2024-04-02T04:20:56.036 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-04-02T04:20:56.036 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/teuthology-2024-04-01_21:08:08-orch-squid-distro-default-smithi/7634838 --name teuthology-2024-04-01_21:08:08-orch-squid-distro-default-smithi --verbose --owner scheduled_teuthology@teuthology --description orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} -- /home/teuthworker/archive/teuthology-2024-04-01_21:08:08-orch-squid-distro-default-smithi/7634838/orig.config.yaml 2024-04-02T04:20:56.092 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-02T04:20:56.186 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-01_21:08:08-orch-squid-distro-default-smithi/7634838 branch: squid description: orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7634838' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/reef ' name: teuthology-2024-04-01_21:08:08-orch-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 6cec793d9073f0594e2364b9b687ecaf22424cca ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 6cec793d9073f0594e2364b9b687ecaf22424cca kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 6cec793d9073f0594e2364b9b687ecaf22424cca owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 6530 sha1: 6cec793d9073f0594e2364b9b687ecaf22424cca sleep_before_teardown: 0 subset: 52/64 suite: orch suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_6cec793d9073f0594e2364b9b687ecaf22424cca/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 6cec793d9073f0594e2364b9b687ecaf22424cca targets: smithi016.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHRZqVhwCciLOONC/HznwhHfKmFctk5739tiOraMTl2h1nkmzK2GPAaBjw6l3WlquIg1JsPPi0NANUoJYARBiF8= smithi027.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH5+d8o2yMuNlHGs4c4hyCuLs2fdq07eanTjpGjK1YZciNmYtpq/Y0+kMTtZ1cweEeM4h1GlamBOqh+N7bcpRlM= tasks: - install: branch: reef exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: compiled_cephadm_branch: reef conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:reef roleless: true - print: '**** done end installing reef cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 1 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay false - cephadm.shell: host.a: - ceph fs set cephfs inline_data true --yes-i-really-really-mean-it - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: - /home/teuthology/src/git.ceph.com_ceph_squid/qa/suites/orch/cephadm/mds_upgrade_sequence/tasks/3-upgrade-mgr-staggered.yaml meta: {} postmerge: [] variables: fail_fs: false teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-04-01_21:08:08 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs false || true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-04-02T04:20:56.186 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_6cec793d9073f0594e2364b9b687ecaf22424cca/qa; will attempt to use it 2024-04-02T04:20:56.187 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_6cec793d9073f0594e2364b9b687ecaf22424cca/qa/tasks 2024-04-02T04:20:56.187 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-02T04:20:56.189 INFO:teuthology.task.internal:Checking packages... 2024-04-02T04:20:56.212 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '6cec793d9073f0594e2364b9b687ecaf22424cca' 2024-04-02T04:20:56.212 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-02T04:20:56.212 INFO:teuthology.packaging:ref: None 2024-04-02T04:20:56.212 INFO:teuthology.packaging:tag: None 2024-04-02T04:20:56.212 INFO:teuthology.packaging:branch: squid 2024-04-02T04:20:56.212 INFO:teuthology.packaging:sha1: 6cec793d9073f0594e2364b9b687ecaf22424cca 2024-04-02T04:20:56.212 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-04-02T04:20:56.392 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1979.g6cec793d 2024-04-02T04:20:56.394 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-02T04:20:56.400 INFO:teuthology.task.internal:no buildpackages task found 2024-04-02T04:20:56.401 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-02T04:20:56.470 INFO:teuthology.task.internal:Saving configuration 2024-04-02T04:20:56.491 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-02T04:20:56.498 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-02T04:20:56.527 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-01_21:08:08-orch-squid-distro-default-smithi/7634838', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 04:14:49.476152', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHRZqVhwCciLOONC/HznwhHfKmFctk5739tiOraMTl2h1nkmzK2GPAaBjw6l3WlquIg1JsPPi0NANUoJYARBiF8='} 2024-04-02T04:20:56.556 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi027.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-01_21:08:08-orch-squid-distro-default-smithi/7634838', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 04:14:49.477957', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH5+d8o2yMuNlHGs4c4hyCuLs2fdq07eanTjpGjK1YZciNmYtpq/Y0+kMTtZ1cweEeM4h1GlamBOqh+N7bcpRlM='} 2024-04-02T04:20:56.556 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-02T04:20:56.562 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-04-02T04:20:56.562 INFO:teuthology.task.internal:roles: ubuntu@smithi027.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-04-02T04:20:56.562 INFO:teuthology.run_tasks:Running task console_log... 2024-04-02T04:20:56.647 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff3083c10d0>, signals=[15]) 2024-04-02T04:20:56.648 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-02T04:20:56.654 INFO:teuthology.task.internal:Opening connections... 2024-04-02T04:20:56.654 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi016.front.sepia.ceph.com 2024-04-02T04:20:56.656 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T04:20:56.772 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi027.front.sepia.ceph.com 2024-04-02T04:20:56.773 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T04:20:56.908 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-02T04:20:56.914 DEBUG:teuthology.orchestra.run.smithi016:> uname -m 2024-04-02T04:20:56.934 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2024-04-02T04:20:56.934 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2024-04-02T04:20:56.992 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="CentOS Stream" 2024-04-02T04:20:56.993 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="9" 2024-04-02T04:20:56.993 INFO:teuthology.orchestra.run.smithi016.stdout:ID="centos" 2024-04-02T04:20:56.993 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="rhel fedora" 2024-04-02T04:20:56.993 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="9" 2024-04-02T04:20:56.993 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el9" 2024-04-02T04:20:56.993 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-02T04:20:56.993 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2024-04-02T04:20:56.993 INFO:teuthology.orchestra.run.smithi016.stdout:LOGO="fedora-logo-icon" 2024-04-02T04:20:56.993 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-02T04:20:56.993 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://centos.org/" 2024-04-02T04:20:56.993 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-02T04:20:56.993 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-02T04:20:56.993 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-02T04:20:56.994 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2024-04-02T04:20:57.130 DEBUG:teuthology.orchestra.run.smithi027:> uname -m 2024-04-02T04:20:57.149 INFO:teuthology.orchestra.run.smithi027.stdout:x86_64 2024-04-02T04:20:57.150 DEBUG:teuthology.orchestra.run.smithi027:> cat /etc/os-release 2024-04-02T04:20:57.208 INFO:teuthology.orchestra.run.smithi027.stdout:NAME="CentOS Stream" 2024-04-02T04:20:57.208 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION="9" 2024-04-02T04:20:57.208 INFO:teuthology.orchestra.run.smithi027.stdout:ID="centos" 2024-04-02T04:20:57.208 INFO:teuthology.orchestra.run.smithi027.stdout:ID_LIKE="rhel fedora" 2024-04-02T04:20:57.208 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_ID="9" 2024-04-02T04:20:57.209 INFO:teuthology.orchestra.run.smithi027.stdout:PLATFORM_ID="platform:el9" 2024-04-02T04:20:57.209 INFO:teuthology.orchestra.run.smithi027.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-02T04:20:57.209 INFO:teuthology.orchestra.run.smithi027.stdout:ANSI_COLOR="0;31" 2024-04-02T04:20:57.209 INFO:teuthology.orchestra.run.smithi027.stdout:LOGO="fedora-logo-icon" 2024-04-02T04:20:57.209 INFO:teuthology.orchestra.run.smithi027.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-02T04:20:57.209 INFO:teuthology.orchestra.run.smithi027.stdout:HOME_URL="https://centos.org/" 2024-04-02T04:20:57.209 INFO:teuthology.orchestra.run.smithi027.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-02T04:20:57.209 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-02T04:20:57.209 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-02T04:20:57.209 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server 2024-04-02T04:20:57.236 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-02T04:20:57.246 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-02T04:20:57.272 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-02T04:20:57.273 DEBUG:teuthology.orchestra.run.smithi016:> test '!' -e /home/ubuntu/cephtest 2024-04-02T04:20:57.276 DEBUG:teuthology.orchestra.run.smithi027:> test '!' -e /home/ubuntu/cephtest 2024-04-02T04:20:57.293 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-02T04:20:57.301 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-02T04:20:57.301 DEBUG:teuthology.orchestra.run.smithi016:> test -z $(ls -A /var/lib/ceph) 2024-04-02T04:20:57.333 DEBUG:teuthology.orchestra.run.smithi027:> test -z $(ls -A /var/lib/ceph) 2024-04-02T04:20:57.363 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-02T04:20:57.422 INFO:teuthology.run_tasks:Running task kernel... 2024-04-02T04:20:57.443 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-04-02T04:20:57.444 INFO:teuthology.task.kernel:config {'host.a': {'kdb': True, 'sha1': 'distro'}, 'host.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-04-02T04:20:57.444 DEBUG:teuthology.orchestra.run.smithi016:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-02T04:20:57.445 DEBUG:teuthology.orchestra.run.smithi027:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-02T04:20:57.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T04:20:57.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T04:20:57.465 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2024-04-02T04:20:57.466 DEBUG:teuthology.orchestra.run.smithi027:> uname -r 2024-04-02T04:20:57.524 INFO:teuthology.orchestra.run.smithi027.stdout:5.14.0-361.el9.x86_64 2024-04-02T04:20:57.524 INFO:teuthology.orchestra.run.smithi016.stdout:5.14.0-361.el9.x86_64 2024-04-02T04:20:57.525 INFO:teuthology.task.kernel:Running kernel on smithi027: 5.14.0-361.el9.x86_64 2024-04-02T04:20:57.525 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum install -y kernel 2024-04-02T04:20:57.525 INFO:teuthology.task.kernel:Running kernel on smithi016: 5.14.0-361.el9.x86_64 2024-04-02T04:20:57.525 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum install -y kernel 2024-04-02T04:20:59.702 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-04-02T04:20:59.705 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-04-02T04:21:01.102 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS-9 - BaseOS 7.6 MB/s | 2.0 MB 00:00 2024-04-02T04:21:01.139 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS-9 - BaseOS 7.5 MB/s | 2.0 MB 00:00 2024-04-02T04:21:02.642 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - BaseOS 6.7 MB/s | 8.0 MB 00:01 2024-04-02T04:21:04.246 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 2.9 MB/s | 8.0 MB 00:02 2024-04-02T04:21:04.602 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-04-02T04:21:08.702 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.1 MB 00:00 2024-04-02T04:21:10.137 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - Extras packages 71 kB/s | 16 kB 00:00 2024-04-02T04:21:12.521 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 9.2 MB/s | 21 MB 00:02 2024-04-02T04:21:17.696 INFO:teuthology.orchestra.run.smithi027.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-04-02T04:21:20.062 INFO:teuthology.orchestra.run.smithi027.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-02T04:21:20.062 INFO:teuthology.orchestra.run.smithi027.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-04-02T04:21:20.163 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-04-02T04:21:20.167 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-02T04:21:20.168 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-04-02T04:21:20.168 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-02T04:21:20.168 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-04-02T04:21:20.168 INFO:teuthology.orchestra.run.smithi027.stdout: kernel x86_64 5.14.0-434.el9 CentOS-BaseOS 192 k 2024-04-02T04:21:20.168 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-04-02T04:21:20.168 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-core x86_64 5.14.0-434.el9 CentOS-BaseOS 15 M 2024-04-02T04:21:20.168 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-modules x86_64 5.14.0-434.el9 CentOS-BaseOS 34 M 2024-04-02T04:21:20.168 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-modules-core x86_64 5.14.0-434.el9 CentOS-BaseOS 28 M 2024-04-02T04:21:20.168 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:21:20.168 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-04-02T04:21:20.168 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-02T04:21:20.168 INFO:teuthology.orchestra.run.smithi027.stdout:Install 4 Packages 2024-04-02T04:21:20.168 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:21:20.169 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 76 M 2024-04-02T04:21:20.169 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 123 M 2024-04-02T04:21:20.169 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-04-02T04:21:20.340 INFO:teuthology.orchestra.run.smithi027.stdout:(1/4): kernel-5.14.0-434.el9.x86_64.rpm 1.1 MB/s | 192 kB 00:00 2024-04-02T04:21:20.790 INFO:teuthology.orchestra.run.smithi027.stdout:(2/4): kernel-core-5.14.0-434.el9.x86_64.rpm 24 MB/s | 15 MB 00:00 2024-04-02T04:21:21.498 INFO:teuthology.orchestra.run.smithi027.stdout:(3/4): kernel-modules-5.14.0-434.el9.x86_64.rpm 25 MB/s | 34 MB 00:01 2024-04-02T04:21:21.907 INFO:teuthology.orchestra.run.smithi027.stdout:(4/4): kernel-modules-core-5.14.0-434.el9.x86_6 18 MB/s | 28 MB 00:01 2024-04-02T04:21:21.909 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:21:21.910 INFO:teuthology.orchestra.run.smithi027.stdout:Total 44 MB/s | 76 MB 00:01 2024-04-02T04:21:21.910 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-04-02T04:21:22.349 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-04-02T04:21:22.349 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-04-02T04:21:23.213 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-04-02T04:21:23.213 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-04-02T04:21:25.658 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-04-02T04:21:26.280 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : kernel-modules-core-5.14.0-434.el9.x86_64 1/4 2024-04-02T04:21:26.385 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-02T04:21:29.251 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-02T04:21:29.387 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-02T04:21:32.795 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 707 kB/s | 19 MB 00:27 2024-04-02T04:21:34.462 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-02T04:21:34.502 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : kernel-5.14.0-434.el9.x86_64 4/4 2024-04-02T04:21:37.970 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 6.1 MB 00:01 2024-04-02T04:21:39.421 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 56 kB/s | 16 kB 00:00 2024-04-02T04:21:39.601 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-04-02T04:21:40.773 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-04-02T04:21:45.856 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-02T04:21:48.197 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-02T04:21:48.197 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-04-02T04:21:48.284 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-02T04:21:48.294 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-02T04:21:48.294 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-04-02T04:21:48.294 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-02T04:21:48.294 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-04-02T04:21:48.294 INFO:teuthology.orchestra.run.smithi016.stdout: kernel x86_64 5.14.0-434.el9 CentOS-BaseOS 192 k 2024-04-02T04:21:48.294 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-04-02T04:21:48.295 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-core x86_64 5.14.0-434.el9 CentOS-BaseOS 15 M 2024-04-02T04:21:48.295 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules x86_64 5.14.0-434.el9 CentOS-BaseOS 34 M 2024-04-02T04:21:48.295 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules-core x86_64 5.14.0-434.el9 CentOS-BaseOS 28 M 2024-04-02T04:21:48.295 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:21:48.295 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-04-02T04:21:48.295 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-02T04:21:48.295 INFO:teuthology.orchestra.run.smithi016.stdout:Install 4 Packages 2024-04-02T04:21:48.295 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:21:48.296 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 76 M 2024-04-02T04:21:48.296 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 123 M 2024-04-02T04:21:48.296 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-04-02T04:21:48.458 INFO:teuthology.orchestra.run.smithi016.stdout:(1/4): kernel-5.14.0-434.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2024-04-02T04:21:49.058 INFO:teuthology.orchestra.run.smithi016.stdout:(2/4): kernel-core-5.14.0-434.el9.x86_64.rpm 20 MB/s | 15 MB 00:00 2024-04-02T04:21:49.708 INFO:teuthology.orchestra.run.smithi016.stdout:(3/4): kernel-modules-5.14.0-434.el9.x86_64.rpm 24 MB/s | 34 MB 00:01 2024-04-02T04:21:49.966 INFO:teuthology.orchestra.run.smithi016.stdout:(4/4): kernel-modules-core-5.14.0-434.el9.x86_6 18 MB/s | 28 MB 00:01 2024-04-02T04:21:49.970 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:21:49.970 INFO:teuthology.orchestra.run.smithi016.stdout:Total 46 MB/s | 76 MB 00:01 2024-04-02T04:21:49.970 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-04-02T04:21:50.442 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-04-02T04:21:50.443 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-04-02T04:21:51.289 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-04-02T04:21:51.289 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-04-02T04:21:53.712 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-04-02T04:21:54.322 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-modules-core-5.14.0-434.el9.x86_64 1/4 2024-04-02T04:21:54.409 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-02T04:21:57.275 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-02T04:21:57.413 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-02T04:22:02.481 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-02T04:22:02.520 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-5.14.0-434.el9.x86_64 4/4 2024-04-02T04:22:07.624 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-04-02T04:22:10.287 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 4/4 2024-04-02T04:22:10.288 INFO:teuthology.orchestra.run.smithi027.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-02T04:22:10.288 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:22:10.332 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 4/4 2024-04-02T04:22:10.669 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-5.14.0-434.el9.x86_64 4/4 2024-04-02T04:22:10.669 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : kernel-5.14.0-434.el9.x86_64 1/4 2024-04-02T04:22:10.669 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-02T04:22:10.669 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-02T04:22:11.051 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-04-02T04:22:11.051 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:22:11.051 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-04-02T04:22:11.051 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-5.14.0-434.el9.x86_64 2024-04-02T04:22:11.052 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-core-5.14.0-434.el9.x86_64 2024-04-02T04:22:11.052 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-modules-5.14.0-434.el9.x86_64 2024-04-02T04:22:11.052 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-modules-core-5.14.0-434.el9.x86_64 2024-04-02T04:22:11.052 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:22:11.052 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-04-02T04:22:11.400 DEBUG:teuthology.orchestra.run.smithi027:> rpm -q kernel | sort -rV | head -n 1 2024-04-02T04:22:11.509 INFO:teuthology.orchestra.run.smithi027.stdout:kernel-5.14.0-434.el9.x86_64 2024-04-02T04:22:11.509 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-04-02T04:22:11.510 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-434.el9.x86_64 2024-04-02T04:22:11.510 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-02T04:22:11.510 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-02T04:22:11.510 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-04-02T04:22:11.510 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi027.front.sepia.ceph.com, path=None, version=distro) 2024-04-02T04:22:11.510 DEBUG:teuthology.orchestra.run.smithi027:> rpm -q kernel | sort -rV | head -n 1 2024-04-02T04:22:11.594 INFO:teuthology.orchestra.run.smithi027.stdout:kernel-5.14.0-434.el9.x86_64 2024-04-02T04:22:11.594 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-04-02T04:22:11.594 DEBUG:teuthology.orchestra.run.smithi027:> sudo rpm -qi grub2-tools 2024-04-02T04:22:11.632 INFO:teuthology.orchestra.run.smithi027.stdout:Name : grub2-tools 2024-04-02T04:22:11.632 INFO:teuthology.orchestra.run.smithi027.stdout:Epoch : 1 2024-04-02T04:22:11.632 INFO:teuthology.orchestra.run.smithi027.stdout:Version : 2.06 2024-04-02T04:22:11.632 INFO:teuthology.orchestra.run.smithi027.stdout:Release : 68.el9 2024-04-02T04:22:11.633 INFO:teuthology.orchestra.run.smithi027.stdout:Architecture: x86_64 2024-04-02T04:22:11.633 INFO:teuthology.orchestra.run.smithi027.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-02T04:22:11.633 INFO:teuthology.orchestra.run.smithi027.stdout:Group : Unspecified 2024-04-02T04:22:11.633 INFO:teuthology.orchestra.run.smithi027.stdout:Size : 8264254 2024-04-02T04:22:11.633 INFO:teuthology.orchestra.run.smithi027.stdout:License : GPLv3+ 2024-04-02T04:22:11.633 INFO:teuthology.orchestra.run.smithi027.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-02T04:22:11.633 INFO:teuthology.orchestra.run.smithi027.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-02T04:22:11.633 INFO:teuthology.orchestra.run.smithi027.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-02T04:22:11.633 INFO:teuthology.orchestra.run.smithi027.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-02T04:22:11.633 INFO:teuthology.orchestra.run.smithi027.stdout:Packager : builder@centos.org 2024-04-02T04:22:11.633 INFO:teuthology.orchestra.run.smithi027.stdout:Vendor : CentOS 2024-04-02T04:22:11.633 INFO:teuthology.orchestra.run.smithi027.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-02T04:22:11.633 INFO:teuthology.orchestra.run.smithi027.stdout:Summary : Support tools for GRUB. 2024-04-02T04:22:11.633 INFO:teuthology.orchestra.run.smithi027.stdout:Description : 2024-04-02T04:22:11.633 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:22:11.634 INFO:teuthology.orchestra.run.smithi027.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-02T04:22:11.634 INFO:teuthology.orchestra.run.smithi027.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-02T04:22:11.634 INFO:teuthology.orchestra.run.smithi027.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-02T04:22:11.634 INFO:teuthology.orchestra.run.smithi027.stdout:hardware devices. 2024-04-02T04:22:11.634 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:22:11.634 INFO:teuthology.orchestra.run.smithi027.stdout:This subpackage provides tools for support of all platforms. 2024-04-02T04:22:11.635 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-02T04:22:11.635 INFO:teuthology.task.kernel:Updating grub on smithi027 to boot 5.14.0-434.el9.x86_64 2024-04-02T04:22:11.635 DEBUG:teuthology.orchestra.run.smithi027:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-02T04:22:12.211 INFO:teuthology.orchestra.run.smithi027.stderr:Generating grub configuration file ... 2024-04-02T04:22:13.623 INFO:teuthology.orchestra.run.smithi027.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-02T04:22:13.647 INFO:teuthology.orchestra.run.smithi027.stderr:done 2024-04-02T04:22:13.649 DEBUG:teuthology.orchestra.run.smithi027:> mktemp 2024-04-02T04:22:13.667 INFO:teuthology.orchestra.run.smithi027.stdout:/tmp/tmp.1n6pBLrSS0 2024-04-02T04:22:13.668 DEBUG:teuthology.orchestra.run.smithi027:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.1n6pBLrSS0 2024-04-02T04:22:13.740 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0666 /tmp/tmp.1n6pBLrSS0 2024-04-02T04:22:13.886 DEBUG:teuthology.orchestra.remote:smithi027:/tmp/tmp.1n6pBLrSS0 is 6KB 2024-04-02T04:22:13.934 DEBUG:teuthology.orchestra.run.smithi027:> rm -fr /tmp/tmp.1n6pBLrSS0 2024-04-02T04:22:13.948 DEBUG:teuthology.orchestra.run.smithi027:> sudo /bin/ls /boot/loader/entries || true 2024-04-02T04:22:14.016 INFO:teuthology.orchestra.run.smithi027.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-02T04:22:14.016 INFO:teuthology.orchestra.run.smithi027.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-02T04:22:14.016 INFO:teuthology.orchestra.run.smithi027.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-02T04:22:14.016 INFO:teuthology.orchestra.run.smithi027.stdout:15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64.conf 2024-04-02T04:22:14.017 DEBUG:teuthology.orchestra.run.smithi027:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64 2024-04-02T04:22:14.177 DEBUG:teuthology.orchestra.run.smithi027:> sudo shutdown -r now 2024-04-02T04:22:38.485 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 4/4 2024-04-02T04:22:38.486 INFO:teuthology.orchestra.run.smithi016.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-02T04:22:38.486 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:22:38.530 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 4/4 2024-04-02T04:22:38.918 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-5.14.0-434.el9.x86_64 4/4 2024-04-02T04:22:38.918 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-5.14.0-434.el9.x86_64 1/4 2024-04-02T04:22:38.919 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-02T04:22:38.919 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-02T04:22:39.350 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-04-02T04:22:39.350 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:22:39.350 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-04-02T04:22:39.350 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-5.14.0-434.el9.x86_64 2024-04-02T04:22:39.350 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-core-5.14.0-434.el9.x86_64 2024-04-02T04:22:39.351 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules-5.14.0-434.el9.x86_64 2024-04-02T04:22:39.351 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules-core-5.14.0-434.el9.x86_64 2024-04-02T04:22:39.351 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:22:39.351 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-02T04:22:39.658 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q kernel | sort -rV | head -n 1 2024-04-02T04:22:39.759 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-5.14.0-434.el9.x86_64 2024-04-02T04:22:39.759 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-04-02T04:22:39.759 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-434.el9.x86_64 2024-04-02T04:22:39.759 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-02T04:22:39.759 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-02T04:22:39.760 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-04-02T04:22:39.760 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi016.front.sepia.ceph.com, path=None, version=distro) 2024-04-02T04:22:39.760 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q kernel | sort -rV | head -n 1 2024-04-02T04:22:39.843 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-5.14.0-434.el9.x86_64 2024-04-02T04:22:39.844 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-04-02T04:22:39.844 DEBUG:teuthology.orchestra.run.smithi016:> sudo rpm -qi grub2-tools 2024-04-02T04:22:39.883 INFO:teuthology.orchestra.run.smithi016.stdout:Name : grub2-tools 2024-04-02T04:22:39.883 INFO:teuthology.orchestra.run.smithi016.stdout:Epoch : 1 2024-04-02T04:22:39.883 INFO:teuthology.orchestra.run.smithi016.stdout:Version : 2.06 2024-04-02T04:22:39.883 INFO:teuthology.orchestra.run.smithi016.stdout:Release : 68.el9 2024-04-02T04:22:39.883 INFO:teuthology.orchestra.run.smithi016.stdout:Architecture: x86_64 2024-04-02T04:22:39.883 INFO:teuthology.orchestra.run.smithi016.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-02T04:22:39.883 INFO:teuthology.orchestra.run.smithi016.stdout:Group : Unspecified 2024-04-02T04:22:39.883 INFO:teuthology.orchestra.run.smithi016.stdout:Size : 8264254 2024-04-02T04:22:39.883 INFO:teuthology.orchestra.run.smithi016.stdout:License : GPLv3+ 2024-04-02T04:22:39.883 INFO:teuthology.orchestra.run.smithi016.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-02T04:22:39.883 INFO:teuthology.orchestra.run.smithi016.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-02T04:22:39.883 INFO:teuthology.orchestra.run.smithi016.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-02T04:22:39.884 INFO:teuthology.orchestra.run.smithi016.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-02T04:22:39.884 INFO:teuthology.orchestra.run.smithi016.stdout:Packager : builder@centos.org 2024-04-02T04:22:39.884 INFO:teuthology.orchestra.run.smithi016.stdout:Vendor : CentOS 2024-04-02T04:22:39.884 INFO:teuthology.orchestra.run.smithi016.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-02T04:22:39.884 INFO:teuthology.orchestra.run.smithi016.stdout:Summary : Support tools for GRUB. 2024-04-02T04:22:39.884 INFO:teuthology.orchestra.run.smithi016.stdout:Description : 2024-04-02T04:22:39.884 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:22:39.884 INFO:teuthology.orchestra.run.smithi016.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-02T04:22:39.884 INFO:teuthology.orchestra.run.smithi016.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-02T04:22:39.884 INFO:teuthology.orchestra.run.smithi016.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-02T04:22:39.884 INFO:teuthology.orchestra.run.smithi016.stdout:hardware devices. 2024-04-02T04:22:39.884 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:22:39.884 INFO:teuthology.orchestra.run.smithi016.stdout:This subpackage provides tools for support of all platforms. 2024-04-02T04:22:39.885 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-02T04:22:39.885 INFO:teuthology.task.kernel:Updating grub on smithi016 to boot 5.14.0-434.el9.x86_64 2024-04-02T04:22:39.886 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-02T04:22:40.493 INFO:teuthology.orchestra.run.smithi016.stderr:Generating grub configuration file ... 2024-04-02T04:22:41.877 INFO:teuthology.orchestra.run.smithi016.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-02T04:22:41.925 INFO:teuthology.orchestra.run.smithi016.stderr:done 2024-04-02T04:22:41.927 DEBUG:teuthology.orchestra.run.smithi016:> mktemp 2024-04-02T04:22:41.942 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/tmp.Qt0OIOu1Eb 2024-04-02T04:22:41.942 DEBUG:teuthology.orchestra.run.smithi016:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Qt0OIOu1Eb 2024-04-02T04:22:42.017 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0666 /tmp/tmp.Qt0OIOu1Eb 2024-04-02T04:22:42.161 DEBUG:teuthology.orchestra.remote:smithi016:/tmp/tmp.Qt0OIOu1Eb is 6KB 2024-04-02T04:22:42.210 DEBUG:teuthology.orchestra.run.smithi016:> rm -fr /tmp/tmp.Qt0OIOu1Eb 2024-04-02T04:22:42.224 DEBUG:teuthology.orchestra.run.smithi016:> sudo /bin/ls /boot/loader/entries || true 2024-04-02T04:22:42.293 INFO:teuthology.orchestra.run.smithi016.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-02T04:22:42.293 INFO:teuthology.orchestra.run.smithi016.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-02T04:22:42.293 INFO:teuthology.orchestra.run.smithi016.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-02T04:22:42.293 INFO:teuthology.orchestra.run.smithi016.stdout:15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64.conf 2024-04-02T04:22:42.294 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64 2024-04-02T04:22:42.453 DEBUG:teuthology.orchestra.run.smithi016:> sudo shutdown -r now 2024-04-02T04:22:44.181 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-04-02T04:22:44.181 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-04-02T04:22:44.182 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T04:23:02.687 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-04-02T04:23:11.695 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-04-02T04:23:11.696 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T04:23:12.456 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-04-02T04:23:12.457 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-04-02T04:23:12.457 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T04:23:14.750 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-04-02T04:23:26.762 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-04-02T04:23:26.763 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T04:23:29.826 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-04-02T04:23:30.846 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.16 2024-04-02T04:23:39.854 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-04-02T04:23:39.855 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T04:23:42.910 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.16 2024-04-02T04:23:44.829 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-04-02T04:23:44.830 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T04:23:47.902 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-04-02T04:23:54.917 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-04-02T04:23:54.917 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T04:24:05.914 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-04-02T04:24:05.915 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T04:24:24.355 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-04-02T04:24:26.730 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-04-02T04:24:27.129 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-04-02T04:24:27.129 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "5.14.0-434.el9.x86_64"... 2024-04-02T04:24:27.129 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2024-04-02T04:24:27.144 INFO:teuthology.orchestra.run.smithi016.stdout:5.14.0-434.el9.x86_64 2024-04-02T04:24:27.145 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-434.el9.x86_64 vs 5.14.0-434.el9.x86_64 2024-04-02T04:24:27.145 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-02T04:24:27.145 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-02T04:24:28.145 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-04-02T04:24:28.146 DEBUG:teuthology.orchestra.run.smithi016:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-02T04:24:28.212 INFO:teuthology.orchestra.run.smithi016.stdout:ttyS1 2024-04-02T04:24:28.249 DEBUG:teuthology.parallel:result is None 2024-04-02T04:24:45.355 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-04-02T04:24:45.356 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T04:24:45.629 DEBUG:teuthology.orchestra.run.smithi027:> true 2024-04-02T04:24:45.999 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-04-02T04:24:45.999 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "5.14.0-434.el9.x86_64"... 2024-04-02T04:24:45.999 DEBUG:teuthology.orchestra.run.smithi027:> uname -r 2024-04-02T04:24:46.015 INFO:teuthology.orchestra.run.smithi027.stdout:5.14.0-434.el9.x86_64 2024-04-02T04:24:46.015 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-434.el9.x86_64 vs 5.14.0-434.el9.x86_64 2024-04-02T04:24:46.015 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-02T04:24:46.015 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-02T04:24:47.016 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-04-02T04:24:47.016 DEBUG:teuthology.orchestra.run.smithi027:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-02T04:24:47.098 INFO:teuthology.orchestra.run.smithi027.stdout:ttyS1 2024-04-02T04:24:47.129 DEBUG:teuthology.parallel:result is None 2024-04-02T04:24:47.129 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-02T04:24:47.140 INFO:teuthology.task.internal:Creating test directory... 2024-04-02T04:24:47.140 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-02T04:24:47.143 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-02T04:24:47.160 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-02T04:24:47.166 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-02T04:24:47.192 INFO:teuthology.task.internal:Creating archive directory... 2024-04-02T04:24:47.193 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-02T04:24:47.200 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-02T04:24:47.230 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-02T04:24:47.251 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-02T04:24:47.251 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-02T04:24:47.263 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-02T04:24:47.301 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T04:24:47.309 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T04:24:47.315 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T04:24:47.325 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T04:24:47.327 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-02T04:24:47.333 INFO:teuthology.task.internal:Configuring sudo... 2024-04-02T04:24:47.333 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-02T04:24:47.360 DEBUG:teuthology.orchestra.run.smithi027:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-02T04:24:47.401 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-02T04:24:47.410 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-02T04:24:47.410 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-02T04:24:47.428 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-02T04:24:47.457 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T04:24:47.525 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T04:24:47.570 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T04:24:47.653 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T04:24:47.696 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-02T04:24:47.696 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-02T04:24:47.763 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T04:24:47.809 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T04:24:47.866 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T04:24:47.951 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T04:24:48.000 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-02T04:24:48.000 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-02T04:24:48.074 DEBUG:teuthology.orchestra.run.smithi016:> sudo service rsyslog restart 2024-04-02T04:24:48.076 DEBUG:teuthology.orchestra.run.smithi027:> sudo service rsyslog restart 2024-04-02T04:24:48.117 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T04:24:48.169 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T04:24:48.681 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-02T04:24:48.689 INFO:teuthology.task.internal:Starting timer... 2024-04-02T04:24:48.690 INFO:teuthology.run_tasks:Running task pcp... 2024-04-02T04:24:48.729 INFO:teuthology.run_tasks:Running task selinux... 2024-04-02T04:24:48.761 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-02T04:24:48.762 DEBUG:teuthology.orchestra.run.smithi016:> sudo service auditd rotate 2024-04-02T04:24:48.836 INFO:teuthology.orchestra.run.smithi016.stdout:Rotating logs: 2024-04-02T04:24:48.837 DEBUG:teuthology.orchestra.run.smithi027:> sudo service auditd rotate 2024-04-02T04:24:48.892 INFO:teuthology.orchestra.run.smithi027.stdout:Rotating logs: 2024-04-02T04:24:48.893 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-02T04:24:48.894 DEBUG:teuthology.orchestra.run.smithi016:> /usr/sbin/getenforce 2024-04-02T04:24:48.914 INFO:teuthology.orchestra.run.smithi016.stdout:Permissive 2024-04-02T04:24:48.914 DEBUG:teuthology.orchestra.run.smithi027:> /usr/sbin/getenforce 2024-04-02T04:24:48.937 INFO:teuthology.orchestra.run.smithi027.stdout:Permissive 2024-04-02T04:24:48.938 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi016.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi027.front.sepia.ceph.com': 'permissive'} 2024-04-02T04:24:48.938 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T04:24:48.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T04:24:48.984 DEBUG:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T04:24:49.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T04:24:49.019 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-02T04:24:49.019 DEBUG:teuthology.orchestra.run.smithi016:> sudo /usr/sbin/setenforce permissive 2024-04-02T04:24:49.057 DEBUG:teuthology.orchestra.run.smithi027:> sudo /usr/sbin/setenforce permissive 2024-04-02T04:24:49.098 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-02T04:24:49.108 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-02T04:24:49.125 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-02T04:24:49.126 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi016.front.sepia.ceph.com,smithi027.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-02T04:29:14.222 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi016.front.sepia.ceph.com'), Remote(name='ubuntu@smithi027.front.sepia.ceph.com')] 2024-04-02T04:29:14.224 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-04-02T04:29:14.225 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T04:29:14.322 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-04-02T04:29:14.397 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-04-02T04:29:14.397 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-04-02T04:29:14.398 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T04:29:14.501 DEBUG:teuthology.orchestra.run.smithi027:> true 2024-04-02T04:29:14.582 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-04-02T04:29:14.582 INFO:teuthology.run_tasks:Running task clock... 2024-04-02T04:29:14.592 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-02T04:29:14.593 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-02T04:29:14.593 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T04:29:14.596 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-02T04:29:14.596 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T04:29:14.625 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-02T04:29:14.642 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-02T04:29:14.658 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-02T04:29:14.671 INFO:teuthology.orchestra.run.smithi016.stderr:sudo: ntpd: command not found 2024-04-02T04:29:14.678 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-02T04:29:14.684 INFO:teuthology.orchestra.run.smithi016.stdout:506 Cannot talk to daemon 2024-04-02T04:29:14.701 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-02T04:29:14.714 INFO:teuthology.orchestra.run.smithi027.stderr:sudo: ntpd: command not found 2024-04-02T04:29:14.717 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-02T04:29:14.731 INFO:teuthology.orchestra.run.smithi027.stdout:506 Cannot talk to daemon 2024-04-02T04:29:14.751 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-02T04:29:14.771 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-02T04:29:14.771 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2024-04-02T04:29:14.776 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T04:29:14.776 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-04-02T04:29:14.776 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T04:29:14.776 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T04:29:14.776 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T04:29:14.776 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T04:29:14.827 INFO:teuthology.orchestra.run.smithi027.stderr:bash: line 1: ntpq: command not found 2024-04-02T04:29:14.830 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T04:29:14.830 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2024-04-02T04:29:14.831 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T04:29:14.831 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T04:29:14.831 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T04:29:14.831 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T04:29:14.831 INFO:teuthology.run_tasks:Running task install... 2024-04-02T04:29:14.840 DEBUG:teuthology.task.install:project ceph 2024-04-02T04:29:14.840 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '6cec793d9073f0594e2364b9b687ecaf22424cca'}} 2024-04-02T04:29:14.840 DEBUG:teuthology.task.install:config {'branch': 'reef', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': '6cec793d9073f0594e2364b9b687ecaf22424cca'} 2024-04-02T04:29:14.840 INFO:teuthology.task.install:Using flavor: default 2024-04-02T04:29:14.849 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-02T04:29:14.849 INFO:teuthology.task.install:extra packages: [] 2024-04-02T04:29:14.850 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'reef', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6cec793d9073f0594e2364b9b687ecaf22424cca', 'tag': None, 'wait_for_package': False} 2024-04-02T04:29:14.850 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-02T04:29:14.850 INFO:teuthology.packaging:ref: None 2024-04-02T04:29:14.850 INFO:teuthology.packaging:tag: None 2024-04-02T04:29:14.850 INFO:teuthology.packaging:branch: reef 2024-04-02T04:29:14.850 INFO:teuthology.packaging:sha1: 6cec793d9073f0594e2364b9b687ecaf22424cca 2024-04-02T04:29:14.851 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-04-02T04:29:14.852 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'reef', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6cec793d9073f0594e2364b9b687ecaf22424cca', 'tag': None, 'wait_for_package': False} 2024-04-02T04:29:14.853 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-02T04:29:14.853 INFO:teuthology.packaging:ref: None 2024-04-02T04:29:14.853 INFO:teuthology.packaging:tag: None 2024-04-02T04:29:14.853 INFO:teuthology.packaging:branch: reef 2024-04-02T04:29:14.853 INFO:teuthology.packaging:sha1: 6cec793d9073f0594e2364b9b687ecaf22424cca 2024-04-02T04:29:14.853 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-04-02T04:29:15.126 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/reef/4d05f4cad464de0fcdd11a2856643e533aea34e8/centos/9/flavors/default/ 2024-04-02T04:29:15.126 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1059.g4d05f4ca 2024-04-02T04:29:15.235 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/reef/4d05f4cad464de0fcdd11a2856643e533aea34e8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/reef/4d05f4cad464de0fcdd11a2856643e533aea34e8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/reef/4d05f4cad464de0fcdd11a2856643e533aea34e8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-02T04:29:15.235 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-02T04:29:15.235 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-02T04:29:15.264 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-02T04:29:15.264 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-02T04:29:15.264 INFO:teuthology.packaging:ref: None 2024-04-02T04:29:15.265 INFO:teuthology.packaging:tag: None 2024-04-02T04:29:15.265 INFO:teuthology.packaging:branch: reef 2024-04-02T04:29:15.265 INFO:teuthology.packaging:sha1: 6cec793d9073f0594e2364b9b687ecaf22424cca 2024-04-02T04:29:15.265 DEBUG:teuthology.orchestra.run.smithi016:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/reef/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-02T04:29:15.332 DEBUG:teuthology.orchestra.run.smithi016:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-02T04:29:15.413 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/reef/4d05f4cad464de0fcdd11a2856643e533aea34e8/centos/9/flavors/default/ 2024-04-02T04:29:15.413 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1059.g4d05f4ca 2024-04-02T04:29:15.419 DEBUG:teuthology.orchestra.run.smithi016:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-02T04:29:15.486 INFO:teuthology.orchestra.run.smithi016.stdout:check_obsoletes = 1 2024-04-02T04:29:15.488 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum clean all 2024-04-02T04:29:15.523 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/reef/4d05f4cad464de0fcdd11a2856643e533aea34e8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/reef/4d05f4cad464de0fcdd11a2856643e533aea34e8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/reef/4d05f4cad464de0fcdd11a2856643e533aea34e8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-02T04:29:15.524 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-02T04:29:15.524 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-02T04:29:15.557 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-02T04:29:15.557 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-02T04:29:15.558 INFO:teuthology.packaging:ref: None 2024-04-02T04:29:15.558 INFO:teuthology.packaging:tag: None 2024-04-02T04:29:15.558 INFO:teuthology.packaging:branch: reef 2024-04-02T04:29:15.558 INFO:teuthology.packaging:sha1: 6cec793d9073f0594e2364b9b687ecaf22424cca 2024-04-02T04:29:15.558 DEBUG:teuthology.orchestra.run.smithi027:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/reef/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-02T04:29:15.630 DEBUG:teuthology.orchestra.run.smithi027:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-02T04:29:15.718 DEBUG:teuthology.orchestra.run.smithi027:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-02T04:29:15.736 INFO:teuthology.orchestra.run.smithi016.stdout:74 files removed 2024-04-02T04:29:15.749 INFO:teuthology.orchestra.run.smithi027.stdout:check_obsoletes = 1 2024-04-02T04:29:15.750 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum clean all 2024-04-02T04:29:15.763 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-radosgw 2024-04-02T04:29:16.006 INFO:teuthology.orchestra.run.smithi027.stdout:74 files removed 2024-04-02T04:29:16.033 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-radosgw 2024-04-02T04:29:16.643 INFO:teuthology.orchestra.run.smithi016.stdout:ceph packages for x86_64 134 kB/s | 77 kB 00:00 2024-04-02T04:29:16.855 INFO:teuthology.orchestra.run.smithi027.stdout:ceph packages for x86_64 153 kB/s | 77 kB 00:00 2024-04-02T04:29:17.035 INFO:teuthology.orchestra.run.smithi016.stdout:ceph noarch packages 29 kB/s | 9.9 kB 00:00 2024-04-02T04:29:17.247 INFO:teuthology.orchestra.run.smithi027.stdout:ceph noarch packages 29 kB/s | 9.9 kB 00:00 2024-04-02T04:29:17.453 INFO:teuthology.orchestra.run.smithi016.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-04-02T04:29:17.673 INFO:teuthology.orchestra.run.smithi027.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-04-02T04:29:17.897 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS-9 - AppStream 17 MB/s | 6.7 MB 00:00 2024-04-02T04:29:18.117 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS-9 - AppStream 17 MB/s | 6.7 MB 00:00 2024-04-02T04:29:19.466 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS-9 - BaseOS 4.9 MB/s | 2.0 MB 00:00 2024-04-02T04:29:19.583 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS-9 - BaseOS 7.4 MB/s | 2.0 MB 00:00 2024-04-02T04:29:20.341 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - BaseOS 20 MB/s | 8.0 MB 00:00 2024-04-02T04:29:22.201 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - AppStream 26 MB/s | 19 MB 00:00 2024-04-02T04:29:25.222 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 1.5 MB/s | 8.0 MB 00:05 2024-04-02T04:29:26.418 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.1 MB 00:00 2024-04-02T04:29:28.028 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-04-02T04:29:28.198 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 10 MB/s | 19 MB 00:01 2024-04-02T04:29:31.079 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 7.1 MB/s | 21 MB 00:02 2024-04-02T04:29:32.208 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.1 MB 00:00 2024-04-02T04:29:33.683 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 60 kB/s | 16 kB 00:00 2024-04-02T04:29:34.837 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-04-02T04:29:36.354 INFO:teuthology.orchestra.run.smithi027.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-04-02T04:29:39.045 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-04-02T04:29:39.062 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-02T04:29:39.063 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-04-02T04:29:39.063 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-02T04:29:39.063 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-04-02T04:29:39.063 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 7.7 M 2024-04-02T04:29:39.063 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrading: 2024-04-02T04:29:39.063 INFO:teuthology.orchestra.run.smithi027.stdout: librados2 x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 3.3 M 2024-04-02T04:29:39.063 INFO:teuthology.orchestra.run.smithi027.stdout: librbd1 x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 3.0 M 2024-04-02T04:29:39.063 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-04-02T04:29:39.063 INFO:teuthology.orchestra.run.smithi027.stdout: boost-program-options 2024-04-02T04:29:39.063 INFO:teuthology.orchestra.run.smithi027.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-02T04:29:39.063 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 5.1 M 2024-04-02T04:29:39.063 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 18 M 2024-04-02T04:29:39.064 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 25 k 2024-04-02T04:29:39.064 INFO:teuthology.orchestra.run.smithi027.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-04-02T04:29:39.064 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-02T04:29:39.064 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-02T04:29:39.064 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-02T04:29:39.064 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2 x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 708 k 2024-04-02T04:29:39.064 INFO:teuthology.orchestra.run.smithi027.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-02T04:29:39.064 INFO:teuthology.orchestra.run.smithi027.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-02T04:29:39.064 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-02T04:29:39.064 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1 x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 474 k 2024-04-02T04:29:39.064 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-02T04:29:39.064 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2 x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 4.5 M 2024-04-02T04:29:39.064 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-02T04:29:39.064 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-02T04:29:39.065 INFO:teuthology.orchestra.run.smithi027.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-02T04:29:39.065 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse 2024-04-02T04:29:39.065 INFO:teuthology.orchestra.run.smithi027.stdout: x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 45 k 2024-04-02T04:29:39.065 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 129 k 2024-04-02T04:29:39.065 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 162 k 2024-04-02T04:29:39.065 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-02T04:29:39.065 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 320 k 2024-04-02T04:29:39.065 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 298 k 2024-04-02T04:29:39.065 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 99 k 2024-04-02T04:29:39.065 INFO:teuthology.orchestra.run.smithi027.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-02T04:29:39.065 INFO:teuthology.orchestra.run.smithi027.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-02T04:29:39.065 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:29:39.065 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-04-02T04:29:39.065 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-02T04:29:39.066 INFO:teuthology.orchestra.run.smithi027.stdout:Install 28 Packages 2024-04-02T04:29:39.066 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrade 2 Packages 2024-04-02T04:29:39.066 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:29:39.066 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 53 M 2024-04-02T04:29:39.066 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-04-02T04:29:39.919 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 48 kB/s | 1.7 kB 00:00 2024-04-02T04:29:40.221 INFO:teuthology.orchestra.run.smithi027.stdout:(1/30): ceph-common-18.2.2-1059.g4d05f4ca.el9.x 17 MB/s | 18 MB 00:01 2024-04-02T04:29:40.264 INFO:teuthology.orchestra.run.smithi027.stdout:(2/30): ceph-selinux-18.2.2-1059.g4d05f4ca.el9. 605 kB/s | 25 kB 00:00 2024-04-02T04:29:40.330 INFO:teuthology.orchestra.run.smithi027.stdout:(3/30): libcephfs2-18.2.2-1059.g4d05f4ca.el9.x8 10 MB/s | 708 kB 00:00 2024-04-02T04:29:40.398 INFO:teuthology.orchestra.run.smithi027.stdout:(4/30): libradosstriper1-18.2.2-1059.g4d05f4ca. 6.9 MB/s | 474 kB 00:00 2024-04-02T04:29:40.648 INFO:teuthology.orchestra.run.smithi027.stdout:(5/30): librgw2-18.2.2-1059.g4d05f4ca.el9.x86_6 18 MB/s | 4.5 MB 00:00 2024-04-02T04:29:40.690 INFO:teuthology.orchestra.run.smithi027.stdout:(6/30): python3-ceph-argparse-18.2.2-1059.g4d05 1.1 MB/s | 45 kB 00:00 2024-04-02T04:29:40.740 INFO:teuthology.orchestra.run.smithi027.stdout:(7/30): python3-ceph-common-18.2.2-1059.g4d05f4 2.5 MB/s | 129 kB 00:00 2024-04-02T04:29:40.791 INFO:teuthology.orchestra.run.smithi027.stdout:(8/30): python3-cephfs-18.2.2-1059.g4d05f4ca.el 3.2 MB/s | 162 kB 00:00 2024-04-02T04:29:40.840 INFO:teuthology.orchestra.run.smithi027.stdout:(9/30): python3-rados-18.2.2-1059.g4d05f4ca.el9 6.3 MB/s | 320 kB 00:00 2024-04-02T04:29:40.891 INFO:teuthology.orchestra.run.smithi027.stdout:(10/30): python3-rbd-18.2.2-1059.g4d05f4ca.el9. 5.8 MB/s | 298 kB 00:00 2024-04-02T04:29:40.933 INFO:teuthology.orchestra.run.smithi027.stdout:(11/30): python3-rgw-18.2.2-1059.g4d05f4ca.el9. 2.3 MB/s | 99 kB 00:00 2024-04-02T04:29:41.125 INFO:teuthology.orchestra.run.smithi027.stdout:(12/30): boost-program-options-1.75.0-8.el9.x86 552 kB/s | 106 kB 00:00 2024-04-02T04:29:41.250 INFO:teuthology.orchestra.run.smithi027.stdout:(13/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 159 kB 00:00 2024-04-02T04:29:41.292 INFO:teuthology.orchestra.run.smithi027.stdout:(14/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-04-02T04:29:41.542 INFO:teuthology.orchestra.run.smithi027.stdout:(15/30): librdkafka-1.6.1-102.el9.x86_64.rpm 2.6 MB/s | 661 kB 00:00 2024-04-02T04:29:41.634 INFO:teuthology.orchestra.run.smithi027.stdout:(16/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.0 MB/s | 282 kB 00:00 2024-04-02T04:29:41.693 INFO:teuthology.orchestra.run.smithi027.stdout:(17/30): python3-prettytable-0.7.2-27.el9.noarc 707 kB/s | 41 kB 00:00 2024-04-02T04:29:41.835 INFO:teuthology.orchestra.run.smithi027.stdout:(18/30): fmt-8.1.1-5.el9.x86_64.rpm 781 kB/s | 111 kB 00:00 2024-04-02T04:29:41.921 INFO:teuthology.orchestra.run.smithi027.stdout:(19/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 3.6 MB/s | 309 kB 00:00 2024-04-02T04:29:42.102 INFO:teuthology.orchestra.run.smithi027.stdout:(20/30): libarrow-9.0.0-10.el9.x86_64.rpm 24 MB/s | 4.4 MB 00:00 2024-04-02T04:29:42.144 INFO:teuthology.orchestra.run.smithi027.stdout:(21/30): libarrow-doc-9.0.0-10.el9.noarch.rpm 627 kB/s | 26 kB 00:00 2024-04-02T04:29:42.186 INFO:teuthology.orchestra.run.smithi027.stdout:(22/30): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-04-02T04:29:42.253 INFO:teuthology.orchestra.run.smithi027.stdout:(23/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-04-02T04:29:42.330 INFO:teuthology.orchestra.run.smithi027.stdout:(24/30): parquet-libs-9.0.0-10.el9.x86_64.rpm 11 MB/s | 838 kB 00:00 2024-04-02T04:29:42.387 INFO:teuthology.orchestra.run.smithi027.stdout:(25/30): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-04-02T04:29:42.454 INFO:teuthology.orchestra.run.smithi027.stdout:(26/30): thrift-0.15.0-2.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-04-02T04:29:42.713 INFO:teuthology.orchestra.run.smithi027.stdout:(27/30): librados2-18.2.2-1059.g4d05f4ca.el9.x8 13 MB/s | 3.3 MB 00:00 2024-04-02T04:29:42.718 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-02T04:29:42.732 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-02T04:29:42.732 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-04-02T04:29:42.732 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-02T04:29:42.732 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-04-02T04:29:42.732 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 7.7 M 2024-04-02T04:29:42.732 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrading: 2024-04-02T04:29:42.732 INFO:teuthology.orchestra.run.smithi016.stdout: librados2 x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 3.3 M 2024-04-02T04:29:42.732 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1 x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 3.0 M 2024-04-02T04:29:42.732 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-04-02T04:29:42.732 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options 2024-04-02T04:29:42.733 INFO:teuthology.orchestra.run.smithi016.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-02T04:29:42.733 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 5.1 M 2024-04-02T04:29:42.733 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 18 M 2024-04-02T04:29:42.733 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 25 k 2024-04-02T04:29:42.733 INFO:teuthology.orchestra.run.smithi016.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-04-02T04:29:42.733 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-02T04:29:42.733 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-02T04:29:42.733 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-02T04:29:42.733 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2 x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 708 k 2024-04-02T04:29:42.733 INFO:teuthology.orchestra.run.smithi016.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-02T04:29:42.733 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-02T04:29:42.733 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-02T04:29:42.733 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1 x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 474 k 2024-04-02T04:29:42.734 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-02T04:29:42.734 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2 x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 4.5 M 2024-04-02T04:29:42.734 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-02T04:29:42.734 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-02T04:29:42.734 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-02T04:29:42.734 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse 2024-04-02T04:29:42.734 INFO:teuthology.orchestra.run.smithi016.stdout: x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 45 k 2024-04-02T04:29:42.734 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 129 k 2024-04-02T04:29:42.734 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 162 k 2024-04-02T04:29:42.734 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-02T04:29:42.734 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 320 k 2024-04-02T04:29:42.734 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 298 k 2024-04-02T04:29:42.734 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 99 k 2024-04-02T04:29:42.735 INFO:teuthology.orchestra.run.smithi016.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-02T04:29:42.735 INFO:teuthology.orchestra.run.smithi016.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-02T04:29:42.735 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:29:42.735 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-04-02T04:29:42.735 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-02T04:29:42.735 INFO:teuthology.orchestra.run.smithi016.stdout:Install 28 Packages 2024-04-02T04:29:42.735 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrade 2 Packages 2024-04-02T04:29:42.735 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:29:42.735 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 53 M 2024-04-02T04:29:42.736 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-04-02T04:29:42.888 INFO:teuthology.orchestra.run.smithi027.stdout:(28/30): librbd1-18.2.2-1059.g4d05f4ca.el9.x86_ 17 MB/s | 3.0 MB 00:00 2024-04-02T04:29:43.737 INFO:teuthology.orchestra.run.smithi016.stdout:(1/30): ceph-base-18.2.2-1059.g4d05f4ca.el9.x86 8.0 MB/s | 5.1 MB 00:00 2024-04-02T04:29:43.796 INFO:teuthology.orchestra.run.smithi016.stdout:(2/30): ceph-selinux-18.2.2-1059.g4d05f4ca.el9. 432 kB/s | 25 kB 00:00 2024-04-02T04:29:43.895 INFO:teuthology.orchestra.run.smithi016.stdout:(3/30): libcephfs2-18.2.2-1059.g4d05f4ca.el9.x8 6.9 MB/s | 708 kB 00:00 2024-04-02T04:29:43.987 INFO:teuthology.orchestra.run.smithi016.stdout:(4/30): libradosstriper1-18.2.2-1059.g4d05f4ca. 5.1 MB/s | 474 kB 00:00 2024-04-02T04:29:44.088 INFO:teuthology.orchestra.run.smithi027.stdout:(29/30): ceph-base-18.2.2-1059.g4d05f4ca.el9.x8 1.0 MB/s | 5.1 MB 00:04 2024-04-02T04:29:44.446 INFO:teuthology.orchestra.run.smithi016.stdout:(5/30): librgw2-18.2.2-1059.g4d05f4ca.el9.x86_6 9.7 MB/s | 4.5 MB 00:00 2024-04-02T04:29:44.647 INFO:teuthology.orchestra.run.smithi016.stdout:(6/30): ceph-common-18.2.2-1059.g4d05f4ca.el9.x 12 MB/s | 18 MB 00:01 2024-04-02T04:29:44.688 INFO:teuthology.orchestra.run.smithi016.stdout:(7/30): python3-ceph-argparse-18.2.2-1059.g4d05 186 kB/s | 45 kB 00:00 2024-04-02T04:29:44.713 INFO:teuthology.orchestra.run.smithi016.stdout:(8/30): python3-ceph-common-18.2.2-1059.g4d05f4 1.9 MB/s | 129 kB 00:00 2024-04-02T04:29:44.738 INFO:teuthology.orchestra.run.smithi016.stdout:(9/30): python3-cephfs-18.2.2-1059.g4d05f4ca.el 3.2 MB/s | 162 kB 00:00 2024-04-02T04:29:44.764 INFO:teuthology.orchestra.run.smithi016.stdout:(10/30): python3-rados-18.2.2-1059.g4d05f4ca.el 6.2 MB/s | 320 kB 00:00 2024-04-02T04:29:44.789 INFO:teuthology.orchestra.run.smithi016.stdout:(11/30): python3-rbd-18.2.2-1059.g4d05f4ca.el9. 5.8 MB/s | 298 kB 00:00 2024-04-02T04:29:44.814 INFO:teuthology.orchestra.run.smithi016.stdout:(12/30): python3-rgw-18.2.2-1059.g4d05f4ca.el9. 1.9 MB/s | 99 kB 00:00 2024-04-02T04:29:44.931 INFO:teuthology.orchestra.run.smithi016.stdout:(13/30): boost-program-options-1.75.0-8.el9.x86 746 kB/s | 106 kB 00:00 2024-04-02T04:29:44.957 INFO:teuthology.orchestra.run.smithi016.stdout:(14/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 159 kB 00:00 2024-04-02T04:29:44.982 INFO:teuthology.orchestra.run.smithi016.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 882 kB/s | 44 kB 00:00 2024-04-02T04:29:45.032 INFO:teuthology.orchestra.run.smithi016.stdout:(16/30): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2024-04-02T04:29:45.058 INFO:teuthology.orchestra.run.smithi016.stdout:(17/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-04-02T04:29:45.083 INFO:teuthology.orchestra.run.smithi016.stdout:(18/30): python3-prettytable-0.7.2-27.el9.noarc 825 kB/s | 41 kB 00:00 2024-04-02T04:29:45.475 INFO:teuthology.orchestra.run.smithi016.stdout:(19/30): fmt-8.1.1-5.el9.x86_64.rpm 266 kB/s | 111 kB 00:00 2024-04-02T04:29:45.600 INFO:teuthology.orchestra.run.smithi016.stdout:(20/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 598 kB/s | 309 kB 00:00 2024-04-02T04:29:45.646 INFO:teuthology.orchestra.run.smithi027.stdout:(30/30): ceph-radosgw-18.2.2-1059.g4d05f4ca.el9 1.2 MB/s | 7.7 MB 00:06 2024-04-02T04:29:45.650 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:29:45.650 INFO:teuthology.orchestra.run.smithi027.stdout:Total 8.0 MB/s | 53 MB 00:06 2024-04-02T04:29:45.651 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-04-02T04:29:45.692 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-04-02T04:29:45.692 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-04-02T04:29:45.700 INFO:teuthology.orchestra.run.smithi016.stdout:(21/30): libarrow-doc-9.0.0-10.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-04-02T04:29:45.825 INFO:teuthology.orchestra.run.smithi016.stdout:(22/30): ceph-radosgw-18.2.2-1059.g4d05f4ca.el9 2.8 MB/s | 7.7 MB 00:02 2024-04-02T04:29:45.851 INFO:teuthology.orchestra.run.smithi016.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 323 kB/s | 49 kB 00:00 2024-04-02T04:29:46.144 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-04-02T04:29:46.145 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-04-02T04:29:46.159 INFO:teuthology.orchestra.run.smithi016.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 202 kB/s | 67 kB 00:00 2024-04-02T04:29:46.185 INFO:teuthology.orchestra.run.smithi016.stdout:(25/30): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.5 MB/s | 838 kB 00:00 2024-04-02T04:29:46.335 INFO:teuthology.orchestra.run.smithi016.stdout:(26/30): re2-20211101-3.el9.x86_64.rpm 1.1 MB/s | 192 kB 00:00 2024-04-02T04:29:46.610 INFO:teuthology.orchestra.run.smithi016.stdout:(27/30): librados2-18.2.2-1059.g4d05f4ca.el9.x8 12 MB/s | 3.3 MB 00:00 2024-04-02T04:29:46.644 INFO:teuthology.orchestra.run.smithi016.stdout:(28/30): thrift-0.15.0-2.el9.x86_64.rpm 3.4 MB/s | 1.6 MB 00:00 2024-04-02T04:29:46.794 INFO:teuthology.orchestra.run.smithi016.stdout:(29/30): librbd1-18.2.2-1059.g4d05f4ca.el9.x86_ 16 MB/s | 3.0 MB 00:00 2024-04-02T04:29:47.111 INFO:teuthology.orchestra.run.smithi016.stdout:(30/30): libarrow-9.0.0-10.el9.x86_64.rpm 2.7 MB/s | 4.4 MB 00:01 2024-04-02T04:29:47.116 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:29:47.116 INFO:teuthology.orchestra.run.smithi016.stdout:Total 12 MB/s | 53 MB 00:04 2024-04-02T04:29:47.117 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-04-02T04:29:47.157 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-04-02T04:29:47.157 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-04-02T04:29:47.548 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-04-02T04:29:47.614 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-04-02T04:29:47.626 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-04-02T04:29:47.627 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-04-02T04:29:47.643 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-04-02T04:29:47.685 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-04-02T04:29:48.020 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-04-02T04:29:48.084 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librados2-2:18.2.2-1059.g4d05f4ca.el9.x86_64 5/32 2024-04-02T04:29:48.168 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-2:18.2.2-1059.g4d05f4ca.el9.x86_64 5/32 2024-04-02T04:29:48.245 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rados-2:18.2.2-1059.g4d05f4ca.el9.x86_64 6/32 2024-04-02T04:29:48.277 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs2-2:18.2.2-1059.g4d05f4ca.el9.x86_64 7/32 2024-04-02T04:29:48.340 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephfs2-2:18.2.2-1059.g4d05f4ca.el9.x86_64 7/32 2024-04-02T04:29:48.360 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-04-02T04:29:48.395 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-argparse-2:18.2.2-1059.g4d05f4ca.el9. 9/32 2024-04-02T04:29:48.463 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cephfs-2:18.2.2-1059.g4d05f4ca.el9.x86_64 10/32 2024-04-02T04:29:48.489 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libradosstriper1-2:18.2.2-1059.g4d05f4ca.el9.x86_6 11/32 2024-04-02T04:29:48.553 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1059.g4d05f4ca.el9.x86_6 11/32 2024-04-02T04:29:48.664 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-04-02T04:29:48.703 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 13/32 2024-04-02T04:29:48.744 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-04-02T04:29:48.784 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-04-02T04:29:48.811 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 16/32 2024-04-02T04:29:48.847 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-04-02T04:29:48.880 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-04-02T04:29:49.041 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-04-02T04:29:49.127 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-04-02T04:29:49.180 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-04-02T04:29:49.202 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-04-02T04:29:49.242 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-04-02T04:29:49.243 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librbd1-2:18.2.2-1059.g4d05f4ca.el9.x86_64 20/32 2024-04-02T04:29:49.334 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-2:18.2.2-1059.g4d05f4ca.el9.x86_64 20/32 2024-04-02T04:29:49.364 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rbd-2:18.2.2-1059.g4d05f4ca.el9.x86_64 21/32 2024-04-02T04:29:49.427 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-04-02T04:29:49.577 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-04-02T04:29:49.623 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librados2-2:18.2.2-1059.g4d05f4ca.el9.x86_64 5/32 2024-04-02T04:29:49.715 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:18.2.2-1059.g4d05f4ca.el9.x86_64 5/32 2024-04-02T04:29:49.802 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rados-2:18.2.2-1059.g4d05f4ca.el9.x86_64 6/32 2024-04-02T04:29:49.833 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs2-2:18.2.2-1059.g4d05f4ca.el9.x86_64 7/32 2024-04-02T04:29:49.849 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 23/32 2024-04-02T04:29:49.879 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librgw2-2:18.2.2-1059.g4d05f4ca.el9.x86_64 24/32 2024-04-02T04:29:49.895 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephfs2-2:18.2.2-1059.g4d05f4ca.el9.x86_64 7/32 2024-04-02T04:29:49.927 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-04-02T04:29:49.950 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librgw2-2:18.2.2-1059.g4d05f4ca.el9.x86_64 24/32 2024-04-02T04:29:49.968 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-argparse-2:18.2.2-1059.g4d05f4ca.el9. 9/32 2024-04-02T04:29:50.029 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rgw-2:18.2.2-1059.g4d05f4ca.el9.x86_64 25/32 2024-04-02T04:29:50.036 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cephfs-2:18.2.2-1059.g4d05f4ca.el9.x86_64 10/32 2024-04-02T04:29:50.056 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-common-2:18.2.2-1059.g4d05f4ca.el9.x8 26/32 2024-04-02T04:29:50.060 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libradosstriper1-2:18.2.2-1059.g4d05f4ca.el9.x86_6 11/32 2024-04-02T04:29:50.126 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1059.g4d05f4ca.el9.x86_6 11/32 2024-04-02T04:29:50.235 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-04-02T04:29:50.268 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 13/32 2024-04-02T04:29:50.309 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-04-02T04:29:50.349 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-04-02T04:29:50.376 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 16/32 2024-04-02T04:29:50.412 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-04-02T04:29:50.444 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-04-02T04:29:50.761 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-04-02T04:29:50.791 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librbd1-2:18.2.2-1059.g4d05f4ca.el9.x86_64 20/32 2024-04-02T04:29:50.882 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:18.2.2-1059.g4d05f4ca.el9.x86_64 20/32 2024-04-02T04:29:50.912 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rbd-2:18.2.2-1059.g4d05f4ca.el9.x86_64 21/32 2024-04-02T04:29:50.952 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-04-02T04:29:51.362 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 23/32 2024-04-02T04:29:51.393 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librgw2-2:18.2.2-1059.g4d05f4ca.el9.x86_64 24/32 2024-04-02T04:29:51.456 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librgw2-2:18.2.2-1059.g4d05f4ca.el9.x86_64 24/32 2024-04-02T04:29:51.527 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rgw-2:18.2.2-1059.g4d05f4ca.el9.x86_64 25/32 2024-04-02T04:29:51.554 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-common-2:18.2.2-1059.g4d05f4ca.el9.x8 26/32 2024-04-02T04:29:52.172 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:18.2.2-1059.g4d05f4ca.el9.x86_64 27/32 2024-04-02T04:29:52.203 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-common-2:18.2.2-1059.g4d05f4ca.el9.x86_64 27/32 2024-04-02T04:29:52.790 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:18.2.2-1059.g4d05f4ca.el9.x86_64 27/32 2024-04-02T04:29:52.823 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-base-2:18.2.2-1059.g4d05f4ca.el9.x86_64 28/32 2024-04-02T04:29:52.897 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-base-2:18.2.2-1059.g4d05f4ca.el9.x86_64 28/32 2024-04-02T04:29:52.898 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-02T04:29:52.898 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-02T04:29:52.898 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:29:52.932 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-selinux-2:18.2.2-1059.g4d05f4ca.el9.x86_64 29/32 2024-04-02T04:29:53.681 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:18.2.2-1059.g4d05f4ca.el9.x86_64 27/32 2024-04-02T04:29:53.717 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-common-2:18.2.2-1059.g4d05f4ca.el9.x86_64 27/32 2024-04-02T04:29:54.283 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:18.2.2-1059.g4d05f4ca.el9.x86_64 27/32 2024-04-02T04:29:54.320 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-base-2:18.2.2-1059.g4d05f4ca.el9.x86_64 28/32 2024-04-02T04:29:54.410 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-base-2:18.2.2-1059.g4d05f4ca.el9.x86_64 28/32 2024-04-02T04:29:54.410 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-02T04:29:54.410 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-02T04:29:54.410 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:29:54.446 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-selinux-2:18.2.2-1059.g4d05f4ca.el9.x86_64 29/32 2024-04-02T04:30:02.731 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1059.g4d05f4ca.el9.x86_64 29/32 2024-04-02T04:30:02.731 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /sys 2024-04-02T04:30:02.731 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /proc 2024-04-02T04:30:02.731 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /mnt 2024-04-02T04:30:02.731 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /var/tmp 2024-04-02T04:30:02.731 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /home 2024-04-02T04:30:02.731 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /root 2024-04-02T04:30:02.731 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /tmp 2024-04-02T04:30:02.731 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:30:03.492 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-radosgw-2:18.2.2-1059.g4d05f4ca.el9.x86_64 30/32 2024-04-02T04:30:03.519 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1059.g4d05f4ca.el9.x86_64 30/32 2024-04-02T04:30:03.519 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T04:30:03.519 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-02T04:30:03.519 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T04:30:03.519 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T04:30:03.519 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:30:03.521 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-04-02T04:30:03.595 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-04-02T04:30:03.595 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-04-02T04:30:04.573 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1059.g4d05f4ca.el9.x86_64 29/32 2024-04-02T04:30:04.574 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /sys 2024-04-02T04:30:04.574 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /proc 2024-04-02T04:30:04.574 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /mnt 2024-04-02T04:30:04.574 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /var/tmp 2024-04-02T04:30:04.574 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /home 2024-04-02T04:30:04.574 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /root 2024-04-02T04:30:04.574 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /tmp 2024-04-02T04:30:04.574 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:30:04.641 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-04-02T04:30:04.641 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-base-2:18.2.2-1059.g4d05f4ca.el9.x86_64 1/32 2024-04-02T04:30:04.641 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-common-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2/32 2024-04-02T04:30:04.641 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-radosgw-2:18.2.2-1059.g4d05f4ca.el9.x86_64 3/32 2024-04-02T04:30:04.641 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-selinux-2:18.2.2-1059.g4d05f4ca.el9.x86_64 4/32 2024-04-02T04:30:04.641 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs2-2:18.2.2-1059.g4d05f4ca.el9.x86_64 5/32 2024-04-02T04:30:04.641 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libradosstriper1-2:18.2.2-1059.g4d05f4ca.el9.x86_6 6/32 2024-04-02T04:30:04.641 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librgw2-2:18.2.2-1059.g4d05f4ca.el9.x86_64 7/32 2024-04-02T04:30:04.641 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1059.g4d05f4ca.el9. 8/32 2024-04-02T04:30:04.641 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-common-2:18.2.2-1059.g4d05f4ca.el9.x8 9/32 2024-04-02T04:30:04.641 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cephfs-2:18.2.2-1059.g4d05f4ca.el9.x86_64 10/32 2024-04-02T04:30:04.641 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rados-2:18.2.2-1059.g4d05f4ca.el9.x86_64 11/32 2024-04-02T04:30:04.642 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rbd-2:18.2.2-1059.g4d05f4ca.el9.x86_64 12/32 2024-04-02T04:30:04.642 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rgw-2:18.2.2-1059.g4d05f4ca.el9.x86_64 13/32 2024-04-02T04:30:04.642 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-04-02T04:30:04.642 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-04-02T04:30:04.642 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-04-02T04:30:04.642 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-04-02T04:30:04.642 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-04-02T04:30:04.642 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-04-02T04:30:04.642 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-04-02T04:30:04.642 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-04-02T04:30:04.642 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 22/32 2024-04-02T04:30:04.642 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 23/32 2024-04-02T04:30:04.642 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-04-02T04:30:04.642 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-04-02T04:30:04.643 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 26/32 2024-04-02T04:30:04.643 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-04-02T04:30:04.643 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-04-02T04:30:04.643 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-2:18.2.2-1059.g4d05f4ca.el9.x86_64 29/32 2024-04-02T04:30:04.643 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-04-02T04:30:04.644 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-2:18.2.2-1059.g4d05f4ca.el9.x86_64 31/32 2024-04-02T04:30:05.342 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-radosgw-2:18.2.2-1059.g4d05f4ca.el9.x86_64 30/32 2024-04-02T04:30:05.366 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1059.g4d05f4ca.el9.x86_64 30/32 2024-04-02T04:30:05.366 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T04:30:05.366 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-02T04:30:05.366 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T04:30:05.366 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T04:30:05.366 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:30:05.368 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-04-02T04:30:05.436 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-04-02T04:30:05.437 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-04-02T04:30:05.601 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-04-02T04:30:05.601 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:30:05.602 INFO:teuthology.orchestra.run.smithi027.stdout:Upgraded: 2024-04-02T04:30:05.602 INFO:teuthology.orchestra.run.smithi027.stdout: librados2-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:05.602 INFO:teuthology.orchestra.run.smithi027.stdout: librbd1-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:05.602 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-04-02T04:30:05.602 INFO:teuthology.orchestra.run.smithi027.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-02T04:30:05.602 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:05.602 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:05.602 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:05.602 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:05.602 INFO:teuthology.orchestra.run.smithi027.stdout: fmt-8.1.1-5.el9.x86_64 2024-04-02T04:30:05.602 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-02T04:30:05.602 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-02T04:30:05.602 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-02T04:30:05.602 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:05.603 INFO:teuthology.orchestra.run.smithi027.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-02T04:30:05.603 INFO:teuthology.orchestra.run.smithi027.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-02T04:30:05.603 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-02T04:30:05.603 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:05.603 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-02T04:30:05.603 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:05.603 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-02T04:30:05.603 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-02T04:30:05.603 INFO:teuthology.orchestra.run.smithi027.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-02T04:30:05.603 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:05.603 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:05.603 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:05.603 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-02T04:30:05.603 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:05.604 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:05.604 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:05.604 INFO:teuthology.orchestra.run.smithi027.stdout: re2-1:20211101-3.el9.x86_64 2024-04-02T04:30:05.604 INFO:teuthology.orchestra.run.smithi027.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-02T04:30:05.604 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:30:05.604 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-04-02T04:30:05.942 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-test 2024-04-02T04:30:06.493 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-04-02T04:30:06.493 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-base-2:18.2.2-1059.g4d05f4ca.el9.x86_64 1/32 2024-04-02T04:30:06.493 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-common-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2/32 2024-04-02T04:30:06.493 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-radosgw-2:18.2.2-1059.g4d05f4ca.el9.x86_64 3/32 2024-04-02T04:30:06.493 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-selinux-2:18.2.2-1059.g4d05f4ca.el9.x86_64 4/32 2024-04-02T04:30:06.493 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs2-2:18.2.2-1059.g4d05f4ca.el9.x86_64 5/32 2024-04-02T04:30:06.493 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libradosstriper1-2:18.2.2-1059.g4d05f4ca.el9.x86_6 6/32 2024-04-02T04:30:06.493 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librgw2-2:18.2.2-1059.g4d05f4ca.el9.x86_64 7/32 2024-04-02T04:30:06.493 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1059.g4d05f4ca.el9. 8/32 2024-04-02T04:30:06.494 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-common-2:18.2.2-1059.g4d05f4ca.el9.x8 9/32 2024-04-02T04:30:06.494 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cephfs-2:18.2.2-1059.g4d05f4ca.el9.x86_64 10/32 2024-04-02T04:30:06.494 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rados-2:18.2.2-1059.g4d05f4ca.el9.x86_64 11/32 2024-04-02T04:30:06.494 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rbd-2:18.2.2-1059.g4d05f4ca.el9.x86_64 12/32 2024-04-02T04:30:06.494 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rgw-2:18.2.2-1059.g4d05f4ca.el9.x86_64 13/32 2024-04-02T04:30:06.494 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-04-02T04:30:06.494 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-04-02T04:30:06.494 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-04-02T04:30:06.494 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-04-02T04:30:06.494 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-04-02T04:30:06.494 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-04-02T04:30:06.494 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-04-02T04:30:06.494 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-04-02T04:30:06.494 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 22/32 2024-04-02T04:30:06.495 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 23/32 2024-04-02T04:30:06.495 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-04-02T04:30:06.495 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-04-02T04:30:06.495 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 26/32 2024-04-02T04:30:06.495 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-04-02T04:30:06.495 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-04-02T04:30:06.495 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:18.2.2-1059.g4d05f4ca.el9.x86_64 29/32 2024-04-02T04:30:06.495 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-04-02T04:30:06.495 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:18.2.2-1059.g4d05f4ca.el9.x86_64 31/32 2024-04-02T04:30:06.532 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:00:30 ago on Tue 02 Apr 2024 04:29:36 AM UTC. 2024-04-02T04:30:06.695 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-04-02T04:30:06.696 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-02T04:30:06.696 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-04-02T04:30:06.696 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-02T04:30:06.696 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-04-02T04:30:06.696 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 40 M 2024-04-02T04:30:06.696 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-04-02T04:30:06.697 INFO:teuthology.orchestra.run.smithi027.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-02T04:30:06.697 INFO:teuthology.orchestra.run.smithi027.stdout: libcephsqlite x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 166 k 2024-04-02T04:30:06.697 INFO:teuthology.orchestra.run.smithi027.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-02T04:30:06.697 INFO:teuthology.orchestra.run.smithi027.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-02T04:30:06.697 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-02T04:30:06.697 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:30:06.697 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-04-02T04:30:06.697 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-02T04:30:06.697 INFO:teuthology.orchestra.run.smithi027.stdout:Install 6 Packages 2024-04-02T04:30:06.697 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:30:06.698 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 41 M 2024-04-02T04:30:06.698 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 164 M 2024-04-02T04:30:06.698 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-04-02T04:30:06.940 INFO:teuthology.orchestra.run.smithi027.stdout:(1/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.0 MB/s | 217 kB 00:00 2024-04-02T04:30:06.999 INFO:teuthology.orchestra.run.smithi027.stdout:(2/6): socat-1.7.4.1-5.el9.x86_64.rpm 5.0 MB/s | 300 kB 00:00 2024-04-02T04:30:07.024 INFO:teuthology.orchestra.run.smithi027.stdout:(3/6): libcephsqlite-18.2.2-1059.g4d05f4ca.el9. 578 kB/s | 166 kB 00:00 2024-04-02T04:30:07.041 INFO:teuthology.orchestra.run.smithi027.stdout:(4/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 63 kB 00:00 2024-04-02T04:30:07.216 INFO:teuthology.orchestra.run.smithi027.stdout:(5/6): jq-1.6-16.el9.x86_64.rpm 976 kB/s | 187 kB 00:00 2024-04-02T04:30:07.680 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-04-02T04:30:07.681 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:30:07.681 INFO:teuthology.orchestra.run.smithi016.stdout:Upgraded: 2024-04-02T04:30:07.681 INFO:teuthology.orchestra.run.smithi016.stdout: librados2-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:07.681 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:07.681 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-04-02T04:30:07.681 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-02T04:30:07.681 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:07.681 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:07.681 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:07.681 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:07.681 INFO:teuthology.orchestra.run.smithi016.stdout: fmt-8.1.1-5.el9.x86_64 2024-04-02T04:30:07.681 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-02T04:30:07.681 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-02T04:30:07.682 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-02T04:30:07.682 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:07.682 INFO:teuthology.orchestra.run.smithi016.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-02T04:30:07.682 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-02T04:30:07.682 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-02T04:30:07.682 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:07.682 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-02T04:30:07.682 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:07.682 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-02T04:30:07.682 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-02T04:30:07.682 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-02T04:30:07.682 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:07.682 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:07.682 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:07.682 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-02T04:30:07.683 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:07.683 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:07.683 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:07.683 INFO:teuthology.orchestra.run.smithi016.stdout: re2-1:20211101-3.el9.x86_64 2024-04-02T04:30:07.683 INFO:teuthology.orchestra.run.smithi016.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-02T04:30:07.683 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:30:07.683 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-02T04:30:08.010 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-test 2024-04-02T04:30:08.602 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:00:29 ago on Tue 02 Apr 2024 04:29:39 AM UTC. 2024-04-02T04:30:08.768 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-02T04:30:08.769 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-02T04:30:08.769 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-04-02T04:30:08.769 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-02T04:30:08.769 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-04-02T04:30:08.769 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 40 M 2024-04-02T04:30:08.769 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-04-02T04:30:08.769 INFO:teuthology.orchestra.run.smithi016.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-02T04:30:08.769 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 166 k 2024-04-02T04:30:08.769 INFO:teuthology.orchestra.run.smithi016.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-02T04:30:08.769 INFO:teuthology.orchestra.run.smithi016.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-02T04:30:08.769 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-02T04:30:08.769 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:30:08.769 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-04-02T04:30:08.770 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-02T04:30:08.770 INFO:teuthology.orchestra.run.smithi016.stdout:Install 6 Packages 2024-04-02T04:30:08.770 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:30:08.770 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 41 M 2024-04-02T04:30:08.770 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 164 M 2024-04-02T04:30:08.770 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-04-02T04:30:08.975 INFO:teuthology.orchestra.run.smithi027.stdout:(6/6): ceph-test-18.2.2-1059.g4d05f4ca.el9.x86_ 18 MB/s | 40 MB 00:02 2024-04-02T04:30:08.978 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:30:08.978 INFO:teuthology.orchestra.run.smithi027.stdout:Total 18 MB/s | 41 MB 00:02 2024-04-02T04:30:08.993 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-04-02T04:30:09.001 INFO:teuthology.orchestra.run.smithi016.stdout:(1/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.3 MB/s | 217 kB 00:00 2024-04-02T04:30:09.017 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-04-02T04:30:09.017 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-04-02T04:30:09.051 INFO:teuthology.orchestra.run.smithi016.stdout:(2/6): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 300 kB 00:00 2024-04-02T04:30:09.085 INFO:teuthology.orchestra.run.smithi016.stdout:(3/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2024-04-02T04:30:09.210 INFO:teuthology.orchestra.run.smithi016.stdout:(4/6): libcephsqlite-18.2.2-1059.g4d05f4ca.el9. 443 kB/s | 166 kB 00:00 2024-04-02T04:30:09.235 INFO:teuthology.orchestra.run.smithi016.stdout:(5/6): jq-1.6-16.el9.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2024-04-02T04:30:09.376 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-04-02T04:30:09.376 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-04-02T04:30:09.920 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-04-02T04:30:09.993 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-04-02T04:30:10.033 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-04-02T04:30:10.073 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-04-02T04:30:10.121 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-04-02T04:30:10.153 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephsqlite-2:18.2.2-1059.g4d05f4ca.el9.x86_64 5/6 2024-04-02T04:30:11.019 INFO:teuthology.orchestra.run.smithi016.stdout:(6/6): ceph-test-18.2.2-1059.g4d05f4ca.el9.x86_ 18 MB/s | 40 MB 00:02 2024-04-02T04:30:11.021 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:30:11.021 INFO:teuthology.orchestra.run.smithi016.stdout:Total 18 MB/s | 41 MB 00:02 2024-04-02T04:30:11.035 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-04-02T04:30:11.057 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-04-02T04:30:11.057 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-04-02T04:30:11.402 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-04-02T04:30:11.402 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-04-02T04:30:11.947 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-04-02T04:30:12.012 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-04-02T04:30:12.058 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-04-02T04:30:12.099 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-04-02T04:30:12.153 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-04-02T04:30:12.180 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephsqlite-2:18.2.2-1059.g4d05f4ca.el9.x86_64 5/6 2024-04-02T04:30:14.048 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1059.g4d05f4ca.el9.x86_64 5/6 2024-04-02T04:30:14.129 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-test-2:18.2.2-1059.g4d05f4ca.el9.x86_64 6/6 2024-04-02T04:30:16.075 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1059.g4d05f4ca.el9.x86_64 5/6 2024-04-02T04:30:16.156 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-test-2:18.2.2-1059.g4d05f4ca.el9.x86_64 6/6 2024-04-02T04:30:16.207 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-test-2:18.2.2-1059.g4d05f4ca.el9.x86_64 6/6 2024-04-02T04:30:16.207 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-test-2:18.2.2-1059.g4d05f4ca.el9.x86_64 1/6 2024-04-02T04:30:16.207 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephsqlite-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2/6 2024-04-02T04:30:16.207 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-04-02T04:30:16.207 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2024-04-02T04:30:16.207 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2024-04-02T04:30:16.585 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : jq-1.6-16.el9.x86_64 6/6 2024-04-02T04:30:16.586 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:30:16.586 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-04-02T04:30:16.586 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:16.586 INFO:teuthology.orchestra.run.smithi027.stdout: jq-1.6-16.el9.x86_64 2024-04-02T04:30:16.586 INFO:teuthology.orchestra.run.smithi027.stdout: libcephsqlite-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:16.587 INFO:teuthology.orchestra.run.smithi027.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-02T04:30:16.587 INFO:teuthology.orchestra.run.smithi027.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-02T04:30:16.587 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-02T04:30:16.587 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:30:16.587 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-04-02T04:30:16.839 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph 2024-04-02T04:30:17.423 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:00:41 ago on Tue 02 Apr 2024 04:29:36 AM UTC. 2024-04-02T04:30:17.584 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-04-02T04:30:17.587 INFO:teuthology.orchestra.run.smithi027.stdout:========================================================================================= 2024-04-02T04:30:17.587 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-04-02T04:30:17.587 INFO:teuthology.orchestra.run.smithi027.stdout:========================================================================================= 2024-04-02T04:30:17.587 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-04-02T04:30:17.587 INFO:teuthology.orchestra.run.smithi027.stdout: ceph x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 6.5 k 2024-04-02T04:30:17.587 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-04-02T04:30:17.587 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 2.1 M 2024-04-02T04:30:17.587 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 1.5 M 2024-04-02T04:30:17.587 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1059.g4d05f4ca.el9 ceph-noarch 245 k 2024-04-02T04:30:17.587 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 4.7 M 2024-04-02T04:30:17.587 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 17 M 2024-04-02T04:30:17.588 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-02T04:30:17.588 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-02T04:30:17.588 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-02T04:30:17.588 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-02T04:30:17.588 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-02T04:30:17.588 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-02T04:30:17.588 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-02T04:30:17.588 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-02T04:30:17.588 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-02T04:30:17.588 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-02T04:30:17.588 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-02T04:30:17.588 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-02T04:30:17.588 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-02T04:30:17.589 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-02T04:30:17.589 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-02T04:30:17.589 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-02T04:30:17.589 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-02T04:30:17.589 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-02T04:30:17.589 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-02T04:30:17.589 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-02T04:30:17.589 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-02T04:30:17.589 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-02T04:30:17.589 INFO:teuthology.orchestra.run.smithi027.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-02T04:30:17.589 INFO:teuthology.orchestra.run.smithi027.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-02T04:30:17.589 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-02T04:30:17.589 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-02T04:30:17.589 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-02T04:30:17.589 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:30:17.590 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-04-02T04:30:17.590 INFO:teuthology.orchestra.run.smithi027.stdout:========================================================================================= 2024-04-02T04:30:17.590 INFO:teuthology.orchestra.run.smithi027.stdout:Install 33 Packages 2024-04-02T04:30:17.590 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:30:17.590 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 29 M 2024-04-02T04:30:17.590 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 104 M 2024-04-02T04:30:17.591 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-04-02T04:30:18.029 INFO:teuthology.orchestra.run.smithi027.stdout:(1/33): ceph-18.2.2-1059.g4d05f4ca.el9.x86_64.r 28 kB/s | 6.5 kB 00:00 2024-04-02T04:30:18.171 INFO:teuthology.orchestra.run.smithi027.stdout:(2/33): ceph-mds-18.2.2-1059.g4d05f4ca.el9.x86_ 5.6 MB/s | 2.1 MB 00:00 2024-04-02T04:30:18.278 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-test-2:18.2.2-1059.g4d05f4ca.el9.x86_64 6/6 2024-04-02T04:30:18.278 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-test-2:18.2.2-1059.g4d05f4ca.el9.x86_64 1/6 2024-04-02T04:30:18.279 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephsqlite-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2/6 2024-04-02T04:30:18.279 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-04-02T04:30:18.279 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2024-04-02T04:30:18.279 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2024-04-02T04:30:18.621 INFO:teuthology.orchestra.run.smithi027.stdout:(3/33): ceph-mgr-18.2.2-1059.g4d05f4ca.el9.x86_ 1.8 MB/s | 1.5 MB 00:00 2024-04-02T04:30:18.630 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : jq-1.6-16.el9.x86_64 6/6 2024-04-02T04:30:18.631 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:30:18.631 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-04-02T04:30:18.631 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:18.631 INFO:teuthology.orchestra.run.smithi016.stdout: jq-1.6-16.el9.x86_64 2024-04-02T04:30:18.631 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:18.631 INFO:teuthology.orchestra.run.smithi016.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-02T04:30:18.631 INFO:teuthology.orchestra.run.smithi016.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-02T04:30:18.631 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-02T04:30:18.631 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:30:18.631 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-02T04:30:18.788 INFO:teuthology.orchestra.run.smithi027.stdout:(4/33): ceph-mgr-modules-core-18.2.2-1059.g4d05 1.4 MB/s | 245 kB 00:00 2024-04-02T04:30:18.895 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph 2024-04-02T04:30:18.896 INFO:teuthology.orchestra.run.smithi027.stdout:(5/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 144 kB 00:00 2024-04-02T04:30:19.055 INFO:teuthology.orchestra.run.smithi027.stdout:(6/33): ceph-osd-18.2.2-1059.g4d05f4ca.el9.x86_ 19 MB/s | 17 MB 00:00 2024-04-02T04:30:19.080 INFO:teuthology.orchestra.run.smithi027.stdout:(7/33): python3-markupsafe-1.1.1-12.el9.x86_64. 172 kB/s | 32 kB 00:00 2024-04-02T04:30:19.105 INFO:teuthology.orchestra.run.smithi027.stdout:(8/33): python3-pytz-2021.1-5.el9.noarch.rpm 947 kB/s | 47 kB 00:00 2024-04-02T04:30:19.131 INFO:teuthology.orchestra.run.smithi027.stdout:(9/33): python3-toml-0.10.2-6.el9.noarch.rpm 768 kB/s | 39 kB 00:00 2024-04-02T04:30:19.156 INFO:teuthology.orchestra.run.smithi027.stdout:(10/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-04-02T04:30:19.206 INFO:teuthology.orchestra.run.smithi027.stdout:(11/33): python3-cryptography-36.0.1-4.el9.x86_ 15 MB/s | 1.1 MB 00:00 2024-04-02T04:30:19.232 INFO:teuthology.orchestra.run.smithi027.stdout:(12/33): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 103 kB 00:00 2024-04-02T04:30:19.274 INFO:teuthology.orchestra.run.smithi027.stdout:(13/33): python3-pycparser-2.20-6.el9.noarch.rp 1.8 MB/s | 124 kB 00:00 2024-04-02T04:30:19.299 INFO:teuthology.orchestra.run.smithi027.stdout:(14/33): python3-urllib3-1.26.5-5.el9.noarch.rp 2.7 MB/s | 187 kB 00:00 2024-04-02T04:30:19.366 INFO:teuthology.orchestra.run.smithi027.stdout:(15/33): ceph-mon-18.2.2-1059.g4d05f4ca.el9.x86 3.5 MB/s | 4.7 MB 00:01 2024-04-02T04:30:19.463 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:00:40 ago on Tue 02 Apr 2024 04:29:39 AM UTC. 2024-04-02T04:30:19.483 INFO:teuthology.orchestra.run.smithi027.stdout:(16/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 237 kB/s | 43 kB 00:00 2024-04-02T04:30:19.583 INFO:teuthology.orchestra.run.smithi027.stdout:(17/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 792 kB/s | 172 kB 00:00 2024-04-02T04:30:19.629 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-02T04:30:19.631 INFO:teuthology.orchestra.run.smithi016.stdout:========================================================================================= 2024-04-02T04:30:19.631 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-04-02T04:30:19.631 INFO:teuthology.orchestra.run.smithi016.stdout:========================================================================================= 2024-04-02T04:30:19.631 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-04-02T04:30:19.632 INFO:teuthology.orchestra.run.smithi016.stdout: ceph x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 6.5 k 2024-04-02T04:30:19.632 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-04-02T04:30:19.632 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 2.1 M 2024-04-02T04:30:19.632 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 1.5 M 2024-04-02T04:30:19.632 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1059.g4d05f4ca.el9 ceph-noarch 245 k 2024-04-02T04:30:19.632 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 4.7 M 2024-04-02T04:30:19.632 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 17 M 2024-04-02T04:30:19.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-02T04:30:19.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-02T04:30:19.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-02T04:30:19.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-02T04:30:19.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-02T04:30:19.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-02T04:30:19.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-02T04:30:19.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-02T04:30:19.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-02T04:30:19.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-02T04:30:19.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-02T04:30:19.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-02T04:30:19.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-02T04:30:19.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-02T04:30:19.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-02T04:30:19.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-02T04:30:19.634 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-02T04:30:19.634 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-02T04:30:19.634 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-02T04:30:19.634 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-02T04:30:19.634 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-02T04:30:19.634 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-02T04:30:19.634 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-02T04:30:19.634 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-02T04:30:19.634 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-02T04:30:19.634 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-02T04:30:19.634 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-02T04:30:19.634 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:30:19.635 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-04-02T04:30:19.635 INFO:teuthology.orchestra.run.smithi016.stdout:========================================================================================= 2024-04-02T04:30:19.635 INFO:teuthology.orchestra.run.smithi016.stdout:Install 33 Packages 2024-04-02T04:30:19.635 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:30:19.636 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 29 M 2024-04-02T04:30:19.636 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 104 M 2024-04-02T04:30:19.636 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-04-02T04:30:19.641 INFO:teuthology.orchestra.run.smithi027.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 183 kB/s | 11 kB 00:00 2024-04-02T04:30:19.667 INFO:teuthology.orchestra.run.smithi027.stdout:(19/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.9 MB/s | 358 kB 00:00 2024-04-02T04:30:19.692 INFO:teuthology.orchestra.run.smithi027.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 351 kB/s | 18 kB 00:00 2024-04-02T04:30:19.717 INFO:teuthology.orchestra.run.smithi027.stdout:(21/33): python3-requests-2.25.1-8.el9.noarch.r 282 kB/s | 125 kB 00:00 2024-04-02T04:30:19.734 INFO:teuthology.orchestra.run.smithi027.stdout:(22/33): python3-jaraco-collections-3.0.0-8.el9 344 kB/s | 23 kB 00:00 2024-04-02T04:30:19.760 INFO:teuthology.orchestra.run.smithi027.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 288 kB/s | 19 kB 00:00 2024-04-02T04:30:19.785 INFO:teuthology.orchestra.run.smithi027.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 922 kB/s | 46 kB 00:00 2024-04-02T04:30:19.819 INFO:teuthology.orchestra.run.smithi027.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 1.3 MB/s | 79 kB 00:00 2024-04-02T04:30:19.836 INFO:teuthology.orchestra.run.smithi027.stdout:(26/33): python3-jaraco-text-3.2.0-6.el9.noarch 167 kB/s | 20 kB 00:00 2024-04-02T04:30:19.861 INFO:teuthology.orchestra.run.smithi027.stdout:(27/33): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-04-02T04:30:19.878 INFO:teuthology.orchestra.run.smithi027.stdout:(28/33): python3-portend-3.1.0-2.el9.noarch.rpm 279 kB/s | 16 kB 00:00 2024-04-02T04:30:19.912 INFO:teuthology.orchestra.run.smithi027.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 714 kB/s | 36 kB 00:00 2024-04-02T04:30:19.953 INFO:teuthology.orchestra.run.smithi027.stdout:(30/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 767 kB/s | 90 kB 00:00 2024-04-02T04:30:19.979 INFO:teuthology.orchestra.run.smithi027.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-02T04:30:20.004 INFO:teuthology.orchestra.run.smithi027.stdout:(32/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.5 MB/s | 427 kB 00:00 2024-04-02T04:30:20.021 INFO:teuthology.orchestra.run.smithi027.stdout:(33/33): python3-zc-lockfile-2.0-10.el9.noarch. 298 kB/s | 20 kB 00:00 2024-04-02T04:30:20.024 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:30:20.024 INFO:teuthology.orchestra.run.smithi027.stdout:Total 12 MB/s | 29 MB 00:02 2024-04-02T04:30:20.039 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-04-02T04:30:20.054 INFO:teuthology.orchestra.run.smithi016.stdout:(1/33): ceph-18.2.2-1059.g4d05f4ca.el9.x86_64.r 42 kB/s | 6.5 kB 00:00 2024-04-02T04:30:20.083 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-04-02T04:30:20.083 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-04-02T04:30:20.288 INFO:teuthology.orchestra.run.smithi016.stdout:(2/33): ceph-mds-18.2.2-1059.g4d05f4ca.el9.x86_ 5.4 MB/s | 2.1 MB 00:00 2024-04-02T04:30:20.346 INFO:teuthology.orchestra.run.smithi016.stdout:(3/33): ceph-mgr-18.2.2-1059.g4d05f4ca.el9.x86_ 3.2 MB/s | 1.5 MB 00:00 2024-04-02T04:30:20.447 INFO:teuthology.orchestra.run.smithi016.stdout:(4/33): ceph-mgr-modules-core-18.2.2-1059.g4d05 2.4 MB/s | 245 kB 00:00 2024-04-02T04:30:20.454 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-04-02T04:30:20.455 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-04-02T04:30:20.572 INFO:teuthology.orchestra.run.smithi016.stdout:(5/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 144 kB 00:00 2024-04-02T04:30:20.639 INFO:teuthology.orchestra.run.smithi016.stdout:(6/33): ceph-mon-18.2.2-1059.g4d05f4ca.el9.x86_ 8.0 MB/s | 4.7 MB 00:00 2024-04-02T04:30:20.666 INFO:teuthology.orchestra.run.smithi016.stdout:(7/33): python3-markupsafe-1.1.1-12.el9.x86_64. 343 kB/s | 32 kB 00:00 2024-04-02T04:30:20.690 INFO:teuthology.orchestra.run.smithi016.stdout:(8/33): python3-pytz-2021.1-5.el9.noarch.rpm 941 kB/s | 47 kB 00:00 2024-04-02T04:30:20.715 INFO:teuthology.orchestra.run.smithi016.stdout:(9/33): python3-toml-0.10.2-6.el9.noarch.rpm 767 kB/s | 39 kB 00:00 2024-04-02T04:30:20.740 INFO:teuthology.orchestra.run.smithi016.stdout:(10/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-04-02T04:30:20.807 INFO:teuthology.orchestra.run.smithi016.stdout:(11/33): python3-cryptography-36.0.1-4.el9.x86_ 13 MB/s | 1.1 MB 00:00 2024-04-02T04:30:20.832 INFO:teuthology.orchestra.run.smithi016.stdout:(12/33): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 103 kB 00:00 2024-04-02T04:30:20.858 INFO:teuthology.orchestra.run.smithi016.stdout:(13/33): python3-pycparser-2.20-6.el9.noarch.rp 2.4 MB/s | 124 kB 00:00 2024-04-02T04:30:20.874 INFO:teuthology.orchestra.run.smithi016.stdout:(14/33): python3-urllib3-1.26.5-5.el9.noarch.rp 4.4 MB/s | 187 kB 00:00 2024-04-02T04:30:20.975 INFO:teuthology.orchestra.run.smithi016.stdout:(15/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 434 kB/s | 43 kB 00:00 2024-04-02T04:30:21.042 INFO:teuthology.orchestra.run.smithi016.stdout:(16/33): python3-requests-2.25.1-8.el9.noarch.r 681 kB/s | 125 kB 00:00 2024-04-02T04:30:21.067 INFO:teuthology.orchestra.run.smithi016.stdout:(17/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-04-02T04:30:21.242 INFO:teuthology.orchestra.run.smithi016.stdout:(18/33): ceph-osd-18.2.2-1059.g4d05f4ca.el9.x86 17 MB/s | 17 MB 00:00 2024-04-02T04:30:21.267 INFO:teuthology.orchestra.run.smithi016.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 53 kB/s | 11 kB 00:00 2024-04-02T04:30:21.293 INFO:teuthology.orchestra.run.smithi016.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 352 kB/s | 18 kB 00:00 2024-04-02T04:30:21.343 INFO:teuthology.orchestra.run.smithi016.stdout:(21/33): python3-jaraco-functools-3.5.0-2.el9.n 390 kB/s | 19 kB 00:00 2024-04-02T04:30:21.368 INFO:teuthology.orchestra.run.smithi016.stdout:(22/33): python3-jaraco-collections-3.0.0-8.el9 230 kB/s | 23 kB 00:00 2024-04-02T04:30:21.394 INFO:teuthology.orchestra.run.smithi016.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 391 kB/s | 20 kB 00:00 2024-04-02T04:30:21.436 INFO:teuthology.orchestra.run.smithi016.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 691 kB/s | 46 kB 00:00 2024-04-02T04:30:21.461 INFO:teuthology.orchestra.run.smithi016.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 1.2 MB/s | 79 kB 00:00 2024-04-02T04:30:21.486 INFO:teuthology.orchestra.run.smithi016.stdout:(26/33): python3-cherrypy-18.6.1-2.el9.noarch.r 807 kB/s | 358 kB 00:00 2024-04-02T04:30:21.511 INFO:teuthology.orchestra.run.smithi016.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-04-02T04:30:21.553 INFO:teuthology.orchestra.run.smithi016.stdout:(28/33): python3-tempora-5.0.0-2.el9.noarch.rpm 858 kB/s | 36 kB 00:00 2024-04-02T04:30:21.578 INFO:teuthology.orchestra.run.smithi016.stdout:(29/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-04-02T04:30:21.604 INFO:teuthology.orchestra.run.smithi016.stdout:(30/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 766 kB/s | 90 kB 00:00 2024-04-02T04:30:21.629 INFO:teuthology.orchestra.run.smithi016.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-02T04:30:21.654 INFO:teuthology.orchestra.run.smithi016.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 399 kB/s | 20 kB 00:00 2024-04-02T04:30:21.682 INFO:teuthology.orchestra.run.smithi016.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.1 MB/s | 427 kB 00:00 2024-04-02T04:30:21.685 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:30:21.686 INFO:teuthology.orchestra.run.smithi016.stdout:Total 14 MB/s | 29 MB 00:02 2024-04-02T04:30:21.701 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-04-02T04:30:21.744 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-04-02T04:30:21.745 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-04-02T04:30:21.767 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-04-02T04:30:21.832 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-04-02T04:30:21.865 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-04-02T04:30:21.898 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-04-02T04:30:21.995 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-04-02T04:30:22.079 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-04-02T04:30:22.113 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-04-02T04:30:22.114 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-04-02T04:30:22.126 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-04-02T04:30:22.201 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-04-02T04:30:22.261 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-04-02T04:30:22.298 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-04-02T04:30:22.357 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-04-02T04:30:22.421 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-04-02T04:30:22.619 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-04-02T04:30:22.677 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-04-02T04:30:22.716 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-04-02T04:30:22.750 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-04-02T04:30:22.789 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-04-02T04:30:22.867 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-04-02T04:30:22.899 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-04-02T04:30:22.932 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-04-02T04:30:22.965 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-04-02T04:30:23.008 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-04-02T04:30:23.043 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-04-02T04:30:23.136 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-04-02T04:30:23.176 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-04-02T04:30:23.257 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-04-02T04:30:23.399 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-04-02T04:30:23.529 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-04-02T04:30:23.569 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-04-02T04:30:23.643 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-04-02T04:30:23.684 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-04-02T04:30:23.691 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1059.g4d05f4ca.el9. 28/33 2024-04-02T04:30:23.720 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-2:18.2.2-1059.g4d05f4ca.el9.x86_64 29/33 2024-04-02T04:30:23.725 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-04-02T04:30:23.746 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1059.g4d05f4ca.el9.x86_64 29/33 2024-04-02T04:30:23.746 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T04:30:23.746 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-02T04:30:23.747 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T04:30:23.747 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T04:30:23.747 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:30:23.821 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-04-02T04:30:23.899 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-04-02T04:30:23.945 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-04-02T04:30:24.020 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-04-02T04:30:24.071 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-04-02T04:30:24.109 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-04-02T04:30:24.167 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-04-02T04:30:24.232 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-04-02T04:30:24.438 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-04-02T04:30:24.489 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-04-02T04:30:24.528 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-04-02T04:30:24.562 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-04-02T04:30:24.592 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-04-02T04:30:24.672 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-04-02T04:30:24.710 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-04-02T04:30:24.743 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-04-02T04:30:24.785 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-04-02T04:30:24.828 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-04-02T04:30:24.862 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-04-02T04:30:24.957 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-04-02T04:30:25.003 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-04-02T04:30:25.077 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-04-02T04:30:25.221 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-04-02T04:30:25.256 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-osd-2:18.2.2-1059.g4d05f4ca.el9.x86_64 30/33 2024-04-02T04:30:25.286 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-osd-2:18.2.2-1059.g4d05f4ca.el9.x86_64 30/33 2024-04-02T04:30:25.286 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T04:30:25.286 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-02T04:30:25.286 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T04:30:25.287 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T04:30:25.287 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:30:25.351 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-04-02T04:30:25.522 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1059.g4d05f4ca.el9. 28/33 2024-04-02T04:30:25.557 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-2:18.2.2-1059.g4d05f4ca.el9.x86_64 29/33 2024-04-02T04:30:25.583 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1059.g4d05f4ca.el9.x86_64 29/33 2024-04-02T04:30:25.584 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T04:30:25.584 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-02T04:30:25.584 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T04:30:25.584 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T04:30:25.584 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:30:25.749 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mon-2:18.2.2-1059.g4d05f4ca.el9.x86_64 31/33 2024-04-02T04:30:25.774 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mon-2:18.2.2-1059.g4d05f4ca.el9.x86_64 31/33 2024-04-02T04:30:25.775 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T04:30:25.775 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-02T04:30:25.775 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T04:30:25.775 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T04:30:25.775 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:30:25.983 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mds-2:18.2.2-1059.g4d05f4ca.el9.x86_64 32/33 2024-04-02T04:30:26.008 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mds-2:18.2.2-1059.g4d05f4ca.el9.x86_64 32/33 2024-04-02T04:30:26.008 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T04:30:26.008 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-02T04:30:26.008 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T04:30:26.008 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T04:30:26.009 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:30:26.074 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-2:18.2.2-1059.g4d05f4ca.el9.x86_64 33/33 2024-04-02T04:30:27.083 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-osd-2:18.2.2-1059.g4d05f4ca.el9.x86_64 30/33 2024-04-02T04:30:27.110 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-osd-2:18.2.2-1059.g4d05f4ca.el9.x86_64 30/33 2024-04-02T04:30:27.110 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T04:30:27.110 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-02T04:30:27.110 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T04:30:27.111 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T04:30:27.111 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:30:27.576 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mon-2:18.2.2-1059.g4d05f4ca.el9.x86_64 31/33 2024-04-02T04:30:27.599 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mon-2:18.2.2-1059.g4d05f4ca.el9.x86_64 31/33 2024-04-02T04:30:27.599 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T04:30:27.599 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-02T04:30:27.599 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T04:30:27.600 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T04:30:27.600 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:30:27.810 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mds-2:18.2.2-1059.g4d05f4ca.el9.x86_64 32/33 2024-04-02T04:30:27.814 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-2:18.2.2-1059.g4d05f4ca.el9.x86_64 33/33 2024-04-02T04:30:27.814 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-2:18.2.2-1059.g4d05f4ca.el9.x86_64 1/33 2024-04-02T04:30:27.814 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mds-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2/33 2024-04-02T04:30:27.814 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-2:18.2.2-1059.g4d05f4ca.el9.x86_64 3/33 2024-04-02T04:30:27.814 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mon-2:18.2.2-1059.g4d05f4ca.el9.x86_64 4/33 2024-04-02T04:30:27.814 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-osd-2:18.2.2-1059.g4d05f4ca.el9.x86_64 5/33 2024-04-02T04:30:27.814 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1059.g4d05f4ca.el9. 6/33 2024-04-02T04:30:27.814 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-04-02T04:30:27.814 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-04-02T04:30:27.814 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-04-02T04:30:27.814 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-04-02T04:30:27.815 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-04-02T04:30:27.815 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-04-02T04:30:27.815 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-04-02T04:30:27.815 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-04-02T04:30:27.815 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 15/33 2024-04-02T04:30:27.815 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/33 2024-04-02T04:30:27.816 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-04-02T04:30:27.816 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-04-02T04:30:27.816 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-04-02T04:30:27.816 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-04-02T04:30:27.816 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-04-02T04:30:27.816 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-04-02T04:30:27.817 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-04-02T04:30:27.817 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-04-02T04:30:27.817 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-04-02T04:30:27.819 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-04-02T04:30:27.820 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-04-02T04:30:27.820 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-04-02T04:30:27.820 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-04-02T04:30:27.820 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-04-02T04:30:27.820 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-04-02T04:30:27.820 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-04-02T04:30:27.833 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mds-2:18.2.2-1059.g4d05f4ca.el9.x86_64 32/33 2024-04-02T04:30:27.833 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T04:30:27.833 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-02T04:30:27.833 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T04:30:27.833 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T04:30:27.833 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:30:27.901 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-2:18.2.2-1059.g4d05f4ca.el9.x86_64 33/33 2024-04-02T04:30:28.530 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-04-02T04:30:28.530 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:30:28.530 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-04-02T04:30:28.531 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:28.531 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:28.531 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:28.531 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core-2:18.2.2-1059.g4d05f4ca.el9.noarch 2024-04-02T04:30:28.531 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:28.531 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:28.531 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-02T04:30:28.531 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-02T04:30:28.531 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-02T04:30:28.531 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-02T04:30:28.531 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-02T04:30:28.531 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-02T04:30:28.531 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-02T04:30:28.531 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-02T04:30:28.532 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-02T04:30:28.532 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-02T04:30:28.532 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-02T04:30:28.532 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-02T04:30:28.532 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-02T04:30:28.532 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-02T04:30:28.532 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-02T04:30:28.532 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ply-3.11-14.el9.noarch 2024-04-02T04:30:28.532 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-02T04:30:28.532 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-02T04:30:28.532 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-02T04:30:28.532 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-02T04:30:28.532 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-02T04:30:28.532 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-02T04:30:28.533 INFO:teuthology.orchestra.run.smithi027.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-02T04:30:28.533 INFO:teuthology.orchestra.run.smithi027.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-02T04:30:28.533 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-02T04:30:28.533 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-02T04:30:28.533 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-02T04:30:28.533 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:30:28.533 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-04-02T04:30:28.867 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-base 2024-04-02T04:30:29.458 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:00:53 ago on Tue 02 Apr 2024 04:29:36 AM UTC. 2024-04-02T04:30:29.559 INFO:teuthology.orchestra.run.smithi027.stdout:Package ceph-base-2:18.2.2-1059.g4d05f4ca.el9.x86_64 is already installed. 2024-04-02T04:30:29.614 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-04-02T04:30:29.615 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2024-04-02T04:30:29.615 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-04-02T04:30:29.631 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-2:18.2.2-1059.g4d05f4ca.el9.x86_64 33/33 2024-04-02T04:30:29.631 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-2:18.2.2-1059.g4d05f4ca.el9.x86_64 1/33 2024-04-02T04:30:29.631 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mds-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2/33 2024-04-02T04:30:29.631 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-2:18.2.2-1059.g4d05f4ca.el9.x86_64 3/33 2024-04-02T04:30:29.631 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mon-2:18.2.2-1059.g4d05f4ca.el9.x86_64 4/33 2024-04-02T04:30:29.631 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-osd-2:18.2.2-1059.g4d05f4ca.el9.x86_64 5/33 2024-04-02T04:30:29.631 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1059.g4d05f4ca.el9. 6/33 2024-04-02T04:30:29.631 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-04-02T04:30:29.632 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-04-02T04:30:29.632 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-04-02T04:30:29.632 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-04-02T04:30:29.632 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-04-02T04:30:29.632 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-04-02T04:30:29.632 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-04-02T04:30:29.632 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-04-02T04:30:29.632 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 15/33 2024-04-02T04:30:29.632 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/33 2024-04-02T04:30:29.632 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-04-02T04:30:29.632 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-04-02T04:30:29.632 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-04-02T04:30:29.632 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-04-02T04:30:29.633 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-04-02T04:30:29.633 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-04-02T04:30:29.633 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-04-02T04:30:29.633 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-04-02T04:30:29.633 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-04-02T04:30:29.633 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-04-02T04:30:29.633 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-04-02T04:30:29.633 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-04-02T04:30:29.633 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-04-02T04:30:29.633 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-04-02T04:30:29.633 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-04-02T04:30:29.633 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-04-02T04:30:29.790 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install cephadm 2024-04-02T04:30:30.377 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:00:54 ago on Tue 02 Apr 2024 04:29:36 AM UTC. 2024-04-02T04:30:30.535 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-04-02T04:30:30.536 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-02T04:30:30.536 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-04-02T04:30:30.536 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-02T04:30:30.536 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-04-02T04:30:30.536 INFO:teuthology.orchestra.run.smithi027.stdout: cephadm noarch 2:18.2.2-1059.g4d05f4ca.el9 ceph-noarch 224 k 2024-04-02T04:30:30.536 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:30:30.536 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-04-02T04:30:30.537 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-02T04:30:30.537 INFO:teuthology.orchestra.run.smithi027.stdout:Install 1 Package 2024-04-02T04:30:30.537 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:30:30.537 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 224 k 2024-04-02T04:30:30.537 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 216 k 2024-04-02T04:30:30.537 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-04-02T04:30:30.555 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-04-02T04:30:30.555 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:30:30.555 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-04-02T04:30:30.555 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:30.555 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:30.555 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:30.555 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core-2:18.2.2-1059.g4d05f4ca.el9.noarch 2024-04-02T04:30:30.555 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:30.555 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:30.555 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-02T04:30:30.555 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-02T04:30:30.555 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-02T04:30:30.555 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-02T04:30:30.556 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-02T04:30:30.556 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-02T04:30:30.556 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-02T04:30:30.556 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-02T04:30:30.556 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-02T04:30:30.556 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-02T04:30:30.556 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-02T04:30:30.556 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-02T04:30:30.556 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-02T04:30:30.556 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-02T04:30:30.556 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-02T04:30:30.556 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply-3.11-14.el9.noarch 2024-04-02T04:30:30.556 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-02T04:30:30.556 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-02T04:30:30.556 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-02T04:30:30.557 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-02T04:30:30.557 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-02T04:30:30.557 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-02T04:30:30.557 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-02T04:30:30.557 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-02T04:30:30.557 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-02T04:30:30.557 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-02T04:30:30.557 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-02T04:30:30.557 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:30:30.557 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-02T04:30:30.782 INFO:teuthology.orchestra.run.smithi027.stdout:cephadm-18.2.2-1059.g4d05f4ca.el9.noarch.rpm 917 kB/s | 224 kB 00:00 2024-04-02T04:30:30.782 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:30:30.782 INFO:teuthology.orchestra.run.smithi027.stdout:Total 910 kB/s | 224 kB 00:00 2024-04-02T04:30:30.783 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-04-02T04:30:30.788 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-04-02T04:30:30.788 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-04-02T04:30:30.792 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-04-02T04:30:30.792 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-04-02T04:30:30.860 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-04-02T04:30:30.872 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-base 2024-04-02T04:30:31.253 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: cephadm-2:18.2.2-1059.g4d05f4ca.el9.noarch 1/1 2024-04-02T04:30:31.347 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephadm-2:18.2.2-1059.g4d05f4ca.el9.noarch 1/1 2024-04-02T04:30:31.447 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:00:52 ago on Tue 02 Apr 2024 04:29:39 AM UTC. 2024-04-02T04:30:31.549 INFO:teuthology.orchestra.run.smithi016.stdout:Package ceph-base-2:18.2.2-1059.g4d05f4ca.el9.x86_64 is already installed. 2024-04-02T04:30:31.605 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-02T04:30:31.605 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2024-04-02T04:30:31.606 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-02T04:30:31.658 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install cephadm 2024-04-02T04:30:31.691 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: cephadm-2:18.2.2-1059.g4d05f4ca.el9.noarch 1/1 2024-04-02T04:30:31.977 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephadm-2:18.2.2-1059.g4d05f4ca.el9.noarch 1/1 2024-04-02T04:30:31.977 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:30:31.977 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-04-02T04:30:31.977 INFO:teuthology.orchestra.run.smithi027.stdout: cephadm-2:18.2.2-1059.g4d05f4ca.el9.noarch 2024-04-02T04:30:31.977 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:30:31.977 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-04-02T04:30:32.173 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-immutable-object-cache 2024-04-02T04:30:32.228 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:00:53 ago on Tue 02 Apr 2024 04:29:39 AM UTC. 2024-04-02T04:30:32.385 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-02T04:30:32.386 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-02T04:30:32.386 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-04-02T04:30:32.386 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-02T04:30:32.386 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-04-02T04:30:32.386 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm noarch 2:18.2.2-1059.g4d05f4ca.el9 ceph-noarch 224 k 2024-04-02T04:30:32.386 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:30:32.386 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-04-02T04:30:32.386 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-02T04:30:32.386 INFO:teuthology.orchestra.run.smithi016.stdout:Install 1 Package 2024-04-02T04:30:32.386 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:30:32.387 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 224 k 2024-04-02T04:30:32.387 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 216 k 2024-04-02T04:30:32.387 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-04-02T04:30:32.609 INFO:teuthology.orchestra.run.smithi016.stdout:cephadm-18.2.2-1059.g4d05f4ca.el9.noarch.rpm 1.0 MB/s | 224 kB 00:00 2024-04-02T04:30:32.609 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:30:32.609 INFO:teuthology.orchestra.run.smithi016.stdout:Total 1.0 MB/s | 224 kB 00:00 2024-04-02T04:30:32.609 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-04-02T04:30:32.615 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-04-02T04:30:32.615 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-04-02T04:30:32.618 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-04-02T04:30:32.618 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-04-02T04:30:32.690 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-04-02T04:30:32.766 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:00:56 ago on Tue 02 Apr 2024 04:29:36 AM UTC. 2024-04-02T04:30:32.923 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-04-02T04:30:32.924 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-02T04:30:32.924 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repo Size 2024-04-02T04:30:32.924 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-02T04:30:32.924 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-04-02T04:30:32.924 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 143 k 2024-04-02T04:30:32.924 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:30:32.924 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-04-02T04:30:32.924 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-02T04:30:32.924 INFO:teuthology.orchestra.run.smithi027.stdout:Install 1 Package 2024-04-02T04:30:32.925 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:30:32.925 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 143 k 2024-04-02T04:30:32.925 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 395 k 2024-04-02T04:30:32.925 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-04-02T04:30:33.068 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephadm-2:18.2.2-1059.g4d05f4ca.el9.noarch 1/1 2024-04-02T04:30:33.149 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephadm-2:18.2.2-1059.g4d05f4ca.el9.noarch 1/1 2024-04-02T04:30:33.280 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-immutable-object-cache-18.2.2-1059.g4d05f4 402 kB/s | 143 kB 00:00 2024-04-02T04:30:33.281 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:30:33.281 INFO:teuthology.orchestra.run.smithi027.stdout:Total 400 kB/s | 143 kB 00:00 2024-04-02T04:30:33.281 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-04-02T04:30:33.289 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-04-02T04:30:33.289 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-04-02T04:30:33.346 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-04-02T04:30:33.346 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-04-02T04:30:33.484 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephadm-2:18.2.2-1059.g4d05f4ca.el9.noarch 1/1 2024-04-02T04:30:33.605 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-04-02T04:30:33.664 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1059.g4d05f4ca. 1/1 2024-04-02T04:30:33.690 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1059.g4d05f4ca. 1/1 2024-04-02T04:30:33.691 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T04:30:33.691 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-02T04:30:33.691 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:30:33.755 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephadm-2:18.2.2-1059.g4d05f4ca.el9.noarch 1/1 2024-04-02T04:30:33.755 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:30:33.755 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-04-02T04:30:33.755 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm-2:18.2.2-1059.g4d05f4ca.el9.noarch 2024-04-02T04:30:33.755 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:30:33.755 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-02T04:30:33.933 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-immutable-object-cache 2024-04-02T04:30:34.509 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:00:55 ago on Tue 02 Apr 2024 04:29:39 AM UTC. 2024-04-02T04:30:34.628 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1059.g4d05f4ca. 1/1 2024-04-02T04:30:34.628 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:30:34.628 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-04-02T04:30:34.628 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-immutable-object-cache-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:34.628 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:30:34.628 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-04-02T04:30:34.667 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-02T04:30:34.668 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-02T04:30:34.668 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repo Size 2024-04-02T04:30:34.668 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-02T04:30:34.668 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-04-02T04:30:34.668 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 143 k 2024-04-02T04:30:34.668 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:30:34.668 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-04-02T04:30:34.668 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-02T04:30:34.669 INFO:teuthology.orchestra.run.smithi016.stdout:Install 1 Package 2024-04-02T04:30:34.669 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:30:34.669 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 143 k 2024-04-02T04:30:34.669 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 395 k 2024-04-02T04:30:34.669 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-04-02T04:30:34.857 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-mgr 2024-04-02T04:30:35.019 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-immutable-object-cache-18.2.2-1059.g4d05f4 407 kB/s | 143 kB 00:00 2024-04-02T04:30:35.020 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:30:35.020 INFO:teuthology.orchestra.run.smithi016.stdout:Total 405 kB/s | 143 kB 00:00 2024-04-02T04:30:35.020 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-04-02T04:30:35.028 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-04-02T04:30:35.029 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-04-02T04:30:35.082 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-04-02T04:30:35.083 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-04-02T04:30:35.225 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-04-02T04:30:35.295 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1059.g4d05f4ca. 1/1 2024-04-02T04:30:35.319 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1059.g4d05f4ca. 1/1 2024-04-02T04:30:35.319 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T04:30:35.319 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-02T04:30:35.320 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:30:35.451 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:00:59 ago on Tue 02 Apr 2024 04:29:36 AM UTC. 2024-04-02T04:30:35.553 INFO:teuthology.orchestra.run.smithi027.stdout:Package ceph-mgr-2:18.2.2-1059.g4d05f4ca.el9.x86_64 is already installed. 2024-04-02T04:30:35.609 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-04-02T04:30:35.610 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2024-04-02T04:30:35.610 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-04-02T04:30:35.674 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-mgr-dashboard 2024-04-02T04:30:36.259 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:00 ago on Tue 02 Apr 2024 04:29:36 AM UTC. 2024-04-02T04:30:36.381 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1059.g4d05f4ca. 1/1 2024-04-02T04:30:36.381 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:30:36.381 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-04-02T04:30:36.381 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:30:36.381 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:30:36.381 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-02T04:30:36.416 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-04-02T04:30:36.417 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-02T04:30:36.417 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-04-02T04:30:36.417 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-02T04:30:36.417 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-04-02T04:30:36.417 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1059.g4d05f4ca.el9 ceph-noarch 1.8 M 2024-04-02T04:30:36.418 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-04-02T04:30:36.418 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1059.g4d05f4ca.el9 ceph-noarch 23 k 2024-04-02T04:30:36.418 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1059.g4d05f4ca.el9 ceph-noarch 15 k 2024-04-02T04:30:36.418 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-02T04:30:36.418 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-02T04:30:36.418 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:30:36.418 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-04-02T04:30:36.418 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-02T04:30:36.418 INFO:teuthology.orchestra.run.smithi027.stdout:Install 5 Packages 2024-04-02T04:30:36.418 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:30:36.418 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 2.0 M 2024-04-02T04:30:36.418 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 8.6 M 2024-04-02T04:30:36.419 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-04-02T04:30:36.633 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-mgr 2024-04-02T04:30:36.742 INFO:teuthology.orchestra.run.smithi027.stdout:(1/5): ceph-grafana-dashboards-18.2.2-1059.g4d0 94 kB/s | 23 kB 00:00 2024-04-02T04:30:36.776 INFO:teuthology.orchestra.run.smithi027.stdout:(2/5): ceph-prometheus-alerts-18.2.2-1059.g4d05 53 kB/s | 15 kB 00:00 2024-04-02T04:30:36.926 INFO:teuthology.orchestra.run.smithi027.stdout:(3/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 167 kB/s | 31 kB 00:00 2024-04-02T04:30:37.018 INFO:teuthology.orchestra.run.smithi027.stdout:(4/5): ceph-mgr-dashboard-18.2.2-1059.g4d05f4ca 3.4 MB/s | 1.8 MB 00:00 2024-04-02T04:30:37.043 INFO:teuthology.orchestra.run.smithi027.stdout:(5/5): python3-routes-2.5.1-5.el9.noarch.rpm 706 kB/s | 188 kB 00:00 2024-04-02T04:30:37.044 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:30:37.045 INFO:teuthology.orchestra.run.smithi027.stdout:Total 3.3 MB/s | 2.0 MB 00:00 2024-04-02T04:30:37.045 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-04-02T04:30:37.052 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-04-02T04:30:37.053 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-04-02T04:30:37.113 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-04-02T04:30:37.113 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-04-02T04:30:37.211 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:00:58 ago on Tue 02 Apr 2024 04:29:39 AM UTC. 2024-04-02T04:30:37.313 INFO:teuthology.orchestra.run.smithi016.stdout:Package ceph-mgr-2:18.2.2-1059.g4d05f4ca.el9.x86_64 is already installed. 2024-04-02T04:30:37.358 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-04-02T04:30:37.368 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-02T04:30:37.369 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2024-04-02T04:30:37.369 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-02T04:30:37.427 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-mgr-dashboard 2024-04-02T04:30:37.442 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-04-02T04:30:37.473 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-04-02T04:30:37.502 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1059.g4d05f4ca.el9.n 3/5 2024-04-02T04:30:37.754 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1059.g4d05f4ca.el9. 4/5 2024-04-02T04:30:37.788 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1059.g4d05f4ca.el9.noarc 5/5 2024-04-02T04:30:37.995 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:00:58 ago on Tue 02 Apr 2024 04:29:39 AM UTC. 2024-04-02T04:30:38.150 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-02T04:30:38.151 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-02T04:30:38.151 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-04-02T04:30:38.151 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-02T04:30:38.151 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-04-02T04:30:38.151 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1059.g4d05f4ca.el9 ceph-noarch 1.8 M 2024-04-02T04:30:38.151 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-04-02T04:30:38.151 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1059.g4d05f4ca.el9 ceph-noarch 23 k 2024-04-02T04:30:38.151 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1059.g4d05f4ca.el9 ceph-noarch 15 k 2024-04-02T04:30:38.151 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-02T04:30:38.151 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-02T04:30:38.152 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:30:38.152 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-04-02T04:30:38.152 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-02T04:30:38.152 INFO:teuthology.orchestra.run.smithi016.stdout:Install 5 Packages 2024-04-02T04:30:38.152 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:30:38.153 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 2.0 M 2024-04-02T04:30:38.153 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 8.6 M 2024-04-02T04:30:38.153 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-04-02T04:30:38.332 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1059.g4d05f4ca.el9.noarc 5/5 2024-04-02T04:30:38.332 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1059.g4d05f4ca.el9. 1/5 2024-04-02T04:30:38.332 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1059.g4d05f4ca.el9.noarc 2/5 2024-04-02T04:30:38.332 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1059.g4d05f4ca.el9.n 3/5 2024-04-02T04:30:38.333 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-04-02T04:30:38.399 INFO:teuthology.orchestra.run.smithi016.stdout:(1/5): ceph-prometheus-alerts-18.2.2-1059.g4d05 78 kB/s | 15 kB 00:00 2024-04-02T04:30:38.491 INFO:teuthology.orchestra.run.smithi016.stdout:(2/5): ceph-grafana-dashboards-18.2.2-1059.g4d0 83 kB/s | 23 kB 00:00 2024-04-02T04:30:38.558 INFO:teuthology.orchestra.run.smithi016.stdout:(3/5): ceph-mgr-dashboard-18.2.2-1059.g4d05f4ca 5.1 MB/s | 1.8 MB 00:00 2024-04-02T04:30:38.574 INFO:teuthology.orchestra.run.smithi016.stdout:(4/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 175 kB/s | 31 kB 00:00 2024-04-02T04:30:38.626 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-04-02T04:30:38.627 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:30:38.627 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-04-02T04:30:38.627 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards-2:18.2.2-1059.g4d05f4ca.el9.noarch 2024-04-02T04:30:38.627 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard-2:18.2.2-1059.g4d05f4ca.el9.noarch 2024-04-02T04:30:38.627 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts-2:18.2.2-1059.g4d05f4ca.el9.noarch 2024-04-02T04:30:38.627 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-02T04:30:38.627 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-02T04:30:38.627 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:30:38.627 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-04-02T04:30:38.741 INFO:teuthology.orchestra.run.smithi016.stdout:(5/5): python3-routes-2.5.1-5.el9.noarch.rpm 753 kB/s | 188 kB 00:00 2024-04-02T04:30:38.743 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:30:38.743 INFO:teuthology.orchestra.run.smithi016.stdout:Total 3.5 MB/s | 2.0 MB 00:00 2024-04-02T04:30:38.743 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-04-02T04:30:38.751 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-04-02T04:30:38.751 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-04-02T04:30:38.810 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-04-02T04:30:38.810 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-04-02T04:30:38.840 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-02T04:30:39.052 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-04-02T04:30:39.141 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-04-02T04:30:39.171 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-04-02T04:30:39.201 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1059.g4d05f4ca.el9.n 3/5 2024-04-02T04:30:39.436 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:03 ago on Tue 02 Apr 2024 04:29:36 AM UTC. 2024-04-02T04:30:39.449 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1059.g4d05f4ca.el9. 4/5 2024-04-02T04:30:39.495 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1059.g4d05f4ca.el9.noarc 5/5 2024-04-02T04:30:39.595 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-04-02T04:30:39.596 INFO:teuthology.orchestra.run.smithi027.stdout:============================================================================================ 2024-04-02T04:30:39.596 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-04-02T04:30:39.596 INFO:teuthology.orchestra.run.smithi027.stdout:============================================================================================ 2024-04-02T04:30:39.596 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-04-02T04:30:39.596 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1059.g4d05f4ca.el9 ceph-noarch 7.4 M 2024-04-02T04:30:39.596 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-04-02T04:30:39.596 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-02T04:30:39.596 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-02T04:30:39.596 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-02T04:30:39.597 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-02T04:30:39.597 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-02T04:30:39.597 INFO:teuthology.orchestra.run.smithi027.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-02T04:30:39.597 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-02T04:30:39.597 INFO:teuthology.orchestra.run.smithi027.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-02T04:30:39.597 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-02T04:30:39.597 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-02T04:30:39.597 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-02T04:30:39.597 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:30:39.597 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-04-02T04:30:39.597 INFO:teuthology.orchestra.run.smithi027.stdout:============================================================================================ 2024-04-02T04:30:39.597 INFO:teuthology.orchestra.run.smithi027.stdout:Install 12 Packages 2024-04-02T04:30:39.597 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:30:39.598 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 37 M 2024-04-02T04:30:39.598 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 209 M 2024-04-02T04:30:39.598 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-04-02T04:30:40.000 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1059.g4d05f4ca.el9.noarc 5/5 2024-04-02T04:30:40.000 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1059.g4d05f4ca.el9. 1/5 2024-04-02T04:30:40.000 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1059.g4d05f4ca.el9.noarc 2/5 2024-04-02T04:30:40.000 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1059.g4d05f4ca.el9.n 3/5 2024-04-02T04:30:40.000 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-04-02T04:30:40.274 INFO:teuthology.orchestra.run.smithi027.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 232 kB/s | 32 kB 00:00 2024-04-02T04:30:40.279 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-04-02T04:30:40.279 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:30:40.279 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-04-02T04:30:40.279 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards-2:18.2.2-1059.g4d05f4ca.el9.noarch 2024-04-02T04:30:40.279 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard-2:18.2.2-1059.g4d05f4ca.el9.noarch 2024-04-02T04:30:40.279 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts-2:18.2.2-1059.g4d05f4ca.el9.noarch 2024-04-02T04:30:40.279 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-02T04:30:40.280 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-02T04:30:40.280 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:30:40.280 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-02T04:30:40.307 INFO:teuthology.orchestra.run.smithi027.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 522 kB/s | 17 kB 00:00 2024-04-02T04:30:40.374 INFO:teuthology.orchestra.run.smithi027.stdout:(3/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 13 MB/s | 3.0 MB 00:00 2024-04-02T04:30:40.400 INFO:teuthology.orchestra.run.smithi027.stdout:(4/12): openblas-0.3.21-2.el9.x86_64.rpm 378 kB/s | 34 kB 00:00 2024-04-02T04:30:40.475 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-02T04:30:40.567 INFO:teuthology.orchestra.run.smithi027.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 25 MB/s | 4.7 MB 00:00 2024-04-02T04:30:40.633 INFO:teuthology.orchestra.run.smithi027.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 21 MB/s | 4.8 MB 00:00 2024-04-02T04:30:40.659 INFO:teuthology.orchestra.run.smithi027.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.4 MB/s | 217 kB 00:00 2024-04-02T04:30:40.776 INFO:teuthology.orchestra.run.smithi027.stdout:(8/12): ceph-mgr-diskprediction-local-18.2.2-10 12 MB/s | 7.4 MB 00:00 2024-04-02T04:30:41.018 INFO:teuthology.orchestra.run.smithi027.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 41 MB/s | 16 MB 00:00 2024-04-02T04:30:41.054 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:02 ago on Tue 02 Apr 2024 04:29:39 AM UTC. 2024-04-02T04:30:41.101 INFO:teuthology.orchestra.run.smithi027.stdout:(10/12): libquadmath-11.4.1-3.el9.x86_64.rpm 605 kB/s | 195 kB 00:00 2024-04-02T04:30:41.135 INFO:teuthology.orchestra.run.smithi027.stdout:(11/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.7 MB/s | 803 kB 00:00 2024-04-02T04:30:41.213 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-02T04:30:41.214 INFO:teuthology.orchestra.run.smithi016.stdout:============================================================================================ 2024-04-02T04:30:41.214 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-04-02T04:30:41.214 INFO:teuthology.orchestra.run.smithi016.stdout:============================================================================================ 2024-04-02T04:30:41.214 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-04-02T04:30:41.214 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1059.g4d05f4ca.el9 ceph-noarch 7.4 M 2024-04-02T04:30:41.215 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-04-02T04:30:41.215 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-02T04:30:41.215 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-02T04:30:41.215 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-02T04:30:41.215 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-02T04:30:41.215 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-02T04:30:41.215 INFO:teuthology.orchestra.run.smithi016.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-02T04:30:41.215 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-02T04:30:41.215 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-02T04:30:41.215 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-02T04:30:41.215 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-02T04:30:41.215 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-02T04:30:41.215 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:30:41.215 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-04-02T04:30:41.215 INFO:teuthology.orchestra.run.smithi016.stdout:============================================================================================ 2024-04-02T04:30:41.216 INFO:teuthology.orchestra.run.smithi016.stdout:Install 12 Packages 2024-04-02T04:30:41.216 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:30:41.216 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 37 M 2024-04-02T04:30:41.216 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 209 M 2024-04-02T04:30:41.216 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-04-02T04:30:41.327 INFO:teuthology.orchestra.run.smithi027.stdout:(12/12): python3-devel-3.9.17-2.el9.x86_64.rpm 795 kB/s | 245 kB 00:00 2024-04-02T04:30:41.330 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:30:41.330 INFO:teuthology.orchestra.run.smithi027.stdout:Total 21 MB/s | 37 MB 00:01 2024-04-02T04:30:41.372 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-04-02T04:30:41.398 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-04-02T04:30:41.398 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-04-02T04:30:41.861 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-04-02T04:30:41.861 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-04-02T04:30:41.898 INFO:teuthology.orchestra.run.smithi016.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 240 kB/s | 32 kB 00:00 2024-04-02T04:30:41.939 INFO:teuthology.orchestra.run.smithi016.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 425 kB/s | 17 kB 00:00 2024-04-02T04:30:41.990 INFO:teuthology.orchestra.run.smithi016.stdout:(3/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 13 MB/s | 3.0 MB 00:00 2024-04-02T04:30:42.015 INFO:teuthology.orchestra.run.smithi016.stdout:(4/12): openblas-0.3.21-2.el9.x86_64.rpm 470 kB/s | 34 kB 00:00 2024-04-02T04:30:42.191 INFO:teuthology.orchestra.run.smithi016.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 24 MB/s | 4.7 MB 00:00 2024-04-02T04:30:42.257 INFO:teuthology.orchestra.run.smithi016.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 4.8 MB 00:00 2024-04-02T04:30:42.283 INFO:teuthology.orchestra.run.smithi016.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.4 MB/s | 217 kB 00:00 2024-04-02T04:30:42.517 INFO:teuthology.orchestra.run.smithi016.stdout:(8/12): ceph-mgr-diskprediction-local-18.2.2-10 9.9 MB/s | 7.4 MB 00:00 2024-04-02T04:30:42.684 INFO:teuthology.orchestra.run.smithi016.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-04-02T04:30:42.710 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-04-02T04:30:42.794 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-04-02T04:30:42.817 INFO:teuthology.orchestra.run.smithi016.stdout:(10/12): python3-devel-3.9.17-2.el9.x86_64.rpm 1.8 MB/s | 245 kB 00:00 2024-04-02T04:30:42.839 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-02T04:30:42.938 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-04-02T04:30:42.985 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-04-02T04:30:43.092 INFO:teuthology.orchestra.run.smithi016.stdout:(11/12): libquadmath-11.4.1-3.el9.x86_64.rpm 341 kB/s | 195 kB 00:00 2024-04-02T04:30:43.126 INFO:teuthology.orchestra.run.smithi016.stdout:(12/12): libgfortran-11.4.1-3.el9.x86_64.rpm 955 kB/s | 803 kB 00:00 2024-04-02T04:30:43.129 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:30:43.129 INFO:teuthology.orchestra.run.smithi016.stdout:Total 19 MB/s | 37 MB 00:01 2024-04-02T04:30:43.170 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-04-02T04:30:43.175 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-02T04:30:43.195 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-04-02T04:30:43.195 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-04-02T04:30:43.208 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-02T04:30:43.287 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-02T04:30:43.648 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-04-02T04:30:43.648 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-04-02T04:30:44.094 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-02T04:30:44.217 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-02T04:30:44.365 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-04-02T04:30:44.444 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-04-02T04:30:44.472 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-02T04:30:44.571 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-04-02T04:30:44.618 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-04-02T04:30:44.810 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-02T04:30:44.841 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-02T04:30:44.920 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-02T04:30:45.617 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-02T04:30:45.643 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-02T04:30:45.733 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-02T04:30:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-02T04:30:46.749 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1059.g4d05f 12/12 2024-04-02T04:30:47.047 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-02T04:30:48.130 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-02T04:30:48.158 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1059.g4d05f 12/12 2024-04-02T04:30:49.436 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1059.g4d05f 12/12 2024-04-02T04:30:49.436 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1059.g4d05f 1/12 2024-04-02T04:30:49.436 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-04-02T04:30:49.436 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-04-02T04:30:49.436 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-04-02T04:30:49.436 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-02T04:30:49.437 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-02T04:30:49.437 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-04-02T04:30:49.437 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-04-02T04:30:49.442 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-04-02T04:30:49.442 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-04-02T04:30:49.443 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-04-02T04:30:49.819 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-04-02T04:30:49.819 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:30:49.819 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-04-02T04:30:49.819 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1059.g4d05f4ca.el9.noarch 2024-04-02T04:30:49.819 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-02T04:30:49.819 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-02T04:30:49.819 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-02T04:30:49.820 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-02T04:30:49.820 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-02T04:30:49.820 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-02T04:30:49.820 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-02T04:30:49.820 INFO:teuthology.orchestra.run.smithi027.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-02T04:30:49.820 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-02T04:30:49.820 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-02T04:30:49.820 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-02T04:30:49.820 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:30:49.820 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-04-02T04:30:50.188 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-mgr-rook 2024-04-02T04:30:50.619 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1059.g4d05f 12/12 2024-04-02T04:30:50.619 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1059.g4d05f 1/12 2024-04-02T04:30:50.619 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-04-02T04:30:50.619 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-04-02T04:30:50.620 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-04-02T04:30:50.620 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-02T04:30:50.620 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-02T04:30:50.620 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-04-02T04:30:50.620 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-04-02T04:30:50.620 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-04-02T04:30:50.620 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-04-02T04:30:50.620 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-04-02T04:30:50.784 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:14 ago on Tue 02 Apr 2024 04:29:36 AM UTC. 2024-04-02T04:30:50.947 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-04-02T04:30:50.948 INFO:teuthology.orchestra.run.smithi027.stdout:======================================================================================== 2024-04-02T04:30:50.948 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-04-02T04:30:50.948 INFO:teuthology.orchestra.run.smithi027.stdout:======================================================================================== 2024-04-02T04:30:50.949 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-04-02T04:30:50.949 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook noarch 2:18.2.2-1059.g4d05f4ca.el9 ceph-noarch 49 k 2024-04-02T04:30:50.949 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-04-02T04:30:50.949 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-02T04:30:50.949 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-02T04:30:50.949 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-02T04:30:50.949 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-02T04:30:50.949 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-02T04:30:50.949 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-02T04:30:50.949 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-02T04:30:50.949 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-02T04:30:50.949 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-02T04:30:50.949 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-02T04:30:50.949 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-02T04:30:50.949 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-02T04:30:50.949 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:30:50.950 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-04-02T04:30:50.950 INFO:teuthology.orchestra.run.smithi027.stdout:======================================================================================== 2024-04-02T04:30:50.950 INFO:teuthology.orchestra.run.smithi027.stdout:Install 13 Packages 2024-04-02T04:30:50.950 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:30:50.950 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 2.0 M 2024-04-02T04:30:50.950 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 26 M 2024-04-02T04:30:50.950 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-04-02T04:30:51.033 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-04-02T04:30:51.034 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:30:51.034 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-04-02T04:30:51.034 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1059.g4d05f4ca.el9.noarch 2024-04-02T04:30:51.034 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-02T04:30:51.034 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-02T04:30:51.034 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-02T04:30:51.034 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-02T04:30:51.034 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-02T04:30:51.034 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-02T04:30:51.034 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-02T04:30:51.034 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-02T04:30:51.034 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-02T04:30:51.035 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-02T04:30:51.035 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-02T04:30:51.035 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:30:51.035 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-02T04:30:51.149 INFO:teuthology.orchestra.run.smithi027.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 203 kB/s | 24 kB 00:00 2024-04-02T04:30:51.183 INFO:teuthology.orchestra.run.smithi027.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 118 kB/s | 18 kB 00:00 2024-04-02T04:30:51.233 INFO:teuthology.orchestra.run.smithi027.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.1 MB/s | 174 kB 00:00 2024-04-02T04:30:51.258 INFO:teuthology.orchestra.run.smithi027.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-04-02T04:30:51.284 INFO:teuthology.orchestra.run.smithi027.stdout:(5/13): ceph-mgr-rook-18.2.2-1059.g4d05f4ca.el9 194 kB/s | 49 kB 00:00 2024-04-02T04:30:51.309 INFO:teuthology.orchestra.run.smithi027.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.7 MB/s | 211 kB 00:00 2024-04-02T04:30:51.326 INFO:teuthology.orchestra.run.smithi027.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 639 kB/s | 43 kB 00:00 2024-04-02T04:30:51.395 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-mgr-rook 2024-04-02T04:30:51.493 INFO:teuthology.orchestra.run.smithi027.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 154 kB/s | 32 kB 00:00 2024-04-02T04:30:51.509 INFO:teuthology.orchestra.run.smithi027.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 82 kB/s | 15 kB 00:00 2024-04-02T04:30:51.660 INFO:teuthology.orchestra.run.smithi027.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 395 kB/s | 59 kB 00:00 2024-04-02T04:30:51.685 INFO:teuthology.orchestra.run.smithi027.stdout:(11/13): python3-google-auth-2.28.2-1.el9.noarc 565 kB/s | 203 kB 00:00 2024-04-02T04:30:51.835 INFO:teuthology.orchestra.run.smithi027.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 511 kB/s | 90 kB 00:00 2024-04-02T04:30:51.969 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:12 ago on Tue 02 Apr 2024 04:29:39 AM UTC. 2024-04-02T04:30:52.127 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-02T04:30:52.129 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================== 2024-04-02T04:30:52.129 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-04-02T04:30:52.129 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================== 2024-04-02T04:30:52.129 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-04-02T04:30:52.129 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook noarch 2:18.2.2-1059.g4d05f4ca.el9 ceph-noarch 49 k 2024-04-02T04:30:52.129 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-04-02T04:30:52.129 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-02T04:30:52.129 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-02T04:30:52.129 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-02T04:30:52.130 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-02T04:30:52.130 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-02T04:30:52.130 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-02T04:30:52.130 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-02T04:30:52.130 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-02T04:30:52.130 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-02T04:30:52.130 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-02T04:30:52.130 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-02T04:30:52.130 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-02T04:30:52.130 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:30:52.130 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-04-02T04:30:52.130 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================== 2024-04-02T04:30:52.131 INFO:teuthology.orchestra.run.smithi016.stdout:Install 13 Packages 2024-04-02T04:30:52.131 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:30:52.131 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 2.0 M 2024-04-02T04:30:52.131 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 26 M 2024-04-02T04:30:52.131 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-04-02T04:30:52.310 INFO:teuthology.orchestra.run.smithi027.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 1.3 MB/s | 1.0 MB 00:00 2024-04-02T04:30:52.313 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:30:52.313 INFO:teuthology.orchestra.run.smithi027.stdout:Total 1.5 MB/s | 2.0 MB 00:01 2024-04-02T04:30:52.313 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-04-02T04:30:52.329 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-04-02T04:30:52.329 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-04-02T04:30:52.342 INFO:teuthology.orchestra.run.smithi016.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 181 kB/s | 24 kB 00:00 2024-04-02T04:30:52.367 INFO:teuthology.orchestra.run.smithi016.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 113 kB/s | 18 kB 00:00 2024-04-02T04:30:52.426 INFO:teuthology.orchestra.run.smithi016.stdout:(3/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 132 kB 00:00 2024-04-02T04:30:52.438 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-04-02T04:30:52.439 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-04-02T04:30:52.451 INFO:teuthology.orchestra.run.smithi016.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.1 MB/s | 174 kB 00:00 2024-04-02T04:30:52.476 INFO:teuthology.orchestra.run.smithi016.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.1 MB/s | 211 kB 00:00 2024-04-02T04:30:52.502 INFO:teuthology.orchestra.run.smithi016.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 853 kB/s | 43 kB 00:00 2024-04-02T04:30:52.602 INFO:teuthology.orchestra.run.smithi016.stdout:(7/13): ceph-mgr-rook-18.2.2-1059.g4d05f4ca.el9 125 kB/s | 49 kB 00:00 2024-04-02T04:30:52.677 INFO:teuthology.orchestra.run.smithi016.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 85 kB/s | 15 kB 00:00 2024-04-02T04:30:52.703 INFO:teuthology.orchestra.run.smithi016.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 142 kB/s | 32 kB 00:00 2024-04-02T04:30:52.794 INFO:teuthology.orchestra.run.smithi016.stdout:(10/13): python3-google-auth-2.28.2-1.el9.noarc 1.0 MB/s | 203 kB 00:00 2024-04-02T04:30:52.895 INFO:teuthology.orchestra.run.smithi016.stdout:(11/13): python3-kubernetes-26.1.0-2.el9.noarch 4.7 MB/s | 1.0 MB 00:00 2024-04-02T04:30:52.919 INFO:teuthology.orchestra.run.smithi016.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 272 kB/s | 59 kB 00:00 2024-04-02T04:30:52.969 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-04-02T04:30:52.995 INFO:teuthology.orchestra.run.smithi016.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 449 kB/s | 90 kB 00:00 2024-04-02T04:30:52.999 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:30:52.999 INFO:teuthology.orchestra.run.smithi016.stdout:Total 2.4 MB/s | 2.0 MB 00:00 2024-04-02T04:30:53.000 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-04-02T04:30:53.015 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-04-02T04:30:53.015 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-04-02T04:30:53.111 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-04-02T04:30:53.112 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-04-02T04:30:53.141 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-02T04:30:53.198 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-02T04:30:53.253 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-02T04:30:53.288 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-02T04:30:53.330 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-02T04:30:53.428 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-02T04:30:53.539 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-04-02T04:30:53.591 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-02T04:30:53.639 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-04-02T04:30:53.808 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-02T04:30:53.865 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-02T04:30:53.929 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-02T04:30:53.964 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-02T04:30:54.006 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-02T04:30:54.104 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-02T04:30:54.222 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-04-02T04:30:54.275 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-02T04:30:54.373 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-02T04:30:54.419 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-02T04:30:54.453 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-02T04:30:54.503 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-02T04:30:54.523 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-rook-2:18.2.2-1059.g4d05f4ca.el9.noarch 13/13 2024-04-02T04:30:55.031 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-02T04:30:55.080 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-02T04:30:55.105 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-02T04:30:55.145 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-02T04:30:55.180 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-rook-2:18.2.2-1059.g4d05f4ca.el9.noarch 13/13 2024-04-02T04:30:55.339 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1059.g4d05f4ca.el9.noarch 13/13 2024-04-02T04:30:55.339 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1059.g4d05f4ca.el9.noarch 1/13 2024-04-02T04:30:55.339 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-02T04:30:55.339 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-02T04:30:55.340 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-02T04:30:55.340 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-02T04:30:55.340 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-02T04:30:55.340 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-02T04:30:55.340 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-02T04:30:55.340 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-02T04:30:55.341 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-04-02T04:30:55.341 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-02T04:30:55.341 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-02T04:30:55.701 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-02T04:30:55.701 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:30:55.701 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-04-02T04:30:55.701 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook-2:18.2.2-1059.g4d05f4ca.el9.noarch 2024-04-02T04:30:55.701 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-02T04:30:55.701 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-02T04:30:55.701 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-02T04:30:55.701 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-02T04:30:55.701 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-02T04:30:55.702 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-02T04:30:55.702 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-02T04:30:55.702 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-02T04:30:55.702 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-02T04:30:55.702 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-02T04:30:55.702 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-02T04:30:55.702 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-02T04:30:55.702 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:30:55.702 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-04-02T04:30:55.928 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-mgr-cephadm 2024-04-02T04:30:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1059.g4d05f4ca.el9.noarch 13/13 2024-04-02T04:30:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1059.g4d05f4ca.el9.noarch 1/13 2024-04-02T04:30:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-02T04:30:55.979 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-02T04:30:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-02T04:30:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-02T04:30:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-02T04:30:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-02T04:30:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-02T04:30:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-02T04:30:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-04-02T04:30:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-02T04:30:55.980 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-02T04:30:56.329 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-02T04:30:56.329 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:30:56.329 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-04-02T04:30:56.329 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook-2:18.2.2-1059.g4d05f4ca.el9.noarch 2024-04-02T04:30:56.329 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-02T04:30:56.329 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-02T04:30:56.329 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-02T04:30:56.329 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-02T04:30:56.329 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-02T04:30:56.329 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-02T04:30:56.330 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-02T04:30:56.330 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-02T04:30:56.330 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-02T04:30:56.330 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-02T04:30:56.330 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-02T04:30:56.330 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-02T04:30:56.330 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:30:56.330 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-02T04:30:56.525 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:20 ago on Tue 02 Apr 2024 04:29:36 AM UTC. 2024-04-02T04:30:56.554 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-mgr-cephadm 2024-04-02T04:30:56.684 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-04-02T04:30:56.685 INFO:teuthology.orchestra.run.smithi027.stdout:======================================================================================== 2024-04-02T04:30:56.685 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-04-02T04:30:56.685 INFO:teuthology.orchestra.run.smithi027.stdout:======================================================================================== 2024-04-02T04:30:56.685 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-04-02T04:30:56.685 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1059.g4d05f4ca.el9 ceph-noarch 138 k 2024-04-02T04:30:56.686 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-04-02T04:30:56.686 INFO:teuthology.orchestra.run.smithi027.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-02T04:30:56.686 INFO:teuthology.orchestra.run.smithi027.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-02T04:30:56.686 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-02T04:30:56.686 INFO:teuthology.orchestra.run.smithi027.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-02T04:30:56.686 INFO:teuthology.orchestra.run.smithi027.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-02T04:30:56.686 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:30:56.686 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-04-02T04:30:56.686 INFO:teuthology.orchestra.run.smithi027.stdout:======================================================================================== 2024-04-02T04:30:56.686 INFO:teuthology.orchestra.run.smithi027.stdout:Install 6 Packages 2024-04-02T04:30:56.686 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:30:56.687 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 6.8 M 2024-04-02T04:30:56.687 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 33 M 2024-04-02T04:30:56.687 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-04-02T04:30:57.130 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:18 ago on Tue 02 Apr 2024 04:29:39 AM UTC. 2024-04-02T04:30:57.179 INFO:teuthology.orchestra.run.smithi027.stdout:(1/6): python3-babel-2.9.1-2.el9.noarch.rpm 21 MB/s | 5.8 MB 00:00 2024-04-02T04:30:57.263 INFO:teuthology.orchestra.run.smithi027.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 690 kB/s | 248 kB 00:00 2024-04-02T04:30:57.289 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-02T04:30:57.289 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================== 2024-04-02T04:30:57.289 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-04-02T04:30:57.289 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================== 2024-04-02T04:30:57.289 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-04-02T04:30:57.290 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1059.g4d05f4ca.el9 ceph-noarch 138 k 2024-04-02T04:30:57.290 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-04-02T04:30:57.290 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-02T04:30:57.290 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-02T04:30:57.290 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-02T04:30:57.290 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-02T04:30:57.290 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-02T04:30:57.290 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:30:57.290 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-04-02T04:30:57.290 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================== 2024-04-02T04:30:57.290 INFO:teuthology.orchestra.run.smithi016.stdout:Install 6 Packages 2024-04-02T04:30:57.290 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:30:57.291 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 6.8 M 2024-04-02T04:30:57.291 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 33 M 2024-04-02T04:30:57.291 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-04-02T04:30:57.313 INFO:teuthology.orchestra.run.smithi027.stdout:(3/6): ceph-mgr-cephadm-18.2.2-1059.g4d05f4ca.e 338 kB/s | 138 kB 00:00 2024-04-02T04:30:57.422 INFO:teuthology.orchestra.run.smithi027.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 365 kB/s | 58 kB 00:00 2024-04-02T04:30:57.447 INFO:teuthology.orchestra.run.smithi027.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 382 kB/s | 51 kB 00:00 2024-04-02T04:30:57.472 INFO:teuthology.orchestra.run.smithi027.stdout:(6/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.8 MB/s | 548 kB 00:00 2024-04-02T04:30:57.475 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:30:57.475 INFO:teuthology.orchestra.run.smithi027.stdout:Total 8.6 MB/s | 6.8 MB 00:00 2024-04-02T04:30:57.488 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-04-02T04:30:57.500 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-04-02T04:30:57.500 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-04-02T04:30:57.594 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-04-02T04:30:57.595 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-04-02T04:30:57.890 INFO:teuthology.orchestra.run.smithi016.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.3 MB/s | 248 kB 00:00 2024-04-02T04:30:57.915 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-04-02T04:30:57.973 INFO:teuthology.orchestra.run.smithi016.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 5.8 MB 00:00 2024-04-02T04:30:58.074 INFO:teuthology.orchestra.run.smithi016.stdout:(3/6): ceph-mgr-cephadm-18.2.2-1059.g4d05f4ca.e 380 kB/s | 138 kB 00:00 2024-04-02T04:30:58.105 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-02T04:30:58.160 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-02T04:30:58.351 INFO:teuthology.orchestra.run.smithi016.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 154 kB/s | 58 kB 00:00 2024-04-02T04:30:58.416 INFO:teuthology.orchestra.run.smithi016.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 149 kB/s | 51 kB 00:00 2024-04-02T04:30:58.532 INFO:teuthology.orchestra.run.smithi016.stdout:(6/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 856 kB/s | 548 kB 00:00 2024-04-02T04:30:58.534 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:30:58.534 INFO:teuthology.orchestra.run.smithi016.stdout:Total 5.5 MB/s | 6.8 MB 00:01 2024-04-02T04:30:58.550 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-04-02T04:30:58.562 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-04-02T04:30:58.562 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-04-02T04:30:58.658 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-04-02T04:30:58.658 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-04-02T04:30:58.854 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-02T04:30:58.950 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-02T04:30:58.976 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-04-02T04:30:59.143 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-02T04:30:59.156 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-02T04:30:59.170 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1059.g4d05f4ca.el9.noarch 6/6 2024-04-02T04:30:59.211 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-02T04:30:59.765 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-02T04:30:59.848 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-02T04:30:59.917 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-02T04:30:59.947 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1059.g4d05f4ca.el9.noarch 6/6 2024-04-02T04:30:59.960 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1059.g4d05f4ca.el9.noarch 6/6 2024-04-02T04:30:59.960 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1059.g4d05f4ca.el9.noarch 1/6 2024-04-02T04:30:59.960 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-02T04:30:59.960 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-02T04:30:59.960 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-02T04:30:59.960 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-02T04:31:00.256 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-02T04:31:00.256 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:31:00.256 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-04-02T04:31:00.256 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-cephadm-2:18.2.2-1059.g4d05f4ca.el9.noarch 2024-04-02T04:31:00.257 INFO:teuthology.orchestra.run.smithi027.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-02T04:31:00.257 INFO:teuthology.orchestra.run.smithi027.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-02T04:31:00.257 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-02T04:31:00.257 INFO:teuthology.orchestra.run.smithi027.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-02T04:31:00.257 INFO:teuthology.orchestra.run.smithi027.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-02T04:31:00.257 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:31:00.257 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-04-02T04:31:00.604 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-fuse 2024-04-02T04:31:00.733 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1059.g4d05f4ca.el9.noarch 6/6 2024-04-02T04:31:00.734 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1059.g4d05f4ca.el9.noarch 1/6 2024-04-02T04:31:00.734 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-02T04:31:00.734 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-02T04:31:00.734 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-02T04:31:00.734 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-02T04:31:01.026 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-02T04:31:01.027 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:31:01.027 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-04-02T04:31:01.027 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm-2:18.2.2-1059.g4d05f4ca.el9.noarch 2024-04-02T04:31:01.027 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-02T04:31:01.027 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-02T04:31:01.027 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-02T04:31:01.027 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-02T04:31:01.027 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-02T04:31:01.027 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:31:01.027 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-02T04:31:01.192 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:25 ago on Tue 02 Apr 2024 04:29:36 AM UTC. 2024-04-02T04:31:01.255 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-fuse 2024-04-02T04:31:01.349 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-04-02T04:31:01.349 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-02T04:31:01.350 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-04-02T04:31:01.350 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-02T04:31:01.350 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-04-02T04:31:01.350 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 849 k 2024-04-02T04:31:01.350 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-04-02T04:31:01.350 INFO:teuthology.orchestra.run.smithi027.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-02T04:31:01.350 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:31:01.350 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-04-02T04:31:01.350 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-02T04:31:01.350 INFO:teuthology.orchestra.run.smithi027.stdout:Install 2 Packages 2024-04-02T04:31:01.350 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:31:01.350 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 928 k 2024-04-02T04:31:01.350 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 2.7 M 2024-04-02T04:31:01.350 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-04-02T04:31:01.474 INFO:teuthology.orchestra.run.smithi027.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 630 kB/s | 79 kB 00:00 2024-04-02T04:31:01.799 INFO:teuthology.orchestra.run.smithi027.stdout:(2/2): ceph-fuse-18.2.2-1059.g4d05f4ca.el9.x86_ 1.8 MB/s | 849 kB 00:00 2024-04-02T04:31:01.800 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:31:01.801 INFO:teuthology.orchestra.run.smithi027.stdout:Total 2.0 MB/s | 928 kB 00:00 2024-04-02T04:31:01.801 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-04-02T04:31:01.822 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:22 ago on Tue 02 Apr 2024 04:29:39 AM UTC. 2024-04-02T04:31:01.823 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-04-02T04:31:01.823 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-04-02T04:31:01.902 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-04-02T04:31:01.902 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-04-02T04:31:01.978 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-02T04:31:01.979 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-02T04:31:01.979 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-04-02T04:31:01.979 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-02T04:31:01.979 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-04-02T04:31:01.979 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 849 k 2024-04-02T04:31:01.979 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-04-02T04:31:01.979 INFO:teuthology.orchestra.run.smithi016.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-02T04:31:01.979 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:31:01.979 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-04-02T04:31:01.979 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-02T04:31:01.979 INFO:teuthology.orchestra.run.smithi016.stdout:Install 2 Packages 2024-04-02T04:31:01.979 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:31:01.979 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 928 k 2024-04-02T04:31:01.980 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 2.7 M 2024-04-02T04:31:01.980 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-04-02T04:31:02.096 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-04-02T04:31:02.118 INFO:teuthology.orchestra.run.smithi016.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 567 kB/s | 79 kB 00:00 2024-04-02T04:31:02.234 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-02T04:31:02.301 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-fuse-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2/2 2024-04-02T04:31:02.435 INFO:teuthology.orchestra.run.smithi016.stdout:(2/2): ceph-fuse-18.2.2-1059.g4d05f4ca.el9.x86_ 1.8 MB/s | 849 kB 00:00 2024-04-02T04:31:02.436 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:31:02.437 INFO:teuthology.orchestra.run.smithi016.stdout:Total 2.0 MB/s | 928 kB 00:00 2024-04-02T04:31:02.437 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-04-02T04:31:02.458 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-04-02T04:31:02.458 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-04-02T04:31:02.534 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-04-02T04:31:02.534 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-04-02T04:31:02.723 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-04-02T04:31:02.863 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-02T04:31:02.937 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-fuse-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2/2 2024-04-02T04:31:02.960 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-fuse-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2/2 2024-04-02T04:31:02.960 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-fuse-2:18.2.2-1059.g4d05f4ca.el9.x86_64 1/2 2024-04-02T04:31:03.275 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-02T04:31:03.275 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:31:03.275 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-04-02T04:31:03.275 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse-2:18.2.2-1059.g4d05f4ca.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-02T04:31:03.275 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:31:03.275 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-04-02T04:31:03.571 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install librados-devel 2024-04-02T04:31:03.618 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-fuse-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2/2 2024-04-02T04:31:03.618 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-fuse-2:18.2.2-1059.g4d05f4ca.el9.x86_64 1/2 2024-04-02T04:31:03.967 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-02T04:31:03.967 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:31:03.967 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-04-02T04:31:03.968 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse-2:18.2.2-1059.g4d05f4ca.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-02T04:31:03.968 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:31:03.968 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-02T04:31:04.165 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:28 ago on Tue 02 Apr 2024 04:29:36 AM UTC. 2024-04-02T04:31:04.223 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install librados-devel 2024-04-02T04:31:04.321 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-04-02T04:31:04.322 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-02T04:31:04.323 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repo Size 2024-04-02T04:31:04.323 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-02T04:31:04.323 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-04-02T04:31:04.323 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 127 k 2024-04-02T04:31:04.323 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:31:04.323 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-04-02T04:31:04.323 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-02T04:31:04.323 INFO:teuthology.orchestra.run.smithi027.stdout:Install 1 Package 2024-04-02T04:31:04.323 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:31:04.323 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 127 k 2024-04-02T04:31:04.323 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 456 k 2024-04-02T04:31:04.323 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-04-02T04:31:04.520 INFO:teuthology.orchestra.run.smithi027.stdout:librados-devel-18.2.2-1059.g4d05f4ca.el9.x86_64 643 kB/s | 127 kB 00:00 2024-04-02T04:31:04.521 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:31:04.522 INFO:teuthology.orchestra.run.smithi027.stdout:Total 638 kB/s | 127 kB 00:00 2024-04-02T04:31:04.522 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-04-02T04:31:04.527 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-04-02T04:31:04.527 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-04-02T04:31:04.585 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-04-02T04:31:04.586 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-04-02T04:31:04.732 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-04-02T04:31:04.800 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:25 ago on Tue 02 Apr 2024 04:29:39 AM UTC. 2024-04-02T04:31:04.863 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librados-devel-2:18.2.2-1059.g4d05f4ca.el9.x86_64 1/1 2024-04-02T04:31:04.955 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-02T04:31:04.956 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-02T04:31:04.956 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repo Size 2024-04-02T04:31:04.956 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-02T04:31:04.956 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-04-02T04:31:04.956 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 127 k 2024-04-02T04:31:04.956 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:31:04.956 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-04-02T04:31:04.956 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-02T04:31:04.956 INFO:teuthology.orchestra.run.smithi016.stdout:Install 1 Package 2024-04-02T04:31:04.956 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:31:04.956 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 127 k 2024-04-02T04:31:04.956 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 456 k 2024-04-02T04:31:04.957 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-04-02T04:31:05.183 INFO:teuthology.orchestra.run.smithi016.stdout:librados-devel-18.2.2-1059.g4d05f4ca.el9.x86_64 562 kB/s | 127 kB 00:00 2024-04-02T04:31:05.184 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:31:05.184 INFO:teuthology.orchestra.run.smithi016.stdout:Total 557 kB/s | 127 kB 00:00 2024-04-02T04:31:05.184 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-04-02T04:31:05.189 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-04-02T04:31:05.190 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-04-02T04:31:05.250 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-04-02T04:31:05.250 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-04-02T04:31:05.395 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-04-02T04:31:05.401 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados-devel-2:18.2.2-1059.g4d05f4ca.el9.x86_64 1/1 2024-04-02T04:31:05.509 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librados-devel-2:18.2.2-1059.g4d05f4ca.el9.x86_64 1/1 2024-04-02T04:31:05.688 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados-devel-2:18.2.2-1059.g4d05f4ca.el9.x86_64 1/1 2024-04-02T04:31:05.688 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:31:05.688 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-04-02T04:31:05.688 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:31:05.689 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:31:05.689 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-04-02T04:31:05.907 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install libcephfs2 2024-04-02T04:31:05.917 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados-devel-2:18.2.2-1059.g4d05f4ca.el9.x86_64 1/1 2024-04-02T04:31:06.184 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados-devel-2:18.2.2-1059.g4d05f4ca.el9.x86_64 1/1 2024-04-02T04:31:06.184 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:31:06.184 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-04-02T04:31:06.184 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:31:06.185 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:31:06.185 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-02T04:31:06.441 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install libcephfs2 2024-04-02T04:31:06.497 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:30 ago on Tue 02 Apr 2024 04:29:36 AM UTC. 2024-04-02T04:31:06.597 INFO:teuthology.orchestra.run.smithi027.stdout:Package libcephfs2-2:18.2.2-1059.g4d05f4ca.el9.x86_64 is already installed. 2024-04-02T04:31:06.652 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-04-02T04:31:06.653 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2024-04-02T04:31:06.653 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-04-02T04:31:06.711 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install libcephfs-devel 2024-04-02T04:31:07.024 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:28 ago on Tue 02 Apr 2024 04:29:39 AM UTC. 2024-04-02T04:31:07.129 INFO:teuthology.orchestra.run.smithi016.stdout:Package libcephfs2-2:18.2.2-1059.g4d05f4ca.el9.x86_64 is already installed. 2024-04-02T04:31:07.184 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-02T04:31:07.185 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2024-04-02T04:31:07.185 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-02T04:31:07.239 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install libcephfs-devel 2024-04-02T04:31:07.302 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:31 ago on Tue 02 Apr 2024 04:29:36 AM UTC. 2024-04-02T04:31:07.457 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-04-02T04:31:07.458 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-02T04:31:07.458 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repo Size 2024-04-02T04:31:07.458 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-02T04:31:07.458 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-04-02T04:31:07.458 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 31 k 2024-04-02T04:31:07.458 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:31:07.458 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-04-02T04:31:07.458 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-02T04:31:07.459 INFO:teuthology.orchestra.run.smithi027.stdout:Install 1 Package 2024-04-02T04:31:07.459 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:31:07.459 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 31 k 2024-04-02T04:31:07.459 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 139 k 2024-04-02T04:31:07.459 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-04-02T04:31:07.632 INFO:teuthology.orchestra.run.smithi027.stdout:libcephfs-devel-18.2.2-1059.g4d05f4ca.el9.x86_6 179 kB/s | 31 kB 00:00 2024-04-02T04:31:07.633 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:31:07.633 INFO:teuthology.orchestra.run.smithi027.stdout:Total 178 kB/s | 31 kB 00:00 2024-04-02T04:31:07.633 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-04-02T04:31:07.637 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-04-02T04:31:07.637 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-04-02T04:31:07.665 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-04-02T04:31:07.666 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-04-02T04:31:07.761 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-04-02T04:31:07.811 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:28 ago on Tue 02 Apr 2024 04:29:39 AM UTC. 2024-04-02T04:31:07.892 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs-devel-2:18.2.2-1059.g4d05f4ca.el9.x86_64 1/1 2024-04-02T04:31:07.966 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-02T04:31:07.967 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-02T04:31:07.967 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repo Size 2024-04-02T04:31:07.967 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-02T04:31:07.967 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-04-02T04:31:07.967 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 31 k 2024-04-02T04:31:07.967 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:31:07.967 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-04-02T04:31:07.968 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-02T04:31:07.968 INFO:teuthology.orchestra.run.smithi016.stdout:Install 1 Package 2024-04-02T04:31:07.968 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:31:07.968 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 31 k 2024-04-02T04:31:07.968 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 139 k 2024-04-02T04:31:07.968 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-04-02T04:31:08.128 INFO:teuthology.orchestra.run.smithi016.stdout:libcephfs-devel-18.2.2-1059.g4d05f4ca.el9.x86_6 194 kB/s | 31 kB 00:00 2024-04-02T04:31:08.129 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:31:08.129 INFO:teuthology.orchestra.run.smithi016.stdout:Total 192 kB/s | 31 kB 00:00 2024-04-02T04:31:08.129 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-04-02T04:31:08.132 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-04-02T04:31:08.133 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-04-02T04:31:08.160 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-04-02T04:31:08.161 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-04-02T04:31:08.240 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephfs-devel-2:18.2.2-1059.g4d05f4ca.el9.x86_64 1/1 2024-04-02T04:31:08.250 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-04-02T04:31:08.362 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs-devel-2:18.2.2-1059.g4d05f4ca.el9.x86_64 1/1 2024-04-02T04:31:08.506 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs-devel-2:18.2.2-1059.g4d05f4ca.el9.x86_64 1/1 2024-04-02T04:31:08.506 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:31:08.506 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-04-02T04:31:08.506 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:31:08.506 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:31:08.506 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-04-02T04:31:08.694 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install librados2 2024-04-02T04:31:08.736 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephfs-devel-2:18.2.2-1059.g4d05f4ca.el9.x86_64 1/1 2024-04-02T04:31:09.001 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs-devel-2:18.2.2-1059.g4d05f4ca.el9.x86_64 1/1 2024-04-02T04:31:09.001 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:31:09.001 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-04-02T04:31:09.001 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:31:09.001 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:31:09.001 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-02T04:31:09.197 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install librados2 2024-04-02T04:31:09.284 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:33 ago on Tue 02 Apr 2024 04:29:36 AM UTC. 2024-04-02T04:31:09.384 INFO:teuthology.orchestra.run.smithi027.stdout:Package librados2-2:18.2.2-1059.g4d05f4ca.el9.x86_64 is already installed. 2024-04-02T04:31:09.440 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-04-02T04:31:09.441 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2024-04-02T04:31:09.441 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-04-02T04:31:09.501 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install librbd1 2024-04-02T04:31:09.769 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:30 ago on Tue 02 Apr 2024 04:29:39 AM UTC. 2024-04-02T04:31:09.869 INFO:teuthology.orchestra.run.smithi016.stdout:Package librados2-2:18.2.2-1059.g4d05f4ca.el9.x86_64 is already installed. 2024-04-02T04:31:09.924 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-02T04:31:09.925 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2024-04-02T04:31:09.925 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-02T04:31:09.982 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install librbd1 2024-04-02T04:31:10.088 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:34 ago on Tue 02 Apr 2024 04:29:36 AM UTC. 2024-04-02T04:31:10.188 INFO:teuthology.orchestra.run.smithi027.stdout:Package librbd1-2:18.2.2-1059.g4d05f4ca.el9.x86_64 is already installed. 2024-04-02T04:31:10.244 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-04-02T04:31:10.245 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2024-04-02T04:31:10.245 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-04-02T04:31:10.309 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install python3-rados 2024-04-02T04:31:10.551 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:31 ago on Tue 02 Apr 2024 04:29:39 AM UTC. 2024-04-02T04:31:10.651 INFO:teuthology.orchestra.run.smithi016.stdout:Package librbd1-2:18.2.2-1059.g4d05f4ca.el9.x86_64 is already installed. 2024-04-02T04:31:10.706 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-02T04:31:10.707 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2024-04-02T04:31:10.708 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-02T04:31:10.765 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install python3-rados 2024-04-02T04:31:10.894 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:34 ago on Tue 02 Apr 2024 04:29:36 AM UTC. 2024-04-02T04:31:10.993 INFO:teuthology.orchestra.run.smithi027.stdout:Package python3-rados-2:18.2.2-1059.g4d05f4ca.el9.x86_64 is already installed. 2024-04-02T04:31:11.049 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-04-02T04:31:11.050 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2024-04-02T04:31:11.050 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-04-02T04:31:11.117 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install python3-rgw 2024-04-02T04:31:11.335 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:32 ago on Tue 02 Apr 2024 04:29:39 AM UTC. 2024-04-02T04:31:11.436 INFO:teuthology.orchestra.run.smithi016.stdout:Package python3-rados-2:18.2.2-1059.g4d05f4ca.el9.x86_64 is already installed. 2024-04-02T04:31:11.491 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-02T04:31:11.492 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2024-04-02T04:31:11.492 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-02T04:31:11.549 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install python3-rgw 2024-04-02T04:31:11.706 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:35 ago on Tue 02 Apr 2024 04:29:36 AM UTC. 2024-04-02T04:31:11.806 INFO:teuthology.orchestra.run.smithi027.stdout:Package python3-rgw-2:18.2.2-1059.g4d05f4ca.el9.x86_64 is already installed. 2024-04-02T04:31:11.862 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-04-02T04:31:11.862 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2024-04-02T04:31:11.862 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-04-02T04:31:11.937 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install python3-cephfs 2024-04-02T04:31:12.119 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:33 ago on Tue 02 Apr 2024 04:29:39 AM UTC. 2024-04-02T04:31:12.220 INFO:teuthology.orchestra.run.smithi016.stdout:Package python3-rgw-2:18.2.2-1059.g4d05f4ca.el9.x86_64 is already installed. 2024-04-02T04:31:12.276 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-02T04:31:12.277 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2024-04-02T04:31:12.277 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-02T04:31:12.341 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install python3-cephfs 2024-04-02T04:31:12.529 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:36 ago on Tue 02 Apr 2024 04:29:36 AM UTC. 2024-04-02T04:31:12.630 INFO:teuthology.orchestra.run.smithi027.stdout:Package python3-cephfs-2:18.2.2-1059.g4d05f4ca.el9.x86_64 is already installed. 2024-04-02T04:31:12.685 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-04-02T04:31:12.686 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2024-04-02T04:31:12.686 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-04-02T04:31:12.753 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install python3-rbd 2024-04-02T04:31:12.914 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:33 ago on Tue 02 Apr 2024 04:29:39 AM UTC. 2024-04-02T04:31:13.015 INFO:teuthology.orchestra.run.smithi016.stdout:Package python3-cephfs-2:18.2.2-1059.g4d05f4ca.el9.x86_64 is already installed. 2024-04-02T04:31:13.070 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-02T04:31:13.071 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2024-04-02T04:31:13.071 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-02T04:31:13.128 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install python3-rbd 2024-04-02T04:31:13.342 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:37 ago on Tue 02 Apr 2024 04:29:36 AM UTC. 2024-04-02T04:31:13.443 INFO:teuthology.orchestra.run.smithi027.stdout:Package python3-rbd-2:18.2.2-1059.g4d05f4ca.el9.x86_64 is already installed. 2024-04-02T04:31:13.498 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-04-02T04:31:13.499 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2024-04-02T04:31:13.499 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-04-02T04:31:13.557 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install rbd-fuse 2024-04-02T04:31:13.700 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:34 ago on Tue 02 Apr 2024 04:29:39 AM UTC. 2024-04-02T04:31:13.800 INFO:teuthology.orchestra.run.smithi016.stdout:Package python3-rbd-2:18.2.2-1059.g4d05f4ca.el9.x86_64 is already installed. 2024-04-02T04:31:13.855 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-02T04:31:13.857 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2024-04-02T04:31:13.857 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-02T04:31:13.911 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install rbd-fuse 2024-04-02T04:31:14.146 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:38 ago on Tue 02 Apr 2024 04:29:36 AM UTC. 2024-04-02T04:31:14.302 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-04-02T04:31:14.303 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-02T04:31:14.303 INFO:teuthology.orchestra.run.smithi027.stdout: Package Architecture Version Repository Size 2024-04-02T04:31:14.303 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-02T04:31:14.303 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-04-02T04:31:14.303 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 87 k 2024-04-02T04:31:14.303 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:31:14.303 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-04-02T04:31:14.304 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-02T04:31:14.304 INFO:teuthology.orchestra.run.smithi027.stdout:Install 1 Package 2024-04-02T04:31:14.304 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:31:14.304 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 87 k 2024-04-02T04:31:14.304 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 227 k 2024-04-02T04:31:14.304 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-04-02T04:31:14.482 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:35 ago on Tue 02 Apr 2024 04:29:39 AM UTC. 2024-04-02T04:31:14.519 INFO:teuthology.orchestra.run.smithi027.stdout:rbd-fuse-18.2.2-1059.g4d05f4ca.el9.x86_64.rpm 402 kB/s | 87 kB 00:00 2024-04-02T04:31:14.520 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:31:14.520 INFO:teuthology.orchestra.run.smithi027.stdout:Total 400 kB/s | 87 kB 00:00 2024-04-02T04:31:14.520 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-04-02T04:31:14.530 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-04-02T04:31:14.530 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-04-02T04:31:14.585 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-04-02T04:31:14.586 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-04-02T04:31:14.639 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-02T04:31:14.639 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-02T04:31:14.639 INFO:teuthology.orchestra.run.smithi016.stdout: Package Architecture Version Repository Size 2024-04-02T04:31:14.639 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-02T04:31:14.639 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-04-02T04:31:14.639 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 87 k 2024-04-02T04:31:14.639 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:31:14.640 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-04-02T04:31:14.640 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-02T04:31:14.640 INFO:teuthology.orchestra.run.smithi016.stdout:Install 1 Package 2024-04-02T04:31:14.640 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:31:14.640 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 87 k 2024-04-02T04:31:14.640 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 227 k 2024-04-02T04:31:14.640 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-04-02T04:31:14.734 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-04-02T04:31:14.827 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-fuse-2:18.2.2-1059.g4d05f4ca.el9.x86_64 1/1 2024-04-02T04:31:14.973 INFO:teuthology.orchestra.run.smithi016.stdout:rbd-fuse-18.2.2-1059.g4d05f4ca.el9.x86_64.rpm 260 kB/s | 87 kB 00:00 2024-04-02T04:31:14.974 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:31:14.974 INFO:teuthology.orchestra.run.smithi016.stdout:Total 259 kB/s | 87 kB 00:00 2024-04-02T04:31:14.974 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-04-02T04:31:14.982 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-04-02T04:31:14.982 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-04-02T04:31:15.040 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-04-02T04:31:15.040 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-04-02T04:31:15.180 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-04-02T04:31:15.211 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: rbd-fuse-2:18.2.2-1059.g4d05f4ca.el9.x86_64 1/1 2024-04-02T04:31:15.291 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-fuse-2:18.2.2-1059.g4d05f4ca.el9.x86_64 1/1 2024-04-02T04:31:15.547 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-fuse-2:18.2.2-1059.g4d05f4ca.el9.x86_64 1/1 2024-04-02T04:31:15.547 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:31:15.547 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-04-02T04:31:15.548 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:31:15.548 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:31:15.548 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-04-02T04:31:15.649 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-fuse-2:18.2.2-1059.g4d05f4ca.el9.x86_64 1/1 2024-04-02T04:31:15.746 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install rbd-mirror 2024-04-02T04:31:15.934 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-fuse-2:18.2.2-1059.g4d05f4ca.el9.x86_64 1/1 2024-04-02T04:31:15.934 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:31:15.934 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-04-02T04:31:15.934 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:31:15.934 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:31:15.934 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-02T04:31:16.150 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install rbd-mirror 2024-04-02T04:31:16.333 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:40 ago on Tue 02 Apr 2024 04:29:36 AM UTC. 2024-04-02T04:31:16.489 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-04-02T04:31:16.490 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-02T04:31:16.490 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repo Size 2024-04-02T04:31:16.490 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-02T04:31:16.490 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-04-02T04:31:16.490 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 3.0 M 2024-04-02T04:31:16.490 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:31:16.490 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-04-02T04:31:16.490 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-02T04:31:16.491 INFO:teuthology.orchestra.run.smithi027.stdout:Install 1 Package 2024-04-02T04:31:16.491 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:31:16.491 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 3.0 M 2024-04-02T04:31:16.491 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 12 M 2024-04-02T04:31:16.491 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-04-02T04:31:16.725 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:37 ago on Tue 02 Apr 2024 04:29:39 AM UTC. 2024-04-02T04:31:16.884 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-02T04:31:16.885 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-02T04:31:16.885 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repo Size 2024-04-02T04:31:16.885 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-02T04:31:16.885 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-04-02T04:31:16.885 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 3.0 M 2024-04-02T04:31:16.885 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:31:16.885 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-04-02T04:31:16.885 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-02T04:31:16.885 INFO:teuthology.orchestra.run.smithi016.stdout:Install 1 Package 2024-04-02T04:31:16.885 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:31:16.885 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 3.0 M 2024-04-02T04:31:16.885 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 12 M 2024-04-02T04:31:16.885 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-04-02T04:31:17.016 INFO:teuthology.orchestra.run.smithi027.stdout:rbd-mirror-18.2.2-1059.g4d05f4ca.el9.x86_64.rpm 5.7 MB/s | 3.0 MB 00:00 2024-04-02T04:31:17.016 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:31:17.017 INFO:teuthology.orchestra.run.smithi027.stdout:Total 5.7 MB/s | 3.0 MB 00:00 2024-04-02T04:31:17.017 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-04-02T04:31:17.027 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-04-02T04:31:17.027 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-04-02T04:31:17.103 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-04-02T04:31:17.103 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-04-02T04:31:17.269 INFO:teuthology.orchestra.run.smithi016.stdout:rbd-mirror-18.2.2-1059.g4d05f4ca.el9.x86_64.rpm 7.8 MB/s | 3.0 MB 00:00 2024-04-02T04:31:17.269 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:31:17.270 INFO:teuthology.orchestra.run.smithi016.stdout:Total 7.8 MB/s | 3.0 MB 00:00 2024-04-02T04:31:17.270 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-04-02T04:31:17.280 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-04-02T04:31:17.280 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-04-02T04:31:17.354 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-04-02T04:31:17.355 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-04-02T04:31:17.537 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-04-02T04:31:17.594 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-mirror-2:18.2.2-1059.g4d05f4ca.el9.x86_64 1/1 2024-04-02T04:31:17.618 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1059.g4d05f4ca.el9.x86_64 1/1 2024-04-02T04:31:17.618 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T04:31:17.618 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-02T04:31:17.618 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T04:31:17.618 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T04:31:17.618 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:31:17.787 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-04-02T04:31:17.853 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-mirror-2:18.2.2-1059.g4d05f4ca.el9.x86_64 1/1 2024-04-02T04:31:17.877 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1059.g4d05f4ca.el9.x86_64 1/1 2024-04-02T04:31:17.877 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T04:31:17.877 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-02T04:31:17.877 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T04:31:17.877 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T04:31:17.877 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:31:18.780 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-mirror-2:18.2.2-1059.g4d05f4ca.el9.x86_64 1/1 2024-04-02T04:31:18.780 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:31:18.780 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-04-02T04:31:18.780 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:31:18.780 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:31:18.780 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-04-02T04:31:19.026 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-mirror-2:18.2.2-1059.g4d05f4ca.el9.x86_64 1/1 2024-04-02T04:31:19.026 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:31:19.026 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-04-02T04:31:19.026 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:31:19.026 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:31:19.026 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-02T04:31:19.055 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install rbd-nbd 2024-04-02T04:31:19.284 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install rbd-nbd 2024-04-02T04:31:19.646 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:43 ago on Tue 02 Apr 2024 04:29:36 AM UTC. 2024-04-02T04:31:19.801 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-04-02T04:31:19.802 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-02T04:31:19.802 INFO:teuthology.orchestra.run.smithi027.stdout: Package Architecture Version Repository Size 2024-04-02T04:31:19.802 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-02T04:31:19.802 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-04-02T04:31:19.803 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 172 k 2024-04-02T04:31:19.803 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:31:19.803 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-04-02T04:31:19.803 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-02T04:31:19.803 INFO:teuthology.orchestra.run.smithi027.stdout:Install 1 Package 2024-04-02T04:31:19.803 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:31:19.803 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 172 k 2024-04-02T04:31:19.803 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 494 k 2024-04-02T04:31:19.803 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-04-02T04:31:19.861 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:40 ago on Tue 02 Apr 2024 04:29:39 AM UTC. 2024-04-02T04:31:20.016 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-02T04:31:20.017 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-02T04:31:20.017 INFO:teuthology.orchestra.run.smithi016.stdout: Package Architecture Version Repository Size 2024-04-02T04:31:20.017 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-02T04:31:20.017 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-04-02T04:31:20.017 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd x86_64 2:18.2.2-1059.g4d05f4ca.el9 ceph 172 k 2024-04-02T04:31:20.017 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:31:20.017 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-04-02T04:31:20.017 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-04-02T04:31:20.017 INFO:teuthology.orchestra.run.smithi016.stdout:Install 1 Package 2024-04-02T04:31:20.018 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:31:20.018 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 172 k 2024-04-02T04:31:20.018 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 494 k 2024-04-02T04:31:20.018 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-04-02T04:31:20.028 INFO:teuthology.orchestra.run.smithi027.stdout:rbd-nbd-18.2.2-1059.g4d05f4ca.el9.x86_64.rpm 768 kB/s | 172 kB 00:00 2024-04-02T04:31:20.028 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:31:20.029 INFO:teuthology.orchestra.run.smithi027.stdout:Total 763 kB/s | 172 kB 00:00 2024-04-02T04:31:20.029 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-04-02T04:31:20.037 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-04-02T04:31:20.037 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-04-02T04:31:20.098 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-04-02T04:31:20.098 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-04-02T04:31:20.253 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-04-02T04:31:20.353 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-nbd-2:18.2.2-1059.g4d05f4ca.el9.x86_64 1/1 2024-04-02T04:31:20.367 INFO:teuthology.orchestra.run.smithi016.stdout:rbd-nbd-18.2.2-1059.g4d05f4ca.el9.x86_64.rpm 492 kB/s | 172 kB 00:00 2024-04-02T04:31:20.367 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:31:20.368 INFO:teuthology.orchestra.run.smithi016.stdout:Total 491 kB/s | 172 kB 00:00 2024-04-02T04:31:20.368 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-04-02T04:31:20.376 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-04-02T04:31:20.376 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-04-02T04:31:20.434 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-04-02T04:31:20.434 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-04-02T04:31:20.588 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-04-02T04:31:20.692 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-nbd-2:18.2.2-1059.g4d05f4ca.el9.x86_64 1/1 2024-04-02T04:31:20.768 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: rbd-nbd-2:18.2.2-1059.g4d05f4ca.el9.x86_64 1/1 2024-04-02T04:31:21.102 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-nbd-2:18.2.2-1059.g4d05f4ca.el9.x86_64 1/1 2024-04-02T04:31:21.162 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-nbd-2:18.2.2-1059.g4d05f4ca.el9.x86_64 1/1 2024-04-02T04:31:21.162 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:31:21.162 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-04-02T04:31:21.162 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:31:21.162 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:31:21.162 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-04-02T04:31:21.370 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-nbd-2:18.2.2-1059.g4d05f4ca.el9.x86_64 1/1 2024-04-02T04:31:21.371 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:31:21.371 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-04-02T04:31:21.371 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd-2:18.2.2-1059.g4d05f4ca.el9.x86_64 2024-04-02T04:31:21.371 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:31:21.371 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-02T04:31:21.382 DEBUG:teuthology.parallel:result is None 2024-04-02T04:31:21.615 DEBUG:teuthology.parallel:result is None 2024-04-02T04:31:21.615 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-02T04:31:21.616 INFO:teuthology.packaging:ref: None 2024-04-02T04:31:21.616 INFO:teuthology.packaging:tag: None 2024-04-02T04:31:21.616 INFO:teuthology.packaging:branch: reef 2024-04-02T04:31:21.616 INFO:teuthology.packaging:sha1: 6cec793d9073f0594e2364b9b687ecaf22424cca 2024-04-02T04:31:21.616 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-04-02T04:31:21.896 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-02T04:31:21.917 INFO:teuthology.orchestra.run.smithi016.stdout:18.2.2-1059.g4d05f4ca.el9 2024-04-02T04:31:21.918 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1059.g4d05f4ca.el9 2024-04-02T04:31:21.918 INFO:teuthology.task.install:The correct ceph version 18.2.2-1059.g4d05f4ca is installed. 2024-04-02T04:31:21.919 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-02T04:31:21.920 INFO:teuthology.packaging:ref: None 2024-04-02T04:31:21.920 INFO:teuthology.packaging:tag: None 2024-04-02T04:31:21.920 INFO:teuthology.packaging:branch: reef 2024-04-02T04:31:21.920 INFO:teuthology.packaging:sha1: 6cec793d9073f0594e2364b9b687ecaf22424cca 2024-04-02T04:31:21.920 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-04-02T04:31:22.305 DEBUG:teuthology.orchestra.run.smithi027:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-02T04:31:22.330 INFO:teuthology.orchestra.run.smithi027.stdout:18.2.2-1059.g4d05f4ca.el9 2024-04-02T04:31:22.331 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1059.g4d05f4ca.el9 2024-04-02T04:31:22.331 INFO:teuthology.task.install:The correct ceph version 18.2.2-1059.g4d05f4ca is installed. 2024-04-02T04:31:22.333 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-02T04:31:22.333 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-02T04:31:22.333 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-02T04:31:22.361 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-02T04:31:22.362 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-02T04:31:22.407 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-02T04:31:22.408 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-02T04:31:22.408 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/daemon-helper 2024-04-02T04:31:22.436 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-02T04:31:22.503 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-02T04:31:22.503 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/daemon-helper 2024-04-02T04:31:22.538 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-02T04:31:22.611 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-02T04:31:22.611 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-02T04:31:22.611 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-02T04:31:22.638 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-02T04:31:22.705 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-02T04:31:22.705 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-02T04:31:22.737 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-02T04:31:22.809 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-02T04:31:22.809 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-02T04:31:22.809 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/stdin-killer 2024-04-02T04:31:22.836 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-02T04:31:22.903 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-02T04:31:22.904 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/stdin-killer 2024-04-02T04:31:22.935 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-02T04:31:23.007 INFO:teuthology.run_tasks:Running task print... 2024-04-02T04:31:23.021 INFO:teuthology.task.print:**** done install task... 2024-04-02T04:31:23.021 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-02T04:31:23.169 INFO:tasks.cephadm:Config: {'compiled_cephadm_branch': 'reef', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}}, 'image': 'quay.ceph.io/ceph-ci/ceph:reef', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '6cec793d9073f0594e2364b9b687ecaf22424cca'} 2024-04-02T04:31:23.169 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:reef 2024-04-02T04:31:23.169 INFO:tasks.cephadm:Cluster fsid is dc5403b4-f0a9-11ee-b647-cb9ed24678a4 2024-04-02T04:31:23.169 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-02T04:31:23.170 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-04-02T04:31:23.170 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi016': '172.21.15.16', 'mon.smithi027': '172.21.15.27'} 2024-04-02T04:31:23.170 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-02T04:31:23.170 DEBUG:teuthology.orchestra.run.smithi016:> sudo hostname $(hostname -s) 2024-04-02T04:31:23.197 DEBUG:teuthology.orchestra.run.smithi027:> sudo hostname $(hostname -s) 2024-04-02T04:31:23.230 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra for reef 2024-04-02T04:31:23.230 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6cec793d9073f0594e2364b9b687ecaf22424cca 2024-04-02T04:31:23.446 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '6cec793d9073f0594e2364b9b687ecaf22424cca', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/54776/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1979-g6cec793d', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1979.g6cec793d'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/6cec793d9073f0594e2364b9b687ecaf22424cca/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-01 16:46:51.658725', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/6cec793d9073f0594e2364b9b687ecaf22424cca/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-04-02T04:31:23.839 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref reef, sha1 4d05f4cad464de0fcdd11a2856643e533aea34e8 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&ref=reef 2024-04-02T04:31:23.841 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/reef/4d05f4cad464de0fcdd11a2856643e533aea34e8/centos/9/x86_64/flavors/default/cephadm 2024-04-02T04:31:23.841 DEBUG:teuthology.orchestra.run.smithi016:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/reef/4d05f4cad464de0fcdd11a2856643e533aea34e8/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-02T04:31:24.401 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r--. 1 ubuntu ubuntu 215333 Apr 2 04:31 /home/ubuntu/cephtest/cephadm 2024-04-02T04:31:24.402 DEBUG:teuthology.orchestra.run.smithi027:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/reef/4d05f4cad464de0fcdd11a2856643e533aea34e8/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-02T04:31:24.988 INFO:teuthology.orchestra.run.smithi027.stdout:-rw-r--r--. 1 ubuntu ubuntu 215333 Apr 2 04:31 /home/ubuntu/cephtest/cephadm 2024-04-02T04:31:24.989 DEBUG:teuthology.orchestra.run.smithi016:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-02T04:31:25.007 DEBUG:teuthology.orchestra.run.smithi027:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-02T04:31:25.038 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:reef on all hosts... 2024-04-02T04:31:25.039 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef pull 2024-04-02T04:31:25.050 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef pull 2024-04-02T04:31:25.929 INFO:teuthology.orchestra.run.smithi027.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:reef... 2024-04-02T04:31:25.953 INFO:teuthology.orchestra.run.smithi016.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:reef... 2024-04-02T04:32:14.162 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:32:14.163 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)", 2024-04-02T04:32:14.163 INFO:teuthology.orchestra.run.smithi016.stdout: "image_id": "909dce4fe53d7167a6fa8180542b49bd6d1a8c351b846c37da562093c6f0cffd", 2024-04-02T04:32:14.163 INFO:teuthology.orchestra.run.smithi016.stdout: "repo_digests": [ 2024-04-02T04:32:14.163 INFO:teuthology.orchestra.run.smithi016.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:397d0c1d908c002c558906e3ea05982bc336d403ee4d900400cc3bdd9243aa2e" 2024-04-02T04:32:14.163 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-04-02T04:32:14.163 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:32:14.167 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-02T04:32:14.167 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)", 2024-04-02T04:32:14.167 INFO:teuthology.orchestra.run.smithi027.stdout: "image_id": "909dce4fe53d7167a6fa8180542b49bd6d1a8c351b846c37da562093c6f0cffd", 2024-04-02T04:32:14.167 INFO:teuthology.orchestra.run.smithi027.stdout: "repo_digests": [ 2024-04-02T04:32:14.167 INFO:teuthology.orchestra.run.smithi027.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:397d0c1d908c002c558906e3ea05982bc336d403ee4d900400cc3bdd9243aa2e" 2024-04-02T04:32:14.167 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-04-02T04:32:14.167 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-02T04:32:14.187 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph 2024-04-02T04:32:14.222 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /etc/ceph 2024-04-02T04:32:14.256 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 777 /etc/ceph 2024-04-02T04:32:14.288 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 777 /etc/ceph 2024-04-02T04:32:14.327 INFO:tasks.cephadm:Writing seed config... 2024-04-02T04:32:14.329 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-04-02T04:32:14.329 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-04-02T04:32:14.329 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-02T04:32:14.329 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-02T04:32:14.329 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-02T04:32:14.329 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-02T04:32:14.329 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-02T04:32:14.329 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-02T04:32:14.329 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-02T04:32:14.329 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-02T04:32:14.329 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-02T04:32:14.329 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-02T04:32:14.330 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-02T04:32:14.330 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-02T04:32:14.330 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-02T04:32:14.330 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-02T04:32:14.330 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-02T04:32:14.330 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-02T04:32:14.330 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-02T04:32:14.331 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-02T04:32:14.331 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-02T04:32:14.331 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-02T04:32:14.331 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-02T04:32:14.331 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-04-02T04:32:14.331 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-02T04:32:14.331 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-02T04:32:14.331 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-02T04:32:14.331 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-02T04:32:14.331 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-02T04:32:14.331 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-02T04:32:14.331 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-02T04:32:14.331 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-02T04:32:14.331 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-02T04:32:14.331 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-02T04:32:14.332 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-02T04:32:14.332 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-02T04:32:14.332 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-02T04:32:14.332 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-02T04:32:14.332 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-02T04:32:14.332 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-02T04:32:14.332 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-02T04:32:14.332 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-02T04:32:14.333 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-02T04:32:14.333 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-02T04:32:14.348 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = dc5403b4-f0a9-11ee-b647-cb9ed24678a4 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-02T04:32:14.348 DEBUG:teuthology.orchestra.run.smithi016:mon.smithi016> sudo journalctl -f -n 0 -u ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@mon.smithi016.service 2024-04-02T04:32:14.390 INFO:tasks.cephadm:Bootstrapping... 2024-04-02T04:32:14.390 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef -v bootstrap --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.16 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-02T04:32:14.550 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-04-02T04:32:14.550 INFO:teuthology.orchestra.run.smithi016.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:reef', '-v', 'bootstrap', '--fsid', 'dc5403b4-f0a9-11ee-b647-cb9ed24678a4', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.16', '--skip-admin-label'] 2024-04-02T04:32:14.575 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-02T04:32:14.576 INFO:teuthology.orchestra.run.smithi016.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-02T04:32:14.576 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying podman|docker is present... 2024-04-02T04:32:14.598 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-02T04:32:14.599 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying lvm2 is present... 2024-04-02T04:32:14.599 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying time synchronization is in place... 2024-04-02T04:32:14.606 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-02T04:32:14.606 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-02T04:32:14.612 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-02T04:32:14.612 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-04-02T04:32:14.618 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout enabled 2024-04-02T04:32:14.624 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout active 2024-04-02T04:32:14.624 INFO:teuthology.orchestra.run.smithi016.stdout:Unit chronyd.service is enabled and running 2024-04-02T04:32:14.624 INFO:teuthology.orchestra.run.smithi016.stdout:Repeating the final host check... 2024-04-02T04:32:14.651 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-02T04:32:14.651 INFO:teuthology.orchestra.run.smithi016.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-02T04:32:14.651 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl is present 2024-04-02T04:32:14.651 INFO:teuthology.orchestra.run.smithi016.stdout:lvcreate is present 2024-04-02T04:32:14.657 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-02T04:32:14.657 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-02T04:32:14.663 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-02T04:32:14.663 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-04-02T04:32:14.670 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout enabled 2024-04-02T04:32:14.676 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout active 2024-04-02T04:32:14.677 INFO:teuthology.orchestra.run.smithi016.stdout:Unit chronyd.service is enabled and running 2024-04-02T04:32:14.677 INFO:teuthology.orchestra.run.smithi016.stdout:Host looks OK 2024-04-02T04:32:14.677 INFO:teuthology.orchestra.run.smithi016.stdout:Cluster fsid: dc5403b4-f0a9-11ee-b647-cb9ed24678a4 2024-04-02T04:32:14.677 INFO:teuthology.orchestra.run.smithi016.stdout:Acquiring lock 140474361612992 on /run/cephadm/dc5403b4-f0a9-11ee-b647-cb9ed24678a4.lock 2024-04-02T04:32:14.677 INFO:teuthology.orchestra.run.smithi016.stdout:Lock 140474361612992 acquired on /run/cephadm/dc5403b4-f0a9-11ee-b647-cb9ed24678a4.lock 2024-04-02T04:32:14.678 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying IP 172.21.15.16 port 3300 ... 2024-04-02T04:32:14.678 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying IP 172.21.15.16 port 6789 ... 2024-04-02T04:32:14.678 INFO:teuthology.orchestra.run.smithi016.stdout:Base mon IP(s) is [172.21.15.16:3300, 172.21.15.16:6789], mon addrv is [v2:172.21.15.16:3300,v1:172.21.15.16:6789] 2024-04-02T04:32:14.681 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.16 metric 100 2024-04-02T04:32:14.681 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.16 metric 100 2024-04-02T04:32:14.684 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-02T04:32:14.684 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-02T04:32:14.684 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-02T04:32:14.687 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-02T04:32:14.687 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-02T04:32:14.687 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-02T04:32:14.688 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-02T04:32:14.688 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2024-04-02T04:32:14.688 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-02T04:32:14.688 INFO:teuthology.orchestra.run.smithi016.stdout:Mon IP `172.21.15.16` is in CIDR network `172.21.0.0/20` 2024-04-02T04:32:14.688 INFO:teuthology.orchestra.run.smithi016.stdout:Mon IP `172.21.15.16` is in CIDR network `172.21.0.0/20` 2024-04-02T04:32:14.688 INFO:teuthology.orchestra.run.smithi016.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-02T04:32:14.689 INFO:teuthology.orchestra.run.smithi016.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-02T04:32:14.689 INFO:teuthology.orchestra.run.smithi016.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:reef... 2024-04-02T04:32:15.172 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stdout 909dce4fe53d7167a6fa8180542b49bd6d1a8c351b846c37da562093c6f0cffd 2024-04-02T04:32:15.173 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:reef... 2024-04-02T04:32:15.173 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Getting image source signatures 2024-04-02T04:32:15.173 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Copying blob sha256:cb7323d9467b19473b36f834f938f39a4497262810bbf67238f5fb8520a9c756 2024-04-02T04:32:15.173 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Copying blob sha256:11da15500eca1f826c328eb3265caa30b0a3730510e2a32023084d6fe9cee20c 2024-04-02T04:32:15.173 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Copying config sha256:909dce4fe53d7167a6fa8180542b49bd6d1a8c351b846c37da562093c6f0cffd 2024-04-02T04:32:15.173 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-02T04:32:16.077 INFO:teuthology.orchestra.run.smithi016.stdout:ceph: stdout ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable) 2024-04-02T04:32:16.077 INFO:teuthology.orchestra.run.smithi016.stdout:Ceph version: ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable) 2024-04-02T04:32:16.077 INFO:teuthology.orchestra.run.smithi016.stdout:Extracting ceph user uid/gid from container image... 2024-04-02T04:32:16.900 INFO:teuthology.orchestra.run.smithi016.stdout:stat: stdout 167 167 2024-04-02T04:32:16.900 INFO:teuthology.orchestra.run.smithi016.stdout:Creating initial keys... 2024-04-02T04:32:17.782 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph-authtool: stdout AQBRigtmUm9UDBAAEgiiueTlRxn+V+eIeVnIjw== 2024-04-02T04:32:18.747 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph-authtool: stdout AQBSigtmvHBBBRAA5NDZNXpMwpjfKE5CkIWbLg== 2024-04-02T04:32:19.694 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph-authtool: stdout AQBTigtmb1ohBRAAfzv42KN7iNF0JOPXK1t1lQ== 2024-04-02T04:32:19.695 INFO:teuthology.orchestra.run.smithi016.stdout:Creating initial monmap... 2024-04-02T04:32:20.535 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-02T04:32:20.535 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-04-02T04:32:20.535 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to dc5403b4-f0a9-11ee-b647-cb9ed24678a4 2024-04-02T04:32:20.535 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-02T04:32:20.535 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool for smithi016 [v2:172.21.15.16:3300,v1:172.21.15.16:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-02T04:32:20.535 INFO:teuthology.orchestra.run.smithi016.stdout:setting min_mon_release = pacific 2024-04-02T04:32:20.535 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: set fsid to dc5403b4-f0a9-11ee-b647-cb9ed24678a4 2024-04-02T04:32:20.535 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-02T04:32:20.535 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:32:20.535 INFO:teuthology.orchestra.run.smithi016.stdout:Creating mon... 2024-04-02T04:32:21.490 INFO:teuthology.orchestra.run.smithi016.stdout:create mon.smithi016 on 2024-04-02T04:32:21.703 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-02T04:32:21.844 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-02T04:32:22.036 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4.target → /etc/systemd/system/ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4.target. 2024-04-02T04:32:22.036 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4.target → /etc/systemd/system/ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4.target. 2024-04-02T04:32:22.260 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@mon.smithi016 2024-04-02T04:32:22.261 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to reset failed state of unit ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@mon.smithi016.service: Unit ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@mon.smithi016.service not loaded. 2024-04-02T04:32:22.425 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4.target.wants/ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@mon.smithi016.service → /etc/systemd/system/ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@.service. 2024-04-02T04:32:23.083 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-02T04:32:23.083 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-04-02T04:32:23.090 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-02T04:32:23.090 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-04-02T04:32:23.090 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-04-02T04:32:23.090 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to enable service . firewalld.service is not available 2024-04-02T04:32:23.090 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mon to start... 2024-04-02T04:32:23.091 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mon... 2024-04-02T04:32:24.091 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:24 smithi016 ceph-mon[28593]: from='client.? 172.21.15.16:0/773621853' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-02T04:32:24.100 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout cluster: 2024-04-02T04:32:24.100 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout id: dc5403b4-f0a9-11ee-b647-cb9ed24678a4 2024-04-02T04:32:24.100 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-02T04:32:24.100 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-04-02T04:32:24.100 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout services: 2024-04-02T04:32:24.100 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi016 (age 0.526416s) 2024-04-02T04:32:24.101 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-02T04:32:24.101 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-02T04:32:24.101 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-04-02T04:32:24.101 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout data: 2024-04-02T04:32:24.101 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-02T04:32:24.101 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-02T04:32:24.101 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-02T04:32:24.101 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout pgs: 2024-04-02T04:32:24.101 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-04-02T04:32:24.101 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.459+0000 7ffafa560700 1 Processor -- start 2024-04-02T04:32:24.101 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.459+0000 7ffafa560700 1 -- start start 2024-04-02T04:32:24.101 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.459+0000 7ffafa560700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf4106fd0 0x7ffaf41093c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:32:24.101 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.459+0000 7ffafa560700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf4079e30 con 0x7ffaf4106fd0 2024-04-02T04:32:24.101 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.459+0000 7ffaf3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf4106fd0 0x7ffaf41093c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:32:24.102 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.459+0000 7ffaf3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf4106fd0 0x7ffaf41093c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52936/0 (socket says 172.21.15.16:52936) 2024-04-02T04:32:24.102 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.459+0000 7ffaf3fff700 1 -- 172.21.15.16:0/1013965821 learned_addr learned my addr 172.21.15.16:0/1013965821 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:32:24.102 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.460+0000 7ffaf3fff700 1 -- 172.21.15.16:0/1013965821 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf4109900 con 0x7ffaf4106fd0 2024-04-02T04:32:24.102 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.460+0000 7ffaf3fff700 1 --2- 172.21.15.16:0/1013965821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf4106fd0 0x7ffaf41093c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ffae40098a0 tx=0x7ffae4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b546b31c941a22df server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:32:24.102 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.460+0000 7ffaf2ffd700 1 -- 172.21.15.16:0/1013965821 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffae4004180 con 0x7ffaf4106fd0 2024-04-02T04:32:24.102 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.460+0000 7ffaf2ffd700 1 -- 172.21.15.16:0/1013965821 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae40042e0 con 0x7ffaf4106fd0 2024-04-02T04:32:24.102 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.460+0000 7ffaf2ffd700 1 -- 172.21.15.16:0/1013965821 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffae40036a0 con 0x7ffaf4106fd0 2024-04-02T04:32:24.102 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.460+0000 7ffafa560700 1 -- 172.21.15.16:0/1013965821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf4106fd0 msgr2=0x7ffaf41093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:32:24.102 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.460+0000 7ffafa560700 1 --2- 172.21.15.16:0/1013965821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf4106fd0 0x7ffaf41093c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ffae40098a0 tx=0x7ffae4009bb0 comp rx=0 tx=0).stop 2024-04-02T04:32:24.102 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.461+0000 7ffafa560700 1 -- 172.21.15.16:0/1013965821 shutdown_connections 2024-04-02T04:32:24.102 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.461+0000 7ffafa560700 1 --2- 172.21.15.16:0/1013965821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf4106fd0 0x7ffaf41093c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:32:24.102 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.461+0000 7ffafa560700 1 -- 172.21.15.16:0/1013965821 >> 172.21.15.16:0/1013965821 conn(0x7ffaf4100a00 msgr2=0x7ffaf4102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:32:24.102 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.461+0000 7ffafa560700 1 -- 172.21.15.16:0/1013965821 shutdown_connections 2024-04-02T04:32:24.103 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.461+0000 7ffafa560700 1 -- 172.21.15.16:0/1013965821 wait complete. 2024-04-02T04:32:24.103 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.461+0000 7ffafa560700 1 Processor -- start 2024-04-02T04:32:24.103 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.461+0000 7ffafa560700 1 -- start start 2024-04-02T04:32:24.103 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.462+0000 7ffafa560700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf4106fd0 0x7ffaf4106800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:32:24.103 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.462+0000 7ffafa560700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf4106d40 con 0x7ffaf4106fd0 2024-04-02T04:32:24.103 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.462+0000 7ffaf3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf4106fd0 0x7ffaf4106800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:32:24.103 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.462+0000 7ffaf3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf4106fd0 0x7ffaf4106800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52942/0 (socket says 172.21.15.16:52942) 2024-04-02T04:32:24.103 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.462+0000 7ffaf3fff700 1 -- 172.21.15.16:0/773621853 learned_addr learned my addr 172.21.15.16:0/773621853 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:32:24.103 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.462+0000 7ffaf3fff700 1 -- 172.21.15.16:0/773621853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae4009550 con 0x7ffaf4106fd0 2024-04-02T04:32:24.103 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.462+0000 7ffaf3fff700 1 --2- 172.21.15.16:0/773621853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf4106fd0 0x7ffaf4106800 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4079270 tx=0x7ffae4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:32:24.103 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.462+0000 7ffaf17fa700 1 -- 172.21.15.16:0/773621853 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffae4003ce0 con 0x7ffaf4106fd0 2024-04-02T04:32:24.103 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.462+0000 7ffaf17fa700 1 -- 172.21.15.16:0/773621853 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae4020070 con 0x7ffaf4106fd0 2024-04-02T04:32:24.104 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.462+0000 7ffafa560700 1 -- 172.21.15.16:0/773621853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaf4104f80 con 0x7ffaf4106fd0 2024-04-02T04:32:24.104 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.462+0000 7ffaf17fa700 1 -- 172.21.15.16:0/773621853 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffae4011630 con 0x7ffaf4106fd0 2024-04-02T04:32:24.104 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.462+0000 7ffafa560700 1 -- 172.21.15.16:0/773621853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaf4105420 con 0x7ffaf4106fd0 2024-04-02T04:32:24.104 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.462+0000 7ffaf17fa700 1 -- 172.21.15.16:0/773621853 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ffae4022030 con 0x7ffaf4106fd0 2024-04-02T04:32:24.104 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.463+0000 7ffaf17fa700 1 -- 172.21.15.16:0/773621853 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ffae4025de0 con 0x7ffaf4106fd0 2024-04-02T04:32:24.104 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.463+0000 7ffafa560700 1 -- 172.21.15.16:0/773621853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaf40623e0 con 0x7ffaf4106fd0 2024-04-02T04:32:24.104 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.466+0000 7ffaf17fa700 1 -- 172.21.15.16:0/773621853 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7ffae4025480 con 0x7ffaf4106fd0 2024-04-02T04:32:24.104 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.539+0000 7ffafa560700 1 -- 172.21.15.16:0/773621853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7ffaf4109d60 con 0x7ffaf4106fd0 2024-04-02T04:32:24.104 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.539+0000 7ffaf17fa700 1 -- 172.21.15.16:0/773621853 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7ffae4011c20 con 0x7ffaf4106fd0 2024-04-02T04:32:24.104 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.540+0000 7ffafa560700 1 -- 172.21.15.16:0/773621853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf4106fd0 msgr2=0x7ffaf4106800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:32:24.104 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.540+0000 7ffafa560700 1 --2- 172.21.15.16:0/773621853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf4106fd0 0x7ffaf4106800 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4079270 tx=0x7ffae4004750 comp rx=0 tx=0).stop 2024-04-02T04:32:24.104 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.540+0000 7ffafa560700 1 -- 172.21.15.16:0/773621853 shutdown_connections 2024-04-02T04:32:24.104 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.540+0000 7ffafa560700 1 --2- 172.21.15.16:0/773621853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf4106fd0 0x7ffaf4106800 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:32:24.104 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.540+0000 7ffafa560700 1 -- 172.21.15.16:0/773621853 >> 172.21.15.16:0/773621853 conn(0x7ffaf4100a00 msgr2=0x7ffaf4102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:32:24.105 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.540+0000 7ffafa560700 1 -- 172.21.15.16:0/773621853 shutdown_connections 2024-04-02T04:32:24.105 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:23.540+0000 7ffafa560700 1 -- 172.21.15.16:0/773621853 wait complete. 2024-04-02T04:32:24.105 INFO:teuthology.orchestra.run.smithi016.stdout:mon is available 2024-04-02T04:32:24.105 INFO:teuthology.orchestra.run.smithi016.stdout:Assimilating anything we can from ceph.conf... 2024-04-02T04:32:25.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:25 smithi016 ceph-mon[28593]: from='client.? 172.21.15.16:0/3927245349' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-02T04:32:25.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:25 smithi016 ceph-mon[28593]: from='client.? 172.21.15.16:0/3927245349' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-02T04:32:25.123 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-04-02T04:32:25.123 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [global] 2024-04-02T04:32:25.123 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout fsid = dc5403b4-f0a9-11ee-b647-cb9ed24678a4 2024-04-02T04:32:25.123 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.16:3300,v1:172.21.15.16:6789] 2024-04-02T04:32:25.124 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-02T04:32:25.124 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-02T04:32:25.124 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-02T04:32:25.124 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-02T04:32:25.124 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-04-02T04:32:25.124 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-02T04:32:25.124 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-02T04:32:25.124 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-04-02T04:32:25.124 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [osd] 2024-04-02T04:32:25.124 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-02T04:32:25.124 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-02T04:32:25.124 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-02T04:32:25.124 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-02T04:32:25.124 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-02T04:32:25.125 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.477+0000 7f1bbdd60700 1 Processor -- start 2024-04-02T04:32:25.125 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.477+0000 7f1bbdd60700 1 -- start start 2024-04-02T04:32:25.125 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.477+0000 7f1bbdd60700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bb8104820 0x7f1bb8104c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:32:25.125 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.477+0000 7f1bbdd60700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb8105210 con 0x7f1bb8104820 2024-04-02T04:32:25.125 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.477+0000 7f1bb77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bb8104820 0x7f1bb8104c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:32:25.125 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.477+0000 7f1bb77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bb8104820 0x7f1bb8104c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52950/0 (socket says 172.21.15.16:52950) 2024-04-02T04:32:25.125 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.477+0000 7f1bb77fe700 1 -- 172.21.15.16:0/2102708584 learned_addr learned my addr 172.21.15.16:0/2102708584 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:32:25.125 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.478+0000 7f1bb77fe700 1 -- 172.21.15.16:0/2102708584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb81077b0 con 0x7f1bb8104820 2024-04-02T04:32:25.125 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.478+0000 7f1bb77fe700 1 --2- 172.21.15.16:0/2102708584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bb8104820 0x7f1bb8104c40 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f1ba80098a0 tx=0x7f1ba8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f028f1c0df8f6d45 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:32:25.125 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.478+0000 7f1bb67fc700 1 -- 172.21.15.16:0/2102708584 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1ba8004180 con 0x7f1bb8104820 2024-04-02T04:32:25.125 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.478+0000 7f1bb67fc700 1 -- 172.21.15.16:0/2102708584 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba80042e0 con 0x7f1bb8104820 2024-04-02T04:32:25.125 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.478+0000 7f1bb67fc700 1 -- 172.21.15.16:0/2102708584 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1ba80036a0 con 0x7f1bb8104820 2024-04-02T04:32:25.125 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.478+0000 7f1bbdd60700 1 -- 172.21.15.16:0/2102708584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bb8104820 msgr2=0x7f1bb8104c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:32:25.125 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.478+0000 7f1bbdd60700 1 --2- 172.21.15.16:0/2102708584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bb8104820 0x7f1bb8104c40 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f1ba80098a0 tx=0x7f1ba8009bb0 comp rx=0 tx=0).stop 2024-04-02T04:32:25.126 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.479+0000 7f1bbdd60700 1 -- 172.21.15.16:0/2102708584 shutdown_connections 2024-04-02T04:32:25.126 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.479+0000 7f1bbdd60700 1 --2- 172.21.15.16:0/2102708584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bb8104820 0x7f1bb8104c40 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:32:25.126 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.479+0000 7f1bbdd60700 1 -- 172.21.15.16:0/2102708584 >> 172.21.15.16:0/2102708584 conn(0x7f1bb8100170 msgr2=0x7f1bb8102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:32:25.126 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.479+0000 7f1bbdd60700 1 -- 172.21.15.16:0/2102708584 shutdown_connections 2024-04-02T04:32:25.126 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.479+0000 7f1bbdd60700 1 -- 172.21.15.16:0/2102708584 wait complete. 2024-04-02T04:32:25.126 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.479+0000 7f1bbdd60700 1 Processor -- start 2024-04-02T04:32:25.126 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.479+0000 7f1bbdd60700 1 -- start start 2024-04-02T04:32:25.126 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.479+0000 7f1bbdd60700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bb8104820 0x7f1bb8106fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:32:25.126 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.479+0000 7f1bbdd60700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb8107520 con 0x7f1bb8104820 2024-04-02T04:32:25.126 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.480+0000 7f1bb77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bb8104820 0x7f1bb8106fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:32:25.126 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.480+0000 7f1bb77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bb8104820 0x7f1bb8106fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52962/0 (socket says 172.21.15.16:52962) 2024-04-02T04:32:25.126 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.480+0000 7f1bb77fe700 1 -- 172.21.15.16:0/3927245349 learned_addr learned my addr 172.21.15.16:0/3927245349 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:32:25.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.480+0000 7f1bb77fe700 1 -- 172.21.15.16:0/3927245349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba8009550 con 0x7f1bb8104820 2024-04-02T04:32:25.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.480+0000 7f1bb77fe700 1 --2- 172.21.15.16:0/3927245349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bb8104820 0x7f1bb8106fe0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8004000 tx=0x7f1ba8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:32:25.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.480+0000 7f1bb4ff9700 1 -- 172.21.15.16:0/3927245349 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1ba8003ce0 con 0x7f1bb8104820 2024-04-02T04:32:25.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.480+0000 7f1bb4ff9700 1 -- 172.21.15.16:0/3927245349 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba8020070 con 0x7f1bb8104820 2024-04-02T04:32:25.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.480+0000 7f1bb4ff9700 1 -- 172.21.15.16:0/3927245349 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1ba8011630 con 0x7f1bb8104820 2024-04-02T04:32:25.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.480+0000 7f1bbdd60700 1 -- 172.21.15.16:0/3927245349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bb8105760 con 0x7f1bb8104820 2024-04-02T04:32:25.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.480+0000 7f1bbdd60700 1 -- 172.21.15.16:0/3927245349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bb8105c00 con 0x7f1bb8104820 2024-04-02T04:32:25.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.480+0000 7f1bb4ff9700 1 -- 172.21.15.16:0/3927245349 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1ba8022030 con 0x7f1bb8104820 2024-04-02T04:32:25.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.480+0000 7f1bb4ff9700 1 -- 172.21.15.16:0/3927245349 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1ba8025d40 con 0x7f1bb8104820 2024-04-02T04:32:25.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.481+0000 7f1bbdd60700 1 -- 172.21.15.16:0/3927245349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bb80623e0 con 0x7f1bb8104820 2024-04-02T04:32:25.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.483+0000 7f1bb4ff9700 1 -- 172.21.15.16:0/3927245349 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f1ba8025480 con 0x7f1bb8104820 2024-04-02T04:32:25.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.554+0000 7f1bbdd60700 1 -- 172.21.15.16:0/3927245349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f1bb8107b20 con 0x7f1bb8104820 2024-04-02T04:32:25.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.557+0000 7f1bb4ff9700 1 -- 172.21.15.16:0/3927245349 <== mon.0 v2:172.21.15.16:3300/0 7 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f1ba8011c20 con 0x7f1bb8104820 2024-04-02T04:32:25.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.557+0000 7f1bb4ff9700 1 -- 172.21.15.16:0/3927245349 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f1ba80375f0 con 0x7f1bb8104820 2024-04-02T04:32:25.128 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.558+0000 7f1bbdd60700 1 -- 172.21.15.16:0/3927245349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bb8104820 msgr2=0x7f1bb8106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:32:25.128 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.558+0000 7f1bbdd60700 1 --2- 172.21.15.16:0/3927245349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bb8104820 0x7f1bb8106fe0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8004000 tx=0x7f1ba8004750 comp rx=0 tx=0).stop 2024-04-02T04:32:25.128 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.558+0000 7f1bbdd60700 1 -- 172.21.15.16:0/3927245349 shutdown_connections 2024-04-02T04:32:25.128 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.558+0000 7f1bbdd60700 1 --2- 172.21.15.16:0/3927245349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bb8104820 0x7f1bb8106fe0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:32:25.128 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.558+0000 7f1bbdd60700 1 -- 172.21.15.16:0/3927245349 >> 172.21.15.16:0/3927245349 conn(0x7f1bb8100170 msgr2=0x7f1bb81908c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:32:25.128 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.558+0000 7f1bbdd60700 1 -- 172.21.15.16:0/3927245349 shutdown_connections 2024-04-02T04:32:25.128 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:24.558+0000 7f1bbdd60700 1 -- 172.21.15.16:0/3927245349 wait complete. 2024-04-02T04:32:25.128 INFO:teuthology.orchestra.run.smithi016.stdout:Generating new minimal ceph.conf... 2024-04-02T04:32:26.055 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.482+0000 7fd66d616700 1 Processor -- start 2024-04-02T04:32:26.055 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.482+0000 7fd66d616700 1 -- start start 2024-04-02T04:32:26.056 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.482+0000 7fd66d616700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd66807be30 0x7fd66807a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:32:26.056 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.482+0000 7fd66d616700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66807a8b0 con 0x7fd66807be30 2024-04-02T04:32:26.056 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.482+0000 7fd666ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd66807be30 0x7fd66807a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:32:26.056 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.482+0000 7fd666ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd66807be30 0x7fd66807a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52972/0 (socket says 172.21.15.16:52972) 2024-04-02T04:32:26.056 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.482+0000 7fd666ffd700 1 -- 172.21.15.16:0/4104861328 learned_addr learned my addr 172.21.15.16:0/4104861328 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:32:26.056 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.483+0000 7fd666ffd700 1 -- 172.21.15.16:0/4104861328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd66807a9f0 con 0x7fd66807be30 2024-04-02T04:32:26.056 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.483+0000 7fd666ffd700 1 --2- 172.21.15.16:0/4104861328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd66807be30 0x7fd66807a2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd6500098a0 tx=0x7fd650009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b58d52b1cc4a7d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:32:26.056 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.483+0000 7fd665ffb700 1 -- 172.21.15.16:0/4104861328 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd650004180 con 0x7fd66807be30 2024-04-02T04:32:26.056 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.483+0000 7fd665ffb700 1 -- 172.21.15.16:0/4104861328 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fd6500042e0 con 0x7fd66807be30 2024-04-02T04:32:26.056 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.483+0000 7fd665ffb700 1 -- 172.21.15.16:0/4104861328 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd650011a70 con 0x7fd66807be30 2024-04-02T04:32:26.056 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.483+0000 7fd66d616700 1 -- 172.21.15.16:0/4104861328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd66807be30 msgr2=0x7fd66807a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:32:26.056 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.483+0000 7fd66d616700 1 --2- 172.21.15.16:0/4104861328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd66807be30 0x7fd66807a2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd6500098a0 tx=0x7fd650009bb0 comp rx=0 tx=0).stop 2024-04-02T04:32:26.056 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.483+0000 7fd66d616700 1 -- 172.21.15.16:0/4104861328 shutdown_connections 2024-04-02T04:32:26.056 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.483+0000 7fd66d616700 1 --2- 172.21.15.16:0/4104861328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd66807be30 0x7fd66807a2e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:32:26.056 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.484+0000 7fd66d616700 1 -- 172.21.15.16:0/4104861328 >> 172.21.15.16:0/4104861328 conn(0x7fd668101b10 msgr2=0x7fd668103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:32:26.057 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.484+0000 7fd66d616700 1 -- 172.21.15.16:0/4104861328 shutdown_connections 2024-04-02T04:32:26.057 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.484+0000 7fd66d616700 1 -- 172.21.15.16:0/4104861328 wait complete. 2024-04-02T04:32:26.057 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.484+0000 7fd66d616700 1 Processor -- start 2024-04-02T04:32:26.057 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.484+0000 7fd66d616700 1 -- start start 2024-04-02T04:32:26.057 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.484+0000 7fd66d616700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd66807be30 0x7fd66819c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:32:26.057 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.484+0000 7fd66d616700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66819c960 con 0x7fd66807be30 2024-04-02T04:32:26.057 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.485+0000 7fd666ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd66807be30 0x7fd66819c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:32:26.057 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.485+0000 7fd666ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd66807be30 0x7fd66819c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52986/0 (socket says 172.21.15.16:52986) 2024-04-02T04:32:26.057 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.485+0000 7fd666ffd700 1 -- 172.21.15.16:0/43238979 learned_addr learned my addr 172.21.15.16:0/43238979 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:32:26.057 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.485+0000 7fd666ffd700 1 -- 172.21.15.16:0/43238979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd650009550 con 0x7fd66807be30 2024-04-02T04:32:26.057 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.485+0000 7fd666ffd700 1 --2- 172.21.15.16:0/43238979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd66807be30 0x7fd66819c420 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd650011710 tx=0x7fd6500047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:32:26.057 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.485+0000 7fd65ffff700 1 -- 172.21.15.16:0/43238979 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd650003b10 con 0x7fd66807be30 2024-04-02T04:32:26.057 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.485+0000 7fd65ffff700 1 -- 172.21.15.16:0/43238979 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fd65001b8c0 con 0x7fd66807be30 2024-04-02T04:32:26.057 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.485+0000 7fd66d616700 1 -- 172.21.15.16:0/43238979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd66819cb60 con 0x7fd66807be30 2024-04-02T04:32:26.057 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.485+0000 7fd65ffff700 1 -- 172.21.15.16:0/43238979 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd6500197e0 con 0x7fd66807be30 2024-04-02T04:32:26.058 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.485+0000 7fd66d616700 1 -- 172.21.15.16:0/43238979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd66819d000 con 0x7fd66807be30 2024-04-02T04:32:26.058 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.485+0000 7fd65ffff700 1 -- 172.21.15.16:0/43238979 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd650019cd0 con 0x7fd66807be30 2024-04-02T04:32:26.058 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.485+0000 7fd65ffff700 1 -- 172.21.15.16:0/43238979 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd65001a710 con 0x7fd66807be30 2024-04-02T04:32:26.058 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.486+0000 7fd66d616700 1 -- 172.21.15.16:0/43238979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd66810c810 con 0x7fd66807be30 2024-04-02T04:32:26.058 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.488+0000 7fd65ffff700 1 -- 172.21.15.16:0/43238979 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7fd650029060 con 0x7fd66807be30 2024-04-02T04:32:26.058 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.557+0000 7fd66d616700 1 -- 172.21.15.16:0/43238979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fd66810c9d0 con 0x7fd66807be30 2024-04-02T04:32:26.058 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.557+0000 7fd65ffff700 1 -- 172.21.15.16:0/43238979 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7fd65001f070 con 0x7fd66807be30 2024-04-02T04:32:26.058 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.558+0000 7fd66d616700 1 -- 172.21.15.16:0/43238979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd66807be30 msgr2=0x7fd66819c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:32:26.058 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.558+0000 7fd66d616700 1 --2- 172.21.15.16:0/43238979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd66807be30 0x7fd66819c420 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd650011710 tx=0x7fd6500047c0 comp rx=0 tx=0).stop 2024-04-02T04:32:26.058 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.558+0000 7fd66d616700 1 -- 172.21.15.16:0/43238979 shutdown_connections 2024-04-02T04:32:26.058 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.558+0000 7fd66d616700 1 --2- 172.21.15.16:0/43238979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd66807be30 0x7fd66819c420 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:32:26.058 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.558+0000 7fd66d616700 1 -- 172.21.15.16:0/43238979 >> 172.21.15.16:0/43238979 conn(0x7fd668101b10 msgr2=0x7fd6681026a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:32:26.058 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.558+0000 7fd66d616700 1 -- 172.21.15.16:0/43238979 shutdown_connections 2024-04-02T04:32:26.059 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:25.558+0000 7fd66d616700 1 -- 172.21.15.16:0/43238979 wait complete. 2024-04-02T04:32:26.059 INFO:teuthology.orchestra.run.smithi016.stdout:Restarting the monitor... 2024-04-02T04:32:26.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:26 smithi016 ceph-mon[28593]: from='client.? 172.21.15.16:0/43238979' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:32:26.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:26 smithi016 systemd[1]: Stopping Ceph mon.smithi016 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4... 2024-04-02T04:32:26.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:26 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi016[28589]: 2024-04-02T04:32:26.323+0000 7f4409c32700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi016 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T04:32:26.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:26 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi016[28589]: 2024-04-02T04:32:26.323+0000 7f4409c32700 -1 mon.smithi016@0(leader) e1 *** Got Signal Terminated *** 2024-04-02T04:32:26.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:26 smithi016 podman[28838]: 2024-04-02 04:32:26.539112433 +0000 UTC m=+0.323386745 container died 9dad17bc9d749f554c2dcd61df1813471c065c98c93cb698b4034c6d3c2bb85f (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi016, RELEASE=reef-4d05f4c, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T04:32:28.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:27 smithi016 podman[28838]: 2024-04-02 04:32:27.866301171 +0000 UTC m=+1.650575493 container cleanup 9dad17bc9d749f554c2dcd61df1813471c065c98c93cb698b4034c6d3c2bb85f (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi016, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=reef-4d05f4c, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-02T04:32:28.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:27 smithi016 bash[28838]: ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi016 2024-04-02T04:32:30.029 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:29 smithi016 podman[28851]: 2024-04-02 04:32:29.727041812 +0000 UTC m=+3.185945771 container remove 9dad17bc9d749f554c2dcd61df1813471c065c98c93cb698b4034c6d3c2bb85f (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi016, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=reef-4d05f4c, org.label-schema.build-date=20240326, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-02T04:32:30.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:30 smithi016 systemd[1]: ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@mon.smithi016.service: Deactivated successfully. 2024-04-02T04:32:30.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:30 smithi016 systemd[1]: Stopped Ceph mon.smithi016 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4. 2024-04-02T04:32:30.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:30 smithi016 systemd[1]: Starting Ceph mon.smithi016 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4... 2024-04-02T04:32:30.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:30 smithi016 podman[28950]: 2024-04-02 04:32:30.162802667 +0000 UTC m=+0.017182508 image pull 909dce4fe53d7167a6fa8180542b49bd6d1a8c351b846c37da562093c6f0cffd quay.ceph.io/ceph-ci/ceph:reef 2024-04-02T04:32:32.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:31 smithi016 podman[28950]: 2024-04-02 04:32:31.706323617 +0000 UTC m=+1.560703445 container create 24d5820dc9942c9a49ee6314cd7bbf0c255fdbdf7bc7e5004950ab521f7caaf4 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi016, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, RELEASE=reef-4d05f4c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-02T04:32:34.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 podman[28950]: 2024-04-02 04:32:33.601428899 +0000 UTC m=+3.455808737 container init 24d5820dc9942c9a49ee6314cd7bbf0c255fdbdf7bc7e5004950ab521f7caaf4 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi016, RELEASE=reef-4d05f4c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-02T04:32:34.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 podman[28950]: 2024-04-02 04:32:33.605305174 +0000 UTC m=+3.459685006 container start 24d5820dc9942c9a49ee6314cd7bbf0c255fdbdf7bc7e5004950ab521f7caaf4 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi016, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=reef-4d05f4c, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T04:32:34.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: set uid:gid to 167:167 (ceph:ceph) 2024-04-02T04:32:34.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable), process ceph-mon, pid 2 2024-04-02T04:32:34.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: pidfile_write: ignore empty --pid-file 2024-04-02T04:32:34.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: load: jerasure load: lrc 2024-04-02T04:32:34.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: RocksDB version: 7.9.2 2024-04-02T04:32:34.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Git sha 0 2024-04-02T04:32:34.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Compile date 2024-04-01 20:31:51 2024-04-02T04:32:34.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: DB SUMMARY 2024-04-02T04:32:34.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: DB Session ID: VYB26239GJZ8JVGRM3HU 2024-04-02T04:32:34.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: CURRENT file: CURRENT 2024-04-02T04:32:34.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: IDENTITY file: IDENTITY 2024-04-02T04:32:34.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-02T04:32:34.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi016/store.db dir, Total Num: 1, files: 000008.sst 2024-04-02T04:32:34.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi016/store.db: 000009.log size: 98461 ; 2024-04-02T04:32:34.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.error_if_exists: 0 2024-04-02T04:32:34.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.create_if_missing: 0 2024-04-02T04:32:34.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.paranoid_checks: 1 2024-04-02T04:32:34.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-02T04:32:34.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-02T04:32:34.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-02T04:32:34.062 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.env: 0x55d71ee5d720 2024-04-02T04:32:34.062 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.fs: PosixFileSystem 2024-04-02T04:32:34.062 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.info_log: 0x55d7216cf380 2024-04-02T04:32:34.062 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.max_file_opening_threads: 16 2024-04-02T04:32:34.062 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.statistics: (nil) 2024-04-02T04:32:34.062 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.use_fsync: 0 2024-04-02T04:32:34.063 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.max_log_file_size: 0 2024-04-02T04:32:34.063 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-02T04:32:34.064 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-02T04:32:34.064 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.keep_log_file_num: 1000 2024-04-02T04:32:34.064 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.recycle_log_file_num: 0 2024-04-02T04:32:34.064 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.allow_fallocate: 1 2024-04-02T04:32:34.064 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.allow_mmap_reads: 0 2024-04-02T04:32:34.064 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.allow_mmap_writes: 0 2024-04-02T04:32:34.064 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.use_direct_reads: 0 2024-04-02T04:32:34.064 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-02T04:32:34.064 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.create_missing_column_families: 0 2024-04-02T04:32:34.064 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.db_log_dir: 2024-04-02T04:32:34.064 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.wal_dir: 2024-04-02T04:32:34.064 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-02T04:32:34.064 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-02T04:32:34.064 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-02T04:32:34.065 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-02T04:32:34.065 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-02T04:32:34.065 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-02T04:32:34.065 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.advise_random_on_open: 1 2024-04-02T04:32:34.065 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.db_write_buffer_size: 0 2024-04-02T04:32:34.065 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.write_buffer_manager: 0x55d720946140 2024-04-02T04:32:34.065 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-02T04:32:34.065 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-02T04:32:34.065 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-02T04:32:34.065 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.rate_limiter: (nil) 2024-04-02T04:32:34.065 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-02T04:32:34.065 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.wal_recovery_mode: 2 2024-04-02T04:32:34.065 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.enable_thread_tracking: 0 2024-04-02T04:32:34.065 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.enable_pipelined_write: 0 2024-04-02T04:32:34.065 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.unordered_write: 0 2024-04-02T04:32:34.066 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-02T04:32:34.066 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-02T04:32:34.066 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-02T04:32:34.066 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-02T04:32:34.066 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.row_cache: None 2024-04-02T04:32:34.066 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.wal_filter: None 2024-04-02T04:32:34.066 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-02T04:32:34.066 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.allow_ingest_behind: 0 2024-04-02T04:32:34.066 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.two_write_queues: 0 2024-04-02T04:32:34.066 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.manual_wal_flush: 0 2024-04-02T04:32:34.066 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.wal_compression: 0 2024-04-02T04:32:34.066 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.atomic_flush: 0 2024-04-02T04:32:34.066 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-02T04:32:34.067 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-02T04:32:34.067 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-02T04:32:34.067 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.log_readahead_size: 0 2024-04-02T04:32:34.067 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-02T04:32:34.067 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.best_efforts_recovery: 0 2024-04-02T04:32:34.067 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-02T04:32:34.067 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-02T04:32:34.067 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.allow_data_in_errors: 0 2024-04-02T04:32:34.067 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.db_host_id: __hostname__ 2024-04-02T04:32:34.067 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-02T04:32:34.067 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.max_background_jobs: 2 2024-04-02T04:32:34.067 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.max_background_compactions: -1 2024-04-02T04:32:34.068 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.max_subcompactions: 1 2024-04-02T04:32:34.068 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-02T04:32:34.068 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-02T04:32:34.068 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-02T04:32:34.068 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.max_total_wal_size: 0 2024-04-02T04:32:34.068 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-02T04:32:34.068 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-02T04:32:34.068 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-02T04:32:34.068 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-02T04:32:34.068 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.max_open_files: -1 2024-04-02T04:32:34.068 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.bytes_per_sync: 0 2024-04-02T04:32:34.068 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-02T04:32:34.068 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-02T04:32:34.069 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.compaction_readahead_size: 0 2024-04-02T04:32:34.069 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.max_background_flushes: -1 2024-04-02T04:32:34.069 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Compression algorithms supported: 2024-04-02T04:32:34.069 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-02T04:32:34.069 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: kZSTD supported: 0 2024-04-02T04:32:34.069 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: kXpressCompression supported: 0 2024-04-02T04:32:34.069 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: kLZ4HCCompression supported: 1 2024-04-02T04:32:34.069 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: kZlibCompression supported: 1 2024-04-02T04:32:34.069 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: kSnappyCompression supported: 1 2024-04-02T04:32:34.069 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: kLZ4Compression supported: 1 2024-04-02T04:32:34.070 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: kBZip2Compression supported: 0 2024-04-02T04:32:34.070 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-02T04:32:34.070 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-02T04:32:34.070 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi016/store.db/MANIFEST-000010 2024-04-02T04:32:34.070 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-02T04:32:34.070 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-02T04:32:34.071 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.merge_operator: 2024-04-02T04:32:34.071 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.compaction_filter: None 2024-04-02T04:32:34.071 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.compaction_filter_factory: None 2024-04-02T04:32:34.071 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.sst_partitioner_factory: None 2024-04-02T04:32:34.071 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-02T04:32:34.071 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-02T04:32:34.072 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d7216cf4a0) 2024-04-02T04:32:34.072 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: cache_index_and_filter_blocks: 1 2024-04-02T04:32:34.072 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-02T04:32:34.072 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-02T04:32:34.072 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: pin_top_level_index_and_filter: 1 2024-04-02T04:32:34.072 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: index_type: 0 2024-04-02T04:32:34.072 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: data_block_index_type: 0 2024-04-02T04:32:34.073 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: index_shortening: 1 2024-04-02T04:32:34.073 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-02T04:32:34.073 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: checksum: 4 2024-04-02T04:32:34.073 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: no_block_cache: 0 2024-04-02T04:32:34.073 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: block_cache: 0x55d7209e71f0 2024-04-02T04:32:34.073 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: block_cache_name: BinnedLRUCache 2024-04-02T04:32:34.074 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: block_cache_options: 2024-04-02T04:32:34.074 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: capacity : 536870912 2024-04-02T04:32:34.074 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: num_shard_bits : 4 2024-04-02T04:32:34.074 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: strict_capacity_limit : 0 2024-04-02T04:32:34.074 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: high_pri_pool_ratio: 0.000 2024-04-02T04:32:34.074 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: block_cache_compressed: (nil) 2024-04-02T04:32:34.075 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: persistent_cache: (nil) 2024-04-02T04:32:34.075 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: block_size: 4096 2024-04-02T04:32:34.075 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: block_size_deviation: 10 2024-04-02T04:32:34.075 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: block_restart_interval: 16 2024-04-02T04:32:34.075 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: index_block_restart_interval: 1 2024-04-02T04:32:34.075 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: metadata_block_size: 4096 2024-04-02T04:32:34.076 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: partition_filters: 0 2024-04-02T04:32:34.076 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: use_delta_encoding: 1 2024-04-02T04:32:34.076 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: filter_policy: bloomfilter 2024-04-02T04:32:34.076 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: whole_key_filtering: 1 2024-04-02T04:32:34.076 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: verify_compression: 0 2024-04-02T04:32:34.076 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: read_amp_bytes_per_bit: 0 2024-04-02T04:32:34.076 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: format_version: 5 2024-04-02T04:32:34.077 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: enable_index_compression: 1 2024-04-02T04:32:34.077 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: block_align: 0 2024-04-02T04:32:34.077 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: max_auto_readahead_size: 262144 2024-04-02T04:32:34.077 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: prepopulate_block_cache: 0 2024-04-02T04:32:34.077 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: initial_auto_readahead_size: 8192 2024-04-02T04:32:34.077 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: num_file_reads_for_auto_readahead: 2 2024-04-02T04:32:34.077 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.write_buffer_size: 33554432 2024-04-02T04:32:34.077 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.max_write_buffer_number: 2 2024-04-02T04:32:34.077 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.compression: NoCompression 2024-04-02T04:32:34.078 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.bottommost_compression: Disabled 2024-04-02T04:32:34.078 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.prefix_extractor: nullptr 2024-04-02T04:32:34.078 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-02T04:32:34.078 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.num_levels: 7 2024-04-02T04:32:34.078 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-02T04:32:34.078 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-02T04:32:34.079 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-02T04:32:34.079 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-02T04:32:34.079 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-02T04:32:34.079 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-02T04:32:34.079 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-02T04:32:34.079 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-02T04:32:34.079 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-02T04:32:34.080 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-02T04:32:34.080 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-02T04:32:34.080 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-02T04:32:34.080 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-02T04:32:34.080 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.compression_opts.level: 32767 2024-04-02T04:32:34.080 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.compression_opts.strategy: 0 2024-04-02T04:32:34.080 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-02T04:32:34.080 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-02T04:32:34.080 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-02T04:32:34.080 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-02T04:32:34.080 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.compression_opts.enabled: false 2024-04-02T04:32:34.080 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-02T04:32:34.081 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-02T04:32:34.081 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-02T04:32:34.081 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-02T04:32:34.081 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.target_file_size_base: 67108864 2024-04-02T04:32:34.081 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-02T04:32:34.081 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-02T04:32:34.081 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-02T04:32:34.081 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-02T04:32:34.081 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-02T04:32:34.081 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-02T04:32:34.081 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-02T04:32:34.081 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-02T04:32:34.081 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-02T04:32:34.081 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-02T04:32:34.081 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-02T04:32:34.081 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-02T04:32:34.082 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-02T04:32:34.082 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-02T04:32:34.082 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.arena_block_size: 1048576 2024-04-02T04:32:34.082 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-02T04:32:34.082 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-02T04:32:34.082 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.disable_auto_compactions: 0 2024-04-02T04:32:34.082 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-02T04:32:34.082 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-02T04:32:34.082 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-02T04:32:34.082 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-02T04:32:34.082 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-02T04:32:34.082 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-02T04:32:34.083 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-02T04:32:34.083 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-02T04:32:34.083 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-02T04:32:34.083 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-02T04:32:34.083 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.table_properties_collectors: 2024-04-02T04:32:34.083 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.inplace_update_support: 0 2024-04-02T04:32:34.083 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-02T04:32:34.083 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-02T04:32:34.083 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-02T04:32:34.083 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-02T04:32:34.083 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.bloom_locality: 0 2024-04-02T04:32:34.083 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.max_successive_merges: 0 2024-04-02T04:32:34.083 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-02T04:32:34.083 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.paranoid_file_checks: 0 2024-04-02T04:32:34.083 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.force_consistency_checks: 1 2024-04-02T04:32:34.083 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.report_bg_io_stats: 0 2024-04-02T04:32:34.084 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.ttl: 2592000 2024-04-02T04:32:34.084 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-02T04:32:34.084 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-02T04:32:34.084 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-02T04:32:34.084 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.enable_blob_files: false 2024-04-02T04:32:34.084 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.min_blob_size: 0 2024-04-02T04:32:34.084 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.blob_file_size: 268435456 2024-04-02T04:32:34.084 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-02T04:32:34.084 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-02T04:32:34.084 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-02T04:32:34.084 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-02T04:32:34.084 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-02T04:32:34.084 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.blob_file_starting_level: 0 2024-04-02T04:32:34.084 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-02T04:32:34.084 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi016/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-02T04:32:34.084 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-02T04:32:34.085 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 05c51ba7-789d-46bb-92d6-8d939af0a557 2024-04-02T04:32:34.085 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712032353636892, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-02T04:32:34.085 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-02T04:32:34.085 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712032353637972, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93738, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 314, "table_properties": {"data_size": 91772, "index_size": 248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13576, "raw_average_key_size": 49, "raw_value_size": 84336, "raw_average_value_size": 308, "num_data_blocks": 11, "num_entries": 273, "num_filter_entries": 273, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1712032353, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "05c51ba7-789d-46bb-92d6-8d939af0a557", "db_session_id": "VYB26239GJZ8JVGRM3HU", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-02T04:32:34.085 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712032353638068, "job": 1, "event": "recovery_finished"} 2024-04-02T04:32:34.085 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-02T04:32:34.085 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi016/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-02T04:32:34.085 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d720a84000 2024-04-02T04:32:34.085 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: DB pointer 0x55d720a72000 2024-04-02T04:32:34.085 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-02T04:32:34.085 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-02T04:32:34.085 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: ** DB Stats ** 2024-04-02T04:32:34.085 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-02T04:32:34.085 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-02T04:32:34.086 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-02T04:32:34.086 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-02T04:32:34.086 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-02T04:32:34.086 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-02T04:32:34.086 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-02T04:32:34.086 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: 2024-04-02T04:32:34.086 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: ** Compaction Stats [default] ** 2024-04-02T04:32:34.086 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-02T04:32:34.086 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-02T04:32:34.086 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: L0 2/0 93.41 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 128.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-02T04:32:34.086 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Sum 2/0 93.41 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 128.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-02T04:32:34.086 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 128.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-02T04:32:34.086 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: 2024-04-02T04:32:34.087 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: ** Compaction Stats [default] ** 2024-04-02T04:32:34.087 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-02T04:32:34.087 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-02T04:32:34.087 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 128.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-02T04:32:34.087 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: 2024-04-02T04:32:34.087 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-02T04:32:34.087 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: 2024-04-02T04:32:34.087 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-02T04:32:34.087 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-02T04:32:34.087 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-02T04:32:34.087 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-02T04:32:34.087 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-02T04:32:34.087 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-02T04:32:34.087 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Cumulative compaction: 0.00 GB write, 26.69 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-02T04:32:34.087 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Interval compaction: 0.00 GB write, 26.69 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-02T04:32:34.087 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-02T04:32:34.088 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Block cache BinnedLRUCache@0x55d7209e71f0#2 capacity: 512.00 MB usage: 1.33 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-04-02T04:32:34.088 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.44 KB,8.34465e-05%) Misc(1,0.00 KB,0%) 2024-04-02T04:32:34.088 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: 2024-04-02T04:32:34.088 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-02T04:32:34.088 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: starting mon.smithi016 rank 0 at public addrs [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] at bind addrs [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi016 fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 2024-04-02T04:32:34.088 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: mon.smithi016@-1(???) e1 preinit fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 2024-04-02T04:32:34.088 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: mon.smithi016@-1(???).mds e1 new map 2024-04-02T04:32:34.088 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: mon.smithi016@-1(???).mds e1 print_map 2024-04-02T04:32:34.088 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: e1 2024-04-02T04:32:34.088 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:32:34.088 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:32:34.088 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: legacy client fscid: -1 2024-04-02T04:32:34.088 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: 2024-04-02T04:32:34.088 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: No filesystems configured 2024-04-02T04:32:34.088 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: mon.smithi016@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-02T04:32:34.088 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: mon.smithi016@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-02T04:32:34.089 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: mon.smithi016@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-02T04:32:34.089 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: mon.smithi016@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-02T04:32:34.089 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: mon.smithi016@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-02T04:32:34.089 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: mon.smithi016 is new leader, mons smithi016 in quorum (ranks 0) 2024-04-02T04:32:34.089 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: monmap e1: 1 mons at {smithi016=[v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-02T04:32:34.089 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: fsmap 2024-04-02T04:32:34.089 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: osdmap e1: 0 total, 0 up, 0 in 2024-04-02T04:32:34.089 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:33 smithi016 ceph-mon[28969]: mgrmap e1: no daemons active 2024-04-02T04:32:34.350 INFO:teuthology.orchestra.run.smithi016.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-02T04:32:34.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:34 smithi016 bash[28950]: 24d5820dc9942c9a49ee6314cd7bbf0c255fdbdf7bc7e5004950ab521f7caaf4 2024-04-02T04:32:34.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:34 smithi016 systemd[1]: Started Ceph mon.smithi016 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4. 2024-04-02T04:32:35.996 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.355+0000 7f156e54e700 1 Processor -- start 2024-04-02T04:32:35.996 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.355+0000 7f156e54e700 1 -- start start 2024-04-02T04:32:35.996 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.355+0000 7f156e54e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1568106f60 0x7f1568109350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:32:35.996 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.355+0000 7f156e54e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1568079e90 con 0x7f1568106f60 2024-04-02T04:32:35.996 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.355+0000 7f1567fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1568106f60 0x7f1568109350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:32:35.996 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.355+0000 7f1567fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1568106f60 0x7f1568109350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36566/0 (socket says 172.21.15.16:36566) 2024-04-02T04:32:35.996 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.356+0000 7f1567fff700 1 -- 172.21.15.16:0/2546811779 learned_addr learned my addr 172.21.15.16:0/2546811779 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:32:35.996 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.356+0000 7f1567fff700 1 -- 172.21.15.16:0/2546811779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1568109890 con 0x7f1568106f60 2024-04-02T04:32:35.997 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.356+0000 7f1567fff700 1 --2- 172.21.15.16:0/2546811779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1568106f60 0x7f1568109350 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f15580098a0 tx=0x7f1558009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d048632c3fafd680 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:32:35.997 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.356+0000 7f1566ffd700 1 -- 172.21.15.16:0/2546811779 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1558004180 con 0x7f1568106f60 2024-04-02T04:32:35.997 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.356+0000 7f1566ffd700 1 -- 172.21.15.16:0/2546811779 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f15580042e0 con 0x7f1568106f60 2024-04-02T04:32:35.997 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.356+0000 7f1566ffd700 1 -- 172.21.15.16:0/2546811779 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1558011a70 con 0x7f1568106f60 2024-04-02T04:32:35.997 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.357+0000 7f156e54e700 1 -- 172.21.15.16:0/2546811779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1568106f60 msgr2=0x7f1568109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:32:35.997 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.357+0000 7f156e54e700 1 --2- 172.21.15.16:0/2546811779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1568106f60 0x7f1568109350 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f15580098a0 tx=0x7f1558009bb0 comp rx=0 tx=0).stop 2024-04-02T04:32:35.997 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.357+0000 7f156e54e700 1 -- 172.21.15.16:0/2546811779 shutdown_connections 2024-04-02T04:32:35.997 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.357+0000 7f156e54e700 1 --2- 172.21.15.16:0/2546811779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1568106f60 0x7f1568109350 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:32:35.997 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.357+0000 7f156e54e700 1 -- 172.21.15.16:0/2546811779 >> 172.21.15.16:0/2546811779 conn(0x7f15681009d0 msgr2=0x7f1568102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:32:35.997 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.357+0000 7f156e54e700 1 -- 172.21.15.16:0/2546811779 shutdown_connections 2024-04-02T04:32:35.997 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.357+0000 7f156e54e700 1 -- 172.21.15.16:0/2546811779 wait complete. 2024-04-02T04:32:35.997 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.357+0000 7f156e54e700 1 Processor -- start 2024-04-02T04:32:35.998 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.358+0000 7f156e54e700 1 -- start start 2024-04-02T04:32:35.998 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.358+0000 7f156e54e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1568106f60 0x7f1568106790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:32:35.998 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.358+0000 7f156e54e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1568106cd0 con 0x7f1568106f60 2024-04-02T04:32:35.998 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.358+0000 7f1567fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1568106f60 0x7f1568106790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:32:35.998 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.358+0000 7f1567fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1568106f60 0x7f1568106790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36570/0 (socket says 172.21.15.16:36570) 2024-04-02T04:32:35.998 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.358+0000 7f1567fff700 1 -- 172.21.15.16:0/3333245743 learned_addr learned my addr 172.21.15.16:0/3333245743 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:32:35.998 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.358+0000 7f1567fff700 1 -- 172.21.15.16:0/3333245743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1558009550 con 0x7f1568106f60 2024-04-02T04:32:35.998 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.358+0000 7f1567fff700 1 --2- 172.21.15.16:0/3333245743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1568106f60 0x7f1568106790 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1558011a20 tx=0x7f15580047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:32:35.998 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.359+0000 7f15657fa700 1 -- 172.21.15.16:0/3333245743 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1558003b10 con 0x7f1568106f60 2024-04-02T04:32:35.998 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.359+0000 7f15657fa700 1 -- 172.21.15.16:0/3333245743 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f155801b8c0 con 0x7f1568106f60 2024-04-02T04:32:35.998 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.359+0000 7f156e54e700 1 -- 172.21.15.16:0/3333245743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1568104f10 con 0x7f1568106f60 2024-04-02T04:32:35.998 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.359+0000 7f15657fa700 1 -- 172.21.15.16:0/3333245743 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f15580237b0 con 0x7f1568106f60 2024-04-02T04:32:35.998 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.359+0000 7f156e54e700 1 -- 172.21.15.16:0/3333245743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15681053b0 con 0x7f1568106f60 2024-04-02T04:32:35.998 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.359+0000 7f15657fa700 1 -- 172.21.15.16:0/3333245743 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1558023ce0 con 0x7f1568106f60 2024-04-02T04:32:35.999 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.359+0000 7f15657fa700 1 -- 172.21.15.16:0/3333245743 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1558019710 con 0x7f1568106f60 2024-04-02T04:32:35.999 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.360+0000 7f156e54e700 1 -- 172.21.15.16:0/3333245743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f156810a610 con 0x7f1568106f60 2024-04-02T04:32:35.999 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.362+0000 7f15657fa700 1 -- 172.21.15.16:0/3333245743 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f155801ba30 con 0x7f1568106f60 2024-04-02T04:32:35.999 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.432+0000 7f156e54e700 1 -- 172.21.15.16:0/3333245743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f15681055f0 con 0x7f1568106f60 2024-04-02T04:32:35.999 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.434+0000 7f15657fa700 1 -- 172.21.15.16:0/3333245743 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f155801bbd0 con 0x7f1568106f60 2024-04-02T04:32:35.999 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.435+0000 7f156e54e700 1 -- 172.21.15.16:0/3333245743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1568106f60 msgr2=0x7f1568106790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:32:35.999 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.435+0000 7f156e54e700 1 --2- 172.21.15.16:0/3333245743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1568106f60 0x7f1568106790 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1558011a20 tx=0x7f15580047c0 comp rx=0 tx=0).stop 2024-04-02T04:32:35.999 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.435+0000 7f156e54e700 1 -- 172.21.15.16:0/3333245743 shutdown_connections 2024-04-02T04:32:35.999 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.435+0000 7f156e54e700 1 --2- 172.21.15.16:0/3333245743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1568106f60 0x7f1568106790 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:32:35.999 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.435+0000 7f156e54e700 1 -- 172.21.15.16:0/3333245743 >> 172.21.15.16:0/3333245743 conn(0x7f15681009d0 msgr2=0x7f15681014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:32:35.999 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.435+0000 7f156e54e700 1 -- 172.21.15.16:0/3333245743 shutdown_connections 2024-04-02T04:32:35.999 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:35.435+0000 7f156e54e700 1 -- 172.21.15.16:0/3333245743 wait complete. 2024-04-02T04:32:35.999 INFO:teuthology.orchestra.run.smithi016.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-02T04:32:36.013 INFO:teuthology.orchestra.run.smithi016.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-02T04:32:36.013 INFO:teuthology.orchestra.run.smithi016.stdout:Creating mgr... 2024-04-02T04:32:36.014 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-02T04:32:36.014 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-02T04:32:36.014 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying port 0.0.0.0:8443 ... 2024-04-02T04:32:36.276 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@mgr.smithi016.eqnebx 2024-04-02T04:32:36.277 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to reset failed state of unit ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@mgr.smithi016.eqnebx.service: Unit ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@mgr.smithi016.eqnebx.service not loaded. 2024-04-02T04:32:36.424 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4.target.wants/ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@mgr.smithi016.eqnebx.service → /etc/systemd/system/ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@.service. 2024-04-02T04:32:36.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:36 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/3333245743' entity='client.admin' 2024-04-02T04:32:36.926 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-02T04:32:36.926 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-04-02T04:32:36.934 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-02T04:32:36.934 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-04-02T04:32:36.934 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-04-02T04:32:36.934 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to enable service . firewalld.service is not available 2024-04-02T04:32:36.944 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-02T04:32:36.944 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-04-02T04:32:36.952 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-02T04:32:36.952 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-04-02T04:32:36.952 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-04-02T04:32:36.952 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to open ports <[9283, 8765, 8443]>. firewalld.service is not available 2024-04-02T04:32:36.952 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr to start... 2024-04-02T04:32:36.953 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr... 2024-04-02T04:32:37.750 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:37 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/3775539119' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-02T04:32:38.030 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-04-02T04:32:38.030 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-04-02T04:32:38.030 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsid": "dc5403b4-f0a9-11ee-b647-cb9ed24678a4", 2024-04-02T04:32:38.031 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "health": { 2024-04-02T04:32:38.031 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-02T04:32:38.031 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-02T04:32:38.031 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-02T04:32:38.031 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:32:38.031 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-02T04:32:38.031 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-02T04:32:38.031 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 0 2024-04-02T04:32:38.031 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-04-02T04:32:38.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-02T04:32:38.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "smithi016" 2024-04-02T04:32:38.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-04-02T04:32:38.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-02T04:32:38.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-02T04:32:38.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T04:32:38.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-04-02T04:32:38.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-02T04:32:38.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:32:38.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-02T04:32:38.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T04:32:38.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-02T04:32:38.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-02T04:32:38.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-02T04:32:38.041 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-02T04:32:38.041 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-02T04:32:38.041 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-02T04:32:38.041 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:32:38.041 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-02T04:32:38.041 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-02T04:32:38.041 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-02T04:32:38.042 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-02T04:32:38.042 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-02T04:32:38.042 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-02T04:32:38.042 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-02T04:32:38.042 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-02T04:32:38.042 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-02T04:32:38.042 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:32:38.042 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-02T04:32:38.042 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T04:32:38.042 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-02T04:32:38.042 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-02T04:32:38.042 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:32:38.042 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-02T04:32:38.042 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-02T04:32:38.043 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-02T04:32:38.043 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-02T04:32:38.043 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-02T04:32:38.043 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-02T04:32:38.043 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "restful" 2024-04-02T04:32:38.043 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-04-02T04:32:38.043 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T04:32:38.043 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:32:38.043 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-02T04:32:38.043 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T04:32:38.043 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modified": "2024-04-02T04:32:23.015012+0000", 2024-04-02T04:32:38.043 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T04:32:38.043 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:32:38.044 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-02T04:32:38.044 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-04-02T04:32:38.044 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.370+0000 7f653f715700 1 Processor -- start 2024-04-02T04:32:38.044 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.370+0000 7f653f715700 1 -- start start 2024-04-02T04:32:38.044 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.370+0000 7f653f715700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6538106520 0x7f6538106940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:32:38.044 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.370+0000 7f653f715700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6538106f10 con 0x7f6538106520 2024-04-02T04:32:38.044 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.370+0000 7f653d4b1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6538106520 0x7f6538106940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:32:38.044 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.370+0000 7f653d4b1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6538106520 0x7f6538106940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36576/0 (socket says 172.21.15.16:36576) 2024-04-02T04:32:38.044 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.370+0000 7f653d4b1700 1 -- 172.21.15.16:0/995698353 learned_addr learned my addr 172.21.15.16:0/995698353 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:32:38.044 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.371+0000 7f653d4b1700 1 -- 172.21.15.16:0/995698353 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6538107680 con 0x7f6538106520 2024-04-02T04:32:38.044 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.371+0000 7f653d4b1700 1 --2- 172.21.15.16:0/995698353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6538106520 0x7f6538106940 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f65340098a0 tx=0x7f6534009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e3565dc8e91f6b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:32:38.044 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.371+0000 7f652ffff700 1 -- 172.21.15.16:0/995698353 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6534004180 con 0x7f6538106520 2024-04-02T04:32:38.045 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.371+0000 7f652ffff700 1 -- 172.21.15.16:0/995698353 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f65340042e0 con 0x7f6538106520 2024-04-02T04:32:38.045 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.371+0000 7f652ffff700 1 -- 172.21.15.16:0/995698353 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6534011a70 con 0x7f6538106520 2024-04-02T04:32:38.045 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.372+0000 7f653f715700 1 -- 172.21.15.16:0/995698353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6538106520 msgr2=0x7f6538106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:32:38.045 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.372+0000 7f653f715700 1 --2- 172.21.15.16:0/995698353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6538106520 0x7f6538106940 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f65340098a0 tx=0x7f6534009bb0 comp rx=0 tx=0).stop 2024-04-02T04:32:38.045 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.372+0000 7f653f715700 1 -- 172.21.15.16:0/995698353 shutdown_connections 2024-04-02T04:32:38.045 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.372+0000 7f653f715700 1 --2- 172.21.15.16:0/995698353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6538106520 0x7f6538106940 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:32:38.045 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.372+0000 7f653f715700 1 -- 172.21.15.16:0/995698353 >> 172.21.15.16:0/995698353 conn(0x7f6538101b40 msgr2=0x7f6538103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:32:38.045 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.372+0000 7f653f715700 1 -- 172.21.15.16:0/995698353 shutdown_connections 2024-04-02T04:32:38.045 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.372+0000 7f653f715700 1 -- 172.21.15.16:0/995698353 wait complete. 2024-04-02T04:32:38.045 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.372+0000 7f653f715700 1 Processor -- start 2024-04-02T04:32:38.045 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.372+0000 7f653f715700 1 -- start start 2024-04-02T04:32:38.045 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.373+0000 7f653f715700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6538106520 0x7f653807bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:32:38.045 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.373+0000 7f653f715700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653807c040 con 0x7f6538106520 2024-04-02T04:32:38.046 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.373+0000 7f653d4b1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6538106520 0x7f653807bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:32:38.046 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.373+0000 7f653d4b1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6538106520 0x7f653807bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36578/0 (socket says 172.21.15.16:36578) 2024-04-02T04:32:38.046 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.373+0000 7f653d4b1700 1 -- 172.21.15.16:0/3775539119 learned_addr learned my addr 172.21.15.16:0/3775539119 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:32:38.046 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.373+0000 7f653d4b1700 1 -- 172.21.15.16:0/3775539119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6534009550 con 0x7f6538106520 2024-04-02T04:32:38.046 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.373+0000 7f653d4b1700 1 --2- 172.21.15.16:0/3775539119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6538106520 0x7f653807bb00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6534011710 tx=0x7f65340047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:32:38.046 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.373+0000 7f652e7fc700 1 -- 172.21.15.16:0/3775539119 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6534003b10 con 0x7f6538106520 2024-04-02T04:32:38.046 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.373+0000 7f652e7fc700 1 -- 172.21.15.16:0/3775539119 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f653401b8c0 con 0x7f6538106520 2024-04-02T04:32:38.046 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.373+0000 7f653f715700 1 -- 172.21.15.16:0/3775539119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f653807a280 con 0x7f6538106520 2024-04-02T04:32:38.046 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.373+0000 7f652e7fc700 1 -- 172.21.15.16:0/3775539119 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6534019710 con 0x7f6538106520 2024-04-02T04:32:38.046 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.374+0000 7f653f715700 1 -- 172.21.15.16:0/3775539119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f653807a720 con 0x7f6538106520 2024-04-02T04:32:38.046 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.374+0000 7f652e7fc700 1 -- 172.21.15.16:0/3775539119 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6534019be0 con 0x7f6538106520 2024-04-02T04:32:38.046 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.374+0000 7f652e7fc700 1 -- 172.21.15.16:0/3775539119 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f653401a710 con 0x7f6538106520 2024-04-02T04:32:38.046 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.375+0000 7f653f715700 1 -- 172.21.15.16:0/3775539119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65380623e0 con 0x7f6538106520 2024-04-02T04:32:38.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.378+0000 7f652e7fc700 1 -- 172.21.15.16:0/3775539119 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f653401ba30 con 0x7f6538106520 2024-04-02T04:32:38.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.452+0000 7f653f715700 1 -- 172.21.15.16:0/3775539119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f653810a8b0 con 0x7f6538106520 2024-04-02T04:32:38.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.453+0000 7f652e7fc700 1 -- 172.21.15.16:0/3775539119 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f653401bbd0 con 0x7f6538106520 2024-04-02T04:32:38.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.454+0000 7f653f715700 1 -- 172.21.15.16:0/3775539119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6538106520 msgr2=0x7f653807bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:32:38.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.454+0000 7f653f715700 1 --2- 172.21.15.16:0/3775539119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6538106520 0x7f653807bb00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6534011710 tx=0x7f65340047c0 comp rx=0 tx=0).stop 2024-04-02T04:32:38.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.454+0000 7f653f715700 1 -- 172.21.15.16:0/3775539119 shutdown_connections 2024-04-02T04:32:38.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.454+0000 7f653f715700 1 --2- 172.21.15.16:0/3775539119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6538106520 0x7f653807bb00 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:32:38.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.454+0000 7f653f715700 1 -- 172.21.15.16:0/3775539119 >> 172.21.15.16:0/3775539119 conn(0x7f6538101b40 msgr2=0x7f6538102610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:32:38.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.455+0000 7f653f715700 1 -- 172.21.15.16:0/3775539119 shutdown_connections 2024-04-02T04:32:38.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:37.455+0000 7f653f715700 1 -- 172.21.15.16:0/3775539119 wait complete. 2024-04-02T04:32:38.047 INFO:teuthology.orchestra.run.smithi016.stdout:mgr not available, waiting (1/15)... 2024-04-02T04:32:42.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:42 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/3417029196' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-02T04:32:43.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-04-02T04:32:43.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-04-02T04:32:43.298 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsid": "dc5403b4-f0a9-11ee-b647-cb9ed24678a4", 2024-04-02T04:32:43.299 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "health": { 2024-04-02T04:32:43.299 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-02T04:32:43.299 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-02T04:32:43.299 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-02T04:32:43.299 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:32:43.299 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-02T04:32:43.299 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-02T04:32:43.299 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 0 2024-04-02T04:32:43.299 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-04-02T04:32:43.299 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-02T04:32:43.299 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "smithi016" 2024-04-02T04:32:43.300 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-04-02T04:32:43.300 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-04-02T04:32:43.300 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-02T04:32:43.300 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T04:32:43.300 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-04-02T04:32:43.300 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-02T04:32:43.300 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:32:43.300 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-02T04:32:43.300 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T04:32:43.300 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-02T04:32:43.300 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-02T04:32:43.300 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-02T04:32:43.300 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-02T04:32:43.300 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-02T04:32:43.301 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-02T04:32:43.301 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:32:43.301 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-02T04:32:43.301 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-02T04:32:43.301 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-02T04:32:43.301 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-02T04:32:43.301 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-02T04:32:43.301 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-02T04:32:43.301 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-02T04:32:43.301 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-02T04:32:43.301 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-02T04:32:43.301 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:32:43.301 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-02T04:32:43.301 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T04:32:43.301 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-02T04:32:43.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-02T04:32:43.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:32:43.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-02T04:32:43.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-02T04:32:43.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-02T04:32:43.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-02T04:32:43.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-02T04:32:43.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-02T04:32:43.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "restful" 2024-04-02T04:32:43.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-04-02T04:32:43.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T04:32:43.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:32:43.302 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-02T04:32:43.303 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T04:32:43.303 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modified": "2024-04-02T04:32:23.015012+0000", 2024-04-02T04:32:43.303 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T04:32:43.303 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:32:43.303 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-02T04:32:43.303 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-04-02T04:32:43.303 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.104+0000 7f828f0bf700 1 Processor -- start 2024-04-02T04:32:43.303 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.104+0000 7f828f0bf700 1 -- start start 2024-04-02T04:32:43.303 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.104+0000 7f828f0bf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8288104db0 0x7f82881051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:32:43.303 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.104+0000 7f828f0bf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8288079e10 con 0x7f8288104db0 2024-04-02T04:32:43.304 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.104+0000 7f828ce5b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8288104db0 0x7f82881051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:32:43.304 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.104+0000 7f828ce5b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8288104db0 0x7f82881051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52346/0 (socket says 172.21.15.16:52346) 2024-04-02T04:32:43.304 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.104+0000 7f828ce5b700 1 -- 172.21.15.16:0/3969897512 learned_addr learned my addr 172.21.15.16:0/3969897512 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:32:43.304 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.104+0000 7f828ce5b700 1 -- 172.21.15.16:0/3969897512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8288105710 con 0x7f8288104db0 2024-04-02T04:32:43.304 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.105+0000 7f828ce5b700 1 --2- 172.21.15.16:0/3969897512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8288104db0 0x7f82881051d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f827c0098a0 tx=0x7f827c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b964fb5247af56 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:32:43.304 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.105+0000 7f82877fe700 1 -- 172.21.15.16:0/3969897512 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f827c004180 con 0x7f8288104db0 2024-04-02T04:32:43.304 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.105+0000 7f82877fe700 1 -- 172.21.15.16:0/3969897512 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f827c0042e0 con 0x7f8288104db0 2024-04-02T04:32:43.304 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.105+0000 7f82877fe700 1 -- 172.21.15.16:0/3969897512 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f827c011a70 con 0x7f8288104db0 2024-04-02T04:32:43.304 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.106+0000 7f828f0bf700 1 -- 172.21.15.16:0/3969897512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8288104db0 msgr2=0x7f82881051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:32:43.304 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.106+0000 7f828f0bf700 1 --2- 172.21.15.16:0/3969897512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8288104db0 0x7f82881051d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f827c0098a0 tx=0x7f827c009bb0 comp rx=0 tx=0).stop 2024-04-02T04:32:43.304 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.106+0000 7f828f0bf700 1 -- 172.21.15.16:0/3969897512 shutdown_connections 2024-04-02T04:32:43.304 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.106+0000 7f828f0bf700 1 --2- 172.21.15.16:0/3969897512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8288104db0 0x7f82881051d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:32:43.304 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.106+0000 7f828f0bf700 1 -- 172.21.15.16:0/3969897512 >> 172.21.15.16:0/3969897512 conn(0x7f8288100950 msgr2=0x7f8288102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:32:43.304 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.106+0000 7f828f0bf700 1 -- 172.21.15.16:0/3969897512 shutdown_connections 2024-04-02T04:32:43.305 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.106+0000 7f828f0bf700 1 -- 172.21.15.16:0/3969897512 wait complete. 2024-04-02T04:32:43.305 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.106+0000 7f828f0bf700 1 Processor -- start 2024-04-02T04:32:43.305 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.106+0000 7f828f0bf700 1 -- start start 2024-04-02T04:32:43.305 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.107+0000 7f828f0bf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8288104db0 0x7f828819c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:32:43.305 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.107+0000 7f828f0bf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828819c9a0 con 0x7f8288104db0 2024-04-02T04:32:43.305 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.107+0000 7f828ce5b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8288104db0 0x7f828819c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:32:43.305 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.107+0000 7f828ce5b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8288104db0 0x7f828819c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52358/0 (socket says 172.21.15.16:52358) 2024-04-02T04:32:43.305 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.107+0000 7f828ce5b700 1 -- 172.21.15.16:0/3417029196 learned_addr learned my addr 172.21.15.16:0/3417029196 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:32:43.305 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.107+0000 7f828ce5b700 1 -- 172.21.15.16:0/3417029196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f827c009550 con 0x7f8288104db0 2024-04-02T04:32:43.305 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.107+0000 7f828ce5b700 1 --2- 172.21.15.16:0/3417029196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8288104db0 0x7f828819c460 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f827c011a20 tx=0x7f827c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:32:43.305 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.107+0000 7f8285ffb700 1 -- 172.21.15.16:0/3417029196 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f827c003b10 con 0x7f8288104db0 2024-04-02T04:32:43.305 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.107+0000 7f828f0bf700 1 -- 172.21.15.16:0/3417029196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f828819cba0 con 0x7f8288104db0 2024-04-02T04:32:43.305 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.107+0000 7f8285ffb700 1 -- 172.21.15.16:0/3417029196 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f827c01b8c0 con 0x7f8288104db0 2024-04-02T04:32:43.305 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.107+0000 7f8285ffb700 1 -- 172.21.15.16:0/3417029196 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f827c0237b0 con 0x7f8288104db0 2024-04-02T04:32:43.306 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.108+0000 7f828f0bf700 1 -- 172.21.15.16:0/3417029196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f828819d040 con 0x7f8288104db0 2024-04-02T04:32:43.306 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.108+0000 7f8285ffb700 1 -- 172.21.15.16:0/3417029196 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f827c023d30 con 0x7f8288104db0 2024-04-02T04:32:43.306 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.108+0000 7f8285ffb700 1 -- 172.21.15.16:0/3417029196 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f827c019710 con 0x7f8288104db0 2024-04-02T04:32:43.306 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.109+0000 7f828f0bf700 1 -- 172.21.15.16:0/3417029196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8288196130 con 0x7f8288104db0 2024-04-02T04:32:43.306 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.113+0000 7f8285ffb700 1 -- 172.21.15.16:0/3417029196 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f827c01ba30 con 0x7f8288104db0 2024-04-02T04:32:43.306 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.185+0000 7f828f0bf700 1 -- 172.21.15.16:0/3417029196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f8288106530 con 0x7f8288104db0 2024-04-02T04:32:43.306 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.185+0000 7f8285ffb700 1 -- 172.21.15.16:0/3417029196 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f827c01bbd0 con 0x7f8288104db0 2024-04-02T04:32:43.306 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.187+0000 7f828f0bf700 1 -- 172.21.15.16:0/3417029196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8288104db0 msgr2=0x7f828819c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:32:43.306 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.187+0000 7f828f0bf700 1 --2- 172.21.15.16:0/3417029196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8288104db0 0x7f828819c460 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f827c011a20 tx=0x7f827c0047c0 comp rx=0 tx=0).stop 2024-04-02T04:32:43.306 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.187+0000 7f828f0bf700 1 -- 172.21.15.16:0/3417029196 shutdown_connections 2024-04-02T04:32:43.306 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.187+0000 7f828f0bf700 1 --2- 172.21.15.16:0/3417029196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8288104db0 0x7f828819c460 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:32:43.306 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.187+0000 7f828f0bf700 1 -- 172.21.15.16:0/3417029196 >> 172.21.15.16:0/3417029196 conn(0x7f8288100950 msgr2=0x7f8288193270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:32:43.306 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.187+0000 7f828f0bf700 1 -- 172.21.15.16:0/3417029196 shutdown_connections 2024-04-02T04:32:43.306 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:42.187+0000 7f828f0bf700 1 -- 172.21.15.16:0/3417029196 wait complete. 2024-04-02T04:32:43.307 INFO:teuthology.orchestra.run.smithi016.stdout:mgr not available, waiting (2/15)... 2024-04-02T04:32:45.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:44 smithi016 ceph-mon[28969]: Activating manager daemon smithi016.eqnebx 2024-04-02T04:32:45.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:44 smithi016 ceph-mon[28969]: mgrmap e2: smithi016.eqnebx(active, starting, since 0.00301421s) 2024-04-02T04:32:45.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:44 smithi016 ceph-mon[28969]: from='mgr.14100 172.21.15.16:0/3486446819' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T04:32:45.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:44 smithi016 ceph-mon[28969]: from='mgr.14100 172.21.15.16:0/3486446819' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T04:32:45.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:44 smithi016 ceph-mon[28969]: from='mgr.14100 172.21.15.16:0/3486446819' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T04:32:45.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:44 smithi016 ceph-mon[28969]: from='mgr.14100 172.21.15.16:0/3486446819' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi016"}]: dispatch 2024-04-02T04:32:45.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:44 smithi016 ceph-mon[28969]: from='mgr.14100 172.21.15.16:0/3486446819' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mgr metadata", "who": "smithi016.eqnebx", "id": "smithi016.eqnebx"}]: dispatch 2024-04-02T04:32:45.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:44 smithi016 ceph-mon[28969]: Manager daemon smithi016.eqnebx is now available 2024-04-02T04:32:45.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:44 smithi016 ceph-mon[28969]: from='mgr.14100 172.21.15.16:0/3486446819' entity='mgr.smithi016.eqnebx' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi016.eqnebx/mirror_snapshot_schedule"}]: dispatch 2024-04-02T04:32:45.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:44 smithi016 ceph-mon[28969]: from='mgr.14100 172.21.15.16:0/3486446819' entity='mgr.smithi016.eqnebx' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi016.eqnebx/trash_purge_schedule"}]: dispatch 2024-04-02T04:32:45.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:44 smithi016 ceph-mon[28969]: from='mgr.14100 172.21.15.16:0/3486446819' entity='mgr.smithi016.eqnebx' 2024-04-02T04:32:45.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:44 smithi016 ceph-mon[28969]: from='mgr.14100 172.21.15.16:0/3486446819' entity='mgr.smithi016.eqnebx' 2024-04-02T04:32:45.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:44 smithi016 ceph-mon[28969]: from='mgr.14100 172.21.15.16:0/3486446819' entity='mgr.smithi016.eqnebx' 2024-04-02T04:32:46.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:45 smithi016 ceph-mon[28969]: mgrmap e3: smithi016.eqnebx(active, since 1.00596s) 2024-04-02T04:32:47.217 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:46 smithi016 ceph-mon[28969]: mgrmap e4: smithi016.eqnebx(active, since 2s) 2024-04-02T04:32:47.217 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:46 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/2307710524' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-02T04:32:47.219 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-04-02T04:32:47.231 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-04-02T04:32:47.231 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsid": "dc5403b4-f0a9-11ee-b647-cb9ed24678a4", 2024-04-02T04:32:47.231 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "health": { 2024-04-02T04:32:47.231 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-02T04:32:47.231 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-02T04:32:47.231 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-02T04:32:47.231 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:32:47.231 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-02T04:32:47.231 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-02T04:32:47.232 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 0 2024-04-02T04:32:47.232 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-04-02T04:32:47.232 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-02T04:32:47.232 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "smithi016" 2024-04-02T04:32:47.232 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-04-02T04:32:47.232 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-04-02T04:32:47.232 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-02T04:32:47.232 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T04:32:47.232 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-04-02T04:32:47.232 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-02T04:32:47.232 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:32:47.232 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-02T04:32:47.232 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T04:32:47.232 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-02T04:32:47.232 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-02T04:32:47.233 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-02T04:32:47.233 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-02T04:32:47.233 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-02T04:32:47.233 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-02T04:32:47.233 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:32:47.233 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-02T04:32:47.233 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-02T04:32:47.233 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-02T04:32:47.233 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-02T04:32:47.233 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-02T04:32:47.233 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-02T04:32:47.233 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-02T04:32:47.233 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-02T04:32:47.233 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-02T04:32:47.234 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:32:47.234 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-02T04:32:47.234 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T04:32:47.234 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-02T04:32:47.234 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-02T04:32:47.234 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:32:47.234 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-02T04:32:47.234 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-02T04:32:47.234 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-02T04:32:47.234 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-02T04:32:47.234 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-02T04:32:47.234 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-02T04:32:47.234 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "restful" 2024-04-02T04:32:47.234 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-04-02T04:32:47.235 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T04:32:47.235 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:32:47.235 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-02T04:32:47.235 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T04:32:47.235 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modified": "2024-04-02T04:32:23.015012+0000", 2024-04-02T04:32:47.235 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T04:32:47.235 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-04-02T04:32:47.235 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-02T04:32:47.235 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-04-02T04:32:47.235 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.717+0000 7fcecce5c700 1 Processor -- start 2024-04-02T04:32:47.235 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.717+0000 7fcecce5c700 1 -- start start 2024-04-02T04:32:47.235 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.717+0000 7fcecce5c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec8106f30 0x7fcec8109320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:32:47.235 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.717+0000 7fcecce5c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec80745f0 con 0x7fcec8106f30 2024-04-02T04:32:47.235 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.718+0000 7fcec659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec8106f30 0x7fcec8109320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:32:47.235 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.718+0000 7fcec659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec8106f30 0x7fcec8109320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52416/0 (socket says 172.21.15.16:52416) 2024-04-02T04:32:47.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.718+0000 7fcec659c700 1 -- 172.21.15.16:0/3063749555 learned_addr learned my addr 172.21.15.16:0/3063749555 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:32:47.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.718+0000 7fcec659c700 1 -- 172.21.15.16:0/3063749555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec8109860 con 0x7fcec8106f30 2024-04-02T04:32:47.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.718+0000 7fcec659c700 1 --2- 172.21.15.16:0/3063749555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec8106f30 0x7fcec8109320 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcec0009e80 tx=0x7fcec000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d45f4e2a291b85 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:32:47.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.718+0000 7fcec559a700 1 -- 172.21.15.16:0/3063749555 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcec000e070 con 0x7fcec8106f30 2024-04-02T04:32:47.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.718+0000 7fcec559a700 1 -- 172.21.15.16:0/3063749555 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fcec000b8c0 con 0x7fcec8106f30 2024-04-02T04:32:47.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.718+0000 7fcec559a700 1 -- 172.21.15.16:0/3063749555 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcec0004440 con 0x7fcec8106f30 2024-04-02T04:32:47.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.719+0000 7fcecce5c700 1 -- 172.21.15.16:0/3063749555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec8106f30 msgr2=0x7fcec8109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:32:47.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.719+0000 7fcecce5c700 1 --2- 172.21.15.16:0/3063749555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec8106f30 0x7fcec8109320 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcec0009e80 tx=0x7fcec000b330 comp rx=0 tx=0).stop 2024-04-02T04:32:47.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.719+0000 7fcecce5c700 1 -- 172.21.15.16:0/3063749555 shutdown_connections 2024-04-02T04:32:47.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.719+0000 7fcecce5c700 1 --2- 172.21.15.16:0/3063749555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec8106f30 0x7fcec8109320 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:32:47.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.719+0000 7fcecce5c700 1 -- 172.21.15.16:0/3063749555 >> 172.21.15.16:0/3063749555 conn(0x7fcec81009c0 msgr2=0x7fcec8102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:32:47.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.719+0000 7fcecce5c700 1 -- 172.21.15.16:0/3063749555 shutdown_connections 2024-04-02T04:32:47.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.719+0000 7fcecce5c700 1 -- 172.21.15.16:0/3063749555 wait complete. 2024-04-02T04:32:47.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.720+0000 7fcecce5c700 1 Processor -- start 2024-04-02T04:32:47.237 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.720+0000 7fcecce5c700 1 -- start start 2024-04-02T04:32:47.237 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.720+0000 7fcecce5c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec8106f30 0x7fcec8106760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:32:47.237 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.720+0000 7fcecce5c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec8106ca0 con 0x7fcec8106f30 2024-04-02T04:32:47.237 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.720+0000 7fcec659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec8106f30 0x7fcec8106760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:32:47.237 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.720+0000 7fcec659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec8106f30 0x7fcec8106760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52418/0 (socket says 172.21.15.16:52418) 2024-04-02T04:32:47.237 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.720+0000 7fcec659c700 1 -- 172.21.15.16:0/2307710524 learned_addr learned my addr 172.21.15.16:0/2307710524 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:32:47.237 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.720+0000 7fcec659c700 1 -- 172.21.15.16:0/2307710524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec0009b30 con 0x7fcec8106f30 2024-04-02T04:32:47.237 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.720+0000 7fcec659c700 1 --2- 172.21.15.16:0/2307710524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec8106f30 0x7fcec8106760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcec00099e0 tx=0x7fcec0003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:32:47.237 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.721+0000 7fcebf7fe700 1 -- 172.21.15.16:0/2307710524 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcec000e040 con 0x7fcec8106f30 2024-04-02T04:32:47.237 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.721+0000 7fcebf7fe700 1 -- 172.21.15.16:0/2307710524 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fcec000bd50 con 0x7fcec8106f30 2024-04-02T04:32:47.237 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.721+0000 7fcebf7fe700 1 -- 172.21.15.16:0/2307710524 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcec00125c0 con 0x7fcec8106f30 2024-04-02T04:32:47.237 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.721+0000 7fcecce5c700 1 -- 172.21.15.16:0/2307710524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec8104ee0 con 0x7fcec8106f30 2024-04-02T04:32:47.237 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.721+0000 7fcecce5c700 1 -- 172.21.15.16:0/2307710524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec81053e0 con 0x7fcec8106f30 2024-04-02T04:32:47.237 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.722+0000 7fcebf7fe700 1 -- 172.21.15.16:0/2307710524 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 3) v1 ==== 45431+0+0 (secure 0 0 0) 0x7fcec0019040 con 0x7fcec8106f30 2024-04-02T04:32:47.238 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.722+0000 7fcebf7fe700 1 --2- 172.21.15.16:0/2307710524 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7fceb4038570 0x7fceb403aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:32:47.238 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.722+0000 7fcecce5c700 1 -- 172.21.15.16:0/2307710524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcec810a5e0 con 0x7fcec8106f30 2024-04-02T04:32:47.238 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.722+0000 7fcebf7fe700 1 -- 172.21.15.16:0/2307710524 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fcec004bdf0 con 0x7fcec8106f30 2024-04-02T04:32:47.238 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.722+0000 7fcec5d9b700 1 --2- 172.21.15.16:0/2307710524 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7fceb4038570 0x7fceb403aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:32:47.238 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.722+0000 7fcec5d9b700 1 --2- 172.21.15.16:0/2307710524 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7fceb4038570 0x7fceb403aa30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fceb0006fd0 tx=0x7fceb0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:32:47.238 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.727+0000 7fcebf7fe700 1 -- 172.21.15.16:0/2307710524 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fcec0011750 con 0x7fcec8106f30 2024-04-02T04:32:47.238 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:45.846+0000 7fcebf7fe700 1 -- 172.21.15.16:0/2307710524 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 4) v1 ==== 45537+0+0 (secure 0 0 0) 0x7fcec0024400 con 0x7fcec8106f30 2024-04-02T04:32:47.238 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:46.016+0000 7fcecce5c700 1 -- 172.21.15.16:0/2307710524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fcec810a820 con 0x7fcec8106f30 2024-04-02T04:32:47.238 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:46.017+0000 7fcebf7fe700 1 -- 172.21.15.16:0/2307710524 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7fcec0011750 con 0x7fcec8106f30 2024-04-02T04:32:47.238 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:46.018+0000 7fcecce5c700 1 -- 172.21.15.16:0/2307710524 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7fceb4038570 msgr2=0x7fceb403aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:32:47.238 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:46.018+0000 7fcecce5c700 1 --2- 172.21.15.16:0/2307710524 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7fceb4038570 0x7fceb403aa30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fceb0006fd0 tx=0x7fceb0006e40 comp rx=0 tx=0).stop 2024-04-02T04:32:47.239 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:46.018+0000 7fcecce5c700 1 -- 172.21.15.16:0/2307710524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec8106f30 msgr2=0x7fcec8106760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:32:47.239 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:46.018+0000 7fcecce5c700 1 --2- 172.21.15.16:0/2307710524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec8106f30 0x7fcec8106760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcec00099e0 tx=0x7fcec0003d00 comp rx=0 tx=0).stop 2024-04-02T04:32:47.239 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:46.018+0000 7fcecce5c700 1 -- 172.21.15.16:0/2307710524 shutdown_connections 2024-04-02T04:32:47.239 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:46.018+0000 7fcecce5c700 1 --2- 172.21.15.16:0/2307710524 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7fceb4038570 0x7fceb403aa30 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:32:47.239 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:46.018+0000 7fcecce5c700 1 --2- 172.21.15.16:0/2307710524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec8106f30 0x7fcec8106760 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:32:47.239 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:46.019+0000 7fcecce5c700 1 -- 172.21.15.16:0/2307710524 >> 172.21.15.16:0/2307710524 conn(0x7fcec81009c0 msgr2=0x7fcec8101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:32:47.239 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:46.019+0000 7fcecce5c700 1 -- 172.21.15.16:0/2307710524 shutdown_connections 2024-04-02T04:32:47.239 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:46.019+0000 7fcecce5c700 1 -- 172.21.15.16:0/2307710524 wait complete. 2024-04-02T04:32:47.239 INFO:teuthology.orchestra.run.smithi016.stdout:mgr is available 2024-04-02T04:32:48.148 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:47 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/2433245518' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-02T04:32:48.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-04-02T04:32:48.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [global] 2024-04-02T04:32:48.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout fsid = dc5403b4-f0a9-11ee-b647-cb9ed24678a4 2024-04-02T04:32:48.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-02T04:32:48.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-02T04:32:48.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-02T04:32:48.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-02T04:32:48.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-04-02T04:32:48.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-02T04:32:48.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-02T04:32:48.383 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-04-02T04:32:48.384 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [osd] 2024-04-02T04:32:48.384 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-02T04:32:48.384 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-02T04:32:48.384 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-02T04:32:48.384 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-02T04:32:48.384 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-02T04:32:48.384 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.632+0000 7ff427fb4700 1 Processor -- start 2024-04-02T04:32:48.384 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.633+0000 7ff427fb4700 1 -- start start 2024-04-02T04:32:48.384 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.633+0000 7ff427fb4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff420104830 0x7ff420104c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:32:48.384 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.633+0000 7ff427fb4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff420105220 con 0x7ff420104830 2024-04-02T04:32:48.384 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.633+0000 7ff425d50700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff420104830 0x7ff420104c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:32:48.384 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.633+0000 7ff425d50700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff420104830 0x7ff420104c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52422/0 (socket says 172.21.15.16:52422) 2024-04-02T04:32:48.384 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.633+0000 7ff425d50700 1 -- 172.21.15.16:0/3964130330 learned_addr learned my addr 172.21.15.16:0/3964130330 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:32:48.384 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.634+0000 7ff425d50700 1 -- 172.21.15.16:0/3964130330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4201077c0 con 0x7ff420104830 2024-04-02T04:32:48.385 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.634+0000 7ff425d50700 1 --2- 172.21.15.16:0/3964130330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff420104830 0x7ff420104c50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff41c0098a0 tx=0x7ff41c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df70851e2aa88c00 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:32:48.385 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.634+0000 7ff424d4e700 1 -- 172.21.15.16:0/3964130330 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff41c004180 con 0x7ff420104830 2024-04-02T04:32:48.385 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.634+0000 7ff424d4e700 1 -- 172.21.15.16:0/3964130330 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7ff41c0042e0 con 0x7ff420104830 2024-04-02T04:32:48.385 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.634+0000 7ff424d4e700 1 -- 172.21.15.16:0/3964130330 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff41c011a70 con 0x7ff420104830 2024-04-02T04:32:48.385 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.634+0000 7ff427fb4700 1 -- 172.21.15.16:0/3964130330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff420104830 msgr2=0x7ff420104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:32:48.385 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.634+0000 7ff427fb4700 1 --2- 172.21.15.16:0/3964130330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff420104830 0x7ff420104c50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff41c0098a0 tx=0x7ff41c009bb0 comp rx=0 tx=0).stop 2024-04-02T04:32:48.385 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.634+0000 7ff427fb4700 1 -- 172.21.15.16:0/3964130330 shutdown_connections 2024-04-02T04:32:48.385 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.634+0000 7ff427fb4700 1 --2- 172.21.15.16:0/3964130330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff420104830 0x7ff420104c50 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:32:48.385 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.634+0000 7ff427fb4700 1 -- 172.21.15.16:0/3964130330 >> 172.21.15.16:0/3964130330 conn(0x7ff4201001a0 msgr2=0x7ff4201025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:32:48.385 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.635+0000 7ff427fb4700 1 -- 172.21.15.16:0/3964130330 shutdown_connections 2024-04-02T04:32:48.385 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.635+0000 7ff427fb4700 1 -- 172.21.15.16:0/3964130330 wait complete. 2024-04-02T04:32:48.385 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.635+0000 7ff427fb4700 1 Processor -- start 2024-04-02T04:32:48.385 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.635+0000 7ff427fb4700 1 -- start start 2024-04-02T04:32:48.385 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.635+0000 7ff427fb4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff420104830 0x7ff420106ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:32:48.386 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.635+0000 7ff427fb4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff420107530 con 0x7ff420104830 2024-04-02T04:32:48.386 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.635+0000 7ff425d50700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff420104830 0x7ff420106ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:32:48.386 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.636+0000 7ff425d50700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff420104830 0x7ff420106ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52436/0 (socket says 172.21.15.16:52436) 2024-04-02T04:32:48.386 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.636+0000 7ff425d50700 1 -- 172.21.15.16:0/2433245518 learned_addr learned my addr 172.21.15.16:0/2433245518 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:32:48.386 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.636+0000 7ff425d50700 1 -- 172.21.15.16:0/2433245518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff41c009550 con 0x7ff420104830 2024-04-02T04:32:48.386 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.636+0000 7ff425d50700 1 --2- 172.21.15.16:0/2433245518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff420104830 0x7ff420106ff0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff41c011710 tx=0x7ff41c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:32:48.386 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.636+0000 7ff412ffd700 1 -- 172.21.15.16:0/2433245518 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff41c003b10 con 0x7ff420104830 2024-04-02T04:32:48.386 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.636+0000 7ff412ffd700 1 -- 172.21.15.16:0/2433245518 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7ff41c01b8c0 con 0x7ff420104830 2024-04-02T04:32:48.386 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.636+0000 7ff427fb4700 1 -- 172.21.15.16:0/2433245518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff420105770 con 0x7ff420104830 2024-04-02T04:32:48.386 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.636+0000 7ff412ffd700 1 -- 172.21.15.16:0/2433245518 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff41c019710 con 0x7ff420104830 2024-04-02T04:32:48.386 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.636+0000 7ff427fb4700 1 -- 172.21.15.16:0/2433245518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff420105c10 con 0x7ff420104830 2024-04-02T04:32:48.386 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.637+0000 7ff412ffd700 1 -- 172.21.15.16:0/2433245518 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 45537+0+0 (secure 0 0 0) 0x7ff41c019930 con 0x7ff420104830 2024-04-02T04:32:48.386 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.637+0000 7ff412ffd700 1 --2- 172.21.15.16:0/2433245518 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7ff40c038690 0x7ff40c03ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:32:48.386 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.637+0000 7ff427fb4700 1 -- 172.21.15.16:0/2433245518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4200623e0 con 0x7ff420104830 2024-04-02T04:32:48.387 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.637+0000 7ff412ffd700 1 -- 172.21.15.16:0/2433245518 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff41c04cbf0 con 0x7ff420104830 2024-04-02T04:32:48.387 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.637+0000 7ff42554f700 1 --2- 172.21.15.16:0/2433245518 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7ff40c038690 0x7ff40c03ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:32:48.387 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.638+0000 7ff42554f700 1 --2- 172.21.15.16:0/2433245518 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7ff40c038690 0x7ff40c03ab50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff414006fd0 tx=0x7ff414006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:32:48.387 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.642+0000 7ff412ffd700 1 -- 172.21.15.16:0/2433245518 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff41c0189a0 con 0x7ff420104830 2024-04-02T04:32:48.387 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.844+0000 7ff427fb4700 1 -- 172.21.15.16:0/2433245518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7ff420105f40 con 0x7ff420104830 2024-04-02T04:32:48.387 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.845+0000 7ff412ffd700 1 -- 172.21.15.16:0/2433245518 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7ff41c018350 con 0x7ff420104830 2024-04-02T04:32:48.387 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.848+0000 7ff427fb4700 1 -- 172.21.15.16:0/2433245518 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7ff40c038690 msgr2=0x7ff40c03ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:32:48.387 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.848+0000 7ff427fb4700 1 --2- 172.21.15.16:0/2433245518 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7ff40c038690 0x7ff40c03ab50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff414006fd0 tx=0x7ff414006e40 comp rx=0 tx=0).stop 2024-04-02T04:32:48.387 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.848+0000 7ff427fb4700 1 -- 172.21.15.16:0/2433245518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff420104830 msgr2=0x7ff420106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:32:48.387 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.848+0000 7ff427fb4700 1 --2- 172.21.15.16:0/2433245518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff420104830 0x7ff420106ff0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff41c011710 tx=0x7ff41c0047c0 comp rx=0 tx=0).stop 2024-04-02T04:32:48.387 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.848+0000 7ff427fb4700 1 -- 172.21.15.16:0/2433245518 shutdown_connections 2024-04-02T04:32:48.387 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.848+0000 7ff427fb4700 1 --2- 172.21.15.16:0/2433245518 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7ff40c038690 0x7ff40c03ab50 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:32:48.387 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.848+0000 7ff427fb4700 1 --2- 172.21.15.16:0/2433245518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff420104830 0x7ff420106ff0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:32:48.387 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.848+0000 7ff427fb4700 1 -- 172.21.15.16:0/2433245518 >> 172.21.15.16:0/2433245518 conn(0x7ff4201001a0 msgr2=0x7ff420100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:32:48.388 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.848+0000 7ff427fb4700 1 -- 172.21.15.16:0/2433245518 shutdown_connections 2024-04-02T04:32:48.388 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:47.848+0000 7ff427fb4700 1 -- 172.21.15.16:0/2433245518 wait complete. 2024-04-02T04:32:48.388 INFO:teuthology.orchestra.run.smithi016.stdout:Enabling cephadm module... 2024-04-02T04:32:49.912 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:49 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/431728401' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-02T04:32:50.179 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.785+0000 7fd94197b700 1 Processor -- start 2024-04-02T04:32:50.179 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.785+0000 7fd94197b700 1 -- start start 2024-04-02T04:32:50.179 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.786+0000 7fd94197b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd93c104b90 0x7fd93c104fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:32:50.179 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.786+0000 7fd94197b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93c105580 con 0x7fd93c104b90 2024-04-02T04:32:50.179 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.786+0000 7fd93affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd93c104b90 0x7fd93c104fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:32:50.180 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.786+0000 7fd93affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd93c104b90 0x7fd93c104fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52442/0 (socket says 172.21.15.16:52442) 2024-04-02T04:32:50.180 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.786+0000 7fd93affd700 1 -- 172.21.15.16:0/2345169423 learned_addr learned my addr 172.21.15.16:0/2345169423 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:32:50.180 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.786+0000 7fd93affd700 1 -- 172.21.15.16:0/2345169423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd93c105d90 con 0x7fd93c104b90 2024-04-02T04:32:50.180 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.786+0000 7fd93affd700 1 --2- 172.21.15.16:0/2345169423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd93c104b90 0x7fd93c104fb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd92c00bb40 tx=0x7fd92c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8781d7c69cf85d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:32:50.180 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.786+0000 7fd939ffb700 1 -- 172.21.15.16:0/2345169423 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd92c00d9a0 con 0x7fd93c104b90 2024-04-02T04:32:50.180 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.786+0000 7fd939ffb700 1 -- 172.21.15.16:0/2345169423 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fd92c004030 con 0x7fd93c104b90 2024-04-02T04:32:50.180 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.786+0000 7fd939ffb700 1 -- 172.21.15.16:0/2345169423 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd92c003cb0 con 0x7fd93c104b90 2024-04-02T04:32:50.180 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.787+0000 7fd94197b700 1 -- 172.21.15.16:0/2345169423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd93c104b90 msgr2=0x7fd93c104fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:32:50.180 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.787+0000 7fd94197b700 1 --2- 172.21.15.16:0/2345169423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd93c104b90 0x7fd93c104fb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd92c00bb40 tx=0x7fd92c00be50 comp rx=0 tx=0).stop 2024-04-02T04:32:50.180 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.787+0000 7fd94197b700 1 -- 172.21.15.16:0/2345169423 shutdown_connections 2024-04-02T04:32:50.180 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.787+0000 7fd94197b700 1 --2- 172.21.15.16:0/2345169423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd93c104b90 0x7fd93c104fb0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:32:50.180 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.787+0000 7fd94197b700 1 -- 172.21.15.16:0/2345169423 >> 172.21.15.16:0/2345169423 conn(0x7fd93c100170 msgr2=0x7fd93c102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:32:50.180 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.787+0000 7fd94197b700 1 -- 172.21.15.16:0/2345169423 shutdown_connections 2024-04-02T04:32:50.180 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.787+0000 7fd94197b700 1 -- 172.21.15.16:0/2345169423 wait complete. 2024-04-02T04:32:50.181 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.788+0000 7fd94197b700 1 Processor -- start 2024-04-02T04:32:50.181 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.788+0000 7fd94197b700 1 -- start start 2024-04-02T04:32:50.181 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.788+0000 7fd94197b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd93c104b90 0x7fd93c19c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:32:50.181 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.788+0000 7fd94197b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93c19c8e0 con 0x7fd93c104b90 2024-04-02T04:32:50.181 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.788+0000 7fd93affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd93c104b90 0x7fd93c19c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:32:50.181 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.788+0000 7fd93affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd93c104b90 0x7fd93c19c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52450/0 (socket says 172.21.15.16:52450) 2024-04-02T04:32:50.181 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.788+0000 7fd93affd700 1 -- 172.21.15.16:0/431728401 learned_addr learned my addr 172.21.15.16:0/431728401 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:32:50.181 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.788+0000 7fd93affd700 1 -- 172.21.15.16:0/431728401 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd92c00b7f0 con 0x7fd93c104b90 2024-04-02T04:32:50.181 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.788+0000 7fd93affd700 1 --2- 172.21.15.16:0/431728401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd93c104b90 0x7fd93c19c3a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd92c00df40 tx=0x7fd92c004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:32:50.181 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.789+0000 7fd940979700 1 -- 172.21.15.16:0/431728401 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd92c009040 con 0x7fd93c104b90 2024-04-02T04:32:50.181 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.789+0000 7fd940979700 1 -- 172.21.15.16:0/431728401 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fd92c004630 con 0x7fd93c104b90 2024-04-02T04:32:50.181 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.789+0000 7fd94197b700 1 -- 172.21.15.16:0/431728401 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd93c19cae0 con 0x7fd93c104b90 2024-04-02T04:32:50.181 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.789+0000 7fd940979700 1 -- 172.21.15.16:0/431728401 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd92c01f6a0 con 0x7fd93c104b90 2024-04-02T04:32:50.181 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.789+0000 7fd94197b700 1 -- 172.21.15.16:0/431728401 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd93c19cfe0 con 0x7fd93c104b90 2024-04-02T04:32:50.181 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.789+0000 7fd940979700 1 -- 172.21.15.16:0/431728401 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 45537+0+0 (secure 0 0 0) 0x7fd92c015070 con 0x7fd93c104b90 2024-04-02T04:32:50.182 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.790+0000 7fd94197b700 1 -- 172.21.15.16:0/431728401 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd93c191cc0 con 0x7fd93c104b90 2024-04-02T04:32:50.182 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.790+0000 7fd940979700 1 --2- 172.21.15.16:0/431728401 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7fd9240386e0 0x7fd92403aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:32:50.182 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.790+0000 7fd940979700 1 -- 172.21.15.16:0/431728401 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd92c04bac0 con 0x7fd93c104b90 2024-04-02T04:32:50.182 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.790+0000 7fd93a7fc700 1 --2- 172.21.15.16:0/431728401 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7fd9240386e0 0x7fd92403aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:32:50.182 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.790+0000 7fd93a7fc700 1 --2- 172.21.15.16:0/431728401 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7fd9240386e0 0x7fd92403aba0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd930006fd0 tx=0x7fd930006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:32:50.182 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:48.795+0000 7fd940979700 1 -- 172.21.15.16:0/431728401 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd92c01e770 con 0x7fd93c104b90 2024-04-02T04:32:50.182 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:49.050+0000 7fd94197b700 1 -- 172.21.15.16:0/431728401 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fd93c1062b0 con 0x7fd93c104b90 2024-04-02T04:32:50.182 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:49.650+0000 7fd940979700 1 -- 172.21.15.16:0/431728401 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 5) v1 ==== 45548+0+0 (secure 0 0 0) 0x7fd92c01da50 con 0x7fd93c104b90 2024-04-02T04:32:50.182 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:49.651+0000 7fd940979700 1 -- 172.21.15.16:0/431728401 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fd92c01d300 con 0x7fd93c104b90 2024-04-02T04:32:50.182 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:49.654+0000 7fd94197b700 1 -- 172.21.15.16:0/431728401 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7fd9240386e0 msgr2=0x7fd92403aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:32:50.182 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:49.654+0000 7fd94197b700 1 --2- 172.21.15.16:0/431728401 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7fd9240386e0 0x7fd92403aba0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd930006fd0 tx=0x7fd930006e40 comp rx=0 tx=0).stop 2024-04-02T04:32:50.182 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:49.654+0000 7fd94197b700 1 -- 172.21.15.16:0/431728401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd93c104b90 msgr2=0x7fd93c19c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:32:50.182 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:49.654+0000 7fd94197b700 1 --2- 172.21.15.16:0/431728401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd93c104b90 0x7fd93c19c3a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd92c00df40 tx=0x7fd92c004250 comp rx=0 tx=0).stop 2024-04-02T04:32:50.182 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:49.654+0000 7fd94197b700 1 -- 172.21.15.16:0/431728401 shutdown_connections 2024-04-02T04:32:50.183 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:49.654+0000 7fd94197b700 1 --2- 172.21.15.16:0/431728401 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7fd9240386e0 0x7fd92403aba0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:32:50.183 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:49.654+0000 7fd94197b700 1 --2- 172.21.15.16:0/431728401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd93c104b90 0x7fd93c19c3a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:32:50.183 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:49.654+0000 7fd94197b700 1 -- 172.21.15.16:0/431728401 >> 172.21.15.16:0/431728401 conn(0x7fd93c100170 msgr2=0x7fd93c18ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:32:50.183 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:49.654+0000 7fd94197b700 1 -- 172.21.15.16:0/431728401 shutdown_connections 2024-04-02T04:32:50.183 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:49.654+0000 7fd94197b700 1 -- 172.21.15.16:0/431728401 wait complete. 2024-04-02T04:32:50.755 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:50 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/431728401' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-02T04:32:50.755 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:50 smithi016 ceph-mon[28969]: mgrmap e5: smithi016.eqnebx(active, since 5s) 2024-04-02T04:32:51.364 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-04-02T04:32:51.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-02T04:32:51.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-02T04:32:51.370 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "active_name": "smithi016.eqnebx", 2024-04-02T04:32:51.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-02T04:32:51.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-04-02T04:32:51.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.546+0000 7f3f3edb5700 1 Processor -- start 2024-04-02T04:32:51.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.546+0000 7f3f3edb5700 1 -- start start 2024-04-02T04:32:51.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.546+0000 7f3f3edb5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f381053c0 0x7f3f381057e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:32:51.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.546+0000 7f3f3edb5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f38105db0 con 0x7f3f381053c0 2024-04-02T04:32:51.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.546+0000 7f3f3cb51700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f381053c0 0x7f3f381057e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:32:51.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.546+0000 7f3f3cb51700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f381053c0 0x7f3f381057e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52470/0 (socket says 172.21.15.16:52470) 2024-04-02T04:32:51.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.547+0000 7f3f3cb51700 1 -- 172.21.15.16:0/1685667921 learned_addr learned my addr 172.21.15.16:0/1685667921 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:32:51.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.547+0000 7f3f3cb51700 1 -- 172.21.15.16:0/1685667921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f381065c0 con 0x7f3f381053c0 2024-04-02T04:32:51.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.547+0000 7f3f3cb51700 1 --2- 172.21.15.16:0/1685667921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f381053c0 0x7f3f381057e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c0098a0 tx=0x7f3f2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d1b1149e3af1f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:32:51.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.547+0000 7f3f377fe700 1 -- 172.21.15.16:0/1685667921 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3f2c004180 con 0x7f3f381053c0 2024-04-02T04:32:51.371 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.547+0000 7f3f377fe700 1 -- 172.21.15.16:0/1685667921 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f3f2c0042e0 con 0x7f3f381053c0 2024-04-02T04:32:51.372 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.547+0000 7f3f377fe700 1 -- 172.21.15.16:0/1685667921 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3f2c011a70 con 0x7f3f381053c0 2024-04-02T04:32:51.372 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.548+0000 7f3f3edb5700 1 -- 172.21.15.16:0/1685667921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f381053c0 msgr2=0x7f3f381057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:32:51.372 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.548+0000 7f3f3edb5700 1 --2- 172.21.15.16:0/1685667921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f381053c0 0x7f3f381057e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c0098a0 tx=0x7f3f2c009bb0 comp rx=0 tx=0).stop 2024-04-02T04:32:51.372 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.548+0000 7f3f3edb5700 1 -- 172.21.15.16:0/1685667921 shutdown_connections 2024-04-02T04:32:51.372 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.548+0000 7f3f3edb5700 1 --2- 172.21.15.16:0/1685667921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f381053c0 0x7f3f381057e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:32:51.372 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.548+0000 7f3f3edb5700 1 -- 172.21.15.16:0/1685667921 >> 172.21.15.16:0/1685667921 conn(0x7f3f38100980 msgr2=0x7f3f38102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:32:51.372 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.548+0000 7f3f3edb5700 1 -- 172.21.15.16:0/1685667921 shutdown_connections 2024-04-02T04:32:51.372 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.548+0000 7f3f3edb5700 1 -- 172.21.15.16:0/1685667921 wait complete. 2024-04-02T04:32:51.372 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.548+0000 7f3f3edb5700 1 Processor -- start 2024-04-02T04:32:51.372 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.548+0000 7f3f3edb5700 1 -- start start 2024-04-02T04:32:51.373 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.549+0000 7f3f3edb5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f381053c0 0x7f3f3819a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:32:51.373 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.549+0000 7f3f3edb5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3819a760 con 0x7f3f381053c0 2024-04-02T04:32:51.373 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.549+0000 7f3f3cb51700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f381053c0 0x7f3f3819a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:32:51.373 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.549+0000 7f3f3cb51700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f381053c0 0x7f3f3819a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52480/0 (socket says 172.21.15.16:52480) 2024-04-02T04:32:51.373 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.549+0000 7f3f3cb51700 1 -- 172.21.15.16:0/2779318114 learned_addr learned my addr 172.21.15.16:0/2779318114 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:32:51.373 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.549+0000 7f3f3cb51700 1 -- 172.21.15.16:0/2779318114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f2c009550 con 0x7f3f381053c0 2024-04-02T04:32:51.373 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.549+0000 7f3f3cb51700 1 --2- 172.21.15.16:0/2779318114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f381053c0 0x7f3f3819a220 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c011710 tx=0x7f3f2c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:32:51.373 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.549+0000 7f3f35ffb700 1 -- 172.21.15.16:0/2779318114 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3f2c003b10 con 0x7f3f381053c0 2024-04-02T04:32:51.373 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.549+0000 7f3f3edb5700 1 -- 172.21.15.16:0/2779318114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f3819a960 con 0x7f3f381053c0 2024-04-02T04:32:51.373 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.549+0000 7f3f35ffb700 1 -- 172.21.15.16:0/2779318114 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f3f2c01b8c0 con 0x7f3f381053c0 2024-04-02T04:32:51.373 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.549+0000 7f3f35ffb700 1 -- 172.21.15.16:0/2779318114 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3f2c019770 con 0x7f3f381053c0 2024-04-02T04:32:51.374 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.550+0000 7f3f3edb5700 1 -- 172.21.15.16:0/2779318114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f3819ae00 con 0x7f3f381053c0 2024-04-02T04:32:51.374 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.550+0000 7f3f35ffb700 1 -- 172.21.15.16:0/2779318114 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 45548+0+0 (secure 0 0 0) 0x7f3f2c0199f0 con 0x7f3f381053c0 2024-04-02T04:32:51.374 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.551+0000 7f3f3edb5700 1 -- 172.21.15.16:0/2779318114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f381943e0 con 0x7f3f381053c0 2024-04-02T04:32:51.374 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.551+0000 7f3f35ffb700 1 --2- 172.21.15.16:0/2779318114 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7f3f28038720 0x7f3f2803abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:32:51.374 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.551+0000 7f3f35ffb700 1 -- 172.21.15.16:0/2779318114 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3f2c04c920 con 0x7f3f381053c0 2024-04-02T04:32:51.374 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.551+0000 7f3f37fff700 1 -- 172.21.15.16:0/2779318114 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7f3f28038720 msgr2=0x7f3f2803abe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/824107799 2024-04-02T04:32:51.374 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.551+0000 7f3f37fff700 1 --2- 172.21.15.16:0/2779318114 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7f3f28038720 0x7f3f2803abe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T04:32:51.374 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.556+0000 7f3f35ffb700 1 -- 172.21.15.16:0/2779318114 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3f2c018db0 con 0x7f3f381053c0 2024-04-02T04:32:51.374 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.752+0000 7f3f37fff700 1 -- 172.21.15.16:0/2779318114 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7f3f28038720 msgr2=0x7f3f2803abe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/824107799 2024-04-02T04:32:51.374 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.752+0000 7f3f37fff700 1 --2- 172.21.15.16:0/2779318114 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7f3f28038720 0x7f3f2803abe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T04:32:51.375 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.815+0000 7f3f3edb5700 1 -- 172.21.15.16:0/2779318114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f3f381097f0 con 0x7f3f381053c0 2024-04-02T04:32:51.375 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.816+0000 7f3f35ffb700 1 -- 172.21.15.16:0/2779318114 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7f3f2c01f070 con 0x7f3f381053c0 2024-04-02T04:32:51.375 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.819+0000 7f3f3edb5700 1 -- 172.21.15.16:0/2779318114 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7f3f28038720 msgr2=0x7f3f2803abe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T04:32:51.375 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.819+0000 7f3f3edb5700 1 --2- 172.21.15.16:0/2779318114 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7f3f28038720 0x7f3f2803abe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:32:51.375 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.819+0000 7f3f3edb5700 1 -- 172.21.15.16:0/2779318114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f381053c0 msgr2=0x7f3f3819a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:32:51.375 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.819+0000 7f3f3edb5700 1 --2- 172.21.15.16:0/2779318114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f381053c0 0x7f3f3819a220 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c011710 tx=0x7f3f2c0047c0 comp rx=0 tx=0).stop 2024-04-02T04:32:51.375 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.819+0000 7f3f3edb5700 1 -- 172.21.15.16:0/2779318114 shutdown_connections 2024-04-02T04:32:51.375 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.819+0000 7f3f3edb5700 1 --2- 172.21.15.16:0/2779318114 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7f3f28038720 0x7f3f2803abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:32:51.375 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.819+0000 7f3f3edb5700 1 --2- 172.21.15.16:0/2779318114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f381053c0 0x7f3f3819a220 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:32:51.375 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.819+0000 7f3f3edb5700 1 -- 172.21.15.16:0/2779318114 >> 172.21.15.16:0/2779318114 conn(0x7f3f38100980 msgr2=0x7f3f38101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:32:51.375 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.819+0000 7f3f3edb5700 1 -- 172.21.15.16:0/2779318114 shutdown_connections 2024-04-02T04:32:51.375 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:50.819+0000 7f3f3edb5700 1 -- 172.21.15.16:0/2779318114 wait complete. 2024-04-02T04:32:51.376 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for the mgr to restart... 2024-04-02T04:32:51.376 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr epoch 5... 2024-04-02T04:32:51.910 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:51 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/2779318114' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-02T04:32:57.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:56 smithi016 ceph-mon[28969]: Active manager daemon smithi016.eqnebx restarted 2024-04-02T04:32:57.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:56 smithi016 ceph-mon[28969]: Activating manager daemon smithi016.eqnebx 2024-04-02T04:32:57.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:56 smithi016 ceph-mon[28969]: osdmap e2: 0 total, 0 up, 0 in 2024-04-02T04:32:57.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:56 smithi016 ceph-mon[28969]: mgrmap e6: smithi016.eqnebx(active, starting, since 0.00309789s) 2024-04-02T04:32:57.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:56 smithi016 ceph-mon[28969]: from='mgr.14118 172.21.15.16:0/872684120' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi016"}]: dispatch 2024-04-02T04:32:57.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:56 smithi016 ceph-mon[28969]: from='mgr.14118 172.21.15.16:0/872684120' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mgr metadata", "who": "smithi016.eqnebx", "id": "smithi016.eqnebx"}]: dispatch 2024-04-02T04:32:57.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:56 smithi016 ceph-mon[28969]: from='mgr.14118 172.21.15.16:0/872684120' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T04:32:57.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:56 smithi016 ceph-mon[28969]: from='mgr.14118 172.21.15.16:0/872684120' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T04:32:57.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:56 smithi016 ceph-mon[28969]: from='mgr.14118 172.21.15.16:0/872684120' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T04:32:57.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:56 smithi016 ceph-mon[28969]: Manager daemon smithi016.eqnebx is now available 2024-04-02T04:32:57.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:56 smithi016 ceph-mon[28969]: from='mgr.14118 172.21.15.16:0/872684120' entity='mgr.smithi016.eqnebx' 2024-04-02T04:32:57.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:56 smithi016 ceph-mon[28969]: from='mgr.14118 172.21.15.16:0/872684120' entity='mgr.smithi016.eqnebx' 2024-04-02T04:32:57.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:56 smithi016 ceph-mon[28969]: from='mgr.14118 172.21.15.16:0/872684120' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:32:57.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:56 smithi016 ceph-mon[28969]: from='mgr.14118 172.21.15.16:0/872684120' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:32:57.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:56 smithi016 ceph-mon[28969]: from='mgr.14118 172.21.15.16:0/872684120' entity='mgr.smithi016.eqnebx' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi016.eqnebx/mirror_snapshot_schedule"}]: dispatch 2024-04-02T04:32:57.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:56 smithi016 ceph-mon[28969]: from='mgr.14118 172.21.15.16:0/872684120' entity='mgr.smithi016.eqnebx' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi016.eqnebx/trash_purge_schedule"}]: dispatch 2024-04-02T04:32:58.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:57 smithi016 ceph-mon[28969]: Found migration_current of "None". Setting to last migration. 2024-04-02T04:32:58.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:57 smithi016 ceph-mon[28969]: from='mgr.14118 172.21.15.16:0/872684120' entity='mgr.smithi016.eqnebx' 2024-04-02T04:32:58.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:57 smithi016 ceph-mon[28969]: from='mgr.14118 172.21.15.16:0/872684120' entity='mgr.smithi016.eqnebx' 2024-04-02T04:32:58.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:57 smithi016 ceph-mon[28969]: mgrmap e7: smithi016.eqnebx(active, since 1.00502s) 2024-04-02T04:32:58.319 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-04-02T04:32:58.319 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-02T04:32:58.319 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-02T04:32:58.320 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-04-02T04:32:58.320 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:51.845+0000 7fe2b714c700 1 Processor -- start 2024-04-02T04:32:58.320 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:51.845+0000 7fe2b714c700 1 -- start start 2024-04-02T04:32:58.320 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:51.845+0000 7fe2b714c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b007bde0 0x7fe2b007a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:32:58.320 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:51.845+0000 7fe2b714c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b007a820 con 0x7fe2b007bde0 2024-04-02T04:32:58.320 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:51.845+0000 7fe2b4ee8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b007bde0 0x7fe2b007a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:32:58.320 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:51.845+0000 7fe2b4ee8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b007bde0 0x7fe2b007a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60290/0 (socket says 172.21.15.16:60290) 2024-04-02T04:32:58.320 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:51.845+0000 7fe2b4ee8700 1 -- 172.21.15.16:0/3306473933 learned_addr learned my addr 172.21.15.16:0/3306473933 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:32:58.320 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:51.846+0000 7fe2b4ee8700 1 -- 172.21.15.16:0/3306473933 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b007a960 con 0x7fe2b007bde0 2024-04-02T04:32:58.320 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:51.846+0000 7fe2b4ee8700 1 --2- 172.21.15.16:0/3306473933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b007bde0 0x7fe2b007a2e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe2a40098a0 tx=0x7fe2a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15c0180495d196c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:32:58.320 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:51.846+0000 7fe2af7fe700 1 -- 172.21.15.16:0/3306473933 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe2a4004180 con 0x7fe2b007bde0 2024-04-02T04:32:58.320 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:51.846+0000 7fe2af7fe700 1 -- 172.21.15.16:0/3306473933 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fe2a40042e0 con 0x7fe2b007bde0 2024-04-02T04:32:58.320 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:51.846+0000 7fe2af7fe700 1 -- 172.21.15.16:0/3306473933 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe2a4011a70 con 0x7fe2b007bde0 2024-04-02T04:32:58.320 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:51.847+0000 7fe2b714c700 1 -- 172.21.15.16:0/3306473933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b007bde0 msgr2=0x7fe2b007a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:32:58.321 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:51.847+0000 7fe2b714c700 1 --2- 172.21.15.16:0/3306473933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b007bde0 0x7fe2b007a2e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe2a40098a0 tx=0x7fe2a4009bb0 comp rx=0 tx=0).stop 2024-04-02T04:32:58.321 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:51.847+0000 7fe2b714c700 1 -- 172.21.15.16:0/3306473933 shutdown_connections 2024-04-02T04:32:58.321 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:51.847+0000 7fe2b714c700 1 --2- 172.21.15.16:0/3306473933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b007bde0 0x7fe2b007a2e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:32:58.321 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:51.847+0000 7fe2b714c700 1 -- 172.21.15.16:0/3306473933 >> 172.21.15.16:0/3306473933 conn(0x7fe2b0101b10 msgr2=0x7fe2b0103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:32:58.321 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:51.847+0000 7fe2b714c700 1 -- 172.21.15.16:0/3306473933 shutdown_connections 2024-04-02T04:32:58.321 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:51.847+0000 7fe2b714c700 1 -- 172.21.15.16:0/3306473933 wait complete. 2024-04-02T04:32:58.321 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:51.847+0000 7fe2b714c700 1 Processor -- start 2024-04-02T04:32:58.321 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:51.847+0000 7fe2b714c700 1 -- start start 2024-04-02T04:32:58.321 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:51.848+0000 7fe2b714c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b007bde0 0x7fe2b019c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:32:58.321 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:51.848+0000 7fe2b714c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b019c8b0 con 0x7fe2b007bde0 2024-04-02T04:32:58.321 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:51.848+0000 7fe2b4ee8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b007bde0 0x7fe2b019c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:32:58.321 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:51.848+0000 7fe2b4ee8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b007bde0 0x7fe2b019c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60296/0 (socket says 172.21.15.16:60296) 2024-04-02T04:32:58.321 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:51.848+0000 7fe2b4ee8700 1 -- 172.21.15.16:0/3571601358 learned_addr learned my addr 172.21.15.16:0/3571601358 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:32:58.321 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:51.848+0000 7fe2b4ee8700 1 -- 172.21.15.16:0/3571601358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a4009550 con 0x7fe2b007bde0 2024-04-02T04:32:58.321 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:51.848+0000 7fe2b4ee8700 1 --2- 172.21.15.16:0/3571601358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b007bde0 0x7fe2b019c370 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4011710 tx=0x7fe2a40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:32:58.322 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:51.848+0000 7fe2adffb700 1 -- 172.21.15.16:0/3571601358 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe2a4003b10 con 0x7fe2b007bde0 2024-04-02T04:32:58.322 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:51.848+0000 7fe2b714c700 1 -- 172.21.15.16:0/3571601358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b019cab0 con 0x7fe2b007bde0 2024-04-02T04:32:58.322 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:51.848+0000 7fe2adffb700 1 -- 172.21.15.16:0/3571601358 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fe2a401b8c0 con 0x7fe2b007bde0 2024-04-02T04:32:58.322 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:51.848+0000 7fe2adffb700 1 -- 172.21.15.16:0/3571601358 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe2a40197e0 con 0x7fe2b007bde0 2024-04-02T04:32:58.322 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:51.848+0000 7fe2b714c700 1 -- 172.21.15.16:0/3571601358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b019cf50 con 0x7fe2b007bde0 2024-04-02T04:32:58.322 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:51.849+0000 7fe2adffb700 1 -- 172.21.15.16:0/3571601358 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 45548+0+0 (secure 0 0 0) 0x7fe2a4019a20 con 0x7fe2b007bde0 2024-04-02T04:32:58.322 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:51.849+0000 7fe2adffb700 1 --2- 172.21.15.16:0/3571601358 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7fe2a0038720 0x7fe2a003abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:32:58.322 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:51.849+0000 7fe2adffb700 1 -- 172.21.15.16:0/3571601358 --> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe2a003b2f0 con 0x7fe2a0038720 2024-04-02T04:32:58.322 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:51.850+0000 7fe2adffb700 1 -- 172.21.15.16:0/3571601358 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe2a404cab0 con 0x7fe2b007bde0 2024-04-02T04:32:58.322 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:51.850+0000 7fe2affff700 1 -- 172.21.15.16:0/3571601358 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7fe2a0038720 msgr2=0x7fe2a003abe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/824107799 2024-04-02T04:32:58.322 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:51.850+0000 7fe2affff700 1 --2- 172.21.15.16:0/3571601358 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7fe2a0038720 0x7fe2a003abe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T04:32:58.322 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:52.051+0000 7fe2affff700 1 -- 172.21.15.16:0/3571601358 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7fe2a0038720 msgr2=0x7fe2a003abe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/824107799 2024-04-02T04:32:58.322 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:52.051+0000 7fe2affff700 1 --2- 172.21.15.16:0/3571601358 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7fe2a0038720 0x7fe2a003abe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T04:32:58.323 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:52.452+0000 7fe2affff700 1 -- 172.21.15.16:0/3571601358 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7fe2a0038720 msgr2=0x7fe2a003abe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/824107799 2024-04-02T04:32:58.323 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:52.452+0000 7fe2affff700 1 --2- 172.21.15.16:0/3571601358 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7fe2a0038720 0x7fe2a003abe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-02T04:32:58.323 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:53.253+0000 7fe2affff700 1 -- 172.21.15.16:0/3571601358 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7fe2a0038720 msgr2=0x7fe2a003abe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/824107799 2024-04-02T04:32:58.323 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:53.253+0000 7fe2affff700 1 --2- 172.21.15.16:0/3571601358 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7fe2a0038720 0x7fe2a003abe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-02T04:32:58.323 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:54.854+0000 7fe2affff700 1 -- 172.21.15.16:0/3571601358 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7fe2a0038720 msgr2=0x7fe2a003abe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/824107799 2024-04-02T04:32:58.323 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:54.854+0000 7fe2affff700 1 --2- 172.21.15.16:0/3571601358 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7fe2a0038720 0x7fe2a003abe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-02T04:32:58.323 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:56.645+0000 7fe2adffb700 1 -- 172.21.15.16:0/3571601358 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mgrmap(e 6) v1 ==== 45315+0+0 (secure 0 0 0) 0x7fe2a401baa0 con 0x7fe2b007bde0 2024-04-02T04:32:58.323 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:56.645+0000 7fe2adffb700 1 -- 172.21.15.16:0/3571601358 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7fe2a0038720 msgr2=0x7fe2a003abe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T04:32:58.323 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:56.645+0000 7fe2adffb700 1 --2- 172.21.15.16:0/3571601358 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7fe2a0038720 0x7fe2a003abe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:32:58.323 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:57.646+0000 7fe2adffb700 1 -- 172.21.15.16:0/3571601358 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 7) v1 ==== 45442+0+0 (secure 0 0 0) 0x7fe2a404e310 con 0x7fe2b007bde0 2024-04-02T04:32:58.323 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:57.647+0000 7fe2adffb700 1 --2- 172.21.15.16:0/3571601358 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fe2a003bfb0 0x7fe2a003e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:32:58.323 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:57.647+0000 7fe2adffb700 1 -- 172.21.15.16:0/3571601358 --> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe2a003b2f0 con 0x7fe2a003bfb0 2024-04-02T04:32:58.323 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:57.647+0000 7fe2affff700 1 --2- 172.21.15.16:0/3571601358 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fe2a003bfb0 0x7fe2a003e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:32:58.323 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:57.648+0000 7fe2affff700 1 --2- 172.21.15.16:0/3571601358 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fe2a003bfb0 0x7fe2a003e3a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe29c003a10 tx=0x7fe29c009290 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:32:58.323 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:57.648+0000 7fe2adffb700 1 -- 172.21.15.16:0/3571601358 <== mgr.14118 v2:172.21.15.16:6800/873531241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fe2a003b2f0 con 0x7fe2a003bfb0 2024-04-02T04:32:58.324 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:57.655+0000 7fe2b714c700 1 -- 172.21.15.16:0/3571601358 --> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fe2b010c7b0 con 0x7fe2a003bfb0 2024-04-02T04:32:58.324 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:57.655+0000 7fe2adffb700 1 -- 172.21.15.16:0/3571601358 <== mgr.14118 v2:172.21.15.16:6800/873531241 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fe2b010c7b0 con 0x7fe2a003bfb0 2024-04-02T04:32:58.324 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:57.655+0000 7fe2b714c700 1 -- 172.21.15.16:0/3571601358 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fe2a003bfb0 msgr2=0x7fe2a003e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:32:58.324 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:57.655+0000 7fe2b714c700 1 --2- 172.21.15.16:0/3571601358 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fe2a003bfb0 0x7fe2a003e3a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe29c003a10 tx=0x7fe29c009290 comp rx=0 tx=0).stop 2024-04-02T04:32:58.324 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:57.655+0000 7fe2b714c700 1 -- 172.21.15.16:0/3571601358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b007bde0 msgr2=0x7fe2b019c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:32:58.324 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:57.655+0000 7fe2b714c700 1 --2- 172.21.15.16:0/3571601358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b007bde0 0x7fe2b019c370 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4011710 tx=0x7fe2a40047c0 comp rx=0 tx=0).stop 2024-04-02T04:32:58.324 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:57.655+0000 7fe2b714c700 1 -- 172.21.15.16:0/3571601358 shutdown_connections 2024-04-02T04:32:58.324 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:57.655+0000 7fe2b714c700 1 --2- 172.21.15.16:0/3571601358 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fe2a003bfb0 0x7fe2a003e3a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:32:58.324 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:57.655+0000 7fe2b714c700 1 --2- 172.21.15.16:0/3571601358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b007bde0 0x7fe2b019c370 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:32:58.324 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:57.655+0000 7fe2b714c700 1 --2- 172.21.15.16:0/3571601358 >> [v2:172.21.15.16:6800/824107799,v1:172.21.15.16:6801/824107799] conn(0x7fe2a0038720 0x7fe2a003abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:32:58.324 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:57.655+0000 7fe2b714c700 1 -- 172.21.15.16:0/3571601358 >> 172.21.15.16:0/3571601358 conn(0x7fe2b0101b10 msgr2=0x7fe2b0102670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:32:58.324 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:57.655+0000 7fe2b714c700 1 -- 172.21.15.16:0/3571601358 shutdown_connections 2024-04-02T04:32:58.324 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:57.655+0000 7fe2b714c700 1 -- 172.21.15.16:0/3571601358 wait complete. 2024-04-02T04:32:58.324 INFO:teuthology.orchestra.run.smithi016.stdout:mgr epoch 5 is available 2024-04-02T04:32:58.324 INFO:teuthology.orchestra.run.smithi016.stdout:Setting orchestrator backend to cephadm... 2024-04-02T04:32:58.978 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:58 smithi016 ceph-mon[28969]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-02T04:32:58.978 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:58 smithi016 ceph-mon[28969]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-02T04:32:58.978 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:58 smithi016 ceph-mon[28969]: from='mgr.14118 172.21.15.16:0/872684120' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:32:59.516 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.746+0000 7f68702d4700 1 Processor -- start 2024-04-02T04:32:59.516 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.746+0000 7f68702d4700 1 -- start start 2024-04-02T04:32:59.516 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.747+0000 7f68702d4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6868106550 0x7f6868106970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:32:59.516 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.747+0000 7f68702d4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6868106f40 con 0x7f6868106550 2024-04-02T04:32:59.516 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.747+0000 7f686e070700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6868106550 0x7f6868106970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:32:59.517 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.747+0000 7f686e070700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6868106550 0x7f6868106970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60384/0 (socket says 172.21.15.16:60384) 2024-04-02T04:32:59.517 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.747+0000 7f686e070700 1 -- 172.21.15.16:0/2944665118 learned_addr learned my addr 172.21.15.16:0/2944665118 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:32:59.517 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.747+0000 7f686e070700 1 -- 172.21.15.16:0/2944665118 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68681076b0 con 0x7f6868106550 2024-04-02T04:32:59.517 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.747+0000 7f686e070700 1 --2- 172.21.15.16:0/2944665118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6868106550 0x7f6868106970 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f68640098a0 tx=0x7f6864009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecb9902ae26d7263 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:32:59.517 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.747+0000 7f686d06e700 1 -- 172.21.15.16:0/2944665118 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6864004180 con 0x7f6868106550 2024-04-02T04:32:59.517 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.747+0000 7f686d06e700 1 -- 172.21.15.16:0/2944665118 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f68640042e0 con 0x7f6868106550 2024-04-02T04:32:59.517 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.748+0000 7f686d06e700 1 -- 172.21.15.16:0/2944665118 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6864011a70 con 0x7f6868106550 2024-04-02T04:32:59.517 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.748+0000 7f68702d4700 1 -- 172.21.15.16:0/2944665118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6868106550 msgr2=0x7f6868106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:32:59.517 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.748+0000 7f68702d4700 1 --2- 172.21.15.16:0/2944665118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6868106550 0x7f6868106970 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f68640098a0 tx=0x7f6864009bb0 comp rx=0 tx=0).stop 2024-04-02T04:32:59.517 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.748+0000 7f68702d4700 1 -- 172.21.15.16:0/2944665118 shutdown_connections 2024-04-02T04:32:59.517 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.748+0000 7f68702d4700 1 --2- 172.21.15.16:0/2944665118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6868106550 0x7f6868106970 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:32:59.518 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.748+0000 7f68702d4700 1 -- 172.21.15.16:0/2944665118 >> 172.21.15.16:0/2944665118 conn(0x7f6868101b10 msgr2=0x7f6868103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:32:59.518 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.748+0000 7f68702d4700 1 -- 172.21.15.16:0/2944665118 shutdown_connections 2024-04-02T04:32:59.518 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.748+0000 7f68702d4700 1 -- 172.21.15.16:0/2944665118 wait complete. 2024-04-02T04:32:59.518 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.749+0000 7f68702d4700 1 Processor -- start 2024-04-02T04:32:59.518 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.749+0000 7f68702d4700 1 -- start start 2024-04-02T04:32:59.518 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.749+0000 7f68702d4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6868106550 0x7f686819a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:32:59.518 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.749+0000 7f68702d4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686819a780 con 0x7f6868106550 2024-04-02T04:32:59.518 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.749+0000 7f686e070700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6868106550 0x7f686819a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:32:59.518 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.749+0000 7f686e070700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6868106550 0x7f686819a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60392/0 (socket says 172.21.15.16:60392) 2024-04-02T04:32:59.518 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.749+0000 7f686e070700 1 -- 172.21.15.16:0/1474918222 learned_addr learned my addr 172.21.15.16:0/1474918222 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:32:59.518 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.749+0000 7f686e070700 1 -- 172.21.15.16:0/1474918222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6864009550 con 0x7f6868106550 2024-04-02T04:32:59.518 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.749+0000 7f686e070700 1 --2- 172.21.15.16:0/1474918222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6868106550 0x7f686819a240 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6864011710 tx=0x7f68640047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:32:59.518 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.750+0000 7f685f7fe700 1 -- 172.21.15.16:0/1474918222 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6864003b10 con 0x7f6868106550 2024-04-02T04:32:59.518 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.750+0000 7f685f7fe700 1 -- 172.21.15.16:0/1474918222 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f686401b8c0 con 0x7f6868106550 2024-04-02T04:32:59.519 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.750+0000 7f68702d4700 1 -- 172.21.15.16:0/1474918222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f686819a980 con 0x7f6868106550 2024-04-02T04:32:59.519 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.750+0000 7f685f7fe700 1 -- 172.21.15.16:0/1474918222 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6864019710 con 0x7f6868106550 2024-04-02T04:32:59.519 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.750+0000 7f68702d4700 1 -- 172.21.15.16:0/1474918222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f686819ae20 con 0x7f6868106550 2024-04-02T04:32:59.519 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.750+0000 7f685f7fe700 1 -- 172.21.15.16:0/1474918222 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 45548+0+0 (secure 0 0 0) 0x7f68640198c0 con 0x7f6868106550 2024-04-02T04:32:59.519 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.750+0000 7f68702d4700 1 -- 172.21.15.16:0/1474918222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f686810a8e0 con 0x7f6868106550 2024-04-02T04:32:59.519 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.751+0000 7f685f7fe700 1 --2- 172.21.15.16:0/1474918222 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f6854038730 0x7f685403abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:32:59.519 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.751+0000 7f685f7fe700 1 -- 172.21.15.16:0/1474918222 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f686404cc10 con 0x7f6868106550 2024-04-02T04:32:59.519 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.751+0000 7f686d86f700 1 --2- 172.21.15.16:0/1474918222 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f6854038730 0x7f685403abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:32:59.519 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.751+0000 7f686d86f700 1 --2- 172.21.15.16:0/1474918222 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f6854038730 0x7f685403abf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6858006fd0 tx=0x7f6858006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:32:59.519 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.756+0000 7f685f7fe700 1 -- 172.21.15.16:0/1474918222 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6864018b80 con 0x7f6868106550 2024-04-02T04:32:59.519 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.974+0000 7f68702d4700 1 -- 172.21.15.16:0/1474918222 --> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f6868102d80 con 0x7f6854038730 2024-04-02T04:32:59.519 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.981+0000 7f685f7fe700 1 -- 172.21.15.16:0/1474918222 <== mgr.14118 v2:172.21.15.16:6800/873531241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6868102d80 con 0x7f6854038730 2024-04-02T04:32:59.519 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.984+0000 7f68702d4700 1 -- 172.21.15.16:0/1474918222 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f6854038730 msgr2=0x7f685403abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:32:59.519 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.984+0000 7f68702d4700 1 --2- 172.21.15.16:0/1474918222 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f6854038730 0x7f685403abf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6858006fd0 tx=0x7f6858006e40 comp rx=0 tx=0).stop 2024-04-02T04:32:59.520 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.984+0000 7f68702d4700 1 -- 172.21.15.16:0/1474918222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6868106550 msgr2=0x7f686819a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:32:59.520 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.984+0000 7f68702d4700 1 --2- 172.21.15.16:0/1474918222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6868106550 0x7f686819a240 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6864011710 tx=0x7f68640047c0 comp rx=0 tx=0).stop 2024-04-02T04:32:59.520 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.984+0000 7f68702d4700 1 -- 172.21.15.16:0/1474918222 shutdown_connections 2024-04-02T04:32:59.520 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.984+0000 7f68702d4700 1 --2- 172.21.15.16:0/1474918222 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f6854038730 0x7f685403abf0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:32:59.520 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.984+0000 7f68702d4700 1 --2- 172.21.15.16:0/1474918222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6868106550 0x7f686819a240 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:32:59.520 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.984+0000 7f68702d4700 1 -- 172.21.15.16:0/1474918222 >> 172.21.15.16:0/1474918222 conn(0x7f6868101b10 msgr2=0x7f6868102670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:32:59.520 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.984+0000 7f68702d4700 1 -- 172.21.15.16:0/1474918222 shutdown_connections 2024-04-02T04:32:59.520 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:58.984+0000 7f68702d4700 1 -- 172.21.15.16:0/1474918222 wait complete. 2024-04-02T04:32:59.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:59 smithi016 ceph-mon[28969]: [02/Apr/2024:04:32:58] ENGINE Bus STARTING 2024-04-02T04:32:59.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:59 smithi016 ceph-mon[28969]: [02/Apr/2024:04:32:58] ENGINE Serving on http://172.21.15.16:8765 2024-04-02T04:32:59.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:59 smithi016 ceph-mon[28969]: [02/Apr/2024:04:32:58] ENGINE Serving on https://172.21.15.16:7150 2024-04-02T04:32:59.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:59 smithi016 ceph-mon[28969]: [02/Apr/2024:04:32:58] ENGINE Bus STARTED 2024-04-02T04:32:59.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:59 smithi016 ceph-mon[28969]: mgrmap e8: smithi016.eqnebx(active, since 2s) 2024-04-02T04:32:59.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:59 smithi016 ceph-mon[28969]: from='mgr.14118 172.21.15.16:0/872684120' entity='mgr.smithi016.eqnebx' 2024-04-02T04:32:59.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:32:59 smithi016 ceph-mon[28969]: from='mgr.14118 172.21.15.16:0/872684120' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:33:00.665 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-02T04:33:00.665 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.920+0000 7fe4f935a700 1 Processor -- start 2024-04-02T04:33:00.665 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.920+0000 7fe4f935a700 1 -- start start 2024-04-02T04:33:00.666 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.920+0000 7fe4f935a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4f41053c0 0x7fe4f41057e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:00.666 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.920+0000 7fe4f935a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f4105db0 con 0x7fe4f41053c0 2024-04-02T04:33:00.666 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.920+0000 7fe4f2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4f41053c0 0x7fe4f41057e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:00.666 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.920+0000 7fe4f2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4f41053c0 0x7fe4f41057e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60408/0 (socket says 172.21.15.16:60408) 2024-04-02T04:33:00.666 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.920+0000 7fe4f2ffd700 1 -- 172.21.15.16:0/4225659765 learned_addr learned my addr 172.21.15.16:0/4225659765 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:00.666 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.921+0000 7fe4f2ffd700 1 -- 172.21.15.16:0/4225659765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f41065c0 con 0x7fe4f41053c0 2024-04-02T04:33:00.666 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.921+0000 7fe4f2ffd700 1 --2- 172.21.15.16:0/4225659765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4f41053c0 0x7fe4f41057e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec0098a0 tx=0x7fe4ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a07420dfce80776 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:00.666 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.921+0000 7fe4f1ffb700 1 -- 172.21.15.16:0/4225659765 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe4ec004180 con 0x7fe4f41053c0 2024-04-02T04:33:00.666 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.921+0000 7fe4f1ffb700 1 -- 172.21.15.16:0/4225659765 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fe4ec0042e0 con 0x7fe4f41053c0 2024-04-02T04:33:00.666 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.921+0000 7fe4f1ffb700 1 -- 172.21.15.16:0/4225659765 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe4ec011a70 con 0x7fe4f41053c0 2024-04-02T04:33:00.666 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.922+0000 7fe4f935a700 1 -- 172.21.15.16:0/4225659765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4f41053c0 msgr2=0x7fe4f41057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:00.666 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.922+0000 7fe4f935a700 1 --2- 172.21.15.16:0/4225659765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4f41053c0 0x7fe4f41057e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec0098a0 tx=0x7fe4ec009bb0 comp rx=0 tx=0).stop 2024-04-02T04:33:00.666 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.922+0000 7fe4f935a700 1 -- 172.21.15.16:0/4225659765 shutdown_connections 2024-04-02T04:33:00.667 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.922+0000 7fe4f935a700 1 --2- 172.21.15.16:0/4225659765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4f41053c0 0x7fe4f41057e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:00.667 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.922+0000 7fe4f935a700 1 -- 172.21.15.16:0/4225659765 >> 172.21.15.16:0/4225659765 conn(0x7fe4f4100980 msgr2=0x7fe4f4102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:00.667 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.922+0000 7fe4f935a700 1 -- 172.21.15.16:0/4225659765 shutdown_connections 2024-04-02T04:33:00.667 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.922+0000 7fe4f935a700 1 -- 172.21.15.16:0/4225659765 wait complete. 2024-04-02T04:33:00.667 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.922+0000 7fe4f935a700 1 Processor -- start 2024-04-02T04:33:00.667 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.923+0000 7fe4f935a700 1 -- start start 2024-04-02T04:33:00.667 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.923+0000 7fe4f935a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4f41053c0 0x7fe4f419a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:00.667 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.923+0000 7fe4f935a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f419a760 con 0x7fe4f41053c0 2024-04-02T04:33:00.667 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.923+0000 7fe4f2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4f41053c0 0x7fe4f419a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:00.667 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.923+0000 7fe4f2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4f41053c0 0x7fe4f419a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60416/0 (socket says 172.21.15.16:60416) 2024-04-02T04:33:00.667 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.923+0000 7fe4f2ffd700 1 -- 172.21.15.16:0/1949388695 learned_addr learned my addr 172.21.15.16:0/1949388695 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:00.667 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.923+0000 7fe4f2ffd700 1 -- 172.21.15.16:0/1949388695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4ec009550 con 0x7fe4f41053c0 2024-04-02T04:33:00.667 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.923+0000 7fe4f2ffd700 1 --2- 172.21.15.16:0/1949388695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4f41053c0 0x7fe4f419a220 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec011710 tx=0x7fe4ec0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:00.667 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.924+0000 7fe4ebfff700 1 -- 172.21.15.16:0/1949388695 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe4ec003b10 con 0x7fe4f41053c0 2024-04-02T04:33:00.668 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.924+0000 7fe4f935a700 1 -- 172.21.15.16:0/1949388695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4f419a960 con 0x7fe4f41053c0 2024-04-02T04:33:00.668 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.924+0000 7fe4ebfff700 1 -- 172.21.15.16:0/1949388695 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fe4ec01b8c0 con 0x7fe4f41053c0 2024-04-02T04:33:00.668 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.924+0000 7fe4ebfff700 1 -- 172.21.15.16:0/1949388695 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe4ec019710 con 0x7fe4f41053c0 2024-04-02T04:33:00.668 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.924+0000 7fe4f935a700 1 -- 172.21.15.16:0/1949388695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f419ae00 con 0x7fe4f41053c0 2024-04-02T04:33:00.668 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.925+0000 7fe4ebfff700 1 -- 172.21.15.16:0/1949388695 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 45548+0+0 (secure 0 0 0) 0x7fe4ec019910 con 0x7fe4f41053c0 2024-04-02T04:33:00.668 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.925+0000 7fe4f935a700 1 -- 172.21.15.16:0/1949388695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4f41943e0 con 0x7fe4f41053c0 2024-04-02T04:33:00.668 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.925+0000 7fe4ebfff700 1 --2- 172.21.15.16:0/1949388695 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fe4e00386e0 0x7fe4e003aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:00.668 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.925+0000 7fe4ebfff700 1 -- 172.21.15.16:0/1949388695 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe4ec04cb10 con 0x7fe4f41053c0 2024-04-02T04:33:00.668 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.925+0000 7fe4f27fc700 1 --2- 172.21.15.16:0/1949388695 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fe4e00386e0 0x7fe4e003aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:00.668 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.925+0000 7fe4f27fc700 1 --2- 172.21.15.16:0/1949388695 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fe4e00386e0 0x7fe4e003aba0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc006fd0 tx=0x7fe4dc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:00.668 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:32:59.930+0000 7fe4ebfff700 1 -- 172.21.15.16:0/1949388695 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe4ec018db0 con 0x7fe4f41053c0 2024-04-02T04:33:00.668 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:00.132+0000 7fe4f935a700 1 -- 172.21.15.16:0/1949388695 --> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fe4f402cf70 con 0x7fe4e00386e0 2024-04-02T04:33:00.668 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:00.133+0000 7fe4ebfff700 1 -- 172.21.15.16:0/1949388695 <== mgr.14118 v2:172.21.15.16:6800/873531241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fe4f402cf70 con 0x7fe4e00386e0 2024-04-02T04:33:00.668 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:00.135+0000 7fe4f935a700 1 -- 172.21.15.16:0/1949388695 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fe4e00386e0 msgr2=0x7fe4e003aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:00.668 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:00.135+0000 7fe4f935a700 1 --2- 172.21.15.16:0/1949388695 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fe4e00386e0 0x7fe4e003aba0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc006fd0 tx=0x7fe4dc006e40 comp rx=0 tx=0).stop 2024-04-02T04:33:00.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:00.135+0000 7fe4f935a700 1 -- 172.21.15.16:0/1949388695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4f41053c0 msgr2=0x7fe4f419a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:00.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:00.135+0000 7fe4f935a700 1 --2- 172.21.15.16:0/1949388695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4f41053c0 0x7fe4f419a220 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec011710 tx=0x7fe4ec0047c0 comp rx=0 tx=0).stop 2024-04-02T04:33:00.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:00.135+0000 7fe4f935a700 1 -- 172.21.15.16:0/1949388695 shutdown_connections 2024-04-02T04:33:00.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:00.135+0000 7fe4f935a700 1 --2- 172.21.15.16:0/1949388695 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fe4e00386e0 0x7fe4e003aba0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:00.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:00.135+0000 7fe4f935a700 1 --2- 172.21.15.16:0/1949388695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4f41053c0 0x7fe4f419a220 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:00.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:00.135+0000 7fe4f935a700 1 -- 172.21.15.16:0/1949388695 >> 172.21.15.16:0/1949388695 conn(0x7fe4f4100980 msgr2=0x7fe4f4101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:00.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:00.136+0000 7fe4f935a700 1 -- 172.21.15.16:0/1949388695 shutdown_connections 2024-04-02T04:33:00.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:00.136+0000 7fe4f935a700 1 -- 172.21.15.16:0/1949388695 wait complete. 2024-04-02T04:33:00.669 INFO:teuthology.orchestra.run.smithi016.stdout:Generating ssh key... 2024-04-02T04:33:01.001 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:00 smithi016 ceph-mon[28969]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:33:02.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:01 smithi016 ceph-mon[28969]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:33:02.329 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.031+0000 7f608149e700 1 Processor -- start 2024-04-02T04:33:02.329 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.031+0000 7f608149e700 1 -- start start 2024-04-02T04:33:02.330 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.031+0000 7f608149e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f607c106510 0x7f607c106930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:02.330 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.032+0000 7f608149e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c106f00 con 0x7f607c106510 2024-04-02T04:33:02.330 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.032+0000 7f607affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f607c106510 0x7f607c106930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:02.330 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.032+0000 7f607affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f607c106510 0x7f607c106930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60430/0 (socket says 172.21.15.16:60430) 2024-04-02T04:33:02.330 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.032+0000 7f607affd700 1 -- 172.21.15.16:0/4238710489 learned_addr learned my addr 172.21.15.16:0/4238710489 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:02.330 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.032+0000 7f607affd700 1 -- 172.21.15.16:0/4238710489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607c1076c0 con 0x7f607c106510 2024-04-02T04:33:02.330 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.032+0000 7f607affd700 1 --2- 172.21.15.16:0/4238710489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f607c106510 0x7f607c106930 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6074009e80 tx=0x7f607400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa38a72d52046f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:02.330 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.032+0000 7f6079ffb700 1 -- 172.21.15.16:0/4238710489 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f607400e070 con 0x7f607c106510 2024-04-02T04:33:02.330 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.032+0000 7f6079ffb700 1 -- 172.21.15.16:0/4238710489 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f607400b8c0 con 0x7f607c106510 2024-04-02T04:33:02.330 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.032+0000 7f6079ffb700 1 -- 172.21.15.16:0/4238710489 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6074004440 con 0x7f607c106510 2024-04-02T04:33:02.330 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.033+0000 7f608149e700 1 -- 172.21.15.16:0/4238710489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f607c106510 msgr2=0x7f607c106930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:02.330 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.033+0000 7f608149e700 1 --2- 172.21.15.16:0/4238710489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f607c106510 0x7f607c106930 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6074009e80 tx=0x7f607400b330 comp rx=0 tx=0).stop 2024-04-02T04:33:02.331 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.033+0000 7f608149e700 1 -- 172.21.15.16:0/4238710489 shutdown_connections 2024-04-02T04:33:02.331 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.033+0000 7f608149e700 1 --2- 172.21.15.16:0/4238710489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f607c106510 0x7f607c106930 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:02.331 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.033+0000 7f608149e700 1 -- 172.21.15.16:0/4238710489 >> 172.21.15.16:0/4238710489 conn(0x7f607c101b10 msgr2=0x7f607c103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:02.331 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.033+0000 7f608149e700 1 -- 172.21.15.16:0/4238710489 shutdown_connections 2024-04-02T04:33:02.331 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.033+0000 7f608149e700 1 -- 172.21.15.16:0/4238710489 wait complete. 2024-04-02T04:33:02.331 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.034+0000 7f608149e700 1 Processor -- start 2024-04-02T04:33:02.331 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.034+0000 7f608149e700 1 -- start start 2024-04-02T04:33:02.331 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.034+0000 7f608149e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f607c106510 0x7f607c19a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:02.331 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.034+0000 7f608149e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c19a6c0 con 0x7f607c106510 2024-04-02T04:33:02.331 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.034+0000 7f607affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f607c106510 0x7f607c19a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:02.331 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.034+0000 7f607affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f607c106510 0x7f607c19a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60436/0 (socket says 172.21.15.16:60436) 2024-04-02T04:33:02.331 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.034+0000 7f607affd700 1 -- 172.21.15.16:0/3575663315 learned_addr learned my addr 172.21.15.16:0/3575663315 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:02.331 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.034+0000 7f607affd700 1 -- 172.21.15.16:0/3575663315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6074009b30 con 0x7f607c106510 2024-04-02T04:33:02.331 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.035+0000 7f607affd700 1 --2- 172.21.15.16:0/3575663315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f607c106510 0x7f607c19a180 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f60740099e0 tx=0x7f6074003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:02.332 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.035+0000 7f6073fff700 1 -- 172.21.15.16:0/3575663315 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f607400e040 con 0x7f607c106510 2024-04-02T04:33:02.332 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.035+0000 7f608149e700 1 -- 172.21.15.16:0/3575663315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f607c19a8c0 con 0x7f607c106510 2024-04-02T04:33:02.332 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.035+0000 7f6073fff700 1 -- 172.21.15.16:0/3575663315 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f607400bc40 con 0x7f607c106510 2024-04-02T04:33:02.332 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.035+0000 7f6073fff700 1 -- 172.21.15.16:0/3575663315 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f60740125e0 con 0x7f607c106510 2024-04-02T04:33:02.332 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.035+0000 7f608149e700 1 -- 172.21.15.16:0/3575663315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f607c19adc0 con 0x7f607c106510 2024-04-02T04:33:02.332 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.036+0000 7f6073fff700 1 -- 172.21.15.16:0/3575663315 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 45548+0+0 (secure 0 0 0) 0x7f6074019040 con 0x7f607c106510 2024-04-02T04:33:02.332 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.036+0000 7f6073fff700 1 --2- 172.21.15.16:0/3575663315 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f6068038700 0x7f606803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:02.332 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.036+0000 7f608149e700 1 -- 172.21.15.16:0/3575663315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f607c0623e0 con 0x7f607c106510 2024-04-02T04:33:02.332 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.036+0000 7f6073fff700 1 -- 172.21.15.16:0/3575663315 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f607404c0a0 con 0x7f607c106510 2024-04-02T04:33:02.332 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.036+0000 7f607a7fc700 1 --2- 172.21.15.16:0/3575663315 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f6068038700 0x7f606803abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:02.332 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.036+0000 7f607a7fc700 1 --2- 172.21.15.16:0/3575663315 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f6068038700 0x7f606803abc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6064006fd0 tx=0x7f6064006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:02.332 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.041+0000 7f6073fff700 1 -- 172.21.15.16:0/3575663315 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f607401ba40 con 0x7f607c106510 2024-04-02T04:33:02.332 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.246+0000 7f608149e700 1 -- 172.21.15.16:0/3575663315 --> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f607c10aaf0 con 0x7f6068038700 2024-04-02T04:33:02.332 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.773+0000 7f6073fff700 1 -- 172.21.15.16:0/3575663315 <== mgr.14118 v2:172.21.15.16:6800/873531241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f607c10aaf0 con 0x7f6068038700 2024-04-02T04:33:02.333 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.775+0000 7f608149e700 1 -- 172.21.15.16:0/3575663315 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f6068038700 msgr2=0x7f606803abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:02.333 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.775+0000 7f608149e700 1 --2- 172.21.15.16:0/3575663315 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f6068038700 0x7f606803abc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6064006fd0 tx=0x7f6064006e40 comp rx=0 tx=0).stop 2024-04-02T04:33:02.333 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.775+0000 7f608149e700 1 -- 172.21.15.16:0/3575663315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f607c106510 msgr2=0x7f607c19a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:02.333 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.775+0000 7f608149e700 1 --2- 172.21.15.16:0/3575663315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f607c106510 0x7f607c19a180 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f60740099e0 tx=0x7f6074003d00 comp rx=0 tx=0).stop 2024-04-02T04:33:02.333 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.775+0000 7f608149e700 1 -- 172.21.15.16:0/3575663315 shutdown_connections 2024-04-02T04:33:02.333 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.775+0000 7f608149e700 1 --2- 172.21.15.16:0/3575663315 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f6068038700 0x7f606803abc0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:02.333 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.775+0000 7f608149e700 1 --2- 172.21.15.16:0/3575663315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f607c106510 0x7f607c19a180 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:02.333 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.775+0000 7f608149e700 1 -- 172.21.15.16:0/3575663315 >> 172.21.15.16:0/3575663315 conn(0x7f607c101b10 msgr2=0x7f607c1026a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:02.333 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.775+0000 7f608149e700 1 -- 172.21.15.16:0/3575663315 shutdown_connections 2024-04-02T04:33:02.333 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:01.775+0000 7f608149e700 1 -- 172.21.15.16:0/3575663315 wait complete. 2024-04-02T04:33:03.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:02 smithi016 ceph-mon[28969]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:33:03.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:02 smithi016 ceph-mon[28969]: Generating ssh key... 2024-04-02T04:33:03.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:02 smithi016 ceph-mon[28969]: from='mgr.14118 172.21.15.16:0/872684120' entity='mgr.smithi016.eqnebx' 2024-04-02T04:33:03.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:02 smithi016 ceph-mon[28969]: from='mgr.14118 172.21.15.16:0/872684120' entity='mgr.smithi016.eqnebx' 2024-04-02T04:33:03.435 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3nrI5QgzO2fY8sZYHpB+09l8v+CM9uFQ3nOL4i67UiiVLFEpF6CMC9qsNfnYOeOWH27Kve1tJ2R/GRGN1A8CpPbeYIwmovEMPmw8yFsYEnZy2nDaGgcc4PHEobGknzl7j0XB52Jm68APZ9i25s1GGmRayNH+c87AjCxgtRL9R/8y99Y+6oalbokf48y229+uaY/hMty3um0oGPOIDkbHP7Qxf8Ur0gDB2Kww4wDNauff0DmrW5Uq6cwD1CzlGYuBqlnKUNCdHRBBlSmo/2byM/4fnq/DjdpqTPVVN981jErA8dPefsle3kHmEojFemfx+HmQP3O4ZhT31/XH17YflocjtUbtF6drqQpZlnXD21gBnPWJ8CQaJpfC8tVpo5B26THiQGEs4Qbo0Il0giYlPze/hUTI8wrgBlIrefIRhWwHIzmuntifFgZZ8vJ6RJ6SAFmLvrXwusC+2j6ei2M8veyVDpzl5fpcahwinw0HOQLvZRfjL9ap3rKmKTigtSZk= ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4 2024-04-02T04:33:03.435 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.711+0000 7f2429d29700 1 Processor -- start 2024-04-02T04:33:03.435 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.711+0000 7f2429d29700 1 -- start start 2024-04-02T04:33:03.435 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.711+0000 7f2429d29700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2424106f40 0x7f2424109330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:03.435 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.711+0000 7f2429d29700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2424079e90 con 0x7f2424106f40 2024-04-02T04:33:03.435 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.712+0000 7f24237fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2424106f40 0x7f2424109330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:03.435 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.712+0000 7f24237fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2424106f40 0x7f2424109330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42068/0 (socket says 172.21.15.16:42068) 2024-04-02T04:33:03.435 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.712+0000 7f24237fe700 1 -- 172.21.15.16:0/2739543606 learned_addr learned my addr 172.21.15.16:0/2739543606 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:03.436 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.712+0000 7f24237fe700 1 -- 172.21.15.16:0/2739543606 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2424109870 con 0x7f2424106f40 2024-04-02T04:33:03.436 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.712+0000 7f24237fe700 1 --2- 172.21.15.16:0/2739543606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2424106f40 0x7f2424109330 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f24140098a0 tx=0x7f2414009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e5d61f650213e9e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:03.436 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.712+0000 7f24227fc700 1 -- 172.21.15.16:0/2739543606 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2414004180 con 0x7f2424106f40 2024-04-02T04:33:03.436 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.712+0000 7f24227fc700 1 -- 172.21.15.16:0/2739543606 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f24140042e0 con 0x7f2424106f40 2024-04-02T04:33:03.436 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.712+0000 7f24227fc700 1 -- 172.21.15.16:0/2739543606 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2414011a70 con 0x7f2424106f40 2024-04-02T04:33:03.436 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.713+0000 7f2429d29700 1 -- 172.21.15.16:0/2739543606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2424106f40 msgr2=0x7f2424109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:03.436 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.713+0000 7f2429d29700 1 --2- 172.21.15.16:0/2739543606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2424106f40 0x7f2424109330 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f24140098a0 tx=0x7f2414009bb0 comp rx=0 tx=0).stop 2024-04-02T04:33:03.436 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.713+0000 7f2429d29700 1 -- 172.21.15.16:0/2739543606 shutdown_connections 2024-04-02T04:33:03.436 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.713+0000 7f2429d29700 1 --2- 172.21.15.16:0/2739543606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2424106f40 0x7f2424109330 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:03.436 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.713+0000 7f2429d29700 1 -- 172.21.15.16:0/2739543606 >> 172.21.15.16:0/2739543606 conn(0x7f2424100950 msgr2=0x7f2424102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:03.436 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.713+0000 7f2429d29700 1 -- 172.21.15.16:0/2739543606 shutdown_connections 2024-04-02T04:33:03.436 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.713+0000 7f2429d29700 1 -- 172.21.15.16:0/2739543606 wait complete. 2024-04-02T04:33:03.436 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.713+0000 7f2429d29700 1 Processor -- start 2024-04-02T04:33:03.436 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.714+0000 7f2429d29700 1 -- start start 2024-04-02T04:33:03.437 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.714+0000 7f2429d29700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2424106f40 0x7f242419a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:03.437 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.714+0000 7f2429d29700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242419a780 con 0x7f2424106f40 2024-04-02T04:33:03.437 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.714+0000 7f24237fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2424106f40 0x7f242419a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:03.437 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.714+0000 7f24237fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2424106f40 0x7f242419a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42084/0 (socket says 172.21.15.16:42084) 2024-04-02T04:33:03.437 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.714+0000 7f24237fe700 1 -- 172.21.15.16:0/1599646247 learned_addr learned my addr 172.21.15.16:0/1599646247 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:03.437 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.714+0000 7f24237fe700 1 -- 172.21.15.16:0/1599646247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2414009550 con 0x7f2424106f40 2024-04-02T04:33:03.437 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.714+0000 7f24237fe700 1 --2- 172.21.15.16:0/1599646247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2424106f40 0x7f242419a240 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2414011a20 tx=0x7f24140047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:03.437 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.714+0000 7f2420ff9700 1 -- 172.21.15.16:0/1599646247 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2414003b10 con 0x7f2424106f40 2024-04-02T04:33:03.437 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.715+0000 7f2429d29700 1 -- 172.21.15.16:0/1599646247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f242419a980 con 0x7f2424106f40 2024-04-02T04:33:03.437 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.715+0000 7f2420ff9700 1 -- 172.21.15.16:0/1599646247 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f241401b8c0 con 0x7f2424106f40 2024-04-02T04:33:03.437 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.715+0000 7f2420ff9700 1 -- 172.21.15.16:0/1599646247 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f24140237b0 con 0x7f2424106f40 2024-04-02T04:33:03.437 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.715+0000 7f2429d29700 1 -- 172.21.15.16:0/1599646247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f242419ae20 con 0x7f2424106f40 2024-04-02T04:33:03.437 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.715+0000 7f2420ff9700 1 -- 172.21.15.16:0/1599646247 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 45548+0+0 (secure 0 0 0) 0x7f2414023a30 con 0x7f2424106f40 2024-04-02T04:33:03.437 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.715+0000 7f2429d29700 1 -- 172.21.15.16:0/1599646247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f242410a5f0 con 0x7f2424106f40 2024-04-02T04:33:03.437 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.716+0000 7f2420ff9700 1 --2- 172.21.15.16:0/1599646247 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f240c0386e0 0x7f240c03aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:03.438 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.716+0000 7f2420ff9700 1 -- 172.21.15.16:0/1599646247 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f241404cbb0 con 0x7f2424106f40 2024-04-02T04:33:03.438 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.716+0000 7f2422ffd700 1 --2- 172.21.15.16:0/1599646247 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f240c0386e0 0x7f240c03aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:03.438 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.716+0000 7f2422ffd700 1 --2- 172.21.15.16:0/1599646247 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f240c0386e0 0x7f240c03aba0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2418006fd0 tx=0x7f2418006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:03.438 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.720+0000 7f2420ff9700 1 -- 172.21.15.16:0/1599646247 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2414023ce0 con 0x7f2424106f40 2024-04-02T04:33:03.438 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.920+0000 7f2429d29700 1 -- 172.21.15.16:0/1599646247 --> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f2424101c30 con 0x7f240c0386e0 2024-04-02T04:33:03.438 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.921+0000 7f2420ff9700 1 -- 172.21.15.16:0/1599646247 <== mgr.14118 v2:172.21.15.16:6800/873531241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f2424101c30 con 0x7f240c0386e0 2024-04-02T04:33:03.438 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.923+0000 7f2429d29700 1 -- 172.21.15.16:0/1599646247 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f240c0386e0 msgr2=0x7f240c03aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:03.438 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.923+0000 7f2429d29700 1 --2- 172.21.15.16:0/1599646247 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f240c0386e0 0x7f240c03aba0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2418006fd0 tx=0x7f2418006e40 comp rx=0 tx=0).stop 2024-04-02T04:33:03.438 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.923+0000 7f2429d29700 1 -- 172.21.15.16:0/1599646247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2424106f40 msgr2=0x7f242419a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:03.438 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.923+0000 7f2429d29700 1 --2- 172.21.15.16:0/1599646247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2424106f40 0x7f242419a240 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2414011a20 tx=0x7f24140047c0 comp rx=0 tx=0).stop 2024-04-02T04:33:03.438 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.923+0000 7f2429d29700 1 -- 172.21.15.16:0/1599646247 shutdown_connections 2024-04-02T04:33:03.438 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.923+0000 7f2429d29700 1 --2- 172.21.15.16:0/1599646247 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f240c0386e0 0x7f240c03aba0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:03.438 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.923+0000 7f2429d29700 1 --2- 172.21.15.16:0/1599646247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2424106f40 0x7f242419a240 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:03.438 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.923+0000 7f2429d29700 1 -- 172.21.15.16:0/1599646247 >> 172.21.15.16:0/1599646247 conn(0x7f2424100950 msgr2=0x7f2424101520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:03.439 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.923+0000 7f2429d29700 1 -- 172.21.15.16:0/1599646247 shutdown_connections 2024-04-02T04:33:03.439 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:02.923+0000 7f2429d29700 1 -- 172.21.15.16:0/1599646247 wait complete. 2024-04-02T04:33:03.439 INFO:teuthology.orchestra.run.smithi016.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-02T04:33:03.439 INFO:teuthology.orchestra.run.smithi016.stdout:Adding key to root@localhost authorized_keys... 2024-04-02T04:33:03.439 INFO:teuthology.orchestra.run.smithi016.stdout:Adding host smithi016... 2024-04-02T04:33:04.681 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:04 smithi016 ceph-mon[28969]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:33:06.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:05 smithi016 ceph-mon[28969]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi016", "addr": "172.21.15.16", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:33:07.010 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:06 smithi016 ceph-mon[28969]: Deploying cephadm binary to smithi016 2024-04-02T04:33:07.294 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Added host 'smithi016' with addr '172.21.15.16' 2024-04-02T04:33:07.294 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.880+0000 7fc1478a8700 1 Processor -- start 2024-04-02T04:33:07.294 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.880+0000 7fc1478a8700 1 -- start start 2024-04-02T04:33:07.294 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.880+0000 7fc1478a8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc140104dd0 0x7fc1401051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:07.294 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.880+0000 7fc1478a8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc140079e10 con 0x7fc140104dd0 2024-04-02T04:33:07.294 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.880+0000 7fc145644700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc140104dd0 0x7fc1401051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:07.294 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.880+0000 7fc145644700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc140104dd0 0x7fc1401051f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42090/0 (socket says 172.21.15.16:42090) 2024-04-02T04:33:07.294 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.880+0000 7fc145644700 1 -- 172.21.15.16:0/3627108424 learned_addr learned my addr 172.21.15.16:0/3627108424 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:07.294 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.880+0000 7fc145644700 1 -- 172.21.15.16:0/3627108424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc140105730 con 0x7fc140104dd0 2024-04-02T04:33:07.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.881+0000 7fc145644700 1 --2- 172.21.15.16:0/3627108424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc140104dd0 0x7fc1401051f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc13c0098a0 tx=0x7fc13c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4877414c78e4568b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:07.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.881+0000 7fc137fff700 1 -- 172.21.15.16:0/3627108424 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc13c004180 con 0x7fc140104dd0 2024-04-02T04:33:07.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.881+0000 7fc137fff700 1 -- 172.21.15.16:0/3627108424 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fc13c0042e0 con 0x7fc140104dd0 2024-04-02T04:33:07.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.881+0000 7fc137fff700 1 -- 172.21.15.16:0/3627108424 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc13c011a70 con 0x7fc140104dd0 2024-04-02T04:33:07.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.881+0000 7fc1478a8700 1 -- 172.21.15.16:0/3627108424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc140104dd0 msgr2=0x7fc1401051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:07.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.881+0000 7fc1478a8700 1 --2- 172.21.15.16:0/3627108424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc140104dd0 0x7fc1401051f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc13c0098a0 tx=0x7fc13c009bb0 comp rx=0 tx=0).stop 2024-04-02T04:33:07.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.881+0000 7fc1478a8700 1 -- 172.21.15.16:0/3627108424 shutdown_connections 2024-04-02T04:33:07.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.881+0000 7fc1478a8700 1 --2- 172.21.15.16:0/3627108424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc140104dd0 0x7fc1401051f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:07.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.881+0000 7fc1478a8700 1 -- 172.21.15.16:0/3627108424 >> 172.21.15.16:0/3627108424 conn(0x7fc1401009d0 msgr2=0x7fc140102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:07.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.881+0000 7fc1478a8700 1 -- 172.21.15.16:0/3627108424 shutdown_connections 2024-04-02T04:33:07.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.882+0000 7fc1478a8700 1 -- 172.21.15.16:0/3627108424 wait complete. 2024-04-02T04:33:07.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.882+0000 7fc1478a8700 1 Processor -- start 2024-04-02T04:33:07.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.882+0000 7fc1478a8700 1 -- start start 2024-04-02T04:33:07.295 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.882+0000 7fc1478a8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc140104dd0 0x7fc14019c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:07.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.882+0000 7fc1478a8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14019c940 con 0x7fc140104dd0 2024-04-02T04:33:07.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.882+0000 7fc145644700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc140104dd0 0x7fc14019c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:07.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.882+0000 7fc145644700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc140104dd0 0x7fc14019c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42098/0 (socket says 172.21.15.16:42098) 2024-04-02T04:33:07.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.882+0000 7fc145644700 1 -- 172.21.15.16:0/3858568497 learned_addr learned my addr 172.21.15.16:0/3858568497 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:07.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.883+0000 7fc145644700 1 -- 172.21.15.16:0/3858568497 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc13c009550 con 0x7fc140104dd0 2024-04-02T04:33:07.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.883+0000 7fc145644700 1 --2- 172.21.15.16:0/3858568497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc140104dd0 0x7fc14019c400 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc13c011a20 tx=0x7fc13c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:07.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.883+0000 7fc1367fc700 1 -- 172.21.15.16:0/3858568497 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc13c003b10 con 0x7fc140104dd0 2024-04-02T04:33:07.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.883+0000 7fc1367fc700 1 -- 172.21.15.16:0/3858568497 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fc13c01b8c0 con 0x7fc140104dd0 2024-04-02T04:33:07.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.883+0000 7fc1478a8700 1 -- 172.21.15.16:0/3858568497 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc14019cb40 con 0x7fc140104dd0 2024-04-02T04:33:07.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.883+0000 7fc1367fc700 1 -- 172.21.15.16:0/3858568497 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc13c0237b0 con 0x7fc140104dd0 2024-04-02T04:33:07.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.883+0000 7fc1478a8700 1 -- 172.21.15.16:0/3858568497 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc14019cfe0 con 0x7fc140104dd0 2024-04-02T04:33:07.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.884+0000 7fc1367fc700 1 -- 172.21.15.16:0/3858568497 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 45548+0+0 (secure 0 0 0) 0x7fc13c023a30 con 0x7fc140104dd0 2024-04-02T04:33:07.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.884+0000 7fc1478a8700 1 -- 172.21.15.16:0/3858568497 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc140196150 con 0x7fc140104dd0 2024-04-02T04:33:07.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.884+0000 7fc1367fc700 1 --2- 172.21.15.16:0/3858568497 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fc12c0386e0 0x7fc12c03aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:07.296 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.884+0000 7fc1367fc700 1 -- 172.21.15.16:0/3858568497 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc13c04cbc0 con 0x7fc140104dd0 2024-04-02T04:33:07.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.884+0000 7fc144e43700 1 --2- 172.21.15.16:0/3858568497 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fc12c0386e0 0x7fc12c03aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:07.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.885+0000 7fc144e43700 1 --2- 172.21.15.16:0/3858568497 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fc12c0386e0 0x7fc12c03aba0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc130006fd0 tx=0x7fc130006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:07.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:03.889+0000 7fc1367fc700 1 -- 172.21.15.16:0/3858568497 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc13c02bd60 con 0x7fc140104dd0 2024-04-02T04:33:07.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:04.096+0000 7fc1478a8700 1 -- 172.21.15.16:0/3858568497 --> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi016", "addr": "172.21.15.16", "target": ["mon-mgr", ""]}) v1 -- 0x7fc14002cee0 con 0x7fc12c0386e0 2024-04-02T04:33:07.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:06.752+0000 7fc1367fc700 1 -- 172.21.15.16:0/3858568497 <== mgr.14118 v2:172.21.15.16:6800/873531241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7fc14002cee0 con 0x7fc12c0386e0 2024-04-02T04:33:07.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:06.754+0000 7fc1478a8700 1 -- 172.21.15.16:0/3858568497 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fc12c0386e0 msgr2=0x7fc12c03aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:07.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:06.754+0000 7fc1478a8700 1 --2- 172.21.15.16:0/3858568497 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fc12c0386e0 0x7fc12c03aba0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc130006fd0 tx=0x7fc130006e40 comp rx=0 tx=0).stop 2024-04-02T04:33:07.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:06.754+0000 7fc1478a8700 1 -- 172.21.15.16:0/3858568497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc140104dd0 msgr2=0x7fc14019c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:07.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:06.754+0000 7fc1478a8700 1 --2- 172.21.15.16:0/3858568497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc140104dd0 0x7fc14019c400 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc13c011a20 tx=0x7fc13c0047c0 comp rx=0 tx=0).stop 2024-04-02T04:33:07.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:06.754+0000 7fc1478a8700 1 -- 172.21.15.16:0/3858568497 shutdown_connections 2024-04-02T04:33:07.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:06.754+0000 7fc1478a8700 1 --2- 172.21.15.16:0/3858568497 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fc12c0386e0 0x7fc12c03aba0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:07.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:06.754+0000 7fc1478a8700 1 --2- 172.21.15.16:0/3858568497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc140104dd0 0x7fc14019c400 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:07.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:06.754+0000 7fc1478a8700 1 -- 172.21.15.16:0/3858568497 >> 172.21.15.16:0/3858568497 conn(0x7fc1401009d0 msgr2=0x7fc140193290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:07.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:06.755+0000 7fc1478a8700 1 -- 172.21.15.16:0/3858568497 shutdown_connections 2024-04-02T04:33:07.297 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:06.755+0000 7fc1478a8700 1 -- 172.21.15.16:0/3858568497 wait complete. 2024-04-02T04:33:07.297 INFO:teuthology.orchestra.run.smithi016.stdout:Deploying mon service with default placement... 2024-04-02T04:33:07.896 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:07 smithi016 ceph-mon[28969]: from='mgr.14118 172.21.15.16:0/872684120' entity='mgr.smithi016.eqnebx' 2024-04-02T04:33:07.896 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:07 smithi016 ceph-mon[28969]: Added host smithi016 2024-04-02T04:33:07.896 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:07 smithi016 ceph-mon[28969]: from='mgr.14118 172.21.15.16:0/872684120' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:33:08.554 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-02T04:33:08.554 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.714+0000 7fd1408b5700 1 Processor -- start 2024-04-02T04:33:08.554 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.714+0000 7fd1408b5700 1 -- start start 2024-04-02T04:33:08.554 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.714+0000 7fd1408b5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd13c1053c0 0x7fd13c1057e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:08.554 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.714+0000 7fd1408b5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c105db0 con 0x7fd13c1053c0 2024-04-02T04:33:08.554 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.715+0000 7fd13a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd13c1053c0 0x7fd13c1057e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:08.554 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.715+0000 7fd13a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd13c1053c0 0x7fd13c1057e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42104/0 (socket says 172.21.15.16:42104) 2024-04-02T04:33:08.554 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.715+0000 7fd13a59c700 1 -- 172.21.15.16:0/2776582224 learned_addr learned my addr 172.21.15.16:0/2776582224 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:08.555 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.715+0000 7fd13a59c700 1 -- 172.21.15.16:0/2776582224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd13c1065c0 con 0x7fd13c1053c0 2024-04-02T04:33:08.555 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.715+0000 7fd13a59c700 1 --2- 172.21.15.16:0/2776582224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd13c1053c0 0x7fd13c1057e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd134009e80 tx=0x7fd13400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bacf1695786dd486 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:08.555 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.715+0000 7fd13959a700 1 -- 172.21.15.16:0/2776582224 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd13400e070 con 0x7fd13c1053c0 2024-04-02T04:33:08.555 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.715+0000 7fd13959a700 1 -- 172.21.15.16:0/2776582224 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fd13400b8c0 con 0x7fd13c1053c0 2024-04-02T04:33:08.555 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.715+0000 7fd13959a700 1 -- 172.21.15.16:0/2776582224 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd134004440 con 0x7fd13c1053c0 2024-04-02T04:33:08.555 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.716+0000 7fd1408b5700 1 -- 172.21.15.16:0/2776582224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd13c1053c0 msgr2=0x7fd13c1057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:08.555 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.716+0000 7fd1408b5700 1 --2- 172.21.15.16:0/2776582224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd13c1053c0 0x7fd13c1057e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd134009e80 tx=0x7fd13400b330 comp rx=0 tx=0).stop 2024-04-02T04:33:08.555 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.716+0000 7fd1408b5700 1 -- 172.21.15.16:0/2776582224 shutdown_connections 2024-04-02T04:33:08.555 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.716+0000 7fd1408b5700 1 --2- 172.21.15.16:0/2776582224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd13c1053c0 0x7fd13c1057e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:08.555 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.716+0000 7fd1408b5700 1 -- 172.21.15.16:0/2776582224 >> 172.21.15.16:0/2776582224 conn(0x7fd13c100980 msgr2=0x7fd13c102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:08.555 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.716+0000 7fd1408b5700 1 -- 172.21.15.16:0/2776582224 shutdown_connections 2024-04-02T04:33:08.555 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.716+0000 7fd1408b5700 1 -- 172.21.15.16:0/2776582224 wait complete. 2024-04-02T04:33:08.555 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.717+0000 7fd1408b5700 1 Processor -- start 2024-04-02T04:33:08.555 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.717+0000 7fd1408b5700 1 -- start start 2024-04-02T04:33:08.555 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.717+0000 7fd1408b5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd13c1053c0 0x7fd13c190410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:08.555 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.717+0000 7fd1408b5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c190950 con 0x7fd13c1053c0 2024-04-02T04:33:08.556 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.717+0000 7fd13a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd13c1053c0 0x7fd13c190410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:08.556 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.717+0000 7fd13a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd13c1053c0 0x7fd13c190410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42112/0 (socket says 172.21.15.16:42112) 2024-04-02T04:33:08.556 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.717+0000 7fd13a59c700 1 -- 172.21.15.16:0/3650090743 learned_addr learned my addr 172.21.15.16:0/3650090743 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:08.556 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.717+0000 7fd13a59c700 1 -- 172.21.15.16:0/3650090743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd134009b30 con 0x7fd13c1053c0 2024-04-02T04:33:08.556 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.717+0000 7fd13a59c700 1 --2- 172.21.15.16:0/3650090743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd13c1053c0 0x7fd13c190410 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd134003c30 tx=0x7fd134003d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:08.556 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.717+0000 7fd1337fe700 1 -- 172.21.15.16:0/3650090743 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd13400e040 con 0x7fd13c1053c0 2024-04-02T04:33:08.556 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.718+0000 7fd1408b5700 1 -- 172.21.15.16:0/3650090743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd13c18ebe0 con 0x7fd13c1053c0 2024-04-02T04:33:08.556 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.718+0000 7fd1337fe700 1 -- 172.21.15.16:0/3650090743 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fd13400bc40 con 0x7fd13c1053c0 2024-04-02T04:33:08.556 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.718+0000 7fd1408b5700 1 -- 172.21.15.16:0/3650090743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd13c18f0e0 con 0x7fd13c1053c0 2024-04-02T04:33:08.556 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.718+0000 7fd1337fe700 1 -- 172.21.15.16:0/3650090743 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd134012750 con 0x7fd13c1053c0 2024-04-02T04:33:08.556 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.719+0000 7fd1337fe700 1 -- 172.21.15.16:0/3650090743 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 45548+0+0 (secure 0 0 0) 0x7fd134019040 con 0x7fd13c1053c0 2024-04-02T04:33:08.556 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.719+0000 7fd1408b5700 1 -- 172.21.15.16:0/3650090743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd13c190a90 con 0x7fd13c1053c0 2024-04-02T04:33:08.556 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.719+0000 7fd1337fe700 1 --2- 172.21.15.16:0/3650090743 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fd1280386e0 0x7fd12803aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:08.556 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.719+0000 7fd1337fe700 1 -- 172.21.15.16:0/3650090743 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd13404c380 con 0x7fd13c1053c0 2024-04-02T04:33:08.556 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.719+0000 7fd139d9b700 1 --2- 172.21.15.16:0/3650090743 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fd1280386e0 0x7fd12803aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:08.557 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.719+0000 7fd139d9b700 1 --2- 172.21.15.16:0/3650090743 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fd1280386e0 0x7fd12803aba0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd124006fd0 tx=0x7fd124006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:08.557 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.724+0000 7fd1337fe700 1 -- 172.21.15.16:0/3650090743 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd13c190a90 con 0x7fd13c1053c0 2024-04-02T04:33:08.557 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.940+0000 7fd1408b5700 1 -- 172.21.15.16:0/3650090743 --> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7fd13c190a90 con 0x7fd1280386e0 2024-04-02T04:33:08.557 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.944+0000 7fd1337fe700 1 -- 172.21.15.16:0/3650090743 <== mgr.14118 v2:172.21.15.16:6800/873531241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fd13c190a90 con 0x7fd1280386e0 2024-04-02T04:33:08.557 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.946+0000 7fd1408b5700 1 -- 172.21.15.16:0/3650090743 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fd1280386e0 msgr2=0x7fd12803aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:08.557 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.946+0000 7fd1408b5700 1 --2- 172.21.15.16:0/3650090743 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fd1280386e0 0x7fd12803aba0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd124006fd0 tx=0x7fd124006e40 comp rx=0 tx=0).stop 2024-04-02T04:33:08.557 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.946+0000 7fd1408b5700 1 -- 172.21.15.16:0/3650090743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd13c1053c0 msgr2=0x7fd13c190410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:08.557 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.946+0000 7fd1408b5700 1 --2- 172.21.15.16:0/3650090743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd13c1053c0 0x7fd13c190410 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd134003c30 tx=0x7fd134003d10 comp rx=0 tx=0).stop 2024-04-02T04:33:08.557 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.946+0000 7fd1408b5700 1 -- 172.21.15.16:0/3650090743 shutdown_connections 2024-04-02T04:33:08.557 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.946+0000 7fd1408b5700 1 --2- 172.21.15.16:0/3650090743 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fd1280386e0 0x7fd12803aba0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:08.557 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.946+0000 7fd1408b5700 1 --2- 172.21.15.16:0/3650090743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd13c1053c0 0x7fd13c190410 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:08.557 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.946+0000 7fd1408b5700 1 -- 172.21.15.16:0/3650090743 >> 172.21.15.16:0/3650090743 conn(0x7fd13c100980 msgr2=0x7fd13c101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:08.557 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.946+0000 7fd1408b5700 1 -- 172.21.15.16:0/3650090743 shutdown_connections 2024-04-02T04:33:08.557 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:07.946+0000 7fd1408b5700 1 -- 172.21.15.16:0/3650090743 wait complete. 2024-04-02T04:33:08.558 INFO:teuthology.orchestra.run.smithi016.stdout:Deploying mgr service with default placement... 2024-04-02T04:33:09.208 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:08 smithi016 ceph-mon[28969]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:33:09.208 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:08 smithi016 ceph-mon[28969]: Saving service mon spec with placement count:5 2024-04-02T04:33:09.208 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:08 smithi016 ceph-mon[28969]: from='mgr.14118 172.21.15.16:0/872684120' entity='mgr.smithi016.eqnebx' 2024-04-02T04:33:09.752 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-02T04:33:09.753 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.982+0000 7f3d00a68700 1 Processor -- start 2024-04-02T04:33:09.753 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.982+0000 7f3d00a68700 1 -- start start 2024-04-02T04:33:09.753 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.982+0000 7f3d00a68700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cfc106fc0 0x7f3cfc1093b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:09.753 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.982+0000 7f3d00a68700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc079e30 con 0x7f3cfc106fc0 2024-04-02T04:33:09.753 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.982+0000 7f3cfa59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cfc106fc0 0x7f3cfc1093b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:09.753 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.982+0000 7f3cfa59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cfc106fc0 0x7f3cfc1093b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42122/0 (socket says 172.21.15.16:42122) 2024-04-02T04:33:09.753 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.982+0000 7f3cfa59c700 1 -- 172.21.15.16:0/3392466072 learned_addr learned my addr 172.21.15.16:0/3392466072 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:09.753 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.983+0000 7f3cfa59c700 1 -- 172.21.15.16:0/3392466072 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cfc1098f0 con 0x7f3cfc106fc0 2024-04-02T04:33:09.753 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.983+0000 7f3cfa59c700 1 --2- 172.21.15.16:0/3392466072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cfc106fc0 0x7f3cfc1093b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3cf00098a0 tx=0x7f3cf0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f496223586f4cfa server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:09.753 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.983+0000 7f3cf959a700 1 -- 172.21.15.16:0/3392466072 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3cf0004180 con 0x7f3cfc106fc0 2024-04-02T04:33:09.753 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.983+0000 7f3cf959a700 1 -- 172.21.15.16:0/3392466072 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f3cf00042e0 con 0x7f3cfc106fc0 2024-04-02T04:33:09.753 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.983+0000 7f3cf959a700 1 -- 172.21.15.16:0/3392466072 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3cf0011a70 con 0x7f3cfc106fc0 2024-04-02T04:33:09.753 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.984+0000 7f3d00a68700 1 -- 172.21.15.16:0/3392466072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cfc106fc0 msgr2=0x7f3cfc1093b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:09.754 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.984+0000 7f3d00a68700 1 --2- 172.21.15.16:0/3392466072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cfc106fc0 0x7f3cfc1093b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3cf00098a0 tx=0x7f3cf0009bb0 comp rx=0 tx=0).stop 2024-04-02T04:33:09.754 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.984+0000 7f3d00a68700 1 -- 172.21.15.16:0/3392466072 shutdown_connections 2024-04-02T04:33:09.754 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.984+0000 7f3d00a68700 1 --2- 172.21.15.16:0/3392466072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cfc106fc0 0x7f3cfc1093b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:09.754 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.984+0000 7f3d00a68700 1 -- 172.21.15.16:0/3392466072 >> 172.21.15.16:0/3392466072 conn(0x7f3cfc1009b0 msgr2=0x7f3cfc102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:09.754 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.984+0000 7f3d00a68700 1 -- 172.21.15.16:0/3392466072 shutdown_connections 2024-04-02T04:33:09.754 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.984+0000 7f3d00a68700 1 -- 172.21.15.16:0/3392466072 wait complete. 2024-04-02T04:33:09.754 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.984+0000 7f3d00a68700 1 Processor -- start 2024-04-02T04:33:09.754 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.984+0000 7f3d00a68700 1 -- start start 2024-04-02T04:33:09.754 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.985+0000 7f3d00a68700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cfc106fc0 0x7f3cfc19a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:09.754 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.985+0000 7f3d00a68700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc19a730 con 0x7f3cfc106fc0 2024-04-02T04:33:09.754 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.985+0000 7f3cfa59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cfc106fc0 0x7f3cfc19a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:09.754 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.985+0000 7f3cfa59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cfc106fc0 0x7f3cfc19a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42132/0 (socket says 172.21.15.16:42132) 2024-04-02T04:33:09.754 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.985+0000 7f3cfa59c700 1 -- 172.21.15.16:0/2319371544 learned_addr learned my addr 172.21.15.16:0/2319371544 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:09.755 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.985+0000 7f3cfa59c700 1 -- 172.21.15.16:0/2319371544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf0009550 con 0x7f3cfc106fc0 2024-04-02T04:33:09.755 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.985+0000 7f3cfa59c700 1 --2- 172.21.15.16:0/2319371544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cfc106fc0 0x7f3cfc19a1f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0011710 tx=0x7f3cf00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:09.755 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.985+0000 7f3ceb7fe700 1 -- 172.21.15.16:0/2319371544 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3cf0003b10 con 0x7f3cfc106fc0 2024-04-02T04:33:09.755 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.985+0000 7f3ceb7fe700 1 -- 172.21.15.16:0/2319371544 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f3cf001b8c0 con 0x7f3cfc106fc0 2024-04-02T04:33:09.755 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.985+0000 7f3d00a68700 1 -- 172.21.15.16:0/2319371544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cfc19a930 con 0x7f3cfc106fc0 2024-04-02T04:33:09.755 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.985+0000 7f3ceb7fe700 1 -- 172.21.15.16:0/2319371544 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3cf00197e0 con 0x7f3cfc106fc0 2024-04-02T04:33:09.755 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.985+0000 7f3d00a68700 1 -- 172.21.15.16:0/2319371544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cfc19add0 con 0x7f3cfc106fc0 2024-04-02T04:33:09.755 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.986+0000 7f3ceb7fe700 1 -- 172.21.15.16:0/2319371544 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 45548+0+0 (secure 0 0 0) 0x7f3cf0019a20 con 0x7f3cfc106fc0 2024-04-02T04:33:09.755 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.986+0000 7f3d00a68700 1 -- 172.21.15.16:0/2319371544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cfc10a610 con 0x7f3cfc106fc0 2024-04-02T04:33:09.755 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.986+0000 7f3ceb7fe700 1 --2- 172.21.15.16:0/2319371544 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f3ce4038720 0x7f3ce403abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:09.755 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.987+0000 7f3ceb7fe700 1 -- 172.21.15.16:0/2319371544 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3cf004c930 con 0x7f3cfc106fc0 2024-04-02T04:33:09.755 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.987+0000 7f3cf9d9b700 1 --2- 172.21.15.16:0/2319371544 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f3ce4038720 0x7f3ce403abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:09.755 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.987+0000 7f3cf9d9b700 1 --2- 172.21.15.16:0/2319371544 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f3ce4038720 0x7f3ce403abe0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3cec006fd0 tx=0x7f3cec006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:09.755 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:08.991+0000 7f3ceb7fe700 1 -- 172.21.15.16:0/2319371544 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3cf002cb40 con 0x7f3cfc106fc0 2024-04-02T04:33:09.756 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:09.203+0000 7f3d00a68700 1 -- 172.21.15.16:0/2319371544 --> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f3cfc101bf0 con 0x7f3ce4038720 2024-04-02T04:33:09.756 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:09.208+0000 7f3ceb7fe700 1 -- 172.21.15.16:0/2319371544 <== mgr.14118 v2:172.21.15.16:6800/873531241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f3cfc101bf0 con 0x7f3ce4038720 2024-04-02T04:33:09.756 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:09.210+0000 7f3d00a68700 1 -- 172.21.15.16:0/2319371544 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f3ce4038720 msgr2=0x7f3ce403abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:09.756 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:09.210+0000 7f3d00a68700 1 --2- 172.21.15.16:0/2319371544 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f3ce4038720 0x7f3ce403abe0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3cec006fd0 tx=0x7f3cec006e40 comp rx=0 tx=0).stop 2024-04-02T04:33:09.756 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:09.210+0000 7f3d00a68700 1 -- 172.21.15.16:0/2319371544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cfc106fc0 msgr2=0x7f3cfc19a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:09.756 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:09.210+0000 7f3d00a68700 1 --2- 172.21.15.16:0/2319371544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cfc106fc0 0x7f3cfc19a1f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0011710 tx=0x7f3cf00047c0 comp rx=0 tx=0).stop 2024-04-02T04:33:09.756 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:09.210+0000 7f3d00a68700 1 -- 172.21.15.16:0/2319371544 shutdown_connections 2024-04-02T04:33:09.756 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:09.210+0000 7f3d00a68700 1 --2- 172.21.15.16:0/2319371544 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f3ce4038720 0x7f3ce403abe0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:09.756 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:09.210+0000 7f3d00a68700 1 --2- 172.21.15.16:0/2319371544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cfc106fc0 0x7f3cfc19a1f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:09.756 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:09.210+0000 7f3d00a68700 1 -- 172.21.15.16:0/2319371544 >> 172.21.15.16:0/2319371544 conn(0x7f3cfc1009b0 msgr2=0x7f3cfc1014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:09.756 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:09.210+0000 7f3d00a68700 1 -- 172.21.15.16:0/2319371544 shutdown_connections 2024-04-02T04:33:09.756 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:09.210+0000 7f3d00a68700 1 -- 172.21.15.16:0/2319371544 wait complete. 2024-04-02T04:33:09.756 INFO:teuthology.orchestra.run.smithi016.stdout:Deploying crash service with default placement... 2024-04-02T04:33:10.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:10 smithi016 ceph-mon[28969]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:33:10.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:10 smithi016 ceph-mon[28969]: Saving service mgr spec with placement count:2 2024-04-02T04:33:10.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:10 smithi016 ceph-mon[28969]: from='mgr.14118 172.21.15.16:0/872684120' entity='mgr.smithi016.eqnebx' 2024-04-02T04:33:11.378 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-04-02T04:33:11.378 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.566+0000 7f88737a8700 1 Processor -- start 2024-04-02T04:33:11.378 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.566+0000 7f88737a8700 1 -- start start 2024-04-02T04:33:11.378 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.566+0000 7f88737a8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f886c106510 0x7f886c106930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:11.378 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.566+0000 7f88737a8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886c106f00 con 0x7f886c106510 2024-04-02T04:33:11.378 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.566+0000 7f8871544700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f886c106510 0x7f886c106930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:11.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.566+0000 7f8871544700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f886c106510 0x7f886c106930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42142/0 (socket says 172.21.15.16:42142) 2024-04-02T04:33:11.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.566+0000 7f8871544700 1 -- 172.21.15.16:0/3916145552 learned_addr learned my addr 172.21.15.16:0/3916145552 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:11.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.567+0000 7f8871544700 1 -- 172.21.15.16:0/3916145552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f886c1076c0 con 0x7f886c106510 2024-04-02T04:33:11.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.567+0000 7f8871544700 1 --2- 172.21.15.16:0/3916145552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f886c106510 0x7f886c106930 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f88680098a0 tx=0x7f8868009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d2bacfa48493b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:11.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.567+0000 7f8863fff700 1 -- 172.21.15.16:0/3916145552 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8868004180 con 0x7f886c106510 2024-04-02T04:33:11.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.567+0000 7f8863fff700 1 -- 172.21.15.16:0/3916145552 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f88680042e0 con 0x7f886c106510 2024-04-02T04:33:11.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.567+0000 7f8863fff700 1 -- 172.21.15.16:0/3916145552 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8868011b10 con 0x7f886c106510 2024-04-02T04:33:11.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.568+0000 7f88737a8700 1 -- 172.21.15.16:0/3916145552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f886c106510 msgr2=0x7f886c106930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:11.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.568+0000 7f88737a8700 1 --2- 172.21.15.16:0/3916145552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f886c106510 0x7f886c106930 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f88680098a0 tx=0x7f8868009bb0 comp rx=0 tx=0).stop 2024-04-02T04:33:11.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.568+0000 7f88737a8700 1 -- 172.21.15.16:0/3916145552 shutdown_connections 2024-04-02T04:33:11.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.568+0000 7f88737a8700 1 --2- 172.21.15.16:0/3916145552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f886c106510 0x7f886c106930 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:11.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.568+0000 7f88737a8700 1 -- 172.21.15.16:0/3916145552 >> 172.21.15.16:0/3916145552 conn(0x7f886c101b70 msgr2=0x7f886c103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:11.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.568+0000 7f88737a8700 1 -- 172.21.15.16:0/3916145552 shutdown_connections 2024-04-02T04:33:11.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.568+0000 7f88737a8700 1 -- 172.21.15.16:0/3916145552 wait complete. 2024-04-02T04:33:11.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.568+0000 7f88737a8700 1 Processor -- start 2024-04-02T04:33:11.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.568+0000 7f88737a8700 1 -- start start 2024-04-02T04:33:11.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.569+0000 7f88737a8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f886c106510 0x7f886c19a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:11.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.569+0000 7f88737a8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886c19a7b0 con 0x7f886c106510 2024-04-02T04:33:11.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.569+0000 7f8871544700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f886c106510 0x7f886c19a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:11.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.569+0000 7f8871544700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f886c106510 0x7f886c19a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42152/0 (socket says 172.21.15.16:42152) 2024-04-02T04:33:11.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.569+0000 7f8871544700 1 -- 172.21.15.16:0/2823490639 learned_addr learned my addr 172.21.15.16:0/2823490639 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:11.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.569+0000 7f8871544700 1 -- 172.21.15.16:0/2823490639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8868009550 con 0x7f886c106510 2024-04-02T04:33:11.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.569+0000 7f8871544700 1 --2- 172.21.15.16:0/2823490639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f886c106510 0x7f886c19a270 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f88680117b0 tx=0x7f8868011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:11.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.569+0000 7f88627fc700 1 -- 172.21.15.16:0/2823490639 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f88680038b0 con 0x7f886c106510 2024-04-02T04:33:11.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.569+0000 7f88627fc700 1 -- 172.21.15.16:0/2823490639 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8868003a10 con 0x7f886c106510 2024-04-02T04:33:11.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.569+0000 7f88737a8700 1 -- 172.21.15.16:0/2823490639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f886c19a9b0 con 0x7f886c106510 2024-04-02T04:33:11.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.569+0000 7f88737a8700 1 -- 172.21.15.16:0/2823490639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f886c19ae50 con 0x7f886c106510 2024-04-02T04:33:11.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.569+0000 7f88627fc700 1 -- 172.21.15.16:0/2823490639 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f88680246e0 con 0x7f886c106510 2024-04-02T04:33:11.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.570+0000 7f88627fc700 1 -- 172.21.15.16:0/2823490639 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 45548+0+0 (secure 0 0 0) 0x7f886802d3f0 con 0x7f886c106510 2024-04-02T04:33:11.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.570+0000 7f88627fc700 1 --2- 172.21.15.16:0/2823490639 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f8858038720 0x7f885803abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:11.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.570+0000 7f88627fc700 1 -- 172.21.15.16:0/2823490639 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f886801ab80 con 0x7f886c106510 2024-04-02T04:33:11.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.571+0000 7f8870d43700 1 --2- 172.21.15.16:0/2823490639 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f8858038720 0x7f885803abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:11.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.571+0000 7f88737a8700 1 -- 172.21.15.16:0/2823490639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f886c0623e0 con 0x7f886c106510 2024-04-02T04:33:11.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.571+0000 7f8870d43700 1 --2- 172.21.15.16:0/2823490639 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f8858038720 0x7f885803abe0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f885c006fd0 tx=0x7f885c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:11.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.576+0000 7f88627fc700 1 -- 172.21.15.16:0/2823490639 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8868049080 con 0x7f886c106510 2024-04-02T04:33:11.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.808+0000 7f88737a8700 1 -- 172.21.15.16:0/2823490639 --> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7f886c10a850 con 0x7f8858038720 2024-04-02T04:33:11.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.813+0000 7f88627fc700 1 -- 172.21.15.16:0/2823490639 <== mgr.14118 v2:172.21.15.16:6800/873531241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f886c10a850 con 0x7f8858038720 2024-04-02T04:33:11.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.815+0000 7f88737a8700 1 -- 172.21.15.16:0/2823490639 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f8858038720 msgr2=0x7f885803abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:11.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.815+0000 7f88737a8700 1 --2- 172.21.15.16:0/2823490639 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f8858038720 0x7f885803abe0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f885c006fd0 tx=0x7f885c006e40 comp rx=0 tx=0).stop 2024-04-02T04:33:11.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.815+0000 7f88737a8700 1 -- 172.21.15.16:0/2823490639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f886c106510 msgr2=0x7f886c19a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:11.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.815+0000 7f88737a8700 1 --2- 172.21.15.16:0/2823490639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f886c106510 0x7f886c19a270 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f88680117b0 tx=0x7f8868011890 comp rx=0 tx=0).stop 2024-04-02T04:33:11.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.815+0000 7f88737a8700 1 -- 172.21.15.16:0/2823490639 shutdown_connections 2024-04-02T04:33:11.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.815+0000 7f88737a8700 1 --2- 172.21.15.16:0/2823490639 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f8858038720 0x7f885803abe0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:11.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.815+0000 7f88737a8700 1 --2- 172.21.15.16:0/2823490639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f886c106510 0x7f886c19a270 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:11.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.815+0000 7f88737a8700 1 -- 172.21.15.16:0/2823490639 >> 172.21.15.16:0/2823490639 conn(0x7f886c101b70 msgr2=0x7f886c102640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:11.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.815+0000 7f88737a8700 1 -- 172.21.15.16:0/2823490639 shutdown_connections 2024-04-02T04:33:11.382 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:10.815+0000 7f88737a8700 1 -- 172.21.15.16:0/2823490639 wait complete. 2024-04-02T04:33:11.382 INFO:teuthology.orchestra.run.smithi016.stdout:Deploying ceph-exporter service with default placement... 2024-04-02T04:33:11.503 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:11 smithi016 ceph-mon[28969]: from='mgr.14118 172.21.15.16:0/872684120' entity='mgr.smithi016.eqnebx' 2024-04-02T04:33:11.503 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:11 smithi016 ceph-mon[28969]: from='mgr.14118 172.21.15.16:0/872684120' entity='mgr.smithi016.eqnebx' 2024-04-02T04:33:11.503 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:11 smithi016 ceph-mon[28969]: from='mgr.14118 172.21.15.16:0/872684120' entity='mgr.smithi016.eqnebx' 2024-04-02T04:33:12.721 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:12 smithi016 ceph-mon[28969]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:33:12.721 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:12 smithi016 ceph-mon[28969]: Saving service crash spec with placement * 2024-04-02T04:33:12.722 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:12 smithi016 ceph-mon[28969]: from='mgr.14118 172.21.15.16:0/872684120' entity='mgr.smithi016.eqnebx' 2024-04-02T04:33:13.046 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Scheduled ceph-exporter update... 2024-04-02T04:33:13.046 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.787+0000 7f6644cf2700 1 Processor -- start 2024-04-02T04:33:13.046 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.787+0000 7f6644cf2700 1 -- start start 2024-04-02T04:33:13.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.788+0000 7f6644cf2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6640106510 0x7f6640106930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:13.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.788+0000 7f6644cf2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6640106f00 con 0x7f6640106510 2024-04-02T04:33:13.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.788+0000 7f663e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6640106510 0x7f6640106930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:13.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.788+0000 7f663e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6640106510 0x7f6640106930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47274/0 (socket says 172.21.15.16:47274) 2024-04-02T04:33:13.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.788+0000 7f663e59c700 1 -- 172.21.15.16:0/2789621457 learned_addr learned my addr 172.21.15.16:0/2789621457 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:13.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.788+0000 7f663e59c700 1 -- 172.21.15.16:0/2789621457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6640107710 con 0x7f6640106510 2024-04-02T04:33:13.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.788+0000 7f663e59c700 1 --2- 172.21.15.16:0/2789621457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6640106510 0x7f6640106930 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f66300098a0 tx=0x7f6630009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e18e43b4ffc8ead2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:13.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.788+0000 7f663d59a700 1 -- 172.21.15.16:0/2789621457 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6630004180 con 0x7f6640106510 2024-04-02T04:33:13.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.788+0000 7f663d59a700 1 -- 172.21.15.16:0/2789621457 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f66300042e0 con 0x7f6640106510 2024-04-02T04:33:13.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.789+0000 7f663d59a700 1 -- 172.21.15.16:0/2789621457 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6630011b10 con 0x7f6640106510 2024-04-02T04:33:13.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.789+0000 7f6644cf2700 1 -- 172.21.15.16:0/2789621457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6640106510 msgr2=0x7f6640106930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:13.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.789+0000 7f6644cf2700 1 --2- 172.21.15.16:0/2789621457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6640106510 0x7f6640106930 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f66300098a0 tx=0x7f6630009bb0 comp rx=0 tx=0).stop 2024-04-02T04:33:13.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.789+0000 7f6644cf2700 1 -- 172.21.15.16:0/2789621457 shutdown_connections 2024-04-02T04:33:13.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.789+0000 7f6644cf2700 1 --2- 172.21.15.16:0/2789621457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6640106510 0x7f6640106930 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:13.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.789+0000 7f6644cf2700 1 -- 172.21.15.16:0/2789621457 >> 172.21.15.16:0/2789621457 conn(0x7f6640101ab0 msgr2=0x7f6640103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:13.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.789+0000 7f6644cf2700 1 -- 172.21.15.16:0/2789621457 shutdown_connections 2024-04-02T04:33:13.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.789+0000 7f6644cf2700 1 -- 172.21.15.16:0/2789621457 wait complete. 2024-04-02T04:33:13.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.790+0000 7f6644cf2700 1 Processor -- start 2024-04-02T04:33:13.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.790+0000 7f6644cf2700 1 -- start start 2024-04-02T04:33:13.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.790+0000 7f6644cf2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6640106510 0x7f664019c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:13.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.790+0000 7f6644cf2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664019ca50 con 0x7f6640106510 2024-04-02T04:33:13.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.790+0000 7f663e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6640106510 0x7f664019c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:13.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.790+0000 7f663e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6640106510 0x7f664019c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47278/0 (socket says 172.21.15.16:47278) 2024-04-02T04:33:13.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.790+0000 7f663e59c700 1 -- 172.21.15.16:0/1136933199 learned_addr learned my addr 172.21.15.16:0/1136933199 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:13.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.791+0000 7f663e59c700 1 -- 172.21.15.16:0/1136933199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6630009550 con 0x7f6640106510 2024-04-02T04:33:13.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.791+0000 7f663e59c700 1 --2- 172.21.15.16:0/1136933199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6640106510 0x7f664019c510 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6630000c00 tx=0x7f6630011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:13.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.791+0000 7f662f7fe700 1 -- 172.21.15.16:0/1136933199 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f66300038b0 con 0x7f6640106510 2024-04-02T04:33:13.049 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.791+0000 7f662f7fe700 1 -- 172.21.15.16:0/1136933199 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6630003a10 con 0x7f6640106510 2024-04-02T04:33:13.049 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.791+0000 7f6644cf2700 1 -- 172.21.15.16:0/1136933199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f664019cc50 con 0x7f6640106510 2024-04-02T04:33:13.049 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.791+0000 7f662f7fe700 1 -- 172.21.15.16:0/1136933199 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6630024780 con 0x7f6640106510 2024-04-02T04:33:13.049 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.791+0000 7f6644cf2700 1 -- 172.21.15.16:0/1136933199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f664019d0f0 con 0x7f6640106510 2024-04-02T04:33:13.049 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.792+0000 7f662f7fe700 1 -- 172.21.15.16:0/1136933199 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 45548+0+0 (secure 0 0 0) 0x7f663002d3f0 con 0x7f6640106510 2024-04-02T04:33:13.049 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.792+0000 7f662f7fe700 1 --2- 172.21.15.16:0/1136933199 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f6628038720 0x7f662803abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:13.049 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.792+0000 7f662f7fe700 1 -- 172.21.15.16:0/1136933199 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f663001ac10 con 0x7f6640106510 2024-04-02T04:33:13.049 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.792+0000 7f663dd9b700 1 --2- 172.21.15.16:0/1136933199 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f6628038720 0x7f662803abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:13.049 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.793+0000 7f6644cf2700 1 -- 172.21.15.16:0/1136933199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66400623e0 con 0x7f6640106510 2024-04-02T04:33:13.049 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.793+0000 7f663dd9b700 1 --2- 172.21.15.16:0/1136933199 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f6628038720 0x7f662803abe0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6634006fd0 tx=0x7f6634006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:13.049 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:11.797+0000 7f662f7fe700 1 -- 172.21.15.16:0/1136933199 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6630019460 con 0x7f6640106510 2024-04-02T04:33:13.049 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:12.016+0000 7f6644cf2700 1 -- 172.21.15.16:0/1136933199 --> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f6640102df0 con 0x7f6628038720 2024-04-02T04:33:13.049 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:12.020+0000 7f662f7fe700 1 -- 172.21.15.16:0/1136933199 <== mgr.14118 v2:172.21.15.16:6800/873531241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f6640102df0 con 0x7f6628038720 2024-04-02T04:33:13.049 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:12.022+0000 7f6644cf2700 1 -- 172.21.15.16:0/1136933199 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f6628038720 msgr2=0x7f662803abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:13.050 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:12.022+0000 7f6644cf2700 1 --2- 172.21.15.16:0/1136933199 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f6628038720 0x7f662803abe0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6634006fd0 tx=0x7f6634006e40 comp rx=0 tx=0).stop 2024-04-02T04:33:13.050 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:12.022+0000 7f6644cf2700 1 -- 172.21.15.16:0/1136933199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6640106510 msgr2=0x7f664019c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:13.050 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:12.022+0000 7f6644cf2700 1 --2- 172.21.15.16:0/1136933199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6640106510 0x7f664019c510 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6630000c00 tx=0x7f6630011890 comp rx=0 tx=0).stop 2024-04-02T04:33:13.050 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:12.022+0000 7f6644cf2700 1 -- 172.21.15.16:0/1136933199 shutdown_connections 2024-04-02T04:33:13.050 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:12.023+0000 7f6644cf2700 1 --2- 172.21.15.16:0/1136933199 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f6628038720 0x7f662803abe0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:13.050 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:12.023+0000 7f6644cf2700 1 --2- 172.21.15.16:0/1136933199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6640106510 0x7f664019c510 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:13.050 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:12.023+0000 7f6644cf2700 1 -- 172.21.15.16:0/1136933199 >> 172.21.15.16:0/1136933199 conn(0x7f6640101ab0 msgr2=0x7f66401026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:13.050 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:12.023+0000 7f6644cf2700 1 -- 172.21.15.16:0/1136933199 shutdown_connections 2024-04-02T04:33:13.050 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:12.023+0000 7f6644cf2700 1 -- 172.21.15.16:0/1136933199 wait complete. 2024-04-02T04:33:13.050 INFO:teuthology.orchestra.run.smithi016.stdout:Deploying prometheus service with default placement... 2024-04-02T04:33:13.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:13 smithi016 ceph-mon[28969]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:33:13.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:13 smithi016 ceph-mon[28969]: Saving service ceph-exporter spec with placement * 2024-04-02T04:33:13.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:13 smithi016 ceph-mon[28969]: from='mgr.14118 172.21.15.16:0/872684120' entity='mgr.smithi016.eqnebx' 2024-04-02T04:33:14.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-04-02T04:33:14.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.497+0000 7fd33c06e700 1 Processor -- start 2024-04-02T04:33:14.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.497+0000 7fd33c06e700 1 -- start start 2024-04-02T04:33:14.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.497+0000 7fd33c06e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd334106f30 0x7fd334109320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:14.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.497+0000 7fd33c06e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3340745f0 con 0x7fd334106f30 2024-04-02T04:33:14.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.497+0000 7fd339e0a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd334106f30 0x7fd334109320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:14.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.497+0000 7fd339e0a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd334106f30 0x7fd334109320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47284/0 (socket says 172.21.15.16:47284) 2024-04-02T04:33:14.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.497+0000 7fd339e0a700 1 -- 172.21.15.16:0/4170285157 learned_addr learned my addr 172.21.15.16:0/4170285157 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:14.281 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.498+0000 7fd339e0a700 1 -- 172.21.15.16:0/4170285157 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd334109860 con 0x7fd334106f30 2024-04-02T04:33:14.281 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.498+0000 7fd339e0a700 1 --2- 172.21.15.16:0/4170285157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd334106f30 0x7fd334109320 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd3240098a0 tx=0x7fd324009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b89dbc2a14fc446 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:14.281 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.498+0000 7fd338e08700 1 -- 172.21.15.16:0/4170285157 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd324004180 con 0x7fd334106f30 2024-04-02T04:33:14.281 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.498+0000 7fd338e08700 1 -- 172.21.15.16:0/4170285157 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd3240042e0 con 0x7fd334106f30 2024-04-02T04:33:14.281 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.498+0000 7fd338e08700 1 -- 172.21.15.16:0/4170285157 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd324011b10 con 0x7fd334106f30 2024-04-02T04:33:14.281 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.499+0000 7fd33c06e700 1 -- 172.21.15.16:0/4170285157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd334106f30 msgr2=0x7fd334109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:14.281 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.499+0000 7fd33c06e700 1 --2- 172.21.15.16:0/4170285157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd334106f30 0x7fd334109320 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd3240098a0 tx=0x7fd324009bb0 comp rx=0 tx=0).stop 2024-04-02T04:33:14.281 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.499+0000 7fd33c06e700 1 -- 172.21.15.16:0/4170285157 shutdown_connections 2024-04-02T04:33:14.281 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.499+0000 7fd33c06e700 1 --2- 172.21.15.16:0/4170285157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd334106f30 0x7fd334109320 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:14.281 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.499+0000 7fd33c06e700 1 -- 172.21.15.16:0/4170285157 >> 172.21.15.16:0/4170285157 conn(0x7fd3341009c0 msgr2=0x7fd334102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:14.281 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.499+0000 7fd33c06e700 1 -- 172.21.15.16:0/4170285157 shutdown_connections 2024-04-02T04:33:14.281 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.499+0000 7fd33c06e700 1 -- 172.21.15.16:0/4170285157 wait complete. 2024-04-02T04:33:14.282 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.499+0000 7fd33c06e700 1 Processor -- start 2024-04-02T04:33:14.282 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.500+0000 7fd33c06e700 1 -- start start 2024-04-02T04:33:14.282 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.500+0000 7fd33c06e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd334106f30 0x7fd33419c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:14.282 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.500+0000 7fd33c06e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33419c980 con 0x7fd334106f30 2024-04-02T04:33:14.282 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.500+0000 7fd339e0a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd334106f30 0x7fd33419c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:14.282 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.500+0000 7fd339e0a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd334106f30 0x7fd33419c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47294/0 (socket says 172.21.15.16:47294) 2024-04-02T04:33:14.282 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.500+0000 7fd339e0a700 1 -- 172.21.15.16:0/2844032504 learned_addr learned my addr 172.21.15.16:0/2844032504 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:14.282 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.500+0000 7fd339e0a700 1 -- 172.21.15.16:0/2844032504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd324009550 con 0x7fd334106f30 2024-04-02T04:33:14.282 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.500+0000 7fd339e0a700 1 --2- 172.21.15.16:0/2844032504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd334106f30 0x7fd33419c440 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd324009fb0 tx=0x7fd324011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:14.282 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.500+0000 7fd32affd700 1 -- 172.21.15.16:0/2844032504 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd3240038b0 con 0x7fd334106f30 2024-04-02T04:33:14.282 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.500+0000 7fd33c06e700 1 -- 172.21.15.16:0/2844032504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd33419cb80 con 0x7fd334106f30 2024-04-02T04:33:14.282 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.500+0000 7fd32affd700 1 -- 172.21.15.16:0/2844032504 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd324003a10 con 0x7fd334106f30 2024-04-02T04:33:14.282 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.500+0000 7fd32affd700 1 -- 172.21.15.16:0/2844032504 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd324019750 con 0x7fd334106f30 2024-04-02T04:33:14.282 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.501+0000 7fd33c06e700 1 -- 172.21.15.16:0/2844032504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd33419d020 con 0x7fd334106f30 2024-04-02T04:33:14.283 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.501+0000 7fd32affd700 1 -- 172.21.15.16:0/2844032504 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 45548+0+0 (secure 0 0 0) 0x7fd32401a430 con 0x7fd334106f30 2024-04-02T04:33:14.283 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.502+0000 7fd33c06e700 1 -- 172.21.15.16:0/2844032504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd33410a650 con 0x7fd334106f30 2024-04-02T04:33:14.283 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.502+0000 7fd32affd700 1 --2- 172.21.15.16:0/2844032504 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fd320038730 0x7fd32003abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:14.283 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.502+0000 7fd32affd700 1 -- 172.21.15.16:0/2844032504 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd32404c840 con 0x7fd334106f30 2024-04-02T04:33:14.283 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.502+0000 7fd339609700 1 --2- 172.21.15.16:0/2844032504 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fd320038730 0x7fd32003abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:14.283 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.502+0000 7fd339609700 1 --2- 172.21.15.16:0/2844032504 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fd320038730 0x7fd32003abf0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd330006fd0 tx=0x7fd330006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:14.283 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.507+0000 7fd32affd700 1 -- 172.21.15.16:0/2844032504 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd324022310 con 0x7fd334106f30 2024-04-02T04:33:14.283 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.730+0000 7fd33c06e700 1 -- 172.21.15.16:0/2844032504 --> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7fd334101b80 con 0x7fd320038730 2024-04-02T04:33:14.283 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.734+0000 7fd32affd700 1 -- 172.21.15.16:0/2844032504 <== mgr.14118 v2:172.21.15.16:6800/873531241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7fd334101b80 con 0x7fd320038730 2024-04-02T04:33:14.284 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.737+0000 7fd33c06e700 1 -- 172.21.15.16:0/2844032504 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fd320038730 msgr2=0x7fd32003abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:14.284 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.737+0000 7fd33c06e700 1 --2- 172.21.15.16:0/2844032504 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fd320038730 0x7fd32003abf0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd330006fd0 tx=0x7fd330006e40 comp rx=0 tx=0).stop 2024-04-02T04:33:14.284 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.737+0000 7fd33c06e700 1 -- 172.21.15.16:0/2844032504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd334106f30 msgr2=0x7fd33419c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:14.284 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.737+0000 7fd33c06e700 1 --2- 172.21.15.16:0/2844032504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd334106f30 0x7fd33419c440 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd324009fb0 tx=0x7fd324011890 comp rx=0 tx=0).stop 2024-04-02T04:33:14.284 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.737+0000 7fd33c06e700 1 -- 172.21.15.16:0/2844032504 shutdown_connections 2024-04-02T04:33:14.284 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.737+0000 7fd33c06e700 1 --2- 172.21.15.16:0/2844032504 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fd320038730 0x7fd32003abf0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:14.284 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.737+0000 7fd33c06e700 1 --2- 172.21.15.16:0/2844032504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd334106f30 0x7fd33419c440 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:14.284 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.737+0000 7fd33c06e700 1 -- 172.21.15.16:0/2844032504 >> 172.21.15.16:0/2844032504 conn(0x7fd3341009c0 msgr2=0x7fd334101470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:14.284 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.737+0000 7fd33c06e700 1 -- 172.21.15.16:0/2844032504 shutdown_connections 2024-04-02T04:33:14.285 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:13.737+0000 7fd33c06e700 1 -- 172.21.15.16:0/2844032504 wait complete. 2024-04-02T04:33:14.285 INFO:teuthology.orchestra.run.smithi016.stdout:Deploying grafana service with default placement... 2024-04-02T04:33:14.878 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:14 smithi016 ceph-mon[28969]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:33:14.879 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:14 smithi016 ceph-mon[28969]: Saving service prometheus spec with placement count:1 2024-04-02T04:33:14.879 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:14 smithi016 ceph-mon[28969]: from='mgr.14118 172.21.15.16:0/872684120' entity='mgr.smithi016.eqnebx' 2024-04-02T04:33:14.879 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:14 smithi016 ceph-mon[28969]: from='mgr.14118 172.21.15.16:0/872684120' entity='mgr.smithi016.eqnebx' 2024-04-02T04:33:15.487 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-04-02T04:33:15.487 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.722+0000 7fcee349a700 1 Processor -- start 2024-04-02T04:33:15.488 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.722+0000 7fcee349a700 1 -- start start 2024-04-02T04:33:15.488 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.722+0000 7fcee349a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcedc07be00 0x7fcedc07a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:15.488 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.722+0000 7fcee349a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc07a840 con 0x7fcedc07be00 2024-04-02T04:33:15.488 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.723+0000 7fcee1236700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcedc07be00 0x7fcedc07a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:15.488 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.723+0000 7fcee1236700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcedc07be00 0x7fcedc07a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47298/0 (socket says 172.21.15.16:47298) 2024-04-02T04:33:15.488 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.723+0000 7fcee1236700 1 -- 172.21.15.16:0/1882895886 learned_addr learned my addr 172.21.15.16:0/1882895886 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:15.488 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.723+0000 7fcee1236700 1 -- 172.21.15.16:0/1882895886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcedc07a980 con 0x7fcedc07be00 2024-04-02T04:33:15.488 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.723+0000 7fcee1236700 1 --2- 172.21.15.16:0/1882895886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcedc07be00 0x7fcedc07a300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fced00098a0 tx=0x7fced0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c89ad15535fc04b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:15.488 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.723+0000 7fcecffff700 1 -- 172.21.15.16:0/1882895886 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fced0004180 con 0x7fcedc07be00 2024-04-02T04:33:15.488 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.723+0000 7fcecffff700 1 -- 172.21.15.16:0/1882895886 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fced00042e0 con 0x7fcedc07be00 2024-04-02T04:33:15.488 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.723+0000 7fcecffff700 1 -- 172.21.15.16:0/1882895886 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fced0011b10 con 0x7fcedc07be00 2024-04-02T04:33:15.488 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.724+0000 7fcee349a700 1 -- 172.21.15.16:0/1882895886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcedc07be00 msgr2=0x7fcedc07a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:15.488 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.724+0000 7fcee349a700 1 --2- 172.21.15.16:0/1882895886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcedc07be00 0x7fcedc07a300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fced00098a0 tx=0x7fced0009bb0 comp rx=0 tx=0).stop 2024-04-02T04:33:15.488 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.724+0000 7fcee349a700 1 -- 172.21.15.16:0/1882895886 shutdown_connections 2024-04-02T04:33:15.489 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.724+0000 7fcee349a700 1 --2- 172.21.15.16:0/1882895886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcedc07be00 0x7fcedc07a300 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:15.489 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.724+0000 7fcee349a700 1 -- 172.21.15.16:0/1882895886 >> 172.21.15.16:0/1882895886 conn(0x7fcedc101b10 msgr2=0x7fcedc103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:15.489 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.724+0000 7fcee349a700 1 -- 172.21.15.16:0/1882895886 shutdown_connections 2024-04-02T04:33:15.489 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.724+0000 7fcee349a700 1 -- 172.21.15.16:0/1882895886 wait complete. 2024-04-02T04:33:15.489 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.725+0000 7fcee349a700 1 Processor -- start 2024-04-02T04:33:15.489 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.725+0000 7fcee349a700 1 -- start start 2024-04-02T04:33:15.489 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.725+0000 7fcee349a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcedc07be00 0x7fcedc19c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:15.489 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.725+0000 7fcee349a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc19c960 con 0x7fcedc07be00 2024-04-02T04:33:15.489 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.725+0000 7fcee1236700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcedc07be00 0x7fcedc19c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:15.489 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.725+0000 7fcee1236700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcedc07be00 0x7fcedc19c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47300/0 (socket says 172.21.15.16:47300) 2024-04-02T04:33:15.489 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.725+0000 7fcee1236700 1 -- 172.21.15.16:0/2864564067 learned_addr learned my addr 172.21.15.16:0/2864564067 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:15.489 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.725+0000 7fcee1236700 1 -- 172.21.15.16:0/2864564067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced0009550 con 0x7fcedc07be00 2024-04-02T04:33:15.489 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.726+0000 7fcee1236700 1 --2- 172.21.15.16:0/2864564067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcedc07be00 0x7fcedc19c420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fced0009fb0 tx=0x7fced0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:15.489 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.726+0000 7fcece7fc700 1 -- 172.21.15.16:0/2864564067 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fced00038b0 con 0x7fcedc07be00 2024-04-02T04:33:15.490 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.726+0000 7fcee349a700 1 -- 172.21.15.16:0/2864564067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcedc19cb60 con 0x7fcedc07be00 2024-04-02T04:33:15.490 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.726+0000 7fcece7fc700 1 -- 172.21.15.16:0/2864564067 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fced0003a10 con 0x7fcedc07be00 2024-04-02T04:33:15.490 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.726+0000 7fcece7fc700 1 -- 172.21.15.16:0/2864564067 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fced0019750 con 0x7fcedc07be00 2024-04-02T04:33:15.490 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.726+0000 7fcee349a700 1 -- 172.21.15.16:0/2864564067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcedc19d000 con 0x7fcedc07be00 2024-04-02T04:33:15.490 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.727+0000 7fcee349a700 1 -- 172.21.15.16:0/2864564067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcedc10c7f0 con 0x7fcedc07be00 2024-04-02T04:33:15.490 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.727+0000 7fcece7fc700 1 -- 172.21.15.16:0/2864564067 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 45548+0+0 (secure 0 0 0) 0x7fced00198b0 con 0x7fcedc07be00 2024-04-02T04:33:15.490 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.727+0000 7fcece7fc700 1 --2- 172.21.15.16:0/2864564067 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fcec8038730 0x7fcec803abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:15.490 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.727+0000 7fcece7fc700 1 -- 172.21.15.16:0/2864564067 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fced004c900 con 0x7fcedc07be00 2024-04-02T04:33:15.490 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.727+0000 7fcee0a35700 1 --2- 172.21.15.16:0/2864564067 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fcec8038730 0x7fcec803abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:15.490 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.728+0000 7fcee0a35700 1 --2- 172.21.15.16:0/2864564067 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fcec8038730 0x7fcec803abf0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fced8006fd0 tx=0x7fced8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:15.490 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.732+0000 7fcece7fc700 1 -- 172.21.15.16:0/2864564067 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fced00226d0 con 0x7fcedc07be00 2024-04-02T04:33:15.490 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.958+0000 7fcee349a700 1 -- 172.21.15.16:0/2864564067 --> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7fcedc102dd0 con 0x7fcec8038730 2024-04-02T04:33:15.491 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.964+0000 7fcece7fc700 1 -- 172.21.15.16:0/2864564067 <== mgr.14118 v2:172.21.15.16:6800/873531241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fcedc102dd0 con 0x7fcec8038730 2024-04-02T04:33:15.491 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.966+0000 7fcee349a700 1 -- 172.21.15.16:0/2864564067 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fcec8038730 msgr2=0x7fcec803abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:15.491 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.966+0000 7fcee349a700 1 --2- 172.21.15.16:0/2864564067 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fcec8038730 0x7fcec803abf0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fced8006fd0 tx=0x7fced8006e40 comp rx=0 tx=0).stop 2024-04-02T04:33:15.491 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.966+0000 7fcee349a700 1 -- 172.21.15.16:0/2864564067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcedc07be00 msgr2=0x7fcedc19c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:15.491 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.966+0000 7fcee349a700 1 --2- 172.21.15.16:0/2864564067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcedc07be00 0x7fcedc19c420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fced0009fb0 tx=0x7fced0011890 comp rx=0 tx=0).stop 2024-04-02T04:33:15.491 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.966+0000 7fcee349a700 1 -- 172.21.15.16:0/2864564067 shutdown_connections 2024-04-02T04:33:15.491 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.966+0000 7fcee349a700 1 --2- 172.21.15.16:0/2864564067 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fcec8038730 0x7fcec803abf0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:15.491 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.966+0000 7fcee349a700 1 --2- 172.21.15.16:0/2864564067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcedc07be00 0x7fcedc19c420 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:15.491 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.966+0000 7fcee349a700 1 -- 172.21.15.16:0/2864564067 >> 172.21.15.16:0/2864564067 conn(0x7fcedc101b10 msgr2=0x7fcedc1026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:15.491 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.966+0000 7fcee349a700 1 -- 172.21.15.16:0/2864564067 shutdown_connections 2024-04-02T04:33:15.491 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:14.966+0000 7fcee349a700 1 -- 172.21.15.16:0/2864564067 wait complete. 2024-04-02T04:33:15.491 INFO:teuthology.orchestra.run.smithi016.stdout:Deploying node-exporter service with default placement... 2024-04-02T04:33:16.125 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:15 smithi016 ceph-mon[28969]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:33:16.125 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:15 smithi016 ceph-mon[28969]: Saving service grafana spec with placement count:1 2024-04-02T04:33:16.125 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:15 smithi016 ceph-mon[28969]: from='mgr.14118 172.21.15.16:0/872684120' entity='mgr.smithi016.eqnebx' 2024-04-02T04:33:16.838 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-04-02T04:33:16.838 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.913+0000 7fad567db700 1 Processor -- start 2024-04-02T04:33:16.838 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.913+0000 7fad567db700 1 -- start start 2024-04-02T04:33:16.838 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.913+0000 7fad567db700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad50106530 0x7fad50106950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:16.838 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.913+0000 7fad567db700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad50106f20 con 0x7fad50106530 2024-04-02T04:33:16.838 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.913+0000 7fad4ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad50106530 0x7fad50106950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:16.838 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.914+0000 7fad4ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad50106530 0x7fad50106950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47308/0 (socket says 172.21.15.16:47308) 2024-04-02T04:33:16.838 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.914+0000 7fad4ffff700 1 -- 172.21.15.16:0/3642285473 learned_addr learned my addr 172.21.15.16:0/3642285473 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:16.838 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.914+0000 7fad4ffff700 1 -- 172.21.15.16:0/3642285473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad50107690 con 0x7fad50106530 2024-04-02T04:33:16.838 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.914+0000 7fad4ffff700 1 --2- 172.21.15.16:0/3642285473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad50106530 0x7fad50106950 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fad400098a0 tx=0x7fad40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7d244851c11779 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:16.838 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.914+0000 7fad4effd700 1 -- 172.21.15.16:0/3642285473 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fad40004180 con 0x7fad50106530 2024-04-02T04:33:16.838 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.914+0000 7fad4effd700 1 -- 172.21.15.16:0/3642285473 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad400042e0 con 0x7fad50106530 2024-04-02T04:33:16.839 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.914+0000 7fad4effd700 1 -- 172.21.15.16:0/3642285473 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fad40011b10 con 0x7fad50106530 2024-04-02T04:33:16.839 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.915+0000 7fad567db700 1 -- 172.21.15.16:0/3642285473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad50106530 msgr2=0x7fad50106950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:16.839 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.915+0000 7fad567db700 1 --2- 172.21.15.16:0/3642285473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad50106530 0x7fad50106950 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fad400098a0 tx=0x7fad40009bb0 comp rx=0 tx=0).stop 2024-04-02T04:33:16.839 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.915+0000 7fad567db700 1 -- 172.21.15.16:0/3642285473 shutdown_connections 2024-04-02T04:33:16.839 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.915+0000 7fad567db700 1 --2- 172.21.15.16:0/3642285473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad50106530 0x7fad50106950 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:16.839 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.915+0000 7fad567db700 1 -- 172.21.15.16:0/3642285473 >> 172.21.15.16:0/3642285473 conn(0x7fad50101b10 msgr2=0x7fad50103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:16.839 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.915+0000 7fad567db700 1 -- 172.21.15.16:0/3642285473 shutdown_connections 2024-04-02T04:33:16.839 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.915+0000 7fad567db700 1 -- 172.21.15.16:0/3642285473 wait complete. 2024-04-02T04:33:16.839 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.915+0000 7fad567db700 1 Processor -- start 2024-04-02T04:33:16.839 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.915+0000 7fad567db700 1 -- start start 2024-04-02T04:33:16.839 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.916+0000 7fad567db700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad50106530 0x7fad5019a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:16.839 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.916+0000 7fad567db700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5019a7f0 con 0x7fad50106530 2024-04-02T04:33:16.839 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.916+0000 7fad4ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad50106530 0x7fad5019a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:16.839 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.916+0000 7fad4ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad50106530 0x7fad5019a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47320/0 (socket says 172.21.15.16:47320) 2024-04-02T04:33:16.839 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.916+0000 7fad4ffff700 1 -- 172.21.15.16:0/564291795 learned_addr learned my addr 172.21.15.16:0/564291795 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:16.840 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.916+0000 7fad4ffff700 1 -- 172.21.15.16:0/564291795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad40009550 con 0x7fad50106530 2024-04-02T04:33:16.840 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.916+0000 7fad4ffff700 1 --2- 172.21.15.16:0/564291795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad50106530 0x7fad5019a2b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fad40011670 tx=0x7fad40011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:16.840 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.916+0000 7fad4d7fa700 1 -- 172.21.15.16:0/564291795 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fad400038b0 con 0x7fad50106530 2024-04-02T04:33:16.840 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.916+0000 7fad4d7fa700 1 -- 172.21.15.16:0/564291795 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad40003a10 con 0x7fad50106530 2024-04-02T04:33:16.840 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.916+0000 7fad567db700 1 -- 172.21.15.16:0/564291795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad5019a9f0 con 0x7fad50106530 2024-04-02T04:33:16.840 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.916+0000 7fad4d7fa700 1 -- 172.21.15.16:0/564291795 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fad4001a7e0 con 0x7fad50106530 2024-04-02T04:33:16.840 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.916+0000 7fad567db700 1 -- 172.21.15.16:0/564291795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad5019ae90 con 0x7fad50106530 2024-04-02T04:33:16.840 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.917+0000 7fad4d7fa700 1 -- 172.21.15.16:0/564291795 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 45548+0+0 (secure 0 0 0) 0x7fad40024c10 con 0x7fad50106530 2024-04-02T04:33:16.840 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.917+0000 7fad567db700 1 -- 172.21.15.16:0/564291795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad5010a8c0 con 0x7fad50106530 2024-04-02T04:33:16.840 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.917+0000 7fad4d7fa700 1 --2- 172.21.15.16:0/564291795 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fad380386e0 0x7fad3803aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:16.840 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.918+0000 7fad4d7fa700 1 -- 172.21.15.16:0/564291795 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fad4004c920 con 0x7fad50106530 2024-04-02T04:33:16.840 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.918+0000 7fad4f7fe700 1 --2- 172.21.15.16:0/564291795 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fad380386e0 0x7fad3803aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:16.840 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.918+0000 7fad4f7fe700 1 --2- 172.21.15.16:0/564291795 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fad380386e0 0x7fad3803aba0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fad44006fd0 tx=0x7fad44006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:16.840 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:15.922+0000 7fad4d7fa700 1 -- 172.21.15.16:0/564291795 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fad40024430 con 0x7fad50106530 2024-04-02T04:33:16.840 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:16.144+0000 7fad567db700 1 -- 172.21.15.16:0/564291795 --> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7fad5019d720 con 0x7fad380386e0 2024-04-02T04:33:16.841 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:16.148+0000 7fad4d7fa700 1 -- 172.21.15.16:0/564291795 <== mgr.14118 v2:172.21.15.16:6800/873531241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7fad5019d720 con 0x7fad380386e0 2024-04-02T04:33:16.841 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:16.150+0000 7fad567db700 1 -- 172.21.15.16:0/564291795 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fad380386e0 msgr2=0x7fad3803aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:16.841 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:16.150+0000 7fad567db700 1 --2- 172.21.15.16:0/564291795 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fad380386e0 0x7fad3803aba0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fad44006fd0 tx=0x7fad44006e40 comp rx=0 tx=0).stop 2024-04-02T04:33:16.841 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:16.150+0000 7fad567db700 1 -- 172.21.15.16:0/564291795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad50106530 msgr2=0x7fad5019a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:16.841 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:16.150+0000 7fad567db700 1 --2- 172.21.15.16:0/564291795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad50106530 0x7fad5019a2b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fad40011670 tx=0x7fad40011890 comp rx=0 tx=0).stop 2024-04-02T04:33:16.841 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:16.150+0000 7fad567db700 1 -- 172.21.15.16:0/564291795 shutdown_connections 2024-04-02T04:33:16.841 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:16.150+0000 7fad567db700 1 --2- 172.21.15.16:0/564291795 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7fad380386e0 0x7fad3803aba0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:16.841 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:16.150+0000 7fad567db700 1 --2- 172.21.15.16:0/564291795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad50106530 0x7fad5019a2b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:16.841 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:16.150+0000 7fad567db700 1 -- 172.21.15.16:0/564291795 >> 172.21.15.16:0/564291795 conn(0x7fad50101b10 msgr2=0x7fad50102730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:16.841 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:16.150+0000 7fad567db700 1 -- 172.21.15.16:0/564291795 shutdown_connections 2024-04-02T04:33:16.841 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:16.150+0000 7fad567db700 1 -- 172.21.15.16:0/564291795 wait complete. 2024-04-02T04:33:16.841 INFO:teuthology.orchestra.run.smithi016.stdout:Deploying alertmanager service with default placement... 2024-04-02T04:33:17.495 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:17 smithi016 ceph-mon[28969]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:33:17.495 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:17 smithi016 ceph-mon[28969]: Saving service node-exporter spec with placement * 2024-04-02T04:33:17.495 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:17 smithi016 ceph-mon[28969]: from='mgr.14118 172.21.15.16:0/872684120' entity='mgr.smithi016.eqnebx' 2024-04-02T04:33:17.496 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:17 smithi016 ceph-mon[28969]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:33:18.146 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-04-02T04:33:18.146 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.264+0000 7f50abdb1700 1 Processor -- start 2024-04-02T04:33:18.146 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.264+0000 7f50abdb1700 1 -- start start 2024-04-02T04:33:18.146 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.264+0000 7f50abdb1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50a4106510 0x7f50a4106930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:18.146 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.264+0000 7f50abdb1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a4106f00 con 0x7f50a4106510 2024-04-02T04:33:18.147 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.265+0000 7f50a9b4d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50a4106510 0x7f50a4106930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:18.147 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.265+0000 7f50a9b4d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50a4106510 0x7f50a4106930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47328/0 (socket says 172.21.15.16:47328) 2024-04-02T04:33:18.147 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.265+0000 7f50a9b4d700 1 -- 172.21.15.16:0/322195325 learned_addr learned my addr 172.21.15.16:0/322195325 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:18.147 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.265+0000 7f50a9b4d700 1 -- 172.21.15.16:0/322195325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a41076c0 con 0x7f50a4106510 2024-04-02T04:33:18.147 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.265+0000 7f50a9b4d700 1 --2- 172.21.15.16:0/322195325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50a4106510 0x7f50a4106930 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f50a00098a0 tx=0x7f50a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c17abf0cbd41f610 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:18.147 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.265+0000 7f50a8b4b700 1 -- 172.21.15.16:0/322195325 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f50a0004180 con 0x7f50a4106510 2024-04-02T04:33:18.147 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.265+0000 7f50a8b4b700 1 -- 172.21.15.16:0/322195325 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f50a00042e0 con 0x7f50a4106510 2024-04-02T04:33:18.147 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.265+0000 7f50a8b4b700 1 -- 172.21.15.16:0/322195325 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f50a0011b10 con 0x7f50a4106510 2024-04-02T04:33:18.147 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.266+0000 7f50abdb1700 1 -- 172.21.15.16:0/322195325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50a4106510 msgr2=0x7f50a4106930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:18.147 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.266+0000 7f50abdb1700 1 --2- 172.21.15.16:0/322195325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50a4106510 0x7f50a4106930 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f50a00098a0 tx=0x7f50a0009bb0 comp rx=0 tx=0).stop 2024-04-02T04:33:18.147 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.266+0000 7f50abdb1700 1 -- 172.21.15.16:0/322195325 shutdown_connections 2024-04-02T04:33:18.147 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.266+0000 7f50abdb1700 1 --2- 172.21.15.16:0/322195325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50a4106510 0x7f50a4106930 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:18.147 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.266+0000 7f50abdb1700 1 -- 172.21.15.16:0/322195325 >> 172.21.15.16:0/322195325 conn(0x7f50a4101b10 msgr2=0x7f50a4103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:18.147 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.266+0000 7f50abdb1700 1 -- 172.21.15.16:0/322195325 shutdown_connections 2024-04-02T04:33:18.148 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.266+0000 7f50abdb1700 1 -- 172.21.15.16:0/322195325 wait complete. 2024-04-02T04:33:18.148 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.266+0000 7f50abdb1700 1 Processor -- start 2024-04-02T04:33:18.148 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.267+0000 7f50abdb1700 1 -- start start 2024-04-02T04:33:18.148 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.267+0000 7f50abdb1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50a4106510 0x7f50a407bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:18.148 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.267+0000 7f50abdb1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a407bff0 con 0x7f50a4106510 2024-04-02T04:33:18.148 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.267+0000 7f50a9b4d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50a4106510 0x7f50a407bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:18.148 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.267+0000 7f50a9b4d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50a4106510 0x7f50a407bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47338/0 (socket says 172.21.15.16:47338) 2024-04-02T04:33:18.148 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.267+0000 7f50a9b4d700 1 -- 172.21.15.16:0/3962424729 learned_addr learned my addr 172.21.15.16:0/3962424729 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:18.148 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.267+0000 7f50a9b4d700 1 -- 172.21.15.16:0/3962424729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a0009550 con 0x7f50a4106510 2024-04-02T04:33:18.148 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.267+0000 7f50a9b4d700 1 --2- 172.21.15.16:0/3962424729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50a4106510 0x7f50a407bab0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f50a0009fb0 tx=0x7f50a0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:18.148 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.267+0000 7f509affd700 1 -- 172.21.15.16:0/3962424729 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f50a00038b0 con 0x7f50a4106510 2024-04-02T04:33:18.148 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.267+0000 7f509affd700 1 -- 172.21.15.16:0/3962424729 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f50a0003a10 con 0x7f50a4106510 2024-04-02T04:33:18.148 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.267+0000 7f50abdb1700 1 -- 172.21.15.16:0/3962424729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a407a280 con 0x7f50a4106510 2024-04-02T04:33:18.148 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.267+0000 7f509affd700 1 -- 172.21.15.16:0/3962424729 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f50a0019770 con 0x7f50a4106510 2024-04-02T04:33:18.149 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.268+0000 7f50abdb1700 1 -- 172.21.15.16:0/3962424729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a407a720 con 0x7f50a4106510 2024-04-02T04:33:18.149 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.268+0000 7f509affd700 1 -- 172.21.15.16:0/3962424729 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 45548+0+0 (secure 0 0 0) 0x7f50a00198d0 con 0x7f50a4106510 2024-04-02T04:33:18.149 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.268+0000 7f509affd700 1 --2- 172.21.15.16:0/3962424729 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f50900386e0 0x7f509003aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:18.149 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.268+0000 7f509affd700 1 -- 172.21.15.16:0/3962424729 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f50a004c7c0 con 0x7f50a4106510 2024-04-02T04:33:18.149 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.269+0000 7f50abdb1700 1 -- 172.21.15.16:0/3962424729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50a407c130 con 0x7f50a4106510 2024-04-02T04:33:18.149 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.269+0000 7f50a934c700 1 --2- 172.21.15.16:0/3962424729 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f50900386e0 0x7f509003aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:18.149 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.269+0000 7f50a934c700 1 --2- 172.21.15.16:0/3962424729 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f50900386e0 0x7f509003aba0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5094006fd0 tx=0x7f5094006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:18.149 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.274+0000 7f509affd700 1 -- 172.21.15.16:0/3962424729 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f50a407c130 con 0x7f50a4106510 2024-04-02T04:33:18.149 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.491+0000 7f50abdb1700 1 -- 172.21.15.16:0/3962424729 --> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7f50a407c130 con 0x7f50900386e0 2024-04-02T04:33:18.149 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.495+0000 7f509affd700 1 -- 172.21.15.16:0/3962424729 <== mgr.14118 v2:172.21.15.16:6800/873531241 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f50a407c130 con 0x7f50900386e0 2024-04-02T04:33:18.149 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.497+0000 7f50abdb1700 1 -- 172.21.15.16:0/3962424729 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f50900386e0 msgr2=0x7f509003aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:18.149 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.497+0000 7f50abdb1700 1 --2- 172.21.15.16:0/3962424729 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f50900386e0 0x7f509003aba0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5094006fd0 tx=0x7f5094006e40 comp rx=0 tx=0).stop 2024-04-02T04:33:18.149 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.497+0000 7f50abdb1700 1 -- 172.21.15.16:0/3962424729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50a4106510 msgr2=0x7f50a407bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:18.149 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.497+0000 7f50abdb1700 1 --2- 172.21.15.16:0/3962424729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50a4106510 0x7f50a407bab0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f50a0009fb0 tx=0x7f50a0011890 comp rx=0 tx=0).stop 2024-04-02T04:33:18.150 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.497+0000 7f50abdb1700 1 -- 172.21.15.16:0/3962424729 shutdown_connections 2024-04-02T04:33:18.150 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.497+0000 7f50abdb1700 1 --2- 172.21.15.16:0/3962424729 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f50900386e0 0x7f509003aba0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:18.150 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.497+0000 7f50abdb1700 1 --2- 172.21.15.16:0/3962424729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50a4106510 0x7f50a407bab0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:18.150 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.497+0000 7f50abdb1700 1 -- 172.21.15.16:0/3962424729 >> 172.21.15.16:0/3962424729 conn(0x7f50a4101b10 msgr2=0x7f50a41026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:18.150 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.498+0000 7f50abdb1700 1 -- 172.21.15.16:0/3962424729 shutdown_connections 2024-04-02T04:33:18.150 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:17.498+0000 7f50abdb1700 1 -- 172.21.15.16:0/3962424729 wait complete. 2024-04-02T04:33:18.759 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:18 smithi016 ceph-mon[28969]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:33:18.759 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:18 smithi016 ceph-mon[28969]: Saving service alertmanager spec with placement count:1 2024-04-02T04:33:18.759 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:18 smithi016 ceph-mon[28969]: from='mgr.14118 172.21.15.16:0/872684120' entity='mgr.smithi016.eqnebx' 2024-04-02T04:33:19.202 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.545+0000 7f7da1220700 1 Processor -- start 2024-04-02T04:33:19.202 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.545+0000 7f7da1220700 1 -- start start 2024-04-02T04:33:19.202 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.545+0000 7f7da1220700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d9c104830 0x7f7d9c104c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:19.202 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.545+0000 7f7da1220700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9c105220 con 0x7f7d9c104830 2024-04-02T04:33:19.203 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.545+0000 7f7d9ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d9c104830 0x7f7d9c104c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:19.203 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.545+0000 7f7d9ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d9c104830 0x7f7d9c104c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47340/0 (socket says 172.21.15.16:47340) 2024-04-02T04:33:19.203 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.545+0000 7f7d9ad9d700 1 -- 172.21.15.16:0/2040190020 learned_addr learned my addr 172.21.15.16:0/2040190020 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:19.203 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.545+0000 7f7d9ad9d700 1 -- 172.21.15.16:0/2040190020 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d9c1077c0 con 0x7f7d9c104830 2024-04-02T04:33:19.203 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.545+0000 7f7d9ad9d700 1 --2- 172.21.15.16:0/2040190020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d9c104830 0x7f7d9c104c50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c0098a0 tx=0x7f7d8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4f36b247e340090 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:19.203 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.546+0000 7f7d99d9b700 1 -- 172.21.15.16:0/2040190020 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7d8c004180 con 0x7f7d9c104830 2024-04-02T04:33:19.203 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.546+0000 7f7d99d9b700 1 -- 172.21.15.16:0/2040190020 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d8c0042e0 con 0x7f7d9c104830 2024-04-02T04:33:19.203 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.546+0000 7f7d99d9b700 1 -- 172.21.15.16:0/2040190020 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7d8c011b10 con 0x7f7d9c104830 2024-04-02T04:33:19.203 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.546+0000 7f7da1220700 1 -- 172.21.15.16:0/2040190020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d9c104830 msgr2=0x7f7d9c104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:19.203 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.546+0000 7f7da1220700 1 --2- 172.21.15.16:0/2040190020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d9c104830 0x7f7d9c104c50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c0098a0 tx=0x7f7d8c009bb0 comp rx=0 tx=0).stop 2024-04-02T04:33:19.203 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.546+0000 7f7da1220700 1 -- 172.21.15.16:0/2040190020 shutdown_connections 2024-04-02T04:33:19.203 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.546+0000 7f7da1220700 1 --2- 172.21.15.16:0/2040190020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d9c104830 0x7f7d9c104c50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:19.203 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.546+0000 7f7da1220700 1 -- 172.21.15.16:0/2040190020 >> 172.21.15.16:0/2040190020 conn(0x7f7d9c1001a0 msgr2=0x7f7d9c1025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:19.203 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.546+0000 7f7da1220700 1 -- 172.21.15.16:0/2040190020 shutdown_connections 2024-04-02T04:33:19.203 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.546+0000 7f7da1220700 1 -- 172.21.15.16:0/2040190020 wait complete. 2024-04-02T04:33:19.203 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.547+0000 7f7da1220700 1 Processor -- start 2024-04-02T04:33:19.204 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.547+0000 7f7da1220700 1 -- start start 2024-04-02T04:33:19.204 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.547+0000 7f7da1220700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d9c104830 0x7f7d9c106ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:19.204 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.547+0000 7f7da1220700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9c107530 con 0x7f7d9c104830 2024-04-02T04:33:19.204 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.547+0000 7f7d9ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d9c104830 0x7f7d9c106ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:19.204 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.547+0000 7f7d9ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d9c104830 0x7f7d9c106ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47342/0 (socket says 172.21.15.16:47342) 2024-04-02T04:33:19.204 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.547+0000 7f7d9ad9d700 1 -- 172.21.15.16:0/2291604396 learned_addr learned my addr 172.21.15.16:0/2291604396 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:19.204 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.548+0000 7f7d9ad9d700 1 -- 172.21.15.16:0/2291604396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d8c009550 con 0x7f7d9c104830 2024-04-02T04:33:19.204 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.548+0000 7f7d9ad9d700 1 --2- 172.21.15.16:0/2291604396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d9c104830 0x7f7d9c106ff0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c0117b0 tx=0x7f7d8c011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:19.204 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.548+0000 7f7d83fff700 1 -- 172.21.15.16:0/2291604396 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7d8c0038b0 con 0x7f7d9c104830 2024-04-02T04:33:19.204 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.548+0000 7f7d83fff700 1 -- 172.21.15.16:0/2291604396 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d8c003a10 con 0x7f7d9c104830 2024-04-02T04:33:19.204 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.548+0000 7f7d83fff700 1 -- 172.21.15.16:0/2291604396 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7d8c0246e0 con 0x7f7d9c104830 2024-04-02T04:33:19.204 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.548+0000 7f7da1220700 1 -- 172.21.15.16:0/2291604396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d9c105770 con 0x7f7d9c104830 2024-04-02T04:33:19.204 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.548+0000 7f7da1220700 1 -- 172.21.15.16:0/2291604396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d9c105c10 con 0x7f7d9c104830 2024-04-02T04:33:19.204 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.549+0000 7f7d83fff700 1 -- 172.21.15.16:0/2291604396 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 45548+0+0 (secure 0 0 0) 0x7f7d8c02d3f0 con 0x7f7d9c104830 2024-04-02T04:33:19.204 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.549+0000 7f7da1220700 1 -- 172.21.15.16:0/2291604396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d9c0623e0 con 0x7f7d9c104830 2024-04-02T04:33:19.204 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.549+0000 7f7d83fff700 1 --2- 172.21.15.16:0/2291604396 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f7d84038770 0x7f7d8403ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:19.204 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.549+0000 7f7d83fff700 1 -- 172.21.15.16:0/2291604396 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7d8c04c820 con 0x7f7d9c104830 2024-04-02T04:33:19.205 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.549+0000 7f7d9a59c700 1 --2- 172.21.15.16:0/2291604396 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f7d84038770 0x7f7d8403ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:19.205 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.550+0000 7f7d9a59c700 1 --2- 172.21.15.16:0/2291604396 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f7d84038770 0x7f7d8403ac30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7d90006fd0 tx=0x7f7d90006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:19.205 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.554+0000 7f7d83fff700 1 -- 172.21.15.16:0/2291604396 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7d8c010bd0 con 0x7f7d9c104830 2024-04-02T04:33:19.205 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.755+0000 7f7da1220700 1 -- 172.21.15.16:0/2291604396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f7d9c105f10 con 0x7f7d9c104830 2024-04-02T04:33:19.205 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.758+0000 7f7d83fff700 1 -- 172.21.15.16:0/2291604396 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f7d8c0107f0 con 0x7f7d9c104830 2024-04-02T04:33:19.205 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.760+0000 7f7da1220700 1 -- 172.21.15.16:0/2291604396 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f7d84038770 msgr2=0x7f7d8403ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:19.205 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.760+0000 7f7da1220700 1 --2- 172.21.15.16:0/2291604396 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f7d84038770 0x7f7d8403ac30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7d90006fd0 tx=0x7f7d90006e40 comp rx=0 tx=0).stop 2024-04-02T04:33:19.205 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.760+0000 7f7da1220700 1 -- 172.21.15.16:0/2291604396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d9c104830 msgr2=0x7f7d9c106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:19.205 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.760+0000 7f7da1220700 1 --2- 172.21.15.16:0/2291604396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d9c104830 0x7f7d9c106ff0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c0117b0 tx=0x7f7d8c011890 comp rx=0 tx=0).stop 2024-04-02T04:33:19.205 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.760+0000 7f7da1220700 1 -- 172.21.15.16:0/2291604396 shutdown_connections 2024-04-02T04:33:19.205 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.760+0000 7f7da1220700 1 --2- 172.21.15.16:0/2291604396 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f7d84038770 0x7f7d8403ac30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:19.205 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.760+0000 7f7da1220700 1 --2- 172.21.15.16:0/2291604396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d9c104830 0x7f7d9c106ff0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:19.205 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.760+0000 7f7da1220700 1 -- 172.21.15.16:0/2291604396 >> 172.21.15.16:0/2291604396 conn(0x7f7d9c1001a0 msgr2=0x7f7d9c100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:19.205 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.760+0000 7f7da1220700 1 -- 172.21.15.16:0/2291604396 shutdown_connections 2024-04-02T04:33:19.205 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:18.760+0000 7f7da1220700 1 -- 172.21.15.16:0/2291604396 wait complete. 2024-04-02T04:33:19.809 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:19 smithi016 ceph-mon[28969]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:33:19.809 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:19 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/2291604396' entity='client.admin' 2024-04-02T04:33:20.375 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.623+0000 7f692a6fd700 1 Processor -- start 2024-04-02T04:33:20.375 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.623+0000 7f692a6fd700 1 -- start start 2024-04-02T04:33:20.375 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.623+0000 7f692a6fd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6924106f60 0x7f6924109350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:20.375 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.623+0000 7f692a6fd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6924079e90 con 0x7f6924106f60 2024-04-02T04:33:20.375 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.623+0000 7f6923fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6924106f60 0x7f6924109350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:20.375 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.623+0000 7f6923fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6924106f60 0x7f6924109350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47354/0 (socket says 172.21.15.16:47354) 2024-04-02T04:33:20.375 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.623+0000 7f6923fff700 1 -- 172.21.15.16:0/4145622159 learned_addr learned my addr 172.21.15.16:0/4145622159 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:20.376 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.623+0000 7f6923fff700 1 -- 172.21.15.16:0/4145622159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6924109890 con 0x7f6924106f60 2024-04-02T04:33:20.376 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.624+0000 7f6923fff700 1 --2- 172.21.15.16:0/4145622159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6924106f60 0x7f6924109350 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f69140098a0 tx=0x7f6914009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad81ec15a36459f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:20.376 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.624+0000 7f6922ffd700 1 -- 172.21.15.16:0/4145622159 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6914004180 con 0x7f6924106f60 2024-04-02T04:33:20.376 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.624+0000 7f6922ffd700 1 -- 172.21.15.16:0/4145622159 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f69140042e0 con 0x7f6924106f60 2024-04-02T04:33:20.376 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.624+0000 7f6922ffd700 1 -- 172.21.15.16:0/4145622159 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6914011b10 con 0x7f6924106f60 2024-04-02T04:33:20.376 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.624+0000 7f692a6fd700 1 -- 172.21.15.16:0/4145622159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6924106f60 msgr2=0x7f6924109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:20.376 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.624+0000 7f692a6fd700 1 --2- 172.21.15.16:0/4145622159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6924106f60 0x7f6924109350 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f69140098a0 tx=0x7f6914009bb0 comp rx=0 tx=0).stop 2024-04-02T04:33:20.376 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.624+0000 7f692a6fd700 1 -- 172.21.15.16:0/4145622159 shutdown_connections 2024-04-02T04:33:20.376 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.624+0000 7f692a6fd700 1 --2- 172.21.15.16:0/4145622159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6924106f60 0x7f6924109350 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:20.376 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.624+0000 7f692a6fd700 1 -- 172.21.15.16:0/4145622159 >> 172.21.15.16:0/4145622159 conn(0x7f69241009d0 msgr2=0x7f6924102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:20.376 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.624+0000 7f692a6fd700 1 -- 172.21.15.16:0/4145622159 shutdown_connections 2024-04-02T04:33:20.376 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.624+0000 7f692a6fd700 1 -- 172.21.15.16:0/4145622159 wait complete. 2024-04-02T04:33:20.377 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.625+0000 7f692a6fd700 1 Processor -- start 2024-04-02T04:33:20.377 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.625+0000 7f692a6fd700 1 -- start start 2024-04-02T04:33:20.377 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.625+0000 7f692a6fd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6924106f60 0x7f6924106790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:20.377 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.625+0000 7f692a6fd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6924106cd0 con 0x7f6924106f60 2024-04-02T04:33:20.377 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.625+0000 7f6923fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6924106f60 0x7f6924106790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:20.377 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.625+0000 7f6923fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6924106f60 0x7f6924106790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47364/0 (socket says 172.21.15.16:47364) 2024-04-02T04:33:20.377 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.625+0000 7f6923fff700 1 -- 172.21.15.16:0/1227479655 learned_addr learned my addr 172.21.15.16:0/1227479655 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:20.377 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.626+0000 7f6923fff700 1 -- 172.21.15.16:0/1227479655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6914009550 con 0x7f6924106f60 2024-04-02T04:33:20.377 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.626+0000 7f6923fff700 1 --2- 172.21.15.16:0/1227479655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6924106f60 0x7f6924106790 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6914009fb0 tx=0x7f6914011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:20.377 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.626+0000 7f69217fa700 1 -- 172.21.15.16:0/1227479655 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f69140038b0 con 0x7f6924106f60 2024-04-02T04:33:20.377 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.626+0000 7f69217fa700 1 -- 172.21.15.16:0/1227479655 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6914003a10 con 0x7f6924106f60 2024-04-02T04:33:20.377 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.626+0000 7f692a6fd700 1 -- 172.21.15.16:0/1227479655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6924104f10 con 0x7f6924106f60 2024-04-02T04:33:20.377 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.626+0000 7f69217fa700 1 -- 172.21.15.16:0/1227479655 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6914019750 con 0x7f6924106f60 2024-04-02T04:33:20.377 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.626+0000 7f692a6fd700 1 -- 172.21.15.16:0/1227479655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69241053b0 con 0x7f6924106f60 2024-04-02T04:33:20.378 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.627+0000 7f69217fa700 1 -- 172.21.15.16:0/1227479655 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 45548+0+0 (secure 0 0 0) 0x7f691401ac10 con 0x7f6924106f60 2024-04-02T04:33:20.378 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.627+0000 7f692a6fd700 1 -- 172.21.15.16:0/1227479655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69240623e0 con 0x7f6924106f60 2024-04-02T04:33:20.378 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.627+0000 7f69217fa700 1 --2- 172.21.15.16:0/1227479655 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f690c0386e0 0x7f690c03aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:20.378 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.627+0000 7f69217fa700 1 -- 172.21.15.16:0/1227479655 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f691404c8e0 con 0x7f6924106f60 2024-04-02T04:33:20.378 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.627+0000 7f69237fe700 1 --2- 172.21.15.16:0/1227479655 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f690c0386e0 0x7f690c03aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:20.378 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.628+0000 7f69237fe700 1 --2- 172.21.15.16:0/1227479655 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f690c0386e0 0x7f690c03aba0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6918006fd0 tx=0x7f6918006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:20.378 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.632+0000 7f69217fa700 1 -- 172.21.15.16:0/1227479655 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f69140199c0 con 0x7f6924106f60 2024-04-02T04:33:20.378 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.831+0000 7f692a6fd700 1 -- 172.21.15.16:0/1227479655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f6924105760 con 0x7f6924106f60 2024-04-02T04:33:20.378 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.834+0000 7f69217fa700 1 -- 172.21.15.16:0/1227479655 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f6914018350 con 0x7f6924106f60 2024-04-02T04:33:20.378 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.836+0000 7f692a6fd700 1 -- 172.21.15.16:0/1227479655 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f690c0386e0 msgr2=0x7f690c03aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:20.378 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.836+0000 7f692a6fd700 1 --2- 172.21.15.16:0/1227479655 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f690c0386e0 0x7f690c03aba0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6918006fd0 tx=0x7f6918006e40 comp rx=0 tx=0).stop 2024-04-02T04:33:20.378 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.836+0000 7f692a6fd700 1 -- 172.21.15.16:0/1227479655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6924106f60 msgr2=0x7f6924106790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:20.378 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.836+0000 7f692a6fd700 1 --2- 172.21.15.16:0/1227479655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6924106f60 0x7f6924106790 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6914009fb0 tx=0x7f6914011890 comp rx=0 tx=0).stop 2024-04-02T04:33:20.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.836+0000 7f692a6fd700 1 -- 172.21.15.16:0/1227479655 shutdown_connections 2024-04-02T04:33:20.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.836+0000 7f692a6fd700 1 --2- 172.21.15.16:0/1227479655 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f690c0386e0 0x7f690c03aba0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:20.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.836+0000 7f692a6fd700 1 --2- 172.21.15.16:0/1227479655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6924106f60 0x7f6924106790 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:20.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.836+0000 7f692a6fd700 1 -- 172.21.15.16:0/1227479655 >> 172.21.15.16:0/1227479655 conn(0x7f69241009d0 msgr2=0x7f69241014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:20.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.836+0000 7f692a6fd700 1 -- 172.21.15.16:0/1227479655 shutdown_connections 2024-04-02T04:33:20.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:19.836+0000 7f692a6fd700 1 -- 172.21.15.16:0/1227479655 wait complete. 2024-04-02T04:33:20.379 INFO:teuthology.orchestra.run.smithi016.stdout:Enabling the dashboard module... 2024-04-02T04:33:21.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:20 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/1227479655' entity='client.admin' 2024-04-02T04:33:21.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:20 smithi016 ceph-mon[28969]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:33:22.198 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:21 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/3008556740' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-02T04:33:22.465 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.752+0000 7f179267a700 1 Processor -- start 2024-04-02T04:33:22.465 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.752+0000 7f179267a700 1 -- start start 2024-04-02T04:33:22.465 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.752+0000 7f179267a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f178c106550 0x7f178c106970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:22.465 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.752+0000 7f179267a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178c106f40 con 0x7f178c106550 2024-04-02T04:33:22.465 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.752+0000 7f178bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f178c106550 0x7f178c106970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:22.465 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.752+0000 7f178bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f178c106550 0x7f178c106970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47374/0 (socket says 172.21.15.16:47374) 2024-04-02T04:33:22.465 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.752+0000 7f178bfff700 1 -- 172.21.15.16:0/883774565 learned_addr learned my addr 172.21.15.16:0/883774565 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:22.466 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.753+0000 7f178bfff700 1 -- 172.21.15.16:0/883774565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f178c1076b0 con 0x7f178c106550 2024-04-02T04:33:22.466 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.753+0000 7f178bfff700 1 --2- 172.21.15.16:0/883774565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f178c106550 0x7f178c106970 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f177c0098a0 tx=0x7f177c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c400b2348367ba37 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:22.466 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.753+0000 7f178affd700 1 -- 172.21.15.16:0/883774565 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f177c004180 con 0x7f178c106550 2024-04-02T04:33:22.466 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.753+0000 7f178affd700 1 -- 172.21.15.16:0/883774565 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f177c0042e0 con 0x7f178c106550 2024-04-02T04:33:22.466 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.753+0000 7f178affd700 1 -- 172.21.15.16:0/883774565 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f177c011b10 con 0x7f178c106550 2024-04-02T04:33:22.466 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.753+0000 7f179267a700 1 -- 172.21.15.16:0/883774565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f178c106550 msgr2=0x7f178c106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:22.466 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.753+0000 7f179267a700 1 --2- 172.21.15.16:0/883774565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f178c106550 0x7f178c106970 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f177c0098a0 tx=0x7f177c009bb0 comp rx=0 tx=0).stop 2024-04-02T04:33:22.466 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.753+0000 7f179267a700 1 -- 172.21.15.16:0/883774565 shutdown_connections 2024-04-02T04:33:22.466 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.753+0000 7f179267a700 1 --2- 172.21.15.16:0/883774565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f178c106550 0x7f178c106970 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:22.466 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.753+0000 7f179267a700 1 -- 172.21.15.16:0/883774565 >> 172.21.15.16:0/883774565 conn(0x7f178c101b10 msgr2=0x7f178c103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:22.467 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.754+0000 7f179267a700 1 -- 172.21.15.16:0/883774565 shutdown_connections 2024-04-02T04:33:22.467 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.754+0000 7f179267a700 1 -- 172.21.15.16:0/883774565 wait complete. 2024-04-02T04:33:22.467 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.754+0000 7f179267a700 1 Processor -- start 2024-04-02T04:33:22.467 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.754+0000 7f179267a700 1 -- start start 2024-04-02T04:33:22.467 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.754+0000 7f179267a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f178c106550 0x7f178c19a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:22.467 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.754+0000 7f179267a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178c19a770 con 0x7f178c106550 2024-04-02T04:33:22.468 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.755+0000 7f178bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f178c106550 0x7f178c19a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:22.468 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.755+0000 7f178bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f178c106550 0x7f178c19a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47390/0 (socket says 172.21.15.16:47390) 2024-04-02T04:33:22.468 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.755+0000 7f178bfff700 1 -- 172.21.15.16:0/3008556740 learned_addr learned my addr 172.21.15.16:0/3008556740 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:22.468 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.755+0000 7f178bfff700 1 -- 172.21.15.16:0/3008556740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f177c009550 con 0x7f178c106550 2024-04-02T04:33:22.468 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.755+0000 7f178bfff700 1 --2- 172.21.15.16:0/3008556740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f178c106550 0x7f178c19a230 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f177c0117b0 tx=0x7f177c011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:22.468 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.755+0000 7f17897fa700 1 -- 172.21.15.16:0/3008556740 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f177c0038b0 con 0x7f178c106550 2024-04-02T04:33:22.468 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.755+0000 7f17897fa700 1 -- 172.21.15.16:0/3008556740 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f177c003a10 con 0x7f178c106550 2024-04-02T04:33:22.468 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.755+0000 7f17897fa700 1 -- 172.21.15.16:0/3008556740 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f177c0246e0 con 0x7f178c106550 2024-04-02T04:33:22.468 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.755+0000 7f179267a700 1 -- 172.21.15.16:0/3008556740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f178c19a970 con 0x7f178c106550 2024-04-02T04:33:22.468 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.755+0000 7f179267a700 1 -- 172.21.15.16:0/3008556740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f178c19ae10 con 0x7f178c106550 2024-04-02T04:33:22.468 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.756+0000 7f17897fa700 1 -- 172.21.15.16:0/3008556740 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 45548+0+0 (secure 0 0 0) 0x7f177c01a430 con 0x7f178c106550 2024-04-02T04:33:22.468 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.756+0000 7f17897fa700 1 --2- 172.21.15.16:0/3008556740 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f1778038770 0x7f177803ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:22.468 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.756+0000 7f179267a700 1 -- 172.21.15.16:0/3008556740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f178c10a8e0 con 0x7f178c106550 2024-04-02T04:33:22.468 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.756+0000 7f17897fa700 1 -- 172.21.15.16:0/3008556740 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f177c04c800 con 0x7f178c106550 2024-04-02T04:33:22.469 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.756+0000 7f178b7fe700 1 --2- 172.21.15.16:0/3008556740 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f1778038770 0x7f177803ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:22.469 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.757+0000 7f178b7fe700 1 --2- 172.21.15.16:0/3008556740 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f1778038770 0x7f177803ac30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1774006fd0 tx=0x7f1774006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:22.469 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:20.762+0000 7f17897fa700 1 -- 172.21.15.16:0/3008556740 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f177c018b40 con 0x7f178c106550 2024-04-02T04:33:22.469 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:21.022+0000 7f179267a700 1 -- 172.21.15.16:0/3008556740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f178c19d6a0 con 0x7f178c106550 2024-04-02T04:33:22.469 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:21.841+0000 7f17897fa700 1 -- 172.21.15.16:0/3008556740 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 9) v1 ==== 45561+0+0 (secure 0 0 0) 0x7f177c01ab80 con 0x7f178c106550 2024-04-02T04:33:22.469 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:21.842+0000 7f17897fa700 1 -- 172.21.15.16:0/3008556740 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f177c02d910 con 0x7f178c106550 2024-04-02T04:33:22.469 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:21.846+0000 7f179267a700 1 -- 172.21.15.16:0/3008556740 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f1778038770 msgr2=0x7f177803ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:22.469 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:21.846+0000 7f179267a700 1 --2- 172.21.15.16:0/3008556740 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f1778038770 0x7f177803ac30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1774006fd0 tx=0x7f1774006e40 comp rx=0 tx=0).stop 2024-04-02T04:33:22.469 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:21.846+0000 7f179267a700 1 -- 172.21.15.16:0/3008556740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f178c106550 msgr2=0x7f178c19a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:22.469 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:21.846+0000 7f179267a700 1 --2- 172.21.15.16:0/3008556740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f178c106550 0x7f178c19a230 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f177c0117b0 tx=0x7f177c011890 comp rx=0 tx=0).stop 2024-04-02T04:33:22.469 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:21.846+0000 7f179267a700 1 -- 172.21.15.16:0/3008556740 shutdown_connections 2024-04-02T04:33:22.469 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:21.846+0000 7f179267a700 1 --2- 172.21.15.16:0/3008556740 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f1778038770 0x7f177803ac30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:22.469 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:21.846+0000 7f179267a700 1 --2- 172.21.15.16:0/3008556740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f178c106550 0x7f178c19a230 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:22.469 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:21.846+0000 7f179267a700 1 -- 172.21.15.16:0/3008556740 >> 172.21.15.16:0/3008556740 conn(0x7f178c101b10 msgr2=0x7f178c102690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:22.470 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:21.847+0000 7f179267a700 1 -- 172.21.15.16:0/3008556740 shutdown_connections 2024-04-02T04:33:22.470 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:21.847+0000 7f179267a700 1 -- 172.21.15.16:0/3008556740 wait complete. 2024-04-02T04:33:23.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:22 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/3008556740' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-02T04:33:23.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:22 smithi016 ceph-mon[28969]: mgrmap e9: smithi016.eqnebx(active, since 25s) 2024-04-02T04:33:23.772 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-04-02T04:33:23.772 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-02T04:33:23.772 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-02T04:33:23.772 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "active_name": "smithi016.eqnebx", 2024-04-02T04:33:23.772 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-02T04:33:23.772 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-04-02T04:33:23.772 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.874+0000 7f0d99565700 1 Processor -- start 2024-04-02T04:33:23.772 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.874+0000 7f0d99565700 1 -- start start 2024-04-02T04:33:23.772 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.874+0000 7f0d99565700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d9407be10 0x7f0d9407a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:23.772 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.874+0000 7f0d99565700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9407a800 con 0x7f0d9407be10 2024-04-02T04:33:23.772 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.874+0000 7f0d92ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d9407be10 0x7f0d9407a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:23.773 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.874+0000 7f0d92ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d9407be10 0x7f0d9407a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39172/0 (socket says 172.21.15.16:39172) 2024-04-02T04:33:23.773 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.874+0000 7f0d92ffd700 1 -- 172.21.15.16:0/825636456 learned_addr learned my addr 172.21.15.16:0/825636456 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:23.773 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.874+0000 7f0d92ffd700 1 -- 172.21.15.16:0/825636456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d9407a940 con 0x7f0d9407be10 2024-04-02T04:33:23.773 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.874+0000 7f0d92ffd700 1 --2- 172.21.15.16:0/825636456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d9407be10 0x7f0d9407a2c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0d840098a0 tx=0x7f0d84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d094a38c5ab0f53 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:23.773 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.875+0000 7f0d91ffb700 1 -- 172.21.15.16:0/825636456 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0d84004180 con 0x7f0d9407be10 2024-04-02T04:33:23.773 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.875+0000 7f0d91ffb700 1 -- 172.21.15.16:0/825636456 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d840042e0 con 0x7f0d9407be10 2024-04-02T04:33:23.773 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.875+0000 7f0d91ffb700 1 -- 172.21.15.16:0/825636456 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0d84011b10 con 0x7f0d9407be10 2024-04-02T04:33:23.773 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.875+0000 7f0d99565700 1 -- 172.21.15.16:0/825636456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d9407be10 msgr2=0x7f0d9407a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:23.773 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.875+0000 7f0d99565700 1 --2- 172.21.15.16:0/825636456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d9407be10 0x7f0d9407a2c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0d840098a0 tx=0x7f0d84009bb0 comp rx=0 tx=0).stop 2024-04-02T04:33:23.773 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.875+0000 7f0d99565700 1 -- 172.21.15.16:0/825636456 shutdown_connections 2024-04-02T04:33:23.773 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.875+0000 7f0d99565700 1 --2- 172.21.15.16:0/825636456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d9407be10 0x7f0d9407a2c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:23.773 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.875+0000 7f0d99565700 1 -- 172.21.15.16:0/825636456 >> 172.21.15.16:0/825636456 conn(0x7f0d94101b10 msgr2=0x7f0d94103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:23.773 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.875+0000 7f0d99565700 1 -- 172.21.15.16:0/825636456 shutdown_connections 2024-04-02T04:33:23.774 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.875+0000 7f0d99565700 1 -- 172.21.15.16:0/825636456 wait complete. 2024-04-02T04:33:23.774 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.876+0000 7f0d99565700 1 Processor -- start 2024-04-02T04:33:23.774 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.876+0000 7f0d99565700 1 -- start start 2024-04-02T04:33:23.774 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.876+0000 7f0d99565700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d9407be10 0x7f0d9419c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:23.774 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.876+0000 7f0d99565700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9419c9c0 con 0x7f0d9407be10 2024-04-02T04:33:23.774 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.876+0000 7f0d92ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d9407be10 0x7f0d9419c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:23.774 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.876+0000 7f0d92ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d9407be10 0x7f0d9419c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39180/0 (socket says 172.21.15.16:39180) 2024-04-02T04:33:23.774 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.876+0000 7f0d92ffd700 1 -- 172.21.15.16:0/1805249350 learned_addr learned my addr 172.21.15.16:0/1805249350 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:23.774 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.877+0000 7f0d92ffd700 1 -- 172.21.15.16:0/1805249350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d84009550 con 0x7f0d9407be10 2024-04-02T04:33:23.774 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.877+0000 7f0d92ffd700 1 --2- 172.21.15.16:0/1805249350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d9407be10 0x7f0d9419c480 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0d840117b0 tx=0x7f0d84011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:23.774 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.877+0000 7f0d7bfff700 1 -- 172.21.15.16:0/1805249350 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0d840038b0 con 0x7f0d9407be10 2024-04-02T04:33:23.774 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.877+0000 7f0d7bfff700 1 -- 172.21.15.16:0/1805249350 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d84003a10 con 0x7f0d9407be10 2024-04-02T04:33:23.774 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.877+0000 7f0d99565700 1 -- 172.21.15.16:0/1805249350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d9419cbc0 con 0x7f0d9407be10 2024-04-02T04:33:23.775 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.877+0000 7f0d7bfff700 1 -- 172.21.15.16:0/1805249350 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0d84024780 con 0x7f0d9407be10 2024-04-02T04:33:23.775 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.877+0000 7f0d99565700 1 -- 172.21.15.16:0/1805249350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d9419d060 con 0x7f0d9407be10 2024-04-02T04:33:23.775 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.878+0000 7f0d7bfff700 1 -- 172.21.15.16:0/1805249350 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 45561+0+0 (secure 0 0 0) 0x7f0d840248e0 con 0x7f0d9407be10 2024-04-02T04:33:23.775 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.878+0000 7f0d99565700 1 -- 172.21.15.16:0/1805249350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d9410c830 con 0x7f0d9407be10 2024-04-02T04:33:23.775 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.878+0000 7f0d7bfff700 1 --2- 172.21.15.16:0/1805249350 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f0d7c038770 0x7f0d7c03ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:23.775 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.878+0000 7f0d7bfff700 1 -- 172.21.15.16:0/1805249350 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0d8404c7c0 con 0x7f0d9407be10 2024-04-02T04:33:23.775 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.878+0000 7f0d927fc700 1 -- 172.21.15.16:0/1805249350 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f0d7c038770 msgr2=0x7f0d7c03ac30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/873531241 2024-04-02T04:33:23.775 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.878+0000 7f0d927fc700 1 --2- 172.21.15.16:0/1805249350 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f0d7c038770 0x7f0d7c03ac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T04:33:23.775 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:22.883+0000 7f0d7bfff700 1 -- 172.21.15.16:0/1805249350 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0d84010380 con 0x7f0d9407be10 2024-04-02T04:33:23.775 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:23.079+0000 7f0d927fc700 1 -- 172.21.15.16:0/1805249350 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f0d7c038770 msgr2=0x7f0d7c03ac30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/873531241 2024-04-02T04:33:23.776 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:23.079+0000 7f0d927fc700 1 --2- 172.21.15.16:0/1805249350 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f0d7c038770 0x7f0d7c03ac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T04:33:23.776 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:23.143+0000 7f0d99565700 1 -- 172.21.15.16:0/1805249350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f0d9419f900 con 0x7f0d9407be10 2024-04-02T04:33:23.776 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:23.143+0000 7f0d7bfff700 1 -- 172.21.15.16:0/1805249350 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7f0d84019e40 con 0x7f0d9407be10 2024-04-02T04:33:23.776 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:23.145+0000 7f0d99565700 1 -- 172.21.15.16:0/1805249350 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f0d7c038770 msgr2=0x7f0d7c03ac30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T04:33:23.776 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:23.145+0000 7f0d99565700 1 --2- 172.21.15.16:0/1805249350 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f0d7c038770 0x7f0d7c03ac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:23.776 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:23.145+0000 7f0d99565700 1 -- 172.21.15.16:0/1805249350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d9407be10 msgr2=0x7f0d9419c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:23.776 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:23.145+0000 7f0d99565700 1 --2- 172.21.15.16:0/1805249350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d9407be10 0x7f0d9419c480 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0d840117b0 tx=0x7f0d84011890 comp rx=0 tx=0).stop 2024-04-02T04:33:23.776 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:23.145+0000 7f0d99565700 1 -- 172.21.15.16:0/1805249350 shutdown_connections 2024-04-02T04:33:23.776 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:23.145+0000 7f0d99565700 1 --2- 172.21.15.16:0/1805249350 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f0d7c038770 0x7f0d7c03ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:23.776 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:23.145+0000 7f0d99565700 1 --2- 172.21.15.16:0/1805249350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d9407be10 0x7f0d9419c480 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:23.776 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:23.145+0000 7f0d99565700 1 -- 172.21.15.16:0/1805249350 >> 172.21.15.16:0/1805249350 conn(0x7f0d94101b10 msgr2=0x7f0d941026a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:23.776 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:23.145+0000 7f0d99565700 1 -- 172.21.15.16:0/1805249350 shutdown_connections 2024-04-02T04:33:23.776 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:23.145+0000 7f0d99565700 1 -- 172.21.15.16:0/1805249350 wait complete. 2024-04-02T04:33:23.776 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for the mgr to restart... 2024-04-02T04:33:23.777 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr epoch 9... 2024-04-02T04:33:24.030 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:23 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/1805249350' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-02T04:33:29.913 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:29 smithi016 ceph-mon[28969]: Active manager daemon smithi016.eqnebx restarted 2024-04-02T04:33:29.914 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:29 smithi016 ceph-mon[28969]: Activating manager daemon smithi016.eqnebx 2024-04-02T04:33:29.914 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:29 smithi016 ceph-mon[28969]: osdmap e3: 0 total, 0 up, 0 in 2024-04-02T04:33:29.914 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:29 smithi016 ceph-mon[28969]: mgrmap e10: smithi016.eqnebx(active, starting, since 0.0032847s) 2024-04-02T04:33:29.914 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:29 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi016"}]: dispatch 2024-04-02T04:33:29.914 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:29 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mgr metadata", "who": "smithi016.eqnebx", "id": "smithi016.eqnebx"}]: dispatch 2024-04-02T04:33:29.914 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:29 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T04:33:29.914 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:29 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T04:33:29.914 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:29 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T04:33:29.914 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:29 smithi016 ceph-mon[28969]: Manager daemon smithi016.eqnebx is now available 2024-04-02T04:33:29.915 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:29 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:33:29.915 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:29 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi016.eqnebx/mirror_snapshot_schedule"}]: dispatch 2024-04-02T04:33:29.915 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:29 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi016.eqnebx/trash_purge_schedule"}]: dispatch 2024-04-02T04:33:30.641 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-04-02T04:33:30.641 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-02T04:33:30.641 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-02T04:33:30.641 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-04-02T04:33:30.641 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.147+0000 7f7dd7c5f700 1 Processor -- start 2024-04-02T04:33:30.642 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.147+0000 7f7dd7c5f700 1 -- start start 2024-04-02T04:33:30.642 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.148+0000 7f7dd7c5f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dd007ad70 0x7f7dd00791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:30.642 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.148+0000 7f7dd7c5f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd00797a0 con 0x7f7dd007ad70 2024-04-02T04:33:30.642 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.148+0000 7f7dd59fb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dd007ad70 0x7f7dd00791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:30.642 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.148+0000 7f7dd59fb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dd007ad70 0x7f7dd00791d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39190/0 (socket says 172.21.15.16:39190) 2024-04-02T04:33:30.642 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.148+0000 7f7dd59fb700 1 -- 172.21.15.16:0/1387980965 learned_addr learned my addr 172.21.15.16:0/1387980965 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:30.642 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.148+0000 7f7dd59fb700 1 -- 172.21.15.16:0/1387980965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd00798e0 con 0x7f7dd007ad70 2024-04-02T04:33:30.642 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.148+0000 7f7dd59fb700 1 --2- 172.21.15.16:0/1387980965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dd007ad70 0x7f7dd00791d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7dc00098a0 tx=0x7f7dc0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a58e7343c2adf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:30.642 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.148+0000 7f7dd49f9700 1 -- 172.21.15.16:0/1387980965 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7dc0004180 con 0x7f7dd007ad70 2024-04-02T04:33:30.642 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.148+0000 7f7dd49f9700 1 -- 172.21.15.16:0/1387980965 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7dc00042e0 con 0x7f7dd007ad70 2024-04-02T04:33:30.642 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.148+0000 7f7dd49f9700 1 -- 172.21.15.16:0/1387980965 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7dc0011b10 con 0x7f7dd007ad70 2024-04-02T04:33:30.643 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.149+0000 7f7dd7c5f700 1 -- 172.21.15.16:0/1387980965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dd007ad70 msgr2=0x7f7dd00791d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:30.643 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.149+0000 7f7dd7c5f700 1 --2- 172.21.15.16:0/1387980965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dd007ad70 0x7f7dd00791d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7dc00098a0 tx=0x7f7dc0009bb0 comp rx=0 tx=0).stop 2024-04-02T04:33:30.643 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.149+0000 7f7dd7c5f700 1 -- 172.21.15.16:0/1387980965 shutdown_connections 2024-04-02T04:33:30.643 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.149+0000 7f7dd7c5f700 1 --2- 172.21.15.16:0/1387980965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dd007ad70 0x7f7dd00791d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:30.643 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.149+0000 7f7dd7c5f700 1 -- 172.21.15.16:0/1387980965 >> 172.21.15.16:0/1387980965 conn(0x7f7dd0101af0 msgr2=0x7f7dd0103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:30.643 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.149+0000 7f7dd7c5f700 1 -- 172.21.15.16:0/1387980965 shutdown_connections 2024-04-02T04:33:30.643 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.149+0000 7f7dd7c5f700 1 -- 172.21.15.16:0/1387980965 wait complete. 2024-04-02T04:33:30.643 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.150+0000 7f7dd7c5f700 1 Processor -- start 2024-04-02T04:33:30.643 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.150+0000 7f7dd7c5f700 1 -- start start 2024-04-02T04:33:30.643 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.150+0000 7f7dd7c5f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dd007ad70 0x7f7dd019c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:30.643 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.150+0000 7f7dd7c5f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd019c9d0 con 0x7f7dd007ad70 2024-04-02T04:33:30.643 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.150+0000 7f7dd59fb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dd007ad70 0x7f7dd019c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:30.643 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.150+0000 7f7dd59fb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dd007ad70 0x7f7dd019c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39200/0 (socket says 172.21.15.16:39200) 2024-04-02T04:33:30.643 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.150+0000 7f7dd59fb700 1 -- 172.21.15.16:0/1987105928 learned_addr learned my addr 172.21.15.16:0/1987105928 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:30.643 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.151+0000 7f7dd59fb700 1 -- 172.21.15.16:0/1987105928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc0009550 con 0x7f7dd007ad70 2024-04-02T04:33:30.644 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.151+0000 7f7dd59fb700 1 --2- 172.21.15.16:0/1987105928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dd007ad70 0x7f7dd019c490 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7dc00117b0 tx=0x7f7dc0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:30.644 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.151+0000 7f7dc6ffd700 1 -- 172.21.15.16:0/1987105928 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7dc00038b0 con 0x7f7dd007ad70 2024-04-02T04:33:30.644 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.151+0000 7f7dc6ffd700 1 -- 172.21.15.16:0/1987105928 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7dc0003a10 con 0x7f7dd007ad70 2024-04-02T04:33:30.644 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.151+0000 7f7dd7c5f700 1 -- 172.21.15.16:0/1987105928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dd019cbd0 con 0x7f7dd007ad70 2024-04-02T04:33:30.644 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.151+0000 7f7dc6ffd700 1 -- 172.21.15.16:0/1987105928 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7dc00246e0 con 0x7f7dd007ad70 2024-04-02T04:33:30.644 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.151+0000 7f7dd7c5f700 1 -- 172.21.15.16:0/1987105928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dd019d070 con 0x7f7dd007ad70 2024-04-02T04:33:30.644 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.152+0000 7f7dc6ffd700 1 -- 172.21.15.16:0/1987105928 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 45561+0+0 (secure 0 0 0) 0x7f7dc002d3f0 con 0x7f7dd007ad70 2024-04-02T04:33:30.644 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.152+0000 7f7dc6ffd700 1 --2- 172.21.15.16:0/1987105928 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f7dbc0387c0 0x7f7dbc03ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:30.644 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.152+0000 7f7dc6ffd700 1 -- 172.21.15.16:0/1987105928 --> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7dbc03b390 con 0x7f7dbc0387c0 2024-04-02T04:33:30.644 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.152+0000 7f7dc6ffd700 1 -- 172.21.15.16:0/1987105928 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7dc004ca90 con 0x7f7dd007ad70 2024-04-02T04:33:30.644 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.152+0000 7f7dd51fa700 1 -- 172.21.15.16:0/1987105928 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f7dbc0387c0 msgr2=0x7f7dbc03ac80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/873531241 2024-04-02T04:33:30.644 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.152+0000 7f7dd51fa700 1 --2- 172.21.15.16:0/1987105928 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f7dbc0387c0 0x7f7dbc03ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T04:33:30.644 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.353+0000 7f7dd51fa700 1 -- 172.21.15.16:0/1987105928 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f7dbc0387c0 msgr2=0x7f7dbc03ac80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/873531241 2024-04-02T04:33:30.644 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.353+0000 7f7dd51fa700 1 --2- 172.21.15.16:0/1987105928 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f7dbc0387c0 0x7f7dbc03ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T04:33:30.645 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.754+0000 7f7dd51fa700 1 -- 172.21.15.16:0/1987105928 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f7dbc0387c0 msgr2=0x7f7dbc03ac80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/873531241 2024-04-02T04:33:30.645 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:24.754+0000 7f7dd51fa700 1 --2- 172.21.15.16:0/1987105928 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f7dbc0387c0 0x7f7dbc03ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-02T04:33:30.645 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:25.555+0000 7f7dd51fa700 1 -- 172.21.15.16:0/1987105928 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f7dbc0387c0 msgr2=0x7f7dbc03ac80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/873531241 2024-04-02T04:33:30.645 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:25.555+0000 7f7dd51fa700 1 --2- 172.21.15.16:0/1987105928 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f7dbc0387c0 0x7f7dbc03ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-02T04:33:30.645 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:27.156+0000 7f7dd51fa700 1 -- 172.21.15.16:0/1987105928 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f7dbc0387c0 msgr2=0x7f7dbc03ac80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/873531241 2024-04-02T04:33:30.645 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:27.156+0000 7f7dd51fa700 1 --2- 172.21.15.16:0/1987105928 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f7dbc0387c0 0x7f7dbc03ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-02T04:33:30.645 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:28.908+0000 7f7dc6ffd700 1 -- 172.21.15.16:0/1987105928 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mgrmap(e 10) v1 ==== 45328+0+0 (secure 0 0 0) 0x7f7dc0021d60 con 0x7f7dd007ad70 2024-04-02T04:33:30.645 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:28.908+0000 7f7dc6ffd700 1 -- 172.21.15.16:0/1987105928 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f7dbc0387c0 msgr2=0x7f7dbc03ac80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T04:33:30.645 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:28.908+0000 7f7dc6ffd700 1 --2- 172.21.15.16:0/1987105928 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f7dbc0387c0 0x7f7dbc03ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:30.645 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:29.909+0000 7f7dc6ffd700 1 -- 172.21.15.16:0/1987105928 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 11) v1 ==== 45455+0+0 (secure 0 0 0) 0x7f7dc0024cf0 con 0x7f7dd007ad70 2024-04-02T04:33:30.645 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:29.910+0000 7f7dc6ffd700 1 --2- 172.21.15.16:0/1987105928 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f7dbc03bfc0 0x7f7dbc03e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:30.645 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:29.910+0000 7f7dc6ffd700 1 -- 172.21.15.16:0/1987105928 --> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7dbc03b390 con 0x7f7dbc03bfc0 2024-04-02T04:33:30.645 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:29.910+0000 7f7dd51fa700 1 --2- 172.21.15.16:0/1987105928 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f7dbc03bfc0 0x7f7dbc03e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:30.645 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:29.910+0000 7f7dd51fa700 1 --2- 172.21.15.16:0/1987105928 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f7dbc03bfc0 0x7f7dbc03e3b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc003a10 tx=0x7f7dcc009270 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:30.645 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:29.911+0000 7f7dc6ffd700 1 -- 172.21.15.16:0/1987105928 <== mgr.14162 v2:172.21.15.16:6800/3912038174 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f7dbc03b390 con 0x7f7dbc03bfc0 2024-04-02T04:33:30.646 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:29.918+0000 7f7dd7c5f700 1 -- 172.21.15.16:0/1987105928 --> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f7dd010c8b0 con 0x7f7dbc03bfc0 2024-04-02T04:33:30.646 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:29.918+0000 7f7dc6ffd700 1 -- 172.21.15.16:0/1987105928 <== mgr.14162 v2:172.21.15.16:6800/3912038174 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f7dd010c8b0 con 0x7f7dbc03bfc0 2024-04-02T04:33:30.646 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:29.918+0000 7f7dd7c5f700 1 -- 172.21.15.16:0/1987105928 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f7dbc03bfc0 msgr2=0x7f7dbc03e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:30.646 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:29.918+0000 7f7dd7c5f700 1 --2- 172.21.15.16:0/1987105928 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f7dbc03bfc0 0x7f7dbc03e3b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc003a10 tx=0x7f7dcc009270 comp rx=0 tx=0).stop 2024-04-02T04:33:30.646 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:29.918+0000 7f7dd7c5f700 1 -- 172.21.15.16:0/1987105928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dd007ad70 msgr2=0x7f7dd019c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:30.646 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:29.918+0000 7f7dd7c5f700 1 --2- 172.21.15.16:0/1987105928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dd007ad70 0x7f7dd019c490 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7dc00117b0 tx=0x7f7dc0011890 comp rx=0 tx=0).stop 2024-04-02T04:33:30.646 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:29.918+0000 7f7dd7c5f700 1 -- 172.21.15.16:0/1987105928 shutdown_connections 2024-04-02T04:33:30.646 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:29.918+0000 7f7dd7c5f700 1 --2- 172.21.15.16:0/1987105928 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f7dbc03bfc0 0x7f7dbc03e3b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:30.646 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:29.918+0000 7f7dd7c5f700 1 --2- 172.21.15.16:0/1987105928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dd007ad70 0x7f7dd019c490 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:30.646 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:29.918+0000 7f7dd7c5f700 1 --2- 172.21.15.16:0/1987105928 >> [v2:172.21.15.16:6800/873531241,v1:172.21.15.16:6801/873531241] conn(0x7f7dbc0387c0 0x7f7dbc03ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:30.646 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:29.918+0000 7f7dd7c5f700 1 -- 172.21.15.16:0/1987105928 >> 172.21.15.16:0/1987105928 conn(0x7f7dd0101af0 msgr2=0x7f7dd0102680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:30.646 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:29.919+0000 7f7dd7c5f700 1 -- 172.21.15.16:0/1987105928 shutdown_connections 2024-04-02T04:33:30.647 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:29.919+0000 7f7dd7c5f700 1 -- 172.21.15.16:0/1987105928 wait complete. 2024-04-02T04:33:30.647 INFO:teuthology.orchestra.run.smithi016.stdout:mgr epoch 9 is available 2024-04-02T04:33:30.647 INFO:teuthology.orchestra.run.smithi016.stdout:Generating a dashboard self-signed certificate... 2024-04-02T04:33:30.926 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:30 smithi016 ceph-mon[28969]: [02/Apr/2024:04:33:29] ENGINE Bus STARTING 2024-04-02T04:33:30.926 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:30 smithi016 ceph-mon[28969]: [02/Apr/2024:04:33:29] ENGINE Serving on https://172.21.15.16:7150 2024-04-02T04:33:30.926 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:30 smithi016 ceph-mon[28969]: mgrmap e11: smithi016.eqnebx(active, since 1.00571s) 2024-04-02T04:33:31.946 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-02T04:33:31.946 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.026+0000 7efea2cd2700 1 Processor -- start 2024-04-02T04:33:31.946 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.026+0000 7efea2cd2700 1 -- start start 2024-04-02T04:33:31.946 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.026+0000 7efea2cd2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe9c106f70 0x7efe9c109360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:31.946 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.026+0000 7efea2cd2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c0745b0 con 0x7efe9c106f70 2024-04-02T04:33:31.946 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.026+0000 7efea0a6e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe9c106f70 0x7efe9c109360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:31.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.026+0000 7efea0a6e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe9c106f70 0x7efe9c109360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39272/0 (socket says 172.21.15.16:39272) 2024-04-02T04:33:31.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.026+0000 7efea0a6e700 1 -- 172.21.15.16:0/1754923553 learned_addr learned my addr 172.21.15.16:0/1754923553 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:31.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.027+0000 7efea0a6e700 1 -- 172.21.15.16:0/1754923553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe9c1098a0 con 0x7efe9c106f70 2024-04-02T04:33:31.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.027+0000 7efea0a6e700 1 --2- 172.21.15.16:0/1754923553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe9c106f70 0x7efe9c109360 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efe900098a0 tx=0x7efe90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32234e580f4c642a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:31.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.027+0000 7efe9b7fe700 1 -- 172.21.15.16:0/1754923553 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efe90004180 con 0x7efe9c106f70 2024-04-02T04:33:31.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.027+0000 7efe9b7fe700 1 -- 172.21.15.16:0/1754923553 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe900042e0 con 0x7efe9c106f70 2024-04-02T04:33:31.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.027+0000 7efe9b7fe700 1 -- 172.21.15.16:0/1754923553 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efe90011b10 con 0x7efe9c106f70 2024-04-02T04:33:31.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.028+0000 7efea2cd2700 1 -- 172.21.15.16:0/1754923553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe9c106f70 msgr2=0x7efe9c109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:31.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.028+0000 7efea2cd2700 1 --2- 172.21.15.16:0/1754923553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe9c106f70 0x7efe9c109360 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efe900098a0 tx=0x7efe90009bb0 comp rx=0 tx=0).stop 2024-04-02T04:33:31.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.028+0000 7efea2cd2700 1 -- 172.21.15.16:0/1754923553 shutdown_connections 2024-04-02T04:33:31.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.028+0000 7efea2cd2700 1 --2- 172.21.15.16:0/1754923553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe9c106f70 0x7efe9c109360 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:31.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.028+0000 7efea2cd2700 1 -- 172.21.15.16:0/1754923553 >> 172.21.15.16:0/1754923553 conn(0x7efe9c1009c0 msgr2=0x7efe9c102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:31.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.028+0000 7efea2cd2700 1 -- 172.21.15.16:0/1754923553 shutdown_connections 2024-04-02T04:33:31.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.028+0000 7efea2cd2700 1 -- 172.21.15.16:0/1754923553 wait complete. 2024-04-02T04:33:31.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.028+0000 7efea2cd2700 1 Processor -- start 2024-04-02T04:33:31.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.028+0000 7efea2cd2700 1 -- start start 2024-04-02T04:33:31.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.029+0000 7efea2cd2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe9c106f70 0x7efe9c1067a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:31.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.029+0000 7efea2cd2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c106ce0 con 0x7efe9c106f70 2024-04-02T04:33:31.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.029+0000 7efea0a6e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe9c106f70 0x7efe9c1067a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:31.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.029+0000 7efea0a6e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe9c106f70 0x7efe9c1067a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39276/0 (socket says 172.21.15.16:39276) 2024-04-02T04:33:31.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.029+0000 7efea0a6e700 1 -- 172.21.15.16:0/2964727724 learned_addr learned my addr 172.21.15.16:0/2964727724 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:31.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.029+0000 7efea0a6e700 1 -- 172.21.15.16:0/2964727724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe90009550 con 0x7efe9c106f70 2024-04-02T04:33:31.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.029+0000 7efea0a6e700 1 --2- 172.21.15.16:0/2964727724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe9c106f70 0x7efe9c1067a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efe90009fb0 tx=0x7efe90011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:31.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.029+0000 7efe99ffb700 1 -- 172.21.15.16:0/2964727724 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efe900038b0 con 0x7efe9c106f70 2024-04-02T04:33:31.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.029+0000 7efe99ffb700 1 -- 172.21.15.16:0/2964727724 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe90003a10 con 0x7efe9c106f70 2024-04-02T04:33:31.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.029+0000 7efea2cd2700 1 -- 172.21.15.16:0/2964727724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe9c104f20 con 0x7efe9c106f70 2024-04-02T04:33:31.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.029+0000 7efe99ffb700 1 -- 172.21.15.16:0/2964727724 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efe90019750 con 0x7efe9c106f70 2024-04-02T04:33:31.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.030+0000 7efea2cd2700 1 -- 172.21.15.16:0/2964727724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe9c1053c0 con 0x7efe9c106f70 2024-04-02T04:33:31.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.030+0000 7efe99ffb700 1 -- 172.21.15.16:0/2964727724 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 11) v1 ==== 45455+0+0 (secure 0 0 0) 0x7efe900198b0 con 0x7efe9c106f70 2024-04-02T04:33:31.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.031+0000 7efe99ffb700 1 --2- 172.21.15.16:0/2964727724 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7efe8c038660 0x7efe8c03ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:31.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.031+0000 7efe99ffb700 1 -- 172.21.15.16:0/2964727724 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efe9004c720 con 0x7efe9c106f70 2024-04-02T04:33:31.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.031+0000 7efe9bfff700 1 --2- 172.21.15.16:0/2964727724 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7efe8c038660 0x7efe8c03ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:31.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.031+0000 7efea2cd2700 1 -- 172.21.15.16:0/2964727724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe9c0623e0 con 0x7efe9c106f70 2024-04-02T04:33:31.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.031+0000 7efe9bfff700 1 --2- 172.21.15.16:0/2964727724 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7efe8c038660 0x7efe8c03ab20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efe88006fd0 tx=0x7efe88006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:31.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.036+0000 7efe99ffb700 1 -- 172.21.15.16:0/2964727724 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7efe90019c00 con 0x7efe9c106f70 2024-04-02T04:33:31.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.250+0000 7efea2cd2700 1 -- 172.21.15.16:0/2964727724 --> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7efe9c1058e0 con 0x7efe8c038660 2024-04-02T04:33:31.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.339+0000 7efe99ffb700 1 -- 172.21.15.16:0/2964727724 <== mgr.14162 v2:172.21.15.16:6800/3912038174 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7efe9c1058e0 con 0x7efe8c038660 2024-04-02T04:33:31.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.341+0000 7efea2cd2700 1 -- 172.21.15.16:0/2964727724 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7efe8c038660 msgr2=0x7efe8c03ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:31.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.341+0000 7efea2cd2700 1 --2- 172.21.15.16:0/2964727724 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7efe8c038660 0x7efe8c03ab20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efe88006fd0 tx=0x7efe88006e40 comp rx=0 tx=0).stop 2024-04-02T04:33:31.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.341+0000 7efea2cd2700 1 -- 172.21.15.16:0/2964727724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe9c106f70 msgr2=0x7efe9c1067a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:31.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.341+0000 7efea2cd2700 1 --2- 172.21.15.16:0/2964727724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe9c106f70 0x7efe9c1067a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efe90009fb0 tx=0x7efe90011890 comp rx=0 tx=0).stop 2024-04-02T04:33:31.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.341+0000 7efea2cd2700 1 -- 172.21.15.16:0/2964727724 shutdown_connections 2024-04-02T04:33:31.950 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.341+0000 7efea2cd2700 1 --2- 172.21.15.16:0/2964727724 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7efe8c038660 0x7efe8c03ab20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:31.950 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.341+0000 7efea2cd2700 1 --2- 172.21.15.16:0/2964727724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe9c106f70 0x7efe9c1067a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:31.950 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.341+0000 7efea2cd2700 1 -- 172.21.15.16:0/2964727724 >> 172.21.15.16:0/2964727724 conn(0x7efe9c1009c0 msgr2=0x7efe9c1014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:31.950 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.341+0000 7efea2cd2700 1 -- 172.21.15.16:0/2964727724 shutdown_connections 2024-04-02T04:33:31.950 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:31.341+0000 7efea2cd2700 1 -- 172.21.15.16:0/2964727724 wait complete. 2024-04-02T04:33:31.950 INFO:teuthology.orchestra.run.smithi016.stdout:Creating initial admin user... 2024-04-02T04:33:32.047 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:31 smithi016 ceph-mon[28969]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-02T04:33:32.048 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:31 smithi016 ceph-mon[28969]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-02T04:33:32.048 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:31 smithi016 ceph-mon[28969]: [02/Apr/2024:04:33:29] ENGINE Serving on http://172.21.15.16:8765 2024-04-02T04:33:32.048 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:31 smithi016 ceph-mon[28969]: [02/Apr/2024:04:33:29] ENGINE Bus STARTED 2024-04-02T04:33:32.048 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:31 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:33:32.048 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:31 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:33:32.048 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:31 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:33:32.048 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:31 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:33:32.839 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:32 smithi016 ceph-mon[28969]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:33:32.840 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:32 smithi016 ceph-mon[28969]: mgrmap e12: smithi016.eqnebx(active, since 2s) 2024-04-02T04:33:33.557 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$1QFIxPHEoqKfrKrJ.mVMK.9M6RuKTTHkTb7B8WOHhK24dG5U2PSQG", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1712032412, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-02T04:33:33.558 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.347+0000 7fee6249f700 1 Processor -- start 2024-04-02T04:33:33.558 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.347+0000 7fee6249f700 1 -- start start 2024-04-02T04:33:33.558 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.347+0000 7fee6249f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee5c106fa0 0x7fee5c109390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:33.558 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.347+0000 7fee6249f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c079e30 con 0x7fee5c106fa0 2024-04-02T04:33:33.558 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.348+0000 7fee5bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee5c106fa0 0x7fee5c109390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:33.558 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.348+0000 7fee5bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee5c106fa0 0x7fee5c109390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36540/0 (socket says 172.21.15.16:36540) 2024-04-02T04:33:33.558 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.348+0000 7fee5bfff700 1 -- 172.21.15.16:0/2229287891 learned_addr learned my addr 172.21.15.16:0/2229287891 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:33.558 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.348+0000 7fee5bfff700 1 -- 172.21.15.16:0/2229287891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee5c1098d0 con 0x7fee5c106fa0 2024-04-02T04:33:33.558 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.348+0000 7fee5bfff700 1 --2- 172.21.15.16:0/2229287891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee5c106fa0 0x7fee5c109390 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fee4c0098a0 tx=0x7fee4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d90b4e34064ca627 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:33.558 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.348+0000 7fee5affd700 1 -- 172.21.15.16:0/2229287891 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee4c004180 con 0x7fee5c106fa0 2024-04-02T04:33:33.558 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.348+0000 7fee5affd700 1 -- 172.21.15.16:0/2229287891 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fee4c0042e0 con 0x7fee5c106fa0 2024-04-02T04:33:33.559 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.349+0000 7fee5affd700 1 -- 172.21.15.16:0/2229287891 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee4c011b10 con 0x7fee5c106fa0 2024-04-02T04:33:33.559 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.349+0000 7fee6249f700 1 -- 172.21.15.16:0/2229287891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee5c106fa0 msgr2=0x7fee5c109390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:33.559 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.349+0000 7fee6249f700 1 --2- 172.21.15.16:0/2229287891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee5c106fa0 0x7fee5c109390 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fee4c0098a0 tx=0x7fee4c009bb0 comp rx=0 tx=0).stop 2024-04-02T04:33:33.559 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.349+0000 7fee6249f700 1 -- 172.21.15.16:0/2229287891 shutdown_connections 2024-04-02T04:33:33.559 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.349+0000 7fee6249f700 1 --2- 172.21.15.16:0/2229287891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee5c106fa0 0x7fee5c109390 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:33.559 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.349+0000 7fee6249f700 1 -- 172.21.15.16:0/2229287891 >> 172.21.15.16:0/2229287891 conn(0x7fee5c100930 msgr2=0x7fee5c102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:33.559 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.349+0000 7fee6249f700 1 -- 172.21.15.16:0/2229287891 shutdown_connections 2024-04-02T04:33:33.559 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.349+0000 7fee6249f700 1 -- 172.21.15.16:0/2229287891 wait complete. 2024-04-02T04:33:33.559 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.350+0000 7fee6249f700 1 Processor -- start 2024-04-02T04:33:33.559 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.350+0000 7fee6249f700 1 -- start start 2024-04-02T04:33:33.559 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.350+0000 7fee6249f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee5c106fa0 0x7fee5c106780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:33.559 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.350+0000 7fee6249f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c106cc0 con 0x7fee5c106fa0 2024-04-02T04:33:33.559 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.351+0000 7fee5bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee5c106fa0 0x7fee5c106780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:33.559 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.351+0000 7fee5bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee5c106fa0 0x7fee5c106780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36548/0 (socket says 172.21.15.16:36548) 2024-04-02T04:33:33.559 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.351+0000 7fee5bfff700 1 -- 172.21.15.16:0/3459401126 learned_addr learned my addr 172.21.15.16:0/3459401126 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:33.560 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.351+0000 7fee5bfff700 1 -- 172.21.15.16:0/3459401126 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4c009550 con 0x7fee5c106fa0 2024-04-02T04:33:33.560 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.351+0000 7fee5bfff700 1 --2- 172.21.15.16:0/3459401126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee5c106fa0 0x7fee5c106780 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009fb0 tx=0x7fee4c011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:33.560 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.351+0000 7fee597fa700 1 -- 172.21.15.16:0/3459401126 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee4c0038b0 con 0x7fee5c106fa0 2024-04-02T04:33:33.560 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.351+0000 7fee597fa700 1 -- 172.21.15.16:0/3459401126 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fee4c003a10 con 0x7fee5c106fa0 2024-04-02T04:33:33.560 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.351+0000 7fee6249f700 1 -- 172.21.15.16:0/3459401126 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee5c104f50 con 0x7fee5c106fa0 2024-04-02T04:33:33.560 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.351+0000 7fee597fa700 1 -- 172.21.15.16:0/3459401126 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee4c019750 con 0x7fee5c106fa0 2024-04-02T04:33:33.560 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.351+0000 7fee6249f700 1 -- 172.21.15.16:0/3459401126 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee5c1053f0 con 0x7fee5c106fa0 2024-04-02T04:33:33.560 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.353+0000 7fee597fa700 1 -- 172.21.15.16:0/3459401126 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 45561+0+0 (secure 0 0 0) 0x7fee4c0199c0 con 0x7fee5c106fa0 2024-04-02T04:33:33.560 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.353+0000 7fee6249f700 1 -- 172.21.15.16:0/3459401126 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee5c0623e0 con 0x7fee5c106fa0 2024-04-02T04:33:33.560 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.353+0000 7fee597fa700 1 --2- 172.21.15.16:0/3459401126 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fee44038730 0x7fee4403abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:33.560 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.353+0000 7fee597fa700 1 -- 172.21.15.16:0/3459401126 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fee4c051660 con 0x7fee5c106fa0 2024-04-02T04:33:33.560 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.353+0000 7fee5b7fe700 1 --2- 172.21.15.16:0/3459401126 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fee44038730 0x7fee4403abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:33.560 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.353+0000 7fee5b7fe700 1 --2- 172.21.15.16:0/3459401126 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fee44038730 0x7fee4403abf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fee50006fd0 tx=0x7fee50006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:33.560 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.358+0000 7fee597fa700 1 -- 172.21.15.16:0/3459401126 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fee4c052060 con 0x7fee5c106fa0 2024-04-02T04:33:33.561 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.578+0000 7fee6249f700 1 -- 172.21.15.16:0/3459401126 --> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fee5c10a910 con 0x7fee44038730 2024-04-02T04:33:33.561 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.823+0000 7fee597fa700 1 -- 172.21.15.16:0/3459401126 <== mgr.14162 v2:172.21.15.16:6800/3912038174 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fee5c10a910 con 0x7fee44038730 2024-04-02T04:33:33.561 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.825+0000 7fee6249f700 1 -- 172.21.15.16:0/3459401126 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fee44038730 msgr2=0x7fee4403abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:33.561 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.825+0000 7fee6249f700 1 --2- 172.21.15.16:0/3459401126 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fee44038730 0x7fee4403abf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fee50006fd0 tx=0x7fee50006e40 comp rx=0 tx=0).stop 2024-04-02T04:33:33.561 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.825+0000 7fee6249f700 1 -- 172.21.15.16:0/3459401126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee5c106fa0 msgr2=0x7fee5c106780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:33.561 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.825+0000 7fee6249f700 1 --2- 172.21.15.16:0/3459401126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee5c106fa0 0x7fee5c106780 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009fb0 tx=0x7fee4c011890 comp rx=0 tx=0).stop 2024-04-02T04:33:33.561 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.826+0000 7fee6249f700 1 -- 172.21.15.16:0/3459401126 shutdown_connections 2024-04-02T04:33:33.561 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.826+0000 7fee6249f700 1 --2- 172.21.15.16:0/3459401126 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fee44038730 0x7fee4403abf0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:33.561 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.826+0000 7fee6249f700 1 --2- 172.21.15.16:0/3459401126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee5c106fa0 0x7fee5c106780 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:33.561 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.826+0000 7fee6249f700 1 -- 172.21.15.16:0/3459401126 >> 172.21.15.16:0/3459401126 conn(0x7fee5c100930 msgr2=0x7fee5c101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:33.561 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.826+0000 7fee6249f700 1 -- 172.21.15.16:0/3459401126 shutdown_connections 2024-04-02T04:33:33.561 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:32.826+0000 7fee6249f700 1 -- 172.21.15.16:0/3459401126 wait complete. 2024-04-02T04:33:33.561 INFO:teuthology.orchestra.run.smithi016.stdout:Fetching dashboard port number... 2024-04-02T04:33:34.158 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:33 smithi016 ceph-mon[28969]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:33:34.158 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:33 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:33:34.819 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 8443 2024-04-02T04:33:34.819 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.942+0000 7fc5817a2700 1 Processor -- start 2024-04-02T04:33:34.819 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.942+0000 7fc5817a2700 1 -- start start 2024-04-02T04:33:34.819 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.943+0000 7fc5817a2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc57c106f70 0x7fc57c109360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:34.819 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.943+0000 7fc5817a2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57c0745b0 con 0x7fc57c106f70 2024-04-02T04:33:34.819 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.943+0000 7fc57affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc57c106f70 0x7fc57c109360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:34.819 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.943+0000 7fc57affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc57c106f70 0x7fc57c109360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36562/0 (socket says 172.21.15.16:36562) 2024-04-02T04:33:34.819 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.943+0000 7fc57affd700 1 -- 172.21.15.16:0/2263138324 learned_addr learned my addr 172.21.15.16:0/2263138324 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:34.819 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.943+0000 7fc57affd700 1 -- 172.21.15.16:0/2263138324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc57c1098a0 con 0x7fc57c106f70 2024-04-02T04:33:34.820 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.943+0000 7fc57affd700 1 --2- 172.21.15.16:0/2263138324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc57c106f70 0x7fc57c109360 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc56c0098a0 tx=0x7fc56c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c73a258658ad7e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:34.820 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.943+0000 7fc579ffb700 1 -- 172.21.15.16:0/2263138324 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc56c004180 con 0x7fc57c106f70 2024-04-02T04:33:34.820 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.944+0000 7fc579ffb700 1 -- 172.21.15.16:0/2263138324 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc56c0042e0 con 0x7fc57c106f70 2024-04-02T04:33:34.820 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.944+0000 7fc579ffb700 1 -- 172.21.15.16:0/2263138324 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc56c011b10 con 0x7fc57c106f70 2024-04-02T04:33:34.820 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.944+0000 7fc5817a2700 1 -- 172.21.15.16:0/2263138324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc57c106f70 msgr2=0x7fc57c109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:34.820 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.944+0000 7fc5817a2700 1 --2- 172.21.15.16:0/2263138324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc57c106f70 0x7fc57c109360 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc56c0098a0 tx=0x7fc56c009bb0 comp rx=0 tx=0).stop 2024-04-02T04:33:34.820 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.944+0000 7fc5817a2700 1 -- 172.21.15.16:0/2263138324 shutdown_connections 2024-04-02T04:33:34.820 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.944+0000 7fc5817a2700 1 --2- 172.21.15.16:0/2263138324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc57c106f70 0x7fc57c109360 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:34.820 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.944+0000 7fc5817a2700 1 -- 172.21.15.16:0/2263138324 >> 172.21.15.16:0/2263138324 conn(0x7fc57c1009c0 msgr2=0x7fc57c102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:34.820 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.944+0000 7fc5817a2700 1 -- 172.21.15.16:0/2263138324 shutdown_connections 2024-04-02T04:33:34.820 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.944+0000 7fc5817a2700 1 -- 172.21.15.16:0/2263138324 wait complete. 2024-04-02T04:33:34.820 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.945+0000 7fc5817a2700 1 Processor -- start 2024-04-02T04:33:34.820 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.945+0000 7fc5817a2700 1 -- start start 2024-04-02T04:33:34.821 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.945+0000 7fc5817a2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc57c106f70 0x7fc57c19a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:34.821 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.945+0000 7fc5817a2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57c19a7c0 con 0x7fc57c106f70 2024-04-02T04:33:34.821 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.945+0000 7fc57affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc57c106f70 0x7fc57c19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:34.821 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.945+0000 7fc57affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc57c106f70 0x7fc57c19a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36568/0 (socket says 172.21.15.16:36568) 2024-04-02T04:33:34.821 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.945+0000 7fc57affd700 1 -- 172.21.15.16:0/1931603540 learned_addr learned my addr 172.21.15.16:0/1931603540 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:34.821 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.946+0000 7fc57affd700 1 -- 172.21.15.16:0/1931603540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc56c009550 con 0x7fc57c106f70 2024-04-02T04:33:34.821 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.946+0000 7fc57affd700 1 --2- 172.21.15.16:0/1931603540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc57c106f70 0x7fc57c19a280 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc56c009fb0 tx=0x7fc56c011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:34.821 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.946+0000 7fc573fff700 1 -- 172.21.15.16:0/1931603540 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc56c0038b0 con 0x7fc57c106f70 2024-04-02T04:33:34.821 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.946+0000 7fc5817a2700 1 -- 172.21.15.16:0/1931603540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc57c19a9c0 con 0x7fc57c106f70 2024-04-02T04:33:34.821 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.946+0000 7fc573fff700 1 -- 172.21.15.16:0/1931603540 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc56c003a10 con 0x7fc57c106f70 2024-04-02T04:33:34.821 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.946+0000 7fc573fff700 1 -- 172.21.15.16:0/1931603540 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc56c019750 con 0x7fc57c106f70 2024-04-02T04:33:34.821 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.946+0000 7fc5817a2700 1 -- 172.21.15.16:0/1931603540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc57c19ae60 con 0x7fc57c106f70 2024-04-02T04:33:34.821 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.947+0000 7fc573fff700 1 -- 172.21.15.16:0/1931603540 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 45561+0+0 (secure 0 0 0) 0x7fc56c0199c0 con 0x7fc57c106f70 2024-04-02T04:33:34.822 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.947+0000 7fc5817a2700 1 -- 172.21.15.16:0/1931603540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc57c10a630 con 0x7fc57c106f70 2024-04-02T04:33:34.822 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.947+0000 7fc573fff700 1 --2- 172.21.15.16:0/1931603540 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fc568038730 0x7fc56803abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:34.822 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.947+0000 7fc573fff700 1 -- 172.21.15.16:0/1931603540 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc56c04c920 con 0x7fc57c106f70 2024-04-02T04:33:34.822 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.947+0000 7fc57a7fc700 1 --2- 172.21.15.16:0/1931603540 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fc568038730 0x7fc56803abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:34.822 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.948+0000 7fc57a7fc700 1 --2- 172.21.15.16:0/1931603540 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fc568038730 0x7fc56803abf0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc564006fd0 tx=0x7fc564006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:34.822 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:33.952+0000 7fc573fff700 1 -- 172.21.15.16:0/1931603540 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc56c010380 con 0x7fc57c106f70 2024-04-02T04:33:34.822 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:34.155+0000 7fc5817a2700 1 -- 172.21.15.16:0/1931603540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fc57c10a870 con 0x7fc57c106f70 2024-04-02T04:33:34.822 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:34.155+0000 7fc573fff700 1 -- 172.21.15.16:0/1931603540 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fc56c018350 con 0x7fc57c106f70 2024-04-02T04:33:34.822 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:34.157+0000 7fc5817a2700 1 -- 172.21.15.16:0/1931603540 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fc568038730 msgr2=0x7fc56803abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:34.822 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:34.157+0000 7fc5817a2700 1 --2- 172.21.15.16:0/1931603540 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fc568038730 0x7fc56803abf0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc564006fd0 tx=0x7fc564006e40 comp rx=0 tx=0).stop 2024-04-02T04:33:34.822 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:34.157+0000 7fc5817a2700 1 -- 172.21.15.16:0/1931603540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc57c106f70 msgr2=0x7fc57c19a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:34.822 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:34.158+0000 7fc5817a2700 1 --2- 172.21.15.16:0/1931603540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc57c106f70 0x7fc57c19a280 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc56c009fb0 tx=0x7fc56c011890 comp rx=0 tx=0).stop 2024-04-02T04:33:34.823 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:34.158+0000 7fc5817a2700 1 -- 172.21.15.16:0/1931603540 shutdown_connections 2024-04-02T04:33:34.823 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:34.158+0000 7fc5817a2700 1 --2- 172.21.15.16:0/1931603540 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fc568038730 0x7fc56803abf0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:34.823 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:34.158+0000 7fc5817a2700 1 --2- 172.21.15.16:0/1931603540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc57c106f70 0x7fc57c19a280 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:34.823 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:34.158+0000 7fc5817a2700 1 -- 172.21.15.16:0/1931603540 >> 172.21.15.16:0/1931603540 conn(0x7fc57c1009c0 msgr2=0x7fc57c1014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:34.823 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:34.158+0000 7fc5817a2700 1 -- 172.21.15.16:0/1931603540 shutdown_connections 2024-04-02T04:33:34.823 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:34.158+0000 7fc5817a2700 1 -- 172.21.15.16:0/1931603540 wait complete. 2024-04-02T04:33:34.826 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-02T04:33:34.826 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-04-02T04:33:34.834 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-02T04:33:34.834 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-04-02T04:33:34.834 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-04-02T04:33:34.834 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-02T04:33:34.836 INFO:teuthology.orchestra.run.smithi016.stdout:Ceph Dashboard is now available at: 2024-04-02T04:33:34.836 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:33:34.836 INFO:teuthology.orchestra.run.smithi016.stdout: URL: https://smithi016.front.sepia.ceph.com:8443/ 2024-04-02T04:33:34.836 INFO:teuthology.orchestra.run.smithi016.stdout: User: admin 2024-04-02T04:33:34.836 INFO:teuthology.orchestra.run.smithi016.stdout: Password: yqsb1bboet 2024-04-02T04:33:34.836 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:33:34.837 INFO:teuthology.orchestra.run.smithi016.stdout:Saving cluster configuration to /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/config directory 2024-04-02T04:33:34.837 INFO:teuthology.orchestra.run.smithi016.stdout:Enabling autotune for osd_memory_target 2024-04-02T04:33:35.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:34 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/1931603540' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-02T04:33:36.026 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.273+0000 7f353c53f700 1 Processor -- start 2024-04-02T04:33:36.027 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.273+0000 7f353c53f700 1 -- start start 2024-04-02T04:33:36.027 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.274+0000 7f353c53f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3534106fa0 0x7f3534109390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:36.027 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.274+0000 7f353c53f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3534079e50 con 0x7f3534106fa0 2024-04-02T04:33:36.027 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.274+0000 7f353a2db700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3534106fa0 0x7f3534109390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:36.027 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.274+0000 7f353a2db700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3534106fa0 0x7f3534109390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36584/0 (socket says 172.21.15.16:36584) 2024-04-02T04:33:36.027 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.274+0000 7f353a2db700 1 -- 172.21.15.16:0/347758859 learned_addr learned my addr 172.21.15.16:0/347758859 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:36.027 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.274+0000 7f353a2db700 1 -- 172.21.15.16:0/347758859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35341098d0 con 0x7f3534106fa0 2024-04-02T04:33:36.027 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.274+0000 7f353a2db700 1 --2- 172.21.15.16:0/347758859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3534106fa0 0x7f3534109390 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f35300098a0 tx=0x7f3530009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c6e25a5b5216ad server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:36.027 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.275+0000 7f35392d9700 1 -- 172.21.15.16:0/347758859 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3530004180 con 0x7f3534106fa0 2024-04-02T04:33:36.027 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.275+0000 7f35392d9700 1 -- 172.21.15.16:0/347758859 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f35300042e0 con 0x7f3534106fa0 2024-04-02T04:33:36.027 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.275+0000 7f35392d9700 1 -- 172.21.15.16:0/347758859 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3530011b10 con 0x7f3534106fa0 2024-04-02T04:33:36.027 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.275+0000 7f353c53f700 1 -- 172.21.15.16:0/347758859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3534106fa0 msgr2=0x7f3534109390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:36.028 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.275+0000 7f353c53f700 1 --2- 172.21.15.16:0/347758859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3534106fa0 0x7f3534109390 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f35300098a0 tx=0x7f3530009bb0 comp rx=0 tx=0).stop 2024-04-02T04:33:36.028 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.275+0000 7f353c53f700 1 -- 172.21.15.16:0/347758859 shutdown_connections 2024-04-02T04:33:36.028 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.275+0000 7f353c53f700 1 --2- 172.21.15.16:0/347758859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3534106fa0 0x7f3534109390 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:36.028 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.275+0000 7f353c53f700 1 -- 172.21.15.16:0/347758859 >> 172.21.15.16:0/347758859 conn(0x7f35341009d0 msgr2=0x7f3534102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:36.028 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.275+0000 7f353c53f700 1 -- 172.21.15.16:0/347758859 shutdown_connections 2024-04-02T04:33:36.028 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.275+0000 7f353c53f700 1 -- 172.21.15.16:0/347758859 wait complete. 2024-04-02T04:33:36.028 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.276+0000 7f353c53f700 1 Processor -- start 2024-04-02T04:33:36.028 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.276+0000 7f353c53f700 1 -- start start 2024-04-02T04:33:36.028 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.276+0000 7f353c53f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3534106fa0 0x7f353419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:36.028 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.276+0000 7f353c53f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f353419a730 con 0x7f3534106fa0 2024-04-02T04:33:36.028 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.276+0000 7f353a2db700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3534106fa0 0x7f353419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:36.028 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.276+0000 7f353a2db700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3534106fa0 0x7f353419a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36600/0 (socket says 172.21.15.16:36600) 2024-04-02T04:33:36.028 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.276+0000 7f353a2db700 1 -- 172.21.15.16:0/310996243 learned_addr learned my addr 172.21.15.16:0/310996243 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:36.029 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.276+0000 7f353a2db700 1 -- 172.21.15.16:0/310996243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3530009550 con 0x7f3534106fa0 2024-04-02T04:33:36.029 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.277+0000 7f353a2db700 1 --2- 172.21.15.16:0/310996243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3534106fa0 0x7f353419a1f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3530009fb0 tx=0x7f3530011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:36.029 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.277+0000 7f352b7fe700 1 -- 172.21.15.16:0/310996243 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f35300038b0 con 0x7f3534106fa0 2024-04-02T04:33:36.029 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.277+0000 7f352b7fe700 1 -- 172.21.15.16:0/310996243 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3530003a10 con 0x7f3534106fa0 2024-04-02T04:33:36.029 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.277+0000 7f353c53f700 1 -- 172.21.15.16:0/310996243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f353419a930 con 0x7f3534106fa0 2024-04-02T04:33:36.029 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.277+0000 7f352b7fe700 1 -- 172.21.15.16:0/310996243 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3530019750 con 0x7f3534106fa0 2024-04-02T04:33:36.029 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.277+0000 7f353c53f700 1 -- 172.21.15.16:0/310996243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f353419add0 con 0x7f3534106fa0 2024-04-02T04:33:36.029 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.278+0000 7f352b7fe700 1 -- 172.21.15.16:0/310996243 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 45561+0+0 (secure 0 0 0) 0x7f35300198b0 con 0x7f3534106fa0 2024-04-02T04:33:36.029 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.278+0000 7f353c53f700 1 -- 172.21.15.16:0/310996243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f353410a660 con 0x7f3534106fa0 2024-04-02T04:33:36.029 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.278+0000 7f352b7fe700 1 --2- 172.21.15.16:0/310996243 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f3520038780 0x7f352003ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:36.029 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.278+0000 7f352b7fe700 1 -- 172.21.15.16:0/310996243 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f353004ca60 con 0x7f3534106fa0 2024-04-02T04:33:36.029 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.278+0000 7f3539ada700 1 --2- 172.21.15.16:0/310996243 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f3520038780 0x7f352003ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:36.029 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.278+0000 7f3539ada700 1 --2- 172.21.15.16:0/310996243 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f3520038780 0x7f352003ac40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3524006fd0 tx=0x7f3524006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:36.030 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.283+0000 7f352b7fe700 1 -- 172.21.15.16:0/310996243 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3530022b40 con 0x7f3534106fa0 2024-04-02T04:33:36.030 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.483+0000 7f353c53f700 1 -- 172.21.15.16:0/310996243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f353410a8a0 con 0x7f3534106fa0 2024-04-02T04:33:36.030 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.483+0000 7f352b7fe700 1 -- 172.21.15.16:0/310996243 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f3530022310 con 0x7f3534106fa0 2024-04-02T04:33:36.030 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.485+0000 7f353c53f700 1 -- 172.21.15.16:0/310996243 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f3520038780 msgr2=0x7f352003ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:36.030 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.485+0000 7f353c53f700 1 --2- 172.21.15.16:0/310996243 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f3520038780 0x7f352003ac40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3524006fd0 tx=0x7f3524006e40 comp rx=0 tx=0).stop 2024-04-02T04:33:36.030 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.485+0000 7f353c53f700 1 -- 172.21.15.16:0/310996243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3534106fa0 msgr2=0x7f353419a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:36.030 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.485+0000 7f353c53f700 1 --2- 172.21.15.16:0/310996243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3534106fa0 0x7f353419a1f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3530009fb0 tx=0x7f3530011890 comp rx=0 tx=0).stop 2024-04-02T04:33:36.030 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.485+0000 7f353c53f700 1 -- 172.21.15.16:0/310996243 shutdown_connections 2024-04-02T04:33:36.030 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.485+0000 7f353c53f700 1 --2- 172.21.15.16:0/310996243 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f3520038780 0x7f352003ac40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:36.030 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.485+0000 7f353c53f700 1 --2- 172.21.15.16:0/310996243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3534106fa0 0x7f353419a1f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:36.030 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.485+0000 7f353c53f700 1 -- 172.21.15.16:0/310996243 >> 172.21.15.16:0/310996243 conn(0x7f35341009d0 msgr2=0x7f3534101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:36.030 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.485+0000 7f353c53f700 1 -- 172.21.15.16:0/310996243 shutdown_connections 2024-04-02T04:33:36.030 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:35.485+0000 7f353c53f700 1 -- 172.21.15.16:0/310996243 wait complete. 2024-04-02T04:33:37.040 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:36 smithi016 ceph-mon[28969]: mgrmap e13: smithi016.eqnebx(active, since 6s) 2024-04-02T04:33:37.040 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:36 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/2250396668' entity='client.admin' 2024-04-02T04:33:37.264 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.419+0000 7f880d914700 1 Processor -- start 2024-04-02T04:33:37.264 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.419+0000 7f880d914700 1 -- start start 2024-04-02T04:33:37.264 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.419+0000 7f880d914700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8808106550 0x7f8808106970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:37.264 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.419+0000 7f880d914700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8808106f40 con 0x7f8808106550 2024-04-02T04:33:37.264 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.419+0000 7f8806ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8808106550 0x7f8808106970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:37.264 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.419+0000 7f8806ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8808106550 0x7f8808106970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36612/0 (socket says 172.21.15.16:36612) 2024-04-02T04:33:37.265 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.419+0000 7f8806ffd700 1 -- 172.21.15.16:0/2425017492 learned_addr learned my addr 172.21.15.16:0/2425017492 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:37.265 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.420+0000 7f8806ffd700 1 -- 172.21.15.16:0/2425017492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88081076b0 con 0x7f8808106550 2024-04-02T04:33:37.265 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.420+0000 7f8806ffd700 1 --2- 172.21.15.16:0/2425017492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8808106550 0x7f8808106970 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8800009e80 tx=0x7f880000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a93b81385e6f61d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:37.265 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.420+0000 7f8805ffb700 1 -- 172.21.15.16:0/2425017492 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f880000e070 con 0x7f8808106550 2024-04-02T04:33:37.265 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.420+0000 7f8805ffb700 1 -- 172.21.15.16:0/2425017492 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f880000b8c0 con 0x7f8808106550 2024-04-02T04:33:37.265 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.420+0000 7f8805ffb700 1 -- 172.21.15.16:0/2425017492 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f88000044d0 con 0x7f8808106550 2024-04-02T04:33:37.265 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.420+0000 7f880d914700 1 -- 172.21.15.16:0/2425017492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8808106550 msgr2=0x7f8808106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:37.265 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.420+0000 7f880d914700 1 --2- 172.21.15.16:0/2425017492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8808106550 0x7f8808106970 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8800009e80 tx=0x7f880000b330 comp rx=0 tx=0).stop 2024-04-02T04:33:37.265 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.420+0000 7f880d914700 1 -- 172.21.15.16:0/2425017492 shutdown_connections 2024-04-02T04:33:37.265 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.420+0000 7f880d914700 1 --2- 172.21.15.16:0/2425017492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8808106550 0x7f8808106970 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:37.265 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.420+0000 7f880d914700 1 -- 172.21.15.16:0/2425017492 >> 172.21.15.16:0/2425017492 conn(0x7f8808101b10 msgr2=0x7f8808103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:37.265 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.421+0000 7f880d914700 1 -- 172.21.15.16:0/2425017492 shutdown_connections 2024-04-02T04:33:37.266 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.421+0000 7f880d914700 1 -- 172.21.15.16:0/2425017492 wait complete. 2024-04-02T04:33:37.266 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.421+0000 7f880d914700 1 Processor -- start 2024-04-02T04:33:37.266 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.421+0000 7f880d914700 1 -- start start 2024-04-02T04:33:37.266 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.421+0000 7f880d914700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8808106550 0x7f880819a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:37.266 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.421+0000 7f880d914700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880819a700 con 0x7f8808106550 2024-04-02T04:33:37.266 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.421+0000 7f8806ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8808106550 0x7f880819a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:37.266 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.421+0000 7f8806ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8808106550 0x7f880819a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36626/0 (socket says 172.21.15.16:36626) 2024-04-02T04:33:37.266 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.421+0000 7f8806ffd700 1 -- 172.21.15.16:0/2250396668 learned_addr learned my addr 172.21.15.16:0/2250396668 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:37.266 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.422+0000 7f8806ffd700 1 -- 172.21.15.16:0/2250396668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8800009b30 con 0x7f8808106550 2024-04-02T04:33:37.266 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.422+0000 7f8806ffd700 1 --2- 172.21.15.16:0/2250396668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8808106550 0x7f880819a1c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f88000072a0 tx=0x7f880000bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:37.266 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.422+0000 7f880c912700 1 -- 172.21.15.16:0/2250396668 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f880000e040 con 0x7f8808106550 2024-04-02T04:33:37.266 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.422+0000 7f880c912700 1 -- 172.21.15.16:0/2250396668 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8800003dc0 con 0x7f8808106550 2024-04-02T04:33:37.267 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.422+0000 7f880d914700 1 -- 172.21.15.16:0/2250396668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f880819a900 con 0x7f8808106550 2024-04-02T04:33:37.267 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.422+0000 7f880c912700 1 -- 172.21.15.16:0/2250396668 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8800012600 con 0x7f8808106550 2024-04-02T04:33:37.267 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.422+0000 7f880d914700 1 -- 172.21.15.16:0/2250396668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f880819ae00 con 0x7f8808106550 2024-04-02T04:33:37.267 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.423+0000 7f880c912700 1 -- 172.21.15.16:0/2250396668 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45604+0+0 (secure 0 0 0) 0x7f8800019040 con 0x7f8808106550 2024-04-02T04:33:37.267 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.423+0000 7f880d914700 1 -- 172.21.15.16:0/2250396668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f880810a8e0 con 0x7f8808106550 2024-04-02T04:33:37.267 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.423+0000 7f880c912700 1 --2- 172.21.15.16:0/2250396668 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f87f4038800 0x7f87f403acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:37.267 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.423+0000 7f880c912700 1 -- 172.21.15.16:0/2250396668 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f880004c3e0 con 0x7f8808106550 2024-04-02T04:33:37.267 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.423+0000 7f88067fc700 1 --2- 172.21.15.16:0/2250396668 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f87f4038800 0x7f87f403acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:37.267 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.424+0000 7f88067fc700 1 --2- 172.21.15.16:0/2250396668 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f87f4038800 0x7f87f403acc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f87f0006fd0 tx=0x7f87f0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:37.267 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.428+0000 7f880c912700 1 -- 172.21.15.16:0/2250396668 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8800011750 con 0x7f8808106550 2024-04-02T04:33:37.268 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.738+0000 7f880d914700 1 -- 172.21.15.16:0/2250396668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f880819d630 con 0x7f8808106550 2024-04-02T04:33:37.268 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.740+0000 7f880c912700 1 -- 172.21.15.16:0/2250396668 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v34)=0 set mgr/dashboard/cluster/status v34) v1 ==== 153+0+0 (secure 0 0 0) 0x7f880001f020 con 0x7f8808106550 2024-04-02T04:33:37.268 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.742+0000 7f880d914700 1 -- 172.21.15.16:0/2250396668 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f87f4038800 msgr2=0x7f87f403acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:37.268 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.742+0000 7f880d914700 1 --2- 172.21.15.16:0/2250396668 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f87f4038800 0x7f87f403acc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f87f0006fd0 tx=0x7f87f0006e40 comp rx=0 tx=0).stop 2024-04-02T04:33:37.268 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.742+0000 7f880d914700 1 -- 172.21.15.16:0/2250396668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8808106550 msgr2=0x7f880819a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:37.268 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.742+0000 7f880d914700 1 --2- 172.21.15.16:0/2250396668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8808106550 0x7f880819a1c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f88000072a0 tx=0x7f880000bf70 comp rx=0 tx=0).stop 2024-04-02T04:33:37.268 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.742+0000 7f880d914700 1 -- 172.21.15.16:0/2250396668 shutdown_connections 2024-04-02T04:33:37.268 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.742+0000 7f880d914700 1 --2- 172.21.15.16:0/2250396668 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f87f4038800 0x7f87f403acc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:37.268 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.742+0000 7f880d914700 1 --2- 172.21.15.16:0/2250396668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8808106550 0x7f880819a1c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:37.268 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.742+0000 7f880d914700 1 -- 172.21.15.16:0/2250396668 >> 172.21.15.16:0/2250396668 conn(0x7f8808101b10 msgr2=0x7f8808102670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:37.268 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.743+0000 7f880d914700 1 -- 172.21.15.16:0/2250396668 shutdown_connections 2024-04-02T04:33:37.268 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-04-02T04:33:36.743+0000 7f880d914700 1 -- 172.21.15.16:0/2250396668 wait complete. 2024-04-02T04:33:37.269 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-02T04:33:37.269 INFO:teuthology.orchestra.run.smithi016.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-02T04:33:37.269 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:33:37.269 INFO:teuthology.orchestra.run.smithi016.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-02T04:33:37.269 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:33:37.269 INFO:teuthology.orchestra.run.smithi016.stdout:Or, if you are only running a single cluster on this host: 2024-04-02T04:33:37.269 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:33:37.269 INFO:teuthology.orchestra.run.smithi016.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-02T04:33:37.269 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:33:37.269 INFO:teuthology.orchestra.run.smithi016.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-02T04:33:37.269 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:33:37.269 INFO:teuthology.orchestra.run.smithi016.stdout: ceph telemetry on 2024-04-02T04:33:37.269 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:33:37.269 INFO:teuthology.orchestra.run.smithi016.stdout:For more information see: 2024-04-02T04:33:37.270 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:33:37.270 INFO:teuthology.orchestra.run.smithi016.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-02T04:33:37.270 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:33:37.270 INFO:teuthology.orchestra.run.smithi016.stdout:Bootstrap complete. 2024-04-02T04:33:37.296 INFO:tasks.cephadm:Fetching config... 2024-04-02T04:33:37.297 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-02T04:33:37.297 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-02T04:33:37.352 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-02T04:33:37.352 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-02T04:33:37.352 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-02T04:33:37.407 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-02T04:33:37.407 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-02T04:33:37.407 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/keyring of=/dev/stdout 2024-04-02T04:33:37.474 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-02T04:33:37.474 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-02T04:33:37.474 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-02T04:33:37.528 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-02T04:33:37.529 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3nrI5QgzO2fY8sZYHpB+09l8v+CM9uFQ3nOL4i67UiiVLFEpF6CMC9qsNfnYOeOWH27Kve1tJ2R/GRGN1A8CpPbeYIwmovEMPmw8yFsYEnZy2nDaGgcc4PHEobGknzl7j0XB52Jm68APZ9i25s1GGmRayNH+c87AjCxgtRL9R/8y99Y+6oalbokf48y229+uaY/hMty3um0oGPOIDkbHP7Qxf8Ur0gDB2Kww4wDNauff0DmrW5Uq6cwD1CzlGYuBqlnKUNCdHRBBlSmo/2byM/4fnq/DjdpqTPVVN981jErA8dPefsle3kHmEojFemfx+HmQP3O4ZhT31/XH17YflocjtUbtF6drqQpZlnXD21gBnPWJ8CQaJpfC8tVpo5B26THiQGEs4Qbo0Il0giYlPze/hUTI8wrgBlIrefIRhWwHIzmuntifFgZZ8vJ6RJ6SAFmLvrXwusC+2j6ei2M8veyVDpzl5fpcahwinw0HOQLvZRfjL9ap3rKmKTigtSZk= ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-02T04:33:37.608 INFO:teuthology.orchestra.run.smithi016.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3nrI5QgzO2fY8sZYHpB+09l8v+CM9uFQ3nOL4i67UiiVLFEpF6CMC9qsNfnYOeOWH27Kve1tJ2R/GRGN1A8CpPbeYIwmovEMPmw8yFsYEnZy2nDaGgcc4PHEobGknzl7j0XB52Jm68APZ9i25s1GGmRayNH+c87AjCxgtRL9R/8y99Y+6oalbokf48y229+uaY/hMty3um0oGPOIDkbHP7Qxf8Ur0gDB2Kww4wDNauff0DmrW5Uq6cwD1CzlGYuBqlnKUNCdHRBBlSmo/2byM/4fnq/DjdpqTPVVN981jErA8dPefsle3kHmEojFemfx+HmQP3O4ZhT31/XH17YflocjtUbtF6drqQpZlnXD21gBnPWJ8CQaJpfC8tVpo5B26THiQGEs4Qbo0Il0giYlPze/hUTI8wrgBlIrefIRhWwHIzmuntifFgZZ8vJ6RJ6SAFmLvrXwusC+2j6ei2M8veyVDpzl5fpcahwinw0HOQLvZRfjL9ap3rKmKTigtSZk= ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4 2024-04-02T04:33:37.621 DEBUG:teuthology.orchestra.run.smithi027:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3nrI5QgzO2fY8sZYHpB+09l8v+CM9uFQ3nOL4i67UiiVLFEpF6CMC9qsNfnYOeOWH27Kve1tJ2R/GRGN1A8CpPbeYIwmovEMPmw8yFsYEnZy2nDaGgcc4PHEobGknzl7j0XB52Jm68APZ9i25s1GGmRayNH+c87AjCxgtRL9R/8y99Y+6oalbokf48y229+uaY/hMty3um0oGPOIDkbHP7Qxf8Ur0gDB2Kww4wDNauff0DmrW5Uq6cwD1CzlGYuBqlnKUNCdHRBBlSmo/2byM/4fnq/DjdpqTPVVN981jErA8dPefsle3kHmEojFemfx+HmQP3O4ZhT31/XH17YflocjtUbtF6drqQpZlnXD21gBnPWJ8CQaJpfC8tVpo5B26THiQGEs4Qbo0Il0giYlPze/hUTI8wrgBlIrefIRhWwHIzmuntifFgZZ8vJ6RJ6SAFmLvrXwusC+2j6ei2M8veyVDpzl5fpcahwinw0HOQLvZRfjL9ap3rKmKTigtSZk= ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-02T04:33:37.676 INFO:teuthology.orchestra.run.smithi027.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3nrI5QgzO2fY8sZYHpB+09l8v+CM9uFQ3nOL4i67UiiVLFEpF6CMC9qsNfnYOeOWH27Kve1tJ2R/GRGN1A8CpPbeYIwmovEMPmw8yFsYEnZy2nDaGgcc4PHEobGknzl7j0XB52Jm68APZ9i25s1GGmRayNH+c87AjCxgtRL9R/8y99Y+6oalbokf48y229+uaY/hMty3um0oGPOIDkbHP7Qxf8Ur0gDB2Kww4wDNauff0DmrW5Uq6cwD1CzlGYuBqlnKUNCdHRBBlSmo/2byM/4fnq/DjdpqTPVVN981jErA8dPefsle3kHmEojFemfx+HmQP3O4ZhT31/XH17YflocjtUbtF6drqQpZlnXD21gBnPWJ8CQaJpfC8tVpo5B26THiQGEs4Qbo0Il0giYlPze/hUTI8wrgBlIrefIRhWwHIzmuntifFgZZ8vJ6RJ6SAFmLvrXwusC+2j6ei2M8veyVDpzl5fpcahwinw0HOQLvZRfjL9ap3rKmKTigtSZk= ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4 2024-04-02T04:33:37.695 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-02T04:33:37.900 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:33:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.052+0000 7fcb87c71700 1 -- 172.21.15.16:0/1154635509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb800fea10 msgr2=0x7fcb800fee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.052+0000 7fcb87c71700 1 --2- 172.21.15.16:0/1154635509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb800fea10 0x7fcb800fee30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c009910 tx=0x7fcb7c009c20 comp rx=0 tx=0).stop 2024-04-02T04:33:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.052+0000 7fcb87c71700 1 -- 172.21.15.16:0/1154635509 shutdown_connections 2024-04-02T04:33:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.052+0000 7fcb87c71700 1 --2- 172.21.15.16:0/1154635509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb800fea10 0x7fcb800fee30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.052+0000 7fcb87c71700 1 -- 172.21.15.16:0/1154635509 >> 172.21.15.16:0/1154635509 conn(0x7fcb800fa370 msgr2=0x7fcb800fc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.052+0000 7fcb87c71700 1 -- 172.21.15.16:0/1154635509 shutdown_connections 2024-04-02T04:33:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.052+0000 7fcb87c71700 1 -- 172.21.15.16:0/1154635509 wait complete. 2024-04-02T04:33:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.053+0000 7fcb87c71700 1 Processor -- start 2024-04-02T04:33:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.053+0000 7fcb87c71700 1 -- start start 2024-04-02T04:33:40.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.053+0000 7fcb87c71700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb800fea10 0x7fcb80101220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:40.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.053+0000 7fcb87c71700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb80101760 con 0x7fcb800fea10 2024-04-02T04:33:40.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.053+0000 7fcb85a0d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb800fea10 0x7fcb80101220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:40.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.053+0000 7fcb85a0d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb800fea10 0x7fcb80101220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36632/0 (socket says 172.21.15.16:36632) 2024-04-02T04:33:40.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.053+0000 7fcb85a0d700 1 -- 172.21.15.16:0/3138958288 learned_addr learned my addr 172.21.15.16:0/3138958288 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:40.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.053+0000 7fcb85a0d700 1 -- 172.21.15.16:0/3138958288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb7c0095f0 con 0x7fcb800fea10 2024-04-02T04:33:40.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.054+0000 7fcb85a0d700 1 --2- 172.21.15.16:0/3138958288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb800fea10 0x7fcb80101220 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c006010 tx=0x7fcb7c009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:40.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.054+0000 7fcb76ffd700 1 -- 172.21.15.16:0/3138958288 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb7c01d070 con 0x7fcb800fea10 2024-04-02T04:33:40.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.054+0000 7fcb87c71700 1 -- 172.21.15.16:0/3138958288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb800ff9a0 con 0x7fcb800fea10 2024-04-02T04:33:40.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.054+0000 7fcb76ffd700 1 -- 172.21.15.16:0/3138958288 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcb7c005dc0 con 0x7fcb800fea10 2024-04-02T04:33:40.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.054+0000 7fcb76ffd700 1 -- 172.21.15.16:0/3138958288 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb7c022e90 con 0x7fcb800fea10 2024-04-02T04:33:40.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.054+0000 7fcb87c71700 1 -- 172.21.15.16:0/3138958288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb800ffe40 con 0x7fcb800fea10 2024-04-02T04:33:40.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.055+0000 7fcb76ffd700 1 -- 172.21.15.16:0/3138958288 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45604+0+0 (secure 0 0 0) 0x7fcb7c00f4f0 con 0x7fcb800fea10 2024-04-02T04:33:40.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.055+0000 7fcb87c71700 1 -- 172.21.15.16:0/3138958288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb80106ec0 con 0x7fcb800fea10 2024-04-02T04:33:40.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.055+0000 7fcb76ffd700 1 --2- 172.21.15.16:0/3138958288 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fcb6c038760 0x7fcb6c03ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:40.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.055+0000 7fcb76ffd700 1 -- 172.21.15.16:0/3138958288 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcb7c04c860 con 0x7fcb800fea10 2024-04-02T04:33:40.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.055+0000 7fcb8520c700 1 --2- 172.21.15.16:0/3138958288 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fcb6c038760 0x7fcb6c03ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:40.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.055+0000 7fcb8520c700 1 --2- 172.21.15.16:0/3138958288 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fcb6c038760 0x7fcb6c03ac20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcb70006fd0 tx=0x7fcb70006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:40.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.060+0000 7fcb76ffd700 1 -- 172.21.15.16:0/3138958288 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fcb7c017440 con 0x7fcb800fea10 2024-04-02T04:33:40.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.265+0000 7fcb87c71700 1 -- 172.21.15.16:0/3138958288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fcb801067e0 con 0x7fcb800fea10 2024-04-02T04:33:40.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.268+0000 7fcb76ffd700 1 -- 172.21.15.16:0/3138958288 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fcb7c027020 con 0x7fcb800fea10 2024-04-02T04:33:40.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.270+0000 7fcb87c71700 1 -- 172.21.15.16:0/3138958288 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fcb6c038760 msgr2=0x7fcb6c03ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:40.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.270+0000 7fcb87c71700 1 --2- 172.21.15.16:0/3138958288 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fcb6c038760 0x7fcb6c03ac20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcb70006fd0 tx=0x7fcb70006e40 comp rx=0 tx=0).stop 2024-04-02T04:33:40.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.270+0000 7fcb87c71700 1 -- 172.21.15.16:0/3138958288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb800fea10 msgr2=0x7fcb80101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:40.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.270+0000 7fcb87c71700 1 --2- 172.21.15.16:0/3138958288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb800fea10 0x7fcb80101220 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c006010 tx=0x7fcb7c009f80 comp rx=0 tx=0).stop 2024-04-02T04:33:40.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.270+0000 7fcb87c71700 1 -- 172.21.15.16:0/3138958288 shutdown_connections 2024-04-02T04:33:40.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.270+0000 7fcb87c71700 1 --2- 172.21.15.16:0/3138958288 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fcb6c038760 0x7fcb6c03ac20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:40.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.270+0000 7fcb87c71700 1 --2- 172.21.15.16:0/3138958288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb800fea10 0x7fcb80101220 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:40.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.270+0000 7fcb87c71700 1 -- 172.21.15.16:0/3138958288 >> 172.21.15.16:0/3138958288 conn(0x7fcb800fa370 msgr2=0x7fcb800fae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:40.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.270+0000 7fcb87c71700 1 -- 172.21.15.16:0/3138958288 shutdown_connections 2024-04-02T04:33:40.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:40.270+0000 7fcb87c71700 1 -- 172.21.15.16:0/3138958288 wait complete. 2024-04-02T04:33:40.857 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:40 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:33:40.857 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:40 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:33:40.857 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:40 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "who": "osd/host:smithi016", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:33:40.857 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:40 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:33:40.857 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:40 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi016", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-02T04:33:40.857 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:40 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi016", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-02T04:33:40.857 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:40 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:33:40.858 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:40 smithi016 ceph-mon[28969]: Deploying daemon ceph-exporter.smithi016 on smithi016 2024-04-02T04:33:40.858 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:40 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/3138958288' entity='client.admin' 2024-04-02T04:33:40.902 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-02T04:33:40.903 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-02T04:33:41.572 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:33:44.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.098+0000 7f41f3a3b700 1 -- 172.21.15.16:0/142789774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41ec100740 msgr2=0x7f41ec100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:44.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.098+0000 7f41f3a3b700 1 --2- 172.21.15.16:0/142789774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41ec100740 0x7f41ec100b60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f41e8009910 tx=0x7f41e8009c20 comp rx=0 tx=0).stop 2024-04-02T04:33:44.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.098+0000 7f41f3a3b700 1 -- 172.21.15.16:0/142789774 shutdown_connections 2024-04-02T04:33:44.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.098+0000 7f41f3a3b700 1 --2- 172.21.15.16:0/142789774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41ec100740 0x7f41ec100b60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:44.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.098+0000 7f41f3a3b700 1 -- 172.21.15.16:0/142789774 >> 172.21.15.16:0/142789774 conn(0x7f41ec0fbcc0 msgr2=0x7f41ec0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:44.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.098+0000 7f41f3a3b700 1 -- 172.21.15.16:0/142789774 shutdown_connections 2024-04-02T04:33:44.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.098+0000 7f41f3a3b700 1 -- 172.21.15.16:0/142789774 wait complete. 2024-04-02T04:33:44.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.098+0000 7f41f3a3b700 1 Processor -- start 2024-04-02T04:33:44.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.098+0000 7f41f3a3b700 1 -- start start 2024-04-02T04:33:44.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.099+0000 7f41f3a3b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41ec100740 0x7f41ec195bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:44.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.099+0000 7f41f3a3b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ec196100 con 0x7f41ec100740 2024-04-02T04:33:44.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.099+0000 7f41f17d7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41ec100740 0x7f41ec195bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:44.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.099+0000 7f41f17d7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41ec100740 0x7f41ec195bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47264/0 (socket says 172.21.15.16:47264) 2024-04-02T04:33:44.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.099+0000 7f41f17d7700 1 -- 172.21.15.16:0/1070979962 learned_addr learned my addr 172.21.15.16:0/1070979962 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:44.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.099+0000 7f41f17d7700 1 -- 172.21.15.16:0/1070979962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e80095f0 con 0x7f41ec100740 2024-04-02T04:33:44.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.099+0000 7f41f17d7700 1 --2- 172.21.15.16:0/1070979962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41ec100740 0x7f41ec195bc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f41e8006010 tx=0x7f41e8009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:44.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.099+0000 7f41de7fc700 1 -- 172.21.15.16:0/1070979962 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f41e801d070 con 0x7f41ec100740 2024-04-02T04:33:44.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.099+0000 7f41f3a3b700 1 -- 172.21.15.16:0/1070979962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41ec196300 con 0x7f41ec100740 2024-04-02T04:33:44.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.099+0000 7f41de7fc700 1 -- 172.21.15.16:0/1070979962 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f41e8005dc0 con 0x7f41ec100740 2024-04-02T04:33:44.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.099+0000 7f41de7fc700 1 -- 172.21.15.16:0/1070979962 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f41e8022e90 con 0x7f41ec100740 2024-04-02T04:33:44.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.100+0000 7f41f3a3b700 1 -- 172.21.15.16:0/1070979962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41ec1967a0 con 0x7f41ec100740 2024-04-02T04:33:44.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.101+0000 7f41de7fc700 1 -- 172.21.15.16:0/1070979962 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45604+0+0 (secure 0 0 0) 0x7f41e800f4f0 con 0x7f41ec100740 2024-04-02T04:33:44.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.101+0000 7f41f3a3b700 1 -- 172.21.15.16:0/1070979962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41ec18f570 con 0x7f41ec100740 2024-04-02T04:33:44.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.101+0000 7f41de7fc700 1 --2- 172.21.15.16:0/1070979962 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f41d8038760 0x7f41d803ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:44.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.101+0000 7f41de7fc700 1 -- 172.21.15.16:0/1070979962 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f41e804c940 con 0x7f41ec100740 2024-04-02T04:33:44.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.101+0000 7f41f0fd6700 1 --2- 172.21.15.16:0/1070979962 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f41d8038760 0x7f41d803ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:44.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.101+0000 7f41f0fd6700 1 --2- 172.21.15.16:0/1070979962 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f41d8038760 0x7f41d803ac20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f41e0006fd0 tx=0x7f41e0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:44.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.106+0000 7f41de7fc700 1 -- 172.21.15.16:0/1070979962 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f41e802a950 con 0x7f41ec100740 2024-04-02T04:33:44.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.321+0000 7f41f3a3b700 1 -- 172.21.15.16:0/1070979962 --> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f41ec02cd20 con 0x7f41d8038760 2024-04-02T04:33:44.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.324+0000 7f41de7fc700 1 -- 172.21.15.16:0/1070979962 <== mgr.14162 v2:172.21.15.16:6800/3912038174 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f41ec02cd20 con 0x7f41d8038760 2024-04-02T04:33:44.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.326+0000 7f41f3a3b700 1 -- 172.21.15.16:0/1070979962 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f41d8038760 msgr2=0x7f41d803ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:44.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.326+0000 7f41f3a3b700 1 --2- 172.21.15.16:0/1070979962 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f41d8038760 0x7f41d803ac20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f41e0006fd0 tx=0x7f41e0006e40 comp rx=0 tx=0).stop 2024-04-02T04:33:44.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.326+0000 7f41f3a3b700 1 -- 172.21.15.16:0/1070979962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41ec100740 msgr2=0x7f41ec195bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:44.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.326+0000 7f41f3a3b700 1 --2- 172.21.15.16:0/1070979962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41ec100740 0x7f41ec195bc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f41e8006010 tx=0x7f41e8009f80 comp rx=0 tx=0).stop 2024-04-02T04:33:44.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.326+0000 7f41f3a3b700 1 -- 172.21.15.16:0/1070979962 shutdown_connections 2024-04-02T04:33:44.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.326+0000 7f41f3a3b700 1 --2- 172.21.15.16:0/1070979962 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f41d8038760 0x7f41d803ac20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:44.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.326+0000 7f41f3a3b700 1 --2- 172.21.15.16:0/1070979962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41ec100740 0x7f41ec195bc0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:44.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.326+0000 7f41f3a3b700 1 -- 172.21.15.16:0/1070979962 >> 172.21.15.16:0/1070979962 conn(0x7f41ec0fbcc0 msgr2=0x7f41ec0fc790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:44.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.326+0000 7f41f3a3b700 1 -- 172.21.15.16:0/1070979962 shutdown_connections 2024-04-02T04:33:44.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:44.326+0000 7f41f3a3b700 1 -- 172.21.15.16:0/1070979962 wait complete. 2024-04-02T04:33:44.939 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi027 2024-04-02T04:33:44.939 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-02T04:33:44.939 DEBUG:teuthology.orchestra.run.smithi027:> dd of=/etc/ceph/ceph.conf 2024-04-02T04:33:44.958 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-02T04:33:44.958 DEBUG:teuthology.orchestra.run.smithi027:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-02T04:33:45.016 INFO:tasks.cephadm:Adding host smithi027 to orchestrator... 2024-04-02T04:33:45.017 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph orch host add smithi027 2024-04-02T04:33:45.233 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:33:45.433 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:45 smithi016 ceph-mon[28969]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:33:45.433 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:45 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:33:45.433 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:45 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:33:45.433 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:45 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:33:45.433 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:45 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:33:45.434 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:45 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:33:45.434 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:45 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi016", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-02T04:33:45.434 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:45 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi016", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-02T04:33:45.434 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:45 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:33:46.514 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:46 smithi016 ceph-mon[28969]: Deploying daemon crash.smithi016 on smithi016 2024-04-02T04:33:46.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:46.715+0000 7f2812fdb700 1 -- 172.21.15.16:0/4147210877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280c100720 msgr2=0x7f280c100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:46.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:46.715+0000 7f2812fdb700 1 --2- 172.21.15.16:0/4147210877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280c100720 0x7f280c100b40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f27fc009910 tx=0x7f27fc009c20 comp rx=0 tx=0).stop 2024-04-02T04:33:46.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:46.715+0000 7f2812fdb700 1 -- 172.21.15.16:0/4147210877 shutdown_connections 2024-04-02T04:33:46.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:46.715+0000 7f2812fdb700 1 --2- 172.21.15.16:0/4147210877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280c100720 0x7f280c100b40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:46.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:46.715+0000 7f2812fdb700 1 -- 172.21.15.16:0/4147210877 >> 172.21.15.16:0/4147210877 conn(0x7f280c0fbcc0 msgr2=0x7f280c0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:46.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:46.715+0000 7f2812fdb700 1 -- 172.21.15.16:0/4147210877 shutdown_connections 2024-04-02T04:33:46.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:46.715+0000 7f2812fdb700 1 -- 172.21.15.16:0/4147210877 wait complete. 2024-04-02T04:33:46.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:46.715+0000 7f2812fdb700 1 Processor -- start 2024-04-02T04:33:46.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:46.716+0000 7f2812fdb700 1 -- start start 2024-04-02T04:33:46.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:46.716+0000 7f2812fdb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280c100720 0x7f280c195ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:46.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:46.716+0000 7f2812fdb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280c1960e0 con 0x7f280c100720 2024-04-02T04:33:46.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:46.716+0000 7f2810d77700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280c100720 0x7f280c195ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:46.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:46.716+0000 7f2810d77700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280c100720 0x7f280c195ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47286/0 (socket says 172.21.15.16:47286) 2024-04-02T04:33:46.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:46.716+0000 7f2810d77700 1 -- 172.21.15.16:0/3467483386 learned_addr learned my addr 172.21.15.16:0/3467483386 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:46.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:46.716+0000 7f2810d77700 1 -- 172.21.15.16:0/3467483386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27fc0095f0 con 0x7f280c100720 2024-04-02T04:33:46.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:46.716+0000 7f2810d77700 1 --2- 172.21.15.16:0/3467483386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280c100720 0x7f280c195ba0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f27fc006010 tx=0x7f27fc0050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:46.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:46.716+0000 7f2809ffb700 1 -- 172.21.15.16:0/3467483386 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f27fc01d070 con 0x7f280c100720 2024-04-02T04:33:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:46.716+0000 7f2809ffb700 1 -- 172.21.15.16:0/3467483386 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f27fc00be20 con 0x7f280c100720 2024-04-02T04:33:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:46.716+0000 7f2809ffb700 1 -- 172.21.15.16:0/3467483386 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f27fc022e50 con 0x7f280c100720 2024-04-02T04:33:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:46.716+0000 7f2812fdb700 1 -- 172.21.15.16:0/3467483386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f280c1962e0 con 0x7f280c100720 2024-04-02T04:33:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:46.717+0000 7f2812fdb700 1 -- 172.21.15.16:0/3467483386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f280c196780 con 0x7f280c100720 2024-04-02T04:33:46.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:46.717+0000 7f2809ffb700 1 -- 172.21.15.16:0/3467483386 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45604+0+0 (secure 0 0 0) 0x7f27fc00f4b0 con 0x7f280c100720 2024-04-02T04:33:46.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:46.718+0000 7f2812fdb700 1 -- 172.21.15.16:0/3467483386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f280c18f550 con 0x7f280c100720 2024-04-02T04:33:46.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:46.718+0000 7f2809ffb700 1 --2- 172.21.15.16:0/3467483386 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f27f40387d0 0x7f27f403ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:46.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:46.718+0000 7f2809ffb700 1 -- 172.21.15.16:0/3467483386 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f27fc04ca10 con 0x7f280c100720 2024-04-02T04:33:46.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:46.718+0000 7f280bfff700 1 --2- 172.21.15.16:0/3467483386 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f27f40387d0 0x7f27f403ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:46.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:46.718+0000 7f280bfff700 1 --2- 172.21.15.16:0/3467483386 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f27f40387d0 0x7f27f403ac90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2800006fd0 tx=0x7f2800006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:46.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:46.723+0000 7f2809ffb700 1 -- 172.21.15.16:0/3467483386 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f27fc017440 con 0x7f280c100720 2024-04-02T04:33:46.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:46.931+0000 7f2812fdb700 1 -- 172.21.15.16:0/3467483386 --> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi027", "target": ["mon-mgr", ""]}) v1 -- 0x7f280c0611f0 con 0x7f27f40387d0 2024-04-02T04:33:48.035 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:47 smithi016 ceph-mon[28969]: from='client.14189 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi027", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:33:48.035 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:47 smithi016 ceph-mon[28969]: Deploying cephadm binary to smithi027 2024-04-02T04:33:49.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:49.686+0000 7f2809ffb700 1 -- 172.21.15.16:0/3467483386 <== mgr.14162 v2:172.21.15.16:6800/3912038174 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f280c0611f0 con 0x7f27f40387d0 2024-04-02T04:33:49.692 INFO:teuthology.orchestra.run.smithi016.stdout:Added host 'smithi027' with addr '172.21.15.27' 2024-04-02T04:33:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:49.689+0000 7f2812fdb700 1 -- 172.21.15.16:0/3467483386 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f27f40387d0 msgr2=0x7f27f403ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:49.689+0000 7f2812fdb700 1 --2- 172.21.15.16:0/3467483386 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f27f40387d0 0x7f27f403ac90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2800006fd0 tx=0x7f2800006e40 comp rx=0 tx=0).stop 2024-04-02T04:33:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:49.689+0000 7f2812fdb700 1 -- 172.21.15.16:0/3467483386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280c100720 msgr2=0x7f280c195ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:49.689+0000 7f2812fdb700 1 --2- 172.21.15.16:0/3467483386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280c100720 0x7f280c195ba0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f27fc006010 tx=0x7f27fc0050b0 comp rx=0 tx=0).stop 2024-04-02T04:33:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:49.689+0000 7f2812fdb700 1 -- 172.21.15.16:0/3467483386 shutdown_connections 2024-04-02T04:33:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:49.689+0000 7f2812fdb700 1 --2- 172.21.15.16:0/3467483386 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f27f40387d0 0x7f27f403ac90 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:49.689+0000 7f2812fdb700 1 --2- 172.21.15.16:0/3467483386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280c100720 0x7f280c195ba0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:49.689+0000 7f2812fdb700 1 -- 172.21.15.16:0/3467483386 >> 172.21.15.16:0/3467483386 conn(0x7f280c0fbcc0 msgr2=0x7f280c0fc790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:49.689+0000 7f2812fdb700 1 -- 172.21.15.16:0/3467483386 shutdown_connections 2024-04-02T04:33:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:49.689+0000 7f2812fdb700 1 -- 172.21.15.16:0/3467483386 wait complete. 2024-04-02T04:33:50.105 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:49 smithi016 ceph-mon[28969]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:33:50.105 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:49 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:33:50.105 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:49 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:33:50.105 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:49 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:33:50.105 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:49 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:33:50.106 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:49 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:33:50.106 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:49 smithi016 ceph-mon[28969]: Deploying daemon node-exporter.smithi016 on smithi016 2024-04-02T04:33:50.106 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:49 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:33:50.106 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:49 smithi016 ceph-mon[28969]: Added host smithi027 2024-04-02T04:33:50.239 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph orch host ls --format=json 2024-04-02T04:33:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:33:52.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:51 smithi016 ceph-mon[28969]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:33:52.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.696+0000 7fa816b6b700 1 -- 172.21.15.16:0/2397414537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8100ff640 msgr2=0x7fa8100ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:52.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.696+0000 7fa816b6b700 1 --2- 172.21.15.16:0/2397414537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8100ff640 0x7fa8100ffa60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa804009910 tx=0x7fa804009c20 comp rx=0 tx=0).stop 2024-04-02T04:33:52.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.696+0000 7fa816b6b700 1 -- 172.21.15.16:0/2397414537 shutdown_connections 2024-04-02T04:33:52.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.696+0000 7fa816b6b700 1 --2- 172.21.15.16:0/2397414537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8100ff640 0x7fa8100ffa60 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:52.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.696+0000 7fa816b6b700 1 -- 172.21.15.16:0/2397414537 >> 172.21.15.16:0/2397414537 conn(0x7fa8100fabe0 msgr2=0x7fa8100fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:52.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.696+0000 7fa816b6b700 1 -- 172.21.15.16:0/2397414537 shutdown_connections 2024-04-02T04:33:52.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.696+0000 7fa816b6b700 1 -- 172.21.15.16:0/2397414537 wait complete. 2024-04-02T04:33:52.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.696+0000 7fa816b6b700 1 Processor -- start 2024-04-02T04:33:52.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.697+0000 7fa816b6b700 1 -- start start 2024-04-02T04:33:52.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.697+0000 7fa816b6b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8100ff640 0x7fa810195bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:52.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.697+0000 7fa816b6b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa810196100 con 0x7fa8100ff640 2024-04-02T04:33:52.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.697+0000 7fa814907700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8100ff640 0x7fa810195bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:52.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.697+0000 7fa814907700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8100ff640 0x7fa810195bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53130/0 (socket says 172.21.15.16:53130) 2024-04-02T04:33:52.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.697+0000 7fa814907700 1 -- 172.21.15.16:0/2954302182 learned_addr learned my addr 172.21.15.16:0/2954302182 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:52.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.697+0000 7fa814907700 1 -- 172.21.15.16:0/2954302182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8040095f0 con 0x7fa8100ff640 2024-04-02T04:33:52.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.697+0000 7fa814907700 1 --2- 172.21.15.16:0/2954302182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8100ff640 0x7fa810195bc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa804006010 tx=0x7fa8040050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.697+0000 7fa80dffb700 1 -- 172.21.15.16:0/2954302182 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa80401d070 con 0x7fa8100ff640 2024-04-02T04:33:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.697+0000 7fa816b6b700 1 -- 172.21.15.16:0/2954302182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa810196300 con 0x7fa8100ff640 2024-04-02T04:33:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.697+0000 7fa80dffb700 1 -- 172.21.15.16:0/2954302182 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa80400be20 con 0x7fa8100ff640 2024-04-02T04:33:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.697+0000 7fa80dffb700 1 -- 172.21.15.16:0/2954302182 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa804022de0 con 0x7fa8100ff640 2024-04-02T04:33:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.698+0000 7fa816b6b700 1 -- 172.21.15.16:0/2954302182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8101967a0 con 0x7fa8100ff640 2024-04-02T04:33:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.698+0000 7fa80dffb700 1 -- 172.21.15.16:0/2954302182 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45604+0+0 (secure 0 0 0) 0x7fa80400f4b0 con 0x7fa8100ff640 2024-04-02T04:33:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.698+0000 7fa816b6b700 1 -- 172.21.15.16:0/2954302182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa81018f6f0 con 0x7fa8100ff640 2024-04-02T04:33:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.699+0000 7fa80dffb700 1 --2- 172.21.15.16:0/2954302182 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fa8000387b0 0x7fa80003ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.699+0000 7fa80dffb700 1 -- 172.21.15.16:0/2954302182 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa80404ccf0 con 0x7fa8100ff640 2024-04-02T04:33:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.699+0000 7fa80ffff700 1 --2- 172.21.15.16:0/2954302182 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fa8000387b0 0x7fa80003ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.699+0000 7fa80ffff700 1 --2- 172.21.15.16:0/2954302182 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fa8000387b0 0x7fa80003ac70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc006fd0 tx=0x7fa7fc006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:52.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.703+0000 7fa80dffb700 1 -- 172.21.15.16:0/2954302182 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa804051070 con 0x7fa8100ff640 2024-04-02T04:33:52.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.912+0000 7fa816b6b700 1 -- 172.21.15.16:0/2954302182 --> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa8100611f0 con 0x7fa8000387b0 2024-04-02T04:33:52.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.913+0000 7fa80dffb700 1 -- 172.21.15.16:0/2954302182 <== mgr.14162 v2:172.21.15.16:6800/3912038174 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7fa8100611f0 con 0x7fa8000387b0 2024-04-02T04:33:52.916 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:33:52.916 INFO:teuthology.orchestra.run.smithi016.stdout:[{"addr": "172.21.15.16", "hostname": "smithi016", "labels": [], "status": ""}, {"addr": "172.21.15.27", "hostname": "smithi027", "labels": [], "status": ""}] 2024-04-02T04:33:52.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.915+0000 7fa816b6b700 1 -- 172.21.15.16:0/2954302182 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fa8000387b0 msgr2=0x7fa80003ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:52.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.915+0000 7fa816b6b700 1 --2- 172.21.15.16:0/2954302182 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fa8000387b0 0x7fa80003ac70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc006fd0 tx=0x7fa7fc006e40 comp rx=0 tx=0).stop 2024-04-02T04:33:52.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.915+0000 7fa816b6b700 1 -- 172.21.15.16:0/2954302182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8100ff640 msgr2=0x7fa810195bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:52.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.915+0000 7fa816b6b700 1 --2- 172.21.15.16:0/2954302182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8100ff640 0x7fa810195bc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa804006010 tx=0x7fa8040050b0 comp rx=0 tx=0).stop 2024-04-02T04:33:52.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.915+0000 7fa816b6b700 1 -- 172.21.15.16:0/2954302182 shutdown_connections 2024-04-02T04:33:52.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.915+0000 7fa816b6b700 1 --2- 172.21.15.16:0/2954302182 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fa8000387b0 0x7fa80003ac70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:52.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.915+0000 7fa816b6b700 1 --2- 172.21.15.16:0/2954302182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8100ff640 0x7fa810195bc0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:52.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.915+0000 7fa816b6b700 1 -- 172.21.15.16:0/2954302182 >> 172.21.15.16:0/2954302182 conn(0x7fa8100fabe0 msgr2=0x7fa810068c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:52.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.915+0000 7fa816b6b700 1 -- 172.21.15.16:0/2954302182 shutdown_connections 2024-04-02T04:33:52.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:52.915+0000 7fa816b6b700 1 -- 172.21.15.16:0/2954302182 wait complete. 2024-04-02T04:33:53.792 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-02T04:33:53.792 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph osd crush tunables default 2024-04-02T04:33:54.228 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:33:55.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:54 smithi016 ceph-mon[28969]: from='client.14191 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T04:33:55.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:54 smithi016 ceph-mon[28969]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:33:55.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:54 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:33:55.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:54 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:33:55.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:54 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:33:55.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:54 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:33:55.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:54 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:33:55.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:55.738+0000 7fb266bde700 1 -- 172.21.15.16:0/3543333728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb260100720 msgr2=0x7fb260100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:55.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:55.738+0000 7fb266bde700 1 --2- 172.21.15.16:0/3543333728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb260100720 0x7fb260100b40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb254009910 tx=0x7fb254009c20 comp rx=0 tx=0).stop 2024-04-02T04:33:55.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:55.738+0000 7fb266bde700 1 -- 172.21.15.16:0/3543333728 shutdown_connections 2024-04-02T04:33:55.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:55.738+0000 7fb266bde700 1 --2- 172.21.15.16:0/3543333728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb260100720 0x7fb260100b40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:55.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:55.738+0000 7fb266bde700 1 -- 172.21.15.16:0/3543333728 >> 172.21.15.16:0/3543333728 conn(0x7fb2600fbcc0 msgr2=0x7fb2600fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:55.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:55.738+0000 7fb266bde700 1 -- 172.21.15.16:0/3543333728 shutdown_connections 2024-04-02T04:33:55.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:55.738+0000 7fb266bde700 1 -- 172.21.15.16:0/3543333728 wait complete. 2024-04-02T04:33:55.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:55.739+0000 7fb266bde700 1 Processor -- start 2024-04-02T04:33:55.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:55.739+0000 7fb266bde700 1 -- start start 2024-04-02T04:33:55.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:55.739+0000 7fb266bde700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb260100720 0x7fb260197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:55.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:55.739+0000 7fb266bde700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb260198320 con 0x7fb260100720 2024-04-02T04:33:55.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:55.739+0000 7fb26497a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb260100720 0x7fb260197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:55.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:55.739+0000 7fb26497a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb260100720 0x7fb260197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53158/0 (socket says 172.21.15.16:53158) 2024-04-02T04:33:55.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:55.739+0000 7fb26497a700 1 -- 172.21.15.16:0/4081788697 learned_addr learned my addr 172.21.15.16:0/4081788697 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:33:55.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:55.739+0000 7fb26497a700 1 -- 172.21.15.16:0/4081788697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2540095f0 con 0x7fb260100720 2024-04-02T04:33:55.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:55.740+0000 7fb26497a700 1 --2- 172.21.15.16:0/4081788697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb260100720 0x7fb260197de0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb254006010 tx=0x7fb254009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:55.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:55.740+0000 7fb25dffb700 1 -- 172.21.15.16:0/4081788697 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb25401d070 con 0x7fb260100720 2024-04-02T04:33:55.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:55.740+0000 7fb25dffb700 1 -- 172.21.15.16:0/4081788697 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb254005dc0 con 0x7fb260100720 2024-04-02T04:33:55.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:55.740+0000 7fb266bde700 1 -- 172.21.15.16:0/4081788697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb260198520 con 0x7fb260100720 2024-04-02T04:33:55.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:55.740+0000 7fb25dffb700 1 -- 172.21.15.16:0/4081788697 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb254022e90 con 0x7fb260100720 2024-04-02T04:33:55.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:55.740+0000 7fb266bde700 1 -- 172.21.15.16:0/4081788697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2601989c0 con 0x7fb260100720 2024-04-02T04:33:55.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:55.741+0000 7fb25dffb700 1 -- 172.21.15.16:0/4081788697 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45604+0+0 (secure 0 0 0) 0x7fb25400f4b0 con 0x7fb260100720 2024-04-02T04:33:55.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:55.741+0000 7fb25dffb700 1 --2- 172.21.15.16:0/4081788697 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fb2500387c0 0x7fb25003ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:55.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:55.741+0000 7fb25dffb700 1 -- 172.21.15.16:0/4081788697 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb2540224e0 con 0x7fb260100720 2024-04-02T04:33:55.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:55.741+0000 7fb266bde700 1 -- 172.21.15.16:0/4081788697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb260191950 con 0x7fb260100720 2024-04-02T04:33:55.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:55.741+0000 7fb25ffff700 1 --2- 172.21.15.16:0/4081788697 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fb2500387c0 0x7fb25003ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:55.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:55.742+0000 7fb25ffff700 1 --2- 172.21.15.16:0/4081788697 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fb2500387c0 0x7fb25003ac80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb24c006fd0 tx=0x7fb24c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:55.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:55.746+0000 7fb25dffb700 1 -- 172.21.15.16:0/4081788697 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb254027070 con 0x7fb260100720 2024-04-02T04:33:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:55.961+0000 7fb266bde700 1 -- 172.21.15.16:0/4081788697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fb2600623e0 con 0x7fb260100720 2024-04-02T04:33:55.964 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:55 smithi016 ceph-mon[28969]: Deploying daemon alertmanager.smithi016 on smithi016 2024-04-02T04:33:56.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:56.661+0000 7fb25dffb700 1 -- 172.21.15.16:0/4081788697 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fb254034070 con 0x7fb260100720 2024-04-02T04:33:56.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:56.663+0000 7fb266bde700 1 -- 172.21.15.16:0/4081788697 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fb2500387c0 msgr2=0x7fb25003ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:56.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:56.663+0000 7fb266bde700 1 --2- 172.21.15.16:0/4081788697 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fb2500387c0 0x7fb25003ac80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb24c006fd0 tx=0x7fb24c006e40 comp rx=0 tx=0).stop 2024-04-02T04:33:56.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:56.663+0000 7fb266bde700 1 -- 172.21.15.16:0/4081788697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb260100720 msgr2=0x7fb260197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:56.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:56.663+0000 7fb266bde700 1 --2- 172.21.15.16:0/4081788697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb260100720 0x7fb260197de0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb254006010 tx=0x7fb254009f80 comp rx=0 tx=0).stop 2024-04-02T04:33:56.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:56.663+0000 7fb266bde700 1 -- 172.21.15.16:0/4081788697 shutdown_connections 2024-04-02T04:33:56.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:56.663+0000 7fb266bde700 1 --2- 172.21.15.16:0/4081788697 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fb2500387c0 0x7fb25003ac80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:56.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:56.663+0000 7fb266bde700 1 --2- 172.21.15.16:0/4081788697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb260100720 0x7fb260197de0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:56.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:56.663+0000 7fb266bde700 1 -- 172.21.15.16:0/4081788697 >> 172.21.15.16:0/4081788697 conn(0x7fb2600fbcc0 msgr2=0x7fb2600fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:56.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:56.664+0000 7fb266bde700 1 -- 172.21.15.16:0/4081788697 shutdown_connections 2024-04-02T04:33:56.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:33:56.664+0000 7fb266bde700 1 -- 172.21.15.16:0/4081788697 wait complete. 2024-04-02T04:33:56.668 INFO:teuthology.orchestra.run.smithi016.stderr:adjusted tunables profile to default 2024-04-02T04:33:57.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:56 smithi016 ceph-mon[28969]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:33:57.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:56 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/4081788697' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-02T04:33:57.798 INFO:tasks.cephadm:Adding mon.smithi016 on smithi016 2024-04-02T04:33:57.799 INFO:tasks.cephadm:Adding mon.smithi027 on smithi027 2024-04-02T04:33:57.799 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph orch apply mon '2;smithi016:172.21.15.16=smithi016;smithi027:172.21.15.27=smithi027' 2024-04-02T04:33:58.013 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:33:58.059 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:33:58.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:57 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/4081788697' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-02T04:33:58.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:57 smithi016 ceph-mon[28969]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T04:33:59.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:58 smithi016 ceph-mon[28969]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:33:59.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.646+0000 7f6e41ac6700 1 -- 172.21.15.27:0/2088878863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e3c100720 msgr2=0x7f6e3c100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:59.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.646+0000 7f6e41ac6700 1 --2- 172.21.15.27:0/2088878863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e3c100720 0x7f6e3c100b40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c009910 tx=0x7f6e2c009c20 comp rx=0 tx=0).stop 2024-04-02T04:33:59.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.646+0000 7f6e41ac6700 1 -- 172.21.15.27:0/2088878863 shutdown_connections 2024-04-02T04:33:59.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.646+0000 7f6e41ac6700 1 --2- 172.21.15.27:0/2088878863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e3c100720 0x7f6e3c100b40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:59.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.646+0000 7f6e41ac6700 1 -- 172.21.15.27:0/2088878863 >> 172.21.15.27:0/2088878863 conn(0x7f6e3c0fbcc0 msgr2=0x7f6e3c0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:59.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.646+0000 7f6e41ac6700 1 -- 172.21.15.27:0/2088878863 shutdown_connections 2024-04-02T04:33:59.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.646+0000 7f6e41ac6700 1 -- 172.21.15.27:0/2088878863 wait complete. 2024-04-02T04:33:59.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.647+0000 7f6e41ac6700 1 Processor -- start 2024-04-02T04:33:59.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.647+0000 7f6e41ac6700 1 -- start start 2024-04-02T04:33:59.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.647+0000 7f6e41ac6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e3c100720 0x7f6e3c195bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:59.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.647+0000 7f6e41ac6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3c196130 con 0x7f6e3c100720 2024-04-02T04:33:59.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.647+0000 7f6e3b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e3c100720 0x7f6e3c195bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:59.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.647+0000 7f6e3b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e3c100720 0x7f6e3c195bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.27:48288/0 (socket says 172.21.15.27:48288) 2024-04-02T04:33:59.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.647+0000 7f6e3b7fe700 1 -- 172.21.15.27:0/1439762769 learned_addr learned my addr 172.21.15.27:0/1439762769 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-02T04:33:59.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.648+0000 7f6e3b7fe700 1 -- 172.21.15.27:0/1439762769 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e2c0095f0 con 0x7f6e3c100720 2024-04-02T04:33:59.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.648+0000 7f6e3b7fe700 1 --2- 172.21.15.27:0/1439762769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e3c100720 0x7f6e3c195bf0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c006010 tx=0x7f6e2c009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:59.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.648+0000 7f6e38ff9700 1 -- 172.21.15.27:0/1439762769 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6e2c01d070 con 0x7f6e3c100720 2024-04-02T04:33:59.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.648+0000 7f6e41ac6700 1 -- 172.21.15.27:0/1439762769 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e3c196330 con 0x7f6e3c100720 2024-04-02T04:33:59.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.648+0000 7f6e38ff9700 1 -- 172.21.15.27:0/1439762769 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e2c005dc0 con 0x7f6e3c100720 2024-04-02T04:33:59.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.648+0000 7f6e41ac6700 1 -- 172.21.15.27:0/1439762769 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e3c1967d0 con 0x7f6e3c100720 2024-04-02T04:33:59.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.648+0000 7f6e38ff9700 1 -- 172.21.15.27:0/1439762769 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6e2c022e50 con 0x7f6e3c100720 2024-04-02T04:33:59.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.649+0000 7f6e38ff9700 1 -- 172.21.15.27:0/1439762769 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45604+0+0 (secure 0 0 0) 0x7f6e2c00f460 con 0x7f6e3c100720 2024-04-02T04:33:59.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.649+0000 7f6e38ff9700 1 --2- 172.21.15.27:0/1439762769 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f6e24038770 0x7f6e2403ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:33:59.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.649+0000 7f6e41ac6700 1 -- 172.21.15.27:0/1439762769 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e3c18f550 con 0x7f6e3c100720 2024-04-02T04:33:59.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.649+0000 7f6e38ff9700 1 -- 172.21.15.27:0/1439762769 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6e2c04c9a0 con 0x7f6e3c100720 2024-04-02T04:33:59.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.649+0000 7f6e3affd700 1 --2- 172.21.15.27:0/1439762769 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f6e24038770 0x7f6e2403ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:33:59.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.650+0000 7f6e3affd700 1 --2- 172.21.15.27:0/1439762769 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f6e24038770 0x7f6e2403ac30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6e30006fd0 tx=0x7f6e30006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:33:59.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.654+0000 7f6e38ff9700 1 -- 172.21.15.27:0/1439762769 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6e2c02a950 con 0x7f6e3c100720 2024-04-02T04:33:59.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.867+0000 7f6e41ac6700 1 -- 172.21.15.27:0/1439762769 --> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi016:172.21.15.16=smithi016;smithi027:172.21.15.27=smithi027", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e3c02cd20 con 0x7f6e24038770 2024-04-02T04:33:59.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.873+0000 7f6e38ff9700 1 -- 172.21.15.27:0/1439762769 <== mgr.14162 v2:172.21.15.16:6800/3912038174 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f6e3c02cd20 con 0x7f6e24038770 2024-04-02T04:33:59.876 INFO:teuthology.orchestra.run.smithi027.stdout:Scheduled mon update... 2024-04-02T04:33:59.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.875+0000 7f6e41ac6700 1 -- 172.21.15.27:0/1439762769 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f6e24038770 msgr2=0x7f6e2403ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:59.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.875+0000 7f6e41ac6700 1 --2- 172.21.15.27:0/1439762769 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f6e24038770 0x7f6e2403ac30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6e30006fd0 tx=0x7f6e30006e40 comp rx=0 tx=0).stop 2024-04-02T04:33:59.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.875+0000 7f6e41ac6700 1 -- 172.21.15.27:0/1439762769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e3c100720 msgr2=0x7f6e3c195bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:33:59.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.875+0000 7f6e41ac6700 1 --2- 172.21.15.27:0/1439762769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e3c100720 0x7f6e3c195bf0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c006010 tx=0x7f6e2c009f80 comp rx=0 tx=0).stop 2024-04-02T04:33:59.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.875+0000 7f6e41ac6700 1 -- 172.21.15.27:0/1439762769 shutdown_connections 2024-04-02T04:33:59.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.875+0000 7f6e41ac6700 1 --2- 172.21.15.27:0/1439762769 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f6e24038770 0x7f6e2403ac30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:59.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.875+0000 7f6e41ac6700 1 --2- 172.21.15.27:0/1439762769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e3c100720 0x7f6e3c195bf0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:33:59.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.875+0000 7f6e41ac6700 1 -- 172.21.15.27:0/1439762769 >> 172.21.15.27:0/1439762769 conn(0x7f6e3c0fbcc0 msgr2=0x7f6e3c0fc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:33:59.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.875+0000 7f6e41ac6700 1 -- 172.21.15.27:0/1439762769 shutdown_connections 2024-04-02T04:33:59.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:33:59.875+0000 7f6e41ac6700 1 -- 172.21.15.27:0/1439762769 wait complete. 2024-04-02T04:34:00.244 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:59 smithi016 ceph-mon[28969]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:34:00.244 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:59 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:00.244 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:59 smithi016 ceph-mon[28969]: from='client.14195 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi016:172.21.15.16=smithi016;smithi027:172.21.15.27=smithi027", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:34:00.244 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:59 smithi016 ceph-mon[28969]: Saving service mon spec with placement smithi016:172.21.15.16=smithi016;smithi027:172.21.15.27=smithi027;count:2 2024-04-02T04:34:00.245 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:33:59 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:00.473 DEBUG:teuthology.orchestra.run.smithi027:mon.smithi027> sudo journalctl -f -n 0 -u ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@mon.smithi027.service 2024-04-02T04:34:00.476 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-02T04:34:00.476 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T04:34:00.717 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:34:00.764 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:34:02.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.261+0000 7f4e06442700 1 -- 172.21.15.27:0/3861097250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e000ff640 msgr2=0x7f4e000ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:02.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.261+0000 7f4e06442700 1 --2- 172.21.15.27:0/3861097250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e000ff640 0x7f4e000ffa60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4df0009910 tx=0x7f4df0009c20 comp rx=0 tx=0).stop 2024-04-02T04:34:02.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.261+0000 7f4e06442700 1 -- 172.21.15.27:0/3861097250 shutdown_connections 2024-04-02T04:34:02.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.261+0000 7f4e06442700 1 --2- 172.21.15.27:0/3861097250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e000ff640 0x7f4e000ffa60 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:02.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.261+0000 7f4e06442700 1 -- 172.21.15.27:0/3861097250 >> 172.21.15.27:0/3861097250 conn(0x7f4e000fac00 msgr2=0x7f4e000fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:34:02.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.261+0000 7f4e06442700 1 -- 172.21.15.27:0/3861097250 shutdown_connections 2024-04-02T04:34:02.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.261+0000 7f4e06442700 1 -- 172.21.15.27:0/3861097250 wait complete. 2024-04-02T04:34:02.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.261+0000 7f4e06442700 1 Processor -- start 2024-04-02T04:34:02.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.261+0000 7f4e06442700 1 -- start start 2024-04-02T04:34:02.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.262+0000 7f4e06442700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e000ff640 0x7f4e00195be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:34:02.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.262+0000 7f4e06442700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e00196120 con 0x7f4e000ff640 2024-04-02T04:34:02.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.262+0000 7f4dfffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e000ff640 0x7f4e00195be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:34:02.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.262+0000 7f4dfffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e000ff640 0x7f4e00195be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.27:57626/0 (socket says 172.21.15.27:57626) 2024-04-02T04:34:02.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.262+0000 7f4dfffff700 1 -- 172.21.15.27:0/1219264305 learned_addr learned my addr 172.21.15.27:0/1219264305 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-02T04:34:02.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.262+0000 7f4dfffff700 1 -- 172.21.15.27:0/1219264305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df00095f0 con 0x7f4e000ff640 2024-04-02T04:34:02.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.262+0000 7f4dfffff700 1 --2- 172.21.15.27:0/1219264305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e000ff640 0x7f4e00195be0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4df0006010 tx=0x7f4df0009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:34:02.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.263+0000 7f4dfd7fa700 1 -- 172.21.15.27:0/1219264305 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4df001d070 con 0x7f4e000ff640 2024-04-02T04:34:02.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.263+0000 7f4e06442700 1 -- 172.21.15.27:0/1219264305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e00196320 con 0x7f4e000ff640 2024-04-02T04:34:02.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.263+0000 7f4dfd7fa700 1 -- 172.21.15.27:0/1219264305 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4df0005dc0 con 0x7f4e000ff640 2024-04-02T04:34:02.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.263+0000 7f4dfd7fa700 1 -- 172.21.15.27:0/1219264305 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4df0022e90 con 0x7f4e000ff640 2024-04-02T04:34:02.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.263+0000 7f4e06442700 1 -- 172.21.15.27:0/1219264305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e001967c0 con 0x7f4e000ff640 2024-04-02T04:34:02.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.264+0000 7f4e06442700 1 -- 172.21.15.27:0/1219264305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e0018f6c0 con 0x7f4e000ff640 2024-04-02T04:34:02.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.264+0000 7f4dfd7fa700 1 -- 172.21.15.27:0/1219264305 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45604+0+0 (secure 0 0 0) 0x7f4df0022470 con 0x7f4e000ff640 2024-04-02T04:34:02.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.264+0000 7f4dfd7fa700 1 --2- 172.21.15.27:0/1219264305 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f4de8038760 0x7f4de803ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:34:02.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.264+0000 7f4dfd7fa700 1 -- 172.21.15.27:0/1219264305 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4df004c8b0 con 0x7f4e000ff640 2024-04-02T04:34:02.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.264+0000 7f4dff7fe700 1 --2- 172.21.15.27:0/1219264305 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f4de8038760 0x7f4de803ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:34:02.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.265+0000 7f4dff7fe700 1 --2- 172.21.15.27:0/1219264305 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f4de8038760 0x7f4de803ac20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4df4006fd0 tx=0x7f4df4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:34:02.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.269+0000 7f4dfd7fa700 1 -- 172.21.15.27:0/1219264305 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4df000b960 con 0x7f4e000ff640 2024-04-02T04:34:02.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:01 smithi016 ceph-mon[28969]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:34:02.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.552+0000 7f4e06442700 1 -- 172.21.15.27:0/1219264305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4e000623e0 con 0x7f4e000ff640 2024-04-02T04:34:02.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.553+0000 7f4dfd7fa700 1 -- 172.21.15.27:0/1219264305 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f4e000623e0 con 0x7f4e000ff640 2024-04-02T04:34:02.555 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:34:02.556 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":1,"fsid":"dc5403b4-f0a9-11ee-b647-cb9ed24678a4","modified":"2024-04-02T04:32:20.006984Z","created":"2024-04-02T04:32:20.006984Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi016","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-02T04:34:02.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.555+0000 7f4e06442700 1 -- 172.21.15.27:0/1219264305 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f4de8038760 msgr2=0x7f4de803ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:02.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.555+0000 7f4e06442700 1 --2- 172.21.15.27:0/1219264305 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f4de8038760 0x7f4de803ac20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4df4006fd0 tx=0x7f4df4006e40 comp rx=0 tx=0).stop 2024-04-02T04:34:02.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.555+0000 7f4e06442700 1 -- 172.21.15.27:0/1219264305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e000ff640 msgr2=0x7f4e00195be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:02.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.555+0000 7f4e06442700 1 --2- 172.21.15.27:0/1219264305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e000ff640 0x7f4e00195be0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4df0006010 tx=0x7f4df0009f80 comp rx=0 tx=0).stop 2024-04-02T04:34:02.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.555+0000 7f4e06442700 1 -- 172.21.15.27:0/1219264305 shutdown_connections 2024-04-02T04:34:02.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.555+0000 7f4e06442700 1 --2- 172.21.15.27:0/1219264305 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f4de8038760 0x7f4de803ac20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:02.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.555+0000 7f4e06442700 1 --2- 172.21.15.27:0/1219264305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e000ff640 0x7f4e00195be0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:02.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.555+0000 7f4e06442700 1 -- 172.21.15.27:0/1219264305 >> 172.21.15.27:0/1219264305 conn(0x7f4e000fac00 msgr2=0x7f4e00068fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:34:02.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.555+0000 7f4e06442700 1 -- 172.21.15.27:0/1219264305 shutdown_connections 2024-04-02T04:34:02.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:02.555+0000 7f4e06442700 1 -- 172.21.15.27:0/1219264305 wait complete. 2024-04-02T04:34:02.559 INFO:teuthology.orchestra.run.smithi027.stderr:dumped monmap epoch 1 2024-04-02T04:34:03.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:02 smithi016 ceph-mon[28969]: from='client.? 172.21.15.27:0/1219264305' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T04:34:04.142 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-02T04:34:04.143 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T04:34:04.356 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:34:04.403 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:34:05.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:04 smithi016 ceph-mon[28969]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:34:05.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:04 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:05.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:04 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:05.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:04 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:05.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:04 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:05.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:04 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:05.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:04 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:05.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:04 smithi016 ceph-mon[28969]: Regenerating cephadm self-signed grafana TLS certificates 2024-04-02T04:34:05.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:04 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:05.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:04 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:05.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:04 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-02T04:34:05.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:04 smithi016 ceph-mon[28969]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-02T04:34:05.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:04 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:05.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:04 smithi016 ceph-mon[28969]: Deploying daemon grafana.smithi016 on smithi016 2024-04-02T04:34:05.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:04 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:05.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:05.952+0000 7ff2b033c700 1 -- 172.21.15.27:0/3030666041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a8100720 msgr2=0x7ff2a8100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:05.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:05.952+0000 7ff2b033c700 1 --2- 172.21.15.27:0/3030666041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a8100720 0x7ff2a8100b40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4009910 tx=0x7ff2a4009c20 comp rx=0 tx=0).stop 2024-04-02T04:34:05.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:05.952+0000 7ff2b033c700 1 -- 172.21.15.27:0/3030666041 shutdown_connections 2024-04-02T04:34:05.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:05.952+0000 7ff2b033c700 1 --2- 172.21.15.27:0/3030666041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a8100720 0x7ff2a8100b40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:05.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:05.952+0000 7ff2b033c700 1 -- 172.21.15.27:0/3030666041 >> 172.21.15.27:0/3030666041 conn(0x7ff2a80fbcc0 msgr2=0x7ff2a80fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:34:05.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:05.952+0000 7ff2b033c700 1 -- 172.21.15.27:0/3030666041 shutdown_connections 2024-04-02T04:34:05.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:05.952+0000 7ff2b033c700 1 -- 172.21.15.27:0/3030666041 wait complete. 2024-04-02T04:34:05.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:05.952+0000 7ff2b033c700 1 Processor -- start 2024-04-02T04:34:05.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:05.952+0000 7ff2b033c700 1 -- start start 2024-04-02T04:34:05.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:05.953+0000 7ff2b033c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a8100720 0x7ff2a8074a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:34:05.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:05.953+0000 7ff2b033c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a8074f60 con 0x7ff2a8100720 2024-04-02T04:34:05.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:05.953+0000 7ff2ae0d8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a8100720 0x7ff2a8074a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:34:05.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:05.953+0000 7ff2ae0d8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a8100720 0x7ff2a8074a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.27:57646/0 (socket says 172.21.15.27:57646) 2024-04-02T04:34:05.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:05.953+0000 7ff2ae0d8700 1 -- 172.21.15.27:0/1254440258 learned_addr learned my addr 172.21.15.27:0/1254440258 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-02T04:34:05.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:05.953+0000 7ff2ae0d8700 1 -- 172.21.15.27:0/1254440258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a40095f0 con 0x7ff2a8100720 2024-04-02T04:34:05.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:05.953+0000 7ff2ae0d8700 1 --2- 172.21.15.27:0/1254440258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a8100720 0x7ff2a8074a20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4006010 tx=0x7ff2a4009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:34:05.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:05.953+0000 7ff29f7fe700 1 -- 172.21.15.27:0/1254440258 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff2a401d070 con 0x7ff2a8100720 2024-04-02T04:34:05.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:05.953+0000 7ff29f7fe700 1 -- 172.21.15.27:0/1254440258 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff2a4005dc0 con 0x7ff2a8100720 2024-04-02T04:34:05.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:05.953+0000 7ff2b033c700 1 -- 172.21.15.27:0/1254440258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a80731a0 con 0x7ff2a8100720 2024-04-02T04:34:05.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:05.953+0000 7ff29f7fe700 1 -- 172.21.15.27:0/1254440258 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff2a4022e90 con 0x7ff2a8100720 2024-04-02T04:34:05.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:05.953+0000 7ff2b033c700 1 -- 172.21.15.27:0/1254440258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a8073640 con 0x7ff2a8100720 2024-04-02T04:34:05.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:05.954+0000 7ff29f7fe700 1 -- 172.21.15.27:0/1254440258 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45604+0+0 (secure 0 0 0) 0x7ff2a400f4f0 con 0x7ff2a8100720 2024-04-02T04:34:05.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:05.954+0000 7ff2b033c700 1 -- 172.21.15.27:0/1254440258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a8106e80 con 0x7ff2a8100720 2024-04-02T04:34:05.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:05.954+0000 7ff29f7fe700 1 --2- 172.21.15.27:0/1254440258 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7ff294038760 0x7ff29403ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:34:05.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:05.955+0000 7ff29f7fe700 1 -- 172.21.15.27:0/1254440258 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff2a404caf0 con 0x7ff2a8100720 2024-04-02T04:34:05.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:05.955+0000 7ff2ad8d7700 1 --2- 172.21.15.27:0/1254440258 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7ff294038760 0x7ff29403ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:34:05.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:05.955+0000 7ff2ad8d7700 1 --2- 172.21.15.27:0/1254440258 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7ff294038760 0x7ff29403ac20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff298006fd0 tx=0x7ff298006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:34:05.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:05.960+0000 7ff29f7fe700 1 -- 172.21.15.27:0/1254440258 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff2a4020d40 con 0x7ff2a8100720 2024-04-02T04:34:06.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:06.246+0000 7ff2b033c700 1 -- 172.21.15.27:0/1254440258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff2a80623e0 con 0x7ff2a8100720 2024-04-02T04:34:06.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:06.246+0000 7ff29f7fe700 1 -- 172.21.15.27:0/1254440258 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7ff2a4027020 con 0x7ff2a8100720 2024-04-02T04:34:06.250 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:34:06.250 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":1,"fsid":"dc5403b4-f0a9-11ee-b647-cb9ed24678a4","modified":"2024-04-02T04:32:20.006984Z","created":"2024-04-02T04:32:20.006984Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi016","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-02T04:34:06.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:06.248+0000 7ff2b033c700 1 -- 172.21.15.27:0/1254440258 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7ff294038760 msgr2=0x7ff29403ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:06.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:06.248+0000 7ff2b033c700 1 --2- 172.21.15.27:0/1254440258 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7ff294038760 0x7ff29403ac20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff298006fd0 tx=0x7ff298006e40 comp rx=0 tx=0).stop 2024-04-02T04:34:06.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:06.248+0000 7ff2b033c700 1 -- 172.21.15.27:0/1254440258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a8100720 msgr2=0x7ff2a8074a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:06.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:06.248+0000 7ff2b033c700 1 --2- 172.21.15.27:0/1254440258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a8100720 0x7ff2a8074a20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4006010 tx=0x7ff2a4009f80 comp rx=0 tx=0).stop 2024-04-02T04:34:06.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:06.248+0000 7ff2b033c700 1 -- 172.21.15.27:0/1254440258 shutdown_connections 2024-04-02T04:34:06.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:06.248+0000 7ff2b033c700 1 --2- 172.21.15.27:0/1254440258 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7ff294038760 0x7ff29403ac20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:06.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:06.248+0000 7ff2b033c700 1 --2- 172.21.15.27:0/1254440258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a8100720 0x7ff2a8074a20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:06.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:06.248+0000 7ff2b033c700 1 -- 172.21.15.27:0/1254440258 >> 172.21.15.27:0/1254440258 conn(0x7ff2a80fbcc0 msgr2=0x7ff2a80fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:34:06.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:06.248+0000 7ff2b033c700 1 -- 172.21.15.27:0/1254440258 shutdown_connections 2024-04-02T04:34:06.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:06.248+0000 7ff2b033c700 1 -- 172.21.15.27:0/1254440258 wait complete. 2024-04-02T04:34:06.253 INFO:teuthology.orchestra.run.smithi027.stderr:dumped monmap epoch 1 2024-04-02T04:34:07.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:06 smithi016 ceph-mon[28969]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:34:07.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:06 smithi016 ceph-mon[28969]: from='client.? 172.21.15.27:0/1254440258' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T04:34:07.874 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-02T04:34:07.874 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T04:34:08.085 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:34:08.132 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:34:09.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:08 smithi016 ceph-mon[28969]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:34:09.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.618+0000 7f894eb41700 1 -- 172.21.15.27:0/616839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89480ff640 msgr2=0x7f89480ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:09.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.618+0000 7f894eb41700 1 --2- 172.21.15.27:0/616839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89480ff640 0x7f89480ffa60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f893c009910 tx=0x7f893c009c20 comp rx=0 tx=0).stop 2024-04-02T04:34:09.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.618+0000 7f894eb41700 1 -- 172.21.15.27:0/616839 shutdown_connections 2024-04-02T04:34:09.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.618+0000 7f894eb41700 1 --2- 172.21.15.27:0/616839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89480ff640 0x7f89480ffa60 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:09.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.618+0000 7f894eb41700 1 -- 172.21.15.27:0/616839 >> 172.21.15.27:0/616839 conn(0x7f89480fabe0 msgr2=0x7f89480fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:34:09.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.618+0000 7f894eb41700 1 -- 172.21.15.27:0/616839 shutdown_connections 2024-04-02T04:34:09.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.618+0000 7f894eb41700 1 -- 172.21.15.27:0/616839 wait complete. 2024-04-02T04:34:09.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.619+0000 7f894eb41700 1 Processor -- start 2024-04-02T04:34:09.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.619+0000 7f894eb41700 1 -- start start 2024-04-02T04:34:09.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.619+0000 7f894eb41700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89480ff640 0x7f8948195bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:34:09.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.619+0000 7f894eb41700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8948196130 con 0x7f89480ff640 2024-04-02T04:34:09.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.619+0000 7f894c8dd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89480ff640 0x7f8948195bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:34:09.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.619+0000 7f894c8dd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89480ff640 0x7f8948195bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.27:57660/0 (socket says 172.21.15.27:57660) 2024-04-02T04:34:09.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.619+0000 7f894c8dd700 1 -- 172.21.15.27:0/18185488 learned_addr learned my addr 172.21.15.27:0/18185488 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-02T04:34:09.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.620+0000 7f894c8dd700 1 -- 172.21.15.27:0/18185488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f893c0095f0 con 0x7f89480ff640 2024-04-02T04:34:09.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.620+0000 7f894c8dd700 1 --2- 172.21.15.27:0/18185488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89480ff640 0x7f8948195bf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f893c006010 tx=0x7f893c009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:34:09.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.620+0000 7f8945ffb700 1 -- 172.21.15.27:0/18185488 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f893c01d070 con 0x7f89480ff640 2024-04-02T04:34:09.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.620+0000 7f8945ffb700 1 -- 172.21.15.27:0/18185488 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f893c005dc0 con 0x7f89480ff640 2024-04-02T04:34:09.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.620+0000 7f894eb41700 1 -- 172.21.15.27:0/18185488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8948196330 con 0x7f89480ff640 2024-04-02T04:34:09.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.620+0000 7f8945ffb700 1 -- 172.21.15.27:0/18185488 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f893c022e90 con 0x7f89480ff640 2024-04-02T04:34:09.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.620+0000 7f894eb41700 1 -- 172.21.15.27:0/18185488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89481967d0 con 0x7f89480ff640 2024-04-02T04:34:09.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.621+0000 7f894eb41700 1 -- 172.21.15.27:0/18185488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89480623e0 con 0x7f89480ff640 2024-04-02T04:34:09.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.621+0000 7f8945ffb700 1 -- 172.21.15.27:0/18185488 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45604+0+0 (secure 0 0 0) 0x7f893c022470 con 0x7f89480ff640 2024-04-02T04:34:09.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.621+0000 7f8945ffb700 1 --2- 172.21.15.27:0/18185488 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f8938038760 0x7f893803ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:34:09.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.622+0000 7f8945ffb700 1 -- 172.21.15.27:0/18185488 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f893c04c920 con 0x7f89480ff640 2024-04-02T04:34:09.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.622+0000 7f8947fff700 1 --2- 172.21.15.27:0/18185488 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f8938038760 0x7f893803ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:34:09.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.622+0000 7f8947fff700 1 --2- 172.21.15.27:0/18185488 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f8938038760 0x7f893803ac20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8934006fd0 tx=0x7f8934006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:34:09.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.626+0000 7f8945ffb700 1 -- 172.21.15.27:0/18185488 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f893c00b960 con 0x7f89480ff640 2024-04-02T04:34:09.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.911+0000 7f894eb41700 1 -- 172.21.15.27:0/18185488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8948105670 con 0x7f89480ff640 2024-04-02T04:34:09.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.912+0000 7f8945ffb700 1 -- 172.21.15.27:0/18185488 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f893c027020 con 0x7f89480ff640 2024-04-02T04:34:09.915 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:34:09.916 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":1,"fsid":"dc5403b4-f0a9-11ee-b647-cb9ed24678a4","modified":"2024-04-02T04:32:20.006984Z","created":"2024-04-02T04:32:20.006984Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi016","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-02T04:34:09.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.914+0000 7f894eb41700 1 -- 172.21.15.27:0/18185488 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f8938038760 msgr2=0x7f893803ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:09.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.914+0000 7f894eb41700 1 --2- 172.21.15.27:0/18185488 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f8938038760 0x7f893803ac20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8934006fd0 tx=0x7f8934006e40 comp rx=0 tx=0).stop 2024-04-02T04:34:09.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.914+0000 7f894eb41700 1 -- 172.21.15.27:0/18185488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89480ff640 msgr2=0x7f8948195bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:09.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.914+0000 7f894eb41700 1 --2- 172.21.15.27:0/18185488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89480ff640 0x7f8948195bf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f893c006010 tx=0x7f893c009f80 comp rx=0 tx=0).stop 2024-04-02T04:34:09.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.914+0000 7f894eb41700 1 -- 172.21.15.27:0/18185488 shutdown_connections 2024-04-02T04:34:09.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.914+0000 7f894eb41700 1 --2- 172.21.15.27:0/18185488 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f8938038760 0x7f893803ac20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:09.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.914+0000 7f894eb41700 1 --2- 172.21.15.27:0/18185488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89480ff640 0x7f8948195bf0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:09.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.914+0000 7f894eb41700 1 -- 172.21.15.27:0/18185488 >> 172.21.15.27:0/18185488 conn(0x7f89480fabe0 msgr2=0x7f8948067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:34:09.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.914+0000 7f894eb41700 1 -- 172.21.15.27:0/18185488 shutdown_connections 2024-04-02T04:34:09.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:09.914+0000 7f894eb41700 1 -- 172.21.15.27:0/18185488 wait complete. 2024-04-02T04:34:09.918 INFO:teuthology.orchestra.run.smithi027.stderr:dumped monmap epoch 1 2024-04-02T04:34:11.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:10 smithi016 ceph-mon[28969]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:34:11.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:10 smithi016 ceph-mon[28969]: from='client.? 172.21.15.27:0/18185488' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T04:34:11.554 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-02T04:34:11.554 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T04:34:11.771 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:34:11.818 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:34:13.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:12 smithi016 ceph-mon[28969]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:34:13.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.343+0000 7fdd75939700 1 -- 172.21.15.27:0/2588719826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd701011d0 msgr2=0x7fdd701035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:13.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.343+0000 7fdd75939700 1 --2- 172.21.15.27:0/2588719826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd701011d0 0x7fdd701035c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdd6800b0a0 tx=0x7fdd6800b3b0 comp rx=0 tx=0).stop 2024-04-02T04:34:13.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.343+0000 7fdd75939700 1 -- 172.21.15.27:0/2588719826 shutdown_connections 2024-04-02T04:34:13.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.343+0000 7fdd75939700 1 --2- 172.21.15.27:0/2588719826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd701011d0 0x7fdd701035c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:13.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.343+0000 7fdd75939700 1 -- 172.21.15.27:0/2588719826 >> 172.21.15.27:0/2588719826 conn(0x7fdd700fabe0 msgr2=0x7fdd700fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:34:13.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.343+0000 7fdd75939700 1 -- 172.21.15.27:0/2588719826 shutdown_connections 2024-04-02T04:34:13.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.343+0000 7fdd75939700 1 -- 172.21.15.27:0/2588719826 wait complete. 2024-04-02T04:34:13.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.344+0000 7fdd75939700 1 Processor -- start 2024-04-02T04:34:13.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.344+0000 7fdd75939700 1 -- start start 2024-04-02T04:34:13.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.344+0000 7fdd75939700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd701011d0 0x7fdd70195b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:34:13.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.344+0000 7fdd75939700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd701960c0 con 0x7fdd701011d0 2024-04-02T04:34:13.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.344+0000 7fdd6effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd701011d0 0x7fdd70195b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:34:13.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.345+0000 7fdd6effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd701011d0 0x7fdd70195b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.27:36604/0 (socket says 172.21.15.27:36604) 2024-04-02T04:34:13.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.345+0000 7fdd6effd700 1 -- 172.21.15.27:0/3621064842 learned_addr learned my addr 172.21.15.27:0/3621064842 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-02T04:34:13.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.345+0000 7fdd6effd700 1 -- 172.21.15.27:0/3621064842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd68009d00 con 0x7fdd701011d0 2024-04-02T04:34:13.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.345+0000 7fdd6effd700 1 --2- 172.21.15.27:0/3621064842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd701011d0 0x7fdd70195b80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdd68015040 tx=0x7fdd6800bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:34:13.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.345+0000 7fdd74937700 1 -- 172.21.15.27:0/3621064842 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdd6800e040 con 0x7fdd701011d0 2024-04-02T04:34:13.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.345+0000 7fdd75939700 1 -- 172.21.15.27:0/3621064842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd701962c0 con 0x7fdd701011d0 2024-04-02T04:34:13.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.345+0000 7fdd74937700 1 -- 172.21.15.27:0/3621064842 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd68003d70 con 0x7fdd701011d0 2024-04-02T04:34:13.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.345+0000 7fdd74937700 1 -- 172.21.15.27:0/3621064842 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdd6801c9e0 con 0x7fdd701011d0 2024-04-02T04:34:13.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.345+0000 7fdd75939700 1 -- 172.21.15.27:0/3621064842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd70196760 con 0x7fdd701011d0 2024-04-02T04:34:13.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.346+0000 7fdd74937700 1 -- 172.21.15.27:0/3621064842 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45604+0+0 (secure 0 0 0) 0x7fdd68012640 con 0x7fdd701011d0 2024-04-02T04:34:13.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.346+0000 7fdd75939700 1 -- 172.21.15.27:0/3621064842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd7018f570 con 0x7fdd701011d0 2024-04-02T04:34:13.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.346+0000 7fdd74937700 1 --2- 172.21.15.27:0/3621064842 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fdd5c038810 0x7fdd5c03acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:34:13.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.346+0000 7fdd74937700 1 -- 172.21.15.27:0/3621064842 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdd6804c930 con 0x7fdd701011d0 2024-04-02T04:34:13.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.346+0000 7fdd6e7fc700 1 --2- 172.21.15.27:0/3621064842 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fdd5c038810 0x7fdd5c03acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:34:13.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.347+0000 7fdd6e7fc700 1 --2- 172.21.15.27:0/3621064842 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fdd5c038810 0x7fdd5c03acd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdd58006fd0 tx=0x7fdd58006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:34:13.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.353+0000 7fdd74937700 1 -- 172.21.15.27:0/3621064842 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fdd6801a590 con 0x7fdd701011d0 2024-04-02T04:34:13.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.636+0000 7fdd75939700 1 -- 172.21.15.27:0/3621064842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fdd700623e0 con 0x7fdd701011d0 2024-04-02T04:34:13.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.636+0000 7fdd74937700 1 -- 172.21.15.27:0/3621064842 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fdd700623e0 con 0x7fdd701011d0 2024-04-02T04:34:13.641 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:34:13.641 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":1,"fsid":"dc5403b4-f0a9-11ee-b647-cb9ed24678a4","modified":"2024-04-02T04:32:20.006984Z","created":"2024-04-02T04:32:20.006984Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi016","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-02T04:34:13.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.638+0000 7fdd75939700 1 -- 172.21.15.27:0/3621064842 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fdd5c038810 msgr2=0x7fdd5c03acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:13.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.638+0000 7fdd75939700 1 --2- 172.21.15.27:0/3621064842 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fdd5c038810 0x7fdd5c03acd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdd58006fd0 tx=0x7fdd58006e40 comp rx=0 tx=0).stop 2024-04-02T04:34:13.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.638+0000 7fdd75939700 1 -- 172.21.15.27:0/3621064842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd701011d0 msgr2=0x7fdd70195b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:13.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.638+0000 7fdd75939700 1 --2- 172.21.15.27:0/3621064842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd701011d0 0x7fdd70195b80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdd68015040 tx=0x7fdd6800bd40 comp rx=0 tx=0).stop 2024-04-02T04:34:13.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.638+0000 7fdd75939700 1 -- 172.21.15.27:0/3621064842 shutdown_connections 2024-04-02T04:34:13.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.638+0000 7fdd75939700 1 --2- 172.21.15.27:0/3621064842 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fdd5c038810 0x7fdd5c03acd0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:13.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.638+0000 7fdd75939700 1 --2- 172.21.15.27:0/3621064842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd701011d0 0x7fdd70195b80 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:13.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.638+0000 7fdd75939700 1 -- 172.21.15.27:0/3621064842 >> 172.21.15.27:0/3621064842 conn(0x7fdd700fabe0 msgr2=0x7fdd700fb5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:34:13.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.639+0000 7fdd75939700 1 -- 172.21.15.27:0/3621064842 shutdown_connections 2024-04-02T04:34:13.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:13.639+0000 7fdd75939700 1 -- 172.21.15.27:0/3621064842 wait complete. 2024-04-02T04:34:13.645 INFO:teuthology.orchestra.run.smithi027.stderr:dumped monmap epoch 1 2024-04-02T04:34:13.809 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:13 smithi016 ceph-mon[28969]: from='client.? 172.21.15.27:0/3621064842' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T04:34:15.039 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:14 smithi016 ceph-mon[28969]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:34:15.223 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-02T04:34:15.223 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T04:34:15.437 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:34:15.484 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:34:17.053 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:16 smithi016 ceph-mon[28969]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:34:17.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.086+0000 7ff3b0809700 1 -- 172.21.15.27:0/451380203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3ac0ff640 msgr2=0x7ff3ac0ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:17.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.086+0000 7ff3b0809700 1 --2- 172.21.15.27:0/451380203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3ac0ff640 0x7ff3ac0ffa60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009910 tx=0x7ff39c009c20 comp rx=0 tx=0).stop 2024-04-02T04:34:17.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.086+0000 7ff3b0809700 1 -- 172.21.15.27:0/451380203 shutdown_connections 2024-04-02T04:34:17.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.086+0000 7ff3b0809700 1 --2- 172.21.15.27:0/451380203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3ac0ff640 0x7ff3ac0ffa60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:17.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.086+0000 7ff3b0809700 1 -- 172.21.15.27:0/451380203 >> 172.21.15.27:0/451380203 conn(0x7ff3ac0fabe0 msgr2=0x7ff3ac0fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:34:17.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.086+0000 7ff3b0809700 1 -- 172.21.15.27:0/451380203 shutdown_connections 2024-04-02T04:34:17.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.086+0000 7ff3b0809700 1 -- 172.21.15.27:0/451380203 wait complete. 2024-04-02T04:34:17.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.087+0000 7ff3b0809700 1 Processor -- start 2024-04-02T04:34:17.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.087+0000 7ff3b0809700 1 -- start start 2024-04-02T04:34:17.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.087+0000 7ff3b0809700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3ac0ff640 0x7ff3ac069570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:34:17.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.087+0000 7ff3b0809700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac069ab0 con 0x7ff3ac0ff640 2024-04-02T04:34:17.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.087+0000 7ff3aa59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3ac0ff640 0x7ff3ac069570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:34:17.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.087+0000 7ff3aa59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3ac0ff640 0x7ff3ac069570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.27:36620/0 (socket says 172.21.15.27:36620) 2024-04-02T04:34:17.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.087+0000 7ff3aa59c700 1 -- 172.21.15.27:0/2526627108 learned_addr learned my addr 172.21.15.27:0/2526627108 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-02T04:34:17.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.087+0000 7ff3aa59c700 1 -- 172.21.15.27:0/2526627108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff39c0095f0 con 0x7ff3ac0ff640 2024-04-02T04:34:17.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.088+0000 7ff3aa59c700 1 --2- 172.21.15.27:0/2526627108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3ac0ff640 0x7ff3ac069570 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff39c006010 tx=0x7ff39c009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:34:17.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.088+0000 7ff3a37fe700 1 -- 172.21.15.27:0/2526627108 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff39c01d070 con 0x7ff3ac0ff640 2024-04-02T04:34:17.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.088+0000 7ff3a37fe700 1 -- 172.21.15.27:0/2526627108 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff39c005dc0 con 0x7ff3ac0ff640 2024-04-02T04:34:17.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.088+0000 7ff3a37fe700 1 -- 172.21.15.27:0/2526627108 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff39c022e90 con 0x7ff3ac0ff640 2024-04-02T04:34:17.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.088+0000 7ff3b0809700 1 -- 172.21.15.27:0/2526627108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3ac198930 con 0x7ff3ac0ff640 2024-04-02T04:34:17.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.088+0000 7ff3b0809700 1 -- 172.21.15.27:0/2526627108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3ac198d70 con 0x7ff3ac0ff640 2024-04-02T04:34:17.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.089+0000 7ff3a37fe700 1 -- 172.21.15.27:0/2526627108 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45604+0+0 (secure 0 0 0) 0x7ff39c00b960 con 0x7ff3ac0ff640 2024-04-02T04:34:17.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.089+0000 7ff3b0809700 1 -- 172.21.15.27:0/2526627108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3ac18e470 con 0x7ff3ac0ff640 2024-04-02T04:34:17.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.089+0000 7ff3a37fe700 1 --2- 172.21.15.27:0/2526627108 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7ff398038760 0x7ff39803ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:34:17.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.089+0000 7ff3a37fe700 1 -- 172.21.15.27:0/2526627108 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff39c04c8c0 con 0x7ff3ac0ff640 2024-04-02T04:34:17.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.089+0000 7ff3a9d9b700 1 --2- 172.21.15.27:0/2526627108 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7ff398038760 0x7ff39803ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:34:17.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.089+0000 7ff3a9d9b700 1 --2- 172.21.15.27:0/2526627108 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7ff398038760 0x7ff39803ac20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff394006fd0 tx=0x7ff394006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:34:17.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.094+0000 7ff3a37fe700 1 -- 172.21.15.27:0/2526627108 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff39c022750 con 0x7ff3ac0ff640 2024-04-02T04:34:17.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.381+0000 7ff3b0809700 1 -- 172.21.15.27:0/2526627108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff3ac0623e0 con 0x7ff3ac0ff640 2024-04-02T04:34:17.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.382+0000 7ff3a37fe700 1 -- 172.21.15.27:0/2526627108 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7ff39c027020 con 0x7ff3ac0ff640 2024-04-02T04:34:17.386 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:34:17.386 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":1,"fsid":"dc5403b4-f0a9-11ee-b647-cb9ed24678a4","modified":"2024-04-02T04:32:20.006984Z","created":"2024-04-02T04:32:20.006984Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi016","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-02T04:34:17.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.384+0000 7ff3b0809700 1 -- 172.21.15.27:0/2526627108 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7ff398038760 msgr2=0x7ff39803ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:17.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.384+0000 7ff3b0809700 1 --2- 172.21.15.27:0/2526627108 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7ff398038760 0x7ff39803ac20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff394006fd0 tx=0x7ff394006e40 comp rx=0 tx=0).stop 2024-04-02T04:34:17.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.384+0000 7ff3b0809700 1 -- 172.21.15.27:0/2526627108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3ac0ff640 msgr2=0x7ff3ac069570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:17.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.384+0000 7ff3b0809700 1 --2- 172.21.15.27:0/2526627108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3ac0ff640 0x7ff3ac069570 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff39c006010 tx=0x7ff39c009f80 comp rx=0 tx=0).stop 2024-04-02T04:34:17.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.384+0000 7ff3b0809700 1 -- 172.21.15.27:0/2526627108 shutdown_connections 2024-04-02T04:34:17.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.384+0000 7ff3b0809700 1 --2- 172.21.15.27:0/2526627108 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7ff398038760 0x7ff39803ac20 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:17.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.384+0000 7ff3b0809700 1 --2- 172.21.15.27:0/2526627108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3ac0ff640 0x7ff3ac069570 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:17.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.384+0000 7ff3b0809700 1 -- 172.21.15.27:0/2526627108 >> 172.21.15.27:0/2526627108 conn(0x7ff3ac0fabe0 msgr2=0x7ff3ac0fb730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:34:17.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.384+0000 7ff3b0809700 1 -- 172.21.15.27:0/2526627108 shutdown_connections 2024-04-02T04:34:17.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:17.384+0000 7ff3b0809700 1 -- 172.21.15.27:0/2526627108 wait complete. 2024-04-02T04:34:17.389 INFO:teuthology.orchestra.run.smithi027.stderr:dumped monmap epoch 1 2024-04-02T04:34:18.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:17 smithi016 ceph-mon[28969]: from='client.? 172.21.15.27:0/2526627108' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T04:34:18.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:17 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:18.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:17 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:18.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:17 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:18.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:17 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:18.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:17 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:18.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:17 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:18.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:17 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:18.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:17 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:18.849 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:18 smithi016 ceph-mon[28969]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:34:18.850 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:18 smithi016 ceph-mon[28969]: Deploying daemon prometheus.smithi016 on smithi016 2024-04-02T04:34:18.985 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-02T04:34:18.985 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T04:34:19.203 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:34:19.249 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:34:20.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:19 smithi016 ceph-mon[28969]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:34:20.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:19 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:20.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:20.799+0000 7f6ff1857700 1 -- 172.21.15.27:0/3672523638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fec100720 msgr2=0x7f6fec100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:20.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:20.799+0000 7f6ff1857700 1 --2- 172.21.15.27:0/3672523638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fec100720 0x7f6fec100b40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc009910 tx=0x7f6fdc009c20 comp rx=0 tx=0).stop 2024-04-02T04:34:20.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:20.799+0000 7f6ff1857700 1 -- 172.21.15.27:0/3672523638 shutdown_connections 2024-04-02T04:34:20.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:20.799+0000 7f6ff1857700 1 --2- 172.21.15.27:0/3672523638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fec100720 0x7f6fec100b40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:20.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:20.799+0000 7f6ff1857700 1 -- 172.21.15.27:0/3672523638 >> 172.21.15.27:0/3672523638 conn(0x7f6fec0fbcc0 msgr2=0x7f6fec0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:34:20.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:20.799+0000 7f6ff1857700 1 -- 172.21.15.27:0/3672523638 shutdown_connections 2024-04-02T04:34:20.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:20.799+0000 7f6ff1857700 1 -- 172.21.15.27:0/3672523638 wait complete. 2024-04-02T04:34:20.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:20.799+0000 7f6ff1857700 1 Processor -- start 2024-04-02T04:34:20.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:20.799+0000 7f6ff1857700 1 -- start start 2024-04-02T04:34:20.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:20.799+0000 7f6ff1857700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fec100720 0x7f6fec195bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:34:20.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:20.799+0000 7f6ff1857700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec196100 con 0x7f6fec100720 2024-04-02T04:34:20.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:20.799+0000 7f6feaffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fec100720 0x7f6fec195bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:34:20.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:20.799+0000 7f6feaffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fec100720 0x7f6fec195bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.27:36644/0 (socket says 172.21.15.27:36644) 2024-04-02T04:34:20.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:20.799+0000 7f6feaffd700 1 -- 172.21.15.27:0/2585167345 learned_addr learned my addr 172.21.15.27:0/2585167345 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-02T04:34:20.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:20.800+0000 7f6feaffd700 1 -- 172.21.15.27:0/2585167345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fdc0095f0 con 0x7f6fec100720 2024-04-02T04:34:20.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:20.800+0000 7f6feaffd700 1 --2- 172.21.15.27:0/2585167345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fec100720 0x7f6fec195bc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc006010 tx=0x7f6fdc0050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:34:20.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:20.800+0000 7f6ff0855700 1 -- 172.21.15.27:0/2585167345 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6fdc01d070 con 0x7f6fec100720 2024-04-02T04:34:20.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:20.800+0000 7f6ff0855700 1 -- 172.21.15.27:0/2585167345 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6fdc00be20 con 0x7f6fec100720 2024-04-02T04:34:20.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:20.800+0000 7f6ff1857700 1 -- 172.21.15.27:0/2585167345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fec196300 con 0x7f6fec100720 2024-04-02T04:34:20.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:20.800+0000 7f6ff0855700 1 -- 172.21.15.27:0/2585167345 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6fdc022de0 con 0x7f6fec100720 2024-04-02T04:34:20.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:20.800+0000 7f6ff1857700 1 -- 172.21.15.27:0/2585167345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fec1967a0 con 0x7f6fec100720 2024-04-02T04:34:20.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:20.801+0000 7f6ff0855700 1 -- 172.21.15.27:0/2585167345 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45604+0+0 (secure 0 0 0) 0x7f6fdc00b960 con 0x7f6fec100720 2024-04-02T04:34:20.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:20.801+0000 7f6ff1857700 1 -- 172.21.15.27:0/2585167345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fec18f550 con 0x7f6fec100720 2024-04-02T04:34:20.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:20.801+0000 7f6ff0855700 1 --2- 172.21.15.27:0/2585167345 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f6fd4038820 0x7f6fd403ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:34:20.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:20.801+0000 7f6ff0855700 1 -- 172.21.15.27:0/2585167345 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6fdc04cae0 con 0x7f6fec100720 2024-04-02T04:34:20.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:20.801+0000 7f6fea7fc700 1 --2- 172.21.15.27:0/2585167345 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f6fd4038820 0x7f6fd403ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:34:20.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:20.802+0000 7f6fea7fc700 1 --2- 172.21.15.27:0/2585167345 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f6fd4038820 0x7f6fd403ace0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0006fd0 tx=0x7f6fe0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:34:20.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:20.810+0000 7f6ff0855700 1 -- 172.21.15.27:0/2585167345 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6fdc02bba0 con 0x7f6fec100720 2024-04-02T04:34:21.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:21.096+0000 7f6ff1857700 1 -- 172.21.15.27:0/2585167345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6fec02ccb0 con 0x7f6fec100720 2024-04-02T04:34:21.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:21.097+0000 7f6ff0855700 1 -- 172.21.15.27:0/2585167345 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f6fdc027030 con 0x7f6fec100720 2024-04-02T04:34:21.100 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:34:21.100 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":1,"fsid":"dc5403b4-f0a9-11ee-b647-cb9ed24678a4","modified":"2024-04-02T04:32:20.006984Z","created":"2024-04-02T04:32:20.006984Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi016","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-02T04:34:21.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:21.098+0000 7f6ff1857700 1 -- 172.21.15.27:0/2585167345 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f6fd4038820 msgr2=0x7f6fd403ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:21.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:21.098+0000 7f6ff1857700 1 --2- 172.21.15.27:0/2585167345 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f6fd4038820 0x7f6fd403ace0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0006fd0 tx=0x7f6fe0006e40 comp rx=0 tx=0).stop 2024-04-02T04:34:21.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:21.099+0000 7f6ff1857700 1 -- 172.21.15.27:0/2585167345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fec100720 msgr2=0x7f6fec195bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:21.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:21.099+0000 7f6ff1857700 1 --2- 172.21.15.27:0/2585167345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fec100720 0x7f6fec195bc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc006010 tx=0x7f6fdc0050b0 comp rx=0 tx=0).stop 2024-04-02T04:34:21.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:21.099+0000 7f6ff1857700 1 -- 172.21.15.27:0/2585167345 shutdown_connections 2024-04-02T04:34:21.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:21.099+0000 7f6ff1857700 1 --2- 172.21.15.27:0/2585167345 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f6fd4038820 0x7f6fd403ace0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:21.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:21.099+0000 7f6ff1857700 1 --2- 172.21.15.27:0/2585167345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fec100720 0x7f6fec195bc0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:21.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:21.099+0000 7f6ff1857700 1 -- 172.21.15.27:0/2585167345 >> 172.21.15.27:0/2585167345 conn(0x7f6fec0fbcc0 msgr2=0x7f6fec0fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:34:21.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:21.099+0000 7f6ff1857700 1 -- 172.21.15.27:0/2585167345 shutdown_connections 2024-04-02T04:34:21.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:21.099+0000 7f6ff1857700 1 -- 172.21.15.27:0/2585167345 wait complete. 2024-04-02T04:34:21.104 INFO:teuthology.orchestra.run.smithi027.stderr:dumped monmap epoch 1 2024-04-02T04:34:22.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:21 smithi016 ceph-mon[28969]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:34:22.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:21 smithi016 ceph-mon[28969]: from='client.? 172.21.15.27:0/2585167345' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T04:34:22.731 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-02T04:34:22.732 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T04:34:22.944 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:34:22.991 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:34:24.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.393+0000 7f300d75d700 1 -- 172.21.15.27:0/3820718342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3008100720 msgr2=0x7f3008100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:24.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.393+0000 7f300d75d700 1 --2- 172.21.15.27:0/3820718342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3008100720 0x7f3008100b40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8009910 tx=0x7f2ff8009c20 comp rx=0 tx=0).stop 2024-04-02T04:34:24.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.393+0000 7f300d75d700 1 -- 172.21.15.27:0/3820718342 shutdown_connections 2024-04-02T04:34:24.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.393+0000 7f300d75d700 1 --2- 172.21.15.27:0/3820718342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3008100720 0x7f3008100b40 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:24.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.393+0000 7f300d75d700 1 -- 172.21.15.27:0/3820718342 >> 172.21.15.27:0/3820718342 conn(0x7f30080fbcc0 msgr2=0x7f30080fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:34:24.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.393+0000 7f300d75d700 1 -- 172.21.15.27:0/3820718342 shutdown_connections 2024-04-02T04:34:24.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.393+0000 7f300d75d700 1 -- 172.21.15.27:0/3820718342 wait complete. 2024-04-02T04:34:24.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.393+0000 7f300d75d700 1 Processor -- start 2024-04-02T04:34:24.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.394+0000 7f300d75d700 1 -- start start 2024-04-02T04:34:24.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.394+0000 7f300d75d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3008100720 0x7f3008195bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:34:24.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.394+0000 7f300d75d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3008196100 con 0x7f3008100720 2024-04-02T04:34:24.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.394+0000 7f3006ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3008100720 0x7f3008195bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:34:24.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.394+0000 7f3006ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3008100720 0x7f3008195bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.27:50750/0 (socket says 172.21.15.27:50750) 2024-04-02T04:34:24.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.394+0000 7f3006ffd700 1 -- 172.21.15.27:0/1297373706 learned_addr learned my addr 172.21.15.27:0/1297373706 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-02T04:34:24.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.394+0000 7f3006ffd700 1 -- 172.21.15.27:0/1297373706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff80095f0 con 0x7f3008100720 2024-04-02T04:34:24.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.394+0000 7f3006ffd700 1 --2- 172.21.15.27:0/1297373706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3008100720 0x7f3008195bc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8006010 tx=0x7f2ff8009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:34:24.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.395+0000 7f2feffff700 1 -- 172.21.15.27:0/1297373706 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2ff801d070 con 0x7f3008100720 2024-04-02T04:34:24.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.395+0000 7f300d75d700 1 -- 172.21.15.27:0/1297373706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3008196300 con 0x7f3008100720 2024-04-02T04:34:24.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.395+0000 7f2feffff700 1 -- 172.21.15.27:0/1297373706 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2ff8005dc0 con 0x7f3008100720 2024-04-02T04:34:24.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.395+0000 7f2feffff700 1 -- 172.21.15.27:0/1297373706 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2ff8022e90 con 0x7f3008100720 2024-04-02T04:34:24.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.395+0000 7f300d75d700 1 -- 172.21.15.27:0/1297373706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30081967a0 con 0x7f3008100720 2024-04-02T04:34:24.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.395+0000 7f2feffff700 1 -- 172.21.15.27:0/1297373706 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45604+0+0 (secure 0 0 0) 0x7f2ff800b960 con 0x7f3008100720 2024-04-02T04:34:24.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.395+0000 7f300d75d700 1 -- 172.21.15.27:0/1297373706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f300818f550 con 0x7f3008100720 2024-04-02T04:34:24.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.396+0000 7f2feffff700 1 --2- 172.21.15.27:0/1297373706 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f2ff0038760 0x7f2ff003ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:34:24.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.396+0000 7f2feffff700 1 -- 172.21.15.27:0/1297373706 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2ff804c880 con 0x7f3008100720 2024-04-02T04:34:24.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.396+0000 7f30067fc700 1 --2- 172.21.15.27:0/1297373706 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f2ff0038760 0x7f2ff003ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:34:24.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.396+0000 7f30067fc700 1 --2- 172.21.15.27:0/1297373706 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f2ff0038760 0x7f2ff003ac20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc006fd0 tx=0x7f2ffc006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:34:24.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.405+0000 7f2feffff700 1 -- 172.21.15.27:0/1297373706 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2ff8022750 con 0x7f3008100720 2024-04-02T04:34:24.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.690+0000 7f300d75d700 1 -- 172.21.15.27:0/1297373706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f30080623e0 con 0x7f3008100720 2024-04-02T04:34:24.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.691+0000 7f2feffff700 1 -- 172.21.15.27:0/1297373706 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f30080623e0 con 0x7f3008100720 2024-04-02T04:34:24.694 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:34:24.694 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":1,"fsid":"dc5403b4-f0a9-11ee-b647-cb9ed24678a4","modified":"2024-04-02T04:32:20.006984Z","created":"2024-04-02T04:32:20.006984Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi016","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-02T04:34:24.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.693+0000 7f300d75d700 1 -- 172.21.15.27:0/1297373706 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f2ff0038760 msgr2=0x7f2ff003ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:24.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.693+0000 7f300d75d700 1 --2- 172.21.15.27:0/1297373706 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f2ff0038760 0x7f2ff003ac20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc006fd0 tx=0x7f2ffc006e40 comp rx=0 tx=0).stop 2024-04-02T04:34:24.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.693+0000 7f300d75d700 1 -- 172.21.15.27:0/1297373706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3008100720 msgr2=0x7f3008195bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:24.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.693+0000 7f300d75d700 1 --2- 172.21.15.27:0/1297373706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3008100720 0x7f3008195bc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8006010 tx=0x7f2ff8009f80 comp rx=0 tx=0).stop 2024-04-02T04:34:24.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.693+0000 7f300d75d700 1 -- 172.21.15.27:0/1297373706 shutdown_connections 2024-04-02T04:34:24.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.693+0000 7f300d75d700 1 --2- 172.21.15.27:0/1297373706 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f2ff0038760 0x7f2ff003ac20 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:24.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.693+0000 7f300d75d700 1 --2- 172.21.15.27:0/1297373706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3008100720 0x7f3008195bc0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:24.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.693+0000 7f300d75d700 1 -- 172.21.15.27:0/1297373706 >> 172.21.15.27:0/1297373706 conn(0x7f30080fbcc0 msgr2=0x7f30080fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:34:24.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.693+0000 7f300d75d700 1 -- 172.21.15.27:0/1297373706 shutdown_connections 2024-04-02T04:34:24.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:24.693+0000 7f300d75d700 1 -- 172.21.15.27:0/1297373706 wait complete. 2024-04-02T04:34:24.697 INFO:teuthology.orchestra.run.smithi027.stderr:dumped monmap epoch 1 2024-04-02T04:34:25.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:24 smithi016 ceph-mon[28969]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:34:26.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:25 smithi016 ceph-mon[28969]: from='client.? 172.21.15.27:0/1297373706' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T04:34:26.240 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-02T04:34:26.241 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T04:34:26.452 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:34:26.500 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:34:27.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:26 smithi016 ceph-mon[28969]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:34:27.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:27.957+0000 7f2add758700 1 -- 172.21.15.27:0/193148384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad8100720 msgr2=0x7f2ad8100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:27.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:27.957+0000 7f2add758700 1 --2- 172.21.15.27:0/193148384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad8100720 0x7f2ad8100b40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2ad000b0a0 tx=0x7f2ad000b3b0 comp rx=0 tx=0).stop 2024-04-02T04:34:27.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:27.957+0000 7f2add758700 1 -- 172.21.15.27:0/193148384 shutdown_connections 2024-04-02T04:34:27.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:27.957+0000 7f2add758700 1 --2- 172.21.15.27:0/193148384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad8100720 0x7f2ad8100b40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:27.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:27.957+0000 7f2add758700 1 -- 172.21.15.27:0/193148384 >> 172.21.15.27:0/193148384 conn(0x7f2ad80fbcc0 msgr2=0x7f2ad80fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:34:27.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:27.957+0000 7f2add758700 1 -- 172.21.15.27:0/193148384 shutdown_connections 2024-04-02T04:34:27.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:27.957+0000 7f2add758700 1 -- 172.21.15.27:0/193148384 wait complete. 2024-04-02T04:34:27.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:27.957+0000 7f2add758700 1 Processor -- start 2024-04-02T04:34:27.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:27.958+0000 7f2add758700 1 -- start start 2024-04-02T04:34:27.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:27.958+0000 7f2add758700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad8100720 0x7f2ad8195bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:34:27.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:27.958+0000 7f2add758700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad8196100 con 0x7f2ad8100720 2024-04-02T04:34:27.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:27.958+0000 7f2ad6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad8100720 0x7f2ad8195bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:34:27.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:27.958+0000 7f2ad6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad8100720 0x7f2ad8195bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.27:50766/0 (socket says 172.21.15.27:50766) 2024-04-02T04:34:27.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:27.958+0000 7f2ad6ffd700 1 -- 172.21.15.27:0/885964485 learned_addr learned my addr 172.21.15.27:0/885964485 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-02T04:34:27.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:27.958+0000 7f2ad6ffd700 1 -- 172.21.15.27:0/885964485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad0009d00 con 0x7f2ad8100720 2024-04-02T04:34:27.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:27.958+0000 7f2ad6ffd700 1 --2- 172.21.15.27:0/885964485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad8100720 0x7f2ad8195bc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2ad00062a0 tx=0x7f2ad000bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:34:27.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:27.959+0000 7f2acffff700 1 -- 172.21.15.27:0/885964485 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2ad000e040 con 0x7f2ad8100720 2024-04-02T04:34:27.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:27.959+0000 7f2add758700 1 -- 172.21.15.27:0/885964485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ad8196300 con 0x7f2ad8100720 2024-04-02T04:34:27.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:27.959+0000 7f2acffff700 1 -- 172.21.15.27:0/885964485 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2ad0003d70 con 0x7f2ad8100720 2024-04-02T04:34:27.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:27.959+0000 7f2acffff700 1 -- 172.21.15.27:0/885964485 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2ad001c890 con 0x7f2ad8100720 2024-04-02T04:34:27.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:27.959+0000 7f2add758700 1 -- 172.21.15.27:0/885964485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ad81967a0 con 0x7f2ad8100720 2024-04-02T04:34:27.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:27.959+0000 7f2add758700 1 -- 172.21.15.27:0/885964485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ad818f550 con 0x7f2ad8100720 2024-04-02T04:34:27.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:27.960+0000 7f2acffff700 1 -- 172.21.15.27:0/885964485 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45604+0+0 (secure 0 0 0) 0x7f2ad001cb50 con 0x7f2ad8100720 2024-04-02T04:34:27.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:27.960+0000 7f2acffff700 1 --2- 172.21.15.27:0/885964485 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f2ac40387c0 0x7f2ac403ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:34:27.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:27.960+0000 7f2acffff700 1 -- 172.21.15.27:0/885964485 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2ad004c9a0 con 0x7f2ad8100720 2024-04-02T04:34:27.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:27.960+0000 7f2ad67fc700 1 --2- 172.21.15.27:0/885964485 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f2ac40387c0 0x7f2ac403ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:34:27.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:27.960+0000 7f2ad67fc700 1 --2- 172.21.15.27:0/885964485 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f2ac40387c0 0x7f2ac403ac80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0006fd0 tx=0x7f2ac0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:34:27.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:27.965+0000 7f2acffff700 1 -- 172.21.15.27:0/885964485 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2ad001a3b0 con 0x7f2ad8100720 2024-04-02T04:34:28.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:28.253+0000 7f2add758700 1 -- 172.21.15.27:0/885964485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2ad80623e0 con 0x7f2ad8100720 2024-04-02T04:34:28.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:28.254+0000 7f2acffff700 1 -- 172.21.15.27:0/885964485 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f2ad001a3b0 con 0x7f2ad8100720 2024-04-02T04:34:28.257 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:34:28.257 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":1,"fsid":"dc5403b4-f0a9-11ee-b647-cb9ed24678a4","modified":"2024-04-02T04:32:20.006984Z","created":"2024-04-02T04:32:20.006984Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi016","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-02T04:34:28.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:28.256+0000 7f2add758700 1 -- 172.21.15.27:0/885964485 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f2ac40387c0 msgr2=0x7f2ac403ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:28.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:28.256+0000 7f2add758700 1 --2- 172.21.15.27:0/885964485 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f2ac40387c0 0x7f2ac403ac80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0006fd0 tx=0x7f2ac0006e40 comp rx=0 tx=0).stop 2024-04-02T04:34:28.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:28.256+0000 7f2add758700 1 -- 172.21.15.27:0/885964485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad8100720 msgr2=0x7f2ad8195bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:28.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:28.256+0000 7f2add758700 1 --2- 172.21.15.27:0/885964485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad8100720 0x7f2ad8195bc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2ad00062a0 tx=0x7f2ad000bd40 comp rx=0 tx=0).stop 2024-04-02T04:34:28.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:28.256+0000 7f2add758700 1 -- 172.21.15.27:0/885964485 shutdown_connections 2024-04-02T04:34:28.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:28.256+0000 7f2add758700 1 --2- 172.21.15.27:0/885964485 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f2ac40387c0 0x7f2ac403ac80 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:28.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:28.256+0000 7f2add758700 1 --2- 172.21.15.27:0/885964485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad8100720 0x7f2ad8195bc0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:28.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:28.256+0000 7f2add758700 1 -- 172.21.15.27:0/885964485 >> 172.21.15.27:0/885964485 conn(0x7f2ad80fbcc0 msgr2=0x7f2ad80fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:34:28.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:28.256+0000 7f2add758700 1 -- 172.21.15.27:0/885964485 shutdown_connections 2024-04-02T04:34:28.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:28.256+0000 7f2add758700 1 -- 172.21.15.27:0/885964485 wait complete. 2024-04-02T04:34:28.265 INFO:teuthology.orchestra.run.smithi027.stderr:dumped monmap epoch 1 2024-04-02T04:34:29.019 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:28 smithi016 ceph-mon[28969]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:34:29.019 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:28 smithi016 ceph-mon[28969]: from='client.? 172.21.15.27:0/885964485' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T04:34:29.831 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-02T04:34:29.831 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T04:34:30.043 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:34:30.091 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:34:31.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:30 smithi016 ceph-mon[28969]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:34:31.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.583+0000 7fbdcc673700 1 -- 172.21.15.27:0/1670899833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdc40ff630 msgr2=0x7fbdc40ffa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:31.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.583+0000 7fbdcc673700 1 --2- 172.21.15.27:0/1670899833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdc40ff630 0x7fbdc40ffa50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0009910 tx=0x7fbdc0009c20 comp rx=0 tx=0).stop 2024-04-02T04:34:31.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.583+0000 7fbdcc673700 1 -- 172.21.15.27:0/1670899833 shutdown_connections 2024-04-02T04:34:31.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.583+0000 7fbdcc673700 1 --2- 172.21.15.27:0/1670899833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdc40ff630 0x7fbdc40ffa50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:31.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.583+0000 7fbdcc673700 1 -- 172.21.15.27:0/1670899833 >> 172.21.15.27:0/1670899833 conn(0x7fbdc40fabd0 msgr2=0x7fbdc40fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:34:31.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.583+0000 7fbdcc673700 1 -- 172.21.15.27:0/1670899833 shutdown_connections 2024-04-02T04:34:31.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.583+0000 7fbdcc673700 1 -- 172.21.15.27:0/1670899833 wait complete. 2024-04-02T04:34:31.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.583+0000 7fbdcc673700 1 Processor -- start 2024-04-02T04:34:31.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.584+0000 7fbdcc673700 1 -- start start 2024-04-02T04:34:31.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.584+0000 7fbdcc673700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdc40ff630 0x7fbdc4195bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:34:31.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.584+0000 7fbdcc673700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc4196130 con 0x7fbdc40ff630 2024-04-02T04:34:31.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.584+0000 7fbdca40f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdc40ff630 0x7fbdc4195bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:34:31.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.584+0000 7fbdca40f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdc40ff630 0x7fbdc4195bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.27:50786/0 (socket says 172.21.15.27:50786) 2024-04-02T04:34:31.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.584+0000 7fbdca40f700 1 -- 172.21.15.27:0/2916140361 learned_addr learned my addr 172.21.15.27:0/2916140361 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-02T04:34:31.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.584+0000 7fbdca40f700 1 -- 172.21.15.27:0/2916140361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc00095f0 con 0x7fbdc40ff630 2024-04-02T04:34:31.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.585+0000 7fbdca40f700 1 --2- 172.21.15.27:0/2916140361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdc40ff630 0x7fbdc4195bf0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0006010 tx=0x7fbdc0009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:34:31.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.585+0000 7fbdbb7fe700 1 -- 172.21.15.27:0/2916140361 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbdc001d070 con 0x7fbdc40ff630 2024-04-02T04:34:31.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.585+0000 7fbdcc673700 1 -- 172.21.15.27:0/2916140361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdc4196330 con 0x7fbdc40ff630 2024-04-02T04:34:31.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.585+0000 7fbdbb7fe700 1 -- 172.21.15.27:0/2916140361 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbdc0005dc0 con 0x7fbdc40ff630 2024-04-02T04:34:31.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.585+0000 7fbdbb7fe700 1 -- 172.21.15.27:0/2916140361 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbdc0022e90 con 0x7fbdc40ff630 2024-04-02T04:34:31.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.585+0000 7fbdcc673700 1 -- 172.21.15.27:0/2916140361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc41967d0 con 0x7fbdc40ff630 2024-04-02T04:34:31.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.586+0000 7fbdcc673700 1 -- 172.21.15.27:0/2916140361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdc418f720 con 0x7fbdc40ff630 2024-04-02T04:34:31.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.586+0000 7fbdbb7fe700 1 -- 172.21.15.27:0/2916140361 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45604+0+0 (secure 0 0 0) 0x7fbdc0022470 con 0x7fbdc40ff630 2024-04-02T04:34:31.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.586+0000 7fbdbb7fe700 1 --2- 172.21.15.27:0/2916140361 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fbdb0038760 0x7fbdb003ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:34:31.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.586+0000 7fbdbb7fe700 1 -- 172.21.15.27:0/2916140361 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbdc004c8b0 con 0x7fbdc40ff630 2024-04-02T04:34:31.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.586+0000 7fbdc9c0e700 1 --2- 172.21.15.27:0/2916140361 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fbdb0038760 0x7fbdb003ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:34:31.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.587+0000 7fbdc9c0e700 1 --2- 172.21.15.27:0/2916140361 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fbdb0038760 0x7fbdb003ac20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4006fd0 tx=0x7fbdb4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:34:31.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.591+0000 7fbdbb7fe700 1 -- 172.21.15.27:0/2916140361 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbdc000b960 con 0x7fbdc40ff630 2024-04-02T04:34:31.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.875+0000 7fbdcc673700 1 -- 172.21.15.27:0/2916140361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fbdc40623e0 con 0x7fbdc40ff630 2024-04-02T04:34:31.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.876+0000 7fbdbb7fe700 1 -- 172.21.15.27:0/2916140361 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fbdc40623e0 con 0x7fbdc40ff630 2024-04-02T04:34:31.879 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:34:31.879 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":1,"fsid":"dc5403b4-f0a9-11ee-b647-cb9ed24678a4","modified":"2024-04-02T04:32:20.006984Z","created":"2024-04-02T04:32:20.006984Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi016","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-02T04:34:31.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.878+0000 7fbdcc673700 1 -- 172.21.15.27:0/2916140361 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fbdb0038760 msgr2=0x7fbdb003ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:31.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.878+0000 7fbdcc673700 1 --2- 172.21.15.27:0/2916140361 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fbdb0038760 0x7fbdb003ac20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4006fd0 tx=0x7fbdb4006e40 comp rx=0 tx=0).stop 2024-04-02T04:34:31.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.878+0000 7fbdcc673700 1 -- 172.21.15.27:0/2916140361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdc40ff630 msgr2=0x7fbdc4195bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:31.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.878+0000 7fbdcc673700 1 --2- 172.21.15.27:0/2916140361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdc40ff630 0x7fbdc4195bf0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0006010 tx=0x7fbdc0009f80 comp rx=0 tx=0).stop 2024-04-02T04:34:31.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.878+0000 7fbdcc673700 1 -- 172.21.15.27:0/2916140361 shutdown_connections 2024-04-02T04:34:31.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.878+0000 7fbdcc673700 1 --2- 172.21.15.27:0/2916140361 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7fbdb0038760 0x7fbdb003ac20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:31.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.878+0000 7fbdcc673700 1 --2- 172.21.15.27:0/2916140361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdc40ff630 0x7fbdc4195bf0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:31.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.878+0000 7fbdcc673700 1 -- 172.21.15.27:0/2916140361 >> 172.21.15.27:0/2916140361 conn(0x7fbdc40fabd0 msgr2=0x7fbdc4068f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:34:31.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.878+0000 7fbdcc673700 1 -- 172.21.15.27:0/2916140361 shutdown_connections 2024-04-02T04:34:31.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:31.878+0000 7fbdcc673700 1 -- 172.21.15.27:0/2916140361 wait complete. 2024-04-02T04:34:31.882 INFO:teuthology.orchestra.run.smithi027.stderr:dumped monmap epoch 1 2024-04-02T04:34:32.858 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:32 smithi016 ceph-mon[28969]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:34:32.858 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:32 smithi016 ceph-mon[28969]: from='client.? 172.21.15.27:0/2916140361' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T04:34:33.466 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-02T04:34:33.466 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T04:34:33.678 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:34:33.725 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:34:35.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:34 smithi016 ceph-mon[28969]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:34:35.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.178+0000 7f514d34b700 1 -- 172.21.15.27:0/451723794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5148100750 msgr2=0x7f5148100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:35.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.178+0000 7f514d34b700 1 --2- 172.21.15.27:0/451723794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5148100750 0x7f5148100b70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5138009910 tx=0x7f5138009c20 comp rx=0 tx=0).stop 2024-04-02T04:34:35.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.178+0000 7f514d34b700 1 -- 172.21.15.27:0/451723794 shutdown_connections 2024-04-02T04:34:35.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.178+0000 7f514d34b700 1 --2- 172.21.15.27:0/451723794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5148100750 0x7f5148100b70 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:35.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.178+0000 7f514d34b700 1 -- 172.21.15.27:0/451723794 >> 172.21.15.27:0/451723794 conn(0x7f51480fbcf0 msgr2=0x7f51480fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:34:35.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.178+0000 7f514d34b700 1 -- 172.21.15.27:0/451723794 shutdown_connections 2024-04-02T04:34:35.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.178+0000 7f514d34b700 1 -- 172.21.15.27:0/451723794 wait complete. 2024-04-02T04:34:35.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.179+0000 7f514d34b700 1 Processor -- start 2024-04-02T04:34:35.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.179+0000 7f514d34b700 1 -- start start 2024-04-02T04:34:35.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.179+0000 7f514d34b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5148100750 0x7f5148195c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:34:35.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.180+0000 7f514d34b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5148196140 con 0x7f5148100750 2024-04-02T04:34:35.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.180+0000 7f5146ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5148100750 0x7f5148195c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:34:35.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.180+0000 7f5146ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5148100750 0x7f5148195c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.27:42648/0 (socket says 172.21.15.27:42648) 2024-04-02T04:34:35.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.180+0000 7f5146ffd700 1 -- 172.21.15.27:0/3477410750 learned_addr learned my addr 172.21.15.27:0/3477410750 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-02T04:34:35.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.180+0000 7f5146ffd700 1 -- 172.21.15.27:0/3477410750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51380095f0 con 0x7f5148100750 2024-04-02T04:34:35.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.180+0000 7f5146ffd700 1 --2- 172.21.15.27:0/3477410750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5148100750 0x7f5148195c00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5138006010 tx=0x7f5138009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:34:35.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.180+0000 7f5144ff9700 1 -- 172.21.15.27:0/3477410750 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f513801d070 con 0x7f5148100750 2024-04-02T04:34:35.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.180+0000 7f5144ff9700 1 -- 172.21.15.27:0/3477410750 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5138005dc0 con 0x7f5148100750 2024-04-02T04:34:35.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.180+0000 7f514d34b700 1 -- 172.21.15.27:0/3477410750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5148196340 con 0x7f5148100750 2024-04-02T04:34:35.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.180+0000 7f5144ff9700 1 -- 172.21.15.27:0/3477410750 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5138022e90 con 0x7f5148100750 2024-04-02T04:34:35.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.180+0000 7f514d34b700 1 -- 172.21.15.27:0/3477410750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51481967e0 con 0x7f5148100750 2024-04-02T04:34:35.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.181+0000 7f5144ff9700 1 -- 172.21.15.27:0/3477410750 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45604+0+0 (secure 0 0 0) 0x7f513800b960 con 0x7f5148100750 2024-04-02T04:34:35.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.181+0000 7f514d34b700 1 -- 172.21.15.27:0/3477410750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f514818f580 con 0x7f5148100750 2024-04-02T04:34:35.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.182+0000 7f5144ff9700 1 --2- 172.21.15.27:0/3477410750 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f5134038760 0x7f513403ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:34:35.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.182+0000 7f5144ff9700 1 -- 172.21.15.27:0/3477410750 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f513804c8c0 con 0x7f5148100750 2024-04-02T04:34:35.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.182+0000 7f513e7fc700 1 --2- 172.21.15.27:0/3477410750 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f5134038760 0x7f513403ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:34:35.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.182+0000 7f513e7fc700 1 --2- 172.21.15.27:0/3477410750 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f5134038760 0x7f513403ac20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5130006fd0 tx=0x7f5130006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:34:35.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.187+0000 7f5144ff9700 1 -- 172.21.15.27:0/3477410750 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5138022750 con 0x7f5148100750 2024-04-02T04:34:35.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.470+0000 7f514d34b700 1 -- 172.21.15.27:0/3477410750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f51480623e0 con 0x7f5148100750 2024-04-02T04:34:35.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.471+0000 7f5144ff9700 1 -- 172.21.15.27:0/3477410750 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f51480623e0 con 0x7f5148100750 2024-04-02T04:34:35.474 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:34:35.474 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":1,"fsid":"dc5403b4-f0a9-11ee-b647-cb9ed24678a4","modified":"2024-04-02T04:32:20.006984Z","created":"2024-04-02T04:32:20.006984Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi016","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-02T04:34:35.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.473+0000 7f514d34b700 1 -- 172.21.15.27:0/3477410750 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f5134038760 msgr2=0x7f513403ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:35.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.473+0000 7f514d34b700 1 --2- 172.21.15.27:0/3477410750 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f5134038760 0x7f513403ac20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5130006fd0 tx=0x7f5130006e40 comp rx=0 tx=0).stop 2024-04-02T04:34:35.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.473+0000 7f514d34b700 1 -- 172.21.15.27:0/3477410750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5148100750 msgr2=0x7f5148195c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:35.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.473+0000 7f514d34b700 1 --2- 172.21.15.27:0/3477410750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5148100750 0x7f5148195c00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5138006010 tx=0x7f5138009f80 comp rx=0 tx=0).stop 2024-04-02T04:34:35.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.473+0000 7f514d34b700 1 -- 172.21.15.27:0/3477410750 shutdown_connections 2024-04-02T04:34:35.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.473+0000 7f514d34b700 1 --2- 172.21.15.27:0/3477410750 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f5134038760 0x7f513403ac20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:35.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.473+0000 7f514d34b700 1 --2- 172.21.15.27:0/3477410750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5148100750 0x7f5148195c00 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:35.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.473+0000 7f514d34b700 1 -- 172.21.15.27:0/3477410750 >> 172.21.15.27:0/3477410750 conn(0x7f51480fbcf0 msgr2=0x7f51480fc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:34:35.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.473+0000 7f514d34b700 1 -- 172.21.15.27:0/3477410750 shutdown_connections 2024-04-02T04:34:35.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:35.473+0000 7f514d34b700 1 -- 172.21.15.27:0/3477410750 wait complete. 2024-04-02T04:34:35.477 INFO:teuthology.orchestra.run.smithi027.stderr:dumped monmap epoch 1 2024-04-02T04:34:36.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:35 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:36.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:35 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:36.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:35 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:36.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:35 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-04-02T04:34:36.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:35 smithi016 ceph-mon[28969]: from='client.? 172.21.15.27:0/3477410750' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T04:34:37.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:36 smithi016 ceph-mon[28969]: from='mgr.14162 172.21.15.16:0/2931115903' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-04-02T04:34:37.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:36 smithi016 ceph-mon[28969]: mgrmap e14: smithi016.eqnebx(active, since 66s) 2024-04-02T04:34:37.130 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-02T04:34:37.130 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T04:34:37.343 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:34:37.391 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:34:38.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:38.862+0000 7f257cdad700 1 -- 172.21.15.27:0/1275215132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25780ff640 msgr2=0x7f25780ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:38.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:38.862+0000 7f257cdad700 1 --2- 172.21.15.27:0/1275215132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25780ff640 0x7f25780ffa60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2568009910 tx=0x7f2568009c20 comp rx=0 tx=0).stop 2024-04-02T04:34:38.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:38.862+0000 7f257cdad700 1 -- 172.21.15.27:0/1275215132 shutdown_connections 2024-04-02T04:34:38.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:38.862+0000 7f257cdad700 1 --2- 172.21.15.27:0/1275215132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25780ff640 0x7f25780ffa60 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:38.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:38.862+0000 7f257cdad700 1 -- 172.21.15.27:0/1275215132 >> 172.21.15.27:0/1275215132 conn(0x7f25780fac00 msgr2=0x7f25780fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:34:38.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:38.862+0000 7f257cdad700 1 -- 172.21.15.27:0/1275215132 shutdown_connections 2024-04-02T04:34:38.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:38.862+0000 7f257cdad700 1 -- 172.21.15.27:0/1275215132 wait complete. 2024-04-02T04:34:38.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:38.863+0000 7f257cdad700 1 Processor -- start 2024-04-02T04:34:38.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:38.863+0000 7f257cdad700 1 -- start start 2024-04-02T04:34:38.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:38.863+0000 7f257cdad700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25780ff640 0x7f25780fca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:34:38.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:38.863+0000 7f257cdad700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25780fcf60 con 0x7f25780ff640 2024-04-02T04:34:38.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:38.863+0000 7f257659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25780ff640 0x7f25780fca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:34:38.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:38.863+0000 7f257659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25780ff640 0x7f25780fca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.27:42666/0 (socket says 172.21.15.27:42666) 2024-04-02T04:34:38.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:38.863+0000 7f257659c700 1 -- 172.21.15.27:0/2467909456 learned_addr learned my addr 172.21.15.27:0/2467909456 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-02T04:34:38.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:38.863+0000 7f257659c700 1 -- 172.21.15.27:0/2467909456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25680095f0 con 0x7f25780ff640 2024-04-02T04:34:38.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:38.863+0000 7f257659c700 1 --2- 172.21.15.27:0/2467909456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25780ff640 0x7f25780fca20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2568006010 tx=0x7f2568009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:34:38.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:38.864+0000 7f256f7fe700 1 -- 172.21.15.27:0/2467909456 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f256801d070 con 0x7f25780ff640 2024-04-02T04:34:38.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:38.864+0000 7f256f7fe700 1 -- 172.21.15.27:0/2467909456 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2568005dc0 con 0x7f25780ff640 2024-04-02T04:34:38.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:38.864+0000 7f257cdad700 1 -- 172.21.15.27:0/2467909456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25780fb1a0 con 0x7f25780ff640 2024-04-02T04:34:38.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:38.864+0000 7f256f7fe700 1 -- 172.21.15.27:0/2467909456 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2568022e90 con 0x7f25780ff640 2024-04-02T04:34:38.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:38.864+0000 7f257cdad700 1 -- 172.21.15.27:0/2467909456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25780fb640 con 0x7f25780ff640 2024-04-02T04:34:38.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:38.864+0000 7f256f7fe700 1 -- 172.21.15.27:0/2467909456 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 14) v1 ==== 45618+0+0 (secure 0 0 0) 0x7f256800b960 con 0x7f25780ff640 2024-04-02T04:34:38.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:38.864+0000 7f257cdad700 1 -- 172.21.15.27:0/2467909456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2578105da0 con 0x7f25780ff640 2024-04-02T04:34:38.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:38.865+0000 7f256f7fe700 1 --2- 172.21.15.27:0/2467909456 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f25640387b0 0x7f256403ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:34:38.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:38.865+0000 7f256f7fe700 1 -- 172.21.15.27:0/2467909456 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f256804c8f0 con 0x7f25780ff640 2024-04-02T04:34:38.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:38.865+0000 7f2575d9b700 1 -- 172.21.15.27:0/2467909456 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f25640387b0 msgr2=0x7f256403ac70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/3912038174 2024-04-02T04:34:38.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:38.865+0000 7f2575d9b700 1 --2- 172.21.15.27:0/2467909456 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f25640387b0 0x7f256403ac70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T04:34:38.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:38.874+0000 7f256f7fe700 1 -- 172.21.15.27:0/2467909456 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f256802a540 con 0x7f25780ff640 2024-04-02T04:34:39.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:39.066+0000 7f2575d9b700 1 -- 172.21.15.27:0/2467909456 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f25640387b0 msgr2=0x7f256403ac70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/3912038174 2024-04-02T04:34:39.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:39.066+0000 7f2575d9b700 1 --2- 172.21.15.27:0/2467909456 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f25640387b0 0x7f256403ac70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T04:34:39.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:39.160+0000 7f257cdad700 1 -- 172.21.15.27:0/2467909456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f25780623e0 con 0x7f25780ff640 2024-04-02T04:34:39.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:39.161+0000 7f256f7fe700 1 -- 172.21.15.27:0/2467909456 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f2568027020 con 0x7f25780ff640 2024-04-02T04:34:39.164 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:34:39.164 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":1,"fsid":"dc5403b4-f0a9-11ee-b647-cb9ed24678a4","modified":"2024-04-02T04:32:20.006984Z","created":"2024-04-02T04:32:20.006984Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi016","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-02T04:34:39.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:39.163+0000 7f257cdad700 1 -- 172.21.15.27:0/2467909456 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f25640387b0 msgr2=0x7f256403ac70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T04:34:39.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:39.163+0000 7f257cdad700 1 --2- 172.21.15.27:0/2467909456 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f25640387b0 0x7f256403ac70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:39.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:39.163+0000 7f257cdad700 1 -- 172.21.15.27:0/2467909456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25780ff640 msgr2=0x7f25780fca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:39.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:39.163+0000 7f257cdad700 1 --2- 172.21.15.27:0/2467909456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25780ff640 0x7f25780fca20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2568006010 tx=0x7f2568009f80 comp rx=0 tx=0).stop 2024-04-02T04:34:39.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:39.163+0000 7f257cdad700 1 -- 172.21.15.27:0/2467909456 shutdown_connections 2024-04-02T04:34:39.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:39.163+0000 7f257cdad700 1 --2- 172.21.15.27:0/2467909456 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f25640387b0 0x7f256403ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:39.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:39.163+0000 7f257cdad700 1 --2- 172.21.15.27:0/2467909456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25780ff640 0x7f25780fca20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:39.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:39.163+0000 7f257cdad700 1 -- 172.21.15.27:0/2467909456 >> 172.21.15.27:0/2467909456 conn(0x7f25780fac00 msgr2=0x7f2578068fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:34:39.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:39.163+0000 7f257cdad700 1 -- 172.21.15.27:0/2467909456 shutdown_connections 2024-04-02T04:34:39.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:39.163+0000 7f257cdad700 1 -- 172.21.15.27:0/2467909456 wait complete. 2024-04-02T04:34:39.167 INFO:teuthology.orchestra.run.smithi027.stderr:dumped monmap epoch 1 2024-04-02T04:34:39.982 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:39 smithi016 ceph-mon[28969]: from='client.? 172.21.15.27:0/2467909456' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T04:34:40.799 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-02T04:34:40.799 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T04:34:41.004 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:34:41.050 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:34:42.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.533+0000 7f74b9b53700 1 -- 172.21.15.27:0/1469812922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b40ff630 msgr2=0x7f74b40ffa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:42.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.533+0000 7f74b9b53700 1 --2- 172.21.15.27:0/1469812922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b40ff630 0x7f74b40ffa50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f74a4009910 tx=0x7f74a4009c20 comp rx=0 tx=0).stop 2024-04-02T04:34:42.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.533+0000 7f74b9b53700 1 -- 172.21.15.27:0/1469812922 shutdown_connections 2024-04-02T04:34:42.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.533+0000 7f74b9b53700 1 --2- 172.21.15.27:0/1469812922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b40ff630 0x7f74b40ffa50 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:42.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.534+0000 7f74b9b53700 1 -- 172.21.15.27:0/1469812922 >> 172.21.15.27:0/1469812922 conn(0x7f74b40fabd0 msgr2=0x7f74b40fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:34:42.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.534+0000 7f74b9b53700 1 -- 172.21.15.27:0/1469812922 shutdown_connections 2024-04-02T04:34:42.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.534+0000 7f74b9b53700 1 -- 172.21.15.27:0/1469812922 wait complete. 2024-04-02T04:34:42.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.534+0000 7f74b9b53700 1 Processor -- start 2024-04-02T04:34:42.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.534+0000 7f74b9b53700 1 -- start start 2024-04-02T04:34:42.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.535+0000 7f74b9b53700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b40ff630 0x7f74b418c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:34:42.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.535+0000 7f74b9b53700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b418cc30 con 0x7f74b40ff630 2024-04-02T04:34:42.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.535+0000 7f74b37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b40ff630 0x7f74b418c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:34:42.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.535+0000 7f74b37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b40ff630 0x7f74b418c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.27:43376/0 (socket says 172.21.15.27:43376) 2024-04-02T04:34:42.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.535+0000 7f74b37fe700 1 -- 172.21.15.27:0/2014726253 learned_addr learned my addr 172.21.15.27:0/2014726253 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-02T04:34:42.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.535+0000 7f74b37fe700 1 -- 172.21.15.27:0/2014726253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a40095f0 con 0x7f74b40ff630 2024-04-02T04:34:42.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.535+0000 7f74b37fe700 1 --2- 172.21.15.27:0/2014726253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b40ff630 0x7f74b418c6f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f74a4004f40 tx=0x7f74a4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:34:42.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.535+0000 7f74b0ff9700 1 -- 172.21.15.27:0/2014726253 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f74a401d070 con 0x7f74b40ff630 2024-04-02T04:34:42.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.535+0000 7f74b0ff9700 1 -- 172.21.15.27:0/2014726253 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f74a4005dc0 con 0x7f74b40ff630 2024-04-02T04:34:42.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.535+0000 7f74b9b53700 1 -- 172.21.15.27:0/2014726253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74b418ae70 con 0x7f74b40ff630 2024-04-02T04:34:42.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.535+0000 7f74b0ff9700 1 -- 172.21.15.27:0/2014726253 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f74a4022e40 con 0x7f74b40ff630 2024-04-02T04:34:42.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.535+0000 7f74b9b53700 1 -- 172.21.15.27:0/2014726253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74b418b310 con 0x7f74b40ff630 2024-04-02T04:34:42.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.536+0000 7f74b0ff9700 1 -- 172.21.15.27:0/2014726253 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 14) v1 ==== 45618+0+0 (secure 0 0 0) 0x7f74a40226c0 con 0x7f74b40ff630 2024-04-02T04:34:42.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.536+0000 7f74b9b53700 1 -- 172.21.15.27:0/2014726253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74b40623e0 con 0x7f74b40ff630 2024-04-02T04:34:42.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.536+0000 7f74b0ff9700 1 --2- 172.21.15.27:0/2014726253 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f749c038810 0x7f749c03acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:34:42.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.537+0000 7f74b0ff9700 1 -- 172.21.15.27:0/2014726253 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f74a404c860 con 0x7f74b40ff630 2024-04-02T04:34:42.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.537+0000 7f74b2ffd700 1 -- 172.21.15.27:0/2014726253 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f749c038810 msgr2=0x7f749c03acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/3912038174 2024-04-02T04:34:42.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.537+0000 7f74b2ffd700 1 --2- 172.21.15.27:0/2014726253 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f749c038810 0x7f749c03acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T04:34:42.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.545+0000 7f74b0ff9700 1 -- 172.21.15.27:0/2014726253 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f74a402a540 con 0x7f74b40ff630 2024-04-02T04:34:42.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.714+0000 7f74b0ff9700 1 -- 172.21.15.27:0/2014726253 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 15) v1 ==== 45342+0+0 (secure 0 0 0) 0x7f74a400f460 con 0x7f74b40ff630 2024-04-02T04:34:42.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.714+0000 7f74b0ff9700 1 -- 172.21.15.27:0/2014726253 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f749c038810 msgr2=0x7f749c03acd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T04:34:42.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.714+0000 7f74b0ff9700 1 --2- 172.21.15.27:0/2014726253 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f749c038810 0x7f749c03acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:42.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.829+0000 7f74b9b53700 1 -- 172.21.15.27:0/2014726253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f74b4105470 con 0x7f74b40ff630 2024-04-02T04:34:42.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.830+0000 7f74b0ff9700 1 -- 172.21.15.27:0/2014726253 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f74a4027020 con 0x7f74b40ff630 2024-04-02T04:34:42.834 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:34:42.834 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":1,"fsid":"dc5403b4-f0a9-11ee-b647-cb9ed24678a4","modified":"2024-04-02T04:32:20.006984Z","created":"2024-04-02T04:32:20.006984Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi016","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-02T04:34:42.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.832+0000 7f74b9b53700 1 -- 172.21.15.27:0/2014726253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b40ff630 msgr2=0x7f74b418c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:42.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.832+0000 7f74b9b53700 1 --2- 172.21.15.27:0/2014726253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b40ff630 0x7f74b418c6f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f74a4004f40 tx=0x7f74a4004740 comp rx=0 tx=0).stop 2024-04-02T04:34:42.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.832+0000 7f74b9b53700 1 -- 172.21.15.27:0/2014726253 shutdown_connections 2024-04-02T04:34:42.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.832+0000 7f74b9b53700 1 --2- 172.21.15.27:0/2014726253 >> [v2:172.21.15.16:6800/3912038174,v1:172.21.15.16:6801/3912038174] conn(0x7f749c038810 0x7f749c03acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:42.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.832+0000 7f74b9b53700 1 --2- 172.21.15.27:0/2014726253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b40ff630 0x7f74b418c6f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:42.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.832+0000 7f74b9b53700 1 -- 172.21.15.27:0/2014726253 >> 172.21.15.27:0/2014726253 conn(0x7f74b40fabd0 msgr2=0x7f74b40fb8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:34:42.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.832+0000 7f74b9b53700 1 -- 172.21.15.27:0/2014726253 shutdown_connections 2024-04-02T04:34:42.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:42.832+0000 7f74b9b53700 1 -- 172.21.15.27:0/2014726253 wait complete. 2024-04-02T04:34:42.836 INFO:teuthology.orchestra.run.smithi027.stderr:dumped monmap epoch 1 2024-04-02T04:34:43.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:42 smithi016 ceph-mon[28969]: Active manager daemon smithi016.eqnebx restarted 2024-04-02T04:34:43.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:42 smithi016 ceph-mon[28969]: Activating manager daemon smithi016.eqnebx 2024-04-02T04:34:43.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:42 smithi016 ceph-mon[28969]: osdmap e5: 0 total, 0 up, 0 in 2024-04-02T04:34:43.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:42 smithi016 ceph-mon[28969]: mgrmap e15: smithi016.eqnebx(active, starting, since 0.0032103s) 2024-04-02T04:34:43.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:42 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi016"}]: dispatch 2024-04-02T04:34:43.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:42 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mgr metadata", "who": "smithi016.eqnebx", "id": "smithi016.eqnebx"}]: dispatch 2024-04-02T04:34:43.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:42 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T04:34:43.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:42 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T04:34:43.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:42 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T04:34:43.062 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:42 smithi016 ceph-mon[28969]: Manager daemon smithi016.eqnebx is now available 2024-04-02T04:34:43.062 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:42 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:34:43.062 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:42 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:43.062 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:42 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:34:43.968 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:43 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi016.eqnebx/mirror_snapshot_schedule"}]: dispatch 2024-04-02T04:34:43.969 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:43 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi016.eqnebx/trash_purge_schedule"}]: dispatch 2024-04-02T04:34:43.969 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:43 smithi016 ceph-mon[28969]: from='client.? 172.21.15.27:0/2014726253' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T04:34:43.969 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:43 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:43.969 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:43 smithi016 ceph-mon[28969]: mgrmap e16: smithi016.eqnebx(active, since 1.00568s) 2024-04-02T04:34:44.459 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-02T04:34:44.459 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T04:34:44.664 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:34:44.712 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:34:45.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:44 smithi016 ceph-mon[28969]: [02/Apr/2024:04:34:43] ENGINE Bus STARTING 2024-04-02T04:34:45.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:44 smithi016 ceph-mon[28969]: [02/Apr/2024:04:34:43] ENGINE Serving on http://172.21.15.16:8765 2024-04-02T04:34:45.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:44 smithi016 ceph-mon[28969]: [02/Apr/2024:04:34:43] ENGINE Serving on https://172.21.15.16:7150 2024-04-02T04:34:45.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:44 smithi016 ceph-mon[28969]: [02/Apr/2024:04:34:43] ENGINE Bus STARTED 2024-04-02T04:34:45.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:44 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:46.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:45 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:46.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:45 smithi016 ceph-mon[28969]: mgrmap e17: smithi016.eqnebx(active, since 2s) 2024-04-02T04:34:46.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.281+0000 7f604a1b1700 1 -- 172.21.15.27:0/815009703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6044100730 msgr2=0x7f6044100b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:46.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.281+0000 7f604a1b1700 1 --2- 172.21.15.27:0/815009703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6044100730 0x7f6044100b50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6038009910 tx=0x7f6038009c20 comp rx=0 tx=0).stop 2024-04-02T04:34:46.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.281+0000 7f604a1b1700 1 -- 172.21.15.27:0/815009703 shutdown_connections 2024-04-02T04:34:46.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.281+0000 7f604a1b1700 1 --2- 172.21.15.27:0/815009703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6044100730 0x7f6044100b50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:46.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.281+0000 7f604a1b1700 1 -- 172.21.15.27:0/815009703 >> 172.21.15.27:0/815009703 conn(0x7f60440fbcd0 msgr2=0x7f60440fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:34:46.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.281+0000 7f604a1b1700 1 -- 172.21.15.27:0/815009703 shutdown_connections 2024-04-02T04:34:46.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.281+0000 7f604a1b1700 1 -- 172.21.15.27:0/815009703 wait complete. 2024-04-02T04:34:46.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.282+0000 7f604a1b1700 1 Processor -- start 2024-04-02T04:34:46.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.282+0000 7f604a1b1700 1 -- start start 2024-04-02T04:34:46.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.282+0000 7f604a1b1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6044100730 0x7f604419c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:34:46.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.282+0000 7f604a1b1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604419c780 con 0x7f6044100730 2024-04-02T04:34:46.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.283+0000 7f60437fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6044100730 0x7f604419c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:34:46.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.283+0000 7f60437fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6044100730 0x7f604419c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.27:43394/0 (socket says 172.21.15.27:43394) 2024-04-02T04:34:46.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.283+0000 7f60437fe700 1 -- 172.21.15.27:0/3589175572 learned_addr learned my addr 172.21.15.27:0/3589175572 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-02T04:34:46.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.283+0000 7f60437fe700 1 -- 172.21.15.27:0/3589175572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60380095f0 con 0x7f6044100730 2024-04-02T04:34:46.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.283+0000 7f60437fe700 1 --2- 172.21.15.27:0/3589175572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6044100730 0x7f604419c240 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6038006010 tx=0x7f6038009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:34:46.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.283+0000 7f6040ff9700 1 -- 172.21.15.27:0/3589175572 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f603801d070 con 0x7f6044100730 2024-04-02T04:34:46.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.284+0000 7f6040ff9700 1 -- 172.21.15.27:0/3589175572 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6038005dc0 con 0x7f6044100730 2024-04-02T04:34:46.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.284+0000 7f604a1b1700 1 -- 172.21.15.27:0/3589175572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f604419c980 con 0x7f6044100730 2024-04-02T04:34:46.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.284+0000 7f6040ff9700 1 -- 172.21.15.27:0/3589175572 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6038022e90 con 0x7f6044100730 2024-04-02T04:34:46.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.284+0000 7f604a1b1700 1 -- 172.21.15.27:0/3589175572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f604419ce20 con 0x7f6044100730 2024-04-02T04:34:46.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.285+0000 7f6040ff9700 1 -- 172.21.15.27:0/3589175572 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 45661+0+0 (secure 0 0 0) 0x7f603800f4f0 con 0x7f6044100730 2024-04-02T04:34:46.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.285+0000 7f602a7fc700 1 -- 172.21.15.27:0/3589175572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60240052f0 con 0x7f6044100730 2024-04-02T04:34:46.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.285+0000 7f6040ff9700 1 --2- 172.21.15.27:0/3589175572 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f602c038850 0x7f602c03ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:34:46.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.285+0000 7f6040ff9700 1 -- 172.21.15.27:0/3589175572 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f603804ce10 con 0x7f6044100730 2024-04-02T04:34:46.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.285+0000 7f6042ffd700 1 --2- 172.21.15.27:0/3589175572 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f602c038850 0x7f602c03ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:34:46.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.286+0000 7f6042ffd700 1 --2- 172.21.15.27:0/3589175572 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f602c038850 0x7f602c03ad10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6034006fd0 tx=0x7f6034006e40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:34:46.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.290+0000 7f6040ff9700 1 -- 172.21.15.27:0/3589175572 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6038050030 con 0x7f6044100730 2024-04-02T04:34:46.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.580+0000 7f602a7fc700 1 -- 172.21.15.27:0/3589175572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6024005160 con 0x7f6044100730 2024-04-02T04:34:46.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.581+0000 7f6040ff9700 1 -- 172.21.15.27:0/3589175572 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f6038027020 con 0x7f6044100730 2024-04-02T04:34:46.585 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:34:46.585 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":1,"fsid":"dc5403b4-f0a9-11ee-b647-cb9ed24678a4","modified":"2024-04-02T04:32:20.006984Z","created":"2024-04-02T04:32:20.006984Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi016","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-02T04:34:46.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.583+0000 7f602a7fc700 1 -- 172.21.15.27:0/3589175572 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f602c038850 msgr2=0x7f602c03ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:46.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.583+0000 7f602a7fc700 1 --2- 172.21.15.27:0/3589175572 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f602c038850 0x7f602c03ad10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6034006fd0 tx=0x7f6034006e40 comp rx=0 tx=0).stop 2024-04-02T04:34:46.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.583+0000 7f602a7fc700 1 -- 172.21.15.27:0/3589175572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6044100730 msgr2=0x7f604419c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:46.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.583+0000 7f602a7fc700 1 --2- 172.21.15.27:0/3589175572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6044100730 0x7f604419c240 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6038006010 tx=0x7f6038009f80 comp rx=0 tx=0).stop 2024-04-02T04:34:46.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.583+0000 7f602a7fc700 1 -- 172.21.15.27:0/3589175572 shutdown_connections 2024-04-02T04:34:46.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.583+0000 7f602a7fc700 1 --2- 172.21.15.27:0/3589175572 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f602c038850 0x7f602c03ad10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:46.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.583+0000 7f602a7fc700 1 --2- 172.21.15.27:0/3589175572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6044100730 0x7f604419c240 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:46.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.583+0000 7f602a7fc700 1 -- 172.21.15.27:0/3589175572 >> 172.21.15.27:0/3589175572 conn(0x7f60440fbcd0 msgr2=0x7f60440fc7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:34:46.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.583+0000 7f602a7fc700 1 -- 172.21.15.27:0/3589175572 shutdown_connections 2024-04-02T04:34:46.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:46.583+0000 7f602a7fc700 1 -- 172.21.15.27:0/3589175572 wait complete. 2024-04-02T04:34:46.587 INFO:teuthology.orchestra.run.smithi027.stderr:dumped monmap epoch 1 2024-04-02T04:34:47.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:46 smithi016 ceph-mon[28969]: from='client.? 172.21.15.27:0/3589175572' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T04:34:48.535 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-02T04:34:48.535 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T04:34:48.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:48 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:48.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:48 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:48.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:48 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:48.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:48 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:48.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:48 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "who": "osd/host:smithi016", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:34:48.746 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:34:48.799 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T04:34:50.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.597+0000 7fd752fc5700 1 -- 172.21.15.27:0/3771021995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd74c0ff630 msgr2=0x7fd74c0ffa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:50.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.597+0000 7fd752fc5700 1 --2- 172.21.15.27:0/3771021995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd74c0ff630 0x7fd74c0ffa50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd73c009910 tx=0x7fd73c009c20 comp rx=0 tx=0).stop 2024-04-02T04:34:50.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.597+0000 7fd752fc5700 1 -- 172.21.15.27:0/3771021995 shutdown_connections 2024-04-02T04:34:50.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.597+0000 7fd752fc5700 1 --2- 172.21.15.27:0/3771021995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd74c0ff630 0x7fd74c0ffa50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:50.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.597+0000 7fd752fc5700 1 -- 172.21.15.27:0/3771021995 >> 172.21.15.27:0/3771021995 conn(0x7fd74c0fabd0 msgr2=0x7fd74c0fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:34:50.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.597+0000 7fd752fc5700 1 -- 172.21.15.27:0/3771021995 shutdown_connections 2024-04-02T04:34:50.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.597+0000 7fd752fc5700 1 -- 172.21.15.27:0/3771021995 wait complete. 2024-04-02T04:34:50.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.598+0000 7fd752fc5700 1 Processor -- start 2024-04-02T04:34:50.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.598+0000 7fd752fc5700 1 -- start start 2024-04-02T04:34:50.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.598+0000 7fd752fc5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd74c0ff630 0x7fd74c197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:34:50.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.598+0000 7fd752fc5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74c198330 con 0x7fd74c0ff630 2024-04-02T04:34:50.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.598+0000 7fd750d61700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd74c0ff630 0x7fd74c197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:34:50.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.598+0000 7fd750d61700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd74c0ff630 0x7fd74c197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.27:43426/0 (socket says 172.21.15.27:43426) 2024-04-02T04:34:50.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.598+0000 7fd750d61700 1 -- 172.21.15.27:0/2057412750 learned_addr learned my addr 172.21.15.27:0/2057412750 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-02T04:34:50.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.599+0000 7fd750d61700 1 -- 172.21.15.27:0/2057412750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd73c0095f0 con 0x7fd74c0ff630 2024-04-02T04:34:50.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.599+0000 7fd750d61700 1 --2- 172.21.15.27:0/2057412750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd74c0ff630 0x7fd74c197df0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd73c006010 tx=0x7fd73c009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:34:50.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.599+0000 7fd749ffb700 1 -- 172.21.15.27:0/2057412750 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd73c01d070 con 0x7fd74c0ff630 2024-04-02T04:34:50.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.599+0000 7fd752fc5700 1 -- 172.21.15.27:0/2057412750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd74c198530 con 0x7fd74c0ff630 2024-04-02T04:34:50.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.599+0000 7fd752fc5700 1 -- 172.21.15.27:0/2057412750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd74c1989d0 con 0x7fd74c0ff630 2024-04-02T04:34:50.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.599+0000 7fd749ffb700 1 -- 172.21.15.27:0/2057412750 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd73c005dc0 con 0x7fd74c0ff630 2024-04-02T04:34:50.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.599+0000 7fd749ffb700 1 -- 172.21.15.27:0/2057412750 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd73c022e70 con 0x7fd74c0ff630 2024-04-02T04:34:50.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.600+0000 7fd749ffb700 1 -- 172.21.15.27:0/2057412750 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 45661+0+0 (secure 0 0 0) 0x7fd73c00b960 con 0x7fd74c0ff630 2024-04-02T04:34:50.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.600+0000 7fd752fc5700 1 -- 172.21.15.27:0/2057412750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd738005320 con 0x7fd74c0ff630 2024-04-02T04:34:50.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.600+0000 7fd749ffb700 1 --2- 172.21.15.27:0/2057412750 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fd7340388b0 0x7fd73403ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:34:50.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.600+0000 7fd749ffb700 1 -- 172.21.15.27:0/2057412750 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fd73c04ca20 con 0x7fd74c0ff630 2024-04-02T04:34:50.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.601+0000 7fd74bfff700 1 --2- 172.21.15.27:0/2057412750 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fd7340388b0 0x7fd73403ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:34:50.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.601+0000 7fd74bfff700 1 --2- 172.21.15.27:0/2057412750 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fd7340388b0 0x7fd73403ad70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd740006fd0 tx=0x7fd740006e40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:34:50.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.606+0000 7fd749ffb700 1 -- 172.21.15.27:0/2057412750 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd73c022730 con 0x7fd74c0ff630 2024-04-02T04:34:50.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.891+0000 7fd752fc5700 1 -- 172.21.15.27:0/2057412750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd738005190 con 0x7fd74c0ff630 2024-04-02T04:34:50.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.892+0000 7fd749ffb700 1 -- 172.21.15.27:0/2057412750 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fd73c027030 con 0x7fd74c0ff630 2024-04-02T04:34:50.896 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:34:50.896 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":1,"fsid":"dc5403b4-f0a9-11ee-b647-cb9ed24678a4","modified":"2024-04-02T04:32:20.006984Z","created":"2024-04-02T04:32:20.006984Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi016","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-02T04:34:50.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.894+0000 7fd752fc5700 1 -- 172.21.15.27:0/2057412750 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fd7340388b0 msgr2=0x7fd73403ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:50.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.894+0000 7fd752fc5700 1 --2- 172.21.15.27:0/2057412750 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fd7340388b0 0x7fd73403ad70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd740006fd0 tx=0x7fd740006e40 comp rx=0 tx=0).stop 2024-04-02T04:34:50.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.894+0000 7fd752fc5700 1 -- 172.21.15.27:0/2057412750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd74c0ff630 msgr2=0x7fd74c197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:50.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.894+0000 7fd752fc5700 1 --2- 172.21.15.27:0/2057412750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd74c0ff630 0x7fd74c197df0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd73c006010 tx=0x7fd73c009f80 comp rx=0 tx=0).stop 2024-04-02T04:34:50.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.894+0000 7fd752fc5700 1 -- 172.21.15.27:0/2057412750 shutdown_connections 2024-04-02T04:34:50.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.894+0000 7fd752fc5700 1 --2- 172.21.15.27:0/2057412750 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fd7340388b0 0x7fd73403ad70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:50.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.894+0000 7fd752fc5700 1 --2- 172.21.15.27:0/2057412750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd74c0ff630 0x7fd74c197df0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:50.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.894+0000 7fd752fc5700 1 -- 172.21.15.27:0/2057412750 >> 172.21.15.27:0/2057412750 conn(0x7fd74c0fabd0 msgr2=0x7fd74c18f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:34:50.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.894+0000 7fd752fc5700 1 -- 172.21.15.27:0/2057412750 shutdown_connections 2024-04-02T04:34:50.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:50.895+0000 7fd752fc5700 1 -- 172.21.15.27:0/2057412750 wait complete. 2024-04-02T04:34:50.899 INFO:teuthology.orchestra.run.smithi027.stderr:dumped monmap epoch 1 2024-04-02T04:34:51.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:50 smithi016 ceph-mon[28969]: from='client.? 172.21.15.27:0/2057412750' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T04:34:52.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:52 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:52.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:52 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:52.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:52 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:52.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:52 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:52.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:52 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:34:52.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:52 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:34:52.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:52 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:34:52.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:52 smithi016 ceph-mon[28969]: Updating smithi016:/etc/ceph/ceph.conf 2024-04-02T04:34:52.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:52 smithi016 ceph-mon[28969]: Updating smithi027:/etc/ceph/ceph.conf 2024-04-02T04:34:52.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:52 smithi016 ceph-mon[28969]: Updating smithi027:/var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:34:52.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:52 smithi016 ceph-mon[28969]: Updating smithi016:/var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:34:52.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:52 smithi016 ceph-mon[28969]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-04-02T04:34:52.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:52 smithi016 ceph-mon[28969]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-04-02T04:34:52.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:52 smithi016 ceph-mon[28969]: Updating smithi016:/var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T04:34:52.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:52 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:52.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:52 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:52.743 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-02T04:34:52.743 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T04:34:53.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:53 smithi016 ceph-mon[28969]: Updating smithi027:/var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T04:34:53.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:53 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:53.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:53 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:53.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:53 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:53.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:53 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi027", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-02T04:34:53.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:53 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi027", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-02T04:34:53.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:53 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:34:53.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:53 smithi016 ceph-mon[28969]: Deploying daemon ceph-exporter.smithi027 on smithi027 2024-04-02T04:34:53.567 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:34:55.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.427+0000 7f9656299700 1 -- 172.21.15.27:0/2926560868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9650101890 msgr2=0x7f9650101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:55.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.427+0000 7f9656299700 1 --2- 172.21.15.27:0/2926560868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9650101890 0x7f9650101c70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9640009910 tx=0x7f9640009c20 comp rx=0 tx=0).stop 2024-04-02T04:34:55.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.427+0000 7f9656299700 1 -- 172.21.15.27:0/2926560868 shutdown_connections 2024-04-02T04:34:55.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.427+0000 7f9656299700 1 --2- 172.21.15.27:0/2926560868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9650101890 0x7f9650101c70 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:55.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.427+0000 7f9656299700 1 -- 172.21.15.27:0/2926560868 >> 172.21.15.27:0/2926560868 conn(0x7f96500fb0e0 msgr2=0x7f96500fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:34:55.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.427+0000 7f9656299700 1 -- 172.21.15.27:0/2926560868 shutdown_connections 2024-04-02T04:34:55.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.427+0000 7f9656299700 1 -- 172.21.15.27:0/2926560868 wait complete. 2024-04-02T04:34:55.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.427+0000 7f9656299700 1 Processor -- start 2024-04-02T04:34:55.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.427+0000 7f9656299700 1 -- start start 2024-04-02T04:34:55.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.428+0000 7f9656299700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9650101890 0x7f96500ff4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:34:55.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.428+0000 7f9656299700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96501013b0 con 0x7f9650101890 2024-04-02T04:34:55.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.428+0000 7f964ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9650101890 0x7f96500ff4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:34:55.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.428+0000 7f964ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9650101890 0x7f96500ff4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.27:39932/0 (socket says 172.21.15.27:39932) 2024-04-02T04:34:55.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.428+0000 7f964ffff700 1 -- 172.21.15.27:0/642696449 learned_addr learned my addr 172.21.15.27:0/642696449 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-02T04:34:55.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.428+0000 7f964ffff700 1 -- 172.21.15.27:0/642696449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96400095f0 con 0x7f9650101890 2024-04-02T04:34:55.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.429+0000 7f964ffff700 1 --2- 172.21.15.27:0/642696449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9650101890 0x7f96500ff4f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9640005490 tx=0x7f9640005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:34:55.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.429+0000 7f964d7fa700 1 -- 172.21.15.27:0/642696449 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f964001d070 con 0x7f9650101890 2024-04-02T04:34:55.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.429+0000 7f9656299700 1 -- 172.21.15.27:0/642696449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96500ffa30 con 0x7f9650101890 2024-04-02T04:34:55.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.429+0000 7f964d7fa700 1 -- 172.21.15.27:0/642696449 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9640005700 con 0x7f9650101890 2024-04-02T04:34:55.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.429+0000 7f964d7fa700 1 -- 172.21.15.27:0/642696449 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9640003950 con 0x7f9650101890 2024-04-02T04:34:55.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.429+0000 7f9656299700 1 -- 172.21.15.27:0/642696449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96500ffe50 con 0x7f9650101890 2024-04-02T04:34:55.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.430+0000 7f964d7fa700 1 -- 172.21.15.27:0/642696449 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 45661+0+0 (secure 0 0 0) 0x7f9640004240 con 0x7f9650101890 2024-04-02T04:34:55.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.430+0000 7f9656299700 1 -- 172.21.15.27:0/642696449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9650069060 con 0x7f9650101890 2024-04-02T04:34:55.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.430+0000 7f964d7fa700 1 --2- 172.21.15.27:0/642696449 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f96380388c0 0x7f963803ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:34:55.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.430+0000 7f964d7fa700 1 -- 172.21.15.27:0/642696449 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f964004c5b0 con 0x7f9650101890 2024-04-02T04:34:55.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.430+0000 7f964f7fe700 1 --2- 172.21.15.27:0/642696449 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f96380388c0 0x7f963803ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:34:55.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.431+0000 7f964f7fe700 1 --2- 172.21.15.27:0/642696449 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f96380388c0 0x7f963803ad80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9644006fd0 tx=0x7f9644006e40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:34:55.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.435+0000 7f964d7fa700 1 -- 172.21.15.27:0/642696449 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f964000bc10 con 0x7f9650101890 2024-04-02T04:34:55.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.725+0000 7f9656299700 1 -- 172.21.15.27:0/642696449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9650100090 con 0x7f9650101890 2024-04-02T04:34:55.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.726+0000 7f964d7fa700 1 -- 172.21.15.27:0/642696449 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f9640026070 con 0x7f9650101890 2024-04-02T04:34:55.729 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:34:55.730 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":1,"fsid":"dc5403b4-f0a9-11ee-b647-cb9ed24678a4","modified":"2024-04-02T04:32:20.006984Z","created":"2024-04-02T04:32:20.006984Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi016","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-02T04:34:55.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.728+0000 7f9656299700 1 -- 172.21.15.27:0/642696449 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f96380388c0 msgr2=0x7f963803ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:55.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.728+0000 7f9656299700 1 --2- 172.21.15.27:0/642696449 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f96380388c0 0x7f963803ad80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9644006fd0 tx=0x7f9644006e40 comp rx=0 tx=0).stop 2024-04-02T04:34:55.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.728+0000 7f9656299700 1 -- 172.21.15.27:0/642696449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9650101890 msgr2=0x7f96500ff4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:55.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.728+0000 7f9656299700 1 --2- 172.21.15.27:0/642696449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9650101890 0x7f96500ff4f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9640005490 tx=0x7f9640005570 comp rx=0 tx=0).stop 2024-04-02T04:34:55.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.728+0000 7f9656299700 1 -- 172.21.15.27:0/642696449 shutdown_connections 2024-04-02T04:34:55.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.728+0000 7f9656299700 1 --2- 172.21.15.27:0/642696449 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f96380388c0 0x7f963803ad80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:55.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.728+0000 7f9656299700 1 --2- 172.21.15.27:0/642696449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9650101890 0x7f96500ff4f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:55.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.728+0000 7f9656299700 1 -- 172.21.15.27:0/642696449 >> 172.21.15.27:0/642696449 conn(0x7f96500fb0e0 msgr2=0x7f9650104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:34:55.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.728+0000 7f9656299700 1 -- 172.21.15.27:0/642696449 shutdown_connections 2024-04-02T04:34:55.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:55.728+0000 7f9656299700 1 -- 172.21.15.27:0/642696449 wait complete. 2024-04-02T04:34:55.745 INFO:teuthology.orchestra.run.smithi027.stderr:dumped monmap epoch 1 2024-04-02T04:34:56.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:55 smithi016 ceph-mon[28969]: from='client.? 172.21.15.27:0/642696449' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T04:34:57.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:56 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:57.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:56 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:57.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:56 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:57.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:56 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:57.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:56 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi027", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-02T04:34:57.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:56 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi027", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-02T04:34:57.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:56 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:34:57.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:56 smithi016 ceph-mon[28969]: Deploying daemon crash.smithi027 on smithi027 2024-04-02T04:34:57.489 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-02T04:34:57.490 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T04:34:57.732 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:34:59.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:58 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:34:59.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:34:58 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:34:59.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:59.947+0000 7fd596020700 1 -- 172.21.15.27:0/2468232801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd590105670 msgr2=0x7fd590105a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:34:59.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:59.947+0000 7fd596020700 1 --2- 172.21.15.27:0/2468232801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd590105670 0x7fd590105a50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd578009910 tx=0x7fd578009c20 comp rx=0 tx=0).stop 2024-04-02T04:34:59.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:59.947+0000 7fd596020700 1 -- 172.21.15.27:0/2468232801 shutdown_connections 2024-04-02T04:34:59.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:59.947+0000 7fd596020700 1 --2- 172.21.15.27:0/2468232801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd590105670 0x7fd590105a50 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:34:59.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:59.947+0000 7fd596020700 1 -- 172.21.15.27:0/2468232801 >> 172.21.15.27:0/2468232801 conn(0x7fd5900fb110 msgr2=0x7fd5900fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:34:59.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:59.947+0000 7fd596020700 1 -- 172.21.15.27:0/2468232801 shutdown_connections 2024-04-02T04:34:59.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:59.947+0000 7fd596020700 1 -- 172.21.15.27:0/2468232801 wait complete. 2024-04-02T04:34:59.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:59.948+0000 7fd596020700 1 Processor -- start 2024-04-02T04:34:59.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:59.948+0000 7fd596020700 1 -- start start 2024-04-02T04:34:59.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:59.948+0000 7fd596020700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd590105670 0x7fd590196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:34:59.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:59.948+0000 7fd596020700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd590069aa0 con 0x7fd590105670 2024-04-02T04:34:59.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:59.948+0000 7fd58f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd590105670 0x7fd590196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:34:59.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:59.949+0000 7fd58f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd590105670 0x7fd590196da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.27:39964/0 (socket says 172.21.15.27:39964) 2024-04-02T04:34:59.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:59.949+0000 7fd58f7fe700 1 -- 172.21.15.27:0/3259610665 learned_addr learned my addr 172.21.15.27:0/3259610665 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-02T04:34:59.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:59.949+0000 7fd58f7fe700 1 -- 172.21.15.27:0/3259610665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5780095f0 con 0x7fd590105670 2024-04-02T04:34:59.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:59.949+0000 7fd58f7fe700 1 --2- 172.21.15.27:0/3259610665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd590105670 0x7fd590196da0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd578005490 tx=0x7fd578005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:34:59.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:59.949+0000 7fd58cff9700 1 -- 172.21.15.27:0/3259610665 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd57801d070 con 0x7fd590105670 2024-04-02T04:34:59.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:59.949+0000 7fd596020700 1 -- 172.21.15.27:0/3259610665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd590197340 con 0x7fd590105670 2024-04-02T04:34:59.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:59.949+0000 7fd58cff9700 1 -- 172.21.15.27:0/3259610665 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd578005dc0 con 0x7fd590105670 2024-04-02T04:34:59.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:59.949+0000 7fd58cff9700 1 -- 172.21.15.27:0/3259610665 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd578022d60 con 0x7fd590105670 2024-04-02T04:34:59.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:59.949+0000 7fd596020700 1 -- 172.21.15.27:0/3259610665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd590197840 con 0x7fd590105670 2024-04-02T04:34:59.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:59.950+0000 7fd58cff9700 1 -- 172.21.15.27:0/3259610665 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 45661+0+0 (secure 0 0 0) 0x7fd57800b960 con 0x7fd590105670 2024-04-02T04:34:59.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:59.950+0000 7fd596020700 1 -- 172.21.15.27:0/3259610665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd590100d20 con 0x7fd590105670 2024-04-02T04:34:59.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:59.950+0000 7fd58cff9700 1 --2- 172.21.15.27:0/3259610665 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fd57c0388b0 0x7fd57c03ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:34:59.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:59.950+0000 7fd58cff9700 1 -- 172.21.15.27:0/3259610665 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fd57804c9f0 con 0x7fd590105670 2024-04-02T04:34:59.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:59.951+0000 7fd58effd700 1 --2- 172.21.15.27:0/3259610665 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fd57c0388b0 0x7fd57c03ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:34:59.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:59.951+0000 7fd58effd700 1 --2- 172.21.15.27:0/3259610665 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fd57c0388b0 0x7fd57c03ad70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd580006fd0 tx=0x7fd580006e40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:34:59.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:34:59.956+0000 7fd58cff9700 1 -- 172.21.15.27:0/3259610665 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd57800f460 con 0x7fd590105670 2024-04-02T04:35:00.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:00.253+0000 7fd596020700 1 -- 172.21.15.27:0/3259610665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd5900690c0 con 0x7fd590105670 2024-04-02T04:35:00.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:00.253+0000 7fd58cff9700 1 -- 172.21.15.27:0/3259610665 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fd57802ab50 con 0x7fd590105670 2024-04-02T04:35:00.256 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:35:00.256 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":1,"fsid":"dc5403b4-f0a9-11ee-b647-cb9ed24678a4","modified":"2024-04-02T04:32:20.006984Z","created":"2024-04-02T04:32:20.006984Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi016","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-02T04:35:00.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:00.255+0000 7fd596020700 1 -- 172.21.15.27:0/3259610665 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fd57c0388b0 msgr2=0x7fd57c03ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:35:00.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:00.255+0000 7fd596020700 1 --2- 172.21.15.27:0/3259610665 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fd57c0388b0 0x7fd57c03ad70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd580006fd0 tx=0x7fd580006e40 comp rx=0 tx=0).stop 2024-04-02T04:35:00.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:00.255+0000 7fd596020700 1 -- 172.21.15.27:0/3259610665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd590105670 msgr2=0x7fd590196da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:35:00.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:00.255+0000 7fd596020700 1 --2- 172.21.15.27:0/3259610665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd590105670 0x7fd590196da0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd578005490 tx=0x7fd578005570 comp rx=0 tx=0).stop 2024-04-02T04:35:00.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:00.255+0000 7fd596020700 1 -- 172.21.15.27:0/3259610665 shutdown_connections 2024-04-02T04:35:00.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:00.255+0000 7fd596020700 1 --2- 172.21.15.27:0/3259610665 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fd57c0388b0 0x7fd57c03ad70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:35:00.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:00.255+0000 7fd596020700 1 --2- 172.21.15.27:0/3259610665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd590105670 0x7fd590196da0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:35:00.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:00.255+0000 7fd596020700 1 -- 172.21.15.27:0/3259610665 >> 172.21.15.27:0/3259610665 conn(0x7fd5900fb110 msgr2=0x7fd5900fc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:35:00.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:00.256+0000 7fd596020700 1 -- 172.21.15.27:0/3259610665 shutdown_connections 2024-04-02T04:35:00.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:00.256+0000 7fd596020700 1 -- 172.21.15.27:0/3259610665 wait complete. 2024-04-02T04:35:00.259 INFO:teuthology.orchestra.run.smithi027.stderr:dumped monmap epoch 1 2024-04-02T04:35:00.809 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:00 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:00.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:00 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:00.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:00 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:00.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:00 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:00.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:00 smithi016 ceph-mon[28969]: Deploying daemon node-exporter.smithi027 on smithi027 2024-04-02T04:35:00.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:00 smithi016 ceph-mon[28969]: from='client.? 172.21.15.27:0/3259610665' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T04:35:01.829 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-02T04:35:01.829 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T04:35:02.047 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:35:04.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:03 smithi016 ceph-mon[28969]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:04.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:03 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:04.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:04.932+0000 7f1e4ec58700 1 -- 172.21.15.27:0/4083758388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e48105630 msgr2=0x7f1e48105a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:35:04.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:04.932+0000 7f1e4ec58700 1 --2- 172.21.15.27:0/4083758388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e48105630 0x7f1e48105a10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c009910 tx=0x7f1e3c009c20 comp rx=0 tx=0).stop 2024-04-02T04:35:04.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:04.932+0000 7f1e4ec58700 1 -- 172.21.15.27:0/4083758388 shutdown_connections 2024-04-02T04:35:04.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:04.932+0000 7f1e4ec58700 1 --2- 172.21.15.27:0/4083758388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e48105630 0x7f1e48105a10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:35:04.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:04.932+0000 7f1e4ec58700 1 -- 172.21.15.27:0/4083758388 >> 172.21.15.27:0/4083758388 conn(0x7f1e480fa820 msgr2=0x7f1e480fcc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:35:04.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:04.932+0000 7f1e4ec58700 1 -- 172.21.15.27:0/4083758388 shutdown_connections 2024-04-02T04:35:04.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:04.932+0000 7f1e4ec58700 1 -- 172.21.15.27:0/4083758388 wait complete. 2024-04-02T04:35:04.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:04.933+0000 7f1e4ec58700 1 Processor -- start 2024-04-02T04:35:04.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:04.933+0000 7f1e4ec58700 1 -- start start 2024-04-02T04:35:04.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:04.933+0000 7f1e4ec58700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e48105630 0x7f1e48196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:35:04.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:04.933+0000 7f1e4ec58700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e480ffb10 con 0x7f1e48105630 2024-04-02T04:35:04.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:04.933+0000 7f1e4c9f4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e48105630 0x7f1e48196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:35:04.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:04.933+0000 7f1e4c9f4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e48105630 0x7f1e48196e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.27:50478/0 (socket says 172.21.15.27:50478) 2024-04-02T04:35:04.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:04.933+0000 7f1e4c9f4700 1 -- 172.21.15.27:0/423762263 learned_addr learned my addr 172.21.15.27:0/423762263 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-02T04:35:04.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:04.933+0000 7f1e4c9f4700 1 -- 172.21.15.27:0/423762263 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e3c0095f0 con 0x7f1e48105630 2024-04-02T04:35:04.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:04.934+0000 7f1e4c9f4700 1 --2- 172.21.15.27:0/423762263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e48105630 0x7f1e48196e50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c0098e0 tx=0x7f1e3c005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:35:04.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:04.934+0000 7f1e45ffb700 1 -- 172.21.15.27:0/423762263 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1e3c01d070 con 0x7f1e48105630 2024-04-02T04:35:04.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:04.934+0000 7f1e4ec58700 1 -- 172.21.15.27:0/423762263 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e481973f0 con 0x7f1e48105630 2024-04-02T04:35:04.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:04.934+0000 7f1e45ffb700 1 -- 172.21.15.27:0/423762263 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e3c0056b0 con 0x7f1e48105630 2024-04-02T04:35:04.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:04.934+0000 7f1e45ffb700 1 -- 172.21.15.27:0/423762263 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1e3c022d80 con 0x7f1e48105630 2024-04-02T04:35:04.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:04.934+0000 7f1e4ec58700 1 -- 172.21.15.27:0/423762263 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e48197870 con 0x7f1e48105630 2024-04-02T04:35:04.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:04.935+0000 7f1e45ffb700 1 -- 172.21.15.27:0/423762263 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 45661+0+0 (secure 0 0 0) 0x7f1e3c00b960 con 0x7f1e48105630 2024-04-02T04:35:04.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:04.935+0000 7f1e45ffb700 1 --2- 172.21.15.27:0/423762263 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f1e380388b0 0x7f1e3803ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:35:04.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:04.935+0000 7f1e45ffb700 1 -- 172.21.15.27:0/423762263 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f1e3c04c940 con 0x7f1e48105630 2024-04-02T04:35:04.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:04.935+0000 7f1e4ec58700 1 -- 172.21.15.27:0/423762263 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e48100eb0 con 0x7f1e48105630 2024-04-02T04:35:04.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:04.935+0000 7f1e47fff700 1 --2- 172.21.15.27:0/423762263 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f1e380388b0 0x7f1e3803ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:35:04.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:04.936+0000 7f1e47fff700 1 --2- 172.21.15.27:0/423762263 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f1e380388b0 0x7f1e3803ad70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1e34006fd0 tx=0x7f1e34006e40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:35:04.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:04.941+0000 7f1e45ffb700 1 -- 172.21.15.27:0/423762263 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1e3c027070 con 0x7f1e48105630 2024-04-02T04:35:05.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:05.230+0000 7f1e4ec58700 1 -- 172.21.15.27:0/423762263 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f1e480ff1d0 con 0x7f1e48105630 2024-04-02T04:35:05.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:05.230+0000 7f1e45ffb700 1 -- 172.21.15.27:0/423762263 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f1e3c02b730 con 0x7f1e48105630 2024-04-02T04:35:05.233 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:35:05.233 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":1,"fsid":"dc5403b4-f0a9-11ee-b647-cb9ed24678a4","modified":"2024-04-02T04:32:20.006984Z","created":"2024-04-02T04:32:20.006984Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi016","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-02T04:35:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:05.233+0000 7f1e4ec58700 1 -- 172.21.15.27:0/423762263 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f1e380388b0 msgr2=0x7f1e3803ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:35:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:05.233+0000 7f1e4ec58700 1 --2- 172.21.15.27:0/423762263 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f1e380388b0 0x7f1e3803ad70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1e34006fd0 tx=0x7f1e34006e40 comp rx=0 tx=0).stop 2024-04-02T04:35:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:05.233+0000 7f1e4ec58700 1 -- 172.21.15.27:0/423762263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e48105630 msgr2=0x7f1e48196e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:35:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:05.233+0000 7f1e4ec58700 1 --2- 172.21.15.27:0/423762263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e48105630 0x7f1e48196e50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c0098e0 tx=0x7f1e3c005520 comp rx=0 tx=0).stop 2024-04-02T04:35:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:05.233+0000 7f1e4ec58700 1 -- 172.21.15.27:0/423762263 shutdown_connections 2024-04-02T04:35:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:05.233+0000 7f1e4ec58700 1 --2- 172.21.15.27:0/423762263 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f1e380388b0 0x7f1e3803ad70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:35:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:05.233+0000 7f1e4ec58700 1 --2- 172.21.15.27:0/423762263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e48105630 0x7f1e48196e50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:35:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:05.233+0000 7f1e4ec58700 1 -- 172.21.15.27:0/423762263 >> 172.21.15.27:0/423762263 conn(0x7f1e480fa820 msgr2=0x7f1e480fb590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:35:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:05.233+0000 7f1e4ec58700 1 -- 172.21.15.27:0/423762263 shutdown_connections 2024-04-02T04:35:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:05.233+0000 7f1e4ec58700 1 -- 172.21.15.27:0/423762263 wait complete. 2024-04-02T04:35:05.237 INFO:teuthology.orchestra.run.smithi027.stderr:dumped monmap epoch 1 2024-04-02T04:35:06.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:05 smithi016 ceph-mon[28969]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:06.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:05 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:06.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:05 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:06.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:05 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:06.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:05 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:06.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:05 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi027.kjzcsc", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T04:35:06.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:05 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi027.kjzcsc", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-02T04:35:06.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:05 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T04:35:06.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:05 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:35:06.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:05 smithi016 ceph-mon[28969]: from='client.? 172.21.15.27:0/423762263' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T04:35:07.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:06 smithi016 ceph-mon[28969]: Deploying daemon mgr.smithi027.kjzcsc on smithi027 2024-04-02T04:35:07.488 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-02T04:35:07.489 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T04:35:07.929 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:35:08.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:07 smithi016 ceph-mon[28969]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:08.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:07 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:09.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:08 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:09.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:08 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:09.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:08 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:09.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:08 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:09.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:08 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T04:35:09.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:08 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:35:09.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:08 smithi016 ceph-mon[28969]: Deploying daemon mon.smithi027 on smithi027 2024-04-02T04:35:09.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:08 smithi016 ceph-mon[28969]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:10.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.133+0000 7f34eb536700 1 -- 172.21.15.27:0/2699007086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34e41066c0 msgr2=0x7f34e4106aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:35:10.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.133+0000 7f34eb536700 1 --2- 172.21.15.27:0/2699007086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34e41066c0 0x7f34e4106aa0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f34e0009910 tx=0x7f34e0009c20 comp rx=0 tx=0).stop 2024-04-02T04:35:10.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.133+0000 7f34eb536700 1 -- 172.21.15.27:0/2699007086 shutdown_connections 2024-04-02T04:35:10.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.133+0000 7f34eb536700 1 --2- 172.21.15.27:0/2699007086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34e41066c0 0x7f34e4106aa0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:35:10.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.133+0000 7f34eb536700 1 -- 172.21.15.27:0/2699007086 >> 172.21.15.27:0/2699007086 conn(0x7f34e40fc1b0 msgr2=0x7f34e40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:35:10.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.134+0000 7f34eb536700 1 -- 172.21.15.27:0/2699007086 shutdown_connections 2024-04-02T04:35:10.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.134+0000 7f34eb536700 1 -- 172.21.15.27:0/2699007086 wait complete. 2024-04-02T04:35:10.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.135+0000 7f34eb536700 1 Processor -- start 2024-04-02T04:35:10.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.135+0000 7f34eb536700 1 -- start start 2024-04-02T04:35:10.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.135+0000 7f34eb536700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34e41066c0 0x7f34e4195f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:35:10.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.135+0000 7f34eb536700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e41964c0 con 0x7f34e41066c0 2024-04-02T04:35:10.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.135+0000 7f34e92d2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34e41066c0 0x7f34e4195f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:35:10.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.135+0000 7f34e92d2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34e41066c0 0x7f34e4195f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.27:50514/0 (socket says 172.21.15.27:50514) 2024-04-02T04:35:10.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.135+0000 7f34e92d2700 1 -- 172.21.15.27:0/1099617311 learned_addr learned my addr 172.21.15.27:0/1099617311 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-02T04:35:10.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.136+0000 7f34e92d2700 1 -- 172.21.15.27:0/1099617311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e00095f0 con 0x7f34e41066c0 2024-04-02T04:35:10.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.136+0000 7f34e92d2700 1 --2- 172.21.15.27:0/1099617311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34e41066c0 0x7f34e4195f80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f34e0005490 tx=0x7f34e0005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:35:10.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.136+0000 7f34da7fc700 1 -- 172.21.15.27:0/1099617311 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f34e001d070 con 0x7f34e41066c0 2024-04-02T04:35:10.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.136+0000 7f34eb536700 1 -- 172.21.15.27:0/1099617311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34e4196720 con 0x7f34e41066c0 2024-04-02T04:35:10.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.136+0000 7f34eb536700 1 -- 172.21.15.27:0/1099617311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34e4196ba0 con 0x7f34e41066c0 2024-04-02T04:35:10.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.137+0000 7f34da7fc700 1 -- 172.21.15.27:0/1099617311 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f34e0005700 con 0x7f34e41066c0 2024-04-02T04:35:10.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.137+0000 7f34da7fc700 1 -- 172.21.15.27:0/1099617311 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f34e00037d0 con 0x7f34e41066c0 2024-04-02T04:35:10.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.137+0000 7f34da7fc700 1 -- 172.21.15.27:0/1099617311 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 45661+0+0 (secure 0 0 0) 0x7f34e00039f0 con 0x7f34e41066c0 2024-04-02T04:35:10.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.138+0000 7f34da7fc700 1 --2- 172.21.15.27:0/1099617311 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f34d0038910 0x7f34d003add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:35:10.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.138+0000 7f34da7fc700 1 -- 172.21.15.27:0/1099617311 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f34e004d4e0 con 0x7f34e41066c0 2024-04-02T04:35:10.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.138+0000 7f34eb536700 1 -- 172.21.15.27:0/1099617311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34c80052f0 con 0x7f34e41066c0 2024-04-02T04:35:10.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.138+0000 7f34e8ad1700 1 --2- 172.21.15.27:0/1099617311 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f34d0038910 0x7f34d003add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:35:10.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.138+0000 7f34e8ad1700 1 --2- 172.21.15.27:0/1099617311 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f34d0038910 0x7f34d003add0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f34d4006fd0 tx=0x7f34d4006e40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:35:10.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.143+0000 7f34da7fc700 1 -- 172.21.15.27:0/1099617311 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f34e0026020 con 0x7f34e41066c0 2024-04-02T04:35:10.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.470+0000 7f34eb536700 1 -- 172.21.15.27:0/1099617311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f34c80055a0 con 0x7f34e41066c0 2024-04-02T04:35:10.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.471+0000 7f34da7fc700 1 -- 172.21.15.27:0/1099617311 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f34e000f460 con 0x7f34e41066c0 2024-04-02T04:35:10.474 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:35:10.474 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":1,"fsid":"dc5403b4-f0a9-11ee-b647-cb9ed24678a4","modified":"2024-04-02T04:32:20.006984Z","created":"2024-04-02T04:32:20.006984Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi016","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-02T04:35:10.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.473+0000 7f34eb536700 1 -- 172.21.15.27:0/1099617311 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f34d0038910 msgr2=0x7f34d003add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:35:10.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.473+0000 7f34eb536700 1 --2- 172.21.15.27:0/1099617311 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f34d0038910 0x7f34d003add0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f34d4006fd0 tx=0x7f34d4006e40 comp rx=0 tx=0).stop 2024-04-02T04:35:10.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.473+0000 7f34eb536700 1 -- 172.21.15.27:0/1099617311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34e41066c0 msgr2=0x7f34e4195f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:35:10.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.473+0000 7f34eb536700 1 --2- 172.21.15.27:0/1099617311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34e41066c0 0x7f34e4195f80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f34e0005490 tx=0x7f34e0005570 comp rx=0 tx=0).stop 2024-04-02T04:35:10.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.473+0000 7f34eb536700 1 -- 172.21.15.27:0/1099617311 shutdown_connections 2024-04-02T04:35:10.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.473+0000 7f34eb536700 1 --2- 172.21.15.27:0/1099617311 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f34d0038910 0x7f34d003add0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:35:10.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.473+0000 7f34eb536700 1 --2- 172.21.15.27:0/1099617311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34e41066c0 0x7f34e4195f80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:35:10.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.473+0000 7f34eb536700 1 -- 172.21.15.27:0/1099617311 >> 172.21.15.27:0/1099617311 conn(0x7f34e40fc1b0 msgr2=0x7f34e4108290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:35:10.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.473+0000 7f34eb536700 1 -- 172.21.15.27:0/1099617311 shutdown_connections 2024-04-02T04:35:10.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:10.473+0000 7f34eb536700 1 -- 172.21.15.27:0/1099617311 wait complete. 2024-04-02T04:35:10.477 INFO:teuthology.orchestra.run.smithi027.stderr:dumped monmap epoch 1 2024-04-02T04:35:11.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:10 smithi016 ceph-mon[28969]: from='client.? 172.21.15.27:0/1099617311' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T04:35:12.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:11 smithi016 ceph-mon[28969]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:12.108 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-02T04:35:12.108 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T04:35:12.341 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi027/config 2024-04-02T04:35:13.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:12 smithi016 ceph-mon[28969]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:13.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:12 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:35:13.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:12 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:14.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:14.265+0000 7f84618a8700 1 -- 172.21.15.27:0/3510522422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845c1067f0 msgr2=0x7f845c106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:35:14.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:14.265+0000 7f84618a8700 1 --2- 172.21.15.27:0/3510522422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845c1067f0 0x7f845c106bd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8450009910 tx=0x7f8450009c20 comp rx=0 tx=0).stop 2024-04-02T04:35:14.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:14.265+0000 7f84618a8700 1 -- 172.21.15.27:0/3510522422 shutdown_connections 2024-04-02T04:35:14.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:14.265+0000 7f84618a8700 1 --2- 172.21.15.27:0/3510522422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845c1067f0 0x7f845c106bd0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:35:14.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:14.265+0000 7f84618a8700 1 -- 172.21.15.27:0/3510522422 >> 172.21.15.27:0/3510522422 conn(0x7f845c0fc300 msgr2=0x7f845c0fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:35:14.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:14.265+0000 7f84618a8700 1 -- 172.21.15.27:0/3510522422 shutdown_connections 2024-04-02T04:35:14.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:14.265+0000 7f84618a8700 1 -- 172.21.15.27:0/3510522422 wait complete. 2024-04-02T04:35:14.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:14.266+0000 7f84618a8700 1 Processor -- start 2024-04-02T04:35:14.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:14.266+0000 7f84618a8700 1 -- start start 2024-04-02T04:35:14.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:14.266+0000 7f84618a8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845c1067f0 0x7f845c1a12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:35:14.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:14.266+0000 7f84618a8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845c1a1820 con 0x7f845c1067f0 2024-04-02T04:35:14.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:14.267+0000 7f845affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845c1067f0 0x7f845c1a12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:35:14.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:14.267+0000 7f845affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845c1067f0 0x7f845c1a12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.27:44520/0 (socket says 172.21.15.27:44520) 2024-04-02T04:35:14.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:14.267+0000 7f845affd700 1 -- 172.21.15.27:0/2560112287 learned_addr learned my addr 172.21.15.27:0/2560112287 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-02T04:35:14.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:14.267+0000 7f845affd700 1 -- 172.21.15.27:0/2560112287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84500095f0 con 0x7f845c1067f0 2024-04-02T04:35:14.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:14.267+0000 7f845affd700 1 --2- 172.21.15.27:0/2560112287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845c1067f0 0x7f845c1a12e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8450005490 tx=0x7f8450005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:35:14.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:14.267+0000 7f84608a6700 1 -- 172.21.15.27:0/2560112287 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f845001d070 con 0x7f845c1067f0 2024-04-02T04:35:14.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:14.267+0000 7f84608a6700 1 -- 172.21.15.27:0/2560112287 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8450005700 con 0x7f845c1067f0 2024-04-02T04:35:14.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:14.267+0000 7f84608a6700 1 -- 172.21.15.27:0/2560112287 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f84500038e0 con 0x7f845c1067f0 2024-04-02T04:35:14.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:14.267+0000 7f84618a8700 1 -- 172.21.15.27:0/2560112287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f845c1a1a20 con 0x7f845c1067f0 2024-04-02T04:35:14.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:14.268+0000 7f84618a8700 1 -- 172.21.15.27:0/2560112287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f845c19ce30 con 0x7f845c1067f0 2024-04-02T04:35:14.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:14.269+0000 7f84608a6700 1 -- 172.21.15.27:0/2560112287 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 45661+0+0 (secure 0 0 0) 0x7f8450003ba0 con 0x7f845c1067f0 2024-04-02T04:35:14.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:14.269+0000 7f84618a8700 1 -- 172.21.15.27:0/2560112287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f845c101f30 con 0x7f845c1067f0 2024-04-02T04:35:14.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:14.269+0000 7f84608a6700 1 --2- 172.21.15.27:0/2560112287 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f84440388c0 0x7f844403ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:35:14.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:14.269+0000 7f84608a6700 1 -- 172.21.15.27:0/2560112287 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f845004c5e0 con 0x7f845c1067f0 2024-04-02T04:35:14.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:14.269+0000 7f845a7fc700 1 --2- 172.21.15.27:0/2560112287 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f84440388c0 0x7f844403ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:35:14.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:14.270+0000 7f845a7fc700 1 --2- 172.21.15.27:0/2560112287 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f84440388c0 0x7f844403ad80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f844c006fd0 tx=0x7f844c006e40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:35:14.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:14.274+0000 7f84608a6700 1 -- 172.21.15.27:0/2560112287 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8450029360 con 0x7f845c1067f0 2024-04-02T04:35:14.414 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 podman[37501]: 2024-04-02 04:35:14.387698511 +0000 UTC m=+0.777743363 container start b77563184a1e85886b48e078ea5bcaaa6a23dcf9594b86bafda62c96cb1b39d8 (image=quay.ceph.io/ceph-ci/ceph@sha256:397d0c1d908c002c558906e3ea05982bc336d403ee4d900400cc3bdd9243aa2e, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi027, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=reef-4d05f4c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240326) 2024-04-02T04:35:14.414 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: set uid:gid to 167:167 (ceph:ceph) 2024-04-02T04:35:14.414 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable), process ceph-mon, pid 2 2024-04-02T04:35:14.414 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: pidfile_write: ignore empty --pid-file 2024-04-02T04:35:14.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:14.456+0000 7f84608a6700 1 -- 172.21.15.27:0/2560112287 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8450029360 con 0x7f845c1067f0 2024-04-02T04:35:14.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:14.565+0000 7f84618a8700 1 -- 172.21.15.27:0/2560112287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f845c194290 con 0x7f845c1067f0 2024-04-02T04:35:14.802 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: load: jerasure load: lrc 2024-04-02T04:35:14.802 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: RocksDB version: 7.9.2 2024-04-02T04:35:14.802 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Git sha 0 2024-04-02T04:35:14.802 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Compile date 2024-04-01 20:31:51 2024-04-02T04:35:14.803 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: DB SUMMARY 2024-04-02T04:35:14.803 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: DB Session ID: VYCMS7OYPLZEJB5JZOOV 2024-04-02T04:35:14.803 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: CURRENT file: CURRENT 2024-04-02T04:35:14.803 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: IDENTITY file: IDENTITY 2024-04-02T04:35:14.803 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-02T04:35:14.803 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi027/store.db dir, Total Num: 0, files: 2024-04-02T04:35:14.803 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi027/store.db: 000004.log size: 511 ; 2024-04-02T04:35:14.803 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.error_if_exists: 0 2024-04-02T04:35:14.803 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.create_if_missing: 0 2024-04-02T04:35:14.803 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.paranoid_checks: 1 2024-04-02T04:35:14.803 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-02T04:35:14.804 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-02T04:35:14.804 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-02T04:35:14.804 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.env: 0x55bd3e4fd720 2024-04-02T04:35:14.804 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.fs: PosixFileSystem 2024-04-02T04:35:14.804 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.info_log: 0x55bd3fcbd380 2024-04-02T04:35:14.804 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.max_file_opening_threads: 16 2024-04-02T04:35:14.804 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.statistics: (nil) 2024-04-02T04:35:14.804 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.use_fsync: 0 2024-04-02T04:35:14.804 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.max_log_file_size: 0 2024-04-02T04:35:14.804 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-02T04:35:14.804 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-02T04:35:14.804 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.keep_log_file_num: 1000 2024-04-02T04:35:14.805 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.recycle_log_file_num: 0 2024-04-02T04:35:14.805 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.allow_fallocate: 1 2024-04-02T04:35:14.805 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.allow_mmap_reads: 0 2024-04-02T04:35:14.805 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.allow_mmap_writes: 0 2024-04-02T04:35:14.805 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.use_direct_reads: 0 2024-04-02T04:35:14.805 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-02T04:35:14.805 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.create_missing_column_families: 0 2024-04-02T04:35:14.805 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.db_log_dir: 2024-04-02T04:35:14.805 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.wal_dir: 2024-04-02T04:35:14.805 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-02T04:35:14.805 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-02T04:35:14.805 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-02T04:35:14.806 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-02T04:35:14.806 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-02T04:35:14.806 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-02T04:35:14.806 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.advise_random_on_open: 1 2024-04-02T04:35:14.806 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.db_write_buffer_size: 0 2024-04-02T04:35:14.806 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.write_buffer_manager: 0x55bd3ef34140 2024-04-02T04:35:14.806 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-02T04:35:14.806 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-02T04:35:14.806 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-02T04:35:14.806 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.rate_limiter: (nil) 2024-04-02T04:35:14.806 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-02T04:35:14.807 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.wal_recovery_mode: 2 2024-04-02T04:35:14.807 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.enable_thread_tracking: 0 2024-04-02T04:35:14.807 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.enable_pipelined_write: 0 2024-04-02T04:35:14.807 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.unordered_write: 0 2024-04-02T04:35:14.807 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-02T04:35:14.807 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-02T04:35:14.807 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-02T04:35:14.807 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-02T04:35:14.807 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.row_cache: None 2024-04-02T04:35:14.807 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.wal_filter: None 2024-04-02T04:35:14.807 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-02T04:35:14.807 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.allow_ingest_behind: 0 2024-04-02T04:35:14.807 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.two_write_queues: 0 2024-04-02T04:35:14.808 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.manual_wal_flush: 0 2024-04-02T04:35:14.808 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.wal_compression: 0 2024-04-02T04:35:14.808 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.atomic_flush: 0 2024-04-02T04:35:14.808 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-02T04:35:14.808 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-02T04:35:14.808 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-02T04:35:14.808 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.log_readahead_size: 0 2024-04-02T04:35:14.808 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-02T04:35:14.809 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.best_efforts_recovery: 0 2024-04-02T04:35:14.809 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-02T04:35:14.809 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-02T04:35:14.809 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.allow_data_in_errors: 0 2024-04-02T04:35:14.809 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.db_host_id: __hostname__ 2024-04-02T04:35:14.809 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-02T04:35:14.809 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.max_background_jobs: 2 2024-04-02T04:35:14.809 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.max_background_compactions: -1 2024-04-02T04:35:14.809 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.max_subcompactions: 1 2024-04-02T04:35:14.809 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-02T04:35:14.809 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-02T04:35:14.809 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-02T04:35:14.809 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.max_total_wal_size: 0 2024-04-02T04:35:14.810 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-02T04:35:14.810 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-02T04:35:14.810 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-02T04:35:14.810 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-02T04:35:14.810 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.max_open_files: -1 2024-04-02T04:35:14.810 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.bytes_per_sync: 0 2024-04-02T04:35:14.810 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-02T04:35:14.810 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-02T04:35:14.810 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.compaction_readahead_size: 0 2024-04-02T04:35:14.810 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.max_background_flushes: -1 2024-04-02T04:35:14.810 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Compression algorithms supported: 2024-04-02T04:35:14.811 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-02T04:35:14.811 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: kZSTD supported: 0 2024-04-02T04:35:14.811 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: kXpressCompression supported: 0 2024-04-02T04:35:14.811 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: kLZ4HCCompression supported: 1 2024-04-02T04:35:14.811 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: kZlibCompression supported: 1 2024-04-02T04:35:14.811 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: kSnappyCompression supported: 1 2024-04-02T04:35:14.811 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: kLZ4Compression supported: 1 2024-04-02T04:35:14.811 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: kBZip2Compression supported: 0 2024-04-02T04:35:14.811 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-02T04:35:14.811 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-02T04:35:14.811 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi027/store.db/MANIFEST-000005 2024-04-02T04:35:14.812 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-02T04:35:14.812 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-02T04:35:14.812 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.merge_operator: 2024-04-02T04:35:14.812 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.compaction_filter: None 2024-04-02T04:35:14.812 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.compaction_filter_factory: None 2024-04-02T04:35:14.812 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.sst_partitioner_factory: None 2024-04-02T04:35:14.812 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-02T04:35:14.812 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-02T04:35:14.812 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55bd3fcbd4a0) 2024-04-02T04:35:14.812 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: cache_index_and_filter_blocks: 1 2024-04-02T04:35:14.812 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-02T04:35:14.812 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-02T04:35:14.812 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: pin_top_level_index_and_filter: 1 2024-04-02T04:35:14.813 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: index_type: 0 2024-04-02T04:35:14.813 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: data_block_index_type: 0 2024-04-02T04:35:14.813 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: index_shortening: 1 2024-04-02T04:35:14.813 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-02T04:35:14.813 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: checksum: 4 2024-04-02T04:35:14.813 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: no_block_cache: 0 2024-04-02T04:35:14.813 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: block_cache: 0x55bd3efd51f0 2024-04-02T04:35:14.813 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: block_cache_name: BinnedLRUCache 2024-04-02T04:35:14.813 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: block_cache_options: 2024-04-02T04:35:14.813 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: capacity : 536870912 2024-04-02T04:35:14.813 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: num_shard_bits : 4 2024-04-02T04:35:14.813 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: strict_capacity_limit : 0 2024-04-02T04:35:14.814 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: high_pri_pool_ratio: 0.000 2024-04-02T04:35:14.814 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: block_cache_compressed: (nil) 2024-04-02T04:35:14.814 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: persistent_cache: (nil) 2024-04-02T04:35:14.814 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: block_size: 4096 2024-04-02T04:35:14.814 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: block_size_deviation: 10 2024-04-02T04:35:14.814 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: block_restart_interval: 16 2024-04-02T04:35:14.814 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: index_block_restart_interval: 1 2024-04-02T04:35:14.814 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: metadata_block_size: 4096 2024-04-02T04:35:14.814 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: partition_filters: 0 2024-04-02T04:35:14.814 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: use_delta_encoding: 1 2024-04-02T04:35:14.814 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: filter_policy: bloomfilter 2024-04-02T04:35:14.815 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: whole_key_filtering: 1 2024-04-02T04:35:14.815 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: verify_compression: 0 2024-04-02T04:35:14.815 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: read_amp_bytes_per_bit: 0 2024-04-02T04:35:14.815 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: format_version: 5 2024-04-02T04:35:14.815 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: enable_index_compression: 1 2024-04-02T04:35:14.815 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: block_align: 0 2024-04-02T04:35:14.815 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: max_auto_readahead_size: 262144 2024-04-02T04:35:14.815 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: prepopulate_block_cache: 0 2024-04-02T04:35:14.815 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: initial_auto_readahead_size: 8192 2024-04-02T04:35:14.815 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: num_file_reads_for_auto_readahead: 2 2024-04-02T04:35:14.815 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.write_buffer_size: 33554432 2024-04-02T04:35:14.816 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.max_write_buffer_number: 2 2024-04-02T04:35:14.816 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.compression: NoCompression 2024-04-02T04:35:14.816 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.bottommost_compression: Disabled 2024-04-02T04:35:14.816 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.prefix_extractor: nullptr 2024-04-02T04:35:14.816 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-02T04:35:14.816 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.num_levels: 7 2024-04-02T04:35:14.816 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-02T04:35:14.816 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-02T04:35:14.816 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-02T04:35:14.816 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-02T04:35:14.816 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-02T04:35:14.816 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-02T04:35:14.817 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-02T04:35:14.817 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-02T04:35:14.817 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-02T04:35:14.817 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-02T04:35:14.817 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-02T04:35:14.818 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-02T04:35:14.818 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-02T04:35:14.818 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.compression_opts.level: 32767 2024-04-02T04:35:14.818 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.compression_opts.strategy: 0 2024-04-02T04:35:14.819 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-02T04:35:14.819 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-02T04:35:14.819 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-02T04:35:14.819 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-02T04:35:14.819 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.compression_opts.enabled: false 2024-04-02T04:35:14.819 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-02T04:35:14.819 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-02T04:35:14.819 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-02T04:35:14.819 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-02T04:35:14.819 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.target_file_size_base: 67108864 2024-04-02T04:35:14.819 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-02T04:35:14.819 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-02T04:35:14.819 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-02T04:35:14.819 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-02T04:35:14.820 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-02T04:35:14.820 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-02T04:35:14.820 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-02T04:35:14.820 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-02T04:35:14.820 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-02T04:35:14.820 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-02T04:35:14.820 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-02T04:35:14.820 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-02T04:35:14.820 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-02T04:35:14.820 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-02T04:35:14.820 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.arena_block_size: 1048576 2024-04-02T04:35:14.820 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-02T04:35:14.820 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-02T04:35:14.820 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.disable_auto_compactions: 0 2024-04-02T04:35:14.821 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-02T04:35:14.821 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-02T04:35:14.821 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-02T04:35:14.821 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-02T04:35:14.821 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-02T04:35:14.821 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-02T04:35:14.821 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-02T04:35:14.821 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-02T04:35:14.821 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-02T04:35:14.821 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-02T04:35:14.821 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.table_properties_collectors: 2024-04-02T04:35:14.821 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.inplace_update_support: 0 2024-04-02T04:35:14.821 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-02T04:35:14.821 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-02T04:35:14.822 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-02T04:35:14.822 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-02T04:35:14.822 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.bloom_locality: 0 2024-04-02T04:35:14.822 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.max_successive_merges: 0 2024-04-02T04:35:14.822 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-02T04:35:14.822 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.paranoid_file_checks: 0 2024-04-02T04:35:14.822 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.force_consistency_checks: 1 2024-04-02T04:35:14.822 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.report_bg_io_stats: 0 2024-04-02T04:35:14.822 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.ttl: 2592000 2024-04-02T04:35:14.822 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-02T04:35:14.822 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-02T04:35:14.822 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-02T04:35:14.823 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.enable_blob_files: false 2024-04-02T04:35:14.823 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.min_blob_size: 0 2024-04-02T04:35:14.823 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.blob_file_size: 268435456 2024-04-02T04:35:14.823 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-02T04:35:14.823 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-02T04:35:14.823 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-02T04:35:14.823 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-02T04:35:14.823 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-02T04:35:14.823 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.blob_file_starting_level: 0 2024-04-02T04:35:14.823 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-02T04:35:14.823 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi027/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-02T04:35:14.823 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-02T04:35:14.824 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: de102378-e598-4743-8929-633e7d254a4b 2024-04-02T04:35:14.824 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712032514428697, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-02T04:35:14.824 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-02T04:35:14.824 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712032514429189, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1712032514, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "de102378-e598-4743-8929-633e7d254a4b", "db_session_id": "VYCMS7OYPLZEJB5JZOOV", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-02T04:35:14.824 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712032514429289, "job": 1, "event": "recovery_finished"} 2024-04-02T04:35:14.824 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-02T04:35:14.824 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi027/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-02T04:35:14.824 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55bd3f072000 2024-04-02T04:35:14.824 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: DB pointer 0x55bd3f060000 2024-04-02T04:35:14.824 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-02T04:35:14.824 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-02T04:35:14.824 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: ** DB Stats ** 2024-04-02T04:35:14.825 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-02T04:35:14.825 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-02T04:35:14.825 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-02T04:35:14.825 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-02T04:35:14.825 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-02T04:35:14.825 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-02T04:35:14.825 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-02T04:35:14.825 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: 2024-04-02T04:35:14.825 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: ** Compaction Stats [default] ** 2024-04-02T04:35:14.825 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-02T04:35:14.825 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-02T04:35:14.825 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-02T04:35:14.825 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-02T04:35:14.826 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-02T04:35:14.826 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: 2024-04-02T04:35:14.826 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: ** Compaction Stats [default] ** 2024-04-02T04:35:14.826 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-02T04:35:14.826 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-02T04:35:14.826 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-02T04:35:14.826 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: 2024-04-02T04:35:14.826 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-02T04:35:14.826 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: 2024-04-02T04:35:14.826 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-02T04:35:14.826 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-02T04:35:14.826 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-02T04:35:14.826 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-02T04:35:14.827 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-02T04:35:14.827 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-02T04:35:14.827 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Cumulative compaction: 0.00 GB write, 0.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-02T04:35:14.827 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Interval compaction: 0.00 GB write, 0.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-02T04:35:14.827 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-02T04:35:14.827 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Block cache BinnedLRUCache@0x55bd3efd51f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-04-02T04:35:14.827 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-02T04:35:14.827 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: 2024-04-02T04:35:14.827 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-02T04:35:14.827 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: mon.smithi027 does not exist in monmap, will attempt to join an existing cluster 2024-04-02T04:35:14.827 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: using public_addr v2:172.21.15.27:0/0 -> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] 2024-04-02T04:35:14.827 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: starting mon.smithi027 rank -1 at public addrs [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] at bind addrs [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi027 fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 2024-04-02T04:35:14.827 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: mon.smithi027@-1(???) e0 preinit fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 2024-04-02T04:35:14.827 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: mon.smithi027@-1(synchronizing).mds e1 new map 2024-04-02T04:35:14.828 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: mon.smithi027@-1(synchronizing).mds e1 print_map 2024-04-02T04:35:14.828 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: e1 2024-04-02T04:35:14.828 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:35:14.828 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:35:14.828 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: legacy client fscid: -1 2024-04-02T04:35:14.828 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: 2024-04-02T04:35:14.828 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: No filesystems configured 2024-04-02T04:35:14.828 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: mon.smithi027@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-02T04:35:14.828 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: mon.smithi027@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-02T04:35:14.828 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: mon.smithi027@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-02T04:35:14.828 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: mon.smithi027@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-02T04:35:14.828 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: mon.smithi027@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-02T04:35:14.828 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: mon.smithi027@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-02T04:35:14.828 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: mon.smithi027@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2024-04-02T04:35:14.829 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: mon.smithi027@-1(synchronizing).osd e5 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-02T04:35:14.829 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: mon.smithi027@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-04-02T04:35:14.829 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: mon.smithi027@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-04-02T04:35:14.829 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: mon.smithi027@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-04-02T04:35:14.829 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: from='client.? 172.21.15.27:0/1099617311' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T04:35:14.829 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:14.829 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:14.829 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:35:14.829 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:14.829 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 ceph-mon[37550]: mon.smithi027@-1(synchronizing).paxosservice(auth 1..8) refresh upgraded, format 0 -> 3 2024-04-02T04:35:14.829 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 bash[37501]: b77563184a1e85886b48e078ea5bcaaa6a23dcf9594b86bafda62c96cb1b39d8 2024-04-02T04:35:14.829 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:14 smithi027 systemd[1]: Started Ceph mon.smithi027 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4. 2024-04-02T04:35:19.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:19.465+0000 7f84608a6700 1 -- 172.21.15.27:0/2560112287 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1018 (secure 0 0 0) 0x7f8450026070 con 0x7f845c1067f0 2024-04-02T04:35:19.469 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-02T04:35:19.469 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":2,"fsid":"dc5403b4-f0a9-11ee-b647-cb9ed24678a4","modified":"2024-04-02T04:35:14.456167Z","created":"2024-04-02T04:32:20.006984Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi016","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi027","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:3300","nonce":0},{"type":"v1","addr":"172.21.15.27:6789","nonce":0}]},"addr":"172.21.15.27:6789/0","public_addr":"172.21.15.27:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-02T04:35:19.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:19.468+0000 7f84618a8700 1 -- 172.21.15.27:0/2560112287 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f84440388c0 msgr2=0x7f844403ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:35:19.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:19.468+0000 7f84618a8700 1 --2- 172.21.15.27:0/2560112287 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f84440388c0 0x7f844403ad80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f844c006fd0 tx=0x7f844c006e40 comp rx=0 tx=0).stop 2024-04-02T04:35:19.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:19.468+0000 7f84618a8700 1 -- 172.21.15.27:0/2560112287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845c1067f0 msgr2=0x7f845c1a12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:35:19.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:19.468+0000 7f84618a8700 1 --2- 172.21.15.27:0/2560112287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845c1067f0 0x7f845c1a12e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8450005490 tx=0x7f8450005570 comp rx=0 tx=0).stop 2024-04-02T04:35:19.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:19.468+0000 7f84618a8700 1 -- 172.21.15.27:0/2560112287 shutdown_connections 2024-04-02T04:35:19.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:19.468+0000 7f84618a8700 1 --2- 172.21.15.27:0/2560112287 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f84440388c0 0x7f844403ad80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:35:19.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:19.468+0000 7f84618a8700 1 --2- 172.21.15.27:0/2560112287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845c1067f0 0x7f845c1a12e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:35:19.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:19.468+0000 7f84618a8700 1 -- 172.21.15.27:0/2560112287 >> 172.21.15.27:0/2560112287 conn(0x7f845c0fc300 msgr2=0x7f845c0fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:35:19.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:19.468+0000 7f84618a8700 1 -- 172.21.15.27:0/2560112287 shutdown_connections 2024-04-02T04:35:19.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:35:19.468+0000 7f84618a8700 1 -- 172.21.15.27:0/2560112287 wait complete. 2024-04-02T04:35:19.473 INFO:teuthology.orchestra.run.smithi027.stderr:dumped monmap epoch 2 2024-04-02T04:35:19.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:19 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi016"}]: dispatch 2024-04-02T04:35:19.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:19 smithi016 ceph-mon[28969]: mon.smithi016 calling monitor election 2024-04-02T04:35:19.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:19 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2024-04-02T04:35:19.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:19 smithi016 ceph-mon[28969]: from='client.? 172.21.15.27:0/2560112287' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T04:35:19.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:19 smithi016 ceph-mon[28969]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:19.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:19 smithi016 ceph-mon[28969]: from='mgr.? 172.21.15.27:0/2032802220' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi027.kjzcsc/crt"}]: dispatch 2024-04-02T04:35:19.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:19 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2024-04-02T04:35:19.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:19 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2024-04-02T04:35:19.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:19 smithi016 ceph-mon[28969]: mon.smithi027 calling monitor election 2024-04-02T04:35:19.811 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:19 smithi016 ceph-mon[28969]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:19.811 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:19 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2024-04-02T04:35:19.811 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:19 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2024-04-02T04:35:19.811 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:19 smithi016 ceph-mon[28969]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:19.811 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:19 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2024-04-02T04:35:19.811 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:19 smithi016 ceph-mon[28969]: mon.smithi016 is new leader, mons smithi016,smithi027 in quorum (ranks 0,1) 2024-04-02T04:35:19.811 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:19 smithi016 ceph-mon[28969]: monmap e2: 2 mons at {smithi016=[v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0],smithi027=[v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-02T04:35:19.811 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:19 smithi016 ceph-mon[28969]: fsmap 2024-04-02T04:35:19.811 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:19 smithi016 ceph-mon[28969]: osdmap e5: 0 total, 0 up, 0 in 2024-04-02T04:35:19.811 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:19 smithi016 ceph-mon[28969]: mgrmap e17: smithi016.eqnebx(active, since 36s) 2024-04-02T04:35:19.811 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:19 smithi016 ceph-mon[28969]: Standby manager daemon smithi027.kjzcsc started 2024-04-02T04:35:19.811 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:19 smithi016 ceph-mon[28969]: from='mgr.? 172.21.15.27:0/2032802220' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T04:35:19.811 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:19 smithi016 ceph-mon[28969]: overall HEALTH_OK 2024-04-02T04:35:19.811 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:19 smithi016 ceph-mon[28969]: from='mgr.? 172.21.15.27:0/2032802220' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi027.kjzcsc/key"}]: dispatch 2024-04-02T04:35:19.811 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:19 smithi016 ceph-mon[28969]: from='mgr.? 172.21.15.27:0/2032802220' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T04:35:19.811 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:19 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:19.812 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:19 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:19.812 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:19 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:19.812 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:19 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:19.816 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:19 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi016"}]: dispatch 2024-04-02T04:35:19.816 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:19 smithi027 ceph-mon[37550]: mon.smithi016 calling monitor election 2024-04-02T04:35:19.816 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:19 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2024-04-02T04:35:19.816 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:19 smithi027 ceph-mon[37550]: from='client.? 172.21.15.27:0/2560112287' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T04:35:19.816 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:19 smithi027 ceph-mon[37550]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:19.816 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:19 smithi027 ceph-mon[37550]: from='mgr.? 172.21.15.27:0/2032802220' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi027.kjzcsc/crt"}]: dispatch 2024-04-02T04:35:19.816 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:19 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2024-04-02T04:35:19.817 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:19 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2024-04-02T04:35:19.817 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:19 smithi027 ceph-mon[37550]: mon.smithi027 calling monitor election 2024-04-02T04:35:19.817 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:19 smithi027 ceph-mon[37550]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:19.817 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:19 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2024-04-02T04:35:19.817 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:19 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2024-04-02T04:35:19.817 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:19 smithi027 ceph-mon[37550]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:19.817 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:19 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2024-04-02T04:35:19.817 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:19 smithi027 ceph-mon[37550]: mon.smithi016 is new leader, mons smithi016,smithi027 in quorum (ranks 0,1) 2024-04-02T04:35:19.817 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:19 smithi027 ceph-mon[37550]: monmap e2: 2 mons at {smithi016=[v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0],smithi027=[v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-02T04:35:19.817 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:19 smithi027 ceph-mon[37550]: fsmap 2024-04-02T04:35:19.817 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:19 smithi027 ceph-mon[37550]: osdmap e5: 0 total, 0 up, 0 in 2024-04-02T04:35:19.817 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:19 smithi027 ceph-mon[37550]: mgrmap e17: smithi016.eqnebx(active, since 36s) 2024-04-02T04:35:19.817 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:19 smithi027 ceph-mon[37550]: Standby manager daemon smithi027.kjzcsc started 2024-04-02T04:35:19.817 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:19 smithi027 ceph-mon[37550]: from='mgr.? 172.21.15.27:0/2032802220' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T04:35:19.818 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:19 smithi027 ceph-mon[37550]: overall HEALTH_OK 2024-04-02T04:35:19.818 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:19 smithi027 ceph-mon[37550]: from='mgr.? 172.21.15.27:0/2032802220' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi027.kjzcsc/key"}]: dispatch 2024-04-02T04:35:19.818 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:19 smithi027 ceph-mon[37550]: from='mgr.? 172.21.15.27:0/2032802220' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T04:35:19.818 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:19 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:19.818 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:19 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:19.818 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:19 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:19.818 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:19 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:20.113 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-02T04:35:20.113 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph config generate-minimal-conf 2024-04-02T04:35:20.332 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:35:20.754 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:20 smithi016 ceph-mon[28969]: mgrmap e18: smithi016.eqnebx(active, since 36s), standbys: smithi027.kjzcsc 2024-04-02T04:35:20.755 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:20 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mgr metadata", "who": "smithi027.kjzcsc", "id": "smithi027.kjzcsc"}]: dispatch 2024-04-02T04:35:20.755 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:20 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:20.755 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:20 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:35:20.755 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:20 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2024-04-02T04:35:20.825 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:20 smithi027 ceph-mon[37550]: mgrmap e18: smithi016.eqnebx(active, since 36s), standbys: smithi027.kjzcsc 2024-04-02T04:35:20.825 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:20 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mgr metadata", "who": "smithi027.kjzcsc", "id": "smithi027.kjzcsc"}]: dispatch 2024-04-02T04:35:20.825 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:20 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:20.825 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:20 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:35:20.825 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:20 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2024-04-02T04:35:21.582 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:21 smithi016 ceph-mon[28969]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:21.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:21.758+0000 7f3dc5080700 1 -- 172.21.15.16:0/3455872319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc0105650 msgr2=0x7f3dc0105a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:35:21.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:21.758+0000 7f3dc5080700 1 --2- 172.21.15.16:0/3455872319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc0105650 0x7f3dc0105a30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3db000b0a0 tx=0x7f3db000b3b0 comp rx=0 tx=0).stop 2024-04-02T04:35:21.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:21.758+0000 7f3dc5080700 1 -- 172.21.15.16:0/3455872319 shutdown_connections 2024-04-02T04:35:21.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:21.758+0000 7f3dc5080700 1 --2- 172.21.15.16:0/3455872319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc0105650 0x7f3dc0105a30 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:35:21.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:21.758+0000 7f3dc5080700 1 -- 172.21.15.16:0/3455872319 >> 172.21.15.16:0/3455872319 conn(0x7f3dc00fb110 msgr2=0x7f3dc00fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:35:21.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:21.758+0000 7f3dc5080700 1 -- 172.21.15.16:0/3455872319 shutdown_connections 2024-04-02T04:35:21.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:21.758+0000 7f3dc5080700 1 -- 172.21.15.16:0/3455872319 wait complete. 2024-04-02T04:35:21.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:21.759+0000 7f3dc5080700 1 Processor -- start 2024-04-02T04:35:21.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:21.759+0000 7f3dc5080700 1 -- start start 2024-04-02T04:35:21.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:21.759+0000 7f3dc5080700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3dc0105650 0x7f3dc0196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:35:21.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:21.759+0000 7f3dc5080700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc01974b0 0x7f3dc0197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:35:21.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:21.759+0000 7f3dc5080700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc019ffe0 con 0x7f3dc01974b0 2024-04-02T04:35:21.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:21.759+0000 7f3dc5080700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc01a0120 con 0x7f3dc0105650 2024-04-02T04:35:21.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:21.759+0000 7f3dbe59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc01974b0 0x7f3dc0197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:35:21.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:21.759+0000 7f3dbe59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc01974b0 0x7f3dc0197930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54878/0 (socket says 172.21.15.16:54878) 2024-04-02T04:35:21.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:21.759+0000 7f3dbe59c700 1 -- 172.21.15.16:0/3972007146 learned_addr learned my addr 172.21.15.16:0/3972007146 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:35:21.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:21.760+0000 7f3dbed9d700 1 --2- 172.21.15.16:0/3972007146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3dc0105650 0x7f3dc0196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:35:21.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:21.760+0000 7f3dbe59c700 1 -- 172.21.15.16:0/3972007146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3dc0105650 msgr2=0x7f3dc0196f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:35:21.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:21.760+0000 7f3dbe59c700 1 --2- 172.21.15.16:0/3972007146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3dc0105650 0x7f3dc0196f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:35:21.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:21.760+0000 7f3dbe59c700 1 -- 172.21.15.16:0/3972007146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db0009d00 con 0x7f3dc01974b0 2024-04-02T04:35:21.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:21.760+0000 7f3dbe59c700 1 --2- 172.21.15.16:0/3972007146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc01974b0 0x7f3dc0197930 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3da800b730 tx=0x7f3da800baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:35:21.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:21.760+0000 7f3db7fff700 1 -- 172.21.15.16:0/3972007146 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3da8004200 con 0x7f3dc01974b0 2024-04-02T04:35:21.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:21.760+0000 7f3dc5080700 1 -- 172.21.15.16:0/3972007146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dc01a0320 con 0x7f3dc01974b0 2024-04-02T04:35:21.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:21.760+0000 7f3db7fff700 1 -- 172.21.15.16:0/3972007146 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3da8004360 con 0x7f3dc01974b0 2024-04-02T04:35:21.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:21.760+0000 7f3db7fff700 1 -- 172.21.15.16:0/3972007146 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3da8011c00 con 0x7f3dc01974b0 2024-04-02T04:35:21.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:21.760+0000 7f3dc5080700 1 -- 172.21.15.16:0/3972007146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc01a0840 con 0x7f3dc01974b0 2024-04-02T04:35:21.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:21.761+0000 7f3dc5080700 1 -- 172.21.15.16:0/3972007146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dc0100d00 con 0x7f3dc01974b0 2024-04-02T04:35:21.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:21.762+0000 7f3db7fff700 1 -- 172.21.15.16:0/3972007146 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 90786+0+0 (secure 0 0 0) 0x7f3da80036a0 con 0x7f3dc01974b0 2024-04-02T04:35:21.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:21.762+0000 7f3db7fff700 1 --2- 172.21.15.16:0/3972007146 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f3dac06cdb0 0x7f3dac06f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:35:21.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:21.762+0000 7f3dbed9d700 1 --2- 172.21.15.16:0/3972007146 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f3dac06cdb0 0x7f3dac06f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:35:21.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:21.762+0000 7f3db7fff700 1 -- 172.21.15.16:0/3972007146 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f3da808acd0 con 0x7f3dc01974b0 2024-04-02T04:35:21.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:21.763+0000 7f3dbed9d700 1 --2- 172.21.15.16:0/3972007146 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f3dac06cdb0 0x7f3dac06f270 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3db000b830 tx=0x7f3db0006210 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:35:21.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:21.767+0000 7f3db7fff700 1 -- 172.21.15.16:0/3972007146 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3da8056180 con 0x7f3dc01974b0 2024-04-02T04:35:21.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:21 smithi027 ceph-mon[37550]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:22.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:22.002+0000 7f3dc5080700 1 -- 172.21.15.16:0/3972007146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f3dc00623e0 con 0x7f3dc01974b0 2024-04-02T04:35:22.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:22.003+0000 7f3db7fff700 1 -- 172.21.15.16:0/3972007146 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+223 (secure 0 0 0) 0x7f3da808a490 con 0x7f3dc01974b0 2024-04-02T04:35:22.007 INFO:teuthology.orchestra.run.smithi016.stdout:# minimal ceph.conf for dc5403b4-f0a9-11ee-b647-cb9ed24678a4 2024-04-02T04:35:22.007 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2024-04-02T04:35:22.007 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = dc5403b4-f0a9-11ee-b647-cb9ed24678a4 2024-04-02T04:35:22.007 INFO:teuthology.orchestra.run.smithi016.stdout: mon_host = [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] 2024-04-02T04:35:22.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:22.005+0000 7f3dc5080700 1 -- 172.21.15.16:0/3972007146 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f3dac06cdb0 msgr2=0x7f3dac06f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:35:22.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:22.005+0000 7f3dc5080700 1 --2- 172.21.15.16:0/3972007146 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f3dac06cdb0 0x7f3dac06f270 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3db000b830 tx=0x7f3db0006210 comp rx=0 tx=0).stop 2024-04-02T04:35:22.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:22.005+0000 7f3dc5080700 1 -- 172.21.15.16:0/3972007146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc01974b0 msgr2=0x7f3dc0197930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:35:22.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:22.005+0000 7f3dc5080700 1 --2- 172.21.15.16:0/3972007146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc01974b0 0x7f3dc0197930 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3da800b730 tx=0x7f3da800baf0 comp rx=0 tx=0).stop 2024-04-02T04:35:22.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:22.005+0000 7f3dc5080700 1 -- 172.21.15.16:0/3972007146 shutdown_connections 2024-04-02T04:35:22.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:22.005+0000 7f3dc5080700 1 --2- 172.21.15.16:0/3972007146 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f3dac06cdb0 0x7f3dac06f270 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:35:22.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:22.005+0000 7f3dc5080700 1 --2- 172.21.15.16:0/3972007146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3dc0105650 0x7f3dc0196f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:35:22.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:22.005+0000 7f3dc5080700 1 --2- 172.21.15.16:0/3972007146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc01974b0 0x7f3dc0197930 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:35:22.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:22.005+0000 7f3dc5080700 1 -- 172.21.15.16:0/3972007146 >> 172.21.15.16:0/3972007146 conn(0x7f3dc00fb110 msgr2=0x7f3dc00fb4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:35:22.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:22.006+0000 7f3dc5080700 1 -- 172.21.15.16:0/3972007146 shutdown_connections 2024-04-02T04:35:22.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:22.006+0000 7f3dc5080700 1 -- 172.21.15.16:0/3972007146 wait complete. 2024-04-02T04:35:22.700 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-02T04:35:22.700 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-02T04:35:22.700 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.conf 2024-04-02T04:35:22.730 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-02T04:35:22.731 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-02T04:35:22.799 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-02T04:35:22.799 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.conf 2024-04-02T04:35:22.833 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-02T04:35:22.833 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-02T04:35:22.905 INFO:tasks.cephadm:Deploying OSDs... 2024-04-02T04:35:22.905 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-02T04:35:22.905 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/scratch_devs of=/dev/stdout 2024-04-02T04:35:22.920 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-02T04:35:22.921 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_1 2024-04-02T04:35:22.963 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:22 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:22.964 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:22 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:22.964 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:22 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:35:22.964 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:22 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:35:22.964 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:22 smithi016 ceph-mon[28969]: Updating smithi016:/etc/ceph/ceph.conf 2024-04-02T04:35:22.964 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:22 smithi016 ceph-mon[28969]: Updating smithi027:/etc/ceph/ceph.conf 2024-04-02T04:35:22.964 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:22 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/3972007146' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:35:22.964 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:22 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:22.964 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:22 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:22.964 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:22 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:22.964 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:22 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:22.964 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:22 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:22.964 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:22 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T04:35:22.964 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:22 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T04:35:22.965 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:22 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:35:22.965 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:22 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:22.976 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-02T04:35:22.976 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T04:35:22.976 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-04-02T04:35:22.976 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T04:35:22.976 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T04:35:22.976 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-04-02 04:34:34.074676908 +0000 2024-04-02T04:35:22.976 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-04-02 04:28:50.577028900 +0000 2024-04-02T04:35:22.976 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-04-02 04:28:50.577028900 +0000 2024-04-02T04:35:22.976 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-04-02 04:28:50.577028900 +0000 2024-04-02T04:35:22.977 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-02T04:35:23.041 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-04-02T04:35:23.041 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-04-02T04:35:23.041 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 8.8473e-05 s, 5.8 MB/s 2024-04-02T04:35:23.042 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-02T04:35:23.097 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_2 2024-04-02T04:35:23.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:22 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:23.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:22 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:23.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:22 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:35:23.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:22 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:35:23.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:22 smithi027 ceph-mon[37550]: Updating smithi016:/etc/ceph/ceph.conf 2024-04-02T04:35:23.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:22 smithi027 ceph-mon[37550]: Updating smithi027:/etc/ceph/ceph.conf 2024-04-02T04:35:23.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:22 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/3972007146' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:35:23.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:22 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:23.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:22 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:23.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:22 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:23.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:22 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:23.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:22 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:23.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:22 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T04:35:23.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:22 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T04:35:23.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:22 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:35:23.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:22 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:23.152 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-02T04:35:23.152 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T04:35:23.152 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-02T04:35:23.152 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T04:35:23.152 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T04:35:23.152 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-04-02 04:34:34.075676892 +0000 2024-04-02T04:35:23.153 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-04-02 04:28:50.910024181 +0000 2024-04-02T04:35:23.153 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-04-02 04:28:50.910024181 +0000 2024-04-02T04:35:23.153 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-04-02 04:28:50.910024181 +0000 2024-04-02T04:35:23.153 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-02T04:35:23.218 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-04-02T04:35:23.218 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-04-02T04:35:23.218 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 9.7002e-05 s, 5.3 MB/s 2024-04-02T04:35:23.219 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-02T04:35:23.275 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_3 2024-04-02T04:35:23.329 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-02T04:35:23.329 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T04:35:23.329 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-02T04:35:23.329 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T04:35:23.329 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T04:35:23.329 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-04-02 04:34:34.075676892 +0000 2024-04-02T04:35:23.330 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-04-02 04:28:51.247019404 +0000 2024-04-02T04:35:23.330 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-04-02 04:28:51.247019404 +0000 2024-04-02T04:35:23.330 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-04-02 04:28:51.247019404 +0000 2024-04-02T04:35:23.330 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-02T04:35:23.395 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-04-02T04:35:23.395 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-04-02T04:35:23.395 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000103613 s, 4.9 MB/s 2024-04-02T04:35:23.396 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-02T04:35:23.452 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_4 2024-04-02T04:35:23.507 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-02T04:35:23.507 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T04:35:23.507 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-02T04:35:23.507 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T04:35:23.507 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T04:35:23.507 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-04-02 04:34:34.076676876 +0000 2024-04-02T04:35:23.508 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-04-02 04:28:51.576014741 +0000 2024-04-02T04:35:23.508 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-04-02 04:28:51.576014741 +0000 2024-04-02T04:35:23.508 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-04-02 04:28:51.576014741 +0000 2024-04-02T04:35:23.508 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-02T04:35:23.573 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-04-02T04:35:23.573 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-04-02T04:35:23.573 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 9.5919e-05 s, 5.3 MB/s 2024-04-02T04:35:23.575 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-02T04:35:23.629 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-02T04:35:23.629 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/scratch_devs of=/dev/stdout 2024-04-02T04:35:23.649 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-02T04:35:23.649 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_1 2024-04-02T04:35:23.708 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-02T04:35:23.708 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T04:35:23.708 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-04-02T04:35:23.708 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T04:35:23.708 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T04:35:23.708 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-04-02 04:35:12.568158274 +0000 2024-04-02T04:35:23.709 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-04-02 04:28:44.870289125 +0000 2024-04-02T04:35:23.709 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-04-02 04:28:44.870289125 +0000 2024-04-02T04:35:23.709 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-04-02 04:28:44.870289125 +0000 2024-04-02T04:35:23.709 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-02T04:35:23.782 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-04-02T04:35:23.783 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-04-02T04:35:23.783 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000227124 s, 2.3 MB/s 2024-04-02T04:35:23.784 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-02T04:35:23.843 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_2 2024-04-02T04:35:23.856 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:23 smithi016 ceph-mon[28969]: Updating smithi016:/var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:35:23.857 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:23 smithi016 ceph-mon[28969]: Updating smithi027:/var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:35:23.857 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:23 smithi016 ceph-mon[28969]: Reconfiguring mon.smithi016 (unknown last config time)... 2024-04-02T04:35:23.857 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:23 smithi016 ceph-mon[28969]: Reconfiguring daemon mon.smithi016 on smithi016 2024-04-02T04:35:23.857 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:23 smithi016 ceph-mon[28969]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:23.900 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-02T04:35:23.900 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T04:35:23.900 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-02T04:35:23.900 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T04:35:23.900 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T04:35:23.900 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-04-02 04:35:12.568158274 +0000 2024-04-02T04:35:23.901 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-04-02 04:28:45.246279286 +0000 2024-04-02T04:35:23.901 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-04-02 04:28:45.246279286 +0000 2024-04-02T04:35:23.901 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-04-02 04:28:45.246279286 +0000 2024-04-02T04:35:23.901 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-02T04:35:23.972 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-04-02T04:35:23.972 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-04-02T04:35:23.972 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000217515 s, 2.4 MB/s 2024-04-02T04:35:23.973 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-02T04:35:24.031 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_3 2024-04-02T04:35:24.087 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-02T04:35:24.088 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T04:35:24.088 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-04-02T04:35:24.088 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T04:35:24.088 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T04:35:24.088 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-04-02 04:35:12.568158274 +0000 2024-04-02T04:35:24.088 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-04-02 04:28:45.598270075 +0000 2024-04-02T04:35:24.088 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-04-02 04:28:45.598270075 +0000 2024-04-02T04:35:24.088 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-04-02 04:28:45.598270075 +0000 2024-04-02T04:35:24.088 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-02T04:35:24.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:23 smithi027 ceph-mon[37550]: Updating smithi016:/var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:35:24.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:23 smithi027 ceph-mon[37550]: Updating smithi027:/var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:35:24.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:23 smithi027 ceph-mon[37550]: Reconfiguring mon.smithi016 (unknown last config time)... 2024-04-02T04:35:24.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:23 smithi027 ceph-mon[37550]: Reconfiguring daemon mon.smithi016 on smithi016 2024-04-02T04:35:24.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:23 smithi027 ceph-mon[37550]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:24.158 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-04-02T04:35:24.158 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-04-02T04:35:24.158 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000215914 s, 2.4 MB/s 2024-04-02T04:35:24.159 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-02T04:35:24.216 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_4 2024-04-02T04:35:24.271 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-02T04:35:24.272 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T04:35:24.272 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-02T04:35:24.272 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T04:35:24.272 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T04:35:24.272 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-04-02 04:35:12.569158248 +0000 2024-04-02T04:35:24.272 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-04-02 04:28:45.947260943 +0000 2024-04-02T04:35:24.272 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-04-02 04:28:45.947260943 +0000 2024-04-02T04:35:24.272 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-04-02 04:28:45.947260943 +0000 2024-04-02T04:35:24.272 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-02T04:35:24.342 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-04-02T04:35:24.342 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-04-02T04:35:24.342 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000200518 s, 2.6 MB/s 2024-04-02T04:35:24.344 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-02T04:35:24.401 INFO:tasks.cephadm:Deploying osd.0 on smithi016 with /dev/vg_nvme/lv_4... 2024-04-02T04:35:24.401 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-02T04:35:25.233 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:24 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:25.233 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:24 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:25.234 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:24 smithi016 ceph-mon[28969]: Reconfiguring mgr.smithi016.eqnebx (unknown last config time)... 2024-04-02T04:35:25.234 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:24 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi016.eqnebx", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T04:35:25.234 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:24 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T04:35:25.234 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:24 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:35:25.234 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:24 smithi016 ceph-mon[28969]: Reconfiguring daemon mgr.smithi016.eqnebx on smithi016 2024-04-02T04:35:25.234 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:24 smithi016 ceph-mon[28969]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:25.386 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:35:25.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:24 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:25.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:24 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:25.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:24 smithi027 ceph-mon[37550]: Reconfiguring mgr.smithi016.eqnebx (unknown last config time)... 2024-04-02T04:35:25.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:24 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi016.eqnebx", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T04:35:25.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:24 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T04:35:25.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:24 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:35:25.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:24 smithi027 ceph-mon[37550]: Reconfiguring daemon mgr.smithi016.eqnebx on smithi016 2024-04-02T04:35:25.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:24 smithi027 ceph-mon[37550]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:26.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:26 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:26.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:26 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:26.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:26 smithi027 ceph-mon[37550]: Reconfiguring ceph-exporter.smithi016 (monmap changed)... 2024-04-02T04:35:26.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:26 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi016", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-02T04:35:26.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:26 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:35:26.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:26 smithi027 ceph-mon[37550]: Reconfiguring daemon ceph-exporter.smithi016 on smithi016 2024-04-02T04:35:26.663 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:26 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:26.663 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:26 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:26.663 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:26 smithi016 ceph-mon[28969]: Reconfiguring ceph-exporter.smithi016 (monmap changed)... 2024-04-02T04:35:26.663 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:26 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi016", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-02T04:35:26.663 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:26 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:35:26.663 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:26 smithi016 ceph-mon[28969]: Reconfiguring daemon ceph-exporter.smithi016 on smithi016 2024-04-02T04:35:27.552 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:27 smithi016 ceph-mon[28969]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:27.645 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:27 smithi027 ceph-mon[37550]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:27.772 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-02T04:35:27.772 INFO:teuthology.orchestra.run.smithi016.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-02T04:35:27.772 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10+0 records in 2024-04-02T04:35:27.772 INFO:teuthology.orchestra.run.smithi016.stderr:10+0 records out 2024-04-02T04:35:27.772 INFO:teuthology.orchestra.run.smithi016.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00888641 s, 1.2 GB/s 2024-04-02T04:35:27.772 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping successful for: 2024-04-02T04:35:28.374 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:28 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:35:28.375 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:28 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:28.375 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:28 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:28.375 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:28 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi016", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-02T04:35:28.375 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:28 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:35:28.397 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi016:vg_nvme/lv_4 2024-04-02T04:35:28.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:28 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:35:28.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:28 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:28.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:28 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:28.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:28 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi016", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-02T04:35:28.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:28 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:35:29.470 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:35:29.720 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:29 smithi016 ceph-mon[28969]: Reconfiguring crash.smithi016 (monmap changed)... 2024-04-02T04:35:29.720 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:29 smithi016 ceph-mon[28969]: Reconfiguring daemon crash.smithi016 on smithi016 2024-04-02T04:35:29.720 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:29 smithi016 ceph-mon[28969]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:29.720 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:29 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:29.720 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:29 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:30.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:29 smithi027 ceph-mon[37550]: Reconfiguring crash.smithi016 (monmap changed)... 2024-04-02T04:35:30.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:29 smithi027 ceph-mon[37550]: Reconfiguring daemon crash.smithi016 on smithi016 2024-04-02T04:35:30.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:29 smithi027 ceph-mon[37550]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:30.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:29 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:30.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:29 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:30.843 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:30 smithi016 ceph-mon[28969]: Reconfiguring alertmanager.smithi016 (dependencies changed)... 2024-04-02T04:35:30.843 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:30 smithi016 ceph-mon[28969]: Reconfiguring daemon alertmanager.smithi016 on smithi016 2024-04-02T04:35:30.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:30.941+0000 7f84e4c2e700 1 -- 172.21.15.16:0/82855874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e01015c0 msgr2=0x7f84e01039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:35:30.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:30.941+0000 7f84e4c2e700 1 --2- 172.21.15.16:0/82855874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e01015c0 0x7f84e01039b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f84d0009910 tx=0x7f84d0009c20 comp rx=0 tx=0).stop 2024-04-02T04:35:30.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:30.941+0000 7f84e4c2e700 1 -- 172.21.15.16:0/82855874 shutdown_connections 2024-04-02T04:35:30.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:30.941+0000 7f84e4c2e700 1 --2- 172.21.15.16:0/82855874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84e0103ef0 0x7f84e01062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:35:30.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:30.941+0000 7f84e4c2e700 1 --2- 172.21.15.16:0/82855874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e01015c0 0x7f84e01039b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:35:30.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:30.941+0000 7f84e4c2e700 1 -- 172.21.15.16:0/82855874 >> 172.21.15.16:0/82855874 conn(0x7f84e00faf70 msgr2=0x7f84e00fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:35:30.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:30.941+0000 7f84e4c2e700 1 -- 172.21.15.16:0/82855874 shutdown_connections 2024-04-02T04:35:30.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:30.941+0000 7f84e4c2e700 1 -- 172.21.15.16:0/82855874 wait complete. 2024-04-02T04:35:30.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:30.942+0000 7f84e4c2e700 1 Processor -- start 2024-04-02T04:35:30.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:30.942+0000 7f84e4c2e700 1 -- start start 2024-04-02T04:35:30.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:30.942+0000 7f84e4c2e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e01015c0 0x7f84e0196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:35:30.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:30.942+0000 7f84e4c2e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84e0103ef0 0x7f84e0196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:35:30.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:30.942+0000 7f84e4c2e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e0197180 con 0x7f84e01015c0 2024-04-02T04:35:30.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:30.942+0000 7f84e4c2e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e01972c0 con 0x7f84e0103ef0 2024-04-02T04:35:30.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:30.942+0000 7f84de59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e01015c0 0x7f84e0196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:35:30.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:30.942+0000 7f84de59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e01015c0 0x7f84e0196620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54898/0 (socket says 172.21.15.16:54898) 2024-04-02T04:35:30.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:30.942+0000 7f84de59c700 1 -- 172.21.15.16:0/2942147684 learned_addr learned my addr 172.21.15.16:0/2942147684 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:35:30.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:30.942+0000 7f84ddd9b700 1 --2- 172.21.15.16:0/2942147684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84e0103ef0 0x7f84e0196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:35:30.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:30.942+0000 7f84de59c700 1 -- 172.21.15.16:0/2942147684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84e0103ef0 msgr2=0x7f84e0196b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:35:30.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:30.942+0000 7f84de59c700 1 --2- 172.21.15.16:0/2942147684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84e0103ef0 0x7f84e0196b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:35:30.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:30.942+0000 7f84de59c700 1 -- 172.21.15.16:0/2942147684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d00095f0 con 0x7f84e01015c0 2024-04-02T04:35:30.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:30.942+0000 7f84ddd9b700 1 --2- 172.21.15.16:0/2942147684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84e0103ef0 0x7f84e0196b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T04:35:30.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:30.943+0000 7f84de59c700 1 --2- 172.21.15.16:0/2942147684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e01015c0 0x7f84e0196620 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f84d00092e0 tx=0x7f84d000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:35:30.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:30.943+0000 7f84cf7fe700 1 -- 172.21.15.16:0/2942147684 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84d001d070 con 0x7f84e01015c0 2024-04-02T04:35:30.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:30.943+0000 7f84cf7fe700 1 -- 172.21.15.16:0/2942147684 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f84d000f460 con 0x7f84e01015c0 2024-04-02T04:35:30.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:30.943+0000 7f84e4c2e700 1 -- 172.21.15.16:0/2942147684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84e019bc70 con 0x7f84e01015c0 2024-04-02T04:35:30.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:30.943+0000 7f84cf7fe700 1 -- 172.21.15.16:0/2942147684 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84d0021640 con 0x7f84e01015c0 2024-04-02T04:35:30.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:30.943+0000 7f84e4c2e700 1 -- 172.21.15.16:0/2942147684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e019c160 con 0x7f84e01015c0 2024-04-02T04:35:30.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:30.944+0000 7f84e4c2e700 1 -- 172.21.15.16:0/2942147684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84e004ea80 con 0x7f84e01015c0 2024-04-02T04:35:30.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:30.944+0000 7f84cf7fe700 1 -- 172.21.15.16:0/2942147684 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 90786+0+0 (secure 0 0 0) 0x7f84d000f5d0 con 0x7f84e01015c0 2024-04-02T04:35:30.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:30.945+0000 7f84cf7fe700 1 --2- 172.21.15.16:0/2942147684 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f84c806cc60 0x7f84c806f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:35:30.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:30.945+0000 7f84cf7fe700 1 -- 172.21.15.16:0/2942147684 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f84d008c510 con 0x7f84e01015c0 2024-04-02T04:35:30.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:30.945+0000 7f84ddd9b700 1 --2- 172.21.15.16:0/2942147684 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f84c806cc60 0x7f84c806f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:35:30.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:30.945+0000 7f84ddd9b700 1 --2- 172.21.15.16:0/2942147684 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f84c806cc60 0x7f84c806f120 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f84d40099d0 tx=0x7f84d4008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:35:30.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:30.949+0000 7f84cf7fe700 1 -- 172.21.15.16:0/2942147684 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f84d005b450 con 0x7f84e01015c0 2024-04-02T04:35:31.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:30 smithi027 ceph-mon[37550]: Reconfiguring alertmanager.smithi016 (dependencies changed)... 2024-04-02T04:35:31.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:30 smithi027 ceph-mon[37550]: Reconfiguring daemon alertmanager.smithi016 on smithi016 2024-04-02T04:35:31.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:35:31.166+0000 7f84e4c2e700 1 -- 172.21.15.16:0/2942147684 --> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f84e0104970 con 0x7f84c806cc60 2024-04-02T04:35:32.049 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:31 smithi016 ceph-mon[28969]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:32.049 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:31 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T04:35:32.049 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:31 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T04:35:32.049 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:31 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:35:32.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:31 smithi027 ceph-mon[37550]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:32.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:31 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T04:35:32.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:31 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T04:35:32.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:31 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:35:33.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:32 smithi016 ceph-mon[28969]: from='client.14252 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:35:33.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:32 smithi027 ceph-mon[37550]: from='client.14252 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:35:33.801 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:33 smithi016 ceph-mon[28969]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:34.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:33 smithi027 ceph-mon[37550]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:36.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:35 smithi016 ceph-mon[28969]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:36.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:35 smithi027 ceph-mon[37550]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:37.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:37 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:37.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:37 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:37.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:37 smithi016 ceph-mon[28969]: Reconfiguring grafana.smithi016 (dependencies changed)... 2024-04-02T04:35:37.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:37 smithi016 ceph-mon[28969]: Reconfiguring daemon grafana.smithi016 on smithi016 2024-04-02T04:35:37.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:37 smithi016 ceph-mon[28969]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:37.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:37 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:37.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:37 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:37.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:37 smithi027 ceph-mon[37550]: Reconfiguring grafana.smithi016 (dependencies changed)... 2024-04-02T04:35:37.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:37 smithi027 ceph-mon[37550]: Reconfiguring daemon grafana.smithi016 on smithi016 2024-04-02T04:35:37.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:37 smithi027 ceph-mon[37550]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:38.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:38 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/3671169394' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b7be29b-0e2a-456b-b33e-c72abaad9136"}]: dispatch 2024-04-02T04:35:38.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:38 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/3671169394' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2b7be29b-0e2a-456b-b33e-c72abaad9136"}]': finished 2024-04-02T04:35:38.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:38 smithi016 ceph-mon[28969]: osdmap e6: 1 total, 0 up, 1 in 2024-04-02T04:35:38.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:38 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T04:35:38.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:38 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/3133435902' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T04:35:38.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:38 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/3671169394' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b7be29b-0e2a-456b-b33e-c72abaad9136"}]: dispatch 2024-04-02T04:35:38.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:38 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/3671169394' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2b7be29b-0e2a-456b-b33e-c72abaad9136"}]': finished 2024-04-02T04:35:38.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:38 smithi027 ceph-mon[37550]: osdmap e6: 1 total, 0 up, 1 in 2024-04-02T04:35:38.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:38 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T04:35:38.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:38 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/3133435902' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T04:35:40.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:39 smithi016 ceph-mon[28969]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:40.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:39 smithi027 ceph-mon[37550]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:42.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:41 smithi016 ceph-mon[28969]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:42.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:41 smithi027 ceph-mon[37550]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:43.047 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:42 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:35:43.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:42 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:35:44.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:43 smithi016 ceph-mon[28969]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:44.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:43 smithi027 ceph-mon[37550]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:46.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:45 smithi016 ceph-mon[28969]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:46.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:45 smithi027 ceph-mon[37550]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:47.809 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:47 smithi016 ceph-mon[28969]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:47.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:47 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:47.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:47 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:47.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:47 smithi016 ceph-mon[28969]: Reconfiguring prometheus.smithi016 (dependencies changed)... 2024-04-02T04:35:48.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:47 smithi027 ceph-mon[37550]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:48.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:47 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:48.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:47 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:48.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:47 smithi027 ceph-mon[37550]: Reconfiguring prometheus.smithi016 (dependencies changed)... 2024-04-02T04:35:49.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:48 smithi016 ceph-mon[28969]: Reconfiguring daemon prometheus.smithi016 on smithi016 2024-04-02T04:35:49.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:48 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-02T04:35:49.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:48 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:35:49.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:48 smithi027 ceph-mon[37550]: Reconfiguring daemon prometheus.smithi016 on smithi016 2024-04-02T04:35:49.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:48 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-02T04:35:49.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:48 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:35:49.937 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:49 smithi016 ceph-mon[28969]: Deploying daemon osd.0 on smithi016 2024-04-02T04:35:49.937 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:49 smithi016 ceph-mon[28969]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:50.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:49 smithi027 ceph-mon[37550]: Deploying daemon osd.0 on smithi016 2024-04-02T04:35:50.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:49 smithi027 ceph-mon[37550]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:52.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:51 smithi016 ceph-mon[28969]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:52.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:51 smithi027 ceph-mon[37550]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:53.400 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:53 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:53.400 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:53 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:53.400 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:53 smithi027 ceph-mon[37550]: Reconfiguring ceph-exporter.smithi027 (monmap changed)... 2024-04-02T04:35:53.400 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:53 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi027", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-02T04:35:53.400 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:53 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:35:53.400 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:53 smithi027 ceph-mon[37550]: Reconfiguring daemon ceph-exporter.smithi027 on smithi027 2024-04-02T04:35:53.400 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:53 smithi027 ceph-mon[37550]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:53.555 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:53 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:53.555 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:53 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:53.555 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:53 smithi016 ceph-mon[28969]: Reconfiguring ceph-exporter.smithi027 (monmap changed)... 2024-04-02T04:35:53.555 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:53 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi027", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-02T04:35:53.555 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:53 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:35:53.555 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:53 smithi016 ceph-mon[28969]: Reconfiguring daemon ceph-exporter.smithi027 on smithi027 2024-04-02T04:35:53.555 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:53 smithi016 ceph-mon[28969]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:55.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:54 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:55.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:54 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:55.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:54 smithi027 ceph-mon[37550]: Reconfiguring crash.smithi027 (monmap changed)... 2024-04-02T04:35:55.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:54 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi027", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-02T04:35:55.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:54 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:35:55.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:54 smithi027 ceph-mon[37550]: Reconfiguring daemon crash.smithi027 on smithi027 2024-04-02T04:35:55.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:54 smithi027 ceph-mon[37550]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:55.153 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:54 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:55.154 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:54 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:55.154 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:54 smithi016 ceph-mon[28969]: Reconfiguring crash.smithi027 (monmap changed)... 2024-04-02T04:35:55.154 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:54 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi027", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-02T04:35:55.154 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:54 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:35:55.154 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:54 smithi016 ceph-mon[28969]: Reconfiguring daemon crash.smithi027 on smithi027 2024-04-02T04:35:55.154 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:54 smithi016 ceph-mon[28969]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:56.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:56 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:56.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:56 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:56.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:56 smithi027 ceph-mon[37550]: Reconfiguring mgr.smithi027.kjzcsc (monmap changed)... 2024-04-02T04:35:56.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:56 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi027.kjzcsc", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T04:35:56.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:56 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T04:35:56.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:56 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:35:56.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:56 smithi027 ceph-mon[37550]: Reconfiguring daemon mgr.smithi027.kjzcsc on smithi027 2024-04-02T04:35:56.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:56 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:56.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:56 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:56.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:56 smithi016 ceph-mon[28969]: Reconfiguring mgr.smithi027.kjzcsc (monmap changed)... 2024-04-02T04:35:56.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:56 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi027.kjzcsc", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T04:35:56.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:56 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T04:35:56.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:56 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:35:56.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:56 smithi016 ceph-mon[28969]: Reconfiguring daemon mgr.smithi027.kjzcsc on smithi027 2024-04-02T04:35:57.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:57 smithi027 ceph-mon[37550]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:57.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:57 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:57.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:57 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:57.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:57 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T04:35:57.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:57 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T04:35:57.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:57 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:35:57.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:57 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:57.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:57 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:57.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:57 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:35:58.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:57 smithi016 ceph-mon[28969]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:35:58.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:57 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:58.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:57 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:58.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:57 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T04:35:58.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:57 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T04:35:58.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:57 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:35:58.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:57 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:58.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:57 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:58.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:57 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:35:59.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:58 smithi016 ceph-mon[28969]: Reconfiguring mon.smithi027 (monmap changed)... 2024-04-02T04:35:59.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:58 smithi016 ceph-mon[28969]: Reconfiguring daemon mon.smithi027 on smithi027 2024-04-02T04:35:59.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:58 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:59.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:58 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:59.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:58 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-02T04:35:59.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:58 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi016.front.sepia.ceph.com:9093"}]: dispatch 2024-04-02T04:35:59.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:58 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:59.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:58 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-02T04:35:59.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:58 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi016.front.sepia.ceph.com:3000"}]: dispatch 2024-04-02T04:35:59.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:58 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:59.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:58 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-02T04:35:59.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:58 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi016.front.sepia.ceph.com:9095"}]: dispatch 2024-04-02T04:35:59.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:58 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:59.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:58 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:35:59.120 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:58 smithi027 ceph-mon[37550]: Reconfiguring mon.smithi027 (monmap changed)... 2024-04-02T04:35:59.121 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:58 smithi027 ceph-mon[37550]: Reconfiguring daemon mon.smithi027 on smithi027 2024-04-02T04:35:59.121 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:58 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:59.121 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:58 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:59.121 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:58 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-02T04:35:59.121 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:58 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi016.front.sepia.ceph.com:9093"}]: dispatch 2024-04-02T04:35:59.121 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:58 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:59.121 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:58 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-02T04:35:59.121 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:58 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi016.front.sepia.ceph.com:3000"}]: dispatch 2024-04-02T04:35:59.121 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:58 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:59.121 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:58 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-02T04:35:59.121 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:58 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi016.front.sepia.ceph.com:9095"}]: dispatch 2024-04-02T04:35:59.121 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:58 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:35:59.121 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:58 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:36:00.030 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:59 smithi016 ceph-mon[28969]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-02T04:36:00.031 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:59 smithi016 ceph-mon[28969]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi016.front.sepia.ceph.com:9093"}]: dispatch 2024-04-02T04:36:00.031 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:59 smithi016 ceph-mon[28969]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-02T04:36:00.031 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:59 smithi016 ceph-mon[28969]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi016.front.sepia.ceph.com:3000"}]: dispatch 2024-04-02T04:36:00.031 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:59 smithi016 ceph-mon[28969]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-02T04:36:00.031 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:59 smithi016 ceph-mon[28969]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi016.front.sepia.ceph.com:9095"}]: dispatch 2024-04-02T04:36:00.031 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:35:59 smithi016 ceph-mon[28969]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:36:00.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:59 smithi027 ceph-mon[37550]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-02T04:36:00.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:59 smithi027 ceph-mon[37550]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi016.front.sepia.ceph.com:9093"}]: dispatch 2024-04-02T04:36:00.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:59 smithi027 ceph-mon[37550]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-02T04:36:00.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:59 smithi027 ceph-mon[37550]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi016.front.sepia.ceph.com:3000"}]: dispatch 2024-04-02T04:36:00.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:59 smithi027 ceph-mon[37550]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-02T04:36:00.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:59 smithi027 ceph-mon[37550]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi016.front.sepia.ceph.com:9095"}]: dispatch 2024-04-02T04:36:00.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:35:59 smithi027 ceph-mon[37550]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:36:00.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:00.358+0000 7f84cf7fe700 1 -- 172.21.15.16:0/2942147684 <== mgr.14217 v2:172.21.15.16:6800/3643600900 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f84e0104970 con 0x7f84c806cc60 2024-04-02T04:36:00.361 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 0 on host 'smithi016' 2024-04-02T04:36:00.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:00.360+0000 7f84e4c2e700 1 -- 172.21.15.16:0/2942147684 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f84c806cc60 msgr2=0x7f84c806f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:36:00.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:00.360+0000 7f84e4c2e700 1 --2- 172.21.15.16:0/2942147684 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f84c806cc60 0x7f84c806f120 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f84d40099d0 tx=0x7f84d4008040 comp rx=0 tx=0).stop 2024-04-02T04:36:00.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:00.360+0000 7f84e4c2e700 1 -- 172.21.15.16:0/2942147684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e01015c0 msgr2=0x7f84e0196620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:36:00.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:00.360+0000 7f84e4c2e700 1 --2- 172.21.15.16:0/2942147684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e01015c0 0x7f84e0196620 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f84d00092e0 tx=0x7f84d000be20 comp rx=0 tx=0).stop 2024-04-02T04:36:00.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:00.360+0000 7f84e4c2e700 1 -- 172.21.15.16:0/2942147684 shutdown_connections 2024-04-02T04:36:00.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:00.360+0000 7f84e4c2e700 1 --2- 172.21.15.16:0/2942147684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e01015c0 0x7f84e0196620 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:36:00.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:00.360+0000 7f84e4c2e700 1 --2- 172.21.15.16:0/2942147684 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f84c806cc60 0x7f84c806f120 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:36:00.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:00.360+0000 7f84e4c2e700 1 --2- 172.21.15.16:0/2942147684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84e0103ef0 0x7f84e0196b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:36:00.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:00.360+0000 7f84e4c2e700 1 -- 172.21.15.16:0/2942147684 >> 172.21.15.16:0/2942147684 conn(0x7f84e00faf70 msgr2=0x7f84e00fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:36:00.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:00.360+0000 7f84e4c2e700 1 -- 172.21.15.16:0/2942147684 shutdown_connections 2024-04-02T04:36:00.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:00.360+0000 7f84e4c2e700 1 -- 172.21.15.16:0/2942147684 wait complete. 2024-04-02T04:36:01.262 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:01 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:01.262 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:01 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:01.262 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:01 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:01.262 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:01 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:01.262 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:01 smithi016 ceph-mon[28969]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:36:01.262 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:01 smithi016 ceph-mon[28969]: from='osd.0 [v2:172.21.15.16:6802/906882291,v1:172.21.15.16:6803/906882291]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-02T04:36:01.282 DEBUG:teuthology.orchestra.run.smithi016:osd.0> sudo journalctl -f -n 0 -u ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@osd.0.service 2024-04-02T04:36:01.285 INFO:tasks.cephadm:Deploying osd.1 on smithi016 with /dev/vg_nvme/lv_3... 2024-04-02T04:36:01.285 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-02T04:36:01.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:01 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:01.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:01 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:01.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:01 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:01.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:01 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:01.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:01 smithi027 ceph-mon[37550]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:36:01.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:01 smithi027 ceph-mon[37550]: from='osd.0 [v2:172.21.15.16:6802/906882291,v1:172.21.15.16:6803/906882291]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-02T04:36:02.513 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:36:02.601 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:02 smithi016 ceph-mon[28969]: from='osd.0 [v2:172.21.15.16:6802/906882291,v1:172.21.15.16:6803/906882291]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-02T04:36:02.601 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:02 smithi016 ceph-mon[28969]: osdmap e7: 1 total, 0 up, 1 in 2024-04-02T04:36:02.601 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:02 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T04:36:02.601 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:02 smithi016 ceph-mon[28969]: from='osd.0 [v2:172.21.15.16:6802/906882291,v1:172.21.15.16:6803/906882291]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-04-02T04:36:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:02 smithi027 ceph-mon[37550]: from='osd.0 [v2:172.21.15.16:6802/906882291,v1:172.21.15.16:6803/906882291]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-02T04:36:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:02 smithi027 ceph-mon[37550]: osdmap e7: 1 total, 0 up, 1 in 2024-04-02T04:36:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:02 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T04:36:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:02 smithi027 ceph-mon[37550]: from='osd.0 [v2:172.21.15.16:6802/906882291,v1:172.21.15.16:6803/906882291]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-04-02T04:36:03.310 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:36:03 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0[47723]: 2024-04-02T04:36:03.072+0000 7f5b45c27700 -1 osd.0 0 waiting for initial osdmap 2024-04-02T04:36:03.310 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:36:03 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0[47723]: 2024-04-02T04:36:03.076+0000 7f5b3e215700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T04:36:03.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:03 smithi027 ceph-mon[37550]: from='osd.0 [v2:172.21.15.16:6802/906882291,v1:172.21.15.16:6803/906882291]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-04-02T04:36:03.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:03 smithi027 ceph-mon[37550]: osdmap e8: 1 total, 0 up, 1 in 2024-04-02T04:36:03.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:03 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T04:36:03.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:03 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T04:36:03.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:03 smithi027 ceph-mon[37550]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:36:03.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:03 smithi027 ceph-mon[37550]: from='osd.0 [v2:172.21.15.16:6802/906882291,v1:172.21.15.16:6803/906882291]' entity='osd.0' 2024-04-02T04:36:03.677 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:03 smithi016 ceph-mon[28969]: from='osd.0 [v2:172.21.15.16:6802/906882291,v1:172.21.15.16:6803/906882291]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-04-02T04:36:03.677 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:03 smithi016 ceph-mon[28969]: osdmap e8: 1 total, 0 up, 1 in 2024-04-02T04:36:03.677 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:03 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T04:36:03.677 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:03 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T04:36:03.677 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:03 smithi016 ceph-mon[28969]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T04:36:03.677 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:03 smithi016 ceph-mon[28969]: from='osd.0 [v2:172.21.15.16:6802/906882291,v1:172.21.15.16:6803/906882291]' entity='osd.0' 2024-04-02T04:36:04.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:04 smithi016 ceph-mon[28969]: purged_snaps scrub starts 2024-04-02T04:36:04.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:04 smithi016 ceph-mon[28969]: purged_snaps scrub ok 2024-04-02T04:36:04.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:04 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T04:36:04.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:04 smithi016 ceph-mon[28969]: osd.0 [v2:172.21.15.16:6802/906882291,v1:172.21.15.16:6803/906882291] boot 2024-04-02T04:36:04.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:04 smithi016 ceph-mon[28969]: osdmap e9: 1 total, 1 up, 1 in 2024-04-02T04:36:04.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:04 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T04:36:04.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:04 smithi027 ceph-mon[37550]: purged_snaps scrub starts 2024-04-02T04:36:04.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:04 smithi027 ceph-mon[37550]: purged_snaps scrub ok 2024-04-02T04:36:04.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:04 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T04:36:04.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:04 smithi027 ceph-mon[37550]: osd.0 [v2:172.21.15.16:6802/906882291,v1:172.21.15.16:6803/906882291] boot 2024-04-02T04:36:04.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:04 smithi027 ceph-mon[37550]: osdmap e9: 1 total, 1 up, 1 in 2024-04-02T04:36:04.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:04 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T04:36:05.199 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-02T04:36:05.200 INFO:teuthology.orchestra.run.smithi016.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-02T04:36:05.200 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10+0 records in 2024-04-02T04:36:05.200 INFO:teuthology.orchestra.run.smithi016.stderr:10+0 records out 2024-04-02T04:36:05.200 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0102211 s, 1.0 GB/s 2024-04-02T04:36:05.200 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping successful for: 2024-04-02T04:36:05.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:05 smithi016 ceph-mon[28969]: pgmap v38: 0 pgs: ; 0 B data, 38 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:36:05.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:05 smithi027 ceph-mon[37550]: pgmap v38: 0 pgs: ; 0 B data, 38 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:36:05.915 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi016:vg_nvme/lv_3 2024-04-02T04:36:06.126 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:36:06.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:06 smithi016 ceph-mon[28969]: osdmap e10: 1 total, 1 up, 1 in 2024-04-02T04:36:06.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:06 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:06.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:06 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:06.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:06 smithi027 ceph-mon[37550]: osdmap e10: 1 total, 1 up, 1 in 2024-04-02T04:36:06.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:06 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:06.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:06 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:07.429 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:07 smithi016 ceph-mon[28969]: pgmap v40: 0 pgs: ; 0 B data, 38 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:36:07.645 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:07 smithi027 ceph-mon[37550]: pgmap v40: 0 pgs: ; 0 B data, 38 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:36:07.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:07.846+0000 7f88a92fb700 1 -- 172.21.15.16:0/926808735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88a41015c0 msgr2=0x7f88a41039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:36:07.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:07.846+0000 7f88a92fb700 1 --2- 172.21.15.16:0/926808735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88a41015c0 0x7f88a41039b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f889c00b0a0 tx=0x7f889c00b3b0 comp rx=0 tx=0).stop 2024-04-02T04:36:07.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:07.846+0000 7f88a92fb700 1 -- 172.21.15.16:0/926808735 shutdown_connections 2024-04-02T04:36:07.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:07.846+0000 7f88a92fb700 1 --2- 172.21.15.16:0/926808735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88a4103ef0 0x7f88a41062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:36:07.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:07.846+0000 7f88a92fb700 1 --2- 172.21.15.16:0/926808735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88a41015c0 0x7f88a41039b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:36:07.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:07.846+0000 7f88a92fb700 1 -- 172.21.15.16:0/926808735 >> 172.21.15.16:0/926808735 conn(0x7f88a40faf70 msgr2=0x7f88a40fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:36:07.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:07.846+0000 7f88a92fb700 1 -- 172.21.15.16:0/926808735 shutdown_connections 2024-04-02T04:36:07.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:07.846+0000 7f88a92fb700 1 -- 172.21.15.16:0/926808735 wait complete. 2024-04-02T04:36:07.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:07.846+0000 7f88a92fb700 1 Processor -- start 2024-04-02T04:36:07.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:07.846+0000 7f88a92fb700 1 -- start start 2024-04-02T04:36:07.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:07.847+0000 7f88a92fb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88a41015c0 0x7f88a4196570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:36:07.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:07.847+0000 7f88a92fb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88a4103ef0 0x7f88a4196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:36:07.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:07.847+0000 7f88a92fb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a41970d0 con 0x7f88a41015c0 2024-04-02T04:36:07.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:07.847+0000 7f88a92fb700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a4197210 con 0x7f88a4103ef0 2024-04-02T04:36:07.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:07.847+0000 7f88a2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88a41015c0 0x7f88a4196570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:36:07.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:07.847+0000 7f88a2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88a41015c0 0x7f88a4196570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52386/0 (socket says 172.21.15.16:52386) 2024-04-02T04:36:07.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:07.847+0000 7f88a2ffd700 1 -- 172.21.15.16:0/2768566498 learned_addr learned my addr 172.21.15.16:0/2768566498 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:36:07.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:07.847+0000 7f88a2ffd700 1 -- 172.21.15.16:0/2768566498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88a4103ef0 msgr2=0x7f88a4196ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:36:07.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:07.847+0000 7f88a2ffd700 1 --2- 172.21.15.16:0/2768566498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88a4103ef0 0x7f88a4196ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:36:07.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:07.847+0000 7f88a2ffd700 1 -- 172.21.15.16:0/2768566498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f889c009d00 con 0x7f88a41015c0 2024-04-02T04:36:07.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:07.847+0000 7f88a2ffd700 1 --2- 172.21.15.16:0/2768566498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88a41015c0 0x7f88a4196570 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f889c006130 tx=0x7f889c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:36:07.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:07.847+0000 7f889bfff700 1 -- 172.21.15.16:0/2768566498 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f889c021070 con 0x7f88a41015c0 2024-04-02T04:36:07.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:07.847+0000 7f889bfff700 1 -- 172.21.15.16:0/2768566498 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f889c00ce60 con 0x7f88a41015c0 2024-04-02T04:36:07.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:07.847+0000 7f88a92fb700 1 -- 172.21.15.16:0/2768566498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88a419bc10 con 0x7f88a41015c0 2024-04-02T04:36:07.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:07.847+0000 7f889bfff700 1 -- 172.21.15.16:0/2768566498 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f889c00e040 con 0x7f88a41015c0 2024-04-02T04:36:07.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:07.847+0000 7f88a92fb700 1 -- 172.21.15.16:0/2768566498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88a419c100 con 0x7f88a41015c0 2024-04-02T04:36:07.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:07.848+0000 7f88a92fb700 1 -- 172.21.15.16:0/2768566498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88a404ea80 con 0x7f88a41015c0 2024-04-02T04:36:07.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:07.849+0000 7f889bfff700 1 -- 172.21.15.16:0/2768566498 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 90786+0+0 (secure 0 0 0) 0x7f889c004250 con 0x7f88a41015c0 2024-04-02T04:36:07.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:07.849+0000 7f889bfff700 1 --2- 172.21.15.16:0/2768566498 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f889006cc60 0x7f889006f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:36:07.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:07.849+0000 7f88a27fc700 1 --2- 172.21.15.16:0/2768566498 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f889006cc60 0x7f889006f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:36:07.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:07.849+0000 7f889bfff700 1 -- 172.21.15.16:0/2768566498 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1982+0+0 (secure 0 0 0) 0x7f889c08c7c0 con 0x7f88a41015c0 2024-04-02T04:36:07.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:07.850+0000 7f88a27fc700 1 --2- 172.21.15.16:0/2768566498 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f889006cc60 0x7f889006f120 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f888c005950 tx=0x7f888c0058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:36:07.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:07.854+0000 7f889bfff700 1 -- 172.21.15.16:0/2768566498 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f889c05b4a0 con 0x7f88a41015c0 2024-04-02T04:36:08.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:08.066+0000 7f88a92fb700 1 -- 172.21.15.16:0/2768566498 --> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f88a4105e80 con 0x7f889006cc60 2024-04-02T04:36:08.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:08 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T04:36:08.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:08 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T04:36:08.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:08 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:36:08.678 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:08 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T04:36:08.678 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:08 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T04:36:08.678 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:08 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:36:09.855 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:09 smithi016 ceph-mon[28969]: from='client.14270 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:36:09.855 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:09 smithi016 ceph-mon[28969]: pgmap v41: 0 pgs: ; 0 B data, 38 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:36:10.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:09 smithi027 ceph-mon[37550]: from='client.14270 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:36:10.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:09 smithi027 ceph-mon[37550]: pgmap v41: 0 pgs: ; 0 B data, 38 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:36:12.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:11 smithi016 ceph-mon[28969]: pgmap v42: 0 pgs: ; 0 B data, 38 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:36:12.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:11 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:12.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:11 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:12.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:11 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:36:12.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:11 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:12.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:11 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:36:12.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:11 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:36:12.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:11 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:12.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:11 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:36:12.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:11 smithi027 ceph-mon[37550]: pgmap v42: 0 pgs: ; 0 B data, 38 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:36:12.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:11 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:12.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:11 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:12.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:11 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:36:12.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:11 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:12.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:11 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:36:12.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:11 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:36:12.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:11 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:12.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:11 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:36:12.999 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:12 smithi016 ceph-mon[28969]: Detected new or changed devices on smithi016 2024-04-02T04:36:12.999 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:12 smithi016 ceph-mon[28969]: Adjusting osd_memory_target on smithi016 to 11845M 2024-04-02T04:36:12.999 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:12 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/4225509327' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "82578291-3c9c-4f3f-8de9-7a8710a4923e"}]: dispatch 2024-04-02T04:36:12.999 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:12 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/4225509327' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "82578291-3c9c-4f3f-8de9-7a8710a4923e"}]': finished 2024-04-02T04:36:12.999 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:12 smithi016 ceph-mon[28969]: osdmap e11: 2 total, 1 up, 2 in 2024-04-02T04:36:12.999 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:12 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T04:36:12.999 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:12 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:36:13.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:12 smithi027 ceph-mon[37550]: Detected new or changed devices on smithi016 2024-04-02T04:36:13.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:12 smithi027 ceph-mon[37550]: Adjusting osd_memory_target on smithi016 to 11845M 2024-04-02T04:36:13.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:12 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/4225509327' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "82578291-3c9c-4f3f-8de9-7a8710a4923e"}]: dispatch 2024-04-02T04:36:13.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:12 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/4225509327' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "82578291-3c9c-4f3f-8de9-7a8710a4923e"}]': finished 2024-04-02T04:36:13.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:12 smithi027 ceph-mon[37550]: osdmap e11: 2 total, 1 up, 2 in 2024-04-02T04:36:13.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:12 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T04:36:13.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:12 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:36:14.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:13 smithi016 ceph-mon[28969]: pgmap v44: 0 pgs: ; 0 B data, 38 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:36:14.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:13 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:14.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:13 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:14.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:13 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/2622110675' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T04:36:14.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:13 smithi027 ceph-mon[37550]: pgmap v44: 0 pgs: ; 0 B data, 38 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:36:14.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:13 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:14.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:13 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:14.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:13 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/2622110675' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T04:36:15.998 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:15 smithi016 ceph-mon[28969]: pgmap v45: 0 pgs: ; 0 B data, 38 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:36:15.998 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:15 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:15.998 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:15 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:15.998 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:15 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:36:15.998 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:15 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:36:15.998 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:15 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:16.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:15 smithi027 ceph-mon[37550]: pgmap v45: 0 pgs: ; 0 B data, 38 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:36:16.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:15 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:16.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:15 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:16.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:15 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:36:16.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:15 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:36:16.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:15 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:18.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:17 smithi016 ceph-mon[28969]: pgmap v46: 0 pgs: ; 0 B data, 38 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:36:18.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:17 smithi027 ceph-mon[37550]: pgmap v46: 0 pgs: ; 0 B data, 38 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:36:19.968 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:19 smithi016 ceph-mon[28969]: pgmap v47: 0 pgs: ; 0 B data, 38 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:36:19.968 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:19 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-02T04:36:19.968 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:19 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:36:20.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:19 smithi027 ceph-mon[37550]: pgmap v47: 0 pgs: ; 0 B data, 38 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:36:20.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:19 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-02T04:36:20.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:19 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:36:20.922 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:20 smithi016 ceph-mon[28969]: Deploying daemon osd.1 on smithi016 2024-04-02T04:36:21.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:20 smithi027 ceph-mon[37550]: Deploying daemon osd.1 on smithi016 2024-04-02T04:36:22.052 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:21 smithi016 ceph-mon[28969]: pgmap v48: 0 pgs: ; 0 B data, 38 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:36:22.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:21 smithi027 ceph-mon[37550]: pgmap v48: 0 pgs: ; 0 B data, 38 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:36:23.987 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:23 smithi016 ceph-mon[28969]: pgmap v49: 0 pgs: ; 0 B data, 38 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:36:24.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:23 smithi027 ceph-mon[37550]: pgmap v49: 0 pgs: ; 0 B data, 38 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:36:25.809 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:25 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:25.809 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:25 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:25.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:25 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:36:25.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:25 smithi016 ceph-mon[28969]: pgmap v50: 0 pgs: ; 0 B data, 38 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:36:25.863 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:25 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:25.864 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:25 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:25.864 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:25 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:36:25.864 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:25 smithi027 ceph-mon[37550]: pgmap v50: 0 pgs: ; 0 B data, 38 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:36:27.657 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:27 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:27.657 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:27 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:27.657 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:27 smithi016 ceph-mon[28969]: pgmap v51: 0 pgs: ; 0 B data, 38 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:36:27.723 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 1 on host 'smithi016' 2024-04-02T04:36:27.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:27.718+0000 7f889bfff700 1 -- 172.21.15.16:0/2768566498 <== mgr.14217 v2:172.21.15.16:6800/3643600900 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f88a4105e80 con 0x7f889006cc60 2024-04-02T04:36:27.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:27.721+0000 7f88a92fb700 1 -- 172.21.15.16:0/2768566498 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f889006cc60 msgr2=0x7f889006f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:36:27.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:27.721+0000 7f88a92fb700 1 --2- 172.21.15.16:0/2768566498 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f889006cc60 0x7f889006f120 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f888c005950 tx=0x7f888c0058e0 comp rx=0 tx=0).stop 2024-04-02T04:36:27.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:27.721+0000 7f88a92fb700 1 -- 172.21.15.16:0/2768566498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88a41015c0 msgr2=0x7f88a4196570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:36:27.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:27.721+0000 7f88a92fb700 1 --2- 172.21.15.16:0/2768566498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88a41015c0 0x7f88a4196570 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f889c006130 tx=0x7f889c003ce0 comp rx=0 tx=0).stop 2024-04-02T04:36:27.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:27.722+0000 7f88a92fb700 1 -- 172.21.15.16:0/2768566498 shutdown_connections 2024-04-02T04:36:27.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:27.722+0000 7f88a92fb700 1 --2- 172.21.15.16:0/2768566498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88a41015c0 0x7f88a4196570 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:36:27.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:27.722+0000 7f88a92fb700 1 --2- 172.21.15.16:0/2768566498 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f889006cc60 0x7f889006f120 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:36:27.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:27.722+0000 7f88a92fb700 1 --2- 172.21.15.16:0/2768566498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88a4103ef0 0x7f88a4196ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:36:27.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:27.722+0000 7f88a92fb700 1 -- 172.21.15.16:0/2768566498 >> 172.21.15.16:0/2768566498 conn(0x7f88a40faf70 msgr2=0x7f88a40fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:36:27.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:27.722+0000 7f88a92fb700 1 -- 172.21.15.16:0/2768566498 shutdown_connections 2024-04-02T04:36:27.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:27.722+0000 7f88a92fb700 1 -- 172.21.15.16:0/2768566498 wait complete. 2024-04-02T04:36:27.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:27 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:27.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:27 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:27.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:27 smithi027 ceph-mon[37550]: pgmap v51: 0 pgs: ; 0 B data, 38 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:36:28.340 DEBUG:teuthology.orchestra.run.smithi016:osd.1> sudo journalctl -f -n 0 -u ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@osd.1.service 2024-04-02T04:36:28.342 INFO:tasks.cephadm:Deploying osd.2 on smithi016 with /dev/vg_nvme/lv_2... 2024-04-02T04:36:28.342 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-02T04:36:28.597 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:36:28 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1[55673]: 2024-04-02T04:36:28.418+0000 7fdf01358640 -1 osd.1 0 log_to_monitors true 2024-04-02T04:36:29.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:28 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:29.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:28 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:29.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:28 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:36:29.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:28 smithi016 ceph-mon[28969]: from='osd.1 [v2:172.21.15.16:6810/3079548081,v1:172.21.15.16:6811/3079548081]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-02T04:36:29.130 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:36:29.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:28 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:29.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:28 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:29.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:28 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:36:29.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:28 smithi027 ceph-mon[37550]: from='osd.1 [v2:172.21.15.16:6810/3079548081,v1:172.21.15.16:6811/3079548081]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-02T04:36:29.979 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:29 smithi016 ceph-mon[28969]: from='osd.1 [v2:172.21.15.16:6810/3079548081,v1:172.21.15.16:6811/3079548081]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-02T04:36:29.980 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:29 smithi016 ceph-mon[28969]: osdmap e12: 2 total, 1 up, 2 in 2024-04-02T04:36:29.980 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:29 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T04:36:29.980 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:29 smithi016 ceph-mon[28969]: from='osd.1 [v2:172.21.15.16:6810/3079548081,v1:172.21.15.16:6811/3079548081]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-04-02T04:36:29.980 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:29 smithi016 ceph-mon[28969]: pgmap v53: 0 pgs: ; 0 B data, 38 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:36:30.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:29 smithi027 ceph-mon[37550]: from='osd.1 [v2:172.21.15.16:6810/3079548081,v1:172.21.15.16:6811/3079548081]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-02T04:36:30.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:29 smithi027 ceph-mon[37550]: osdmap e12: 2 total, 1 up, 2 in 2024-04-02T04:36:30.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:29 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T04:36:30.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:29 smithi027 ceph-mon[37550]: from='osd.1 [v2:172.21.15.16:6810/3079548081,v1:172.21.15.16:6811/3079548081]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-04-02T04:36:30.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:29 smithi027 ceph-mon[37550]: pgmap v53: 0 pgs: ; 0 B data, 38 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:36:30.698 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:36:30 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1[55673]: 2024-04-02T04:36:30.418+0000 7fdef61bb700 -1 osd.1 0 waiting for initial osdmap 2024-04-02T04:36:30.698 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:36:30 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1[55673]: 2024-04-02T04:36:30.425+0000 7fdeee7a9700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T04:36:31.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:30 smithi016 ceph-mon[28969]: from='osd.1 [v2:172.21.15.16:6810/3079548081,v1:172.21.15.16:6811/3079548081]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-04-02T04:36:31.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:30 smithi016 ceph-mon[28969]: osdmap e13: 2 total, 1 up, 2 in 2024-04-02T04:36:31.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:30 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T04:36:31.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:30 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T04:36:31.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:30 smithi016 ceph-mon[28969]: from='osd.1 [v2:172.21.15.16:6810/3079548081,v1:172.21.15.16:6811/3079548081]' entity='osd.1' 2024-04-02T04:36:31.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:30 smithi027 ceph-mon[37550]: from='osd.1 [v2:172.21.15.16:6810/3079548081,v1:172.21.15.16:6811/3079548081]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-04-02T04:36:31.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:30 smithi027 ceph-mon[37550]: osdmap e13: 2 total, 1 up, 2 in 2024-04-02T04:36:31.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:30 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T04:36:31.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:30 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T04:36:31.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:30 smithi027 ceph-mon[37550]: from='osd.1 [v2:172.21.15.16:6810/3079548081,v1:172.21.15.16:6811/3079548081]' entity='osd.1' 2024-04-02T04:36:31.990 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:31 smithi016 ceph-mon[28969]: purged_snaps scrub starts 2024-04-02T04:36:31.990 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:31 smithi016 ceph-mon[28969]: purged_snaps scrub ok 2024-04-02T04:36:31.991 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:31 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T04:36:31.991 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:31 smithi016 ceph-mon[28969]: pgmap v55: 0 pgs: ; 0 B data, 38 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:36:31.991 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:31 smithi016 ceph-mon[28969]: osd.1 [v2:172.21.15.16:6810/3079548081,v1:172.21.15.16:6811/3079548081] boot 2024-04-02T04:36:31.991 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:31 smithi016 ceph-mon[28969]: osdmap e14: 2 total, 2 up, 2 in 2024-04-02T04:36:31.991 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:31 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T04:36:32.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:31 smithi027 ceph-mon[37550]: purged_snaps scrub starts 2024-04-02T04:36:32.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:31 smithi027 ceph-mon[37550]: purged_snaps scrub ok 2024-04-02T04:36:32.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:31 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T04:36:32.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:31 smithi027 ceph-mon[37550]: pgmap v55: 0 pgs: ; 0 B data, 38 MiB used, 89 GiB / 89 GiB avail 2024-04-02T04:36:32.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:31 smithi027 ceph-mon[37550]: osd.1 [v2:172.21.15.16:6810/3079548081,v1:172.21.15.16:6811/3079548081] boot 2024-04-02T04:36:32.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:31 smithi027 ceph-mon[37550]: osdmap e14: 2 total, 2 up, 2 in 2024-04-02T04:36:32.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:31 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T04:36:32.361 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-02T04:36:32.361 INFO:teuthology.orchestra.run.smithi016.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-02T04:36:32.361 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10+0 records in 2024-04-02T04:36:32.361 INFO:teuthology.orchestra.run.smithi016.stderr:10+0 records out 2024-04-02T04:36:32.361 INFO:teuthology.orchestra.run.smithi016.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00985242 s, 1.1 GB/s 2024-04-02T04:36:32.361 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping successful for: 2024-04-02T04:36:33.329 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi016:vg_nvme/lv_2 2024-04-02T04:36:33.553 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:36:33.968 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:33 smithi016 ceph-mon[28969]: osdmap e15: 2 total, 2 up, 2 in 2024-04-02T04:36:33.969 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:33 smithi016 ceph-mon[28969]: pgmap v58: 0 pgs: ; 0 B data, 73 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:36:33.969 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:33 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:33.969 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:33 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:34.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:33 smithi027 ceph-mon[37550]: osdmap e15: 2 total, 2 up, 2 in 2024-04-02T04:36:34.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:33 smithi027 ceph-mon[37550]: pgmap v58: 0 pgs: ; 0 B data, 73 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:36:34.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:33 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:34.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:33 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:35.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:35.683+0000 7fc5f13e7700 1 -- 172.21.15.16:0/1524200785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5ec100bf0 msgr2=0x7fc5ec101050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:36:35.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:35.683+0000 7fc5f13e7700 1 --2- 172.21.15.16:0/1524200785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5ec100bf0 0x7fc5ec101050 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0009910 tx=0x7fc5e0009c20 comp rx=0 tx=0).stop 2024-04-02T04:36:35.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:35.683+0000 7fc5f13e7700 1 -- 172.21.15.16:0/1524200785 shutdown_connections 2024-04-02T04:36:35.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:35.683+0000 7fc5f13e7700 1 --2- 172.21.15.16:0/1524200785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5ec100bf0 0x7fc5ec101050 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:36:35.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:35.683+0000 7fc5f13e7700 1 --2- 172.21.15.16:0/1524200785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5ec0ff9f0 0x7fc5ec0ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:36:35.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:35.683+0000 7fc5f13e7700 1 -- 172.21.15.16:0/1524200785 >> 172.21.15.16:0/1524200785 conn(0x7fc5ec0faf90 msgr2=0x7fc5ec0fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:36:35.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:35.683+0000 7fc5f13e7700 1 -- 172.21.15.16:0/1524200785 shutdown_connections 2024-04-02T04:36:35.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:35.683+0000 7fc5f13e7700 1 -- 172.21.15.16:0/1524200785 wait complete. 2024-04-02T04:36:35.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:35.684+0000 7fc5f13e7700 1 Processor -- start 2024-04-02T04:36:35.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:35.684+0000 7fc5f13e7700 1 -- start start 2024-04-02T04:36:35.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:35.684+0000 7fc5f13e7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5ec0ff9f0 0x7fc5ec103890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:36:35.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:35.684+0000 7fc5f13e7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5ec100bf0 0x7fc5ec101fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:36:35.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:35.684+0000 7fc5f13e7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec103eb0 con 0x7fc5ec100bf0 2024-04-02T04:36:35.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:35.684+0000 7fc5f13e7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec102510 con 0x7fc5ec0ff9f0 2024-04-02T04:36:35.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:35.684+0000 7fc5ea7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5ec100bf0 0x7fc5ec101fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:36:35.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:35.684+0000 7fc5ea7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5ec100bf0 0x7fc5ec101fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36442/0 (socket says 172.21.15.16:36442) 2024-04-02T04:36:35.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:35.684+0000 7fc5ea7fc700 1 -- 172.21.15.16:0/3712219532 learned_addr learned my addr 172.21.15.16:0/3712219532 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:36:35.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:35.684+0000 7fc5eaffd700 1 --2- 172.21.15.16:0/3712219532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5ec0ff9f0 0x7fc5ec103890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:36:35.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:35.684+0000 7fc5ea7fc700 1 -- 172.21.15.16:0/3712219532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5ec0ff9f0 msgr2=0x7fc5ec103890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:36:35.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:35.684+0000 7fc5ea7fc700 1 --2- 172.21.15.16:0/3712219532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5ec0ff9f0 0x7fc5ec103890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:36:35.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:35.684+0000 7fc5ea7fc700 1 -- 172.21.15.16:0/3712219532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e00095f0 con 0x7fc5ec100bf0 2024-04-02T04:36:35.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:35.685+0000 7fc5ea7fc700 1 --2- 172.21.15.16:0/3712219532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5ec100bf0 0x7fc5ec101fd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0005e50 tx=0x7fc5e000f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:36:35.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:35.685+0000 7fc5d3fff700 1 -- 172.21.15.16:0/3712219532 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5e001d070 con 0x7fc5ec100bf0 2024-04-02T04:36:35.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:35.685+0000 7fc5f13e7700 1 -- 172.21.15.16:0/3712219532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5ec102760 con 0x7fc5ec100bf0 2024-04-02T04:36:35.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:35.685+0000 7fc5d3fff700 1 -- 172.21.15.16:0/3712219532 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc5e00052a0 con 0x7fc5ec100bf0 2024-04-02T04:36:35.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:35.685+0000 7fc5d3fff700 1 -- 172.21.15.16:0/3712219532 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5e0022470 con 0x7fc5ec100bf0 2024-04-02T04:36:35.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:35.685+0000 7fc5f13e7700 1 -- 172.21.15.16:0/3712219532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5ec102cd0 con 0x7fc5ec100bf0 2024-04-02T04:36:35.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:35.686+0000 7fc5f13e7700 1 -- 172.21.15.16:0/3712219532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5ec04ea80 con 0x7fc5ec100bf0 2024-04-02T04:36:35.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:35.686+0000 7fc5d3fff700 1 -- 172.21.15.16:0/3712219532 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 90786+0+0 (secure 0 0 0) 0x7fc5e000f8c0 con 0x7fc5ec100bf0 2024-04-02T04:36:35.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:35.687+0000 7fc5d3fff700 1 --2- 172.21.15.16:0/3712219532 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc5d406cc50 0x7fc5d406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:36:35.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:35.687+0000 7fc5d3fff700 1 -- 172.21.15.16:0/3712219532 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2414+0+0 (secure 0 0 0) 0x7fc5e008c9a0 con 0x7fc5ec100bf0 2024-04-02T04:36:35.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:35.687+0000 7fc5eaffd700 1 --2- 172.21.15.16:0/3712219532 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc5d406cc50 0x7fc5d406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:36:35.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:35.687+0000 7fc5eaffd700 1 --2- 172.21.15.16:0/3712219532 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc5d406cc50 0x7fc5d406f110 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec100a50 tx=0x7fc5dc009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:36:35.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:35.691+0000 7fc5d3fff700 1 -- 172.21.15.16:0/3712219532 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc5e005b550 con 0x7fc5ec100bf0 2024-04-02T04:36:35.908 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:35 smithi016 ceph-mon[28969]: pgmap v59: 0 pgs: ; 0 B data, 73 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:36:35.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:35.904+0000 7fc5f13e7700 1 -- 172.21.15.16:0/3712219532 --> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5ec0697f0 con 0x7fc5d406cc50 2024-04-02T04:36:36.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:35 smithi027 ceph-mon[37550]: pgmap v59: 0 pgs: ; 0 B data, 73 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:36:36.809 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:36 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T04:36:36.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:36 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T04:36:36.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:36 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:36:37.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:36 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T04:36:37.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:36 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T04:36:37.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:36 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:36:37.886 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:37 smithi016 ceph-mon[28969]: from='client.14288 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:36:37.886 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:37 smithi016 ceph-mon[28969]: pgmap v60: 0 pgs: ; 0 B data, 73 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:36:38.047 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:37 smithi027 ceph-mon[37550]: from='client.14288 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:36:38.047 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:37 smithi027 ceph-mon[37550]: pgmap v60: 0 pgs: ; 0 B data, 73 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:36:39.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:39 smithi016 ceph-mon[28969]: Detected new or changed devices on smithi016 2024-04-02T04:36:39.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:39 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:39.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:39 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:39.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:39 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:36:39.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:39 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:36:39.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:39 smithi016 ceph-mon[28969]: Adjusting osd_memory_target on smithi016 to 5922M 2024-04-02T04:36:39.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:39 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:39.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:39 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:36:39.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:39 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:36:39.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:39 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:39.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:39 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:36:39.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:39 smithi016 ceph-mon[28969]: pgmap v61: 0 pgs: ; 0 B data, 73 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:36:39.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:39 smithi027 ceph-mon[37550]: Detected new or changed devices on smithi016 2024-04-02T04:36:39.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:39 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:39.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:39 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:39.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:39 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:36:39.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:39 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:36:39.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:39 smithi027 ceph-mon[37550]: Adjusting osd_memory_target on smithi016 to 5922M 2024-04-02T04:36:39.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:39 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:39.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:39 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:36:39.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:39 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:36:39.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:39 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:39.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:39 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:36:39.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:39 smithi027 ceph-mon[37550]: pgmap v61: 0 pgs: ; 0 B data, 73 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:36:40.475 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:40 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/843203925' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "046262b5-386e-4270-863c-e97a61a038b3"}]: dispatch 2024-04-02T04:36:40.476 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:40 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/843203925' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "046262b5-386e-4270-863c-e97a61a038b3"}]': finished 2024-04-02T04:36:40.476 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:40 smithi016 ceph-mon[28969]: osdmap e16: 3 total, 2 up, 3 in 2024-04-02T04:36:40.476 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:40 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T04:36:40.476 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:40 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/1994935309' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T04:36:40.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:40 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/843203925' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "046262b5-386e-4270-863c-e97a61a038b3"}]: dispatch 2024-04-02T04:36:40.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:40 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/843203925' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "046262b5-386e-4270-863c-e97a61a038b3"}]': finished 2024-04-02T04:36:40.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:40 smithi027 ceph-mon[37550]: osdmap e16: 3 total, 2 up, 3 in 2024-04-02T04:36:40.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:40 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T04:36:40.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:40 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/1994935309' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T04:36:41.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:41 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:41.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:41 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:41.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:41 smithi027 ceph-mon[37550]: pgmap v63: 0 pgs: ; 0 B data, 73 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:36:41.809 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:41 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:41.809 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:41 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:41.809 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:41 smithi016 ceph-mon[28969]: pgmap v63: 0 pgs: ; 0 B data, 73 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:36:43.809 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:43 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:43.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:43 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:43.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:43 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:36:43.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:43 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:36:43.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:43 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:43.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:43 smithi016 ceph-mon[28969]: pgmap v64: 0 pgs: ; 0 B data, 73 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:36:43.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:43 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:36:43.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:43 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:43.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:43 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:43.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:43 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:36:43.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:43 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:36:43.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:43 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:43.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:43 smithi027 ceph-mon[37550]: pgmap v64: 0 pgs: ; 0 B data, 73 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:36:43.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:43 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:36:46.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:45 smithi016 ceph-mon[28969]: pgmap v65: 0 pgs: ; 0 B data, 73 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:36:46.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:45 smithi027 ceph-mon[37550]: pgmap v65: 0 pgs: ; 0 B data, 73 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:36:47.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:46 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-02T04:36:47.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:46 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:36:47.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:46 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-02T04:36:47.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:46 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:36:48.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:47 smithi016 ceph-mon[28969]: Deploying daemon osd.2 on smithi016 2024-04-02T04:36:48.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:47 smithi016 ceph-mon[28969]: pgmap v66: 0 pgs: ; 0 B data, 73 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:36:48.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:47 smithi027 ceph-mon[37550]: Deploying daemon osd.2 on smithi016 2024-04-02T04:36:48.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:47 smithi027 ceph-mon[37550]: pgmap v66: 0 pgs: ; 0 B data, 73 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:36:49.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:49 smithi016 ceph-mon[28969]: pgmap v67: 0 pgs: ; 0 B data, 73 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:36:50.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:49 smithi027 ceph-mon[37550]: pgmap v67: 0 pgs: ; 0 B data, 73 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:36:52.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:51 smithi016 ceph-mon[28969]: pgmap v68: 0 pgs: ; 0 B data, 73 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:36:52.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:51 smithi027 ceph-mon[37550]: pgmap v68: 0 pgs: ; 0 B data, 73 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:36:53.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:52 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:53.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:52 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:53.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:52 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:36:53.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:52 smithi016 ceph-mon[28969]: pgmap v69: 0 pgs: ; 0 B data, 73 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:36:53.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:52 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:53.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:52 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:53.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:52 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:36:53.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:52 smithi027 ceph-mon[37550]: pgmap v69: 0 pgs: ; 0 B data, 73 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:36:55.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:55.557+0000 7fc5d3fff700 1 -- 172.21.15.16:0/3712219532 <== mgr.14217 v2:172.21.15.16:6800/3643600900 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc5ec0697f0 con 0x7fc5d406cc50 2024-04-02T04:36:55.561 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 2 on host 'smithi016' 2024-04-02T04:36:55.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:55.559+0000 7fc5f13e7700 1 -- 172.21.15.16:0/3712219532 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc5d406cc50 msgr2=0x7fc5d406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:36:55.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:55.559+0000 7fc5f13e7700 1 --2- 172.21.15.16:0/3712219532 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc5d406cc50 0x7fc5d406f110 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec100a50 tx=0x7fc5dc009260 comp rx=0 tx=0).stop 2024-04-02T04:36:55.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:55.559+0000 7fc5f13e7700 1 -- 172.21.15.16:0/3712219532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5ec100bf0 msgr2=0x7fc5ec101fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:36:55.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:55.559+0000 7fc5f13e7700 1 --2- 172.21.15.16:0/3712219532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5ec100bf0 0x7fc5ec101fd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0005e50 tx=0x7fc5e000f460 comp rx=0 tx=0).stop 2024-04-02T04:36:55.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:55.559+0000 7fc5f13e7700 1 -- 172.21.15.16:0/3712219532 shutdown_connections 2024-04-02T04:36:55.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:55.559+0000 7fc5f13e7700 1 --2- 172.21.15.16:0/3712219532 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc5d406cc50 0x7fc5d406f110 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:36:55.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:55.559+0000 7fc5f13e7700 1 --2- 172.21.15.16:0/3712219532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5ec0ff9f0 0x7fc5ec103890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:36:55.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:55.559+0000 7fc5f13e7700 1 --2- 172.21.15.16:0/3712219532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5ec100bf0 0x7fc5ec101fd0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:36:55.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:55.559+0000 7fc5f13e7700 1 -- 172.21.15.16:0/3712219532 >> 172.21.15.16:0/3712219532 conn(0x7fc5ec0faf90 msgr2=0x7fc5ec0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:36:55.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:55.559+0000 7fc5f13e7700 1 -- 172.21.15.16:0/3712219532 shutdown_connections 2024-04-02T04:36:55.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:36:55.559+0000 7fc5f13e7700 1 -- 172.21.15.16:0/3712219532 wait complete. 2024-04-02T04:36:56.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:55 smithi016 ceph-mon[28969]: pgmap v70: 0 pgs: ; 0 B data, 73 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:36:56.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:55 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:56.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:55 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:56.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:55 smithi027 ceph-mon[37550]: pgmap v70: 0 pgs: ; 0 B data, 73 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:36:56.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:55 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:56.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:55 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:56.287 DEBUG:teuthology.orchestra.run.smithi016:osd.2> sudo journalctl -f -n 0 -u ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@osd.2.service 2024-04-02T04:36:56.290 INFO:tasks.cephadm:Deploying osd.3 on smithi027 with /dev/vg_nvme/lv_4... 2024-04-02T04:36:56.290 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-02T04:36:56.521 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi027/config 2024-04-02T04:36:56.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:56 smithi027 ceph-mon[37550]: from='osd.2 [v2:172.21.15.16:6818/3734332239,v1:172.21.15.16:6819/3734332239]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-02T04:36:57.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:56 smithi016 ceph-mon[28969]: from='osd.2 [v2:172.21.15.16:6818/3734332239,v1:172.21.15.16:6819/3734332239]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-02T04:36:58.056 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:57 smithi016 ceph-mon[28969]: pgmap v71: 0 pgs: ; 0 B data, 73 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:36:58.056 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:57 smithi016 ceph-mon[28969]: from='osd.2 [v2:172.21.15.16:6818/3734332239,v1:172.21.15.16:6819/3734332239]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-02T04:36:58.056 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:57 smithi016 ceph-mon[28969]: osdmap e17: 3 total, 2 up, 3 in 2024-04-02T04:36:58.056 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:57 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T04:36:58.056 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:57 smithi016 ceph-mon[28969]: from='osd.2 [v2:172.21.15.16:6818/3734332239,v1:172.21.15.16:6819/3734332239]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-04-02T04:36:58.056 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:57 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:36:58.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:57 smithi027 ceph-mon[37550]: pgmap v71: 0 pgs: ; 0 B data, 73 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:36:58.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:57 smithi027 ceph-mon[37550]: from='osd.2 [v2:172.21.15.16:6818/3734332239,v1:172.21.15.16:6819/3734332239]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-02T04:36:58.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:57 smithi027 ceph-mon[37550]: osdmap e17: 3 total, 2 up, 3 in 2024-04-02T04:36:58.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:57 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T04:36:58.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:57 smithi027 ceph-mon[37550]: from='osd.2 [v2:172.21.15.16:6818/3734332239,v1:172.21.15.16:6819/3734332239]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-04-02T04:36:58.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:57 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:36:58.387 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-02T04:36:58.387 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-02T04:36:58.387 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10+0 records in 2024-04-02T04:36:58.387 INFO:teuthology.orchestra.run.smithi027.stderr:10+0 records out 2024-04-02T04:36:58.388 INFO:teuthology.orchestra.run.smithi027.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111706 s, 939 MB/s 2024-04-02T04:36:58.388 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping successful for: 2024-04-02T04:36:58.802 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:36:58 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2[63084]: 2024-04-02T04:36:58.486+0000 7f4e2488f700 -1 osd.2 0 waiting for initial osdmap 2024-04-02T04:36:58.802 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:36:58 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2[63084]: 2024-04-02T04:36:58.492+0000 7f4e1b67a700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T04:36:59.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:58 smithi016 ceph-mon[28969]: from='osd.2 [v2:172.21.15.16:6818/3734332239,v1:172.21.15.16:6819/3734332239]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-04-02T04:36:59.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:58 smithi016 ceph-mon[28969]: osdmap e18: 3 total, 2 up, 3 in 2024-04-02T04:36:59.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:58 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T04:36:59.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:58 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T04:36:59.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:58 smithi016 ceph-mon[28969]: from='osd.2 [v2:172.21.15.16:6818/3734332239,v1:172.21.15.16:6819/3734332239]' entity='osd.2' 2024-04-02T04:36:59.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:58 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:59.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:58 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:59.073 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:58 smithi027 ceph-mon[37550]: from='osd.2 [v2:172.21.15.16:6818/3734332239,v1:172.21.15.16:6819/3734332239]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-04-02T04:36:59.074 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:58 smithi027 ceph-mon[37550]: osdmap e18: 3 total, 2 up, 3 in 2024-04-02T04:36:59.074 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:58 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T04:36:59.074 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:58 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T04:36:59.074 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:58 smithi027 ceph-mon[37550]: from='osd.2 [v2:172.21.15.16:6818/3734332239,v1:172.21.15.16:6819/3734332239]' entity='osd.2' 2024-04-02T04:36:59.074 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:58 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:59.074 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:58 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:36:59.095 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi027:vg_nvme/lv_4 2024-04-02T04:36:59.310 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi027/config 2024-04-02T04:37:00.048 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:59 smithi027 ceph-mon[37550]: purged_snaps scrub starts 2024-04-02T04:37:00.049 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:59 smithi027 ceph-mon[37550]: purged_snaps scrub ok 2024-04-02T04:37:00.049 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:59 smithi027 ceph-mon[37550]: pgmap v74: 0 pgs: ; 0 B data, 73 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:37:00.049 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:59 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T04:37:00.049 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:59 smithi027 ceph-mon[37550]: osd.2 [v2:172.21.15.16:6818/3734332239,v1:172.21.15.16:6819/3734332239] boot 2024-04-02T04:37:00.049 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:59 smithi027 ceph-mon[37550]: osdmap e19: 3 total, 3 up, 3 in 2024-04-02T04:37:00.049 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:36:59 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T04:37:00.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:59 smithi016 ceph-mon[28969]: purged_snaps scrub starts 2024-04-02T04:37:00.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:59 smithi016 ceph-mon[28969]: purged_snaps scrub ok 2024-04-02T04:37:00.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:59 smithi016 ceph-mon[28969]: pgmap v74: 0 pgs: ; 0 B data, 73 MiB used, 179 GiB / 179 GiB avail 2024-04-02T04:37:00.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:59 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T04:37:00.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:59 smithi016 ceph-mon[28969]: osd.2 [v2:172.21.15.16:6818/3734332239,v1:172.21.15.16:6819/3734332239] boot 2024-04-02T04:37:00.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:59 smithi016 ceph-mon[28969]: osdmap e19: 3 total, 3 up, 3 in 2024-04-02T04:37:00.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:36:59 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T04:37:00.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:00.859+0000 7fe3a0452700 1 -- 172.21.15.27:0/1785875685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe398101d10 msgr2=0x7fe398102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:37:00.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:00.859+0000 7fe3a0452700 1 --2- 172.21.15.27:0/1785875685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe398101d10 0x7fe398102170 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe388009910 tx=0x7fe388009c20 comp rx=0 tx=0).stop 2024-04-02T04:37:00.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:00.859+0000 7fe3a0452700 1 -- 172.21.15.27:0/1785875685 shutdown_connections 2024-04-02T04:37:00.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:00.859+0000 7fe3a0452700 1 --2- 172.21.15.27:0/1785875685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe398101d10 0x7fe398102170 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:37:00.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:00.859+0000 7fe3a0452700 1 --2- 172.21.15.27:0/1785875685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe398100b10 0x7fe398100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:37:00.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:00.859+0000 7fe3a0452700 1 -- 172.21.15.27:0/1785875685 >> 172.21.15.27:0/1785875685 conn(0x7fe3980fc0d0 msgr2=0x7fe3980fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:37:00.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:00.859+0000 7fe3a0452700 1 -- 172.21.15.27:0/1785875685 shutdown_connections 2024-04-02T04:37:00.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:00.859+0000 7fe3a0452700 1 -- 172.21.15.27:0/1785875685 wait complete. 2024-04-02T04:37:00.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:00.859+0000 7fe3a0452700 1 Processor -- start 2024-04-02T04:37:00.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:00.859+0000 7fe3a0452700 1 -- start start 2024-04-02T04:37:00.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:00.860+0000 7fe3a0452700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe398100b10 0x7fe3981964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:37:00.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:00.860+0000 7fe3a0452700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe398101d10 0x7fe398196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:37:00.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:00.860+0000 7fe3a0452700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe398197050 con 0x7fe398101d10 2024-04-02T04:37:00.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:00.860+0000 7fe3a0452700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe398197190 con 0x7fe398100b10 2024-04-02T04:37:00.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:00.860+0000 7fe39e1ee700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe398100b10 0x7fe3981964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:37:00.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:00.860+0000 7fe39e1ee700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe398100b10 0x7fe3981964f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34078/0 (socket says 172.21.15.27:34078) 2024-04-02T04:37:00.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:00.860+0000 7fe39e1ee700 1 -- 172.21.15.27:0/344806205 learned_addr learned my addr 172.21.15.27:0/344806205 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-02T04:37:00.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:00.860+0000 7fe39d9ed700 1 --2- 172.21.15.27:0/344806205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe398101d10 0x7fe398196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:37:00.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:00.860+0000 7fe39e1ee700 1 -- 172.21.15.27:0/344806205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe398101d10 msgr2=0x7fe398196a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:37:00.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:00.860+0000 7fe39e1ee700 1 --2- 172.21.15.27:0/344806205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe398101d10 0x7fe398196a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:37:00.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:00.860+0000 7fe39e1ee700 1 -- 172.21.15.27:0/344806205 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3880095f0 con 0x7fe398100b10 2024-04-02T04:37:00.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:00.860+0000 7fe39e1ee700 1 --2- 172.21.15.27:0/344806205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe398100b10 0x7fe3981964f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe39400d900 tx=0x7fe39400dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:37:00.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:00.860+0000 7fe38f7fe700 1 -- 172.21.15.27:0/344806205 <== mon.1 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3940041f0 con 0x7fe398100b10 2024-04-02T04:37:00.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:00.860+0000 7fe3a0452700 1 -- 172.21.15.27:0/344806205 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe39819bbf0 con 0x7fe398100b10 2024-04-02T04:37:00.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:00.860+0000 7fe38f7fe700 1 -- 172.21.15.27:0/344806205 <== mon.1 v2:172.21.15.27:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe394004350 con 0x7fe398100b10 2024-04-02T04:37:00.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:00.861+0000 7fe38f7fe700 1 -- 172.21.15.27:0/344806205 <== mon.1 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe394003cf0 con 0x7fe398100b10 2024-04-02T04:37:00.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:00.861+0000 7fe3a0452700 1 -- 172.21.15.27:0/344806205 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe39819c140 con 0x7fe398100b10 2024-04-02T04:37:00.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:00.861+0000 7fe3a0452700 1 -- 172.21.15.27:0/344806205 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe398190690 con 0x7fe398100b10 2024-04-02T04:37:00.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:00.862+0000 7fe38f7fe700 1 -- 172.21.15.27:0/344806205 <== mon.1 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 90786+0+0 (secure 0 0 0) 0x7fe394009730 con 0x7fe398100b10 2024-04-02T04:37:00.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:00.863+0000 7fe38f7fe700 1 --2- 172.21.15.27:0/344806205 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fe38406cc50 0x7fe38406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:37:00.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:00.863+0000 7fe38f7fe700 1 -- 172.21.15.27:0/344806205 <== mon.1 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 2846+0+0 (secure 0 0 0) 0x7fe39408b560 con 0x7fe398100b10 2024-04-02T04:37:00.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:00.863+0000 7fe39d9ed700 1 --2- 172.21.15.27:0/344806205 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fe38406cc50 0x7fe38406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:37:00.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:00.863+0000 7fe39d9ed700 1 --2- 172.21.15.27:0/344806205 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fe38406cc50 0x7fe38406f110 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe388006010 tx=0x7fe38800b540 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:37:00.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:00.868+0000 7fe38f7fe700 1 -- 172.21.15.27:0/344806205 <== mon.1 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe394059ee0 con 0x7fe398100b10 2024-04-02T04:37:01.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:01.082+0000 7fe3a0452700 1 -- 172.21.15.27:0/344806205 --> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fe39802ce40 con 0x7fe38406cc50 2024-04-02T04:37:01.722 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:01 smithi016 ceph-mon[28969]: osdmap e20: 3 total, 3 up, 3 in 2024-04-02T04:37:01.722 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:01 smithi016 ceph-mon[28969]: pgmap v77: 0 pgs: ; 0 B data, 108 MiB used, 268 GiB / 268 GiB avail 2024-04-02T04:37:01.722 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:01 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-02T04:37:01.722 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:01 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T04:37:01.723 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:01 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T04:37:01.723 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:01 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:37:01.894 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:01 smithi027 ceph-mon[37550]: osdmap e20: 3 total, 3 up, 3 in 2024-04-02T04:37:01.895 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:01 smithi027 ceph-mon[37550]: pgmap v77: 0 pgs: ; 0 B data, 108 MiB used, 268 GiB / 268 GiB avail 2024-04-02T04:37:01.895 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:01 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-02T04:37:01.895 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:01 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T04:37:01.895 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:01 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T04:37:01.895 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:01 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:37:02.795 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:02 smithi027 ceph-mon[37550]: from='client.24131 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:37:02.795 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:02 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-02T04:37:02.795 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:02 smithi027 ceph-mon[37550]: osdmap e21: 3 total, 3 up, 3 in 2024-04-02T04:37:02.795 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:02 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T04:37:02.795 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:02 smithi027 ceph-mon[37550]: Detected new or changed devices on smithi016 2024-04-02T04:37:02.795 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:02 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:02.795 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:02 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:02.796 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:02 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:37:02.796 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:02 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:37:02.796 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:02 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:37:02.796 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:02 smithi027 ceph-mon[37550]: Adjusting osd_memory_target on smithi016 to 3948M 2024-04-02T04:37:02.796 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:02 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:02.796 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:02 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:37:02.796 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:02 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:37:02.796 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:02 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:02.796 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:02 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:37:03.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:02 smithi016 ceph-mon[28969]: from='client.24131 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:37:03.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:02 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-02T04:37:03.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:02 smithi016 ceph-mon[28969]: osdmap e21: 3 total, 3 up, 3 in 2024-04-02T04:37:03.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:02 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T04:37:03.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:02 smithi016 ceph-mon[28969]: Detected new or changed devices on smithi016 2024-04-02T04:37:03.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:02 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:03.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:02 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:03.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:02 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:37:03.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:02 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:37:03.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:02 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:37:03.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:02 smithi016 ceph-mon[28969]: Adjusting osd_memory_target on smithi016 to 3948M 2024-04-02T04:37:03.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:02 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:03.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:02 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:37:03.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:02 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:37:03.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:02 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:03.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:02 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:37:03.423 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:37:03 smithi016 sudo[67563]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T04:37:03.423 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:37:03 smithi016 sudo[67563]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T04:37:03.423 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:37:03 smithi016 sudo[67563]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T04:37:03.423 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:37:03 smithi016 sudo[67563]: pam_unix(sudo:session): session closed for user root 2024-04-02T04:37:03.706 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:37:03 smithi016 sudo[67595]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T04:37:03.706 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:37:03 smithi016 sudo[67595]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T04:37:03.706 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:37:03 smithi016 sudo[67595]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T04:37:03.706 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:37:03 smithi016 sudo[67595]: pam_unix(sudo:session): session closed for user root 2024-04-02T04:37:03.972 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:37:03 smithi016 sudo[67598]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T04:37:03.972 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:37:03 smithi016 sudo[67598]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T04:37:03.972 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:37:03 smithi016 sudo[67598]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T04:37:03.972 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:37:03 smithi016 sudo[67598]: pam_unix(sudo:session): session closed for user root 2024-04-02T04:37:03.972 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:03 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-02T04:37:03.972 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:03 smithi016 ceph-mon[28969]: osdmap e22: 3 total, 3 up, 3 in 2024-04-02T04:37:03.972 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:03 smithi016 ceph-mon[28969]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 108 MiB used, 268 GiB / 268 GiB avail 2024-04-02T04:37:03.972 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:03 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:03.972 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:03 smithi016 ceph-mon[28969]: from='client.? 172.21.15.27:0/1855581557' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "637f2a77-59a2-44fb-b47e-fcbd70f4423c"}]: dispatch 2024-04-02T04:37:03.973 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:03 smithi016 ceph-mon[28969]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "637f2a77-59a2-44fb-b47e-fcbd70f4423c"}]: dispatch 2024-04-02T04:37:03.973 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:03 smithi016 ceph-mon[28969]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "637f2a77-59a2-44fb-b47e-fcbd70f4423c"}]': finished 2024-04-02T04:37:03.973 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:03 smithi016 ceph-mon[28969]: osdmap e23: 4 total, 3 up, 4 in 2024-04-02T04:37:03.973 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:03 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T04:37:03.978 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:03 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-02T04:37:03.978 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:03 smithi027 ceph-mon[37550]: osdmap e22: 3 total, 3 up, 3 in 2024-04-02T04:37:03.978 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:03 smithi027 ceph-mon[37550]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 108 MiB used, 268 GiB / 268 GiB avail 2024-04-02T04:37:03.978 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:03 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:03.979 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:03 smithi027 ceph-mon[37550]: from='client.? 172.21.15.27:0/1855581557' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "637f2a77-59a2-44fb-b47e-fcbd70f4423c"}]: dispatch 2024-04-02T04:37:03.979 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:03 smithi027 ceph-mon[37550]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "637f2a77-59a2-44fb-b47e-fcbd70f4423c"}]: dispatch 2024-04-02T04:37:03.979 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:03 smithi027 ceph-mon[37550]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "637f2a77-59a2-44fb-b47e-fcbd70f4423c"}]': finished 2024-04-02T04:37:03.979 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:03 smithi027 ceph-mon[37550]: osdmap e23: 4 total, 3 up, 4 in 2024-04-02T04:37:03.979 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:03 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T04:37:04.241 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:37:03 smithi016 sudo[67656]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T04:37:04.241 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:37:03 smithi016 sudo[67656]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T04:37:04.242 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:37:03 smithi016 sudo[67656]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T04:37:04.242 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:37:03 smithi016 sudo[67656]: pam_unix(sudo:session): session closed for user root 2024-04-02T04:37:04.513 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:37:04 smithi016 sudo[67677]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T04:37:04.513 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:37:04 smithi016 sudo[67677]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T04:37:04.513 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:37:04 smithi016 sudo[67677]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T04:37:04.514 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:37:04 smithi016 sudo[67677]: pam_unix(sudo:session): session closed for user root 2024-04-02T04:37:04.774 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:37:04 smithi016 sudo[67726]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T04:37:04.774 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:37:04 smithi016 sudo[67726]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T04:37:04.774 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:37:04 smithi016 sudo[67726]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T04:37:04.774 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:37:04 smithi016 sudo[67726]: pam_unix(sudo:session): session closed for user root 2024-04-02T04:37:05.044 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:04 smithi016 sudo[67746]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T04:37:05.044 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:04 smithi016 sudo[67746]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T04:37:05.045 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:04 smithi016 sudo[67746]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T04:37:05.045 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:04 smithi016 sudo[67746]: pam_unix(sudo:session): session closed for user root 2024-04-02T04:37:05.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:05 smithi016 sudo[67761]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T04:37:05.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:05 smithi016 sudo[67761]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T04:37:05.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:05 smithi016 sudo[67761]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T04:37:05.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:05 smithi016 sudo[67761]: pam_unix(sudo:session): session closed for user root 2024-04-02T04:37:05.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:05 smithi016 ceph-mon[28969]: from='client.? 172.21.15.27:0/2090613006' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T04:37:05.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:05 smithi016 ceph-mon[28969]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T04:37:05.395 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:05 smithi027 ceph-mon[37550]: from='client.? 172.21.15.27:0/2090613006' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T04:37:05.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:05 smithi027 ceph-mon[37550]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T04:37:05.895 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:05 smithi027 sudo[41839]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T04:37:05.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:05 smithi027 sudo[41839]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T04:37:05.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:05 smithi027 sudo[41839]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T04:37:05.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:05 smithi027 sudo[41839]: pam_unix(sudo:session): session closed for user root 2024-04-02T04:37:05.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:05.972+0000 7fe38f7fe700 1 -- 172.21.15.27:0/344806205 <== mon.1 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fe394056820 con 0x7fe398100b10 2024-04-02T04:37:06.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:06 smithi016 ceph-mon[28969]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 109 MiB used, 268 GiB / 268 GiB avail 2024-04-02T04:37:06.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:06 smithi016 ceph-mon[28969]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T04:37:06.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:06 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi016"}]: dispatch 2024-04-02T04:37:06.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:06 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2024-04-02T04:37:06.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:06 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi016"}]: dispatch 2024-04-02T04:37:06.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:06 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2024-04-02T04:37:06.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:06 smithi016 ceph-mon[28969]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T04:37:06.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:06 smithi016 ceph-mon[28969]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T04:37:06.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:06 smithi016 ceph-mon[28969]: mgrmap e19: smithi016.eqnebx(active, since 2m), standbys: smithi027.kjzcsc 2024-04-02T04:37:06.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:06 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:06.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:06 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:06.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:06 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:06.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:06 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:37:06.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:06 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:37:06.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:06 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:06.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:05 smithi027 sudo[42029]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T04:37:06.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:05 smithi027 sudo[42029]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T04:37:06.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:05 smithi027 sudo[42029]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T04:37:06.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:05 smithi027 sudo[42029]: pam_unix(sudo:session): session closed for user root 2024-04-02T04:37:06.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:06 smithi027 ceph-mon[37550]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 109 MiB used, 268 GiB / 268 GiB avail 2024-04-02T04:37:06.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:06 smithi027 ceph-mon[37550]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T04:37:06.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:06 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi016"}]: dispatch 2024-04-02T04:37:06.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:06 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2024-04-02T04:37:06.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:06 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi016"}]: dispatch 2024-04-02T04:37:06.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:06 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2024-04-02T04:37:06.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:06 smithi027 ceph-mon[37550]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T04:37:06.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:06 smithi027 ceph-mon[37550]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T04:37:06.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:06 smithi027 ceph-mon[37550]: mgrmap e19: smithi016.eqnebx(active, since 2m), standbys: smithi027.kjzcsc 2024-04-02T04:37:06.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:06 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:06.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:06 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:06.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:06 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:06.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:06 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:37:06.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:06 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:37:06.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:06 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:07.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:07 smithi016 ceph-mon[28969]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 114 MiB used, 268 GiB / 268 GiB avail 2024-04-02T04:37:07.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:07 smithi027 ceph-mon[37550]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 114 MiB used, 268 GiB / 268 GiB avail 2024-04-02T04:37:09.785 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:09 smithi027 ceph-mon[37550]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 114 MiB used, 268 GiB / 268 GiB avail 2024-04-02T04:37:10.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:09 smithi016 ceph-mon[28969]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 114 MiB used, 268 GiB / 268 GiB avail 2024-04-02T04:37:12.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:11 smithi016 ceph-mon[28969]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 114 MiB used, 268 GiB / 268 GiB avail 2024-04-02T04:37:12.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:11 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-02T04:37:12.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:11 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:37:12.084 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:11 smithi027 ceph-mon[37550]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 114 MiB used, 268 GiB / 268 GiB avail 2024-04-02T04:37:12.085 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:11 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-02T04:37:12.085 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:11 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:37:13.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:12 smithi016 ceph-mon[28969]: Deploying daemon osd.3 on smithi027 2024-04-02T04:37:13.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:12 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:37:13.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:12 smithi027 ceph-mon[37550]: Deploying daemon osd.3 on smithi027 2024-04-02T04:37:13.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:12 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:37:13.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:13 smithi027 ceph-mon[37550]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 114 MiB used, 268 GiB / 268 GiB avail 2024-04-02T04:37:14.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:13 smithi016 ceph-mon[28969]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 114 MiB used, 268 GiB / 268 GiB avail 2024-04-02T04:37:15.988 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:15 smithi027 ceph-mon[37550]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 114 MiB used, 268 GiB / 268 GiB avail 2024-04-02T04:37:16.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:15 smithi016 ceph-mon[28969]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 114 MiB used, 268 GiB / 268 GiB avail 2024-04-02T04:37:17.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:17 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:17.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:17 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:17.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:17 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:37:17.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:17 smithi016 ceph-mon[28969]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 114 MiB used, 268 GiB / 268 GiB avail 2024-04-02T04:37:17.622 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:17 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:17.622 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:17 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:17.622 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:17 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:37:17.622 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:17 smithi027 ceph-mon[37550]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 114 MiB used, 268 GiB / 268 GiB avail 2024-04-02T04:37:19.882 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:19 smithi027 ceph-mon[37550]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 114 MiB used, 268 GiB / 268 GiB avail 2024-04-02T04:37:20.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:19 smithi016 ceph-mon[28969]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 114 MiB used, 268 GiB / 268 GiB avail 2024-04-02T04:37:20.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:20.192+0000 7fe38f7fe700 1 -- 172.21.15.27:0/344806205 <== mgr.14217 v2:172.21.15.16:6800/3643600900 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe39802ce40 con 0x7fe38406cc50 2024-04-02T04:37:20.196 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 3 on host 'smithi027' 2024-04-02T04:37:20.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:20.194+0000 7fe3a0452700 1 -- 172.21.15.27:0/344806205 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fe38406cc50 msgr2=0x7fe38406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:37:20.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:20.194+0000 7fe3a0452700 1 --2- 172.21.15.27:0/344806205 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fe38406cc50 0x7fe38406f110 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe388006010 tx=0x7fe38800b540 comp rx=0 tx=0).stop 2024-04-02T04:37:20.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:20.194+0000 7fe3a0452700 1 -- 172.21.15.27:0/344806205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe398100b10 msgr2=0x7fe3981964f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:37:20.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:20.194+0000 7fe3a0452700 1 --2- 172.21.15.27:0/344806205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe398100b10 0x7fe3981964f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe39400d900 tx=0x7fe39400dc10 comp rx=0 tx=0).stop 2024-04-02T04:37:20.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:20.194+0000 7fe3a0452700 1 -- 172.21.15.27:0/344806205 shutdown_connections 2024-04-02T04:37:20.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:20.194+0000 7fe3a0452700 1 --2- 172.21.15.27:0/344806205 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fe38406cc50 0x7fe38406f110 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:37:20.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:20.194+0000 7fe3a0452700 1 --2- 172.21.15.27:0/344806205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe398100b10 0x7fe3981964f0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:37:20.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:20.194+0000 7fe3a0452700 1 --2- 172.21.15.27:0/344806205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe398101d10 0x7fe398196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:37:20.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:20.194+0000 7fe3a0452700 1 -- 172.21.15.27:0/344806205 >> 172.21.15.27:0/344806205 conn(0x7fe3980fc0d0 msgr2=0x7fe398105260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:37:20.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:20.194+0000 7fe3a0452700 1 -- 172.21.15.27:0/344806205 shutdown_connections 2024-04-02T04:37:20.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:20.194+0000 7fe3a0452700 1 -- 172.21.15.27:0/344806205 wait complete. 2024-04-02T04:37:20.797 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:20 smithi027 ceph-mon[37550]: from='osd.3 [v2:172.21.15.27:6800/388124754,v1:172.21.15.27:6801/388124754]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-02T04:37:20.797 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:20 smithi027 ceph-mon[37550]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-02T04:37:20.797 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:20 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:20.797 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:20 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:20.820 DEBUG:teuthology.orchestra.run.smithi027:osd.3> sudo journalctl -f -n 0 -u ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@osd.3.service 2024-04-02T04:37:20.823 INFO:tasks.cephadm:Deploying osd.4 on smithi027 with /dev/vg_nvme/lv_3... 2024-04-02T04:37:20.823 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-02T04:37:21.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:20 smithi016 ceph-mon[28969]: from='osd.3 [v2:172.21.15.27:6800/388124754,v1:172.21.15.27:6801/388124754]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-02T04:37:21.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:20 smithi016 ceph-mon[28969]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-02T04:37:21.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:20 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:21.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:20 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:21.081 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi027/config 2024-04-02T04:37:21.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:21 smithi027 ceph-mon[37550]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-02T04:37:21.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:21 smithi027 ceph-mon[37550]: osdmap e24: 4 total, 3 up, 4 in 2024-04-02T04:37:21.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:21 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T04:37:21.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:21 smithi027 ceph-mon[37550]: from='osd.3 [v2:172.21.15.27:6800/388124754,v1:172.21.15.27:6801/388124754]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-02T04:37:21.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:21 smithi027 ceph-mon[37550]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-02T04:37:21.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:21 smithi027 ceph-mon[37550]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 114 MiB used, 268 GiB / 268 GiB avail 2024-04-02T04:37:21.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:21 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:21.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:21 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:21.897 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:37:21 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3[43496]: 2024-04-02T04:37:21.712+0000 7f7172119700 -1 osd.3 0 waiting for initial osdmap 2024-04-02T04:37:21.897 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:37:21 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3[43496]: 2024-04-02T04:37:21.718+0000 7f716e70f700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T04:37:22.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:21 smithi016 ceph-mon[28969]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-02T04:37:22.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:21 smithi016 ceph-mon[28969]: osdmap e24: 4 total, 3 up, 4 in 2024-04-02T04:37:22.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:21 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T04:37:22.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:21 smithi016 ceph-mon[28969]: from='osd.3 [v2:172.21.15.27:6800/388124754,v1:172.21.15.27:6801/388124754]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-02T04:37:22.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:21 smithi016 ceph-mon[28969]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-02T04:37:22.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:21 smithi016 ceph-mon[28969]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 114 MiB used, 268 GiB / 268 GiB avail 2024-04-02T04:37:22.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:21 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:22.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:21 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:23.055 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-02T04:37:23.055 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-02T04:37:23.055 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10+0 records in 2024-04-02T04:37:23.055 INFO:teuthology.orchestra.run.smithi027.stderr:10+0 records out 2024-04-02T04:37:23.055 INFO:teuthology.orchestra.run.smithi027.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0121344 s, 864 MB/s 2024-04-02T04:37:23.055 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping successful for: 2024-04-02T04:37:23.056 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:22 smithi027 ceph-mon[37550]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-04-02T04:37:23.056 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:22 smithi027 ceph-mon[37550]: osdmap e25: 4 total, 3 up, 4 in 2024-04-02T04:37:23.056 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:22 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T04:37:23.056 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:22 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T04:37:23.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:22 smithi016 ceph-mon[28969]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-04-02T04:37:23.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:22 smithi016 ceph-mon[28969]: osdmap e25: 4 total, 3 up, 4 in 2024-04-02T04:37:23.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:22 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T04:37:23.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:22 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T04:37:23.977 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:23 smithi027 ceph-mon[37550]: purged_snaps scrub starts 2024-04-02T04:37:23.977 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:23 smithi027 ceph-mon[37550]: purged_snaps scrub ok 2024-04-02T04:37:23.977 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:23 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T04:37:23.977 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:23 smithi027 ceph-mon[37550]: osd.3 [v2:172.21.15.27:6800/388124754,v1:172.21.15.27:6801/388124754] boot 2024-04-02T04:37:23.977 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:23 smithi027 ceph-mon[37550]: osdmap e26: 4 total, 4 up, 4 in 2024-04-02T04:37:23.977 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:23 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T04:37:23.978 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:23 smithi027 ceph-mon[37550]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 114 MiB used, 268 GiB / 268 GiB avail 2024-04-02T04:37:23.978 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:23 smithi027 ceph-mon[37550]: osdmap e27: 4 total, 4 up, 4 in 2024-04-02T04:37:24.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:23 smithi016 ceph-mon[28969]: purged_snaps scrub starts 2024-04-02T04:37:24.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:23 smithi016 ceph-mon[28969]: purged_snaps scrub ok 2024-04-02T04:37:24.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:23 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T04:37:24.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:23 smithi016 ceph-mon[28969]: osd.3 [v2:172.21.15.27:6800/388124754,v1:172.21.15.27:6801/388124754] boot 2024-04-02T04:37:24.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:23 smithi016 ceph-mon[28969]: osdmap e26: 4 total, 4 up, 4 in 2024-04-02T04:37:24.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:23 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T04:37:24.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:23 smithi016 ceph-mon[28969]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 114 MiB used, 268 GiB / 268 GiB avail 2024-04-02T04:37:24.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:23 smithi016 ceph-mon[28969]: osdmap e27: 4 total, 4 up, 4 in 2024-04-02T04:37:24.886 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi027:vg_nvme/lv_3 2024-04-02T04:37:25.100 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi027/config 2024-04-02T04:37:25.751 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:25 smithi027 ceph-mon[37550]: osdmap e28: 4 total, 4 up, 4 in 2024-04-02T04:37:25.751 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:25 smithi027 ceph-mon[37550]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 140 MiB used, 357 GiB / 358 GiB avail 2024-04-02T04:37:26.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:25 smithi016 ceph-mon[28969]: osdmap e28: 4 total, 4 up, 4 in 2024-04-02T04:37:26.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:25 smithi016 ceph-mon[28969]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 140 MiB used, 357 GiB / 358 GiB avail 2024-04-02T04:37:26.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:26 smithi027 ceph-mon[37550]: osdmap e29: 4 total, 4 up, 4 in 2024-04-02T04:37:26.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:26 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:26.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:26 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:26.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:26 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:37:26.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:26 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:26.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:26 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:37:26.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:26 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:37:26.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:26 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:26.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:26 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:37:26.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:26 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:37:26.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:26 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:37:26.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:26 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:27.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:26 smithi016 ceph-mon[28969]: osdmap e29: 4 total, 4 up, 4 in 2024-04-02T04:37:27.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:26 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:27.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:26 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:27.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:26 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:37:27.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:26 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:27.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:26 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:37:27.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:26 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:37:27.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:26 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:27.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:26 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:37:27.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:26 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:37:27.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:26 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:37:27.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:26 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:27.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:27.410+0000 7fc3e832c700 1 -- 172.21.15.27:0/2410074259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3e00ff9d0 msgr2=0x7fc3e00ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:37:27.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:27.410+0000 7fc3e832c700 1 --2- 172.21.15.27:0/2410074259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3e00ff9d0 0x7fc3e00ffdf0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4009910 tx=0x7fc3d4009c20 comp rx=0 tx=0).stop 2024-04-02T04:37:27.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:27.410+0000 7fc3e832c700 1 -- 172.21.15.27:0/2410074259 shutdown_connections 2024-04-02T04:37:27.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:27.410+0000 7fc3e832c700 1 --2- 172.21.15.27:0/2410074259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e0100bd0 0x7fc3e0101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:37:27.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:27.410+0000 7fc3e832c700 1 --2- 172.21.15.27:0/2410074259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3e00ff9d0 0x7fc3e00ffdf0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:37:27.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:27.410+0000 7fc3e832c700 1 -- 172.21.15.27:0/2410074259 >> 172.21.15.27:0/2410074259 conn(0x7fc3e00faf70 msgr2=0x7fc3e00fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:37:27.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:27.410+0000 7fc3e832c700 1 -- 172.21.15.27:0/2410074259 shutdown_connections 2024-04-02T04:37:27.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:27.410+0000 7fc3e832c700 1 -- 172.21.15.27:0/2410074259 wait complete. 2024-04-02T04:37:27.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:27.411+0000 7fc3e832c700 1 Processor -- start 2024-04-02T04:37:27.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:27.411+0000 7fc3e832c700 1 -- start start 2024-04-02T04:37:27.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:27.411+0000 7fc3e832c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e00ff9d0 0x7fc3e0068f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:37:27.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:27.411+0000 7fc3e832c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3e0100bd0 0x7fc3e019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:37:27.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:27.411+0000 7fc3e832c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e019b8a0 con 0x7fc3e00ff9d0 2024-04-02T04:37:27.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:27.411+0000 7fc3e832c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e019b9e0 con 0x7fc3e0100bd0 2024-04-02T04:37:27.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:27.411+0000 7fc3e58c7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3e0100bd0 0x7fc3e019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:37:27.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:27.411+0000 7fc3e58c7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3e0100bd0 0x7fc3e019b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35780/0 (socket says 172.21.15.27:35780) 2024-04-02T04:37:27.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:27.411+0000 7fc3e58c7700 1 -- 172.21.15.27:0/1565531449 learned_addr learned my addr 172.21.15.27:0/1565531449 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-02T04:37:27.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:27.411+0000 7fc3e60c8700 1 --2- 172.21.15.27:0/1565531449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e00ff9d0 0x7fc3e0068f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:37:27.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:27.412+0000 7fc3e58c7700 1 -- 172.21.15.27:0/1565531449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e00ff9d0 msgr2=0x7fc3e0068f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:37:27.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:27.412+0000 7fc3e58c7700 1 --2- 172.21.15.27:0/1565531449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e00ff9d0 0x7fc3e0068f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:37:27.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:27.412+0000 7fc3e58c7700 1 -- 172.21.15.27:0/1565531449 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d40095f0 con 0x7fc3e0100bd0 2024-04-02T04:37:27.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:27.412+0000 7fc3e58c7700 1 --2- 172.21.15.27:0/1565531449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3e0100bd0 0x7fc3e019b360 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc00b4b0 tx=0x7fc3dc00b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:37:27.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:27.412+0000 7fc3d37fe700 1 -- 172.21.15.27:0/1565531449 <== mon.1 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3dc011800 con 0x7fc3e0100bd0 2024-04-02T04:37:27.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:27.412+0000 7fc3d37fe700 1 -- 172.21.15.27:0/1565531449 <== mon.1 v2:172.21.15.27:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc3dc011e20 con 0x7fc3e0100bd0 2024-04-02T04:37:27.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:27.412+0000 7fc3e832c700 1 -- 172.21.15.27:0/1565531449 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3e019bc90 con 0x7fc3e0100bd0 2024-04-02T04:37:27.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:27.412+0000 7fc3d37fe700 1 -- 172.21.15.27:0/1565531449 <== mon.1 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3dc00f2e0 con 0x7fc3e0100bd0 2024-04-02T04:37:27.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:27.412+0000 7fc3e832c700 1 -- 172.21.15.27:0/1565531449 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3e019c260 con 0x7fc3e0100bd0 2024-04-02T04:37:27.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:27.413+0000 7fc3e832c700 1 -- 172.21.15.27:0/1565531449 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3e018f5d0 con 0x7fc3e0100bd0 2024-04-02T04:37:27.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:27.413+0000 7fc3d37fe700 1 -- 172.21.15.27:0/1565531449 <== mon.1 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fc3dc010380 con 0x7fc3e0100bd0 2024-04-02T04:37:27.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:27.414+0000 7fc3d37fe700 1 --2- 172.21.15.27:0/1565531449 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc3cc06cce0 0x7fc3cc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:37:27.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:27.414+0000 7fc3d37fe700 1 -- 172.21.15.27:0/1565531449 <== mon.1 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3774+0+0 (secure 0 0 0) 0x7fc3dc08b790 con 0x7fc3e0100bd0 2024-04-02T04:37:27.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:27.414+0000 7fc3e60c8700 1 --2- 172.21.15.27:0/1565531449 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc3cc06cce0 0x7fc3cc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:37:27.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:27.415+0000 7fc3e60c8700 1 --2- 172.21.15.27:0/1565531449 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc3cc06cce0 0x7fc3cc06f1a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4006010 tx=0x7fc3d400b580 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:37:27.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:27.419+0000 7fc3d37fe700 1 -- 172.21.15.27:0/1565531449 <== mon.1 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc3dc059c90 con 0x7fc3e0100bd0 2024-04-02T04:37:27.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:27.628+0000 7fc3e832c700 1 -- 172.21.15.27:0/1565531449 --> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fc3e002cd20 con 0x7fc3cc06cce0 2024-04-02T04:37:28.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:27 smithi016 ceph-mon[28969]: Detected new or changed devices on smithi027 2024-04-02T04:37:28.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:27 smithi016 ceph-mon[28969]: Adjusting osd_memory_target on smithi027 to 14912M 2024-04-02T04:37:28.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:27 smithi016 ceph-mon[28969]: osdmap e30: 4 total, 4 up, 4 in 2024-04-02T04:37:28.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:27 smithi016 ceph-mon[28969]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-02T04:37:28.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:27 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T04:37:28.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:27 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T04:37:28.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:27 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:37:28.126 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:27 smithi027 ceph-mon[37550]: Detected new or changed devices on smithi027 2024-04-02T04:37:28.127 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:27 smithi027 ceph-mon[37550]: Adjusting osd_memory_target on smithi027 to 14912M 2024-04-02T04:37:28.127 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:27 smithi027 ceph-mon[37550]: osdmap e30: 4 total, 4 up, 4 in 2024-04-02T04:37:28.127 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:27 smithi027 ceph-mon[37550]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-02T04:37:28.127 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:27 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T04:37:28.127 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:27 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T04:37:28.127 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:27 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:37:28.837 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:28 smithi027 ceph-mon[37550]: from='client.24153 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:37:28.838 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:28 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:37:29.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:28 smithi016 ceph-mon[28969]: from='client.24153 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:37:29.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:28 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:37:30.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:29 smithi016 ceph-mon[28969]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-04-02T04:37:30.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:29 smithi027 ceph-mon[37550]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-04-02T04:37:30.912 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:30 smithi027 ceph-mon[37550]: from='client.? 172.21.15.27:0/2018058104' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "334b7b3d-505f-45b6-85b0-59ec8eeb8e91"}]: dispatch 2024-04-02T04:37:30.912 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:30 smithi027 ceph-mon[37550]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "334b7b3d-505f-45b6-85b0-59ec8eeb8e91"}]: dispatch 2024-04-02T04:37:30.912 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:30 smithi027 ceph-mon[37550]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "334b7b3d-505f-45b6-85b0-59ec8eeb8e91"}]': finished 2024-04-02T04:37:30.912 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:30 smithi027 ceph-mon[37550]: osdmap e31: 5 total, 4 up, 5 in 2024-04-02T04:37:30.912 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:30 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T04:37:31.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:30 smithi016 ceph-mon[28969]: from='client.? 172.21.15.27:0/2018058104' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "334b7b3d-505f-45b6-85b0-59ec8eeb8e91"}]: dispatch 2024-04-02T04:37:31.097 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:30 smithi016 ceph-mon[28969]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "334b7b3d-505f-45b6-85b0-59ec8eeb8e91"}]: dispatch 2024-04-02T04:37:31.097 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:30 smithi016 ceph-mon[28969]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "334b7b3d-505f-45b6-85b0-59ec8eeb8e91"}]': finished 2024-04-02T04:37:31.098 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:30 smithi016 ceph-mon[28969]: osdmap e31: 5 total, 4 up, 5 in 2024-04-02T04:37:31.098 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:30 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T04:37:32.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:31 smithi016 ceph-mon[28969]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-02T04:37:32.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:31 smithi016 ceph-mon[28969]: from='client.? 172.21.15.27:0/997125733' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T04:37:32.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:31 smithi027 ceph-mon[37550]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-02T04:37:32.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:31 smithi027 ceph-mon[37550]: from='client.? 172.21.15.27:0/997125733' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T04:37:34.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:33 smithi016 ceph-mon[28969]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 357 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-04-02T04:37:34.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:33 smithi027 ceph-mon[37550]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 357 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-04-02T04:37:35.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:35 smithi027 ceph-mon[37550]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-02T04:37:36.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:35 smithi016 ceph-mon[28969]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-02T04:37:37.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:37 smithi027 ceph-mon[37550]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 357 GiB / 358 GiB avail 2024-04-02T04:37:37.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:37 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-02T04:37:37.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:37 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:37:38.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:37 smithi016 ceph-mon[28969]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 357 GiB / 358 GiB avail 2024-04-02T04:37:38.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:37 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-02T04:37:38.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:37 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:37:39.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:38 smithi016 ceph-mon[28969]: Deploying daemon osd.4 on smithi027 2024-04-02T04:37:39.125 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:38 smithi027 ceph-mon[37550]: Deploying daemon osd.4 on smithi027 2024-04-02T04:37:40.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:39 smithi016 ceph-mon[28969]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 357 GiB / 358 GiB avail 2024-04-02T04:37:40.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:39 smithi027 ceph-mon[37550]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 357 GiB / 358 GiB avail 2024-04-02T04:37:42.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:41 smithi016 ceph-mon[28969]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 357 GiB / 358 GiB avail 2024-04-02T04:37:42.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:41 smithi027 ceph-mon[37550]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 357 GiB / 358 GiB avail 2024-04-02T04:37:43.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:42 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:37:43.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:42 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:37:44.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:43 smithi027 ceph-mon[37550]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 357 GiB / 358 GiB avail 2024-04-02T04:37:44.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:43 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:44.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:43 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:37:44.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:43 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:44.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:43 smithi016 ceph-mon[28969]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 357 GiB / 358 GiB avail 2024-04-02T04:37:44.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:43 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:44.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:43 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:37:44.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:43 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:45.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:45 smithi027 ceph-mon[37550]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 357 GiB / 358 GiB avail 2024-04-02T04:37:46.111 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:45 smithi016 ceph-mon[28969]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 357 GiB / 358 GiB avail 2024-04-02T04:37:46.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:46.392+0000 7fc3d37fe700 1 -- 172.21.15.27:0/1565531449 <== mgr.14217 v2:172.21.15.16:6800/3643600900 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc3e002cd20 con 0x7fc3cc06cce0 2024-04-02T04:37:46.396 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 4 on host 'smithi027' 2024-04-02T04:37:46.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:46.395+0000 7fc3e832c700 1 -- 172.21.15.27:0/1565531449 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc3cc06cce0 msgr2=0x7fc3cc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:37:46.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:46.395+0000 7fc3e832c700 1 --2- 172.21.15.27:0/1565531449 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc3cc06cce0 0x7fc3cc06f1a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4006010 tx=0x7fc3d400b580 comp rx=0 tx=0).stop 2024-04-02T04:37:46.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:46.395+0000 7fc3e832c700 1 -- 172.21.15.27:0/1565531449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3e0100bd0 msgr2=0x7fc3e019b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:37:46.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:46.395+0000 7fc3e832c700 1 --2- 172.21.15.27:0/1565531449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3e0100bd0 0x7fc3e019b360 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc00b4b0 tx=0x7fc3dc00b870 comp rx=0 tx=0).stop 2024-04-02T04:37:46.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:46.395+0000 7fc3e832c700 1 -- 172.21.15.27:0/1565531449 shutdown_connections 2024-04-02T04:37:46.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:46.395+0000 7fc3e832c700 1 --2- 172.21.15.27:0/1565531449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e00ff9d0 0x7fc3e0068f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:37:46.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:46.395+0000 7fc3e832c700 1 --2- 172.21.15.27:0/1565531449 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc3cc06cce0 0x7fc3cc06f1a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:37:46.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:46.395+0000 7fc3e832c700 1 --2- 172.21.15.27:0/1565531449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3e0100bd0 0x7fc3e019b360 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:37:46.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:46.395+0000 7fc3e832c700 1 -- 172.21.15.27:0/1565531449 >> 172.21.15.27:0/1565531449 conn(0x7fc3e00faf70 msgr2=0x7fc3e01042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:37:46.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:46.395+0000 7fc3e832c700 1 -- 172.21.15.27:0/1565531449 shutdown_connections 2024-04-02T04:37:46.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:46.395+0000 7fc3e832c700 1 -- 172.21.15.27:0/1565531449 wait complete. 2024-04-02T04:37:47.044 DEBUG:teuthology.orchestra.run.smithi027:osd.4> sudo journalctl -f -n 0 -u ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@osd.4.service 2024-04-02T04:37:47.047 INFO:tasks.cephadm:Deploying osd.5 on smithi027 with /dev/vg_nvme/lv_2... 2024-04-02T04:37:47.047 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-02T04:37:47.645 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:47 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:47.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:47 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:47.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:47 smithi027 ceph-mon[37550]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 357 GiB / 358 GiB avail 2024-04-02T04:37:47.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:47 smithi027 ceph-mon[37550]: from='osd.4 [v2:172.21.15.27:6808/935993381,v1:172.21.15.27:6809/935993381]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T04:37:47.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:47 smithi027 ceph-mon[37550]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T04:37:47.777 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi027/config 2024-04-02T04:37:47.809 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:47 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:47.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:47 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:47.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:47 smithi016 ceph-mon[28969]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 357 GiB / 358 GiB avail 2024-04-02T04:37:47.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:47 smithi016 ceph-mon[28969]: from='osd.4 [v2:172.21.15.27:6808/935993381,v1:172.21.15.27:6809/935993381]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T04:37:47.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:47 smithi016 ceph-mon[28969]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T04:37:48.645 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:48 smithi027 ceph-mon[37550]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-02T04:37:48.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:48 smithi027 ceph-mon[37550]: osdmap e32: 5 total, 4 up, 5 in 2024-04-02T04:37:48.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:48 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T04:37:48.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:48 smithi027 ceph-mon[37550]: from='osd.4 [v2:172.21.15.27:6808/935993381,v1:172.21.15.27:6809/935993381]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-02T04:37:48.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:48 smithi027 ceph-mon[37550]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-02T04:37:48.809 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:48 smithi016 ceph-mon[28969]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-02T04:37:48.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:48 smithi016 ceph-mon[28969]: osdmap e32: 5 total, 4 up, 5 in 2024-04-02T04:37:48.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:48 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T04:37:48.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:48 smithi016 ceph-mon[28969]: from='osd.4 [v2:172.21.15.27:6808/935993381,v1:172.21.15.27:6809/935993381]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-02T04:37:48.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:48 smithi016 ceph-mon[28969]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-02T04:37:49.579 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:37:49 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4[50292]: 2024-04-02T04:37:49.211+0000 7f0570bcc700 -1 osd.4 0 waiting for initial osdmap 2024-04-02T04:37:49.579 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:37:49 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4[50292]: 2024-04-02T04:37:49.222+0000 7f05691ba700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T04:37:49.579 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:49 smithi027 ceph-mon[37550]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-04-02T04:37:49.579 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:49 smithi027 ceph-mon[37550]: osdmap e33: 5 total, 4 up, 5 in 2024-04-02T04:37:49.579 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:49 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T04:37:49.580 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:49 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T04:37:49.580 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:49 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:49.580 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:49 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:49.580 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:49 smithi027 ceph-mon[37550]: pgmap v114: 1 pgs: 1 peering; 577 KiB data, 140 MiB used, 357 GiB / 358 GiB avail 2024-04-02T04:37:49.580 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:49 smithi027 ceph-mon[37550]: from='osd.4 ' entity='osd.4' 2024-04-02T04:37:49.580 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:49 smithi027 ceph-mon[37550]: osd.4 [v2:172.21.15.27:6808/935993381,v1:172.21.15.27:6809/935993381] boot 2024-04-02T04:37:49.580 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:49 smithi027 ceph-mon[37550]: osdmap e34: 5 total, 5 up, 5 in 2024-04-02T04:37:49.580 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:49 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T04:37:49.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:49 smithi016 ceph-mon[28969]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-04-02T04:37:49.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:49 smithi016 ceph-mon[28969]: osdmap e33: 5 total, 4 up, 5 in 2024-04-02T04:37:49.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:49 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T04:37:49.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:49 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T04:37:49.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:49 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:49.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:49 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:49.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:49 smithi016 ceph-mon[28969]: pgmap v114: 1 pgs: 1 peering; 577 KiB data, 140 MiB used, 357 GiB / 358 GiB avail 2024-04-02T04:37:49.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:49 smithi016 ceph-mon[28969]: from='osd.4 ' entity='osd.4' 2024-04-02T04:37:49.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:49 smithi016 ceph-mon[28969]: osd.4 [v2:172.21.15.27:6808/935993381,v1:172.21.15.27:6809/935993381] boot 2024-04-02T04:37:49.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:49 smithi016 ceph-mon[28969]: osdmap e34: 5 total, 5 up, 5 in 2024-04-02T04:37:49.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:49 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T04:37:49.827 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-02T04:37:49.827 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-02T04:37:49.827 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10+0 records in 2024-04-02T04:37:49.827 INFO:teuthology.orchestra.run.smithi027.stderr:10+0 records out 2024-04-02T04:37:49.828 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0121516 s, 863 MB/s 2024-04-02T04:37:49.828 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping successful for: 2024-04-02T04:37:50.741 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi027:vg_nvme/lv_2 2024-04-02T04:37:50.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:50 smithi027 ceph-mon[37550]: purged_snaps scrub starts 2024-04-02T04:37:50.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:50 smithi027 ceph-mon[37550]: purged_snaps scrub ok 2024-04-02T04:37:50.989 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:50 smithi016 ceph-mon[28969]: purged_snaps scrub starts 2024-04-02T04:37:50.989 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:50 smithi016 ceph-mon[28969]: purged_snaps scrub ok 2024-04-02T04:37:51.202 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi027/config 2024-04-02T04:37:52.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:51 smithi016 ceph-mon[28969]: osdmap e35: 5 total, 5 up, 5 in 2024-04-02T04:37:52.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:51 smithi016 ceph-mon[28969]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 572 MiB used, 446 GiB / 447 GiB avail 2024-04-02T04:37:52.078 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:51 smithi027 ceph-mon[37550]: osdmap e35: 5 total, 5 up, 5 in 2024-04-02T04:37:52.078 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:51 smithi027 ceph-mon[37550]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 572 MiB used, 446 GiB / 447 GiB avail 2024-04-02T04:37:53.874 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:53 smithi027 ceph-mon[37550]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 572 MiB used, 446 GiB / 447 GiB avail 2024-04-02T04:37:53.874 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:53 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:53.874 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:53 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:53.874 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:53 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:37:53.874 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:53 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:37:53.874 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:53 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:53.874 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:53 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:37:53.874 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:53 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:37:53.875 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:53 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:53.875 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:53 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:37:53.875 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:53 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:37:53.875 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:53 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:37:53.875 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:53 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:53.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:53.979+0000 7ff063a26700 1 -- 172.21.15.27:0/641158843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff05c100be0 msgr2=0x7ff05c101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:37:53.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:53.979+0000 7ff063a26700 1 --2- 172.21.15.27:0/641158843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff05c100be0 0x7ff05c101040 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff04c009910 tx=0x7ff04c009c20 comp rx=0 tx=0).stop 2024-04-02T04:37:53.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:53.979+0000 7ff063a26700 1 -- 172.21.15.27:0/641158843 shutdown_connections 2024-04-02T04:37:53.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:53.979+0000 7ff063a26700 1 --2- 172.21.15.27:0/641158843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff05c100be0 0x7ff05c101040 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:37:53.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:53.979+0000 7ff063a26700 1 --2- 172.21.15.27:0/641158843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff05c0ff9e0 0x7ff05c0ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:37:53.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:53.979+0000 7ff063a26700 1 -- 172.21.15.27:0/641158843 >> 172.21.15.27:0/641158843 conn(0x7ff05c0faf80 msgr2=0x7ff05c0fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:37:53.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:53.979+0000 7ff063a26700 1 -- 172.21.15.27:0/641158843 shutdown_connections 2024-04-02T04:37:53.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:53.979+0000 7ff063a26700 1 -- 172.21.15.27:0/641158843 wait complete. 2024-04-02T04:37:53.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:53.979+0000 7ff063a26700 1 Processor -- start 2024-04-02T04:37:53.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:53.980+0000 7ff063a26700 1 -- start start 2024-04-02T04:37:53.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:53.980+0000 7ff063a26700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff05c0ff9e0 0x7ff05c069af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:37:53.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:53.980+0000 7ff063a26700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff05c100be0 0x7ff05c19d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:37:53.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:53.980+0000 7ff063a26700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05c197170 con 0x7ff05c100be0 2024-04-02T04:37:53.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:53.980+0000 7ff063a26700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05c19dcd0 con 0x7ff05c0ff9e0 2024-04-02T04:37:53.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:53.980+0000 7ff0617c2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff05c0ff9e0 0x7ff05c069af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:37:53.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:53.980+0000 7ff0617c2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff05c0ff9e0 0x7ff05c069af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47048/0 (socket says 172.21.15.27:47048) 2024-04-02T04:37:53.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:53.980+0000 7ff0617c2700 1 -- 172.21.15.27:0/429110960 learned_addr learned my addr 172.21.15.27:0/429110960 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-02T04:37:53.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:53.980+0000 7ff060fc1700 1 --2- 172.21.15.27:0/429110960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff05c100be0 0x7ff05c19d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:37:53.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:53.980+0000 7ff0617c2700 1 -- 172.21.15.27:0/429110960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff05c100be0 msgr2=0x7ff05c19d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:37:53.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:53.980+0000 7ff0617c2700 1 --2- 172.21.15.27:0/429110960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff05c100be0 0x7ff05c19d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:37:53.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:53.980+0000 7ff0617c2700 1 -- 172.21.15.27:0/429110960 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff04c0095f0 con 0x7ff05c0ff9e0 2024-04-02T04:37:53.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:53.980+0000 7ff060fc1700 1 --2- 172.21.15.27:0/429110960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff05c100be0 0x7ff05c19d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T04:37:53.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:53.980+0000 7ff0617c2700 1 --2- 172.21.15.27:0/429110960 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff05c0ff9e0 0x7ff05c069af0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff05800b480 tx=0x7ff05800b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:37:53.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:53.980+0000 7ff0527fc700 1 -- 172.21.15.27:0/429110960 <== mon.1 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0580118c0 con 0x7ff05c0ff9e0 2024-04-02T04:37:53.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:53.981+0000 7ff063a26700 1 -- 172.21.15.27:0/429110960 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff05c19ded0 con 0x7ff05c0ff9e0 2024-04-02T04:37:53.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:53.981+0000 7ff0527fc700 1 -- 172.21.15.27:0/429110960 <== mon.1 v2:172.21.15.27:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff058010380 con 0x7ff05c0ff9e0 2024-04-02T04:37:53.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:53.981+0000 7ff0527fc700 1 -- 172.21.15.27:0/429110960 <== mon.1 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff05800f4f0 con 0x7ff05c0ff9e0 2024-04-02T04:37:53.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:53.981+0000 7ff063a26700 1 -- 172.21.15.27:0/429110960 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff05c19e3c0 con 0x7ff05c0ff9e0 2024-04-02T04:37:53.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:53.982+0000 7ff063a26700 1 -- 172.21.15.27:0/429110960 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff05c18f600 con 0x7ff05c0ff9e0 2024-04-02T04:37:53.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:53.982+0000 7ff0527fc700 1 -- 172.21.15.27:0/429110960 <== mon.1 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7ff0580104f0 con 0x7ff05c0ff9e0 2024-04-02T04:37:53.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:53.982+0000 7ff0527fc700 1 --2- 172.21.15.27:0/429110960 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7ff04806cce0 0x7ff04806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:37:53.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:53.983+0000 7ff0527fc700 1 -- 172.21.15.27:0/429110960 <== mon.1 v2:172.21.15.27:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4206+0+0 (secure 0 0 0) 0x7ff05808b9d0 con 0x7ff05c0ff9e0 2024-04-02T04:37:53.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:53.983+0000 7ff060fc1700 1 --2- 172.21.15.27:0/429110960 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7ff04806cce0 0x7ff04806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:37:53.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:53.983+0000 7ff060fc1700 1 --2- 172.21.15.27:0/429110960 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7ff04806cce0 0x7ff04806f1a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff04c0051c0 tx=0x7ff04c005920 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:37:53.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:53.987+0000 7ff0527fc700 1 -- 172.21.15.27:0/429110960 <== mon.1 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff058059d20 con 0x7ff05c0ff9e0 2024-04-02T04:37:54.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:53 smithi016 ceph-mon[28969]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 572 MiB used, 446 GiB / 447 GiB avail 2024-04-02T04:37:54.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:53 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:54.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:53 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:54.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:53 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:37:54.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:53 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:37:54.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:53 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:54.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:53 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:37:54.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:53 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:37:54.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:53 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:54.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:53 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:37:54.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:53 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:37:54.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:53 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:37:54.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:53 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:37:54.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:37:54.199+0000 7ff063a26700 1 -- 172.21.15.27:0/429110960 --> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7ff05c02ce10 con 0x7ff04806cce0 2024-04-02T04:37:55.017 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:54 smithi027 ceph-mon[37550]: Detected new or changed devices on smithi027 2024-04-02T04:37:55.017 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:54 smithi027 ceph-mon[37550]: Adjusting osd_memory_target on smithi027 to 7456M 2024-04-02T04:37:55.017 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:54 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T04:37:55.017 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:54 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T04:37:55.017 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:54 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:37:55.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:54 smithi016 ceph-mon[28969]: Detected new or changed devices on smithi027 2024-04-02T04:37:55.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:54 smithi016 ceph-mon[28969]: Adjusting osd_memory_target on smithi027 to 7456M 2024-04-02T04:37:55.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:54 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T04:37:55.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:54 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T04:37:55.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:54 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:37:55.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:55 smithi027 ceph-mon[37550]: from='client.24171 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:37:55.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:55 smithi027 ceph-mon[37550]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:37:56.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:55 smithi016 ceph-mon[28969]: from='client.24171 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:37:56.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:55 smithi016 ceph-mon[28969]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:37:57.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:56 smithi027 ceph-mon[37550]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:37:57.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:56 smithi027 ceph-mon[37550]: from='client.? 172.21.15.27:0/978995314' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f83802ee-4f47-47de-91dd-a667f6cd22e2"}]: dispatch 2024-04-02T04:37:57.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:56 smithi027 ceph-mon[37550]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f83802ee-4f47-47de-91dd-a667f6cd22e2"}]: dispatch 2024-04-02T04:37:57.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:56 smithi027 ceph-mon[37550]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f83802ee-4f47-47de-91dd-a667f6cd22e2"}]': finished 2024-04-02T04:37:57.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:56 smithi027 ceph-mon[37550]: osdmap e36: 6 total, 5 up, 6 in 2024-04-02T04:37:57.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:56 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T04:37:57.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:56 smithi016 ceph-mon[28969]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:37:57.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:56 smithi016 ceph-mon[28969]: from='client.? 172.21.15.27:0/978995314' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f83802ee-4f47-47de-91dd-a667f6cd22e2"}]: dispatch 2024-04-02T04:37:57.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:56 smithi016 ceph-mon[28969]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f83802ee-4f47-47de-91dd-a667f6cd22e2"}]: dispatch 2024-04-02T04:37:57.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:56 smithi016 ceph-mon[28969]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f83802ee-4f47-47de-91dd-a667f6cd22e2"}]': finished 2024-04-02T04:37:57.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:56 smithi016 ceph-mon[28969]: osdmap e36: 6 total, 5 up, 6 in 2024-04-02T04:37:57.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:56 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T04:37:58.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:57 smithi027 ceph-mon[37550]: from='client.? 172.21.15.27:0/863035102' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T04:37:58.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:57 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:37:58.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:57 smithi016 ceph-mon[28969]: from='client.? 172.21.15.27:0/863035102' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T04:37:58.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:57 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:38:00.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:37:59 smithi016 ceph-mon[28969]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:38:00.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:37:59 smithi027 ceph-mon[37550]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:38:02.055 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:01 smithi027 ceph-mon[37550]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:38:02.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:01 smithi016 ceph-mon[28969]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:38:04.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:03 smithi016 ceph-mon[28969]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:38:04.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:03 smithi027 ceph-mon[37550]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:38:05.069 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:04 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-02T04:38:05.069 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:04 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:38:05.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:04 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-02T04:38:05.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:04 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:38:05.991 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:05 smithi027 ceph-mon[37550]: Deploying daemon osd.5 on smithi027 2024-04-02T04:38:05.991 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:05 smithi027 ceph-mon[37550]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:38:06.106 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:05 smithi016 ceph-mon[28969]: Deploying daemon osd.5 on smithi027 2024-04-02T04:38:06.106 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:05 smithi016 ceph-mon[28969]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:38:08.087 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:07 smithi027 ceph-mon[37550]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:38:08.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:07 smithi016 ceph-mon[28969]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:38:09.813 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:09 smithi027 ceph-mon[37550]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:38:10.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:09 smithi016 ceph-mon[28969]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:38:11.392 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:11 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:38:11.393 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:11 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:38:11.393 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:11 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:38:11.393 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:11 smithi016 ceph-mon[28969]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:38:11.395 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:11 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:38:11.395 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:11 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:38:11.395 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:11 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:38:11.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:11 smithi027 ceph-mon[37550]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:38:13.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:12 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:38:13.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:12 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:38:13.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:13 smithi027 ceph-mon[37550]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:38:14.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:14.017+0000 7ff0527fc700 1 -- 172.21.15.27:0/429110960 <== mgr.14217 v2:172.21.15.16:6800/3643600900 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff05c02ce10 con 0x7ff04806cce0 2024-04-02T04:38:14.020 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 5 on host 'smithi027' 2024-04-02T04:38:14.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:14.021+0000 7ff063a26700 1 -- 172.21.15.27:0/429110960 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7ff04806cce0 msgr2=0x7ff04806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:14.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:14.021+0000 7ff063a26700 1 --2- 172.21.15.27:0/429110960 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7ff04806cce0 0x7ff04806f1a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff04c0051c0 tx=0x7ff04c005920 comp rx=0 tx=0).stop 2024-04-02T04:38:14.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:14.021+0000 7ff063a26700 1 -- 172.21.15.27:0/429110960 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff05c0ff9e0 msgr2=0x7ff05c069af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:14.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:14.021+0000 7ff063a26700 1 --2- 172.21.15.27:0/429110960 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff05c0ff9e0 0x7ff05c069af0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff05800b480 tx=0x7ff05800b840 comp rx=0 tx=0).stop 2024-04-02T04:38:14.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:14.021+0000 7ff063a26700 1 -- 172.21.15.27:0/429110960 shutdown_connections 2024-04-02T04:38:14.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:14.021+0000 7ff063a26700 1 --2- 172.21.15.27:0/429110960 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7ff04806cce0 0x7ff04806f1a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:14.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:14.021+0000 7ff063a26700 1 --2- 172.21.15.27:0/429110960 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff05c0ff9e0 0x7ff05c069af0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:14.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:14.022+0000 7ff063a26700 1 --2- 172.21.15.27:0/429110960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff05c100be0 0x7ff05c19d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:14.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:14.022+0000 7ff063a26700 1 -- 172.21.15.27:0/429110960 >> 172.21.15.27:0/429110960 conn(0x7ff05c0faf80 msgr2=0x7ff05c104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:14.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:14.022+0000 7ff063a26700 1 -- 172.21.15.27:0/429110960 shutdown_connections 2024-04-02T04:38:14.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:14.022+0000 7ff063a26700 1 -- 172.21.15.27:0/429110960 wait complete. 2024-04-02T04:38:14.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:13 smithi016 ceph-mon[28969]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:38:14.838 DEBUG:teuthology.orchestra.run.smithi027:osd.5> sudo journalctl -f -n 0 -u ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@osd.5.service 2024-04-02T04:38:14.880 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-04-02T04:38:14.880 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph osd stat -f json 2024-04-02T04:38:15.106 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:38:15.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:14 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:38:15.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:14 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:38:15.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:14 smithi027 ceph-mon[37550]: from='osd.5 [v2:172.21.15.27:6816/2454420135,v1:172.21.15.27:6817/2454420135]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T04:38:15.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:14 smithi027 ceph-mon[37550]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T04:38:15.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:14 smithi027 ceph-mon[37550]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:38:15.228 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:14 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:38:15.228 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:14 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:38:15.228 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:14 smithi016 ceph-mon[28969]: from='osd.5 [v2:172.21.15.27:6816/2454420135,v1:172.21.15.27:6817/2454420135]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T04:38:15.228 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:14 smithi016 ceph-mon[28969]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T04:38:15.228 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:14 smithi016 ceph-mon[28969]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:38:16.231 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:15 smithi016 ceph-mon[28969]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-02T04:38:16.232 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:15 smithi016 ceph-mon[28969]: osdmap e37: 6 total, 5 up, 6 in 2024-04-02T04:38:16.232 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:15 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T04:38:16.232 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:15 smithi016 ceph-mon[28969]: from='osd.5 [v2:172.21.15.27:6816/2454420135,v1:172.21.15.27:6817/2454420135]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-02T04:38:16.232 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:15 smithi016 ceph-mon[28969]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-02T04:38:16.232 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:15 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:38:16.232 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:15 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:38:16.275 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:15 smithi027 ceph-mon[37550]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-02T04:38:16.275 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:15 smithi027 ceph-mon[37550]: osdmap e37: 6 total, 5 up, 6 in 2024-04-02T04:38:16.276 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:15 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T04:38:16.276 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:15 smithi027 ceph-mon[37550]: from='osd.5 [v2:172.21.15.27:6816/2454420135,v1:172.21.15.27:6817/2454420135]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-02T04:38:16.276 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:15 smithi027 ceph-mon[37550]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-02T04:38:16.276 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:15 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:38:16.276 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:15 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:38:16.276 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:38:15 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[56824]: 2024-04-02T04:38:15.980+0000 7fa429534700 -1 osd.5 0 waiting for initial osdmap 2024-04-02T04:38:16.276 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:38:15 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[56824]: 2024-04-02T04:38:15.987+0000 7fa425b2a700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T04:38:16.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:16.829+0000 7f816ca3e700 1 -- 172.21.15.16:0/1753321532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8168101ce0 msgr2=0x7f8168102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:16.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:16.829+0000 7f816ca3e700 1 --2- 172.21.15.16:0/1753321532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8168101ce0 0x7f8168102140 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8150009910 tx=0x7f8150009c20 comp rx=0 tx=0).stop 2024-04-02T04:38:16.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:16.829+0000 7f816ca3e700 1 -- 172.21.15.16:0/1753321532 shutdown_connections 2024-04-02T04:38:16.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:16.829+0000 7f816ca3e700 1 --2- 172.21.15.16:0/1753321532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8168101ce0 0x7f8168102140 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:16.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:16.829+0000 7f816ca3e700 1 --2- 172.21.15.16:0/1753321532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8168100ae0 0x7f8168100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:16.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:16.829+0000 7f816ca3e700 1 -- 172.21.15.16:0/1753321532 >> 172.21.15.16:0/1753321532 conn(0x7f81680fc0a0 msgr2=0x7f81680fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:16.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:16.829+0000 7f816ca3e700 1 -- 172.21.15.16:0/1753321532 shutdown_connections 2024-04-02T04:38:16.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:16.829+0000 7f816ca3e700 1 -- 172.21.15.16:0/1753321532 wait complete. 2024-04-02T04:38:16.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:16.830+0000 7f816ca3e700 1 Processor -- start 2024-04-02T04:38:16.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:16.830+0000 7f816ca3e700 1 -- start start 2024-04-02T04:38:16.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:16.830+0000 7f816ca3e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8168100ae0 0x7f8168074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:16.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:16.830+0000 7f816ca3e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8168101ce0 0x7f81680731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:16.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:16.830+0000 7f816ca3e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81680736e0 con 0x7f8168101ce0 2024-04-02T04:38:16.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:16.830+0000 7f816ca3e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8168073820 con 0x7f8168100ae0 2024-04-02T04:38:16.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:16.830+0000 7f8165d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8168101ce0 0x7f81680731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:16.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:16.830+0000 7f816659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8168100ae0 0x7f8168074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:16.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:16.830+0000 7f8165d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8168101ce0 0x7f81680731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48542/0 (socket says 172.21.15.16:48542) 2024-04-02T04:38:16.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:16.830+0000 7f8165d9b700 1 -- 172.21.15.16:0/1212721236 learned_addr learned my addr 172.21.15.16:0/1212721236 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:38:16.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:16.831+0000 7f8165d9b700 1 -- 172.21.15.16:0/1212721236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8168100ae0 msgr2=0x7f8168074b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:16.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:16.831+0000 7f8165d9b700 1 --2- 172.21.15.16:0/1212721236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8168100ae0 0x7f8168074b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:16.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:16.831+0000 7f8165d9b700 1 -- 172.21.15.16:0/1212721236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81500095f0 con 0x7f8168101ce0 2024-04-02T04:38:16.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:16.831+0000 7f816659c700 1 --2- 172.21.15.16:0/1212721236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8168100ae0 0x7f8168074b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T04:38:16.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:16.831+0000 7f8165d9b700 1 --2- 172.21.15.16:0/1212721236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8168101ce0 0x7f81680731a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f81500092e0 tx=0x7f815000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:16.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:16.831+0000 7f815f7fe700 1 -- 172.21.15.16:0/1212721236 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f815001d070 con 0x7f8168101ce0 2024-04-02T04:38:16.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:16.831+0000 7f816ca3e700 1 -- 172.21.15.16:0/1212721236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8168073aa0 con 0x7f8168101ce0 2024-04-02T04:38:16.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:16.831+0000 7f815f7fe700 1 -- 172.21.15.16:0/1212721236 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f815000f460 con 0x7f8168101ce0 2024-04-02T04:38:16.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:16.831+0000 7f815f7fe700 1 -- 172.21.15.16:0/1212721236 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8150021640 con 0x7f8168101ce0 2024-04-02T04:38:16.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:16.831+0000 7f816ca3e700 1 -- 172.21.15.16:0/1212721236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8168074010 con 0x7f8168101ce0 2024-04-02T04:38:16.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:16.832+0000 7f816ca3e700 1 -- 172.21.15.16:0/1212721236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8168190700 con 0x7f8168101ce0 2024-04-02T04:38:16.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:16.833+0000 7f815f7fe700 1 -- 172.21.15.16:0/1212721236 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f815000f5d0 con 0x7f8168101ce0 2024-04-02T04:38:16.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:16.833+0000 7f815f7fe700 1 --2- 172.21.15.16:0/1212721236 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f815406cce0 0x7f815406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:16.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:16.833+0000 7f816659c700 1 --2- 172.21.15.16:0/1212721236 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f815406cce0 0x7f815406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:16.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:16.833+0000 7f815f7fe700 1 -- 172.21.15.16:0/1212721236 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4358+0+0 (secure 0 0 0) 0x7f815008cfa0 con 0x7f8168101ce0 2024-04-02T04:38:16.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:16.834+0000 7f816659c700 1 --2- 172.21.15.16:0/1212721236 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f815406cce0 0x7f815406f1a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8168101b40 tx=0x7f8158009190 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:16.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:16.838+0000 7f815f7fe700 1 -- 172.21.15.16:0/1212721236 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f815005b250 con 0x7f8168101ce0 2024-04-02T04:38:17.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:17.039+0000 7f816ca3e700 1 -- 172.21.15.16:0/1212721236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f8168074010 con 0x7f8168101ce0 2024-04-02T04:38:17.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:17.039+0000 7f815f7fe700 1 -- 172.21.15.16:0/1212721236 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v39) v1 ==== 74+0+130 (secure 0 0 0) 0x7f8168074010 con 0x7f8168101ce0 2024-04-02T04:38:17.042 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:38:17.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:17.041+0000 7f816ca3e700 1 -- 172.21.15.16:0/1212721236 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f815406cce0 msgr2=0x7f815406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:17.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:17.041+0000 7f816ca3e700 1 --2- 172.21.15.16:0/1212721236 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f815406cce0 0x7f815406f1a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8168101b40 tx=0x7f8158009190 comp rx=0 tx=0).stop 2024-04-02T04:38:17.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:17.041+0000 7f816ca3e700 1 -- 172.21.15.16:0/1212721236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8168101ce0 msgr2=0x7f81680731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:17.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:17.041+0000 7f816ca3e700 1 --2- 172.21.15.16:0/1212721236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8168101ce0 0x7f81680731a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f81500092e0 tx=0x7f815000bd70 comp rx=0 tx=0).stop 2024-04-02T04:38:17.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:17.041+0000 7f816ca3e700 1 -- 172.21.15.16:0/1212721236 shutdown_connections 2024-04-02T04:38:17.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:17.041+0000 7f816ca3e700 1 --2- 172.21.15.16:0/1212721236 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f815406cce0 0x7f815406f1a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:17.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:17.041+0000 7f816ca3e700 1 --2- 172.21.15.16:0/1212721236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8168100ae0 0x7f8168074b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:17.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:17.041+0000 7f816ca3e700 1 --2- 172.21.15.16:0/1212721236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8168101ce0 0x7f81680731a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:17.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:17.041+0000 7f816ca3e700 1 -- 172.21.15.16:0/1212721236 >> 172.21.15.16:0/1212721236 conn(0x7f81680fc0a0 msgr2=0x7f8168105330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:17.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:17.041+0000 7f816ca3e700 1 -- 172.21.15.16:0/1212721236 shutdown_connections 2024-04-02T04:38:17.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:17.041+0000 7f816ca3e700 1 -- 172.21.15.16:0/1212721236 wait complete. 2024-04-02T04:38:17.071 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:16 smithi016 ceph-mon[28969]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-04-02T04:38:17.071 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:16 smithi016 ceph-mon[28969]: osdmap e38: 6 total, 5 up, 6 in 2024-04-02T04:38:17.071 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:16 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T04:38:17.071 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:16 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T04:38:17.072 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:16 smithi016 ceph-mon[28969]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:38:17.337 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:16 smithi027 ceph-mon[37550]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-04-02T04:38:17.337 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:16 smithi027 ceph-mon[37550]: osdmap e38: 6 total, 5 up, 6 in 2024-04-02T04:38:17.337 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:16 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T04:38:17.337 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:16 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T04:38:17.337 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:16 smithi027 ceph-mon[37550]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 447 GiB / 447 GiB avail 2024-04-02T04:38:17.798 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":39,"num_osds":6,"num_up_osds":6,"osd_up_since":1712032696,"num_in_osds":6,"osd_in_since":1712032676,"num_remapped_pgs":0} 2024-04-02T04:38:17.798 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph osd dump --format=json 2024-04-02T04:38:18.019 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:38:18.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:17 smithi016 ceph-mon[28969]: purged_snaps scrub starts 2024-04-02T04:38:18.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:17 smithi016 ceph-mon[28969]: purged_snaps scrub ok 2024-04-02T04:38:18.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:17 smithi016 ceph-mon[28969]: osd.5 [v2:172.21.15.27:6816/2454420135,v1:172.21.15.27:6817/2454420135] boot 2024-04-02T04:38:18.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:17 smithi016 ceph-mon[28969]: osdmap e39: 6 total, 6 up, 6 in 2024-04-02T04:38:18.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:17 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T04:38:18.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:17 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/1212721236' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-02T04:38:18.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:17 smithi027 ceph-mon[37550]: purged_snaps scrub starts 2024-04-02T04:38:18.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:17 smithi027 ceph-mon[37550]: purged_snaps scrub ok 2024-04-02T04:38:18.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:17 smithi027 ceph-mon[37550]: osd.5 [v2:172.21.15.27:6816/2454420135,v1:172.21.15.27:6817/2454420135] boot 2024-04-02T04:38:18.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:17 smithi027 ceph-mon[37550]: osdmap e39: 6 total, 6 up, 6 in 2024-04-02T04:38:18.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:17 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T04:38:18.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:17 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/1212721236' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-02T04:38:19.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:18 smithi027 ceph-mon[37550]: osdmap e40: 6 total, 6 up, 6 in 2024-04-02T04:38:19.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:18 smithi027 ceph-mon[37550]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:19.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:18 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:38:19.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:18 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:38:19.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:18 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:38:19.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:18 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:38:19.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:18 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:38:19.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:18 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:38:19.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:18 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:38:19.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:18 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:38:19.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:18 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:38:19.303 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:18 smithi016 ceph-mon[28969]: osdmap e40: 6 total, 6 up, 6 in 2024-04-02T04:38:19.303 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:18 smithi016 ceph-mon[28969]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:19.303 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:18 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:38:19.303 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:18 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:38:19.303 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:18 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:38:19.303 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:18 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:38:19.304 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:18 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:38:19.304 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:18 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:38:19.304 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:18 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:38:19.304 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:18 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:38:19.304 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:18 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:38:19.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:19.831+0000 7f7964bc6700 1 -- 172.21.15.16:0/3233788688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79600ff9a0 msgr2=0x7f79600ffe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:19.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:19.831+0000 7f7964bc6700 1 --2- 172.21.15.16:0/3233788688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79600ff9a0 0x7f79600ffe20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7954009910 tx=0x7f7954009c20 comp rx=0 tx=0).stop 2024-04-02T04:38:19.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:19.831+0000 7f7964bc6700 1 -- 172.21.15.16:0/3233788688 shutdown_connections 2024-04-02T04:38:19.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:19.831+0000 7f7964bc6700 1 --2- 172.21.15.16:0/3233788688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79600ff9a0 0x7f79600ffe20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:19.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:19.831+0000 7f7964bc6700 1 --2- 172.21.15.16:0/3233788688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79601059a0 0x7f7960105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:19.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:19.831+0000 7f7964bc6700 1 -- 172.21.15.16:0/3233788688 >> 172.21.15.16:0/3233788688 conn(0x7f79600fb4b0 msgr2=0x7f79600fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:19.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:19.831+0000 7f7964bc6700 1 -- 172.21.15.16:0/3233788688 shutdown_connections 2024-04-02T04:38:19.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:19.831+0000 7f7964bc6700 1 -- 172.21.15.16:0/3233788688 wait complete. 2024-04-02T04:38:19.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:19.831+0000 7f7964bc6700 1 Processor -- start 2024-04-02T04:38:19.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:19.831+0000 7f7964bc6700 1 -- start start 2024-04-02T04:38:19.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:19.832+0000 7f7964bc6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79600ff9a0 0x7f79600fbd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:19.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:19.832+0000 7f7964bc6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79601059a0 0x7f79600fc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:19.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:19.832+0000 7f7964bc6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79600fc7a0 con 0x7f79601059a0 2024-04-02T04:38:19.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:19.832+0000 7f7964bc6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79600fc8e0 con 0x7f79600ff9a0 2024-04-02T04:38:19.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:19.832+0000 7f795dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79601059a0 0x7f79600fc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:19.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:19.832+0000 7f795dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79601059a0 0x7f79600fc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48560/0 (socket says 172.21.15.16:48560) 2024-04-02T04:38:19.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:19.832+0000 7f795dd9b700 1 -- 172.21.15.16:0/2482936535 learned_addr learned my addr 172.21.15.16:0/2482936535 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:38:19.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:19.832+0000 7f795e59c700 1 --2- 172.21.15.16:0/2482936535 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79600ff9a0 0x7f79600fbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:19.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:19.832+0000 7f795dd9b700 1 -- 172.21.15.16:0/2482936535 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79600ff9a0 msgr2=0x7f79600fbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:19.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:19.832+0000 7f795dd9b700 1 --2- 172.21.15.16:0/2482936535 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79600ff9a0 0x7f79600fbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:19.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:19.832+0000 7f795dd9b700 1 -- 172.21.15.16:0/2482936535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79540095f0 con 0x7f79601059a0 2024-04-02T04:38:19.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:19.832+0000 7f795dd9b700 1 --2- 172.21.15.16:0/2482936535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79601059a0 0x7f79600fc260 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f79540092e0 tx=0x7f795400b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:19.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:19.832+0000 7f794f7fe700 1 -- 172.21.15.16:0/2482936535 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f795401d070 con 0x7f79601059a0 2024-04-02T04:38:19.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:19.832+0000 7f7964bc6700 1 -- 172.21.15.16:0/2482936535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79600fcb60 con 0x7f79601059a0 2024-04-02T04:38:19.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:19.832+0000 7f794f7fe700 1 -- 172.21.15.16:0/2482936535 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f79540052a0 con 0x7f79601059a0 2024-04-02T04:38:19.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:19.832+0000 7f794f7fe700 1 -- 172.21.15.16:0/2482936535 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7954017400 con 0x7f79601059a0 2024-04-02T04:38:19.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:19.833+0000 7f7964bc6700 1 -- 172.21.15.16:0/2482936535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79601a2e90 con 0x7f79601059a0 2024-04-02T04:38:19.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:19.833+0000 7f7964bc6700 1 -- 172.21.15.16:0/2482936535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7960107fc0 con 0x7f79601059a0 2024-04-02T04:38:19.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:19.834+0000 7f794f7fe700 1 -- 172.21.15.16:0/2482936535 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f7954003710 con 0x7f79601059a0 2024-04-02T04:38:19.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:19.835+0000 7f794f7fe700 1 --2- 172.21.15.16:0/2482936535 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f794806ccf0 0x7f794806f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:19.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:19.835+0000 7f794f7fe700 1 -- 172.21.15.16:0/2482936535 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f795408d300 con 0x7f79601059a0 2024-04-02T04:38:19.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:19.835+0000 7f795e59c700 1 --2- 172.21.15.16:0/2482936535 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f794806ccf0 0x7f794806f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:19.835+0000 7f795e59c700 1 --2- 172.21.15.16:0/2482936535 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f794806ccf0 0x7f794806f1b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7950009c60 tx=0x7f7950009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:19.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:19.839+0000 7f794f7fe700 1 -- 172.21.15.16:0/2482936535 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7954057a90 con 0x7f79601059a0 2024-04-02T04:38:20.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:20.038+0000 7f7964bc6700 1 -- 172.21.15.16:0/2482936535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79601010e0 con 0x7f79601059a0 2024-04-02T04:38:20.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:20.038+0000 7f794f7fe700 1 -- 172.21.15.16:0/2482936535 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11273 (secure 0 0 0) 0x7f795405b0b0 con 0x7f79601059a0 2024-04-02T04:38:20.041 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:38:20.042 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":40,"fsid":"dc5403b4-f0a9-11ee-b647-cb9ed24678a4","created":"2024-04-02T04:32:23.014546+0000","modified":"2024-04-02T04:38:17.982526+0000","last_up_change":"2024-04-02T04:38:16.977561+0000","last_in_change":"2024-04-02T04:37:56.807214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T04:37:01.047338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2b7be29b-0e2a-456b-b33e-c72abaad9136","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":906882291},{"type":"v1","addr":"172.21.15.16:6803","nonce":906882291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":906882291},{"type":"v1","addr":"172.21.15.16:6805","nonce":906882291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":906882291},{"type":"v1","addr":"172.21.15.16:6809","nonce":906882291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":906882291},{"type":"v1","addr":"172.21.15.16:6807","nonce":906882291}]},"public_addr":"172.21.15.16:6803/906882291","cluster_addr":"172.21.15.16:6805/906882291","heartbeat_back_addr":"172.21.15.16:6809/906882291","heartbeat_front_addr":"172.21.15.16:6807/906882291","state":["exists","up"]},{"osd":1,"uuid":"82578291-3c9c-4f3f-8de9-7a8710a4923e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3079548081},{"type":"v1","addr":"172.21.15.16:6811","nonce":3079548081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3079548081},{"type":"v1","addr":"172.21.15.16:6813","nonce":3079548081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3079548081},{"type":"v1","addr":"172.21.15.16:6817","nonce":3079548081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3079548081},{"type":"v1","addr":"172.21.15.16:6815","nonce":3079548081}]},"public_addr":"172.21.15.16:6811/3079548081","cluster_addr":"172.21.15.16:6813/3079548081","heartbeat_back_addr":"172.21.15.16:6817/3079548081","heartbeat_front_addr":"172.21.15.16:6815/3079548081","state":["exists","up"]},{"osd":2,"uuid":"046262b5-386e-4270-863c-e97a61a038b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3734332239},{"type":"v1","addr":"172.21.15.16:6819","nonce":3734332239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3734332239},{"type":"v1","addr":"172.21.15.16:6821","nonce":3734332239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3734332239},{"type":"v1","addr":"172.21.15.16:6825","nonce":3734332239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3734332239},{"type":"v1","addr":"172.21.15.16:6823","nonce":3734332239}]},"public_addr":"172.21.15.16:6819/3734332239","cluster_addr":"172.21.15.16:6821/3734332239","heartbeat_back_addr":"172.21.15.16:6825/3734332239","heartbeat_front_addr":"172.21.15.16:6823/3734332239","state":["exists","up"]},{"osd":3,"uuid":"637f2a77-59a2-44fb-b47e-fcbd70f4423c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":388124754},{"type":"v1","addr":"172.21.15.27:6801","nonce":388124754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":388124754},{"type":"v1","addr":"172.21.15.27:6803","nonce":388124754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":388124754},{"type":"v1","addr":"172.21.15.27:6807","nonce":388124754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":388124754},{"type":"v1","addr":"172.21.15.27:6805","nonce":388124754}]},"public_addr":"172.21.15.27:6801/388124754","cluster_addr":"172.21.15.27:6803/388124754","heartbeat_back_addr":"172.21.15.27:6807/388124754","heartbeat_front_addr":"172.21.15.27:6805/388124754","state":["exists","up"]},{"osd":4,"uuid":"334b7b3d-505f-45b6-85b0-59ec8eeb8e91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":935993381},{"type":"v1","addr":"172.21.15.27:6809","nonce":935993381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":935993381},{"type":"v1","addr":"172.21.15.27:6811","nonce":935993381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":935993381},{"type":"v1","addr":"172.21.15.27:6815","nonce":935993381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":935993381},{"type":"v1","addr":"172.21.15.27:6813","nonce":935993381}]},"public_addr":"172.21.15.27:6809/935993381","cluster_addr":"172.21.15.27:6811/935993381","heartbeat_back_addr":"172.21.15.27:6815/935993381","heartbeat_front_addr":"172.21.15.27:6813/935993381","state":["exists","up"]},{"osd":5,"uuid":"f83802ee-4f47-47de-91dd-a667f6cd22e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2454420135},{"type":"v1","addr":"172.21.15.27:6817","nonce":2454420135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":2454420135},{"type":"v1","addr":"172.21.15.27:6819","nonce":2454420135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":2454420135},{"type":"v1","addr":"172.21.15.27:6823","nonce":2454420135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":2454420135},{"type":"v1","addr":"172.21.15.27:6821","nonce":2454420135}]},"public_addr":"172.21.15.27:6817/2454420135","cluster_addr":"172.21.15.27:6819/2454420135","heartbeat_back_addr":"172.21.15.27:6823/2454420135","heartbeat_front_addr":"172.21.15.27:6821/2454420135","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:36:02.027852+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:36:29.393593+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:36:56.847224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:37:21.195876+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:37:47.992167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:38:15.012477+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/4209328561":"2024-04-03T04:34:42.713541+0000","172.21.15.16:0/3733852550":"2024-04-03T04:34:42.713541+0000","172.21.15.16:0/3405957790":"2024-04-03T04:34:42.713541+0000","172.21.15.16:6800/3912038174":"2024-04-03T04:34:42.713541+0000","172.21.15.16:6801/3912038174":"2024-04-03T04:34:42.713541+0000","172.21.15.16:0/4067493740":"2024-04-03T04:33:28.907145+0000","172.21.15.16:6801/873531241":"2024-04-03T04:33:28.907145+0000","172.21.15.16:6800/873531241":"2024-04-03T04:33:28.907145+0000","172.21.15.16:0/1228221358":"2024-04-03T04:33:28.907145+0000","172.21.15.16:6800/824107799":"2024-04-03T04:32:56.644452+0000","172.21.15.16:0/2255383579":"2024-04-03T04:32:56.644452+0000","172.21.15.16:0/401549934":"2024-04-03T04:32:56.644452+0000","172.21.15.16:6801/824107799":"2024-04-03T04:32:56.644452+0000","172.21.15.16:0/1884756010":"2024-04-03T04:32:56.644452+0000","172.21.15.16:0/166410319":"2024-04-03T04:33:28.907145+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T04:38:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:20.040+0000 7f7964bc6700 1 -- 172.21.15.16:0/2482936535 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f794806ccf0 msgr2=0x7f794806f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:20.040+0000 7f7964bc6700 1 --2- 172.21.15.16:0/2482936535 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f794806ccf0 0x7f794806f1b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7950009c60 tx=0x7f7950009260 comp rx=0 tx=0).stop 2024-04-02T04:38:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:20.041+0000 7f7964bc6700 1 -- 172.21.15.16:0/2482936535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79601059a0 msgr2=0x7f79600fc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:20.041+0000 7f7964bc6700 1 --2- 172.21.15.16:0/2482936535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79601059a0 0x7f79600fc260 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f79540092e0 tx=0x7f795400b610 comp rx=0 tx=0).stop 2024-04-02T04:38:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:20.041+0000 7f7964bc6700 1 -- 172.21.15.16:0/2482936535 shutdown_connections 2024-04-02T04:38:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:20.041+0000 7f7964bc6700 1 --2- 172.21.15.16:0/2482936535 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f794806ccf0 0x7f794806f1b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:20.041+0000 7f7964bc6700 1 --2- 172.21.15.16:0/2482936535 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79600ff9a0 0x7f79600fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:20.041+0000 7f7964bc6700 1 --2- 172.21.15.16:0/2482936535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79601059a0 0x7f79600fc260 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:20.041+0000 7f7964bc6700 1 -- 172.21.15.16:0/2482936535 >> 172.21.15.16:0/2482936535 conn(0x7f79600fb4b0 msgr2=0x7f7960109d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:20.041+0000 7f7964bc6700 1 -- 172.21.15.16:0/2482936535 shutdown_connections 2024-04-02T04:38:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:20.041+0000 7f7964bc6700 1 -- 172.21.15.16:0/2482936535 wait complete. 2024-04-02T04:38:20.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:19 smithi027 ceph-mon[37550]: Detected new or changed devices on smithi027 2024-04-02T04:38:20.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:19 smithi027 ceph-mon[37550]: Adjusting osd_memory_target on smithi027 to 4970M 2024-04-02T04:38:20.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:19 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:38:20.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:19 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:38:20.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:19 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:38:20.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:19 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:38:20.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:19 smithi016 ceph-mon[28969]: Detected new or changed devices on smithi027 2024-04-02T04:38:20.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:19 smithi016 ceph-mon[28969]: Adjusting osd_memory_target on smithi027 to 4970M 2024-04-02T04:38:20.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:19 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:38:20.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:19 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:38:20.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:19 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:38:20.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:19 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:38:20.812 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T04:37:01.047338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T04:38:20.813 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph osd pool get .mgr pg_num 2024-04-02T04:38:21.029 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:38:21.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:20 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/2482936535' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T04:38:21.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:20 smithi016 ceph-mon[28969]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:21.312 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:20 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/2482936535' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T04:38:21.313 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:20 smithi027 ceph-mon[37550]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:22.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.724+0000 7fc0c80c1700 1 -- 172.21.15.16:0/2297186505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0c0102570 msgr2=0x7fc0c010aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:22.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.724+0000 7fc0c80c1700 1 --2- 172.21.15.16:0/2297186505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0c0102570 0x7fc0c010aa60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0009910 tx=0x7fc0b0009c20 comp rx=0 tx=0).stop 2024-04-02T04:38:22.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.724+0000 7fc0c80c1700 1 -- 172.21.15.16:0/2297186505 shutdown_connections 2024-04-02T04:38:22.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.724+0000 7fc0c80c1700 1 --2- 172.21.15.16:0/2297186505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0c0102570 0x7fc0c010aa60 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:22.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.724+0000 7fc0c80c1700 1 --2- 172.21.15.16:0/2297186505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c0101c50 0x7fc0c0102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:22.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.724+0000 7fc0c80c1700 1 -- 172.21.15.16:0/2297186505 >> 172.21.15.16:0/2297186505 conn(0x7fc0c00fb4c0 msgr2=0x7fc0c00fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:22.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.724+0000 7fc0c80c1700 1 -- 172.21.15.16:0/2297186505 shutdown_connections 2024-04-02T04:38:22.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.724+0000 7fc0c80c1700 1 -- 172.21.15.16:0/2297186505 wait complete. 2024-04-02T04:38:22.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.725+0000 7fc0c80c1700 1 Processor -- start 2024-04-02T04:38:22.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.725+0000 7fc0c80c1700 1 -- start start 2024-04-02T04:38:22.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.725+0000 7fc0c80c1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0c0101c50 0x7fc0c019ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:22.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.725+0000 7fc0c80c1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c0102570 0x7fc0c019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:22.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.725+0000 7fc0c80c1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c0198b00 con 0x7fc0c0101c50 2024-04-02T04:38:22.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.725+0000 7fc0c80c1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c0198c70 con 0x7fc0c0102570 2024-04-02T04:38:22.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.725+0000 7fc0c5e5d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0c0101c50 0x7fc0c019ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:22.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.725+0000 7fc0c5e5d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0c0101c50 0x7fc0c019ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45862/0 (socket says 172.21.15.16:45862) 2024-04-02T04:38:22.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.725+0000 7fc0c5e5d700 1 -- 172.21.15.16:0/1495838521 learned_addr learned my addr 172.21.15.16:0/1495838521 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:38:22.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.725+0000 7fc0c565c700 1 --2- 172.21.15.16:0/1495838521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c0102570 0x7fc0c019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:22.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.726+0000 7fc0c5e5d700 1 -- 172.21.15.16:0/1495838521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c0102570 msgr2=0x7fc0c019ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:22.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.726+0000 7fc0c5e5d700 1 --2- 172.21.15.16:0/1495838521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c0102570 0x7fc0c019ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:22.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.726+0000 7fc0c5e5d700 1 -- 172.21.15.16:0/1495838521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b00095f0 con 0x7fc0c0101c50 2024-04-02T04:38:22.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.726+0000 7fc0c5e5d700 1 --2- 172.21.15.16:0/1495838521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0c0101c50 0x7fc0c019ea50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc00b4b0 tx=0x7fc0bc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:22.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.726+0000 7fc0b6ffd700 1 -- 172.21.15.16:0/1495838521 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0bc011800 con 0x7fc0c0101c50 2024-04-02T04:38:22.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.726+0000 7fc0b6ffd700 1 -- 172.21.15.16:0/1495838521 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc0bc011e20 con 0x7fc0c0101c50 2024-04-02T04:38:22.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.726+0000 7fc0c80c1700 1 -- 172.21.15.16:0/1495838521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0c0198f50 con 0x7fc0c0101c50 2024-04-02T04:38:22.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.726+0000 7fc0b6ffd700 1 -- 172.21.15.16:0/1495838521 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0bc00f2e0 con 0x7fc0c0101c50 2024-04-02T04:38:22.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.726+0000 7fc0c80c1700 1 -- 172.21.15.16:0/1495838521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0c0199470 con 0x7fc0c0101c50 2024-04-02T04:38:22.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.727+0000 7fc0c80c1700 1 -- 172.21.15.16:0/1495838521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0c010a5c0 con 0x7fc0c0101c50 2024-04-02T04:38:22.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.727+0000 7fc0b6ffd700 1 -- 172.21.15.16:0/1495838521 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fc0bc010380 con 0x7fc0c0101c50 2024-04-02T04:38:22.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.728+0000 7fc0b6ffd700 1 --2- 172.21.15.16:0/1495838521 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc0ac06cce0 0x7fc0ac06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:22.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.728+0000 7fc0b6ffd700 1 -- 172.21.15.16:0/1495838521 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fc0bc08bae0 con 0x7fc0c0101c50 2024-04-02T04:38:22.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.728+0000 7fc0c565c700 1 --2- 172.21.15.16:0/1495838521 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc0ac06cce0 0x7fc0ac06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:22.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.728+0000 7fc0c565c700 1 --2- 172.21.15.16:0/1495838521 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc0ac06cce0 0x7fc0ac06f1a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc0c019a300 tx=0x7fc0b001a040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:22.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.732+0000 7fc0b6ffd700 1 -- 172.21.15.16:0/1495838521 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc0bc0561f0 con 0x7fc0c0101c50 2024-04-02T04:38:22.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.933+0000 7fc0c80c1700 1 -- 172.21.15.16:0/1495838521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc0c0199fe0 con 0x7fc0c0101c50 2024-04-02T04:38:22.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.933+0000 7fc0b6ffd700 1 -- 172.21.15.16:0/1495838521 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc0bc059810 con 0x7fc0c0101c50 2024-04-02T04:38:22.936 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-02T04:38:22.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.935+0000 7fc0c80c1700 1 -- 172.21.15.16:0/1495838521 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc0ac06cce0 msgr2=0x7fc0ac06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:22.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.935+0000 7fc0c80c1700 1 --2- 172.21.15.16:0/1495838521 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc0ac06cce0 0x7fc0ac06f1a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc0c019a300 tx=0x7fc0b001a040 comp rx=0 tx=0).stop 2024-04-02T04:38:22.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.935+0000 7fc0c80c1700 1 -- 172.21.15.16:0/1495838521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0c0101c50 msgr2=0x7fc0c019ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:22.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.935+0000 7fc0c80c1700 1 --2- 172.21.15.16:0/1495838521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0c0101c50 0x7fc0c019ea50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc00b4b0 tx=0x7fc0bc00b870 comp rx=0 tx=0).stop 2024-04-02T04:38:22.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.935+0000 7fc0c80c1700 1 -- 172.21.15.16:0/1495838521 shutdown_connections 2024-04-02T04:38:22.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.935+0000 7fc0c80c1700 1 --2- 172.21.15.16:0/1495838521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0c0101c50 0x7fc0c019ea50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:22.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.935+0000 7fc0c80c1700 1 --2- 172.21.15.16:0/1495838521 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc0ac06cce0 0x7fc0ac06f1a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:22.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.935+0000 7fc0c80c1700 1 --2- 172.21.15.16:0/1495838521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c0102570 0x7fc0c019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:22.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.935+0000 7fc0c80c1700 1 -- 172.21.15.16:0/1495838521 >> 172.21.15.16:0/1495838521 conn(0x7fc0c00fb4c0 msgr2=0x7fc0c0100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:22.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.935+0000 7fc0c80c1700 1 -- 172.21.15.16:0/1495838521 shutdown_connections 2024-04-02T04:38:22.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:22.935+0000 7fc0c80c1700 1 -- 172.21.15.16:0/1495838521 wait complete. 2024-04-02T04:38:23.635 INFO:tasks.cephadm:Setting up client nodes... 2024-04-02T04:38:23.635 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-02T04:38:23.850 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:38:23.960 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:23 smithi016 ceph-mon[28969]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:23.960 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:23 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/1495838521' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T04:38:24.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:23 smithi027 ceph-mon[37550]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:24.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:23 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/1495838521' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T04:38:25.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.378+0000 7fbff6298700 1 -- 172.21.15.16:0/2496940853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbff0100be0 msgr2=0x7fbff0101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:25.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.378+0000 7fbff6298700 1 --2- 172.21.15.16:0/2496940853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbff0100be0 0x7fbff0101040 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4009910 tx=0x7fbfe4009c20 comp rx=0 tx=0).stop 2024-04-02T04:38:25.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.378+0000 7fbff6298700 1 -- 172.21.15.16:0/2496940853 shutdown_connections 2024-04-02T04:38:25.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.378+0000 7fbff6298700 1 --2- 172.21.15.16:0/2496940853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbff0100be0 0x7fbff0101040 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:25.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.378+0000 7fbff6298700 1 --2- 172.21.15.16:0/2496940853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbff00ff9e0 0x7fbff00ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:25.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.378+0000 7fbff6298700 1 -- 172.21.15.16:0/2496940853 >> 172.21.15.16:0/2496940853 conn(0x7fbff00fafc0 msgr2=0x7fbff00fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:25.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.378+0000 7fbff6298700 1 -- 172.21.15.16:0/2496940853 shutdown_connections 2024-04-02T04:38:25.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.378+0000 7fbff6298700 1 -- 172.21.15.16:0/2496940853 wait complete. 2024-04-02T04:38:25.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.378+0000 7fbff6298700 1 Processor -- start 2024-04-02T04:38:25.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.378+0000 7fbff6298700 1 -- start start 2024-04-02T04:38:25.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.379+0000 7fbff6298700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbff00ff9e0 0x7fbff01986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:25.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.379+0000 7fbff6298700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbff0100be0 0x7fbff0198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:25.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.379+0000 7fbff6298700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff0199230 con 0x7fbff00ff9e0 2024-04-02T04:38:25.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.379+0000 7fbff6298700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff0199370 con 0x7fbff0100be0 2024-04-02T04:38:25.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.379+0000 7fbfeffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbff00ff9e0 0x7fbff01986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:25.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.379+0000 7fbfeffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbff00ff9e0 0x7fbff01986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45874/0 (socket says 172.21.15.16:45874) 2024-04-02T04:38:25.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.379+0000 7fbfeffff700 1 -- 172.21.15.16:0/2527487071 learned_addr learned my addr 172.21.15.16:0/2527487071 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:38:25.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.379+0000 7fbfef7fe700 1 --2- 172.21.15.16:0/2527487071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbff0100be0 0x7fbff0198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:25.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.379+0000 7fbfeffff700 1 -- 172.21.15.16:0/2527487071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbff0100be0 msgr2=0x7fbff0198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:25.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.379+0000 7fbfeffff700 1 --2- 172.21.15.16:0/2527487071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbff0100be0 0x7fbff0198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:25.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.379+0000 7fbfeffff700 1 -- 172.21.15.16:0/2527487071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe40095f0 con 0x7fbff00ff9e0 2024-04-02T04:38:25.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.379+0000 7fbfeffff700 1 --2- 172.21.15.16:0/2527487071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbff00ff9e0 0x7fbff01986d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbfe000d900 tx=0x7fbfe000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:25.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.379+0000 7fbfed7fa700 1 -- 172.21.15.16:0/2527487071 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfe000b920 con 0x7fbff00ff9e0 2024-04-02T04:38:25.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.380+0000 7fbfed7fa700 1 -- 172.21.15.16:0/2527487071 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbfe0010460 con 0x7fbff00ff9e0 2024-04-02T04:38:25.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.380+0000 7fbff6298700 1 -- 172.21.15.16:0/2527487071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff019ddd0 con 0x7fbff00ff9e0 2024-04-02T04:38:25.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.380+0000 7fbfed7fa700 1 -- 172.21.15.16:0/2527487071 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfe000f590 con 0x7fbff00ff9e0 2024-04-02T04:38:25.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.380+0000 7fbff6298700 1 -- 172.21.15.16:0/2527487071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff019e2a0 con 0x7fbff00ff9e0 2024-04-02T04:38:25.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.381+0000 7fbfed7fa700 1 -- 172.21.15.16:0/2527487071 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fbfe00105d0 con 0x7fbff00ff9e0 2024-04-02T04:38:25.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.381+0000 7fbff6298700 1 -- 172.21.15.16:0/2527487071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbff004ea80 con 0x7fbff00ff9e0 2024-04-02T04:38:25.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.381+0000 7fbfed7fa700 1 --2- 172.21.15.16:0/2527487071 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fbfd806cce0 0x7fbfd806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:25.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.381+0000 7fbfed7fa700 1 -- 172.21.15.16:0/2527487071 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fbfe008baa0 con 0x7fbff00ff9e0 2024-04-02T04:38:25.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.382+0000 7fbfef7fe700 1 --2- 172.21.15.16:0/2527487071 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fbfd806cce0 0x7fbfd806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:25.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.382+0000 7fbfef7fe700 1 --2- 172.21.15.16:0/2527487071 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fbfd806cce0 0x7fbfd806f1a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4009fd0 tx=0x7fbfe4005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:25.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.386+0000 7fbfed7fa700 1 -- 172.21.15.16:0/2527487071 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbfe0059cc0 con 0x7fbff00ff9e0 2024-04-02T04:38:25.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.674+0000 7fbff6298700 1 -- 172.21.15.16:0/2527487071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fbff019e510 con 0x7fbff00ff9e0 2024-04-02T04:38:25.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.678+0000 7fbfed7fa700 1 -- 172.21.15.16:0/2527487071 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7fbfe0059850 con 0x7fbff00ff9e0 2024-04-02T04:38:25.681 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-04-02T04:38:25.681 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDBiwtmprlFKBAAHaYxzK38bpqt6rJD/2V8Pw== 2024-04-02T04:38:25.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.680+0000 7fbff6298700 1 -- 172.21.15.16:0/2527487071 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fbfd806cce0 msgr2=0x7fbfd806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:25.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.680+0000 7fbff6298700 1 --2- 172.21.15.16:0/2527487071 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fbfd806cce0 0x7fbfd806f1a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4009fd0 tx=0x7fbfe4005960 comp rx=0 tx=0).stop 2024-04-02T04:38:25.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.680+0000 7fbff6298700 1 -- 172.21.15.16:0/2527487071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbff00ff9e0 msgr2=0x7fbff01986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:25.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.680+0000 7fbff6298700 1 --2- 172.21.15.16:0/2527487071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbff00ff9e0 0x7fbff01986d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbfe000d900 tx=0x7fbfe000dc10 comp rx=0 tx=0).stop 2024-04-02T04:38:25.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.680+0000 7fbff6298700 1 -- 172.21.15.16:0/2527487071 shutdown_connections 2024-04-02T04:38:25.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.680+0000 7fbff6298700 1 --2- 172.21.15.16:0/2527487071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbff00ff9e0 0x7fbff01986d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:25.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.680+0000 7fbff6298700 1 --2- 172.21.15.16:0/2527487071 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fbfd806cce0 0x7fbfd806f1a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:25.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.680+0000 7fbff6298700 1 --2- 172.21.15.16:0/2527487071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbff0100be0 0x7fbff0198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:25.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.680+0000 7fbff6298700 1 -- 172.21.15.16:0/2527487071 >> 172.21.15.16:0/2527487071 conn(0x7fbff00fafc0 msgr2=0x7fbff00684f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:25.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.680+0000 7fbff6298700 1 -- 172.21.15.16:0/2527487071 shutdown_connections 2024-04-02T04:38:25.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:25.680+0000 7fbff6298700 1 -- 172.21.15.16:0/2527487071 wait complete. 2024-04-02T04:38:25.963 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:25 smithi016 ceph-mon[28969]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:25.964 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:25 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/2527487071' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-02T04:38:25.964 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:25 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/2527487071' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-02T04:38:26.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:25 smithi027 ceph-mon[37550]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:26.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:25 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/2527487071' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-02T04:38:26.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:25 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/2527487071' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-02T04:38:26.308 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-02T04:38:26.308 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-02T04:38:26.309 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-02T04:38:26.353 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-02T04:38:26.584 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi027/config 2024-04-02T04:38:27.959 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:27 smithi027 ceph-mon[37550]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:27.959 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:27 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:38:28.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:27 smithi016 ceph-mon[28969]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:28.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:27 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:38:28.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.187+0000 7f012be06700 1 -- 172.21.15.27:0/3186921127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0124101cf0 msgr2=0x7f0124102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:28.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.187+0000 7f012be06700 1 --2- 172.21.15.27:0/3186921127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0124101cf0 0x7f0124102150 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0114009910 tx=0x7f0114009c20 comp rx=0 tx=0).stop 2024-04-02T04:38:28.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.187+0000 7f012be06700 1 -- 172.21.15.27:0/3186921127 shutdown_connections 2024-04-02T04:38:28.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.187+0000 7f012be06700 1 --2- 172.21.15.27:0/3186921127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0124101cf0 0x7f0124102150 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:28.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.187+0000 7f012be06700 1 --2- 172.21.15.27:0/3186921127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0124100af0 0x7f0124100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:28.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.187+0000 7f012be06700 1 -- 172.21.15.27:0/3186921127 >> 172.21.15.27:0/3186921127 conn(0x7f01240fc0b0 msgr2=0x7f01240fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:28.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.187+0000 7f012be06700 1 -- 172.21.15.27:0/3186921127 shutdown_connections 2024-04-02T04:38:28.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.187+0000 7f012be06700 1 -- 172.21.15.27:0/3186921127 wait complete. 2024-04-02T04:38:28.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.188+0000 7f012be06700 1 Processor -- start 2024-04-02T04:38:28.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.188+0000 7f012be06700 1 -- start start 2024-04-02T04:38:28.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.188+0000 7f012be06700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0124100af0 0x7f01241964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:28.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.188+0000 7f012be06700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0124101cf0 0x7f01241969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:28.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.188+0000 7f012be06700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0124197010 con 0x7f0124100af0 2024-04-02T04:38:28.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.188+0000 7f012be06700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0124197150 con 0x7f0124101cf0 2024-04-02T04:38:28.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.188+0000 7f01293a1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0124101cf0 0x7f01241969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:28.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.188+0000 7f01293a1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0124101cf0 0x7f01241969f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33626/0 (socket says 172.21.15.27:33626) 2024-04-02T04:38:28.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.188+0000 7f01293a1700 1 -- 172.21.15.27:0/2001778667 learned_addr learned my addr 172.21.15.27:0/2001778667 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-02T04:38:28.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.188+0000 7f0129ba2700 1 --2- 172.21.15.27:0/2001778667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0124100af0 0x7f01241964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:28.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.189+0000 7f01293a1700 1 -- 172.21.15.27:0/2001778667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0124100af0 msgr2=0x7f01241964b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:28.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.189+0000 7f01293a1700 1 --2- 172.21.15.27:0/2001778667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0124100af0 0x7f01241964b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:28.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.189+0000 7f01293a1700 1 -- 172.21.15.27:0/2001778667 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01140095f0 con 0x7f0124101cf0 2024-04-02T04:38:28.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.189+0000 7f0129ba2700 1 --2- 172.21.15.27:0/2001778667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0124100af0 0x7f01241964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T04:38:28.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.189+0000 7f01293a1700 1 --2- 172.21.15.27:0/2001778667 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0124101cf0 0x7f01241969f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f01140092e0 tx=0x7f011400be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:28.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.189+0000 7f011affd700 1 -- 172.21.15.27:0/2001778667 <== mon.1 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f011401d070 con 0x7f0124101cf0 2024-04-02T04:38:28.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.189+0000 7f012be06700 1 -- 172.21.15.27:0/2001778667 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f012419bb50 con 0x7f0124101cf0 2024-04-02T04:38:28.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.189+0000 7f011affd700 1 -- 172.21.15.27:0/2001778667 <== mon.1 v2:172.21.15.27:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f011400f460 con 0x7f0124101cf0 2024-04-02T04:38:28.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.189+0000 7f011affd700 1 -- 172.21.15.27:0/2001778667 <== mon.1 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0114021640 con 0x7f0124101cf0 2024-04-02T04:38:28.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.189+0000 7f012be06700 1 -- 172.21.15.27:0/2001778667 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f012419c0c0 con 0x7f0124101cf0 2024-04-02T04:38:28.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.190+0000 7f012be06700 1 -- 172.21.15.27:0/2001778667 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0124190670 con 0x7f0124101cf0 2024-04-02T04:38:28.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.190+0000 7f011affd700 1 -- 172.21.15.27:0/2001778667 <== mon.1 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f011400f5d0 con 0x7f0124101cf0 2024-04-02T04:38:28.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.191+0000 7f011affd700 1 --2- 172.21.15.27:0/2001778667 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f011006cce0 0x7f011006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:28.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.191+0000 7f011affd700 1 -- 172.21.15.27:0/2001778667 <== mon.1 v2:172.21.15.27:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f011408d280 con 0x7f0124101cf0 2024-04-02T04:38:28.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.191+0000 7f0129ba2700 1 --2- 172.21.15.27:0/2001778667 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f011006cce0 0x7f011006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:28.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.192+0000 7f0129ba2700 1 --2- 172.21.15.27:0/2001778667 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f011006cce0 0x7f011006f1a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0124101b50 tx=0x7f0120005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:28.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.196+0000 7f011affd700 1 -- 172.21.15.27:0/2001778667 <== mon.1 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f011405b420 con 0x7f0124101cf0 2024-04-02T04:38:28.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.483+0000 7f012be06700 1 -- 172.21.15.27:0/2001778667 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f012404ea80 con 0x7f0124101cf0 2024-04-02T04:38:28.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.488+0000 7f011affd700 1 -- 172.21.15.27:0/2001778667 <== mon.1 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7f011405afb0 con 0x7f0124101cf0 2024-04-02T04:38:28.491 INFO:teuthology.orchestra.run.smithi027.stdout:[client.1] 2024-04-02T04:38:28.492 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQDEiwtm/+zxHBAAQYq0KzccO1xCi2FHrSKADQ== 2024-04-02T04:38:28.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.490+0000 7f012be06700 1 -- 172.21.15.27:0/2001778667 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f011006cce0 msgr2=0x7f011006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:28.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.490+0000 7f012be06700 1 --2- 172.21.15.27:0/2001778667 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f011006cce0 0x7f011006f1a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0124101b50 tx=0x7f0120005f90 comp rx=0 tx=0).stop 2024-04-02T04:38:28.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.490+0000 7f012be06700 1 -- 172.21.15.27:0/2001778667 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0124101cf0 msgr2=0x7f01241969f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:28.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.490+0000 7f012be06700 1 --2- 172.21.15.27:0/2001778667 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0124101cf0 0x7f01241969f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f01140092e0 tx=0x7f011400be20 comp rx=0 tx=0).stop 2024-04-02T04:38:28.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.490+0000 7f012be06700 1 -- 172.21.15.27:0/2001778667 shutdown_connections 2024-04-02T04:38:28.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.490+0000 7f012be06700 1 --2- 172.21.15.27:0/2001778667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0124100af0 0x7f01241964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:28.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.490+0000 7f012be06700 1 --2- 172.21.15.27:0/2001778667 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f011006cce0 0x7f011006f1a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:28.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.490+0000 7f012be06700 1 --2- 172.21.15.27:0/2001778667 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0124101cf0 0x7f01241969f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:28.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.490+0000 7f012be06700 1 -- 172.21.15.27:0/2001778667 >> 172.21.15.27:0/2001778667 conn(0x7f01240fc0b0 msgr2=0x7f0124105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:28.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.490+0000 7f012be06700 1 -- 172.21.15.27:0/2001778667 shutdown_connections 2024-04-02T04:38:28.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:38:28.490+0000 7f012be06700 1 -- 172.21.15.27:0/2001778667 wait complete. 2024-04-02T04:38:29.103 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:28 smithi027 ceph-mon[37550]: from='client.? 172.21.15.27:0/2001778667' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-02T04:38:29.103 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:28 smithi027 ceph-mon[37550]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-02T04:38:29.103 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:28 smithi027 ceph-mon[37550]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-02T04:38:29.125 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-02T04:38:29.125 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-04-02T04:38:29.125 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-02T04:38:29.175 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-02T04:38:29.176 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-02T04:38:29.176 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph mgr dump --format=json 2024-04-02T04:38:29.205 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:28 smithi016 ceph-mon[28969]: from='client.? 172.21.15.27:0/2001778667' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-02T04:38:29.205 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:28 smithi016 ceph-mon[28969]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-02T04:38:29.205 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:28 smithi016 ceph-mon[28969]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-02T04:38:29.394 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:38:30.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:29 smithi016 ceph-mon[28969]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:30.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:29 smithi027 ceph-mon[37550]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:30.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:30.908+0000 7f9b29822700 1 -- 172.21.15.16:0/1626567522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b24106a70 msgr2=0x7f9b24106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:30.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:30.908+0000 7f9b29822700 1 --2- 172.21.15.16:0/1626567522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b24106a70 0x7f9b24106e50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9b18009910 tx=0x7f9b18009c20 comp rx=0 tx=0).stop 2024-04-02T04:38:30.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:30.908+0000 7f9b29822700 1 -- 172.21.15.16:0/1626567522 shutdown_connections 2024-04-02T04:38:30.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:30.908+0000 7f9b29822700 1 --2- 172.21.15.16:0/1626567522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b24100a70 0x7f9b24100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:30.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:30.908+0000 7f9b29822700 1 --2- 172.21.15.16:0/1626567522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b24106a70 0x7f9b24106e50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:30.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:30.908+0000 7f9b29822700 1 -- 172.21.15.16:0/1626567522 >> 172.21.15.16:0/1626567522 conn(0x7f9b240fc5a0 msgr2=0x7f9b240fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:30.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:30.908+0000 7f9b29822700 1 -- 172.21.15.16:0/1626567522 shutdown_connections 2024-04-02T04:38:30.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:30.908+0000 7f9b29822700 1 -- 172.21.15.16:0/1626567522 wait complete. 2024-04-02T04:38:30.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:30.909+0000 7f9b29822700 1 Processor -- start 2024-04-02T04:38:30.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:30.909+0000 7f9b29822700 1 -- start start 2024-04-02T04:38:30.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:30.909+0000 7f9b29822700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b24100a70 0x7f9b240730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:30.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:30.909+0000 7f9b29822700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b24106a70 0x7f9b240735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:30.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:30.909+0000 7f9b29822700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b24073b30 con 0x7f9b24100a70 2024-04-02T04:38:30.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:30.909+0000 7f9b29822700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b24073c70 con 0x7f9b24106a70 2024-04-02T04:38:30.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:30.909+0000 7f9b22ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b24100a70 0x7f9b240730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:30.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:30.909+0000 7f9b22ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b24100a70 0x7f9b240730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45892/0 (socket says 172.21.15.16:45892) 2024-04-02T04:38:30.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:30.909+0000 7f9b22ffd700 1 -- 172.21.15.16:0/2193818085 learned_addr learned my addr 172.21.15.16:0/2193818085 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:38:30.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:30.909+0000 7f9b227fc700 1 --2- 172.21.15.16:0/2193818085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b24106a70 0x7f9b240735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:30.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:30.909+0000 7f9b22ffd700 1 -- 172.21.15.16:0/2193818085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b24106a70 msgr2=0x7f9b240735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:30.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:30.909+0000 7f9b22ffd700 1 --2- 172.21.15.16:0/2193818085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b24106a70 0x7f9b240735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:30.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:30.909+0000 7f9b22ffd700 1 -- 172.21.15.16:0/2193818085 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b180095f0 con 0x7f9b24100a70 2024-04-02T04:38:30.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:30.910+0000 7f9b22ffd700 1 --2- 172.21.15.16:0/2193818085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b24100a70 0x7f9b240730b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9b1800bd70 tx=0x7f9b1800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:30.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:30.910+0000 7f9b28820700 1 -- 172.21.15.16:0/2193818085 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b1801d070 con 0x7f9b24100a70 2024-04-02T04:38:30.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:30.910+0000 7f9b28820700 1 -- 172.21.15.16:0/2193818085 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b180056e0 con 0x7f9b24100a70 2024-04-02T04:38:30.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:30.910+0000 7f9b29822700 1 -- 172.21.15.16:0/2193818085 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b24073ef0 con 0x7f9b24100a70 2024-04-02T04:38:30.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:30.910+0000 7f9b28820700 1 -- 172.21.15.16:0/2193818085 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b1800fe40 con 0x7f9b24100a70 2024-04-02T04:38:30.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:30.910+0000 7f9b29822700 1 -- 172.21.15.16:0/2193818085 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b241a2df0 con 0x7f9b24100a70 2024-04-02T04:38:30.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:30.911+0000 7f9b29822700 1 -- 172.21.15.16:0/2193818085 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b24109070 con 0x7f9b24100a70 2024-04-02T04:38:30.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:30.911+0000 7f9b28820700 1 -- 172.21.15.16:0/2193818085 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f9b1800f680 con 0x7f9b24100a70 2024-04-02T04:38:30.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:30.912+0000 7f9b28820700 1 --2- 172.21.15.16:0/2193818085 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f9b0c06ccd0 0x7f9b0c06f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:30.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:30.912+0000 7f9b227fc700 1 --2- 172.21.15.16:0/2193818085 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f9b0c06ccd0 0x7f9b0c06f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:30.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:30.912+0000 7f9b28820700 1 -- 172.21.15.16:0/2193818085 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f9b1808d390 con 0x7f9b24100a70 2024-04-02T04:38:30.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:30.912+0000 7f9b227fc700 1 --2- 172.21.15.16:0/2193818085 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f9b0c06ccd0 0x7f9b0c06f190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9b240747e0 tx=0x7f9b1400b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:30.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:30.916+0000 7f9b28820700 1 -- 172.21.15.16:0/2193818085 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9b18057a00 con 0x7f9b24100a70 2024-04-02T04:38:31.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:31.178+0000 7f9b29822700 1 -- 172.21.15.16:0/2193818085 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f9b24074630 con 0x7f9b24100a70 2024-04-02T04:38:31.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:31.182+0000 7f9b28820700 1 -- 172.21.15.16:0/2193818085 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+174003 (secure 0 0 0) 0x7f9b1805b020 con 0x7f9b24100a70 2024-04-02T04:38:31.188 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:38:31.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:31.185+0000 7f9b29822700 1 -- 172.21.15.16:0/2193818085 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f9b0c06ccd0 msgr2=0x7f9b0c06f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:31.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:31.185+0000 7f9b29822700 1 --2- 172.21.15.16:0/2193818085 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f9b0c06ccd0 0x7f9b0c06f190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9b240747e0 tx=0x7f9b1400b410 comp rx=0 tx=0).stop 2024-04-02T04:38:31.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:31.185+0000 7f9b29822700 1 -- 172.21.15.16:0/2193818085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b24100a70 msgr2=0x7f9b240730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:31.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:31.185+0000 7f9b29822700 1 --2- 172.21.15.16:0/2193818085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b24100a70 0x7f9b240730b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9b1800bd70 tx=0x7f9b1800bda0 comp rx=0 tx=0).stop 2024-04-02T04:38:31.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:31.185+0000 7f9b29822700 1 -- 172.21.15.16:0/2193818085 shutdown_connections 2024-04-02T04:38:31.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:31.185+0000 7f9b29822700 1 --2- 172.21.15.16:0/2193818085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b24100a70 0x7f9b240730b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:31.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:31.185+0000 7f9b29822700 1 --2- 172.21.15.16:0/2193818085 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f9b0c06ccd0 0x7f9b0c06f190 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:31.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:31.185+0000 7f9b29822700 1 --2- 172.21.15.16:0/2193818085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b24106a70 0x7f9b240735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:31.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:31.185+0000 7f9b29822700 1 -- 172.21.15.16:0/2193818085 >> 172.21.15.16:0/2193818085 conn(0x7f9b240fc5a0 msgr2=0x7f9b240fd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:31.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:31.186+0000 7f9b29822700 1 -- 172.21.15.16:0/2193818085 shutdown_connections 2024-04-02T04:38:31.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:31.186+0000 7f9b29822700 1 -- 172.21.15.16:0/2193818085 wait complete. 2024-04-02T04:38:31.930 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":19,"active_gid":14217,"active_name":"smithi016.eqnebx","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3643600900},{"type":"v1","addr":"172.21.15.16:6801","nonce":3643600900}]},"active_addr":"172.21.15.16:6801/3643600900","active_change":"2024-04-02T04:34:42.713645+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":14240,"name":"smithi027.kjzcsc","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.16:8443/","prometheus":"http://172.21.15.16:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":1664387083}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":1383394471}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":1920858754}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":2373084595}]}]} 2024-04-02T04:38:31.934 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-02T04:38:31.934 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-02T04:38:31.934 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph osd dump --format=json 2024-04-02T04:38:32.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:31 smithi027 ceph-mon[37550]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:32.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:31 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/2193818085' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-02T04:38:32.183 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:38:32.304 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:31 smithi016 ceph-mon[28969]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:32.304 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:31 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/2193818085' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-02T04:38:33.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:33.848+0000 7fc036428700 1 -- 172.21.15.16:0/117211979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0300fefb0 msgr2=0x7fc0300ff430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:33.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:33.848+0000 7fc036428700 1 --2- 172.21.15.16:0/117211979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0300fefb0 0x7fc0300ff430 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc018009910 tx=0x7fc018009c20 comp rx=0 tx=0).stop 2024-04-02T04:38:33.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:33.848+0000 7fc036428700 1 -- 172.21.15.16:0/117211979 shutdown_connections 2024-04-02T04:38:33.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:33.848+0000 7fc036428700 1 --2- 172.21.15.16:0/117211979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0300fefb0 0x7fc0300ff430 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:33.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:33.848+0000 7fc036428700 1 --2- 172.21.15.16:0/117211979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc030105a80 0x7fc030105e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:33.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:33.848+0000 7fc036428700 1 -- 172.21.15.16:0/117211979 >> 172.21.15.16:0/117211979 conn(0x7fc030075e10 msgr2=0x7fc0300fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:33.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:33.848+0000 7fc036428700 1 -- 172.21.15.16:0/117211979 shutdown_connections 2024-04-02T04:38:33.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:33.848+0000 7fc036428700 1 -- 172.21.15.16:0/117211979 wait complete. 2024-04-02T04:38:33.854 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:33 smithi016 ceph-mon[28969]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:33.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:33.849+0000 7fc036428700 1 Processor -- start 2024-04-02T04:38:33.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:33.849+0000 7fc036428700 1 -- start start 2024-04-02T04:38:33.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:33.850+0000 7fc036428700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0300fefb0 0x7fc030100190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:33.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:33.850+0000 7fc036428700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc030105a80 0x7fc0301006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:33.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:33.850+0000 7fc036428700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc030100c10 con 0x7fc0300fefb0 2024-04-02T04:38:33.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:33.850+0000 7fc036428700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc030100d80 con 0x7fc030105a80 2024-04-02T04:38:33.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:33.850+0000 7fc02ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0300fefb0 0x7fc030100190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:33.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:33.850+0000 7fc02ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0300fefb0 0x7fc030100190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49418/0 (socket says 172.21.15.16:49418) 2024-04-02T04:38:33.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:33.850+0000 7fc02ffff700 1 -- 172.21.15.16:0/2865909874 learned_addr learned my addr 172.21.15.16:0/2865909874 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:38:33.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:33.850+0000 7fc02f7fe700 1 --2- 172.21.15.16:0/2865909874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc030105a80 0x7fc0301006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:33.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:33.850+0000 7fc02ffff700 1 -- 172.21.15.16:0/2865909874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc030105a80 msgr2=0x7fc0301006d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:33.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:33.850+0000 7fc02ffff700 1 --2- 172.21.15.16:0/2865909874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc030105a80 0x7fc0301006d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:33.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:33.850+0000 7fc02ffff700 1 -- 172.21.15.16:0/2865909874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0180095f0 con 0x7fc0300fefb0 2024-04-02T04:38:33.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:33.850+0000 7fc02ffff700 1 --2- 172.21.15.16:0/2865909874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0300fefb0 0x7fc030100190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc02000c8a0 tx=0x7fc02000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:33.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:33.850+0000 7fc02d7fa700 1 -- 172.21.15.16:0/2865909874 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0200078e0 con 0x7fc0300fefb0 2024-04-02T04:38:33.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:33.850+0000 7fc02d7fa700 1 -- 172.21.15.16:0/2865909874 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc02000f450 con 0x7fc0300fefb0 2024-04-02T04:38:33.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:33.851+0000 7fc036428700 1 -- 172.21.15.16:0/2865909874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0301a2c20 con 0x7fc0300fefb0 2024-04-02T04:38:33.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:33.851+0000 7fc02d7fa700 1 -- 172.21.15.16:0/2865909874 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc02000e5e0 con 0x7fc0300fefb0 2024-04-02T04:38:33.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:33.851+0000 7fc036428700 1 -- 172.21.15.16:0/2865909874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0301a2f80 con 0x7fc0300fefb0 2024-04-02T04:38:33.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:33.852+0000 7fc036428700 1 -- 172.21.15.16:0/2865909874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc030108fc0 con 0x7fc0300fefb0 2024-04-02T04:38:33.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:33.852+0000 7fc02d7fa700 1 -- 172.21.15.16:0/2865909874 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fc02000f5c0 con 0x7fc0300fefb0 2024-04-02T04:38:33.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:33.853+0000 7fc02d7fa700 1 --2- 172.21.15.16:0/2865909874 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc01c06cce0 0x7fc01c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:33.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:33.853+0000 7fc02d7fa700 1 -- 172.21.15.16:0/2865909874 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fc02008bed0 con 0x7fc0300fefb0 2024-04-02T04:38:33.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:33.853+0000 7fc02f7fe700 1 --2- 172.21.15.16:0/2865909874 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc01c06cce0 0x7fc01c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:33.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:33.853+0000 7fc02f7fe700 1 --2- 172.21.15.16:0/2865909874 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc01c06cce0 0x7fc01c06f1a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc0300fe5a0 tx=0x7fc01801a040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:33.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:33.857+0000 7fc02d7fa700 1 -- 172.21.15.16:0/2865909874 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc0200565e0 con 0x7fc0300fefb0 2024-04-02T04:38:34.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:34.057+0000 7fc036428700 1 -- 172.21.15.16:0/2865909874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc03004ea80 con 0x7fc0300fefb0 2024-04-02T04:38:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:34.058+0000 7fc02d7fa700 1 -- 172.21.15.16:0/2865909874 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11273 (secure 0 0 0) 0x7fc020059c00 con 0x7fc0300fefb0 2024-04-02T04:38:34.062 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:38:34.062 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":40,"fsid":"dc5403b4-f0a9-11ee-b647-cb9ed24678a4","created":"2024-04-02T04:32:23.014546+0000","modified":"2024-04-02T04:38:17.982526+0000","last_up_change":"2024-04-02T04:38:16.977561+0000","last_in_change":"2024-04-02T04:37:56.807214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T04:37:01.047338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2b7be29b-0e2a-456b-b33e-c72abaad9136","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":906882291},{"type":"v1","addr":"172.21.15.16:6803","nonce":906882291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":906882291},{"type":"v1","addr":"172.21.15.16:6805","nonce":906882291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":906882291},{"type":"v1","addr":"172.21.15.16:6809","nonce":906882291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":906882291},{"type":"v1","addr":"172.21.15.16:6807","nonce":906882291}]},"public_addr":"172.21.15.16:6803/906882291","cluster_addr":"172.21.15.16:6805/906882291","heartbeat_back_addr":"172.21.15.16:6809/906882291","heartbeat_front_addr":"172.21.15.16:6807/906882291","state":["exists","up"]},{"osd":1,"uuid":"82578291-3c9c-4f3f-8de9-7a8710a4923e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3079548081},{"type":"v1","addr":"172.21.15.16:6811","nonce":3079548081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3079548081},{"type":"v1","addr":"172.21.15.16:6813","nonce":3079548081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3079548081},{"type":"v1","addr":"172.21.15.16:6817","nonce":3079548081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3079548081},{"type":"v1","addr":"172.21.15.16:6815","nonce":3079548081}]},"public_addr":"172.21.15.16:6811/3079548081","cluster_addr":"172.21.15.16:6813/3079548081","heartbeat_back_addr":"172.21.15.16:6817/3079548081","heartbeat_front_addr":"172.21.15.16:6815/3079548081","state":["exists","up"]},{"osd":2,"uuid":"046262b5-386e-4270-863c-e97a61a038b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3734332239},{"type":"v1","addr":"172.21.15.16:6819","nonce":3734332239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3734332239},{"type":"v1","addr":"172.21.15.16:6821","nonce":3734332239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3734332239},{"type":"v1","addr":"172.21.15.16:6825","nonce":3734332239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3734332239},{"type":"v1","addr":"172.21.15.16:6823","nonce":3734332239}]},"public_addr":"172.21.15.16:6819/3734332239","cluster_addr":"172.21.15.16:6821/3734332239","heartbeat_back_addr":"172.21.15.16:6825/3734332239","heartbeat_front_addr":"172.21.15.16:6823/3734332239","state":["exists","up"]},{"osd":3,"uuid":"637f2a77-59a2-44fb-b47e-fcbd70f4423c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":388124754},{"type":"v1","addr":"172.21.15.27:6801","nonce":388124754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":388124754},{"type":"v1","addr":"172.21.15.27:6803","nonce":388124754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":388124754},{"type":"v1","addr":"172.21.15.27:6807","nonce":388124754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":388124754},{"type":"v1","addr":"172.21.15.27:6805","nonce":388124754}]},"public_addr":"172.21.15.27:6801/388124754","cluster_addr":"172.21.15.27:6803/388124754","heartbeat_back_addr":"172.21.15.27:6807/388124754","heartbeat_front_addr":"172.21.15.27:6805/388124754","state":["exists","up"]},{"osd":4,"uuid":"334b7b3d-505f-45b6-85b0-59ec8eeb8e91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":935993381},{"type":"v1","addr":"172.21.15.27:6809","nonce":935993381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":935993381},{"type":"v1","addr":"172.21.15.27:6811","nonce":935993381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":935993381},{"type":"v1","addr":"172.21.15.27:6815","nonce":935993381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":935993381},{"type":"v1","addr":"172.21.15.27:6813","nonce":935993381}]},"public_addr":"172.21.15.27:6809/935993381","cluster_addr":"172.21.15.27:6811/935993381","heartbeat_back_addr":"172.21.15.27:6815/935993381","heartbeat_front_addr":"172.21.15.27:6813/935993381","state":["exists","up"]},{"osd":5,"uuid":"f83802ee-4f47-47de-91dd-a667f6cd22e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2454420135},{"type":"v1","addr":"172.21.15.27:6817","nonce":2454420135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":2454420135},{"type":"v1","addr":"172.21.15.27:6819","nonce":2454420135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":2454420135},{"type":"v1","addr":"172.21.15.27:6823","nonce":2454420135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":2454420135},{"type":"v1","addr":"172.21.15.27:6821","nonce":2454420135}]},"public_addr":"172.21.15.27:6817/2454420135","cluster_addr":"172.21.15.27:6819/2454420135","heartbeat_back_addr":"172.21.15.27:6823/2454420135","heartbeat_front_addr":"172.21.15.27:6821/2454420135","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:36:02.027852+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:36:29.393593+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:36:56.847224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:37:21.195876+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:37:47.992167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:38:15.012477+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/4209328561":"2024-04-03T04:34:42.713541+0000","172.21.15.16:0/3733852550":"2024-04-03T04:34:42.713541+0000","172.21.15.16:0/3405957790":"2024-04-03T04:34:42.713541+0000","172.21.15.16:6800/3912038174":"2024-04-03T04:34:42.713541+0000","172.21.15.16:6801/3912038174":"2024-04-03T04:34:42.713541+0000","172.21.15.16:0/4067493740":"2024-04-03T04:33:28.907145+0000","172.21.15.16:6801/873531241":"2024-04-03T04:33:28.907145+0000","172.21.15.16:6800/873531241":"2024-04-03T04:33:28.907145+0000","172.21.15.16:0/1228221358":"2024-04-03T04:33:28.907145+0000","172.21.15.16:6800/824107799":"2024-04-03T04:32:56.644452+0000","172.21.15.16:0/2255383579":"2024-04-03T04:32:56.644452+0000","172.21.15.16:0/401549934":"2024-04-03T04:32:56.644452+0000","172.21.15.16:6801/824107799":"2024-04-03T04:32:56.644452+0000","172.21.15.16:0/1884756010":"2024-04-03T04:32:56.644452+0000","172.21.15.16:0/166410319":"2024-04-03T04:33:28.907145+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T04:38:34.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:34.060+0000 7fc036428700 1 -- 172.21.15.16:0/2865909874 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc01c06cce0 msgr2=0x7fc01c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:34.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:34.060+0000 7fc036428700 1 --2- 172.21.15.16:0/2865909874 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc01c06cce0 0x7fc01c06f1a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc0300fe5a0 tx=0x7fc01801a040 comp rx=0 tx=0).stop 2024-04-02T04:38:34.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:34.060+0000 7fc036428700 1 -- 172.21.15.16:0/2865909874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0300fefb0 msgr2=0x7fc030100190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:34.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:34.060+0000 7fc036428700 1 --2- 172.21.15.16:0/2865909874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0300fefb0 0x7fc030100190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc02000c8a0 tx=0x7fc02000cbb0 comp rx=0 tx=0).stop 2024-04-02T04:38:34.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:34.060+0000 7fc036428700 1 -- 172.21.15.16:0/2865909874 shutdown_connections 2024-04-02T04:38:34.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:34.060+0000 7fc036428700 1 --2- 172.21.15.16:0/2865909874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0300fefb0 0x7fc030100190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:34.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:34.060+0000 7fc036428700 1 --2- 172.21.15.16:0/2865909874 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc01c06cce0 0x7fc01c06f1a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:34.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:34.060+0000 7fc036428700 1 --2- 172.21.15.16:0/2865909874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc030105a80 0x7fc0301006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:34.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:34.060+0000 7fc036428700 1 -- 172.21.15.16:0/2865909874 >> 172.21.15.16:0/2865909874 conn(0x7fc030075e10 msgr2=0x7fc0300fb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:34.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:34.060+0000 7fc036428700 1 -- 172.21.15.16:0/2865909874 shutdown_connections 2024-04-02T04:38:34.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:34.060+0000 7fc036428700 1 -- 172.21.15.16:0/2865909874 wait complete. 2024-04-02T04:38:34.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:33 smithi027 ceph-mon[37550]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:34.763 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-02T04:38:34.764 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph osd dump --format=json 2024-04-02T04:38:34.979 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:38:35.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:34 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/2865909874' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T04:38:35.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:34 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/2865909874' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T04:38:36.092 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:35 smithi016 ceph-mon[28969]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:36.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:35 smithi027 ceph-mon[37550]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:36.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.587+0000 7f51300f9700 1 -- 172.21.15.16:0/2877301725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5128100a80 msgr2=0x7f5128100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:36.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.587+0000 7f51300f9700 1 --2- 172.21.15.16:0/2877301725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5128100a80 0x7f5128100f00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5124009910 tx=0x7f5124009c20 comp rx=0 tx=0).stop 2024-04-02T04:38:36.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.587+0000 7f51300f9700 1 -- 172.21.15.16:0/2877301725 shutdown_connections 2024-04-02T04:38:36.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.587+0000 7f51300f9700 1 --2- 172.21.15.16:0/2877301725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5128100a80 0x7f5128100f00 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:36.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.587+0000 7f51300f9700 1 --2- 172.21.15.16:0/2877301725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5128106a80 0x7f5128106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:36.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.587+0000 7f51300f9700 1 -- 172.21.15.16:0/2877301725 >> 172.21.15.16:0/2877301725 conn(0x7f51280fc5b0 msgr2=0x7f51280fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:36.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.587+0000 7f51300f9700 1 -- 172.21.15.16:0/2877301725 shutdown_connections 2024-04-02T04:38:36.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.587+0000 7f51300f9700 1 -- 172.21.15.16:0/2877301725 wait complete. 2024-04-02T04:38:36.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.588+0000 7f51300f9700 1 Processor -- start 2024-04-02T04:38:36.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.588+0000 7f51300f9700 1 -- start start 2024-04-02T04:38:36.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.588+0000 7f51300f9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5128100a80 0x7f512819c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:36.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.588+0000 7f51300f9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5128106a80 0x7f512819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:36.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.588+0000 7f51300f9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512819d280 con 0x7f5128100a80 2024-04-02T04:38:36.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.588+0000 7f51300f9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5128196840 con 0x7f5128106a80 2024-04-02T04:38:36.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.588+0000 7f512de95700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5128100a80 0x7f512819c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:36.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.588+0000 7f512de95700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5128100a80 0x7f512819c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49430/0 (socket says 172.21.15.16:49430) 2024-04-02T04:38:36.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.588+0000 7f512de95700 1 -- 172.21.15.16:0/3659599746 learned_addr learned my addr 172.21.15.16:0/3659599746 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:38:36.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.588+0000 7f512d694700 1 --2- 172.21.15.16:0/3659599746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5128106a80 0x7f512819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:36.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.588+0000 7f512de95700 1 -- 172.21.15.16:0/3659599746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5128106a80 msgr2=0x7f512819cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:36.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.588+0000 7f512de95700 1 --2- 172.21.15.16:0/3659599746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5128106a80 0x7f512819cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:36.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.588+0000 7f512de95700 1 -- 172.21.15.16:0/3659599746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51240095f0 con 0x7f5128100a80 2024-04-02T04:38:36.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.588+0000 7f512d694700 1 --2- 172.21.15.16:0/3659599746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5128106a80 0x7f512819cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T04:38:36.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.589+0000 7f512de95700 1 --2- 172.21.15.16:0/3659599746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5128100a80 0x7f512819c6b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f511800d8d0 tx=0x7f511800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:36.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.589+0000 7f511effd700 1 -- 172.21.15.16:0/3659599746 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5118009940 con 0x7f5128100a80 2024-04-02T04:38:36.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.589+0000 7f511effd700 1 -- 172.21.15.16:0/3659599746 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5118010460 con 0x7f5128100a80 2024-04-02T04:38:36.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.589+0000 7f511effd700 1 -- 172.21.15.16:0/3659599746 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f511800f5f0 con 0x7f5128100a80 2024-04-02T04:38:36.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.589+0000 7f51300f9700 1 -- 172.21.15.16:0/3659599746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5128196b20 con 0x7f5128100a80 2024-04-02T04:38:36.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.589+0000 7f51300f9700 1 -- 172.21.15.16:0/3659599746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5128197070 con 0x7f5128100a80 2024-04-02T04:38:36.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.590+0000 7f51300f9700 1 -- 172.21.15.16:0/3659599746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51281090a0 con 0x7f5128100a80 2024-04-02T04:38:36.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.590+0000 7f511effd700 1 -- 172.21.15.16:0/3659599746 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f51180105d0 con 0x7f5128100a80 2024-04-02T04:38:36.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.591+0000 7f511effd700 1 --2- 172.21.15.16:0/3659599746 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f511406cd30 0x7f511406f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:36.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.591+0000 7f511effd700 1 -- 172.21.15.16:0/3659599746 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f511808bfb0 con 0x7f5128100a80 2024-04-02T04:38:36.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.591+0000 7f512d694700 1 --2- 172.21.15.16:0/3659599746 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f511406cd30 0x7f511406f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:36.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.591+0000 7f512d694700 1 --2- 172.21.15.16:0/3659599746 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f511406cd30 0x7f511406f1f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f51240059d0 tx=0x7f512401a040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:36.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.595+0000 7f511effd700 1 -- 172.21.15.16:0/3659599746 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f511805a0b0 con 0x7f5128100a80 2024-04-02T04:38:36.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.798+0000 7f51300f9700 1 -- 172.21.15.16:0/3659599746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5128197d50 con 0x7f5128100a80 2024-04-02T04:38:36.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.799+0000 7f511effd700 1 -- 172.21.15.16:0/3659599746 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11273 (secure 0 0 0) 0x7f5118059c40 con 0x7f5128100a80 2024-04-02T04:38:36.803 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:38:36.803 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":40,"fsid":"dc5403b4-f0a9-11ee-b647-cb9ed24678a4","created":"2024-04-02T04:32:23.014546+0000","modified":"2024-04-02T04:38:17.982526+0000","last_up_change":"2024-04-02T04:38:16.977561+0000","last_in_change":"2024-04-02T04:37:56.807214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T04:37:01.047338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2b7be29b-0e2a-456b-b33e-c72abaad9136","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":906882291},{"type":"v1","addr":"172.21.15.16:6803","nonce":906882291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":906882291},{"type":"v1","addr":"172.21.15.16:6805","nonce":906882291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":906882291},{"type":"v1","addr":"172.21.15.16:6809","nonce":906882291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":906882291},{"type":"v1","addr":"172.21.15.16:6807","nonce":906882291}]},"public_addr":"172.21.15.16:6803/906882291","cluster_addr":"172.21.15.16:6805/906882291","heartbeat_back_addr":"172.21.15.16:6809/906882291","heartbeat_front_addr":"172.21.15.16:6807/906882291","state":["exists","up"]},{"osd":1,"uuid":"82578291-3c9c-4f3f-8de9-7a8710a4923e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3079548081},{"type":"v1","addr":"172.21.15.16:6811","nonce":3079548081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3079548081},{"type":"v1","addr":"172.21.15.16:6813","nonce":3079548081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3079548081},{"type":"v1","addr":"172.21.15.16:6817","nonce":3079548081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3079548081},{"type":"v1","addr":"172.21.15.16:6815","nonce":3079548081}]},"public_addr":"172.21.15.16:6811/3079548081","cluster_addr":"172.21.15.16:6813/3079548081","heartbeat_back_addr":"172.21.15.16:6817/3079548081","heartbeat_front_addr":"172.21.15.16:6815/3079548081","state":["exists","up"]},{"osd":2,"uuid":"046262b5-386e-4270-863c-e97a61a038b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3734332239},{"type":"v1","addr":"172.21.15.16:6819","nonce":3734332239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3734332239},{"type":"v1","addr":"172.21.15.16:6821","nonce":3734332239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3734332239},{"type":"v1","addr":"172.21.15.16:6825","nonce":3734332239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3734332239},{"type":"v1","addr":"172.21.15.16:6823","nonce":3734332239}]},"public_addr":"172.21.15.16:6819/3734332239","cluster_addr":"172.21.15.16:6821/3734332239","heartbeat_back_addr":"172.21.15.16:6825/3734332239","heartbeat_front_addr":"172.21.15.16:6823/3734332239","state":["exists","up"]},{"osd":3,"uuid":"637f2a77-59a2-44fb-b47e-fcbd70f4423c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":388124754},{"type":"v1","addr":"172.21.15.27:6801","nonce":388124754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":388124754},{"type":"v1","addr":"172.21.15.27:6803","nonce":388124754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":388124754},{"type":"v1","addr":"172.21.15.27:6807","nonce":388124754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":388124754},{"type":"v1","addr":"172.21.15.27:6805","nonce":388124754}]},"public_addr":"172.21.15.27:6801/388124754","cluster_addr":"172.21.15.27:6803/388124754","heartbeat_back_addr":"172.21.15.27:6807/388124754","heartbeat_front_addr":"172.21.15.27:6805/388124754","state":["exists","up"]},{"osd":4,"uuid":"334b7b3d-505f-45b6-85b0-59ec8eeb8e91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":935993381},{"type":"v1","addr":"172.21.15.27:6809","nonce":935993381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":935993381},{"type":"v1","addr":"172.21.15.27:6811","nonce":935993381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":935993381},{"type":"v1","addr":"172.21.15.27:6815","nonce":935993381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":935993381},{"type":"v1","addr":"172.21.15.27:6813","nonce":935993381}]},"public_addr":"172.21.15.27:6809/935993381","cluster_addr":"172.21.15.27:6811/935993381","heartbeat_back_addr":"172.21.15.27:6815/935993381","heartbeat_front_addr":"172.21.15.27:6813/935993381","state":["exists","up"]},{"osd":5,"uuid":"f83802ee-4f47-47de-91dd-a667f6cd22e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2454420135},{"type":"v1","addr":"172.21.15.27:6817","nonce":2454420135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":2454420135},{"type":"v1","addr":"172.21.15.27:6819","nonce":2454420135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":2454420135},{"type":"v1","addr":"172.21.15.27:6823","nonce":2454420135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":2454420135},{"type":"v1","addr":"172.21.15.27:6821","nonce":2454420135}]},"public_addr":"172.21.15.27:6817/2454420135","cluster_addr":"172.21.15.27:6819/2454420135","heartbeat_back_addr":"172.21.15.27:6823/2454420135","heartbeat_front_addr":"172.21.15.27:6821/2454420135","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:36:02.027852+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:36:29.393593+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:36:56.847224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:37:21.195876+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:37:47.992167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:38:15.012477+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/4209328561":"2024-04-03T04:34:42.713541+0000","172.21.15.16:0/3733852550":"2024-04-03T04:34:42.713541+0000","172.21.15.16:0/3405957790":"2024-04-03T04:34:42.713541+0000","172.21.15.16:6800/3912038174":"2024-04-03T04:34:42.713541+0000","172.21.15.16:6801/3912038174":"2024-04-03T04:34:42.713541+0000","172.21.15.16:0/4067493740":"2024-04-03T04:33:28.907145+0000","172.21.15.16:6801/873531241":"2024-04-03T04:33:28.907145+0000","172.21.15.16:6800/873531241":"2024-04-03T04:33:28.907145+0000","172.21.15.16:0/1228221358":"2024-04-03T04:33:28.907145+0000","172.21.15.16:6800/824107799":"2024-04-03T04:32:56.644452+0000","172.21.15.16:0/2255383579":"2024-04-03T04:32:56.644452+0000","172.21.15.16:0/401549934":"2024-04-03T04:32:56.644452+0000","172.21.15.16:6801/824107799":"2024-04-03T04:32:56.644452+0000","172.21.15.16:0/1884756010":"2024-04-03T04:32:56.644452+0000","172.21.15.16:0/166410319":"2024-04-03T04:33:28.907145+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T04:38:36.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.801+0000 7f51300f9700 1 -- 172.21.15.16:0/3659599746 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f511406cd30 msgr2=0x7f511406f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:36.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.801+0000 7f51300f9700 1 --2- 172.21.15.16:0/3659599746 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f511406cd30 0x7f511406f1f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f51240059d0 tx=0x7f512401a040 comp rx=0 tx=0).stop 2024-04-02T04:38:36.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.801+0000 7f51300f9700 1 -- 172.21.15.16:0/3659599746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5128100a80 msgr2=0x7f512819c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:36.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.801+0000 7f51300f9700 1 --2- 172.21.15.16:0/3659599746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5128100a80 0x7f512819c6b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f511800d8d0 tx=0x7f511800dc90 comp rx=0 tx=0).stop 2024-04-02T04:38:36.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.801+0000 7f51300f9700 1 -- 172.21.15.16:0/3659599746 shutdown_connections 2024-04-02T04:38:36.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.801+0000 7f51300f9700 1 --2- 172.21.15.16:0/3659599746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5128100a80 0x7f512819c6b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:36.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.801+0000 7f51300f9700 1 --2- 172.21.15.16:0/3659599746 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f511406cd30 0x7f511406f1f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:36.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.801+0000 7f51300f9700 1 --2- 172.21.15.16:0/3659599746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5128106a80 0x7f512819cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:36.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.801+0000 7f51300f9700 1 -- 172.21.15.16:0/3659599746 >> 172.21.15.16:0/3659599746 conn(0x7f51280fc5b0 msgr2=0x7f51280fd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:36.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.801+0000 7f51300f9700 1 -- 172.21.15.16:0/3659599746 shutdown_connections 2024-04-02T04:38:36.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:36.802+0000 7f51300f9700 1 -- 172.21.15.16:0/3659599746 wait complete. 2024-04-02T04:38:37.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:36 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/3659599746' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T04:38:37.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:36 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/3659599746' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T04:38:37.500 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph tell osd.0 flush_pg_stats 2024-04-02T04:38:37.501 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph tell osd.1 flush_pg_stats 2024-04-02T04:38:37.501 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph tell osd.2 flush_pg_stats 2024-04-02T04:38:37.502 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph tell osd.3 flush_pg_stats 2024-04-02T04:38:37.502 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph tell osd.4 flush_pg_stats 2024-04-02T04:38:37.502 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph tell osd.5 flush_pg_stats 2024-04-02T04:38:37.770 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:38:37.820 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:38:38.137 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:37 smithi016 ceph-mon[28969]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:38.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:37 smithi027 ceph-mon[37550]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:39.809 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:39 smithi016 ceph-mon[28969]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:38:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:38:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:38:39.942 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:38:40.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:39 smithi027 ceph-mon[37550]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:40.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.896+0000 7f4dba728700 1 -- 172.21.15.16:0/2127690187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db41025e0 msgr2=0x7f4db410aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:40.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.896+0000 7f4dba728700 1 --2- 172.21.15.16:0/2127690187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db41025e0 0x7f4db410aad0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4da8009940 tx=0x7f4da8009c50 comp rx=0 tx=0).stop 2024-04-02T04:38:40.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.896+0000 7f4dba728700 1 -- 172.21.15.16:0/2127690187 shutdown_connections 2024-04-02T04:38:40.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.896+0000 7f4dba728700 1 --2- 172.21.15.16:0/2127690187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db41025e0 0x7f4db410aad0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:40.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.896+0000 7f4dba728700 1 --2- 172.21.15.16:0/2127690187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4db4101cc0 0x7f4db41020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:40.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.896+0000 7f4dba728700 1 -- 172.21.15.16:0/2127690187 >> 172.21.15.16:0/2127690187 conn(0x7f4db40fb4b0 msgr2=0x7f4db40fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:40.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.896+0000 7f4dba728700 1 -- 172.21.15.16:0/2127690187 shutdown_connections 2024-04-02T04:38:40.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.896+0000 7f4dba728700 1 -- 172.21.15.16:0/2127690187 wait complete. 2024-04-02T04:38:40.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.897+0000 7f4dba728700 1 Processor -- start 2024-04-02T04:38:40.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.897+0000 7f4dba728700 1 -- start start 2024-04-02T04:38:40.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.897+0000 7f4dba728700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db4101cc0 0x7f4db4196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:40.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.897+0000 7f4dba728700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4db41025e0 0x7f4db4196f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:40.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.897+0000 7f4dba728700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db41975e0 con 0x7f4db4101cc0 2024-04-02T04:38:40.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.897+0000 7f4dba728700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db419b330 con 0x7f4db41025e0 2024-04-02T04:38:40.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.897+0000 7f4db3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db4101cc0 0x7f4db4196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:40.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.897+0000 7f4db3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db4101cc0 0x7f4db4196a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49474/0 (socket says 172.21.15.16:49474) 2024-04-02T04:38:40.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.897+0000 7f4db3fff700 1 -- 172.21.15.16:0/3748891348 learned_addr learned my addr 172.21.15.16:0/3748891348 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:38:40.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.897+0000 7f4db37fe700 1 --2- 172.21.15.16:0/3748891348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4db41025e0 0x7f4db4196f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:40.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.898+0000 7f4db3fff700 1 -- 172.21.15.16:0/3748891348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4db41025e0 msgr2=0x7f4db4196f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:40.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.898+0000 7f4db3fff700 1 --2- 172.21.15.16:0/3748891348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4db41025e0 0x7f4db4196f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:40.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.898+0000 7f4db3fff700 1 -- 172.21.15.16:0/3748891348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da80095f0 con 0x7f4db4101cc0 2024-04-02T04:38:40.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.898+0000 7f4db37fe700 1 --2- 172.21.15.16:0/3748891348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4db41025e0 0x7f4db4196f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T04:38:40.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.898+0000 7f4db3fff700 1 --2- 172.21.15.16:0/3748891348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db4101cc0 0x7f4db4196a10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4da400d8d0 tx=0x7f4da400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:40.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.898+0000 7f4db17fa700 1 -- 172.21.15.16:0/3748891348 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4da4009940 con 0x7f4db4101cc0 2024-04-02T04:38:40.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.898+0000 7f4dba728700 1 -- 172.21.15.16:0/3748891348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4db419b610 con 0x7f4db4101cc0 2024-04-02T04:38:40.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.898+0000 7f4db17fa700 1 -- 172.21.15.16:0/3748891348 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4da4010460 con 0x7f4db4101cc0 2024-04-02T04:38:40.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.898+0000 7f4dba728700 1 -- 172.21.15.16:0/3748891348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4db419bb30 con 0x7f4db4101cc0 2024-04-02T04:38:40.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.898+0000 7f4db17fa700 1 -- 172.21.15.16:0/3748891348 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4da400f5f0 con 0x7f4db4101cc0 2024-04-02T04:38:40.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.899+0000 7f4dba728700 1 -- 172.21.15.16:0/3748891348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4db410dfc0 con 0x7f4db4101cc0 2024-04-02T04:38:40.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.899+0000 7f4db17fa700 1 -- 172.21.15.16:0/3748891348 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f4da40105d0 con 0x7f4db4101cc0 2024-04-02T04:38:40.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.900+0000 7f4db17fa700 1 --2- 172.21.15.16:0/3748891348 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f4d9c06cce0 0x7f4d9c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:40.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.900+0000 7f4db17fa700 1 -- 172.21.15.16:0/3748891348 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f4da408c400 con 0x7f4db4101cc0 2024-04-02T04:38:40.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.900+0000 7f4db37fe700 1 --2- 172.21.15.16:0/3748891348 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f4d9c06cce0 0x7f4d9c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:40.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.900+0000 7f4db17fa700 1 --2- 172.21.15.16:0/3748891348 >> [v2:172.21.15.27:6816/2454420135,v1:172.21.15.27:6817/2454420135] conn(0x7f4d9c0729c0 0x7f4d9c074de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:40.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.900+0000 7f4db17fa700 1 -- 172.21.15.16:0/3748891348 --> [v2:172.21.15.27:6816/2454420135,v1:172.21.15.27:6817/2454420135] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4d9c075490 con 0x7f4d9c0729c0 2024-04-02T04:38:40.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.900+0000 7f4db17fa700 1 -- 172.21.15.16:0/3748891348 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4da408c860 con 0x7f4db4101cc0 2024-04-02T04:38:40.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.900+0000 7f4db37fe700 1 --2- 172.21.15.16:0/3748891348 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f4d9c06cce0 0x7f4d9c06f1a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4da8006010 tx=0x7f4da8005940 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:40.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.900+0000 7f4db8cc5700 1 --2- 172.21.15.16:0/3748891348 >> [v2:172.21.15.27:6816/2454420135,v1:172.21.15.27:6817/2454420135] conn(0x7f4d9c0729c0 0x7f4d9c074de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:40.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.901+0000 7f4db8cc5700 1 --2- 172.21.15.16:0/3748891348 >> [v2:172.21.15.27:6816/2454420135,v1:172.21.15.27:6817/2454420135] conn(0x7f4d9c0729c0 0x7f4d9c074de0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:40.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.902+0000 7f4db17fa700 1 -- 172.21.15.16:0/3748891348 <== osd.5 v2:172.21.15.27:6816/2454420135 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f4d9c075490 con 0x7f4d9c0729c0 2024-04-02T04:38:40.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.917+0000 7f4dba728700 1 -- 172.21.15.16:0/3748891348 --> [v2:172.21.15.27:6816/2454420135,v1:172.21.15.27:6817/2454420135] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4db4069850 con 0x7f4d9c0729c0 2024-04-02T04:38:40.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.918+0000 7f4db17fa700 1 -- 172.21.15.16:0/3748891348 <== osd.5 v2:172.21.15.27:6816/2454420135 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f4db4069850 con 0x7f4d9c0729c0 2024-04-02T04:38:40.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.918+0000 7f4dba728700 1 -- 172.21.15.16:0/3748891348 >> [v2:172.21.15.27:6816/2454420135,v1:172.21.15.27:6817/2454420135] conn(0x7f4d9c0729c0 msgr2=0x7f4d9c074de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:40.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.918+0000 7f4dba728700 1 --2- 172.21.15.16:0/3748891348 >> [v2:172.21.15.27:6816/2454420135,v1:172.21.15.27:6817/2454420135] conn(0x7f4d9c0729c0 0x7f4d9c074de0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:40.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.918+0000 7f4dba728700 1 -- 172.21.15.16:0/3748891348 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f4d9c06cce0 msgr2=0x7f4d9c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:40.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.918+0000 7f4dba728700 1 --2- 172.21.15.16:0/3748891348 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f4d9c06cce0 0x7f4d9c06f1a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4da8006010 tx=0x7f4da8005940 comp rx=0 tx=0).stop 2024-04-02T04:38:40.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.918+0000 7f4dba728700 1 -- 172.21.15.16:0/3748891348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db4101cc0 msgr2=0x7f4db4196a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:40.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.918+0000 7f4dba728700 1 --2- 172.21.15.16:0/3748891348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db4101cc0 0x7f4db4196a10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4da400d8d0 tx=0x7f4da400dc90 comp rx=0 tx=0).stop 2024-04-02T04:38:40.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.918+0000 7f4dba728700 1 -- 172.21.15.16:0/3748891348 shutdown_connections 2024-04-02T04:38:40.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.918+0000 7f4dba728700 1 --2- 172.21.15.16:0/3748891348 >> [v2:172.21.15.27:6816/2454420135,v1:172.21.15.27:6817/2454420135] conn(0x7f4d9c0729c0 0x7f4d9c074de0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:40.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.918+0000 7f4dba728700 1 --2- 172.21.15.16:0/3748891348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4db4101cc0 0x7f4db4196a10 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:40.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.918+0000 7f4dba728700 1 --2- 172.21.15.16:0/3748891348 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f4d9c06cce0 0x7f4d9c06f1a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:40.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.918+0000 7f4dba728700 1 --2- 172.21.15.16:0/3748891348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4db41025e0 0x7f4db4196f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:40.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.919+0000 7f4dba728700 1 -- 172.21.15.16:0/3748891348 >> 172.21.15.16:0/3748891348 conn(0x7f4db40fb4b0 msgr2=0x7f4db4102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:40.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.919+0000 7f4dba728700 1 -- 172.21.15.16:0/3748891348 shutdown_connections 2024-04-02T04:38:40.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.919+0000 7f4dba728700 1 -- 172.21.15.16:0/3748891348 wait complete. 2024-04-02T04:38:41.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.780+0000 7fa40b2fa700 1 -- 172.21.15.16:0/479966645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa404106a70 msgr2=0x7fa404106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:41.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.780+0000 7fa40b2fa700 1 --2- 172.21.15.16:0/479966645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa404106a70 0x7fa404106e50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa400009910 tx=0x7fa400009c20 comp rx=0 tx=0).stop 2024-04-02T04:38:41.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.781+0000 7fa40b2fa700 1 -- 172.21.15.16:0/479966645 shutdown_connections 2024-04-02T04:38:41.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.781+0000 7fa40b2fa700 1 --2- 172.21.15.16:0/479966645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa404100a70 0x7fa404100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:41.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.781+0000 7fa40b2fa700 1 --2- 172.21.15.16:0/479966645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa404106a70 0x7fa404106e50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:41.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.781+0000 7fa40b2fa700 1 -- 172.21.15.16:0/479966645 >> 172.21.15.16:0/479966645 conn(0x7fa4040fc5a0 msgr2=0x7fa4040fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:41.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.781+0000 7fa40b2fa700 1 -- 172.21.15.16:0/479966645 shutdown_connections 2024-04-02T04:38:41.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.781+0000 7fa40b2fa700 1 -- 172.21.15.16:0/479966645 wait complete. 2024-04-02T04:38:41.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.781+0000 7fa40b2fa700 1 Processor -- start 2024-04-02T04:38:41.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.781+0000 7fa40b2fa700 1 -- start start 2024-04-02T04:38:41.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.781+0000 7fa40b2fa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa404100a70 0x7fa4040730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:41.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.781+0000 7fa40b2fa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa404106a70 0x7fa4040735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:41.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.781+0000 7fa40b2fa700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa404073b30 con 0x7fa404106a70 2024-04-02T04:38:41.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.781+0000 7fa40b2fa700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa404073c70 con 0x7fa404100a70 2024-04-02T04:38:41.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.782+0000 7fa408895700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa404106a70 0x7fa4040735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:41.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.782+0000 7fa408895700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa404106a70 0x7fa4040735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49458/0 (socket says 172.21.15.16:49458) 2024-04-02T04:38:41.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.782+0000 7fa408895700 1 -- 172.21.15.16:0/3993502738 learned_addr learned my addr 172.21.15.16:0/3993502738 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:38:41.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.782+0000 7fa409096700 1 --2- 172.21.15.16:0/3993502738 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa404100a70 0x7fa4040730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:41.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.782+0000 7fa408895700 1 -- 172.21.15.16:0/3993502738 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa404100a70 msgr2=0x7fa4040730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:41.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.782+0000 7fa408895700 1 --2- 172.21.15.16:0/3993502738 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa404100a70 0x7fa4040730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:41.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.782+0000 7fa408895700 1 -- 172.21.15.16:0/3993502738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4000095f0 con 0x7fa404106a70 2024-04-02T04:38:41.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.782+0000 7fa408895700 1 --2- 172.21.15.16:0/3993502738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa404106a70 0x7fa4040735f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa3f400d8d0 tx=0x7fa3f400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:41.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.782+0000 7fa3fa7fc700 1 -- 172.21.15.16:0/3993502738 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3f40098a0 con 0x7fa404106a70 2024-04-02T04:38:41.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.782+0000 7fa3fa7fc700 1 -- 172.21.15.16:0/3993502738 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa3f4010460 con 0x7fa404106a70 2024-04-02T04:38:41.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.782+0000 7fa40b2fa700 1 -- 172.21.15.16:0/3993502738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4041a2b70 con 0x7fa404106a70 2024-04-02T04:38:41.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.782+0000 7fa3fa7fc700 1 -- 172.21.15.16:0/3993502738 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3f400f5f0 con 0x7fa404106a70 2024-04-02T04:38:41.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.782+0000 7fa40b2fa700 1 -- 172.21.15.16:0/3993502738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4041a2ed0 con 0x7fa404106a70 2024-04-02T04:38:41.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.784+0000 7fa40b2fa700 1 -- 172.21.15.16:0/3993502738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa40410df70 con 0x7fa404106a70 2024-04-02T04:38:41.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.784+0000 7fa3fa7fc700 1 -- 172.21.15.16:0/3993502738 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fa3f4009a00 con 0x7fa404106a70 2024-04-02T04:38:41.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.784+0000 7fa3fa7fc700 1 --2- 172.21.15.16:0/3993502738 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fa3f006cce0 0x7fa3f006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:41.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.785+0000 7fa409096700 1 --2- 172.21.15.16:0/3993502738 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fa3f006cce0 0x7fa3f006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:41.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.785+0000 7fa3fa7fc700 1 -- 172.21.15.16:0/3993502738 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fa3f408bdc0 con 0x7fa404106a70 2024-04-02T04:38:41.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.785+0000 7fa3fa7fc700 1 --2- 172.21.15.16:0/3993502738 >> [v2:172.21.15.16:6818/3734332239,v1:172.21.15.16:6819/3734332239] conn(0x7fa3f00729c0 0x7fa3f0074de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:41.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.785+0000 7fa3fa7fc700 1 -- 172.21.15.16:0/3993502738 --> [v2:172.21.15.16:6818/3734332239,v1:172.21.15.16:6819/3734332239] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa3f0075490 con 0x7fa3f00729c0 2024-04-02T04:38:41.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.785+0000 7fa3fa7fc700 1 -- 172.21.15.16:0/3993502738 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3f40914a0 con 0x7fa404106a70 2024-04-02T04:38:41.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.785+0000 7fa409897700 1 --2- 172.21.15.16:0/3993502738 >> [v2:172.21.15.16:6818/3734332239,v1:172.21.15.16:6819/3734332239] conn(0x7fa3f00729c0 0x7fa3f0074de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:41.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.785+0000 7fa409096700 1 --2- 172.21.15.16:0/3993502738 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fa3f006cce0 0x7fa3f006f1a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa400006010 tx=0x7fa40001a040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:41.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.785+0000 7fa409897700 1 --2- 172.21.15.16:0/3993502738 >> [v2:172.21.15.16:6818/3734332239,v1:172.21.15.16:6819/3734332239] conn(0x7fa3f00729c0 0x7fa3f0074de0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:41.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.786+0000 7fa3fa7fc700 1 -- 172.21.15.16:0/3993502738 <== osd.2 v2:172.21.15.16:6818/3734332239 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fa3f0075490 con 0x7fa3f00729c0 2024-04-02T04:38:41.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.817+0000 7fa40b2fa700 1 -- 172.21.15.16:0/3993502738 --> [v2:172.21.15.16:6818/3734332239,v1:172.21.15.16:6819/3734332239] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa4040614f0 con 0x7fa3f00729c0 2024-04-02T04:38:41.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.818+0000 7fa3fa7fc700 1 -- 172.21.15.16:0/3993502738 <== osd.2 v2:172.21.15.16:6818/3734332239 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa4040614f0 con 0x7fa3f00729c0 2024-04-02T04:38:41.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.818+0000 7fa40b2fa700 1 -- 172.21.15.16:0/3993502738 >> [v2:172.21.15.16:6818/3734332239,v1:172.21.15.16:6819/3734332239] conn(0x7fa3f00729c0 msgr2=0x7fa3f0074de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:41.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.818+0000 7fa40b2fa700 1 --2- 172.21.15.16:0/3993502738 >> [v2:172.21.15.16:6818/3734332239,v1:172.21.15.16:6819/3734332239] conn(0x7fa3f00729c0 0x7fa3f0074de0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:41.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.818+0000 7fa40b2fa700 1 -- 172.21.15.16:0/3993502738 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fa3f006cce0 msgr2=0x7fa3f006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:41.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.818+0000 7fa40b2fa700 1 --2- 172.21.15.16:0/3993502738 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fa3f006cce0 0x7fa3f006f1a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa400006010 tx=0x7fa40001a040 comp rx=0 tx=0).stop 2024-04-02T04:38:41.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.818+0000 7fa40b2fa700 1 -- 172.21.15.16:0/3993502738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa404106a70 msgr2=0x7fa4040735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:41.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.818+0000 7fa40b2fa700 1 --2- 172.21.15.16:0/3993502738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa404106a70 0x7fa4040735f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa3f400d8d0 tx=0x7fa3f400dbe0 comp rx=0 tx=0).stop 2024-04-02T04:38:41.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.818+0000 7fa40b2fa700 1 -- 172.21.15.16:0/3993502738 shutdown_connections 2024-04-02T04:38:41.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.818+0000 7fa40b2fa700 1 --2- 172.21.15.16:0/3993502738 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fa3f006cce0 0x7fa3f006f1a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:41.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.818+0000 7fa40b2fa700 1 --2- 172.21.15.16:0/3993502738 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa404100a70 0x7fa4040730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:41.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.818+0000 7fa40b2fa700 1 --2- 172.21.15.16:0/3993502738 >> [v2:172.21.15.16:6818/3734332239,v1:172.21.15.16:6819/3734332239] conn(0x7fa3f00729c0 0x7fa3f0074de0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:41.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.818+0000 7fa40b2fa700 1 --2- 172.21.15.16:0/3993502738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa404106a70 0x7fa4040735f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:41.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.818+0000 7fa40b2fa700 1 -- 172.21.15.16:0/3993502738 >> 172.21.15.16:0/3993502738 conn(0x7fa4040fc5a0 msgr2=0x7fa4040fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:41.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.818+0000 7fa40b2fa700 1 -- 172.21.15.16:0/3993502738 shutdown_connections 2024-04-02T04:38:41.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:40.818+0000 7fa40b2fa700 1 -- 172.21.15.16:0/3993502738 wait complete. 2024-04-02T04:38:41.693 INFO:teuthology.orchestra.run.smithi016.stdout:81604378648 2024-04-02T04:38:41.693 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.2 2024-04-02T04:38:41.927 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:41 smithi016 ceph-mon[28969]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:42.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:41 smithi027 ceph-mon[37550]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:43.069 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:42 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:38:43.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:42 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:38:44.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:43 smithi027 ceph-mon[37550]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:44.296 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:43 smithi016 ceph-mon[28969]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:44.323 INFO:teuthology.orchestra.run.smithi016.stdout:167503724550 2024-04-02T04:38:44.323 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.5 2024-04-02T04:38:46.100 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:45 smithi016 ceph-mon[28969]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:46.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:45 smithi027 ceph-mon[37550]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:46.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.791+0000 7f476742b700 1 -- 172.21.15.16:0/1820174201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4760101cb0 msgr2=0x7f4760102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:46.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.791+0000 7f476742b700 1 --2- 172.21.15.16:0/1820174201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4760101cb0 0x7f4760102090 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f475c009910 tx=0x7f475c009c20 comp rx=0 tx=0).stop 2024-04-02T04:38:46.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.791+0000 7f476742b700 1 -- 172.21.15.16:0/1820174201 shutdown_connections 2024-04-02T04:38:46.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.791+0000 7f476742b700 1 --2- 172.21.15.16:0/1820174201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47601025d0 0x7f476010aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:46.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.791+0000 7f476742b700 1 --2- 172.21.15.16:0/1820174201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4760101cb0 0x7f4760102090 secure :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f475c009910 tx=0x7f475c009c20 comp rx=0 tx=0).stop 2024-04-02T04:38:46.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.791+0000 7f476742b700 1 -- 172.21.15.16:0/1820174201 >> 172.21.15.16:0/1820174201 conn(0x7f4760076640 msgr2=0x7f47600fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:46.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.791+0000 7f476742b700 1 -- 172.21.15.16:0/1820174201 shutdown_connections 2024-04-02T04:38:46.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.791+0000 7f476742b700 1 -- 172.21.15.16:0/1820174201 wait complete. 2024-04-02T04:38:46.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.792+0000 7f476742b700 1 Processor -- start 2024-04-02T04:38:46.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.792+0000 7f476742b700 1 -- start start 2024-04-02T04:38:46.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.792+0000 7f476742b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47601025d0 0x7f476019c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:46.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.792+0000 7f476742b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f476019ce80 0x7f4760196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:46.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.793+0000 7f476742b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476019d390 con 0x7f476019ce80 2024-04-02T04:38:46.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.793+0000 7f476742b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476019d500 con 0x7f47601025d0 2024-04-02T04:38:46.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.793+0000 7f47651c7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47601025d0 0x7f476019c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:46.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.793+0000 7f47649c6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f476019ce80 0x7f4760196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:46.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.793+0000 7f47651c7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47601025d0 0x7f476019c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.16:35706/0 (socket says 172.21.15.16:35706) 2024-04-02T04:38:46.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.793+0000 7f47651c7700 1 -- 172.21.15.16:0/2632122662 learned_addr learned my addr 172.21.15.16:0/2632122662 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:38:46.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.793+0000 7f47651c7700 1 -- 172.21.15.16:0/2632122662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f476019ce80 msgr2=0x7f4760196ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:46.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.793+0000 7f47651c7700 1 --2- 172.21.15.16:0/2632122662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f476019ce80 0x7f4760196ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:46.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.793+0000 7f47651c7700 1 -- 172.21.15.16:0/2632122662 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f475c0095f0 con 0x7f47601025d0 2024-04-02T04:38:46.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.793+0000 7f47649c6700 1 --2- 172.21.15.16:0/2632122662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f476019ce80 0x7f4760196ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T04:38:46.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.793+0000 7f47651c7700 1 --2- 172.21.15.16:0/2632122662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47601025d0 0x7f476019c940 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f475c009910 tx=0x7f475c00bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:46.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.794+0000 7f47567fc700 1 -- 172.21.15.16:0/2632122662 <== mon.1 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f475c01d070 con 0x7f47601025d0 2024-04-02T04:38:46.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.794+0000 7f476742b700 1 -- 172.21.15.16:0/2632122662 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4760197150 con 0x7f47601025d0 2024-04-02T04:38:46.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.794+0000 7f47567fc700 1 -- 172.21.15.16:0/2632122662 <== mon.1 v2:172.21.15.27:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f475c0056c0 con 0x7f47601025d0 2024-04-02T04:38:46.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.794+0000 7f47567fc700 1 -- 172.21.15.16:0/2632122662 <== mon.1 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f475c00fe50 con 0x7f47601025d0 2024-04-02T04:38:46.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.794+0000 7f476742b700 1 -- 172.21.15.16:0/2632122662 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4760197640 con 0x7f47601025d0 2024-04-02T04:38:46.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.794+0000 7f476742b700 1 -- 172.21.15.16:0/2632122662 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f476010df50 con 0x7f47601025d0 2024-04-02T04:38:46.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.795+0000 7f47567fc700 1 -- 172.21.15.16:0/2632122662 <== mon.1 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f475c003a60 con 0x7f47601025d0 2024-04-02T04:38:46.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.796+0000 7f47567fc700 1 --2- 172.21.15.16:0/2632122662 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f474c06cce0 0x7f474c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:46.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.796+0000 7f47649c6700 1 --2- 172.21.15.16:0/2632122662 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f474c06cce0 0x7f474c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:46.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.796+0000 7f47567fc700 1 -- 172.21.15.16:0/2632122662 <== mon.1 v2:172.21.15.27:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f475c08d4b0 con 0x7f47601025d0 2024-04-02T04:38:46.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.796+0000 7f47567fc700 1 --2- 172.21.15.16:0/2632122662 >> [v2:172.21.15.16:6802/906882291,v1:172.21.15.16:6803/906882291] conn(0x7f474c0729c0 0x7f474c074de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:46.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.796+0000 7f47567fc700 1 -- 172.21.15.16:0/2632122662 --> [v2:172.21.15.16:6802/906882291,v1:172.21.15.16:6803/906882291] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f474c075490 con 0x7f474c0729c0 2024-04-02T04:38:46.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.796+0000 7f47567fc700 1 -- 172.21.15.16:0/2632122662 <== mon.1 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f475c08d910 con 0x7f47601025d0 2024-04-02T04:38:46.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.796+0000 7f47659c8700 1 --2- 172.21.15.16:0/2632122662 >> [v2:172.21.15.16:6802/906882291,v1:172.21.15.16:6803/906882291] conn(0x7f474c0729c0 0x7f474c074de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:46.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.797+0000 7f47649c6700 1 --2- 172.21.15.16:0/2632122662 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f474c06cce0 0x7f474c06f1a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4750005950 tx=0x7f475000a300 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:46.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.797+0000 7f47659c8700 1 --2- 172.21.15.16:0/2632122662 >> [v2:172.21.15.16:6802/906882291,v1:172.21.15.16:6803/906882291] conn(0x7f474c0729c0 0x7f474c074de0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:46.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.798+0000 7f47567fc700 1 -- 172.21.15.16:0/2632122662 <== osd.0 v2:172.21.15.16:6802/906882291 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f474c075490 con 0x7f474c0729c0 2024-04-02T04:38:46.803 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:38:46.808 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:38:46.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.817+0000 7f476742b700 1 -- 172.21.15.16:0/2632122662 --> [v2:172.21.15.16:6802/906882291,v1:172.21.15.16:6803/906882291] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4760061430 con 0x7f474c0729c0 2024-04-02T04:38:46.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.817+0000 7f47567fc700 1 -- 172.21.15.16:0/2632122662 <== osd.0 v2:172.21.15.16:6802/906882291 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4760061430 con 0x7f474c0729c0 2024-04-02T04:38:46.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.818+0000 7f476742b700 1 -- 172.21.15.16:0/2632122662 >> [v2:172.21.15.16:6802/906882291,v1:172.21.15.16:6803/906882291] conn(0x7f474c0729c0 msgr2=0x7f474c074de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:46.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.818+0000 7f476742b700 1 --2- 172.21.15.16:0/2632122662 >> [v2:172.21.15.16:6802/906882291,v1:172.21.15.16:6803/906882291] conn(0x7f474c0729c0 0x7f474c074de0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:46.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.818+0000 7f476742b700 1 -- 172.21.15.16:0/2632122662 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f474c06cce0 msgr2=0x7f474c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:46.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.818+0000 7f476742b700 1 --2- 172.21.15.16:0/2632122662 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f474c06cce0 0x7f474c06f1a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4750005950 tx=0x7f475000a300 comp rx=0 tx=0).stop 2024-04-02T04:38:46.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.818+0000 7f476742b700 1 -- 172.21.15.16:0/2632122662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47601025d0 msgr2=0x7f476019c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:46.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.818+0000 7f476742b700 1 --2- 172.21.15.16:0/2632122662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47601025d0 0x7f476019c940 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f475c009910 tx=0x7f475c00bd70 comp rx=0 tx=0).stop 2024-04-02T04:38:46.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.818+0000 7f476742b700 1 -- 172.21.15.16:0/2632122662 shutdown_connections 2024-04-02T04:38:46.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.818+0000 7f476742b700 1 --2- 172.21.15.16:0/2632122662 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f474c06cce0 0x7f474c06f1a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:46.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.818+0000 7f476742b700 1 --2- 172.21.15.16:0/2632122662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47601025d0 0x7f476019c940 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:46.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.818+0000 7f476742b700 1 --2- 172.21.15.16:0/2632122662 >> [v2:172.21.15.16:6802/906882291,v1:172.21.15.16:6803/906882291] conn(0x7f474c0729c0 0x7f474c074de0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:46.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.818+0000 7f476742b700 1 --2- 172.21.15.16:0/2632122662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f476019ce80 0x7f4760196ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:46.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.818+0000 7f476742b700 1 -- 172.21.15.16:0/2632122662 >> 172.21.15.16:0/2632122662 conn(0x7f4760076640 msgr2=0x7f4760102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:46.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.819+0000 7f476742b700 1 -- 172.21.15.16:0/2632122662 shutdown_connections 2024-04-02T04:38:46.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:46.819+0000 7f476742b700 1 -- 172.21.15.16:0/2632122662 wait complete. 2024-04-02T04:38:47.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.576+0000 7f60a74d6700 1 -- 172.21.15.16:0/3297711240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60a00ff170 msgr2=0x7f60a00ff550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:47.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.576+0000 7f60a74d6700 1 --2- 172.21.15.16:0/3297711240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60a00ff170 0x7f60a00ff550 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6090009910 tx=0x7f6090009c20 comp rx=0 tx=0).stop 2024-04-02T04:38:47.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.576+0000 7f60a74d6700 1 -- 172.21.15.16:0/3297711240 shutdown_connections 2024-04-02T04:38:47.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.576+0000 7f60a74d6700 1 --2- 172.21.15.16:0/3297711240 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60a00ffb20 0x7f60a0105660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:47.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.576+0000 7f60a74d6700 1 --2- 172.21.15.16:0/3297711240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60a00ff170 0x7f60a00ff550 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:47.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.576+0000 7f60a74d6700 1 -- 172.21.15.16:0/3297711240 >> 172.21.15.16:0/3297711240 conn(0x7f60a0075d60 msgr2=0x7f60a00fd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:47.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.576+0000 7f60a74d6700 1 -- 172.21.15.16:0/3297711240 shutdown_connections 2024-04-02T04:38:47.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.576+0000 7f60a74d6700 1 -- 172.21.15.16:0/3297711240 wait complete. 2024-04-02T04:38:47.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.577+0000 7f60a74d6700 1 Processor -- start 2024-04-02T04:38:47.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.577+0000 7f60a74d6700 1 -- start start 2024-04-02T04:38:47.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.577+0000 7f60a74d6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60a00ff170 0x7f60a01000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:47.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.577+0000 7f60a74d6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60a00ffb20 0x7f60a0100630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:47.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.577+0000 7f60a74d6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a0101de0 con 0x7f60a00ff170 2024-04-02T04:38:47.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.577+0000 7f60a74d6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a0101f50 con 0x7f60a00ffb20 2024-04-02T04:38:47.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.577+0000 7f60a5272700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60a00ff170 0x7f60a01000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:47.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.577+0000 7f60a5272700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60a00ff170 0x7f60a01000f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39480/0 (socket says 172.21.15.16:39480) 2024-04-02T04:38:47.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.577+0000 7f60a5272700 1 -- 172.21.15.16:0/271335406 learned_addr learned my addr 172.21.15.16:0/271335406 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:38:47.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.577+0000 7f60a4a71700 1 --2- 172.21.15.16:0/271335406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60a00ffb20 0x7f60a0100630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:47.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.577+0000 7f60a5272700 1 -- 172.21.15.16:0/271335406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60a00ffb20 msgr2=0x7f60a0100630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:47.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.577+0000 7f60a5272700 1 --2- 172.21.15.16:0/271335406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60a00ffb20 0x7f60a0100630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:47.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.577+0000 7f60a5272700 1 -- 172.21.15.16:0/271335406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60900095f0 con 0x7f60a00ff170 2024-04-02T04:38:47.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.577+0000 7f60a4a71700 1 --2- 172.21.15.16:0/271335406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60a00ffb20 0x7f60a0100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T04:38:47.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.578+0000 7f60a5272700 1 --2- 172.21.15.16:0/271335406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60a00ff170 0x7f60a01000f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f609000bd70 tx=0x7f609000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:47.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.578+0000 7f60967fc700 1 -- 172.21.15.16:0/271335406 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f609001d070 con 0x7f60a00ff170 2024-04-02T04:38:47.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.578+0000 7f60a74d6700 1 -- 172.21.15.16:0/271335406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60a0100b70 con 0x7f60a00ff170 2024-04-02T04:38:47.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.578+0000 7f60967fc700 1 -- 172.21.15.16:0/271335406 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f60900056e0 con 0x7f60a00ff170 2024-04-02T04:38:47.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.578+0000 7f60967fc700 1 -- 172.21.15.16:0/271335406 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6090017400 con 0x7f60a00ff170 2024-04-02T04:38:47.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.578+0000 7f60a74d6700 1 -- 172.21.15.16:0/271335406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60a01a72b0 con 0x7f60a00ff170 2024-04-02T04:38:47.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.578+0000 7f60a74d6700 1 -- 172.21.15.16:0/271335406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f60a010dea0 con 0x7f60a00ff170 2024-04-02T04:38:47.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.579+0000 7f60967fc700 1 -- 172.21.15.16:0/271335406 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f6090003a60 con 0x7f60a00ff170 2024-04-02T04:38:47.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.579+0000 7f60967fc700 1 --2- 172.21.15.16:0/271335406 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f608c06cce0 0x7f608c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:47.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.580+0000 7f60967fc700 1 -- 172.21.15.16:0/271335406 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f609008d770 con 0x7f60a00ff170 2024-04-02T04:38:47.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.580+0000 7f60a4a71700 1 --2- 172.21.15.16:0/271335406 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f608c06cce0 0x7f608c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:47.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.580+0000 7f60967fc700 1 --2- 172.21.15.16:0/271335406 >> [v2:172.21.15.16:6810/3079548081,v1:172.21.15.16:6811/3079548081] conn(0x7f608c0729c0 0x7f608c074de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:47.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.580+0000 7f60967fc700 1 -- 172.21.15.16:0/271335406 --> [v2:172.21.15.16:6810/3079548081,v1:172.21.15.16:6811/3079548081] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f608c075490 con 0x7f608c0729c0 2024-04-02T04:38:47.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.580+0000 7f60967fc700 1 -- 172.21.15.16:0/271335406 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f609008dbd0 con 0x7f60a00ff170 2024-04-02T04:38:47.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.580+0000 7f60a5a73700 1 --2- 172.21.15.16:0/271335406 >> [v2:172.21.15.16:6810/3079548081,v1:172.21.15.16:6811/3079548081] conn(0x7f608c0729c0 0x7f608c074de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:47.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.580+0000 7f60a4a71700 1 --2- 172.21.15.16:0/271335406 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f608c06cce0 0x7f608c06f1a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f609c005950 tx=0x7f609c0058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:47.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.580+0000 7f60a5a73700 1 --2- 172.21.15.16:0/271335406 >> [v2:172.21.15.16:6810/3079548081,v1:172.21.15.16:6811/3079548081] conn(0x7f608c0729c0 0x7f608c074de0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:47.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.581+0000 7f60967fc700 1 -- 172.21.15.16:0/271335406 <== osd.1 v2:172.21.15.16:6810/3079548081 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f608c075490 con 0x7f608c0729c0 2024-04-02T04:38:47.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.600+0000 7f60a74d6700 1 -- 172.21.15.16:0/271335406 --> [v2:172.21.15.16:6810/3079548081,v1:172.21.15.16:6811/3079548081] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f60a0102230 con 0x7f608c0729c0 2024-04-02T04:38:47.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.600+0000 7f60967fc700 1 -- 172.21.15.16:0/271335406 <== osd.1 v2:172.21.15.16:6810/3079548081 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f60a0102230 con 0x7f608c0729c0 2024-04-02T04:38:47.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.601+0000 7f60a74d6700 1 -- 172.21.15.16:0/271335406 >> [v2:172.21.15.16:6810/3079548081,v1:172.21.15.16:6811/3079548081] conn(0x7f608c0729c0 msgr2=0x7f608c074de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:47.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.601+0000 7f60a74d6700 1 --2- 172.21.15.16:0/271335406 >> [v2:172.21.15.16:6810/3079548081,v1:172.21.15.16:6811/3079548081] conn(0x7f608c0729c0 0x7f608c074de0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:47.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.601+0000 7f60a74d6700 1 -- 172.21.15.16:0/271335406 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f608c06cce0 msgr2=0x7f608c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:47.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.601+0000 7f60a74d6700 1 --2- 172.21.15.16:0/271335406 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f608c06cce0 0x7f608c06f1a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f609c005950 tx=0x7f609c0058e0 comp rx=0 tx=0).stop 2024-04-02T04:38:47.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.601+0000 7f60a74d6700 1 -- 172.21.15.16:0/271335406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60a00ff170 msgr2=0x7f60a01000f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:47.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.601+0000 7f60a74d6700 1 --2- 172.21.15.16:0/271335406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60a00ff170 0x7f60a01000f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f609000bd70 tx=0x7f609000be50 comp rx=0 tx=0).stop 2024-04-02T04:38:47.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.601+0000 7f60a74d6700 1 -- 172.21.15.16:0/271335406 shutdown_connections 2024-04-02T04:38:47.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.601+0000 7f60a74d6700 1 --2- 172.21.15.16:0/271335406 >> [v2:172.21.15.16:6810/3079548081,v1:172.21.15.16:6811/3079548081] conn(0x7f608c0729c0 0x7f608c074de0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:47.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.601+0000 7f60a74d6700 1 --2- 172.21.15.16:0/271335406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60a00ff170 0x7f60a01000f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:47.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.601+0000 7f60a74d6700 1 --2- 172.21.15.16:0/271335406 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f608c06cce0 0x7f608c06f1a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:47.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.601+0000 7f60a74d6700 1 --2- 172.21.15.16:0/271335406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60a00ffb20 0x7f60a0100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:47.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.601+0000 7f60a74d6700 1 -- 172.21.15.16:0/271335406 >> 172.21.15.16:0/271335406 conn(0x7f60a0075d60 msgr2=0x7f60a0104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:47.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.601+0000 7f60a74d6700 1 -- 172.21.15.16:0/271335406 shutdown_connections 2024-04-02T04:38:47.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:47.601+0000 7f60a74d6700 1 -- 172.21.15.16:0/271335406 wait complete. 2024-04-02T04:38:48.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:47 smithi027 ceph-mon[37550]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:48.264 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:47 smithi016 ceph-mon[28969]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:48.286 INFO:teuthology.orchestra.run.smithi016.stdout:38654705700 2024-04-02T04:38:48.286 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.0 2024-04-02T04:38:49.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.813+0000 7f31c457d700 1 -- 172.21.15.16:0/3686297765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31bc1049a0 msgr2=0x7f31bc104d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:49.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.813+0000 7f31c457d700 1 --2- 172.21.15.16:0/3686297765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31bc1049a0 0x7f31bc104d80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f31ac00b0a0 tx=0x7f31ac00b3b0 comp rx=0 tx=0).stop 2024-04-02T04:38:49.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.813+0000 7f31c457d700 1 -- 172.21.15.16:0/3686297765 shutdown_connections 2024-04-02T04:38:49.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.813+0000 7f31c457d700 1 --2- 172.21.15.16:0/3686297765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31bc100150 0x7f31bc1005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:49.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.813+0000 7f31c457d700 1 --2- 172.21.15.16:0/3686297765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31bc1049a0 0x7f31bc104d80 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:49.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.813+0000 7f31c457d700 1 -- 172.21.15.16:0/3686297765 >> 172.21.15.16:0/3686297765 conn(0x7f31bc075e10 msgr2=0x7f31bc0fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:49.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.813+0000 7f31c457d700 1 -- 172.21.15.16:0/3686297765 shutdown_connections 2024-04-02T04:38:49.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.813+0000 7f31c457d700 1 -- 172.21.15.16:0/3686297765 wait complete. 2024-04-02T04:38:49.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.813+0000 7f31c457d700 1 Processor -- start 2024-04-02T04:38:49.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.814+0000 7f31c457d700 1 -- start start 2024-04-02T04:38:49.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.814+0000 7f31c457d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31bc100150 0x7f31bc195720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:49.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.814+0000 7f31c457d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31bc1049a0 0x7f31bc195c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:49.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.814+0000 7f31c457d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc0fffd0 con 0x7f31bc1049a0 2024-04-02T04:38:49.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.814+0000 7f31c457d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc1961a0 con 0x7f31bc100150 2024-04-02T04:38:49.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.814+0000 7f31c1b18700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31bc1049a0 0x7f31bc195c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:49.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.814+0000 7f31c1b18700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31bc1049a0 0x7f31bc195c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39498/0 (socket says 172.21.15.16:39498) 2024-04-02T04:38:49.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.814+0000 7f31c1b18700 1 -- 172.21.15.16:0/3022013275 learned_addr learned my addr 172.21.15.16:0/3022013275 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:38:49.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.814+0000 7f31c2319700 1 --2- 172.21.15.16:0/3022013275 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31bc100150 0x7f31bc195720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:49.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.814+0000 7f31c1b18700 1 -- 172.21.15.16:0/3022013275 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31bc100150 msgr2=0x7f31bc195720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:49.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.814+0000 7f31c1b18700 1 --2- 172.21.15.16:0/3022013275 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31bc100150 0x7f31bc195720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:49.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.814+0000 7f31c1b18700 1 -- 172.21.15.16:0/3022013275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31ac009d00 con 0x7f31bc1049a0 2024-04-02T04:38:49.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.814+0000 7f31c2319700 1 --2- 172.21.15.16:0/3022013275 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31bc100150 0x7f31bc195720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T04:38:49.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.814+0000 7f31c1b18700 1 --2- 172.21.15.16:0/3022013275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31bc1049a0 0x7f31bc195c60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f31b000b4b0 tx=0x7f31b000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:49.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.814+0000 7f31bb7fe700 1 -- 172.21.15.16:0/3022013275 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31b0011800 con 0x7f31bc1049a0 2024-04-02T04:38:49.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.814+0000 7f31bb7fe700 1 -- 172.21.15.16:0/3022013275 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f31b0011e20 con 0x7f31bc1049a0 2024-04-02T04:38:49.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.814+0000 7f31c457d700 1 -- 172.21.15.16:0/3022013275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31bc196480 con 0x7f31bc1049a0 2024-04-02T04:38:49.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.814+0000 7f31bb7fe700 1 -- 172.21.15.16:0/3022013275 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31b000f2e0 con 0x7f31bc1049a0 2024-04-02T04:38:49.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.815+0000 7f31c457d700 1 -- 172.21.15.16:0/3022013275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31bc1a3010 con 0x7f31bc1049a0 2024-04-02T04:38:49.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.815+0000 7f31c457d700 1 -- 172.21.15.16:0/3022013275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f31bc10cfd0 con 0x7f31bc1049a0 2024-04-02T04:38:49.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.816+0000 7f31bb7fe700 1 -- 172.21.15.16:0/3022013275 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f31b0010380 con 0x7f31bc1049a0 2024-04-02T04:38:49.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.816+0000 7f31bb7fe700 1 --2- 172.21.15.16:0/3022013275 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f31a406cce0 0x7f31a406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:49.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.816+0000 7f31bb7fe700 1 -- 172.21.15.16:0/3022013275 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f31b008ba90 con 0x7f31bc1049a0 2024-04-02T04:38:49.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.816+0000 7f31c2319700 1 --2- 172.21.15.16:0/3022013275 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f31a406cce0 0x7f31a406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:49.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.817+0000 7f31bb7fe700 1 --2- 172.21.15.16:0/3022013275 >> [v2:172.21.15.27:6808/935993381,v1:172.21.15.27:6809/935993381] conn(0x7f31a40729c0 0x7f31a4074de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:49.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.817+0000 7f31bb7fe700 1 -- 172.21.15.16:0/3022013275 --> [v2:172.21.15.27:6808/935993381,v1:172.21.15.27:6809/935993381] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f31a4075490 con 0x7f31a40729c0 2024-04-02T04:38:49.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.817+0000 7f31bb7fe700 1 -- 172.21.15.16:0/3022013275 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31b00902a0 con 0x7f31bc1049a0 2024-04-02T04:38:49.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.817+0000 7f31c2319700 1 --2- 172.21.15.16:0/3022013275 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f31a406cce0 0x7f31a406f1a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f31ac009480 tx=0x7f31ac0097c0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:49.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.817+0000 7f31c2b1a700 1 --2- 172.21.15.16:0/3022013275 >> [v2:172.21.15.27:6808/935993381,v1:172.21.15.27:6809/935993381] conn(0x7f31a40729c0 0x7f31a4074de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:49.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.817+0000 7f31c2b1a700 1 --2- 172.21.15.16:0/3022013275 >> [v2:172.21.15.27:6808/935993381,v1:172.21.15.27:6809/935993381] conn(0x7f31a40729c0 0x7f31a4074de0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:49.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.818+0000 7f31bb7fe700 1 -- 172.21.15.16:0/3022013275 <== osd.4 v2:172.21.15.27:6808/935993381 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f31a4075490 con 0x7f31a40729c0 2024-04-02T04:38:49.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.834+0000 7f31c457d700 1 -- 172.21.15.16:0/3022013275 --> [v2:172.21.15.27:6808/935993381,v1:172.21.15.27:6809/935993381] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f31bc061460 con 0x7f31a40729c0 2024-04-02T04:38:49.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.834+0000 7f31bb7fe700 1 -- 172.21.15.16:0/3022013275 <== osd.4 v2:172.21.15.27:6808/935993381 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f31bc061460 con 0x7f31a40729c0 2024-04-02T04:38:49.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.834+0000 7f31c457d700 1 -- 172.21.15.16:0/3022013275 >> [v2:172.21.15.27:6808/935993381,v1:172.21.15.27:6809/935993381] conn(0x7f31a40729c0 msgr2=0x7f31a4074de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:49.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.834+0000 7f31c457d700 1 --2- 172.21.15.16:0/3022013275 >> [v2:172.21.15.27:6808/935993381,v1:172.21.15.27:6809/935993381] conn(0x7f31a40729c0 0x7f31a4074de0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:49.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.834+0000 7f31c457d700 1 -- 172.21.15.16:0/3022013275 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f31a406cce0 msgr2=0x7f31a406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:49.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.834+0000 7f31c457d700 1 --2- 172.21.15.16:0/3022013275 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f31a406cce0 0x7f31a406f1a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f31ac009480 tx=0x7f31ac0097c0 comp rx=0 tx=0).stop 2024-04-02T04:38:49.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.834+0000 7f31c457d700 1 -- 172.21.15.16:0/3022013275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31bc1049a0 msgr2=0x7f31bc195c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:49.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.834+0000 7f31c457d700 1 --2- 172.21.15.16:0/3022013275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31bc1049a0 0x7f31bc195c60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f31b000b4b0 tx=0x7f31b000b870 comp rx=0 tx=0).stop 2024-04-02T04:38:49.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.835+0000 7f31c457d700 1 -- 172.21.15.16:0/3022013275 shutdown_connections 2024-04-02T04:38:49.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.835+0000 7f31c457d700 1 --2- 172.21.15.16:0/3022013275 >> [v2:172.21.15.27:6808/935993381,v1:172.21.15.27:6809/935993381] conn(0x7f31a40729c0 0x7f31a4074de0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:49.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.835+0000 7f31c457d700 1 --2- 172.21.15.16:0/3022013275 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f31a406cce0 0x7f31a406f1a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:49.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.835+0000 7f31c457d700 1 --2- 172.21.15.16:0/3022013275 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31bc100150 0x7f31bc195720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:49.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.835+0000 7f31c457d700 1 --2- 172.21.15.16:0/3022013275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31bc1049a0 0x7f31bc195c60 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:49.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.835+0000 7f31c457d700 1 -- 172.21.15.16:0/3022013275 >> 172.21.15.16:0/3022013275 conn(0x7f31bc075e10 msgr2=0x7f31bc0fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:49.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.835+0000 7f31c457d700 1 -- 172.21.15.16:0/3022013275 shutdown_connections 2024-04-02T04:38:49.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:49.835+0000 7f31c457d700 1 -- 172.21.15.16:0/3022013275 wait complete. 2024-04-02T04:38:50.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:49 smithi016 ceph-mon[28969]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:50.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:49 smithi027 ceph-mon[37550]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.268+0000 7f4d41875700 1 -- 172.21.15.16:0/2360439772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d3c100180 msgr2=0x7f4d3c10ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.268+0000 7f4d41875700 1 --2- 172.21.15.16:0/2360439772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d3c100180 0x7f4d3c10ce40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4d30009910 tx=0x7f4d30009c20 comp rx=0 tx=0).stop 2024-04-02T04:38:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.268+0000 7f4d41875700 1 -- 172.21.15.16:0/2360439772 shutdown_connections 2024-04-02T04:38:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.268+0000 7f4d41875700 1 --2- 172.21.15.16:0/2360439772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d3c100180 0x7f4d3c10ce40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.268+0000 7f4d41875700 1 --2- 172.21.15.16:0/2360439772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d3c0ff7d0 0x7f4d3c0ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.268+0000 7f4d41875700 1 -- 172.21.15.16:0/2360439772 >> 172.21.15.16:0/2360439772 conn(0x7f4d3c0fb490 msgr2=0x7f4d3c0fd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.269+0000 7f4d41875700 1 -- 172.21.15.16:0/2360439772 shutdown_connections 2024-04-02T04:38:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.269+0000 7f4d41875700 1 -- 172.21.15.16:0/2360439772 wait complete. 2024-04-02T04:38:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.269+0000 7f4d41875700 1 Processor -- start 2024-04-02T04:38:50.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.269+0000 7f4d41875700 1 -- start start 2024-04-02T04:38:50.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.270+0000 7f4d41875700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d3c0ff7d0 0x7f4d3c19e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:50.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.270+0000 7f4d41875700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d3c100180 0x7f4d3c19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:50.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.270+0000 7f4d41875700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c19f430 con 0x7f4d3c0ff7d0 2024-04-02T04:38:50.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.270+0000 7f4d41875700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c198a10 con 0x7f4d3c100180 2024-04-02T04:38:50.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.270+0000 7f4d3affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d3c0ff7d0 0x7f4d3c19e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:50.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.270+0000 7f4d3affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d3c0ff7d0 0x7f4d3c19e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39510/0 (socket says 172.21.15.16:39510) 2024-04-02T04:38:50.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.270+0000 7f4d3affd700 1 -- 172.21.15.16:0/1453588576 learned_addr learned my addr 172.21.15.16:0/1453588576 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:38:50.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.270+0000 7f4d3a7fc700 1 --2- 172.21.15.16:0/1453588576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d3c100180 0x7f4d3c19ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:50.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.270+0000 7f4d3affd700 1 -- 172.21.15.16:0/1453588576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d3c100180 msgr2=0x7f4d3c19ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:50.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.270+0000 7f4d3affd700 1 --2- 172.21.15.16:0/1453588576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d3c100180 0x7f4d3c19ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:50.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.270+0000 7f4d3affd700 1 -- 172.21.15.16:0/1453588576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d300095f0 con 0x7f4d3c0ff7d0 2024-04-02T04:38:50.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.270+0000 7f4d3affd700 1 --2- 172.21.15.16:0/1453588576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d3c0ff7d0 0x7f4d3c19e8d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c00d8d0 tx=0x7f4d2c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:50.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.270+0000 7f4d40873700 1 -- 172.21.15.16:0/1453588576 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d2c0098a0 con 0x7f4d3c0ff7d0 2024-04-02T04:38:50.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.270+0000 7f4d40873700 1 -- 172.21.15.16:0/1453588576 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4d2c010460 con 0x7f4d3c0ff7d0 2024-04-02T04:38:50.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.270+0000 7f4d41875700 1 -- 172.21.15.16:0/1453588576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d3c198cf0 con 0x7f4d3c0ff7d0 2024-04-02T04:38:50.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.270+0000 7f4d40873700 1 -- 172.21.15.16:0/1453588576 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d2c00f5f0 con 0x7f4d3c0ff7d0 2024-04-02T04:38:50.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.270+0000 7f4d41875700 1 -- 172.21.15.16:0/1453588576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d3c199240 con 0x7f4d3c0ff7d0 2024-04-02T04:38:50.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.271+0000 7f4d41875700 1 -- 172.21.15.16:0/1453588576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4d3c110150 con 0x7f4d3c0ff7d0 2024-04-02T04:38:50.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.272+0000 7f4d40873700 1 -- 172.21.15.16:0/1453588576 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f4d2c009a00 con 0x7f4d3c0ff7d0 2024-04-02T04:38:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.272+0000 7f4d40873700 1 --2- 172.21.15.16:0/1453588576 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f4d2406cd30 0x7f4d2406f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.272+0000 7f4d40873700 1 -- 172.21.15.16:0/1453588576 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f4d2c08bdc0 con 0x7f4d3c0ff7d0 2024-04-02T04:38:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.272+0000 7f4d3a7fc700 1 --2- 172.21.15.16:0/1453588576 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f4d2406cd30 0x7f4d2406f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.272+0000 7f4d40873700 1 --2- 172.21.15.16:0/1453588576 >> [v2:172.21.15.27:6800/388124754,v1:172.21.15.27:6801/388124754] conn(0x7f4d24072a10 0x7f4d24074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.272+0000 7f4d40873700 1 -- 172.21.15.16:0/1453588576 --> [v2:172.21.15.27:6800/388124754,v1:172.21.15.27:6801/388124754] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4d240754e0 con 0x7f4d24072a10 2024-04-02T04:38:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.272+0000 7f4d40873700 1 -- 172.21.15.16:0/1453588576 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d2c091350 con 0x7f4d3c0ff7d0 2024-04-02T04:38:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.273+0000 7f4d3a7fc700 1 --2- 172.21.15.16:0/1453588576 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f4d2406cd30 0x7f4d2406f1f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c199bd0 tx=0x7f4d3000b560 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.273+0000 7f4d3b7fe700 1 --2- 172.21.15.16:0/1453588576 >> [v2:172.21.15.27:6800/388124754,v1:172.21.15.27:6801/388124754] conn(0x7f4d24072a10 0x7f4d24074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.273+0000 7f4d3b7fe700 1 --2- 172.21.15.16:0/1453588576 >> [v2:172.21.15.27:6800/388124754,v1:172.21.15.27:6801/388124754] conn(0x7f4d24072a10 0x7f4d24074e30 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.274+0000 7f4d40873700 1 -- 172.21.15.16:0/1453588576 <== osd.3 v2:172.21.15.27:6800/388124754 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f4d240754e0 con 0x7f4d24072a10 2024-04-02T04:38:50.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.289+0000 7f4d41875700 1 -- 172.21.15.16:0/1453588576 --> [v2:172.21.15.27:6800/388124754,v1:172.21.15.27:6801/388124754] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4d3c1996f0 con 0x7f4d24072a10 2024-04-02T04:38:50.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.290+0000 7f4d40873700 1 -- 172.21.15.16:0/1453588576 <== osd.3 v2:172.21.15.27:6800/388124754 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f4d3c1996f0 con 0x7f4d24072a10 2024-04-02T04:38:50.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.290+0000 7f4d41875700 1 -- 172.21.15.16:0/1453588576 >> [v2:172.21.15.27:6800/388124754,v1:172.21.15.27:6801/388124754] conn(0x7f4d24072a10 msgr2=0x7f4d24074e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:50.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.290+0000 7f4d41875700 1 --2- 172.21.15.16:0/1453588576 >> [v2:172.21.15.27:6800/388124754,v1:172.21.15.27:6801/388124754] conn(0x7f4d24072a10 0x7f4d24074e30 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:50.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.290+0000 7f4d41875700 1 -- 172.21.15.16:0/1453588576 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f4d2406cd30 msgr2=0x7f4d2406f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:50.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.290+0000 7f4d41875700 1 --2- 172.21.15.16:0/1453588576 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f4d2406cd30 0x7f4d2406f1f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c199bd0 tx=0x7f4d3000b560 comp rx=0 tx=0).stop 2024-04-02T04:38:50.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.290+0000 7f4d41875700 1 -- 172.21.15.16:0/1453588576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d3c0ff7d0 msgr2=0x7f4d3c19e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:50.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.290+0000 7f4d41875700 1 --2- 172.21.15.16:0/1453588576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d3c0ff7d0 0x7f4d3c19e8d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c00d8d0 tx=0x7f4d2c00dbe0 comp rx=0 tx=0).stop 2024-04-02T04:38:50.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.290+0000 7f4d41875700 1 -- 172.21.15.16:0/1453588576 shutdown_connections 2024-04-02T04:38:50.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.290+0000 7f4d41875700 1 --2- 172.21.15.16:0/1453588576 >> [v2:172.21.15.27:6800/388124754,v1:172.21.15.27:6801/388124754] conn(0x7f4d24072a10 0x7f4d24074e30 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:50.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.290+0000 7f4d41875700 1 --2- 172.21.15.16:0/1453588576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d3c0ff7d0 0x7f4d3c19e8d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:50.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.290+0000 7f4d41875700 1 --2- 172.21.15.16:0/1453588576 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f4d2406cd30 0x7f4d2406f1f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:50.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.290+0000 7f4d41875700 1 --2- 172.21.15.16:0/1453588576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d3c100180 0x7f4d3c19ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:50.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.290+0000 7f4d41875700 1 -- 172.21.15.16:0/1453588576 >> 172.21.15.16:0/1453588576 conn(0x7f4d3c0fb490 msgr2=0x7f4d3c107e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:50.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.291+0000 7f4d41875700 1 -- 172.21.15.16:0/1453588576 shutdown_connections 2024-04-02T04:38:50.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:50.291+0000 7f4d41875700 1 -- 172.21.15.16:0/1453588576 wait complete. 2024-04-02T04:38:50.905 INFO:teuthology.orchestra.run.smithi016.stdout:60129542174 2024-04-02T04:38:50.905 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.1 2024-04-02T04:38:51.914 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:51 smithi016 ceph-mon[28969]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:51.936 INFO:teuthology.orchestra.run.smithi016.stdout:111669149717 2024-04-02T04:38:51.936 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.3 2024-04-02T04:38:52.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:51 smithi027 ceph-mon[37550]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:52.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.376+0000 7f7f594ac700 1 -- 172.21.15.16:0/802953474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f54101c10 msgr2=0x7f7f54101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:52.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.376+0000 7f7f594ac700 1 --2- 172.21.15.16:0/802953474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f54101c10 0x7f7f54101ff0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7f44009910 tx=0x7f7f44009c20 comp rx=0 tx=0).stop 2024-04-02T04:38:52.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.376+0000 7f7f594ac700 1 -- 172.21.15.16:0/802953474 shutdown_connections 2024-04-02T04:38:52.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.376+0000 7f7f594ac700 1 --2- 172.21.15.16:0/802953474 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f54102530 0x7f7f5410aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:52.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.376+0000 7f7f594ac700 1 --2- 172.21.15.16:0/802953474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f54101c10 0x7f7f54101ff0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:52.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.376+0000 7f7f594ac700 1 -- 172.21.15.16:0/802953474 >> 172.21.15.16:0/802953474 conn(0x7f7f54076640 msgr2=0x7f7f540fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:52.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.376+0000 7f7f594ac700 1 -- 172.21.15.16:0/802953474 shutdown_connections 2024-04-02T04:38:52.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.376+0000 7f7f594ac700 1 -- 172.21.15.16:0/802953474 wait complete. 2024-04-02T04:38:52.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.377+0000 7f7f594ac700 1 Processor -- start 2024-04-02T04:38:52.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.377+0000 7f7f594ac700 1 -- start start 2024-04-02T04:38:52.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.377+0000 7f7f594ac700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f54101c10 0x7f7f54196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:52.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.377+0000 7f7f594ac700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f54102530 0x7f7f54196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:52.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.377+0000 7f7f594ac700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f54197580 con 0x7f7f54101c10 2024-04-02T04:38:52.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.377+0000 7f7f594ac700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f541976f0 con 0x7f7f54102530 2024-04-02T04:38:52.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.377+0000 7f7f52ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f54101c10 0x7f7f54196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:52.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.377+0000 7f7f52ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f54101c10 0x7f7f54196a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53020/0 (socket says 172.21.15.16:53020) 2024-04-02T04:38:52.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.377+0000 7f7f52ffd700 1 -- 172.21.15.16:0/2496611392 learned_addr learned my addr 172.21.15.16:0/2496611392 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:38:52.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.378+0000 7f7f527fc700 1 --2- 172.21.15.16:0/2496611392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f54102530 0x7f7f54196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:52.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.378+0000 7f7f52ffd700 1 -- 172.21.15.16:0/2496611392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f54102530 msgr2=0x7f7f54196f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:52.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.378+0000 7f7f52ffd700 1 --2- 172.21.15.16:0/2496611392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f54102530 0x7f7f54196f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:52.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.378+0000 7f7f52ffd700 1 -- 172.21.15.16:0/2496611392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f440095f0 con 0x7f7f54101c10 2024-04-02T04:38:52.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.378+0000 7f7f52ffd700 1 --2- 172.21.15.16:0/2496611392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f54101c10 0x7f7f54196a40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7f44006010 tx=0x7f7f4400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:52.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.378+0000 7f7f3bfff700 1 -- 172.21.15.16:0/2496611392 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f4401d070 con 0x7f7f54101c10 2024-04-02T04:38:52.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.378+0000 7f7f594ac700 1 -- 172.21.15.16:0/2496611392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f5419b450 con 0x7f7f54101c10 2024-04-02T04:38:52.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.378+0000 7f7f3bfff700 1 -- 172.21.15.16:0/2496611392 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7f4400f460 con 0x7f7f54101c10 2024-04-02T04:38:52.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.378+0000 7f7f3bfff700 1 -- 172.21.15.16:0/2496611392 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f440175f0 con 0x7f7f54101c10 2024-04-02T04:38:52.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.378+0000 7f7f594ac700 1 -- 172.21.15.16:0/2496611392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f5419b940 con 0x7f7f54101c10 2024-04-02T04:38:52.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.379+0000 7f7f594ac700 1 -- 172.21.15.16:0/2496611392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f54068d60 con 0x7f7f54101c10 2024-04-02T04:38:52.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.380+0000 7f7f3bfff700 1 -- 172.21.15.16:0/2496611392 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f7f4400f5d0 con 0x7f7f54101c10 2024-04-02T04:38:52.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.380+0000 7f7f3bfff700 1 --2- 172.21.15.16:0/2496611392 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f7f3c06cce0 0x7f7f3c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:52.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.380+0000 7f7f3bfff700 1 -- 172.21.15.16:0/2496611392 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f7f4408d340 con 0x7f7f54101c10 2024-04-02T04:38:52.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.380+0000 7f7f527fc700 1 --2- 172.21.15.16:0/2496611392 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f7f3c06cce0 0x7f7f3c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:52.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.381+0000 7f7f527fc700 1 --2- 172.21.15.16:0/2496611392 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f7f3c06cce0 0x7f7f3c06f1a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7f54197fa0 tx=0x7f7f4800a2d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:52.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.385+0000 7f7f3bfff700 1 -- 172.21.15.16:0/2496611392 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7f4405b4e0 con 0x7f7f54101c10 2024-04-02T04:38:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.590+0000 7f7f594ac700 1 -- 172.21.15.16:0/2496611392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7f5410dc90 con 0x7f7f54101c10 2024-04-02T04:38:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.591+0000 7f7f3bfff700 1 -- 172.21.15.16:0/2496611392 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7f44005740 con 0x7f7f54101c10 2024-04-02T04:38:52.594 INFO:teuthology.orchestra.run.smithi016.stdout:81604378650 2024-04-02T04:38:52.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.593+0000 7f7f594ac700 1 -- 172.21.15.16:0/2496611392 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f7f3c06cce0 msgr2=0x7f7f3c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:52.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.593+0000 7f7f594ac700 1 --2- 172.21.15.16:0/2496611392 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f7f3c06cce0 0x7f7f3c06f1a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7f54197fa0 tx=0x7f7f4800a2d0 comp rx=0 tx=0).stop 2024-04-02T04:38:52.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.593+0000 7f7f594ac700 1 -- 172.21.15.16:0/2496611392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f54101c10 msgr2=0x7f7f54196a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:52.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.593+0000 7f7f594ac700 1 --2- 172.21.15.16:0/2496611392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f54101c10 0x7f7f54196a40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7f44006010 tx=0x7f7f4400bd70 comp rx=0 tx=0).stop 2024-04-02T04:38:52.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.593+0000 7f7f594ac700 1 -- 172.21.15.16:0/2496611392 shutdown_connections 2024-04-02T04:38:52.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.593+0000 7f7f594ac700 1 --2- 172.21.15.16:0/2496611392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f54101c10 0x7f7f54196a40 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:52.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.593+0000 7f7f594ac700 1 --2- 172.21.15.16:0/2496611392 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f7f3c06cce0 0x7f7f3c06f1a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:52.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.593+0000 7f7f594ac700 1 --2- 172.21.15.16:0/2496611392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f54102530 0x7f7f54196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:52.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.593+0000 7f7f594ac700 1 -- 172.21.15.16:0/2496611392 >> 172.21.15.16:0/2496611392 conn(0x7f7f54076640 msgr2=0x7f7f54102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:52.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.593+0000 7f7f594ac700 1 -- 172.21.15.16:0/2496611392 shutdown_connections 2024-04-02T04:38:52.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:52.593+0000 7f7f594ac700 1 -- 172.21.15.16:0/2496611392 wait complete. 2024-04-02T04:38:53.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:52 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/2496611392' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-02T04:38:53.299 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:52 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/2496611392' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-02T04:38:53.614 INFO:teuthology.orchestra.run.smithi016.stdout:146028888079 2024-04-02T04:38:53.614 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.4 2024-04-02T04:38:53.849 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:53 smithi016 ceph-mon[28969]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:54.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:53 smithi027 ceph-mon[37550]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:54.377 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378648 got 81604378650 for osd.2 2024-04-02T04:38:54.377 DEBUG:teuthology.parallel:result is None 2024-04-02T04:38:54.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.558+0000 7f5a8ee9c700 1 -- 172.21.15.16:0/1831042406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a880ff800 msgr2=0x7f5a880ffbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:54.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.558+0000 7f5a8ee9c700 1 --2- 172.21.15.16:0/1831042406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a880ff800 0x7f5a880ffbe0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c009910 tx=0x7f5a7c009c20 comp rx=0 tx=0).stop 2024-04-02T04:38:54.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.558+0000 7f5a8ee9c700 1 -- 172.21.15.16:0/1831042406 shutdown_connections 2024-04-02T04:38:54.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.558+0000 7f5a8ee9c700 1 --2- 172.21.15.16:0/1831042406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a881001b0 0x7f5a8810ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:54.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.558+0000 7f5a8ee9c700 1 --2- 172.21.15.16:0/1831042406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a880ff800 0x7f5a880ffbe0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:54.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.558+0000 7f5a8ee9c700 1 -- 172.21.15.16:0/1831042406 >> 172.21.15.16:0/1831042406 conn(0x7f5a880fb4e0 msgr2=0x7f5a880fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:54.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.558+0000 7f5a8ee9c700 1 -- 172.21.15.16:0/1831042406 shutdown_connections 2024-04-02T04:38:54.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.558+0000 7f5a8ee9c700 1 -- 172.21.15.16:0/1831042406 wait complete. 2024-04-02T04:38:54.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.559+0000 7f5a8ee9c700 1 Processor -- start 2024-04-02T04:38:54.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.559+0000 7f5a8ee9c700 1 -- start start 2024-04-02T04:38:54.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.559+0000 7f5a8ee9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a880ff800 0x7f5a8819e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:54.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.559+0000 7f5a8ee9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a881001b0 0x7f5a8819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:54.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.559+0000 7f5a8ee9c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8819f500 con 0x7f5a881001b0 2024-04-02T04:38:54.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.559+0000 7f5a8ee9c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a88198b30 con 0x7f5a880ff800 2024-04-02T04:38:54.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.559+0000 7f5a87fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a881001b0 0x7f5a8819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:54.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.559+0000 7f5a8cc38700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a880ff800 0x7f5a8819e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:54.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.559+0000 7f5a87fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a881001b0 0x7f5a8819eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53046/0 (socket says 172.21.15.16:53046) 2024-04-02T04:38:54.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.559+0000 7f5a87fff700 1 -- 172.21.15.16:0/2873227770 learned_addr learned my addr 172.21.15.16:0/2873227770 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:38:54.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.560+0000 7f5a87fff700 1 -- 172.21.15.16:0/2873227770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a880ff800 msgr2=0x7f5a8819e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:54.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.560+0000 7f5a87fff700 1 --2- 172.21.15.16:0/2873227770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a880ff800 0x7f5a8819e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:54.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.560+0000 7f5a87fff700 1 -- 172.21.15.16:0/2873227770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a7c0095f0 con 0x7f5a881001b0 2024-04-02T04:38:54.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.560+0000 7f5a87fff700 1 --2- 172.21.15.16:0/2873227770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a881001b0 0x7f5a8819eee0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5a7400b4b0 tx=0x7f5a7400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:54.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.560+0000 7f5a85ffb700 1 -- 172.21.15.16:0/2873227770 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a74011800 con 0x7f5a881001b0 2024-04-02T04:38:54.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.560+0000 7f5a8ee9c700 1 -- 172.21.15.16:0/2873227770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a88198e10 con 0x7f5a881001b0 2024-04-02T04:38:54.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.560+0000 7f5a85ffb700 1 -- 172.21.15.16:0/2873227770 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a74011e20 con 0x7f5a881001b0 2024-04-02T04:38:54.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.560+0000 7f5a85ffb700 1 -- 172.21.15.16:0/2873227770 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a7400f2e0 con 0x7f5a881001b0 2024-04-02T04:38:54.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.560+0000 7f5a8ee9c700 1 -- 172.21.15.16:0/2873227770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a88199360 con 0x7f5a881001b0 2024-04-02T04:38:54.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.561+0000 7f5a8ee9c700 1 -- 172.21.15.16:0/2873227770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a88107850 con 0x7f5a881001b0 2024-04-02T04:38:54.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.562+0000 7f5a85ffb700 1 -- 172.21.15.16:0/2873227770 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f5a74010380 con 0x7f5a881001b0 2024-04-02T04:38:54.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.562+0000 7f5a85ffb700 1 --2- 172.21.15.16:0/2873227770 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f5a7806cce0 0x7f5a7806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:54.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.562+0000 7f5a85ffb700 1 -- 172.21.15.16:0/2873227770 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f5a7408ba50 con 0x7f5a881001b0 2024-04-02T04:38:54.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.562+0000 7f5a8cc38700 1 --2- 172.21.15.16:0/2873227770 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f5a7806cce0 0x7f5a7806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:54.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.562+0000 7f5a8cc38700 1 --2- 172.21.15.16:0/2873227770 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f5a7806cce0 0x7f5a7806f1a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c00b850 tx=0x7f5a7c00b560 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:54.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.567+0000 7f5a85ffb700 1 -- 172.21.15.16:0/2873227770 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5a74059ca0 con 0x7f5a881001b0 2024-04-02T04:38:54.647 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:38:54.648 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:38:54.648 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:38:54.649 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:38:54.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.804+0000 7f5a8ee9c700 1 -- 172.21.15.16:0/2873227770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f5a880698f0 con 0x7f5a881001b0 2024-04-02T04:38:54.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.804+0000 7f5a85ffb700 1 -- 172.21.15.16:0/2873227770 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5a74059830 con 0x7f5a881001b0 2024-04-02T04:38:54.808 INFO:teuthology.orchestra.run.smithi016.stdout:167503724552 2024-04-02T04:38:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.806+0000 7f5a8ee9c700 1 -- 172.21.15.16:0/2873227770 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f5a7806cce0 msgr2=0x7f5a7806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.806+0000 7f5a8ee9c700 1 --2- 172.21.15.16:0/2873227770 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f5a7806cce0 0x7f5a7806f1a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c00b850 tx=0x7f5a7c00b560 comp rx=0 tx=0).stop 2024-04-02T04:38:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.806+0000 7f5a8ee9c700 1 -- 172.21.15.16:0/2873227770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a881001b0 msgr2=0x7f5a8819eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.806+0000 7f5a8ee9c700 1 --2- 172.21.15.16:0/2873227770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a881001b0 0x7f5a8819eee0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5a7400b4b0 tx=0x7f5a7400b7c0 comp rx=0 tx=0).stop 2024-04-02T04:38:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.806+0000 7f5a8ee9c700 1 -- 172.21.15.16:0/2873227770 shutdown_connections 2024-04-02T04:38:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.806+0000 7f5a8ee9c700 1 --2- 172.21.15.16:0/2873227770 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f5a7806cce0 0x7f5a7806f1a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.806+0000 7f5a8ee9c700 1 --2- 172.21.15.16:0/2873227770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a880ff800 0x7f5a8819e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.806+0000 7f5a8ee9c700 1 --2- 172.21.15.16:0/2873227770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a881001b0 0x7f5a8819eee0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.806+0000 7f5a8ee9c700 1 -- 172.21.15.16:0/2873227770 >> 172.21.15.16:0/2873227770 conn(0x7f5a880fb4e0 msgr2=0x7f5a880fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.807+0000 7f5a8ee9c700 1 -- 172.21.15.16:0/2873227770 shutdown_connections 2024-04-02T04:38:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:54.807+0000 7f5a8ee9c700 1 -- 172.21.15.16:0/2873227770 wait complete. 2024-04-02T04:38:55.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:54 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/2873227770' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-02T04:38:55.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:54 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/2873227770' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-02T04:38:55.418 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724550 got 167503724552 for osd.5 2024-04-02T04:38:55.418 DEBUG:teuthology.parallel:result is None 2024-04-02T04:38:55.918 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:55 smithi016 ceph-mon[28969]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:56.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:55 smithi027 ceph-mon[37550]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:58.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:57 smithi016 ceph-mon[28969]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:58.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:57 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:38:58.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:57 smithi027 ceph-mon[37550]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:38:58.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:57 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:38:59.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:59.937+0000 7f880d62e700 1 -- 172.21.15.16:0/1190617457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8808106a70 msgr2=0x7f8808106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:59.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:59.937+0000 7f880d62e700 1 --2- 172.21.15.16:0/1190617457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8808106a70 0x7f8808106e50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8800009910 tx=0x7f8800009c20 comp rx=0 tx=0).stop 2024-04-02T04:38:59.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:59.937+0000 7f880d62e700 1 -- 172.21.15.16:0/1190617457 shutdown_connections 2024-04-02T04:38:59.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:59.937+0000 7f880d62e700 1 --2- 172.21.15.16:0/1190617457 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8808100a70 0x7f8808100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:59.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:59.937+0000 7f880d62e700 1 --2- 172.21.15.16:0/1190617457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8808106a70 0x7f8808106e50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:59.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:59.937+0000 7f880d62e700 1 -- 172.21.15.16:0/1190617457 >> 172.21.15.16:0/1190617457 conn(0x7f88080fc580 msgr2=0x7f88080fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:38:59.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:59.937+0000 7f880d62e700 1 -- 172.21.15.16:0/1190617457 shutdown_connections 2024-04-02T04:38:59.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:59.937+0000 7f880d62e700 1 -- 172.21.15.16:0/1190617457 wait complete. 2024-04-02T04:38:59.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:59.937+0000 7f880d62e700 1 Processor -- start 2024-04-02T04:38:59.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:59.938+0000 7f880d62e700 1 -- start start 2024-04-02T04:38:59.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:59.938+0000 7f880d62e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8808100a70 0x7f880819c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:59.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:59.938+0000 7f880d62e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8808106a70 0x7f880819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:59.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:59.938+0000 7f880d62e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880819d370 con 0x7f8808106a70 2024-04-02T04:38:59.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:59.938+0000 7f880d62e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8808196950 con 0x7f8808100a70 2024-04-02T04:38:59.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:59.938+0000 7f88067fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8808106a70 0x7f880819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:59.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:59.938+0000 7f88067fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8808106a70 0x7f880819cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53076/0 (socket says 172.21.15.16:53076) 2024-04-02T04:38:59.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:59.938+0000 7f88067fc700 1 -- 172.21.15.16:0/4119472502 learned_addr learned my addr 172.21.15.16:0/4119472502 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:38:59.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:59.938+0000 7f8806ffd700 1 --2- 172.21.15.16:0/4119472502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8808100a70 0x7f880819c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:59.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:59.938+0000 7f88067fc700 1 -- 172.21.15.16:0/4119472502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8808100a70 msgr2=0x7f880819c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:38:59.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:59.938+0000 7f88067fc700 1 --2- 172.21.15.16:0/4119472502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8808100a70 0x7f880819c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:38:59.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:59.938+0000 7f88067fc700 1 -- 172.21.15.16:0/4119472502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88000095f0 con 0x7f8808106a70 2024-04-02T04:38:59.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:59.939+0000 7f88067fc700 1 --2- 172.21.15.16:0/4119472502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8808106a70 0x7f880819cd50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f87f000b4b0 tx=0x7f87f000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:59.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:59.939+0000 7f87fffff700 1 -- 172.21.15.16:0/4119472502 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87f0011800 con 0x7f8808106a70 2024-04-02T04:38:59.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:59.939+0000 7f87fffff700 1 -- 172.21.15.16:0/4119472502 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f87f0011e20 con 0x7f8808106a70 2024-04-02T04:38:59.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:59.939+0000 7f880d62e700 1 -- 172.21.15.16:0/4119472502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8808196c30 con 0x7f8808106a70 2024-04-02T04:38:59.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:59.939+0000 7f87fffff700 1 -- 172.21.15.16:0/4119472502 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87f000f2e0 con 0x7f8808106a70 2024-04-02T04:38:59.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:59.939+0000 7f880d62e700 1 -- 172.21.15.16:0/4119472502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8808197180 con 0x7f8808106a70 2024-04-02T04:38:59.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:59.940+0000 7f880d62e700 1 -- 172.21.15.16:0/4119472502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8808109070 con 0x7f8808106a70 2024-04-02T04:38:59.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:59.940+0000 7f87fffff700 1 -- 172.21.15.16:0/4119472502 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f87f0010380 con 0x7f8808106a70 2024-04-02T04:38:59.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:59.941+0000 7f87fffff700 1 --2- 172.21.15.16:0/4119472502 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f87f406cce0 0x7f87f406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:38:59.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:59.941+0000 7f8806ffd700 1 --2- 172.21.15.16:0/4119472502 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f87f406cce0 0x7f87f406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:38:59.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:59.941+0000 7f87fffff700 1 -- 172.21.15.16:0/4119472502 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f87f008bb80 con 0x7f8808106a70 2024-04-02T04:38:59.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:59.941+0000 7f8806ffd700 1 --2- 172.21.15.16:0/4119472502 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f87f406cce0 0x7f87f406f1a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f880000b5c0 tx=0x7f880001a040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:38:59.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:38:59.945+0000 7f87fffff700 1 -- 172.21.15.16:0/4119472502 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f87f0059c80 con 0x7f8808106a70 2024-04-02T04:39:00.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:38:59 smithi016 ceph-mon[28969]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:39:00.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:38:59 smithi027 ceph-mon[37550]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:39:00.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.149+0000 7f880d62e700 1 -- 172.21.15.16:0/4119472502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f88081021b0 con 0x7f8808106a70 2024-04-02T04:39:00.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.149+0000 7f87fffff700 1 -- 172.21.15.16:0/4119472502 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f87f0059810 con 0x7f8808106a70 2024-04-02T04:39:00.152 INFO:teuthology.orchestra.run.smithi016.stdout:38654705702 2024-04-02T04:39:00.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.151+0000 7f880d62e700 1 -- 172.21.15.16:0/4119472502 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f87f406cce0 msgr2=0x7f87f406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:00.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.151+0000 7f880d62e700 1 --2- 172.21.15.16:0/4119472502 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f87f406cce0 0x7f87f406f1a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f880000b5c0 tx=0x7f880001a040 comp rx=0 tx=0).stop 2024-04-02T04:39:00.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.151+0000 7f880d62e700 1 -- 172.21.15.16:0/4119472502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8808106a70 msgr2=0x7f880819cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:00.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.151+0000 7f880d62e700 1 --2- 172.21.15.16:0/4119472502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8808106a70 0x7f880819cd50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f87f000b4b0 tx=0x7f87f000b870 comp rx=0 tx=0).stop 2024-04-02T04:39:00.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.151+0000 7f880d62e700 1 -- 172.21.15.16:0/4119472502 shutdown_connections 2024-04-02T04:39:00.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.151+0000 7f880d62e700 1 --2- 172.21.15.16:0/4119472502 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f87f406cce0 0x7f87f406f1a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:00.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.151+0000 7f880d62e700 1 --2- 172.21.15.16:0/4119472502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8808100a70 0x7f880819c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:00.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.151+0000 7f880d62e700 1 --2- 172.21.15.16:0/4119472502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8808106a70 0x7f880819cd50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:00.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.151+0000 7f880d62e700 1 -- 172.21.15.16:0/4119472502 >> 172.21.15.16:0/4119472502 conn(0x7f88080fc580 msgr2=0x7f88080fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:00.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.151+0000 7f880d62e700 1 -- 172.21.15.16:0/4119472502 shutdown_connections 2024-04-02T04:39:00.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.151+0000 7f880d62e700 1 -- 172.21.15.16:0/4119472502 wait complete. 2024-04-02T04:39:00.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.572+0000 7f71399bd700 1 -- 172.21.15.16:0/308275544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71340ff380 msgr2=0x7f71340ff800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:00.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.572+0000 7f71399bd700 1 --2- 172.21.15.16:0/308275544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71340ff380 0x7f71340ff800 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f711c009910 tx=0x7f711c009c20 comp rx=0 tx=0).stop 2024-04-02T04:39:00.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.572+0000 7f71399bd700 1 -- 172.21.15.16:0/308275544 shutdown_connections 2024-04-02T04:39:00.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.572+0000 7f71399bd700 1 --2- 172.21.15.16:0/308275544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71340ff380 0x7f71340ff800 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:00.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.572+0000 7f71399bd700 1 --2- 172.21.15.16:0/308275544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7134106d20 0x7f7134107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:00.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.572+0000 7f71399bd700 1 -- 172.21.15.16:0/308275544 >> 172.21.15.16:0/308275544 conn(0x7f7134074c60 msgr2=0x7f7134075070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:00.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.572+0000 7f71399bd700 1 -- 172.21.15.16:0/308275544 shutdown_connections 2024-04-02T04:39:00.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.572+0000 7f71399bd700 1 -- 172.21.15.16:0/308275544 wait complete. 2024-04-02T04:39:00.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.572+0000 7f71399bd700 1 Processor -- start 2024-04-02T04:39:00.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.573+0000 7f71399bd700 1 -- start start 2024-04-02T04:39:00.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.573+0000 7f71399bd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71340ff380 0x7f7134100560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:00.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.573+0000 7f71399bd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7134106d20 0x7f7134100aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:00.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.573+0000 7f71399bd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7134100fe0 con 0x7f7134106d20 2024-04-02T04:39:00.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.573+0000 7f71399bd700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7134101120 con 0x7f71340ff380 2024-04-02T04:39:00.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.573+0000 7f71327fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7134106d20 0x7f7134100aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:00.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.573+0000 7f71327fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7134106d20 0x7f7134100aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53096/0 (socket says 172.21.15.16:53096) 2024-04-02T04:39:00.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.573+0000 7f71327fc700 1 -- 172.21.15.16:0/510582487 learned_addr learned my addr 172.21.15.16:0/510582487 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:39:00.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.573+0000 7f7132ffd700 1 --2- 172.21.15.16:0/510582487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71340ff380 0x7f7134100560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:00.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.573+0000 7f71327fc700 1 -- 172.21.15.16:0/510582487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71340ff380 msgr2=0x7f7134100560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:00.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.573+0000 7f71327fc700 1 --2- 172.21.15.16:0/510582487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71340ff380 0x7f7134100560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:00.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.573+0000 7f71327fc700 1 -- 172.21.15.16:0/510582487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f711c0095f0 con 0x7f7134106d20 2024-04-02T04:39:00.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.573+0000 7f71327fc700 1 --2- 172.21.15.16:0/510582487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7134106d20 0x7f7134100aa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f711c006010 tx=0x7f711c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:00.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.574+0000 7f71389bb700 1 -- 172.21.15.16:0/510582487 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f711c01d070 con 0x7f7134106d20 2024-04-02T04:39:00.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.574+0000 7f71389bb700 1 -- 172.21.15.16:0/510582487 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f711c0056e0 con 0x7f7134106d20 2024-04-02T04:39:00.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.574+0000 7f71399bd700 1 -- 172.21.15.16:0/510582487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71341013a0 con 0x7f7134106d20 2024-04-02T04:39:00.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.574+0000 7f71389bb700 1 -- 172.21.15.16:0/510582487 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f711c00fdf0 con 0x7f7134106d20 2024-04-02T04:39:00.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.574+0000 7f71399bd700 1 -- 172.21.15.16:0/510582487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71341a3030 con 0x7f7134106d20 2024-04-02T04:39:00.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.575+0000 7f71399bd700 1 -- 172.21.15.16:0/510582487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7134109220 con 0x7f7134106d20 2024-04-02T04:39:00.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.575+0000 7f71389bb700 1 -- 172.21.15.16:0/510582487 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f711c00f5f0 con 0x7f7134106d20 2024-04-02T04:39:00.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.576+0000 7f71389bb700 1 --2- 172.21.15.16:0/510582487 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f712006cce0 0x7f712006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:00.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.576+0000 7f71389bb700 1 -- 172.21.15.16:0/510582487 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f711c08d1e0 con 0x7f7134106d20 2024-04-02T04:39:00.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.576+0000 7f7132ffd700 1 --2- 172.21.15.16:0/510582487 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f712006cce0 0x7f712006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:00.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.576+0000 7f7132ffd700 1 --2- 172.21.15.16:0/510582487 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f712006cce0 0x7f712006f1a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f712c005950 tx=0x7f712c00b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:00.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.580+0000 7f71389bb700 1 -- 172.21.15.16:0/510582487 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f711c0578f0 con 0x7f7134106d20 2024-04-02T04:39:00.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.782+0000 7f71399bd700 1 -- 172.21.15.16:0/510582487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7134068cb0 con 0x7f7134106d20 2024-04-02T04:39:00.786 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:00 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/4119472502' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-02T04:39:00.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.782+0000 7f71389bb700 1 -- 172.21.15.16:0/510582487 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f711c05af10 con 0x7f7134106d20 2024-04-02T04:39:00.786 INFO:teuthology.orchestra.run.smithi016.stdout:111669149719 2024-04-02T04:39:00.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.784+0000 7f71399bd700 1 -- 172.21.15.16:0/510582487 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f712006cce0 msgr2=0x7f712006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:00.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.784+0000 7f71399bd700 1 --2- 172.21.15.16:0/510582487 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f712006cce0 0x7f712006f1a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f712c005950 tx=0x7f712c00b410 comp rx=0 tx=0).stop 2024-04-02T04:39:00.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.784+0000 7f71399bd700 1 -- 172.21.15.16:0/510582487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7134106d20 msgr2=0x7f7134100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:00.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.784+0000 7f71399bd700 1 --2- 172.21.15.16:0/510582487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7134106d20 0x7f7134100aa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f711c006010 tx=0x7f711c00bd70 comp rx=0 tx=0).stop 2024-04-02T04:39:00.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.784+0000 7f71399bd700 1 -- 172.21.15.16:0/510582487 shutdown_connections 2024-04-02T04:39:00.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.784+0000 7f71399bd700 1 --2- 172.21.15.16:0/510582487 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f712006cce0 0x7f712006f1a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:00.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.784+0000 7f71399bd700 1 --2- 172.21.15.16:0/510582487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71340ff380 0x7f7134100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:00.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.784+0000 7f71399bd700 1 --2- 172.21.15.16:0/510582487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7134106d20 0x7f7134100aa0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:00.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.784+0000 7f71399bd700 1 -- 172.21.15.16:0/510582487 >> 172.21.15.16:0/510582487 conn(0x7f7134074c60 msgr2=0x7f71340feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:00.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.785+0000 7f71399bd700 1 -- 172.21.15.16:0/510582487 shutdown_connections 2024-04-02T04:39:00.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:00.785+0000 7f71399bd700 1 -- 172.21.15.16:0/510582487 wait complete. 2024-04-02T04:39:01.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:00 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/4119472502' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-02T04:39:01.515 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149717 got 111669149719 for osd.3 2024-04-02T04:39:01.515 DEBUG:teuthology.parallel:result is None 2024-04-02T04:39:01.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:01 smithi016 ceph-mon[28969]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:39:01.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:01 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/510582487' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-02T04:39:01.895 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:01 smithi027 ceph-mon[37550]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:39:01.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:01 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/510582487' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-02T04:39:01.899 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705700 got 38654705702 for osd.0 2024-04-02T04:39:01.899 DEBUG:teuthology.parallel:result is None 2024-04-02T04:39:02.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.108+0000 7efde3916700 1 -- 172.21.15.16:0/1925875878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efddc0ff990 msgr2=0x7efddc0ffe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:02.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.108+0000 7efde3916700 1 --2- 172.21.15.16:0/1925875878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efddc0ff990 0x7efddc0ffe10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7efdcc009910 tx=0x7efdcc009c20 comp rx=0 tx=0).stop 2024-04-02T04:39:02.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.109+0000 7efde3916700 1 -- 172.21.15.16:0/1925875878 shutdown_connections 2024-04-02T04:39:02.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.109+0000 7efde3916700 1 --2- 172.21.15.16:0/1925875878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efddc0ff990 0x7efddc0ffe10 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:02.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.109+0000 7efde3916700 1 --2- 172.21.15.16:0/1925875878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efddc105990 0x7efddc105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:02.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.109+0000 7efde3916700 1 -- 172.21.15.16:0/1925875878 >> 172.21.15.16:0/1925875878 conn(0x7efddc0fb4c0 msgr2=0x7efddc0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:02.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.109+0000 7efde3916700 1 -- 172.21.15.16:0/1925875878 shutdown_connections 2024-04-02T04:39:02.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.109+0000 7efde3916700 1 -- 172.21.15.16:0/1925875878 wait complete. 2024-04-02T04:39:02.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.109+0000 7efde3916700 1 Processor -- start 2024-04-02T04:39:02.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.109+0000 7efde3916700 1 -- start start 2024-04-02T04:39:02.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.109+0000 7efde3916700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efddc0ff990 0x7efddc19e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:02.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.109+0000 7efde3916700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efddc105990 0x7efddc19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:02.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.109+0000 7efde3916700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efddc1989f0 con 0x7efddc0ff990 2024-04-02T04:39:02.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.109+0000 7efde3916700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efddc198b60 con 0x7efddc105990 2024-04-02T04:39:02.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.110+0000 7efde16b2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efddc0ff990 0x7efddc19e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:02.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.110+0000 7efde16b2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efddc0ff990 0x7efddc19e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55150/0 (socket says 172.21.15.16:55150) 2024-04-02T04:39:02.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.110+0000 7efde16b2700 1 -- 172.21.15.16:0/31343071 learned_addr learned my addr 172.21.15.16:0/31343071 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:39:02.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.110+0000 7efde0eb1700 1 --2- 172.21.15.16:0/31343071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efddc105990 0x7efddc19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:02.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.110+0000 7efde16b2700 1 -- 172.21.15.16:0/31343071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efddc105990 msgr2=0x7efddc19ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:02.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.110+0000 7efde16b2700 1 --2- 172.21.15.16:0/31343071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efddc105990 0x7efddc19ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:02.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.110+0000 7efde16b2700 1 -- 172.21.15.16:0/31343071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdcc0095f0 con 0x7efddc0ff990 2024-04-02T04:39:02.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.110+0000 7efde16b2700 1 --2- 172.21.15.16:0/31343071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efddc0ff990 0x7efddc19e940 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efdd800d8d0 tx=0x7efdd800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:02.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.110+0000 7efdd27fc700 1 -- 172.21.15.16:0/31343071 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdd80098a0 con 0x7efddc0ff990 2024-04-02T04:39:02.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.110+0000 7efdd27fc700 1 -- 172.21.15.16:0/31343071 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efdd8010460 con 0x7efddc0ff990 2024-04-02T04:39:02.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.110+0000 7efdd27fc700 1 -- 172.21.15.16:0/31343071 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdd800f5f0 con 0x7efddc0ff990 2024-04-02T04:39:02.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.110+0000 7efde3916700 1 -- 172.21.15.16:0/31343071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efddc198e40 con 0x7efddc0ff990 2024-04-02T04:39:02.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.110+0000 7efde3916700 1 -- 172.21.15.16:0/31343071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efddc199390 con 0x7efddc0ff990 2024-04-02T04:39:02.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.111+0000 7efde3916700 1 -- 172.21.15.16:0/31343071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efddc1010d0 con 0x7efddc0ff990 2024-04-02T04:39:02.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.112+0000 7efdd27fc700 1 -- 172.21.15.16:0/31343071 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7efdd8009a00 con 0x7efddc0ff990 2024-04-02T04:39:02.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.112+0000 7efdd27fc700 1 --2- 172.21.15.16:0/31343071 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7efdc806cd30 0x7efdc806f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:02.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.112+0000 7efdd27fc700 1 -- 172.21.15.16:0/31343071 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7efdd808bdc0 con 0x7efddc0ff990 2024-04-02T04:39:02.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.112+0000 7efde0eb1700 1 --2- 172.21.15.16:0/31343071 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7efdc806cd30 0x7efdc806f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:02.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.113+0000 7efde0eb1700 1 --2- 172.21.15.16:0/31343071 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7efdc806cd30 0x7efdc806f1f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efddc19a160 tx=0x7efdcc01a040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:02.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.117+0000 7efdd27fc700 1 -- 172.21.15.16:0/31343071 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7efdd80564b0 con 0x7efddc0ff990 2024-04-02T04:39:02.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.320+0000 7efde3916700 1 -- 172.21.15.16:0/31343071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7efddc10cb90 con 0x7efddc0ff990 2024-04-02T04:39:02.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.320+0000 7efdd27fc700 1 -- 172.21.15.16:0/31343071 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efdd8059ad0 con 0x7efddc0ff990 2024-04-02T04:39:02.324 INFO:teuthology.orchestra.run.smithi016.stdout:146028888081 2024-04-02T04:39:02.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.322+0000 7efde3916700 1 -- 172.21.15.16:0/31343071 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7efdc806cd30 msgr2=0x7efdc806f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:02.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.322+0000 7efde3916700 1 --2- 172.21.15.16:0/31343071 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7efdc806cd30 0x7efdc806f1f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efddc19a160 tx=0x7efdcc01a040 comp rx=0 tx=0).stop 2024-04-02T04:39:02.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.322+0000 7efde3916700 1 -- 172.21.15.16:0/31343071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efddc0ff990 msgr2=0x7efddc19e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:02.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.322+0000 7efde3916700 1 --2- 172.21.15.16:0/31343071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efddc0ff990 0x7efddc19e940 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efdd800d8d0 tx=0x7efdd800dbe0 comp rx=0 tx=0).stop 2024-04-02T04:39:02.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.322+0000 7efde3916700 1 -- 172.21.15.16:0/31343071 shutdown_connections 2024-04-02T04:39:02.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.322+0000 7efde3916700 1 --2- 172.21.15.16:0/31343071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efddc0ff990 0x7efddc19e940 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:02.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.322+0000 7efde3916700 1 --2- 172.21.15.16:0/31343071 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7efdc806cd30 0x7efdc806f1f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:02.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.322+0000 7efde3916700 1 --2- 172.21.15.16:0/31343071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efddc105990 0x7efddc19ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:02.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.322+0000 7efde3916700 1 -- 172.21.15.16:0/31343071 >> 172.21.15.16:0/31343071 conn(0x7efddc0fb4c0 msgr2=0x7efddc0fc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:02.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.323+0000 7efde3916700 1 -- 172.21.15.16:0/31343071 shutdown_connections 2024-04-02T04:39:02.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.323+0000 7efde3916700 1 -- 172.21.15.16:0/31343071 wait complete. 2024-04-02T04:39:02.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.375+0000 7f7f6c7b2700 1 -- 172.21.15.16:0/3404816223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f64101d30 msgr2=0x7f7f64102110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:02.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.375+0000 7f7f6c7b2700 1 --2- 172.21.15.16:0/3404816223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f64101d30 0x7f7f64102110 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7f60009940 tx=0x7f7f60009c50 comp rx=0 tx=0).stop 2024-04-02T04:39:02.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.375+0000 7f7f6c7b2700 1 -- 172.21.15.16:0/3404816223 shutdown_connections 2024-04-02T04:39:02.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.375+0000 7f7f6c7b2700 1 --2- 172.21.15.16:0/3404816223 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f64102650 0x7f7f6410ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:02.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.375+0000 7f7f6c7b2700 1 --2- 172.21.15.16:0/3404816223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f64101d30 0x7f7f64102110 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:02.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.375+0000 7f7f6c7b2700 1 -- 172.21.15.16:0/3404816223 >> 172.21.15.16:0/3404816223 conn(0x7f7f640fb540 msgr2=0x7f7f640fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:02.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.375+0000 7f7f6c7b2700 1 -- 172.21.15.16:0/3404816223 shutdown_connections 2024-04-02T04:39:02.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.375+0000 7f7f6c7b2700 1 -- 172.21.15.16:0/3404816223 wait complete. 2024-04-02T04:39:02.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.375+0000 7f7f6c7b2700 1 Processor -- start 2024-04-02T04:39:02.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.375+0000 7f7f6c7b2700 1 -- start start 2024-04-02T04:39:02.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.376+0000 7f7f6c7b2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f64101d30 0x7f7f6419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:02.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.376+0000 7f7f6c7b2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f64102650 0x7f7f6419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:02.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.376+0000 7f7f6c7b2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6419d310 con 0x7f7f64102650 2024-04-02T04:39:02.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.376+0000 7f7f6c7b2700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f64196890 con 0x7f7f64101d30 2024-04-02T04:39:02.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.376+0000 7f7f69d4d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f64102650 0x7f7f6419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:02.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.376+0000 7f7f69d4d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f64102650 0x7f7f6419cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55170/0 (socket says 172.21.15.16:55170) 2024-04-02T04:39:02.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.376+0000 7f7f69d4d700 1 -- 172.21.15.16:0/2421346308 learned_addr learned my addr 172.21.15.16:0/2421346308 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:39:02.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.376+0000 7f7f6a54e700 1 --2- 172.21.15.16:0/2421346308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f64101d30 0x7f7f6419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:02.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.376+0000 7f7f69d4d700 1 -- 172.21.15.16:0/2421346308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f64101d30 msgr2=0x7f7f6419c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:02.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.376+0000 7f7f69d4d700 1 --2- 172.21.15.16:0/2421346308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f64101d30 0x7f7f6419c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:02.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.376+0000 7f7f69d4d700 1 -- 172.21.15.16:0/2421346308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f54009520 con 0x7f7f64102650 2024-04-02T04:39:02.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.376+0000 7f7f69d4d700 1 --2- 172.21.15.16:0/2421346308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f64102650 0x7f7f6419cc30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400e9b0 tx=0x7f7f5400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:02.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.376+0000 7f7f5b7fe700 1 -- 172.21.15.16:0/2421346308 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f54009820 con 0x7f7f64102650 2024-04-02T04:39:02.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.376+0000 7f7f5b7fe700 1 -- 172.21.15.16:0/2421346308 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7f540044e0 con 0x7f7f64102650 2024-04-02T04:39:02.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.376+0000 7f7f5b7fe700 1 -- 172.21.15.16:0/2421346308 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f54005020 con 0x7f7f64102650 2024-04-02T04:39:02.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.376+0000 7f7f6c7b2700 1 -- 172.21.15.16:0/2421346308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f600095f0 con 0x7f7f64102650 2024-04-02T04:39:02.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.377+0000 7f7f6c7b2700 1 -- 172.21.15.16:0/2421346308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f64196f30 con 0x7f7f64102650 2024-04-02T04:39:02.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.378+0000 7f7f6c7b2700 1 -- 172.21.15.16:0/2421346308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f64105300 con 0x7f7f64102650 2024-04-02T04:39:02.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.378+0000 7f7f5b7fe700 1 -- 172.21.15.16:0/2421346308 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f7f5400cba0 con 0x7f7f64102650 2024-04-02T04:39:02.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.379+0000 7f7f5b7fe700 1 --2- 172.21.15.16:0/2421346308 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f7f5006cce0 0x7f7f5006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:02.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.379+0000 7f7f6a54e700 1 --2- 172.21.15.16:0/2421346308 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f7f5006cce0 0x7f7f5006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:02.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.379+0000 7f7f5b7fe700 1 -- 172.21.15.16:0/2421346308 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f7f54014070 con 0x7f7f64102650 2024-04-02T04:39:02.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.379+0000 7f7f6a54e700 1 --2- 172.21.15.16:0/2421346308 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f7f5006cce0 0x7f7f5006f1a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7f60009fa0 tx=0x7f7f600058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:02.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.383+0000 7f7f5b7fe700 1 -- 172.21.15.16:0/2421346308 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7f5405a150 con 0x7f7f64102650 2024-04-02T04:39:02.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.588+0000 7f7f6c7b2700 1 -- 172.21.15.16:0/2421346308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7f64197d70 con 0x7f7f64102650 2024-04-02T04:39:02.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.588+0000 7f7f5b7fe700 1 -- 172.21.15.16:0/2421346308 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7f54059ce0 con 0x7f7f64102650 2024-04-02T04:39:02.591 INFO:teuthology.orchestra.run.smithi016.stdout:60129542176 2024-04-02T04:39:02.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.590+0000 7f7f6c7b2700 1 -- 172.21.15.16:0/2421346308 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f7f5006cce0 msgr2=0x7f7f5006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:02.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.590+0000 7f7f6c7b2700 1 --2- 172.21.15.16:0/2421346308 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f7f5006cce0 0x7f7f5006f1a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7f60009fa0 tx=0x7f7f600058e0 comp rx=0 tx=0).stop 2024-04-02T04:39:02.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.590+0000 7f7f6c7b2700 1 -- 172.21.15.16:0/2421346308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f64102650 msgr2=0x7f7f6419cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:02.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.590+0000 7f7f6c7b2700 1 --2- 172.21.15.16:0/2421346308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f64102650 0x7f7f6419cc30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400e9b0 tx=0x7f7f5400ecc0 comp rx=0 tx=0).stop 2024-04-02T04:39:02.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.590+0000 7f7f6c7b2700 1 -- 172.21.15.16:0/2421346308 shutdown_connections 2024-04-02T04:39:02.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.590+0000 7f7f6c7b2700 1 --2- 172.21.15.16:0/2421346308 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f7f5006cce0 0x7f7f5006f1a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:02.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.590+0000 7f7f6c7b2700 1 --2- 172.21.15.16:0/2421346308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f64101d30 0x7f7f6419c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:02.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.590+0000 7f7f6c7b2700 1 --2- 172.21.15.16:0/2421346308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f64102650 0x7f7f6419cc30 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:02.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.590+0000 7f7f6c7b2700 1 -- 172.21.15.16:0/2421346308 >> 172.21.15.16:0/2421346308 conn(0x7f7f640fb540 msgr2=0x7f7f64103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:02.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.590+0000 7f7f6c7b2700 1 -- 172.21.15.16:0/2421346308 shutdown_connections 2024-04-02T04:39:02.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:02.590+0000 7f7f6c7b2700 1 -- 172.21.15.16:0/2421346308 wait complete. 2024-04-02T04:39:03.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:02 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/31343071' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-02T04:39:03.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:02 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/2421346308' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-02T04:39:03.111 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888079 got 146028888081 for osd.4 2024-04-02T04:39:03.112 DEBUG:teuthology.parallel:result is None 2024-04-02T04:39:03.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:02 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/31343071' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-02T04:39:03.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:02 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/2421346308' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-02T04:39:03.526 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542174 got 60129542176 for osd.1 2024-04-02T04:39:03.527 DEBUG:teuthology.parallel:result is None 2024-04-02T04:39:03.527 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-02T04:39:03.527 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph pg dump --format=json 2024-04-02T04:39:03.739 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:39:03.809 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:03 smithi016 ceph-mon[28969]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:39:04.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:03 smithi027 ceph-mon[37550]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:39:05.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.171+0000 7f2861ffb700 1 -- 172.21.15.16:0/920681168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285c106a70 msgr2=0x7f285c106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:05.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.171+0000 7f2861ffb700 1 --2- 172.21.15.16:0/920681168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285c106a70 0x7f285c106e50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f284c009910 tx=0x7f284c009c20 comp rx=0 tx=0).stop 2024-04-02T04:39:05.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.171+0000 7f2861ffb700 1 -- 172.21.15.16:0/920681168 shutdown_connections 2024-04-02T04:39:05.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.171+0000 7f2861ffb700 1 --2- 172.21.15.16:0/920681168 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f285c100a70 0x7f285c100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:05.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.171+0000 7f2861ffb700 1 --2- 172.21.15.16:0/920681168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285c106a70 0x7f285c106e50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:05.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.171+0000 7f2861ffb700 1 -- 172.21.15.16:0/920681168 >> 172.21.15.16:0/920681168 conn(0x7f285c0fc5a0 msgr2=0x7f285c0fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:05.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.171+0000 7f2861ffb700 1 -- 172.21.15.16:0/920681168 shutdown_connections 2024-04-02T04:39:05.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.171+0000 7f2861ffb700 1 -- 172.21.15.16:0/920681168 wait complete. 2024-04-02T04:39:05.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.171+0000 7f2861ffb700 1 Processor -- start 2024-04-02T04:39:05.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.171+0000 7f2861ffb700 1 -- start start 2024-04-02T04:39:05.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.172+0000 7f2861ffb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285c100a70 0x7f285c0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:05.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.172+0000 7f2861ffb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f285c106a70 0x7f285c0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:05.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.172+0000 7f2861ffb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285c073b30 con 0x7f285c100a70 2024-04-02T04:39:05.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.172+0000 7f2861ffb700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285c073c70 con 0x7f285c106a70 2024-04-02T04:39:05.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.172+0000 7f285b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285c100a70 0x7f285c0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:05.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.172+0000 7f285b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285c100a70 0x7f285c0730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55188/0 (socket says 172.21.15.16:55188) 2024-04-02T04:39:05.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.172+0000 7f285b7fe700 1 -- 172.21.15.16:0/2831937263 learned_addr learned my addr 172.21.15.16:0/2831937263 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:39:05.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.172+0000 7f285affd700 1 --2- 172.21.15.16:0/2831937263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f285c106a70 0x7f285c0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:05.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.172+0000 7f285b7fe700 1 -- 172.21.15.16:0/2831937263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f285c106a70 msgr2=0x7f285c0735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:05.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.172+0000 7f285b7fe700 1 --2- 172.21.15.16:0/2831937263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f285c106a70 0x7f285c0735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:05.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.172+0000 7f285b7fe700 1 -- 172.21.15.16:0/2831937263 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f284c0095f0 con 0x7f285c100a70 2024-04-02T04:39:05.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.172+0000 7f285affd700 1 --2- 172.21.15.16:0/2831937263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f285c106a70 0x7f285c0735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T04:39:05.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.172+0000 7f285b7fe700 1 --2- 172.21.15.16:0/2831937263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285c100a70 0x7f285c0730b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f284c00bd70 tx=0x7f284c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:05.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.172+0000 7f2858ff9700 1 -- 172.21.15.16:0/2831937263 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f284c01d070 con 0x7f285c100a70 2024-04-02T04:39:05.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.172+0000 7f2858ff9700 1 -- 172.21.15.16:0/2831937263 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f284c00f460 con 0x7f285c100a70 2024-04-02T04:39:05.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.172+0000 7f2861ffb700 1 -- 172.21.15.16:0/2831937263 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f285c073ef0 con 0x7f285c100a70 2024-04-02T04:39:05.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.172+0000 7f2858ff9700 1 -- 172.21.15.16:0/2831937263 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f284c017630 con 0x7f285c100a70 2024-04-02T04:39:05.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.173+0000 7f2861ffb700 1 -- 172.21.15.16:0/2831937263 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f285c1a2df0 con 0x7f285c100a70 2024-04-02T04:39:05.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.174+0000 7f2861ffb700 1 -- 172.21.15.16:0/2831937263 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f285c109070 con 0x7f285c100a70 2024-04-02T04:39:05.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.174+0000 7f2858ff9700 1 -- 172.21.15.16:0/2831937263 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f284c00f5d0 con 0x7f285c100a70 2024-04-02T04:39:05.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.174+0000 7f2858ff9700 1 --2- 172.21.15.16:0/2831937263 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f284406cce0 0x7f284406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:05.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.174+0000 7f2858ff9700 1 -- 172.21.15.16:0/2831937263 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f284c08d380 con 0x7f285c100a70 2024-04-02T04:39:05.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.174+0000 7f285affd700 1 --2- 172.21.15.16:0/2831937263 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f284406cce0 0x7f284406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:05.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.175+0000 7f285affd700 1 --2- 172.21.15.16:0/2831937263 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f284406cce0 0x7f284406f1a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2850005fd0 tx=0x7f2850005dc0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:05.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.179+0000 7f2858ff9700 1 -- 172.21.15.16:0/2831937263 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f284c057b10 con 0x7f285c100a70 2024-04-02T04:39:05.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.378+0000 7f2861ffb700 1 -- 172.21.15.16:0/2831937263 --> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f285c10dc70 con 0x7f284406cce0 2024-04-02T04:39:05.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.380+0000 7f2858ff9700 1 -- 172.21.15.16:0/2831937263 <== mgr.14217 v2:172.21.15.16:6800/3643600900 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27947 (secure 0 0 0) 0x7f285c10dc70 con 0x7f284406cce0 2024-04-02T04:39:05.383 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:39:05.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.382+0000 7f2861ffb700 1 -- 172.21.15.16:0/2831937263 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f284406cce0 msgr2=0x7f284406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:05.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.382+0000 7f2861ffb700 1 --2- 172.21.15.16:0/2831937263 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f284406cce0 0x7f284406f1a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2850005fd0 tx=0x7f2850005dc0 comp rx=0 tx=0).stop 2024-04-02T04:39:05.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.382+0000 7f2861ffb700 1 -- 172.21.15.16:0/2831937263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285c100a70 msgr2=0x7f285c0730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:05.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.382+0000 7f2861ffb700 1 --2- 172.21.15.16:0/2831937263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285c100a70 0x7f285c0730b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f284c00bd70 tx=0x7f284c00be50 comp rx=0 tx=0).stop 2024-04-02T04:39:05.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.382+0000 7f2861ffb700 1 -- 172.21.15.16:0/2831937263 shutdown_connections 2024-04-02T04:39:05.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.382+0000 7f2861ffb700 1 --2- 172.21.15.16:0/2831937263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285c100a70 0x7f285c0730b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:05.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.382+0000 7f2861ffb700 1 --2- 172.21.15.16:0/2831937263 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f284406cce0 0x7f284406f1a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:05.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.382+0000 7f2861ffb700 1 --2- 172.21.15.16:0/2831937263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f285c106a70 0x7f285c0735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:05.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.382+0000 7f2861ffb700 1 -- 172.21.15.16:0/2831937263 >> 172.21.15.16:0/2831937263 conn(0x7f285c0fc5a0 msgr2=0x7f285c0fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:05.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.382+0000 7f2861ffb700 1 -- 172.21.15.16:0/2831937263 shutdown_connections 2024-04-02T04:39:05.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:05.382+0000 7f2861ffb700 1 -- 172.21.15.16:0/2831937263 wait complete. 2024-04-02T04:39:05.386 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-02T04:39:05.975 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:05 smithi016 ceph-mon[28969]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:39:05.996 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":159,"stamp":"2024-04-02T04:39:04.760130+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":202888,"kb_used_data":3876,"kb_used_omap":9,"kb_used_meta":198902,"kb_avail":562268024,"statfs":{"total":575970213888,"available":575762456576,"internally_reserved":0,"allocated":3969024,"data_stored":2606850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":203676326},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001912"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":186,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T04:38:16.981168+0000","last_change":"2024-04-02T04:37:49.357662+0000","last_active":"2024-04-02T04:38:16.981168+0000","last_peered":"2024-04-02T04:38:16.981168+0000","last_clean":"2024-04-02T04:38:16.981168+0000","last_became_active":"2024-04-02T04:37:49.357233+0000","last_became_peered":"2024-04-02T04:37:49.357233+0000","last_unstale":"2024-04-02T04:38:16.981168+0000","last_undegraded":"2024-04-02T04:38:16.981168+0000","last_fullsized":"2024-04-02T04:38:16.981168+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T04:37:01.692885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T04:37:01.692885+0000","last_clean_scrub_stamp":"2024-04-02T04:37:01.692885+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T11:22:59.386529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724555,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27376,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717776,"statfs":{"total":95995035648,"available":95967002624,"internally_reserved":0,"allocated":364544,"data_stored":139291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]}]},{"osd":4,"up_from":34,"seq":146028888082,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":31476,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713676,"statfs":{"total":95995035648,"available":95962804224,"internally_reserved":0,"allocated":364544,"data_stored":139291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Apr 2 04:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.435},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.52500000000000002}]},{"osd":1,"last update":"Tue Apr 2 04:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.60499999999999998}]},{"osd":2,"last update":"Tue Apr 2 04:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.58599999999999997}]},{"osd":3,"last update":"Tue Apr 2 04:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.495}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149720,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":729659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Apr 2 04:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.47999999999999998}]},{"osd":1,"last update":"Tue Apr 2 04:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.443}]},{"osd":2,"last update":"Tue Apr 2 04:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.55700000000000005}]},{"osd":4,"last update":"Tue Apr 2 04:38:49 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.46600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]}]},{"osd":2,"up_from":19,"seq":81604378652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":35576,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709576,"statfs":{"total":95995035648,"available":95958605824,"internally_reserved":0,"allocated":364544,"data_stored":139291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Apr 2 04:38:02 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"last":0.317},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.499,"5min":0.499,"15min":0.499},"last":0.27800000000000002}]},{"osd":1,"last update":"Tue Apr 2 04:38:02 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.501,"5min":0.501,"15min":0.501},"last":0.30199999999999999}]},{"osd":3,"last update":"Tue Apr 2 04:38:23 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.56100000000000005}]},{"osd":4,"last update":"Tue Apr 2 04:38:51 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.505},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":3.9889999999999999,"5min":3.9889999999999999,"15min":3.9889999999999999},"last":0.52100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]}]},{"osd":0,"up_from":9,"seq":38654705704,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40252,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":93704900,"statfs":{"total":95995035648,"available":95953817600,"internally_reserved":0,"allocated":958464,"data_stored":729659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Apr 2 04:38:36 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.32600000000000001,"15min":0.32300000000000001},"min":{"1min":0.23799999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.45500000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.34000000000000002,"15min":0.33500000000000002},"min":{"1min":0.255,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.52300000000000002,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.23400000000000001}]},{"osd":2,"last update":"Tue Apr 2 04:39:02 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.375,"15min":0.373},"min":{"1min":0.252,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.39700000000000002,"15min":0.40200000000000002},"min":{"1min":0.27000000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.46999999999999997,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.27000000000000002}]},{"osd":3,"last update":"Tue Apr 2 04:38:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.437},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.42099999999999999}]},{"osd":4,"last update":"Tue Apr 2 04:38:51 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.50700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40252,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":93704900,"statfs":{"total":95995035648,"available":95953817600,"internally_reserved":0,"allocated":958464,"data_stored":729659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Apr 2 04:38:33 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.317,"15min":0.314},"min":{"1min":0.17999999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.247},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.33300000000000002,"15min":0.32700000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.502,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.30499999999999999}]},{"osd":2,"last update":"Tue Apr 2 04:38:02 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.26700000000000002}]},{"osd":3,"last update":"Tue Apr 2 04:38:27 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.44900000000000001}]},{"osd":4,"last update":"Tue Apr 2 04:38:51 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.435}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T04:39:05.997 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph pg dump --format=json 2024-04-02T04:39:06.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:05 smithi027 ceph-mon[37550]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:39:06.205 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:39:07.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:06 smithi016 ceph-mon[28969]: from='client.14432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T04:39:07.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:06 smithi027 ceph-mon[37550]: from='client.14432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T04:39:07.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.624+0000 7f9497db9700 1 -- 172.21.15.16:0/2750081883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9490106b40 msgr2=0x7f9490106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:07.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.624+0000 7f9497db9700 1 --2- 172.21.15.16:0/2750081883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9490106b40 0x7f9490106f20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f948000b0a0 tx=0x7f948000b3b0 comp rx=0 tx=0).stop 2024-04-02T04:39:07.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.624+0000 7f9497db9700 1 -- 172.21.15.16:0/2750081883 shutdown_connections 2024-04-02T04:39:07.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.624+0000 7f9497db9700 1 --2- 172.21.15.16:0/2750081883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94900ff240 0x7f94900ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.624+0000 7f9497db9700 1 --2- 172.21.15.16:0/2750081883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9490106b40 0x7f9490106f20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.624+0000 7f9497db9700 1 -- 172.21.15.16:0/2750081883 >> 172.21.15.16:0/2750081883 conn(0x7f9490074cc0 msgr2=0x7f94900750d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.625+0000 7f9497db9700 1 -- 172.21.15.16:0/2750081883 shutdown_connections 2024-04-02T04:39:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.625+0000 7f9497db9700 1 -- 172.21.15.16:0/2750081883 wait complete. 2024-04-02T04:39:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.625+0000 7f9497db9700 1 Processor -- start 2024-04-02T04:39:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.625+0000 7f9497db9700 1 -- start start 2024-04-02T04:39:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.625+0000 7f9497db9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94900ff240 0x7f9490100420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.625+0000 7f9497db9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9490106b40 0x7f9490100960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.625+0000 7f9497db9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9490100ea0 con 0x7f94900ff240 2024-04-02T04:39:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.625+0000 7f9497db9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9490100fe0 con 0x7f9490106b40 2024-04-02T04:39:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.625+0000 7f9495b55700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94900ff240 0x7f9490100420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.625+0000 7f9495b55700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94900ff240 0x7f9490100420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55198/0 (socket says 172.21.15.16:55198) 2024-04-02T04:39:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.625+0000 7f9495b55700 1 -- 172.21.15.16:0/9502743 learned_addr learned my addr 172.21.15.16:0/9502743 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:39:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.626+0000 7f9495354700 1 --2- 172.21.15.16:0/9502743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9490106b40 0x7f9490100960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.626+0000 7f9495b55700 1 -- 172.21.15.16:0/9502743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9490106b40 msgr2=0x7f9490100960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.626+0000 7f9495b55700 1 --2- 172.21.15.16:0/9502743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9490106b40 0x7f9490100960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.626+0000 7f9495b55700 1 -- 172.21.15.16:0/9502743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9480009d00 con 0x7f94900ff240 2024-04-02T04:39:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.626+0000 7f9495b55700 1 --2- 172.21.15.16:0/9502743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94900ff240 0x7f9490100420 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f948000b070 tx=0x7f9480007b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.626+0000 7f948effd700 1 -- 172.21.15.16:0/9502743 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f948000e040 con 0x7f94900ff240 2024-04-02T04:39:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.626+0000 7f948effd700 1 -- 172.21.15.16:0/9502743 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9480003c10 con 0x7f94900ff240 2024-04-02T04:39:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.626+0000 7f9497db9700 1 -- 172.21.15.16:0/9502743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9490101260 con 0x7f94900ff240 2024-04-02T04:39:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.626+0000 7f948effd700 1 -- 172.21.15.16:0/9502743 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f948001d6b0 con 0x7f94900ff240 2024-04-02T04:39:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.626+0000 7f9497db9700 1 -- 172.21.15.16:0/9502743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94901a2e30 con 0x7f94900ff240 2024-04-02T04:39:07.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.627+0000 7f9497db9700 1 -- 172.21.15.16:0/9502743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9490109040 con 0x7f94900ff240 2024-04-02T04:39:07.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.628+0000 7f948effd700 1 -- 172.21.15.16:0/9502743 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f9480004600 con 0x7f94900ff240 2024-04-02T04:39:07.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.628+0000 7f948effd700 1 --2- 172.21.15.16:0/9502743 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f947806ccf0 0x7f947806f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:07.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.628+0000 7f948effd700 1 -- 172.21.15.16:0/9502743 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f948008d6b0 con 0x7f94900ff240 2024-04-02T04:39:07.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.628+0000 7f9495354700 1 --2- 172.21.15.16:0/9502743 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f947806ccf0 0x7f947806f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:07.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.628+0000 7f9495354700 1 --2- 172.21.15.16:0/9502743 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f947806ccf0 0x7f947806f1b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9490101b50 tx=0x7f948400a300 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:07.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.633+0000 7f948effd700 1 -- 172.21.15.16:0/9502743 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9480057e40 con 0x7f94900ff240 2024-04-02T04:39:07.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.832+0000 7f9497db9700 1 -- 172.21.15.16:0/9502743 --> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f949010dc90 con 0x7f947806ccf0 2024-04-02T04:39:07.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.833+0000 7f948effd700 1 -- 172.21.15.16:0/9502743 <== mgr.14217 v2:172.21.15.16:6800/3643600900 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28073 (secure 0 0 0) 0x7f949010dc90 con 0x7f947806ccf0 2024-04-02T04:39:07.837 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:39:07.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.835+0000 7f9497db9700 1 -- 172.21.15.16:0/9502743 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f947806ccf0 msgr2=0x7f947806f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:07.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.835+0000 7f9497db9700 1 --2- 172.21.15.16:0/9502743 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f947806ccf0 0x7f947806f1b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9490101b50 tx=0x7f948400a300 comp rx=0 tx=0).stop 2024-04-02T04:39:07.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.835+0000 7f9497db9700 1 -- 172.21.15.16:0/9502743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94900ff240 msgr2=0x7f9490100420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:07.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.835+0000 7f9497db9700 1 --2- 172.21.15.16:0/9502743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94900ff240 0x7f9490100420 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f948000b070 tx=0x7f9480007b20 comp rx=0 tx=0).stop 2024-04-02T04:39:07.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.835+0000 7f9497db9700 1 -- 172.21.15.16:0/9502743 shutdown_connections 2024-04-02T04:39:07.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.835+0000 7f9497db9700 1 --2- 172.21.15.16:0/9502743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94900ff240 0x7f9490100420 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:07.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.835+0000 7f9497db9700 1 --2- 172.21.15.16:0/9502743 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f947806ccf0 0x7f947806f1b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:07.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.835+0000 7f9497db9700 1 --2- 172.21.15.16:0/9502743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9490106b40 0x7f9490100960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:07.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.835+0000 7f9497db9700 1 -- 172.21.15.16:0/9502743 >> 172.21.15.16:0/9502743 conn(0x7f9490074cc0 msgr2=0x7f9490105ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:07.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.836+0000 7f9497db9700 1 -- 172.21.15.16:0/9502743 shutdown_connections 2024-04-02T04:39:07.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:07.836+0000 7f9497db9700 1 -- 172.21.15.16:0/9502743 wait complete. 2024-04-02T04:39:07.840 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-04-02T04:39:08.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:07 smithi027 ceph-mon[37550]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:39:08.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:07 smithi016 ceph-mon[28969]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:39:08.428 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-04-02T04:39:06.760441+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":202888,"kb_used_data":3876,"kb_used_omap":9,"kb_used_meta":198902,"kb_avail":562268024,"statfs":{"total":575970213888,"available":575762456576,"internally_reserved":0,"allocated":3969024,"data_stored":2606850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":203676326},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001937"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":186,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-02T04:38:16.981168+0000","last_change":"2024-04-02T04:37:49.357662+0000","last_active":"2024-04-02T04:38:16.981168+0000","last_peered":"2024-04-02T04:38:16.981168+0000","last_clean":"2024-04-02T04:38:16.981168+0000","last_became_active":"2024-04-02T04:37:49.357233+0000","last_became_peered":"2024-04-02T04:37:49.357233+0000","last_unstale":"2024-04-02T04:38:16.981168+0000","last_undegraded":"2024-04-02T04:38:16.981168+0000","last_fullsized":"2024-04-02T04:38:16.981168+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T04:37:01.692885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T04:37:01.692885+0000","last_clean_scrub_stamp":"2024-04-02T04:37:01.692885+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T11:22:59.386529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724555,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27376,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717776,"statfs":{"total":95995035648,"available":95967002624,"internally_reserved":0,"allocated":364544,"data_stored":139291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]}]},{"osd":4,"up_from":34,"seq":146028888082,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":31476,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713676,"statfs":{"total":95995035648,"available":95962804224,"internally_reserved":0,"allocated":364544,"data_stored":139291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Apr 2 04:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.435},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.52500000000000002}]},{"osd":1,"last update":"Tue Apr 2 04:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.60499999999999998}]},{"osd":2,"last update":"Tue Apr 2 04:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.58599999999999997}]},{"osd":3,"last update":"Tue Apr 2 04:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.495}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149720,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":729659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Apr 2 04:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.47999999999999998}]},{"osd":1,"last update":"Tue Apr 2 04:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.443}]},{"osd":2,"last update":"Tue Apr 2 04:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.55700000000000005}]},{"osd":4,"last update":"Tue Apr 2 04:38:49 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.46600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]}]},{"osd":2,"up_from":19,"seq":81604378653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":35576,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709576,"statfs":{"total":95995035648,"available":95958605824,"internally_reserved":0,"allocated":364544,"data_stored":139291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Apr 2 04:39:03 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.34699999999999998,"15min":0.34899999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35799999999999998,"15min":0.35899999999999999},"min":{"1min":0.26300000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.29399999999999998}]},{"osd":1,"last update":"Tue Apr 2 04:39:03 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.39100000000000001,"15min":0.39600000000000002},"min":{"1min":0.254,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.55600000000000005,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.36699999999999999,"15min":0.36899999999999999},"min":{"1min":0.27700000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.311}]},{"osd":3,"last update":"Tue Apr 2 04:38:23 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.45900000000000002}]},{"osd":4,"last update":"Tue Apr 2 04:38:51 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":3.9889999999999999,"5min":3.9889999999999999,"15min":3.9889999999999999},"last":0.436}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]}]},{"osd":0,"up_from":9,"seq":38654705704,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40252,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":93704900,"statfs":{"total":95995035648,"available":95953817600,"internally_reserved":0,"allocated":958464,"data_stored":729659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Apr 2 04:38:36 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.32600000000000001,"15min":0.32300000000000001},"min":{"1min":0.23799999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.45500000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.34000000000000002,"15min":0.33500000000000002},"min":{"1min":0.255,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.52300000000000002,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.23400000000000001}]},{"osd":2,"last update":"Tue Apr 2 04:39:02 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.375,"15min":0.373},"min":{"1min":0.252,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.39700000000000002,"15min":0.40200000000000002},"min":{"1min":0.27000000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.46999999999999997,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.27000000000000002}]},{"osd":3,"last update":"Tue Apr 2 04:38:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.437},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.42099999999999999}]},{"osd":4,"last update":"Tue Apr 2 04:38:51 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.50700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40252,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":93704900,"statfs":{"total":95995035648,"available":95953817600,"internally_reserved":0,"allocated":958464,"data_stored":729659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Apr 2 04:38:33 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.317,"15min":0.314},"min":{"1min":0.17999999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.33300000000000002,"15min":0.32700000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.502,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.379}]},{"osd":2,"last update":"Tue Apr 2 04:39:04 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.38100000000000001,"15min":0.38400000000000001},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.40600000000000003}]},{"osd":3,"last update":"Tue Apr 2 04:38:27 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.624},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.63700000000000001}]},{"osd":4,"last update":"Tue Apr 2 04:38:51 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.66400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T04:39:08.429 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-02T04:39:08.429 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-02T04:39:08.429 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-02T04:39:08.429 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph health --format=json 2024-04-02T04:39:08.637 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:39:09.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:08 smithi016 ceph-mon[28969]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T04:39:09.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:08 smithi027 ceph-mon[37550]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T04:39:09.967 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:09 smithi016 ceph-mon[28969]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:39:10.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:09 smithi027 ceph-mon[37550]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:39:10.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.175+0000 7fd4b2997700 1 -- 172.21.15.16:0/3609360116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4ac105950 msgr2=0x7fd4ac105d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:10.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.175+0000 7fd4b2997700 1 --2- 172.21.15.16:0/3609360116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4ac105950 0x7fd4ac105d30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd49c009910 tx=0x7fd49c009c20 comp rx=0 tx=0).stop 2024-04-02T04:39:10.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.175+0000 7fd4b2997700 1 -- 172.21.15.16:0/3609360116 shutdown_connections 2024-04-02T04:39:10.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.175+0000 7fd4b2997700 1 --2- 172.21.15.16:0/3609360116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4ac0ff950 0x7fd4ac0ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:10.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.175+0000 7fd4b2997700 1 --2- 172.21.15.16:0/3609360116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4ac105950 0x7fd4ac105d30 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:10.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.175+0000 7fd4b2997700 1 -- 172.21.15.16:0/3609360116 >> 172.21.15.16:0/3609360116 conn(0x7fd4ac076640 msgr2=0x7fd4ac0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:10.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.175+0000 7fd4b2997700 1 -- 172.21.15.16:0/3609360116 shutdown_connections 2024-04-02T04:39:10.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.175+0000 7fd4b2997700 1 -- 172.21.15.16:0/3609360116 wait complete. 2024-04-02T04:39:10.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.176+0000 7fd4b2997700 1 Processor -- start 2024-04-02T04:39:10.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.176+0000 7fd4b2997700 1 -- start start 2024-04-02T04:39:10.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.176+0000 7fd4b2997700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4ac0ff950 0x7fd4ac19c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:10.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.176+0000 7fd4b2997700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4ac105950 0x7fd4ac19cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:10.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.176+0000 7fd4b2997700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac19d230 con 0x7fd4ac0ff950 2024-04-02T04:39:10.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.176+0000 7fd4b2997700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac196770 con 0x7fd4ac105950 2024-04-02T04:39:10.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.176+0000 7fd4abfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4ac0ff950 0x7fd4ac19c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:10.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.176+0000 7fd4abfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4ac0ff950 0x7fd4ac19c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55212/0 (socket says 172.21.15.16:55212) 2024-04-02T04:39:10.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.176+0000 7fd4abfff700 1 -- 172.21.15.16:0/2268653179 learned_addr learned my addr 172.21.15.16:0/2268653179 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:39:10.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.176+0000 7fd4ab7fe700 1 --2- 172.21.15.16:0/2268653179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4ac105950 0x7fd4ac19cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:10.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.176+0000 7fd4abfff700 1 -- 172.21.15.16:0/2268653179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4ac105950 msgr2=0x7fd4ac19cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:10.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.176+0000 7fd4abfff700 1 --2- 172.21.15.16:0/2268653179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4ac105950 0x7fd4ac19cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:10.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.176+0000 7fd4abfff700 1 -- 172.21.15.16:0/2268653179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd49c0095f0 con 0x7fd4ac0ff950 2024-04-02T04:39:10.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.177+0000 7fd4abfff700 1 --2- 172.21.15.16:0/2268653179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4ac0ff950 0x7fd4ac19c610 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd49c00bd70 tx=0x7fd49c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:10.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.177+0000 7fd4a97fa700 1 -- 172.21.15.16:0/2268653179 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd49c01d070 con 0x7fd4ac0ff950 2024-04-02T04:39:10.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.177+0000 7fd4a97fa700 1 -- 172.21.15.16:0/2268653179 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd49c0056e0 con 0x7fd4ac0ff950 2024-04-02T04:39:10.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.177+0000 7fd4b2997700 1 -- 172.21.15.16:0/2268653179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4ac1969f0 con 0x7fd4ac0ff950 2024-04-02T04:39:10.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.177+0000 7fd4a97fa700 1 -- 172.21.15.16:0/2268653179 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd49c00fe40 con 0x7fd4ac0ff950 2024-04-02T04:39:10.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.177+0000 7fd4b2997700 1 -- 172.21.15.16:0/2268653179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4ac196ee0 con 0x7fd4ac0ff950 2024-04-02T04:39:10.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.178+0000 7fd4b2997700 1 -- 172.21.15.16:0/2268653179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4ac101090 con 0x7fd4ac0ff950 2024-04-02T04:39:10.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.178+0000 7fd4a97fa700 1 -- 172.21.15.16:0/2268653179 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fd49c003a60 con 0x7fd4ac0ff950 2024-04-02T04:39:10.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.178+0000 7fd4a97fa700 1 --2- 172.21.15.16:0/2268653179 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fd49806cc80 0x7fd49806f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:10.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.179+0000 7fd4a97fa700 1 -- 172.21.15.16:0/2268653179 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fd49c08d340 con 0x7fd4ac0ff950 2024-04-02T04:39:10.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.179+0000 7fd4ab7fe700 1 --2- 172.21.15.16:0/2268653179 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fd49806cc80 0x7fd49806f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:10.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.179+0000 7fd4ab7fe700 1 --2- 172.21.15.16:0/2268653179 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fd49806cc80 0x7fd49806f140 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac197ea0 tx=0x7fd49400b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:10.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.183+0000 7fd4a97fa700 1 -- 172.21.15.16:0/2268653179 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd49c05b480 con 0x7fd4ac0ff950 2024-04-02T04:39:10.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.430+0000 7fd4b2997700 1 -- 172.21.15.16:0/2268653179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd4ac197c50 con 0x7fd4ac0ff950 2024-04-02T04:39:10.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.431+0000 7fd4a97fa700 1 -- 172.21.15.16:0/2268653179 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fd49c05b010 con 0x7fd4ac0ff950 2024-04-02T04:39:10.434 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:39:10.434 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-02T04:39:10.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.433+0000 7fd4b2997700 1 -- 172.21.15.16:0/2268653179 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fd49806cc80 msgr2=0x7fd49806f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:10.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.433+0000 7fd4b2997700 1 --2- 172.21.15.16:0/2268653179 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fd49806cc80 0x7fd49806f140 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac197ea0 tx=0x7fd49400b410 comp rx=0 tx=0).stop 2024-04-02T04:39:10.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.433+0000 7fd4b2997700 1 -- 172.21.15.16:0/2268653179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4ac0ff950 msgr2=0x7fd4ac19c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:10.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.433+0000 7fd4b2997700 1 --2- 172.21.15.16:0/2268653179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4ac0ff950 0x7fd4ac19c610 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd49c00bd70 tx=0x7fd49c00bda0 comp rx=0 tx=0).stop 2024-04-02T04:39:10.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.433+0000 7fd4b2997700 1 -- 172.21.15.16:0/2268653179 shutdown_connections 2024-04-02T04:39:10.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.433+0000 7fd4b2997700 1 --2- 172.21.15.16:0/2268653179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4ac0ff950 0x7fd4ac19c610 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:10.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.433+0000 7fd4b2997700 1 --2- 172.21.15.16:0/2268653179 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fd49806cc80 0x7fd49806f140 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:10.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.433+0000 7fd4b2997700 1 --2- 172.21.15.16:0/2268653179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4ac105950 0x7fd4ac19cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:10.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.433+0000 7fd4b2997700 1 -- 172.21.15.16:0/2268653179 >> 172.21.15.16:0/2268653179 conn(0x7fd4ac076640 msgr2=0x7fd4ac0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:10.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.433+0000 7fd4b2997700 1 -- 172.21.15.16:0/2268653179 shutdown_connections 2024-04-02T04:39:10.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:10.433+0000 7fd4b2997700 1 -- 172.21.15.16:0/2268653179 wait complete. 2024-04-02T04:39:10.993 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:10 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/2268653179' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-02T04:39:11.016 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-02T04:39:11.016 INFO:tasks.cephadm:Setup complete, yielding 2024-04-02T04:39:11.017 INFO:teuthology.run_tasks:Running task print... 2024-04-02T04:39:11.026 INFO:teuthology.task.print:**** done end installing reef cephadm ... 2024-04-02T04:39:11.026 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-02T04:39:11.063 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi016.front.sepia.ceph.com 2024-04-02T04:39:11.064 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-04-02T04:39:11.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:10 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/2268653179' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-02T04:39:11.275 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:39:12.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:11 smithi016 ceph-mon[28969]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:39:12.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:11 smithi027 ceph-mon[37550]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:39:12.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:12.810+0000 7faad9ff3700 1 -- 172.21.15.16:0/2098261315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faad4100ad0 msgr2=0x7faad4100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:12.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:12.810+0000 7faad9ff3700 1 --2- 172.21.15.16:0/2098261315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faad4100ad0 0x7faad4100ef0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7faac400b0a0 tx=0x7faac400b3b0 comp rx=0 tx=0).stop 2024-04-02T04:39:12.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:12.810+0000 7faad9ff3700 1 -- 172.21.15.16:0/2098261315 shutdown_connections 2024-04-02T04:39:12.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:12.810+0000 7faad9ff3700 1 --2- 172.21.15.16:0/2098261315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faad4101cd0 0x7faad4102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:12.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:12.810+0000 7faad9ff3700 1 --2- 172.21.15.16:0/2098261315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faad4100ad0 0x7faad4100ef0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:12.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:12.810+0000 7faad9ff3700 1 -- 172.21.15.16:0/2098261315 >> 172.21.15.16:0/2098261315 conn(0x7faad40fc090 msgr2=0x7faad40fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:12.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:12.810+0000 7faad9ff3700 1 -- 172.21.15.16:0/2098261315 shutdown_connections 2024-04-02T04:39:12.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:12.810+0000 7faad9ff3700 1 -- 172.21.15.16:0/2098261315 wait complete. 2024-04-02T04:39:12.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:12.810+0000 7faad9ff3700 1 Processor -- start 2024-04-02T04:39:12.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:12.811+0000 7faad9ff3700 1 -- start start 2024-04-02T04:39:12.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:12.811+0000 7faad9ff3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faad4100ad0 0x7faad4196530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:12.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:12.811+0000 7faad9ff3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faad4101cd0 0x7faad4196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:12.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:12.811+0000 7faad9ff3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad4197090 con 0x7faad4101cd0 2024-04-02T04:39:12.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:12.811+0000 7faad9ff3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad41971d0 con 0x7faad4100ad0 2024-04-02T04:39:12.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:12.811+0000 7faad2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faad4101cd0 0x7faad4196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:12.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:12.811+0000 7faad2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faad4101cd0 0x7faad4196a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42486/0 (socket says 172.21.15.16:42486) 2024-04-02T04:39:12.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:12.811+0000 7faad2ffd700 1 -- 172.21.15.16:0/3561151870 learned_addr learned my addr 172.21.15.16:0/3561151870 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:39:12.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:12.811+0000 7faad37fe700 1 --2- 172.21.15.16:0/3561151870 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faad4100ad0 0x7faad4196530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:12.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:12.811+0000 7faad2ffd700 1 -- 172.21.15.16:0/3561151870 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faad4100ad0 msgr2=0x7faad4196530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:12.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:12.811+0000 7faad2ffd700 1 --2- 172.21.15.16:0/3561151870 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faad4100ad0 0x7faad4196530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:12.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:12.811+0000 7faad2ffd700 1 -- 172.21.15.16:0/3561151870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac4009d00 con 0x7faad4101cd0 2024-04-02T04:39:12.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:12.811+0000 7faad2ffd700 1 --2- 172.21.15.16:0/3561151870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faad4101cd0 0x7faad4196a70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7faac800d8d0 tx=0x7faac800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:12.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:12.812+0000 7faad0ff9700 1 -- 172.21.15.16:0/3561151870 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faac80098a0 con 0x7faad4101cd0 2024-04-02T04:39:12.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:12.812+0000 7faad9ff3700 1 -- 172.21.15.16:0/3561151870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faad419bbe0 con 0x7faad4101cd0 2024-04-02T04:39:12.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:12.812+0000 7faad0ff9700 1 -- 172.21.15.16:0/3561151870 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faac8010460 con 0x7faad4101cd0 2024-04-02T04:39:12.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:12.812+0000 7faad0ff9700 1 -- 172.21.15.16:0/3561151870 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faac800f5f0 con 0x7faad4101cd0 2024-04-02T04:39:12.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:12.812+0000 7faad9ff3700 1 -- 172.21.15.16:0/3561151870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faad419c1b0 con 0x7faad4101cd0 2024-04-02T04:39:12.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:12.813+0000 7faad9ff3700 1 -- 172.21.15.16:0/3561151870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faad4190650 con 0x7faad4101cd0 2024-04-02T04:39:12.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:12.813+0000 7faad0ff9700 1 -- 172.21.15.16:0/3561151870 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7faac8009a00 con 0x7faad4101cd0 2024-04-02T04:39:12.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:12.814+0000 7faad0ff9700 1 --2- 172.21.15.16:0/3561151870 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7faabc06cce0 0x7faabc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:12.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:12.814+0000 7faad0ff9700 1 -- 172.21.15.16:0/3561151870 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7faac808bda0 con 0x7faad4101cd0 2024-04-02T04:39:12.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:12.814+0000 7faad37fe700 1 --2- 172.21.15.16:0/3561151870 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7faabc06cce0 0x7faabc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:12.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:12.814+0000 7faad37fe700 1 --2- 172.21.15.16:0/3561151870 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7faabc06cce0 0x7faabc06f1a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faac4009fd0 tx=0x7faac4006000 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:12.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:12.818+0000 7faad0ff9700 1 -- 172.21.15.16:0/3561151870 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7faac8059f40 con 0x7faad4101cd0 2024-04-02T04:39:13.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:13.020+0000 7faad9ff3700 1 -- 172.21.15.16:0/3561151870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7faad402cd50 con 0x7faad4101cd0 2024-04-02T04:39:13.024 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:12 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:39:13.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:13.025+0000 7faad0ff9700 1 -- 172.21.15.16:0/3561151870 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v24)=0 v24) v1 ==== 143+0+0 (secure 0 0 0) 0x7faac8059ad0 con 0x7faad4101cd0 2024-04-02T04:39:13.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:13.027+0000 7faad9ff3700 1 -- 172.21.15.16:0/3561151870 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7faabc06cce0 msgr2=0x7faabc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:13.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:13.027+0000 7faad9ff3700 1 --2- 172.21.15.16:0/3561151870 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7faabc06cce0 0x7faabc06f1a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faac4009fd0 tx=0x7faac4006000 comp rx=0 tx=0).stop 2024-04-02T04:39:13.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:13.027+0000 7faad9ff3700 1 -- 172.21.15.16:0/3561151870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faad4101cd0 msgr2=0x7faad4196a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:13.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:13.027+0000 7faad9ff3700 1 --2- 172.21.15.16:0/3561151870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faad4101cd0 0x7faad4196a70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7faac800d8d0 tx=0x7faac800dbe0 comp rx=0 tx=0).stop 2024-04-02T04:39:13.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:13.027+0000 7faad9ff3700 1 -- 172.21.15.16:0/3561151870 shutdown_connections 2024-04-02T04:39:13.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:13.027+0000 7faad9ff3700 1 --2- 172.21.15.16:0/3561151870 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7faabc06cce0 0x7faabc06f1a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:13.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:13.027+0000 7faad9ff3700 1 --2- 172.21.15.16:0/3561151870 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faad4100ad0 0x7faad4196530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:13.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:13.027+0000 7faad9ff3700 1 --2- 172.21.15.16:0/3561151870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faad4101cd0 0x7faad4196a70 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:13.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:13.027+0000 7faad9ff3700 1 -- 172.21.15.16:0/3561151870 >> 172.21.15.16:0/3561151870 conn(0x7faad40fc090 msgr2=0x7faad41052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:13.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:13.027+0000 7faad9ff3700 1 -- 172.21.15.16:0/3561151870 shutdown_connections 2024-04-02T04:39:13.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:13.028+0000 7faad9ff3700 1 -- 172.21.15.16:0/3561151870 wait complete. 2024-04-02T04:39:13.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:12 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:39:13.629 INFO:teuthology.run_tasks:Running task print... 2024-04-02T04:39:13.637 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-04-02T04:39:13.637 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-02T04:39:13.679 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi016.front.sepia.ceph.com 2024-04-02T04:39:13.680 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch status' 2024-04-02T04:39:13.888 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:39:14.010 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:13 smithi016 ceph-mon[28969]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:39:14.011 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:13 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/3561151870' entity='client.admin' 2024-04-02T04:39:14.011 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:13 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:39:14.011 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:13 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:39:14.011 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:13 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:39:14.011 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:13 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:14.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:13 smithi027 ceph-mon[37550]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:39:14.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:13 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/3561151870' entity='client.admin' 2024-04-02T04:39:14.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:13 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:39:14.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:13 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:39:14.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:13 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:39:14.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:13 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:15.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.596+0000 7f471cb38700 1 -- 172.21.15.16:0/1781335797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4718101ce0 msgr2=0x7f4718102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:15.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.596+0000 7f471cb38700 1 --2- 172.21.15.16:0/1781335797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4718101ce0 0x7f4718102140 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f470c009910 tx=0x7f470c009c20 comp rx=0 tx=0).stop 2024-04-02T04:39:15.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.596+0000 7f471cb38700 1 -- 172.21.15.16:0/1781335797 shutdown_connections 2024-04-02T04:39:15.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.596+0000 7f471cb38700 1 --2- 172.21.15.16:0/1781335797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4718101ce0 0x7f4718102140 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:15.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.596+0000 7f471cb38700 1 --2- 172.21.15.16:0/1781335797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4718100ae0 0x7f4718100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:15.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.596+0000 7f471cb38700 1 -- 172.21.15.16:0/1781335797 >> 172.21.15.16:0/1781335797 conn(0x7f47180fc0a0 msgr2=0x7f47180fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:15.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.596+0000 7f471cb38700 1 -- 172.21.15.16:0/1781335797 shutdown_connections 2024-04-02T04:39:15.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.596+0000 7f471cb38700 1 -- 172.21.15.16:0/1781335797 wait complete. 2024-04-02T04:39:15.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.597+0000 7f471cb38700 1 Processor -- start 2024-04-02T04:39:15.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.597+0000 7f471cb38700 1 -- start start 2024-04-02T04:39:15.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.597+0000 7f471cb38700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4718100ae0 0x7f47181964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:15.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.597+0000 7f471cb38700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4718101ce0 0x7f4718196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:15.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.597+0000 7f471cb38700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4718197020 con 0x7f4718100ae0 2024-04-02T04:39:15.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.597+0000 7f471cb38700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4718197160 con 0x7f4718101ce0 2024-04-02T04:39:15.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.597+0000 7f471659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4718100ae0 0x7f47181964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:15.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.597+0000 7f471659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4718100ae0 0x7f47181964c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42504/0 (socket says 172.21.15.16:42504) 2024-04-02T04:39:15.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.597+0000 7f471659c700 1 -- 172.21.15.16:0/1266324848 learned_addr learned my addr 172.21.15.16:0/1266324848 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:39:15.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.597+0000 7f4715d9b700 1 --2- 172.21.15.16:0/1266324848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4718101ce0 0x7f4718196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:15.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.597+0000 7f471659c700 1 -- 172.21.15.16:0/1266324848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4718101ce0 msgr2=0x7f4718196a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:15.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.597+0000 7f471659c700 1 --2- 172.21.15.16:0/1266324848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4718101ce0 0x7f4718196a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:15.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.597+0000 7f471659c700 1 -- 172.21.15.16:0/1266324848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f470c0095f0 con 0x7f4718100ae0 2024-04-02T04:39:15.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.597+0000 7f471659c700 1 --2- 172.21.15.16:0/1266324848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4718100ae0 0x7f47181964c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f470800c8f0 tx=0x7f470800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:15.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.598+0000 7f47077fe700 1 -- 172.21.15.16:0/1266324848 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47080078e0 con 0x7f4718100ae0 2024-04-02T04:39:15.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.598+0000 7f471cb38700 1 -- 172.21.15.16:0/1266324848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f471819bbc0 con 0x7f4718100ae0 2024-04-02T04:39:15.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.598+0000 7f47077fe700 1 -- 172.21.15.16:0/1266324848 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f470800f450 con 0x7f4718100ae0 2024-04-02T04:39:15.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.598+0000 7f47077fe700 1 -- 172.21.15.16:0/1266324848 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4708018610 con 0x7f4718100ae0 2024-04-02T04:39:15.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.598+0000 7f471cb38700 1 -- 172.21.15.16:0/1266324848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f471819c110 con 0x7f4718100ae0 2024-04-02T04:39:15.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.599+0000 7f471cb38700 1 -- 172.21.15.16:0/1266324848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4718190660 con 0x7f4718100ae0 2024-04-02T04:39:15.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.599+0000 7f47077fe700 1 -- 172.21.15.16:0/1266324848 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f470800f5c0 con 0x7f4718100ae0 2024-04-02T04:39:15.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.600+0000 7f47077fe700 1 --2- 172.21.15.16:0/1266324848 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f470006ccf0 0x7f470006f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:15.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.600+0000 7f4715d9b700 1 --2- 172.21.15.16:0/1266324848 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f470006ccf0 0x7f470006f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:15.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.600+0000 7f47077fe700 1 -- 172.21.15.16:0/1266324848 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f470808be10 con 0x7f4718100ae0 2024-04-02T04:39:15.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.600+0000 7f4715d9b700 1 --2- 172.21.15.16:0/1266324848 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f470006ccf0 0x7f470006f1b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f470c006010 tx=0x7f470c00b540 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:15.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.604+0000 7f47077fe700 1 -- 172.21.15.16:0/1266324848 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4708059fb0 con 0x7f4718100ae0 2024-04-02T04:39:15.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.822+0000 7f471cb38700 1 -- 172.21.15.16:0/1266324848 --> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f471802cdc0 con 0x7f470006ccf0 2024-04-02T04:39:15.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.823+0000 7f47077fe700 1 -- 172.21.15.16:0/1266324848 <== mgr.14217 v2:172.21.15.16:6800/3643600900 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f471802cdc0 con 0x7f470006ccf0 2024-04-02T04:39:15.826 INFO:teuthology.orchestra.run.smithi016.stdout:Backend: cephadm 2024-04-02T04:39:15.826 INFO:teuthology.orchestra.run.smithi016.stdout:Available: Yes 2024-04-02T04:39:15.826 INFO:teuthology.orchestra.run.smithi016.stdout:Paused: No 2024-04-02T04:39:15.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.825+0000 7f471cb38700 1 -- 172.21.15.16:0/1266324848 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f470006ccf0 msgr2=0x7f470006f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:15.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.825+0000 7f471cb38700 1 --2- 172.21.15.16:0/1266324848 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f470006ccf0 0x7f470006f1b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f470c006010 tx=0x7f470c00b540 comp rx=0 tx=0).stop 2024-04-02T04:39:15.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.825+0000 7f471cb38700 1 -- 172.21.15.16:0/1266324848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4718100ae0 msgr2=0x7f47181964c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:15.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.825+0000 7f471cb38700 1 --2- 172.21.15.16:0/1266324848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4718100ae0 0x7f47181964c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f470800c8f0 tx=0x7f470800cc00 comp rx=0 tx=0).stop 2024-04-02T04:39:15.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.825+0000 7f471cb38700 1 -- 172.21.15.16:0/1266324848 shutdown_connections 2024-04-02T04:39:15.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.825+0000 7f471cb38700 1 --2- 172.21.15.16:0/1266324848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4718100ae0 0x7f47181964c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:15.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.825+0000 7f471cb38700 1 --2- 172.21.15.16:0/1266324848 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f470006ccf0 0x7f470006f1b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:15.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.825+0000 7f471cb38700 1 --2- 172.21.15.16:0/1266324848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4718101ce0 0x7f4718196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:15.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.825+0000 7f471cb38700 1 -- 172.21.15.16:0/1266324848 >> 172.21.15.16:0/1266324848 conn(0x7f47180fc0a0 msgr2=0x7f4718105280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:15.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.825+0000 7f471cb38700 1 -- 172.21.15.16:0/1266324848 shutdown_connections 2024-04-02T04:39:15.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:15.826+0000 7f471cb38700 1 -- 172.21.15.16:0/1266324848 wait complete. 2024-04-02T04:39:16.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:15 smithi016 ceph-mon[28969]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:39:16.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:15 smithi027 ceph-mon[37550]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:39:16.488 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch ps' 2024-04-02T04:39:16.699 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:39:17.083 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:16 smithi016 ceph-mon[28969]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:39:17.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:16 smithi027 ceph-mon[37550]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:39:18.127 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:17 smithi016 ceph-mon[28969]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:39:18.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:17 smithi027 ceph-mon[37550]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:39:18.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.340+0000 7fdefe9a5700 1 -- 172.21.15.16:0/1351716252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdef8100ad0 msgr2=0x7fdef8100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:18.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.340+0000 7fdefe9a5700 1 --2- 172.21.15.16:0/1351716252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdef8100ad0 0x7fdef8100ef0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdee8009910 tx=0x7fdee8009c20 comp rx=0 tx=0).stop 2024-04-02T04:39:18.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.340+0000 7fdefe9a5700 1 -- 172.21.15.16:0/1351716252 shutdown_connections 2024-04-02T04:39:18.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.340+0000 7fdefe9a5700 1 --2- 172.21.15.16:0/1351716252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdef8101cd0 0x7fdef8102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:18.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.340+0000 7fdefe9a5700 1 --2- 172.21.15.16:0/1351716252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdef8100ad0 0x7fdef8100ef0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:18.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.340+0000 7fdefe9a5700 1 -- 172.21.15.16:0/1351716252 >> 172.21.15.16:0/1351716252 conn(0x7fdef80fc090 msgr2=0x7fdef80fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:18.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.340+0000 7fdefe9a5700 1 -- 172.21.15.16:0/1351716252 shutdown_connections 2024-04-02T04:39:18.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.340+0000 7fdefe9a5700 1 -- 172.21.15.16:0/1351716252 wait complete. 2024-04-02T04:39:18.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.341+0000 7fdefe9a5700 1 Processor -- start 2024-04-02T04:39:18.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.341+0000 7fdefe9a5700 1 -- start start 2024-04-02T04:39:18.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.341+0000 7fdefe9a5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdef8100ad0 0x7fdef8074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:18.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.341+0000 7fdefe9a5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdef8101cd0 0x7fdef80731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:18.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.341+0000 7fdefe9a5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef80736e0 con 0x7fdef8100ad0 2024-04-02T04:39:18.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.341+0000 7fdefe9a5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef8073820 con 0x7fdef8101cd0 2024-04-02T04:39:18.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.341+0000 7fdef7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdef8100ad0 0x7fdef8074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:18.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.341+0000 7fdef7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdef8100ad0 0x7fdef8074b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42518/0 (socket says 172.21.15.16:42518) 2024-04-02T04:39:18.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.341+0000 7fdef7fff700 1 -- 172.21.15.16:0/1711682308 learned_addr learned my addr 172.21.15.16:0/1711682308 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:39:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.341+0000 7fdef77fe700 1 --2- 172.21.15.16:0/1711682308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdef8101cd0 0x7fdef80731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.342+0000 7fdef7fff700 1 -- 172.21.15.16:0/1711682308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdef8101cd0 msgr2=0x7fdef80731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.342+0000 7fdef7fff700 1 --2- 172.21.15.16:0/1711682308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdef8101cd0 0x7fdef80731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.342+0000 7fdef7fff700 1 -- 172.21.15.16:0/1711682308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee80095f0 con 0x7fdef8100ad0 2024-04-02T04:39:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.342+0000 7fdef7fff700 1 --2- 172.21.15.16:0/1711682308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdef8100ad0 0x7fdef8074b00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdee800bd70 tx=0x7fdee800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.342+0000 7fdef57fa700 1 -- 172.21.15.16:0/1711682308 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdee801d070 con 0x7fdef8100ad0 2024-04-02T04:39:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.342+0000 7fdefe9a5700 1 -- 172.21.15.16:0/1711682308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdef8073aa0 con 0x7fdef8100ad0 2024-04-02T04:39:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.342+0000 7fdef57fa700 1 -- 172.21.15.16:0/1711682308 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdee8005dc0 con 0x7fdef8100ad0 2024-04-02T04:39:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.342+0000 7fdef57fa700 1 -- 172.21.15.16:0/1711682308 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdee8021410 con 0x7fdef8100ad0 2024-04-02T04:39:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.342+0000 7fdefe9a5700 1 -- 172.21.15.16:0/1711682308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef8074010 con 0x7fdef8100ad0 2024-04-02T04:39:18.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.343+0000 7fdefe9a5700 1 -- 172.21.15.16:0/1711682308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdef8190650 con 0x7fdef8100ad0 2024-04-02T04:39:18.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.344+0000 7fdef57fa700 1 -- 172.21.15.16:0/1711682308 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fdee800f7f0 con 0x7fdef8100ad0 2024-04-02T04:39:18.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.344+0000 7fdef57fa700 1 --2- 172.21.15.16:0/1711682308 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fdee406ccf0 0x7fdee406f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:18.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.344+0000 7fdef57fa700 1 -- 172.21.15.16:0/1711682308 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fdee808ce10 con 0x7fdef8100ad0 2024-04-02T04:39:18.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.344+0000 7fdef77fe700 1 --2- 172.21.15.16:0/1711682308 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fdee406ccf0 0x7fdee406f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:18.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.344+0000 7fdef77fe700 1 --2- 172.21.15.16:0/1711682308 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fdee406ccf0 0x7fdee406f1b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdee0005950 tx=0x7fdee000b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:18.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.349+0000 7fdef57fa700 1 -- 172.21.15.16:0/1711682308 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fdee805b030 con 0x7fdef8100ad0 2024-04-02T04:39:18.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.569+0000 7fdefe9a5700 1 -- 172.21.15.16:0/1711682308 --> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdef8074010 con 0x7fdee406ccf0 2024-04-02T04:39:18.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.578+0000 7fdef57fa700 1 -- 172.21.15.16:0/1711682308 <== mgr.14217 v2:172.21.15.16:6800/3643600900 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3100 (secure 0 0 0) 0x7fdef8074010 con 0x7fdee406ccf0 2024-04-02T04:39:18.581 INFO:teuthology.orchestra.run.smithi016.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:39:18.581 INFO:teuthology.orchestra.run.smithi016.stdout:alertmanager.smithi016 smithi016 *:9093,9094 running (3m) 2m ago 5m 25.6M - 0.25.0 c8568f914cd2 b95d8354b016 2024-04-02T04:39:18.581 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi016 smithi016 running (5m) 2m ago 5m 6995k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 9de6c16a487f 2024-04-02T04:39:18.581 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi027 smithi027 running (4m) 63s ago 4m 6971k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 99956592471a 2024-04-02T04:39:18.581 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi016 smithi016 running (5m) 2m ago 5m 6656k - 18.2.2-1059-g4d05f4ca 909dce4fe53d bd73919b0711 2024-04-02T04:39:18.581 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi027 smithi027 running (4m) 63s ago 4m 6660k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 7a54c37f202e 2024-04-02T04:39:18.582 INFO:teuthology.orchestra.run.smithi016.stdout:grafana.smithi016 smithi016 *:3000 running (3m) 2m ago 5m 80.5M - 9.4.7 954c08fa6188 7c43d133569c 2024-04-02T04:39:18.582 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi016.eqnebx smithi016 *:9283,8765,8443 running (6m) 2m ago 6m 487M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 7d7ad6d79d01 2024-04-02T04:39:18.582 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi027.kjzcsc smithi027 *:8443,9283,8765 running (4m) 63s ago 4m 450M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 1ced128585bc 2024-04-02T04:39:18.582 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi016 smithi016 running (6m) 2m ago 6m 44.4M 2048M 18.2.2-1059-g4d05f4ca 909dce4fe53d 24d5820dc994 2024-04-02T04:39:18.582 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi027 smithi027 running (4m) 63s ago 4m 40.9M 2048M 18.2.2-1059-g4d05f4ca 909dce4fe53d b77563184a1e 2024-04-02T04:39:18.582 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi016 smithi016 *:9100 running (5m) 2m ago 5m 13.7M - 1.5.0 0da6a335fe13 09f54c65b53c 2024-04-02T04:39:18.582 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi027 smithi027 *:9100 running (4m) 63s ago 4m 15.1M - 1.5.0 0da6a335fe13 e41a657c7443 2024-04-02T04:39:18.582 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 smithi016 running (3m) 2m ago 3m 47.5M 3948M 18.2.2-1059-g4d05f4ca 909dce4fe53d 3f40108debd6 2024-04-02T04:39:18.582 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 smithi016 running (2m) 2m ago 2m 44.2M 3948M 18.2.2-1059-g4d05f4ca 909dce4fe53d 7361abc9e1b6 2024-04-02T04:39:18.582 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 smithi016 running (2m) 2m ago 2m 43.4M 3948M 18.2.2-1059-g4d05f4ca 909dce4fe53d 52e592b4d1f2 2024-04-02T04:39:18.582 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 smithi027 running (2m) 63s ago 2m 43.9M 4970M 18.2.2-1059-g4d05f4ca 909dce4fe53d cf7c7fe022f0 2024-04-02T04:39:18.582 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 smithi027 running (96s) 63s ago 95s 44.9M 4970M 18.2.2-1059-g4d05f4ca 909dce4fe53d 8cbbedf8cac9 2024-04-02T04:39:18.582 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 smithi027 running (68s) 63s ago 68s 13.8M 4970M 18.2.2-1059-g4d05f4ca 909dce4fe53d ac6094415529 2024-04-02T04:39:18.582 INFO:teuthology.orchestra.run.smithi016.stdout:prometheus.smithi016 smithi016 *:9095 running (3m) 2m ago 4m 40.2M - 2.43.0 a07b618ecd1d 232d70bae1ba 2024-04-02T04:39:18.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.580+0000 7fdefe9a5700 1 -- 172.21.15.16:0/1711682308 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fdee406ccf0 msgr2=0x7fdee406f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:18.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.580+0000 7fdefe9a5700 1 --2- 172.21.15.16:0/1711682308 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fdee406ccf0 0x7fdee406f1b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdee0005950 tx=0x7fdee000b410 comp rx=0 tx=0).stop 2024-04-02T04:39:18.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.580+0000 7fdefe9a5700 1 -- 172.21.15.16:0/1711682308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdef8100ad0 msgr2=0x7fdef8074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:18.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.580+0000 7fdefe9a5700 1 --2- 172.21.15.16:0/1711682308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdef8100ad0 0x7fdef8074b00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdee800bd70 tx=0x7fdee800bda0 comp rx=0 tx=0).stop 2024-04-02T04:39:18.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.580+0000 7fdefe9a5700 1 -- 172.21.15.16:0/1711682308 shutdown_connections 2024-04-02T04:39:18.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.580+0000 7fdefe9a5700 1 --2- 172.21.15.16:0/1711682308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdef8100ad0 0x7fdef8074b00 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:18.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.580+0000 7fdefe9a5700 1 --2- 172.21.15.16:0/1711682308 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fdee406ccf0 0x7fdee406f1b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:18.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.580+0000 7fdefe9a5700 1 --2- 172.21.15.16:0/1711682308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdef8101cd0 0x7fdef80731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:18.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.580+0000 7fdefe9a5700 1 -- 172.21.15.16:0/1711682308 >> 172.21.15.16:0/1711682308 conn(0x7fdef80fc090 msgr2=0x7fdef81052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:18.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.580+0000 7fdefe9a5700 1 -- 172.21.15.16:0/1711682308 shutdown_connections 2024-04-02T04:39:18.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:18.580+0000 7fdefe9a5700 1 -- 172.21.15.16:0/1711682308 wait complete. 2024-04-02T04:39:19.232 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch ls' 2024-04-02T04:39:19.443 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:39:19.839 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:19 smithi016 ceph-mon[28969]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:39:19.839 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:19 smithi016 ceph-mon[28969]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:39:20.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:19 smithi027 ceph-mon[37550]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:39:20.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:19 smithi027 ceph-mon[37550]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:39:20.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:20.972+0000 7f8a4fc4c700 1 -- 172.21.15.16:0/3294747755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a48103e90 msgr2=0x7f8a48106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:20.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:20.972+0000 7f8a4fc4c700 1 --2- 172.21.15.16:0/3294747755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a48103e90 0x7f8a48106280 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c009910 tx=0x7f8a3c009c20 comp rx=0 tx=0).stop 2024-04-02T04:39:20.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:20.972+0000 7f8a4fc4c700 1 -- 172.21.15.16:0/3294747755 shutdown_connections 2024-04-02T04:39:20.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:20.972+0000 7f8a4fc4c700 1 --2- 172.21.15.16:0/3294747755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a48103e90 0x7f8a48106280 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:20.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:20.972+0000 7f8a4fc4c700 1 --2- 172.21.15.16:0/3294747755 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a48101560 0x7f8a48103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:20.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:20.972+0000 7f8a4fc4c700 1 -- 172.21.15.16:0/3294747755 >> 172.21.15.16:0/3294747755 conn(0x7f8a480faf70 msgr2=0x7f8a480fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:20.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:20.972+0000 7f8a4fc4c700 1 -- 172.21.15.16:0/3294747755 shutdown_connections 2024-04-02T04:39:20.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:20.972+0000 7f8a4fc4c700 1 -- 172.21.15.16:0/3294747755 wait complete. 2024-04-02T04:39:20.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:20.973+0000 7f8a4fc4c700 1 Processor -- start 2024-04-02T04:39:20.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:20.973+0000 7f8a4fc4c700 1 -- start start 2024-04-02T04:39:20.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:20.973+0000 7f8a4fc4c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a48101560 0x7f8a481965b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:20.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:20.973+0000 7f8a4fc4c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a48103e90 0x7f8a48196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:20.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:20.973+0000 7f8a4fc4c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a48197110 con 0x7f8a48103e90 2024-04-02T04:39:20.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:20.973+0000 7f8a4fc4c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a48197250 con 0x7f8a48101560 2024-04-02T04:39:20.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:20.973+0000 7f8a4d1e7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a48103e90 0x7f8a48196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:20.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:20.973+0000 7f8a4d1e7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a48103e90 0x7f8a48196af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42530/0 (socket says 172.21.15.16:42530) 2024-04-02T04:39:20.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:20.973+0000 7f8a4d1e7700 1 -- 172.21.15.16:0/1361628253 learned_addr learned my addr 172.21.15.16:0/1361628253 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:39:20.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:20.973+0000 7f8a4d9e8700 1 --2- 172.21.15.16:0/1361628253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a48101560 0x7f8a481965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:20.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:20.973+0000 7f8a4d1e7700 1 -- 172.21.15.16:0/1361628253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a48101560 msgr2=0x7f8a481965b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:20.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:20.973+0000 7f8a4d1e7700 1 --2- 172.21.15.16:0/1361628253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a48101560 0x7f8a481965b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:20.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:20.973+0000 7f8a4d1e7700 1 -- 172.21.15.16:0/1361628253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a3c0095f0 con 0x7f8a48103e90 2024-04-02T04:39:20.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:20.973+0000 7f8a4d9e8700 1 --2- 172.21.15.16:0/1361628253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a48101560 0x7f8a481965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T04:39:20.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:20.974+0000 7f8a4d1e7700 1 --2- 172.21.15.16:0/1361628253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a48103e90 0x7f8a48196af0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00bd70 tx=0x7f8a3c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:20.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:20.974+0000 7f8a46ffd700 1 -- 172.21.15.16:0/1361628253 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a3c01d070 con 0x7f8a48103e90 2024-04-02T04:39:20.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:20.974+0000 7f8a46ffd700 1 -- 172.21.15.16:0/1361628253 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8a3c00f460 con 0x7f8a48103e90 2024-04-02T04:39:20.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:20.974+0000 7f8a4fc4c700 1 -- 172.21.15.16:0/1361628253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a4819bc50 con 0x7f8a48103e90 2024-04-02T04:39:20.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:20.974+0000 7f8a46ffd700 1 -- 172.21.15.16:0/1361628253 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a3c021620 con 0x7f8a48103e90 2024-04-02T04:39:20.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:20.974+0000 7f8a4fc4c700 1 -- 172.21.15.16:0/1361628253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a4819c1c0 con 0x7f8a48103e90 2024-04-02T04:39:20.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:20.975+0000 7f8a4fc4c700 1 -- 172.21.15.16:0/1361628253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a4804ea80 con 0x7f8a48103e90 2024-04-02T04:39:20.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:20.975+0000 7f8a46ffd700 1 -- 172.21.15.16:0/1361628253 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f8a3c00fa70 con 0x7f8a48103e90 2024-04-02T04:39:20.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:20.976+0000 7f8a46ffd700 1 --2- 172.21.15.16:0/1361628253 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f8a3006ccf0 0x7f8a3006f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:20.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:20.976+0000 7f8a4d9e8700 1 --2- 172.21.15.16:0/1361628253 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f8a3006ccf0 0x7f8a3006f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:20.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:20.976+0000 7f8a46ffd700 1 -- 172.21.15.16:0/1361628253 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f8a3c08d2a0 con 0x7f8a48103e90 2024-04-02T04:39:20.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:20.976+0000 7f8a4d9e8700 1 --2- 172.21.15.16:0/1361628253 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f8a3006ccf0 0x7f8a3006f1b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8a38007eb0 tx=0x7f8a380073c0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:20.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:20.980+0000 7f8a46ffd700 1 -- 172.21.15.16:0/1361628253 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8a3c05b440 con 0x7f8a48103e90 2024-04-02T04:39:21.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:21.195+0000 7f8a4fc4c700 1 -- 172.21.15.16:0/1361628253 --> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a481048f0 con 0x7f8a3006ccf0 2024-04-02T04:39:21.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:21.200+0000 7f8a46ffd700 1 -- 172.21.15.16:0/1361628253 <== mgr.14217 v2:172.21.15.16:6800/3643600900 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1290 (secure 0 0 0) 0x7f8a481048f0 con 0x7f8a3006ccf0 2024-04-02T04:39:21.203 INFO:teuthology.orchestra.run.smithi016.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-02T04:39:21.203 INFO:teuthology.orchestra.run.smithi016.stdout:alertmanager ?:9093,9094 1/1 2m ago 6m count:1 2024-04-02T04:39:21.203 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter 2/2 2m ago 6m * 2024-04-02T04:39:21.203 INFO:teuthology.orchestra.run.smithi016.stdout:crash 2/2 2m ago 6m * 2024-04-02T04:39:21.203 INFO:teuthology.orchestra.run.smithi016.stdout:grafana ?:3000 1/1 2m ago 6m count:1 2024-04-02T04:39:21.204 INFO:teuthology.orchestra.run.smithi016.stdout:mgr 2/2 2m ago 6m count:2 2024-04-02T04:39:21.204 INFO:teuthology.orchestra.run.smithi016.stdout:mon 2/2 2m ago 5m smithi016:172.21.15.16=smithi016;smithi027:172.21.15.27=smithi027;count:2 2024-04-02T04:39:21.204 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter ?:9100 2/2 2m ago 6m * 2024-04-02T04:39:21.204 INFO:teuthology.orchestra.run.smithi016.stdout:osd 6 2m ago - 2024-04-02T04:39:21.204 INFO:teuthology.orchestra.run.smithi016.stdout:prometheus ?:9095 1/1 2m ago 6m count:1 2024-04-02T04:39:21.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:21.202+0000 7f8a4fc4c700 1 -- 172.21.15.16:0/1361628253 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f8a3006ccf0 msgr2=0x7f8a3006f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:21.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:21.202+0000 7f8a4fc4c700 1 --2- 172.21.15.16:0/1361628253 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f8a3006ccf0 0x7f8a3006f1b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8a38007eb0 tx=0x7f8a380073c0 comp rx=0 tx=0).stop 2024-04-02T04:39:21.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:21.202+0000 7f8a4fc4c700 1 -- 172.21.15.16:0/1361628253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a48103e90 msgr2=0x7f8a48196af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:21.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:21.202+0000 7f8a4fc4c700 1 --2- 172.21.15.16:0/1361628253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a48103e90 0x7f8a48196af0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00bd70 tx=0x7f8a3c00be50 comp rx=0 tx=0).stop 2024-04-02T04:39:21.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:21.202+0000 7f8a4fc4c700 1 -- 172.21.15.16:0/1361628253 shutdown_connections 2024-04-02T04:39:21.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:21.202+0000 7f8a4fc4c700 1 --2- 172.21.15.16:0/1361628253 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f8a3006ccf0 0x7f8a3006f1b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:21.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:21.202+0000 7f8a4fc4c700 1 --2- 172.21.15.16:0/1361628253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a48101560 0x7f8a481965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:21.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:21.202+0000 7f8a4fc4c700 1 --2- 172.21.15.16:0/1361628253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a48103e90 0x7f8a48196af0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:21.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:21.202+0000 7f8a4fc4c700 1 -- 172.21.15.16:0/1361628253 >> 172.21.15.16:0/1361628253 conn(0x7f8a480faf70 msgr2=0x7f8a480fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:21.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:21.202+0000 7f8a4fc4c700 1 -- 172.21.15.16:0/1361628253 shutdown_connections 2024-04-02T04:39:21.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:21.202+0000 7f8a4fc4c700 1 -- 172.21.15.16:0/1361628253 wait complete. 2024-04-02T04:39:21.822 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch host ls' 2024-04-02T04:39:22.031 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:39:22.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:21 smithi016 ceph-mon[28969]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:39:22.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:21 smithi027 ceph-mon[37550]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:39:23.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:22 smithi027 ceph-mon[37550]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:39:23.152 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:22 smithi016 ceph-mon[28969]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:39:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.582+0000 7f7b4ab59700 1 -- 172.21.15.16:0/3169851625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b44101cd0 msgr2=0x7f7b44102130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.582+0000 7f7b4ab59700 1 --2- 172.21.15.16:0/3169851625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b44101cd0 0x7f7b44102130 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7b30009910 tx=0x7f7b30009c20 comp rx=0 tx=0).stop 2024-04-02T04:39:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.582+0000 7f7b4ab59700 1 -- 172.21.15.16:0/3169851625 shutdown_connections 2024-04-02T04:39:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.582+0000 7f7b4ab59700 1 --2- 172.21.15.16:0/3169851625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b44101cd0 0x7f7b44102130 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.582+0000 7f7b4ab59700 1 --2- 172.21.15.16:0/3169851625 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b44100ad0 0x7f7b44100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.582+0000 7f7b4ab59700 1 -- 172.21.15.16:0/3169851625 >> 172.21.15.16:0/3169851625 conn(0x7f7b440fc070 msgr2=0x7f7b440fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.582+0000 7f7b4ab59700 1 -- 172.21.15.16:0/3169851625 shutdown_connections 2024-04-02T04:39:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.582+0000 7f7b4ab59700 1 -- 172.21.15.16:0/3169851625 wait complete. 2024-04-02T04:39:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.583+0000 7f7b4ab59700 1 Processor -- start 2024-04-02T04:39:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.583+0000 7f7b4ab59700 1 -- start start 2024-04-02T04:39:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.583+0000 7f7b4ab59700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b44100ad0 0x7f7b441964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.583+0000 7f7b4ab59700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b44101cd0 0x7f7b44196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.583+0000 7f7b4ab59700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b44197020 con 0x7f7b44101cd0 2024-04-02T04:39:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.583+0000 7f7b4ab59700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b44197160 con 0x7f7b44100ad0 2024-04-02T04:39:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.583+0000 7f7b43fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b44101cd0 0x7f7b44196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.583+0000 7f7b43fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b44101cd0 0x7f7b44196a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52630/0 (socket says 172.21.15.16:52630) 2024-04-02T04:39:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.583+0000 7f7b43fff700 1 -- 172.21.15.16:0/2425233581 learned_addr learned my addr 172.21.15.16:0/2425233581 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:39:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.583+0000 7f7b488f5700 1 --2- 172.21.15.16:0/2425233581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b44100ad0 0x7f7b441964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.583+0000 7f7b43fff700 1 -- 172.21.15.16:0/2425233581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b44100ad0 msgr2=0x7f7b441964c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.583+0000 7f7b43fff700 1 --2- 172.21.15.16:0/2425233581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b44100ad0 0x7f7b441964c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.583+0000 7f7b43fff700 1 -- 172.21.15.16:0/2425233581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b300095f0 con 0x7f7b44101cd0 2024-04-02T04:39:23.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.583+0000 7f7b488f5700 1 --2- 172.21.15.16:0/2425233581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b44100ad0 0x7f7b441964c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T04:39:23.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.584+0000 7f7b43fff700 1 --2- 172.21.15.16:0/2425233581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b44101cd0 0x7f7b44196a00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7b300092e0 tx=0x7f7b3000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:23.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.584+0000 7f7b41ffb700 1 -- 172.21.15.16:0/2425233581 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b3001d070 con 0x7f7b44101cd0 2024-04-02T04:39:23.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.584+0000 7f7b4ab59700 1 -- 172.21.15.16:0/2425233581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b4419bb10 con 0x7f7b44101cd0 2024-04-02T04:39:23.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.584+0000 7f7b41ffb700 1 -- 172.21.15.16:0/2425233581 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b3000f460 con 0x7f7b44101cd0 2024-04-02T04:39:23.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.584+0000 7f7b41ffb700 1 -- 172.21.15.16:0/2425233581 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b30021640 con 0x7f7b44101cd0 2024-04-02T04:39:23.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.584+0000 7f7b4ab59700 1 -- 172.21.15.16:0/2425233581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b4419c080 con 0x7f7b44101cd0 2024-04-02T04:39:23.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.585+0000 7f7b4ab59700 1 -- 172.21.15.16:0/2425233581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b44190650 con 0x7f7b44101cd0 2024-04-02T04:39:23.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.585+0000 7f7b41ffb700 1 -- 172.21.15.16:0/2425233581 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f7b3000fad0 con 0x7f7b44101cd0 2024-04-02T04:39:23.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.586+0000 7f7b41ffb700 1 --2- 172.21.15.16:0/2425233581 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f7b3406cce0 0x7f7b3406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:23.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.586+0000 7f7b41ffb700 1 -- 172.21.15.16:0/2425233581 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f7b3008d150 con 0x7f7b44101cd0 2024-04-02T04:39:23.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.586+0000 7f7b488f5700 1 --2- 172.21.15.16:0/2425233581 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f7b3406cce0 0x7f7b3406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:23.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.586+0000 7f7b488f5700 1 --2- 172.21.15.16:0/2425233581 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f7b3406cce0 0x7f7b3406f1a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7b38005fd0 tx=0x7f7b38005f00 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:23.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.590+0000 7f7b41ffb700 1 -- 172.21.15.16:0/2425233581 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7b3005b2f0 con 0x7f7b44101cd0 2024-04-02T04:39:23.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.799+0000 7f7b4ab59700 1 -- 172.21.15.16:0/2425233581 --> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b4402ce60 con 0x7f7b3406cce0 2024-04-02T04:39:23.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.800+0000 7f7b41ffb700 1 -- 172.21.15.16:0/2425233581 <== mgr.14217 v2:172.21.15.16:6800/3643600900 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+145 (secure 0 0 0) 0x7f7b4402ce60 con 0x7f7b3406cce0 2024-04-02T04:39:23.803 INFO:teuthology.orchestra.run.smithi016.stdout:HOST ADDR LABELS STATUS 2024-04-02T04:39:23.804 INFO:teuthology.orchestra.run.smithi016.stdout:smithi016 172.21.15.16 2024-04-02T04:39:23.804 INFO:teuthology.orchestra.run.smithi016.stdout:smithi027 172.21.15.27 2024-04-02T04:39:23.804 INFO:teuthology.orchestra.run.smithi016.stdout:2 hosts in cluster 2024-04-02T04:39:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.803+0000 7f7b4ab59700 1 -- 172.21.15.16:0/2425233581 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f7b3406cce0 msgr2=0x7f7b3406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.803+0000 7f7b4ab59700 1 --2- 172.21.15.16:0/2425233581 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f7b3406cce0 0x7f7b3406f1a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7b38005fd0 tx=0x7f7b38005f00 comp rx=0 tx=0).stop 2024-04-02T04:39:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.803+0000 7f7b4ab59700 1 -- 172.21.15.16:0/2425233581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b44101cd0 msgr2=0x7f7b44196a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.803+0000 7f7b4ab59700 1 --2- 172.21.15.16:0/2425233581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b44101cd0 0x7f7b44196a00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7b300092e0 tx=0x7f7b3000be20 comp rx=0 tx=0).stop 2024-04-02T04:39:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.803+0000 7f7b4ab59700 1 -- 172.21.15.16:0/2425233581 shutdown_connections 2024-04-02T04:39:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.803+0000 7f7b4ab59700 1 --2- 172.21.15.16:0/2425233581 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f7b3406cce0 0x7f7b3406f1a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.803+0000 7f7b4ab59700 1 --2- 172.21.15.16:0/2425233581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b44100ad0 0x7f7b441964c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.803+0000 7f7b4ab59700 1 --2- 172.21.15.16:0/2425233581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b44101cd0 0x7f7b44196a00 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.803+0000 7f7b4ab59700 1 -- 172.21.15.16:0/2425233581 >> 172.21.15.16:0/2425233581 conn(0x7f7b440fc070 msgr2=0x7f7b44105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.803+0000 7f7b4ab59700 1 -- 172.21.15.16:0/2425233581 shutdown_connections 2024-04-02T04:39:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:23.803+0000 7f7b4ab59700 1 -- 172.21.15.16:0/2425233581 wait complete. 2024-04-02T04:39:24.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:23 smithi016 ceph-mon[28969]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:39:24.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:23 smithi027 ceph-mon[37550]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:39:24.427 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch device ls' 2024-04-02T04:39:24.634 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:39:25.050 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:24 smithi016 ceph-mon[28969]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:39:25.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:24 smithi027 ceph-mon[37550]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:39:25.998 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:25 smithi016 ceph-mon[28969]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:39:26.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.104+0000 7f51b4afc700 1 -- 172.21.15.16:0/3393770308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51b0101ce0 msgr2=0x7f51b0102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:26.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.104+0000 7f51b4afc700 1 --2- 172.21.15.16:0/3393770308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51b0101ce0 0x7f51b0102140 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f51a4009910 tx=0x7f51a4009c20 comp rx=0 tx=0).stop 2024-04-02T04:39:26.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.104+0000 7f51b4afc700 1 -- 172.21.15.16:0/3393770308 shutdown_connections 2024-04-02T04:39:26.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.104+0000 7f51b4afc700 1 --2- 172.21.15.16:0/3393770308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51b0101ce0 0x7f51b0102140 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:26.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.104+0000 7f51b4afc700 1 --2- 172.21.15.16:0/3393770308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51b0100ae0 0x7f51b0100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:26.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.104+0000 7f51b4afc700 1 -- 172.21.15.16:0/3393770308 >> 172.21.15.16:0/3393770308 conn(0x7f51b00fc0a0 msgr2=0x7f51b00fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:26.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.104+0000 7f51b4afc700 1 -- 172.21.15.16:0/3393770308 shutdown_connections 2024-04-02T04:39:26.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.104+0000 7f51b4afc700 1 -- 172.21.15.16:0/3393770308 wait complete. 2024-04-02T04:39:26.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.105+0000 7f51b4afc700 1 Processor -- start 2024-04-02T04:39:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.105+0000 7f51b4afc700 1 -- start start 2024-04-02T04:39:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.105+0000 7f51b4afc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51b0100ae0 0x7f51b01964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.105+0000 7f51b4afc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51b0101ce0 0x7f51b0196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.105+0000 7f51b4afc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b0196f90 con 0x7f51b0100ae0 2024-04-02T04:39:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.105+0000 7f51b4afc700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b01970d0 con 0x7f51b0101ce0 2024-04-02T04:39:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.105+0000 7f51ae59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51b0100ae0 0x7f51b01964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.105+0000 7f51ae59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51b0100ae0 0x7f51b01964c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52646/0 (socket says 172.21.15.16:52646) 2024-04-02T04:39:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.105+0000 7f51ae59c700 1 -- 172.21.15.16:0/3791299035 learned_addr learned my addr 172.21.15.16:0/3791299035 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:39:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.105+0000 7f51add9b700 1 --2- 172.21.15.16:0/3791299035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51b0101ce0 0x7f51b0196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.106+0000 7f51ae59c700 1 -- 172.21.15.16:0/3791299035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51b0101ce0 msgr2=0x7f51b0196a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.106+0000 7f51ae59c700 1 --2- 172.21.15.16:0/3791299035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51b0101ce0 0x7f51b0196a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.106+0000 7f51ae59c700 1 -- 172.21.15.16:0/3791299035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a40095f0 con 0x7f51b0100ae0 2024-04-02T04:39:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.106+0000 7f51ae59c700 1 --2- 172.21.15.16:0/3791299035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51b0100ae0 0x7f51b01964c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f51a000c910 tx=0x7f51a000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.106+0000 7f519f7fe700 1 -- 172.21.15.16:0/3791299035 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51a00041f0 con 0x7f51b0100ae0 2024-04-02T04:39:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.106+0000 7f51b4afc700 1 -- 172.21.15.16:0/3791299035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b019bb40 con 0x7f51b0100ae0 2024-04-02T04:39:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.106+0000 7f519f7fe700 1 -- 172.21.15.16:0/3791299035 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f51a000ce60 con 0x7f51b0100ae0 2024-04-02T04:39:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.106+0000 7f519f7fe700 1 -- 172.21.15.16:0/3791299035 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51a0003d90 con 0x7f51b0100ae0 2024-04-02T04:39:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.106+0000 7f51b4afc700 1 -- 172.21.15.16:0/3791299035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b019c060 con 0x7f51b0100ae0 2024-04-02T04:39:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.107+0000 7f51b4afc700 1 -- 172.21.15.16:0/3791299035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51b0190660 con 0x7f51b0100ae0 2024-04-02T04:39:26.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.107+0000 7f519f7fe700 1 -- 172.21.15.16:0/3791299035 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f51a0007900 con 0x7f51b0100ae0 2024-04-02T04:39:26.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.108+0000 7f519f7fe700 1 --2- 172.21.15.16:0/3791299035 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f519806cce0 0x7f519806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:26.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.108+0000 7f519f7fe700 1 -- 172.21.15.16:0/3791299035 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f51a008bbe0 con 0x7f51b0100ae0 2024-04-02T04:39:26.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.108+0000 7f51add9b700 1 --2- 172.21.15.16:0/3791299035 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f519806cce0 0x7f519806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:26.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.108+0000 7f51add9b700 1 --2- 172.21.15.16:0/3791299035 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f519806cce0 0x7f519806f1a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f51a4005090 tx=0x7f51a4005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:26.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.112+0000 7f519f7fe700 1 -- 172.21.15.16:0/3791299035 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f51a0059d80 con 0x7f51b0100ae0 2024-04-02T04:39:26.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:25 smithi027 ceph-mon[37550]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:39:26.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.328+0000 7f51b4afc700 1 -- 172.21.15.16:0/3791299035 --> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f51b002ce20 con 0x7f519806cce0 2024-04-02T04:39:26.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.329+0000 7f519f7fe700 1 -- 172.21.15.16:0/3791299035 <== mgr.14217 v2:172.21.15.16:6800/3643600900 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7f51b002ce20 con 0x7f519806cce0 2024-04-02T04:39:26.332 INFO:teuthology.orchestra.run.smithi016.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-02T04:39:26.332 INFO:teuthology.orchestra.run.smithi016.stdout:smithi016 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513002C480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-04-02T04:39:26.332 INFO:teuthology.orchestra.run.smithi016.stdout:smithi027 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331001C400BGN 372G No 67s ago Has a FileSystem, LVM detected 2024-04-02T04:39:26.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.331+0000 7f51b4afc700 1 -- 172.21.15.16:0/3791299035 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f519806cce0 msgr2=0x7f519806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:26.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.331+0000 7f51b4afc700 1 --2- 172.21.15.16:0/3791299035 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f519806cce0 0x7f519806f1a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f51a4005090 tx=0x7f51a4005960 comp rx=0 tx=0).stop 2024-04-02T04:39:26.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.331+0000 7f51b4afc700 1 -- 172.21.15.16:0/3791299035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51b0100ae0 msgr2=0x7f51b01964c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:26.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.331+0000 7f51b4afc700 1 --2- 172.21.15.16:0/3791299035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51b0100ae0 0x7f51b01964c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f51a000c910 tx=0x7f51a000cc20 comp rx=0 tx=0).stop 2024-04-02T04:39:26.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.331+0000 7f51b4afc700 1 -- 172.21.15.16:0/3791299035 shutdown_connections 2024-04-02T04:39:26.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.331+0000 7f51b4afc700 1 --2- 172.21.15.16:0/3791299035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51b0100ae0 0x7f51b01964c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:26.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.331+0000 7f51b4afc700 1 --2- 172.21.15.16:0/3791299035 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f519806cce0 0x7f519806f1a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:26.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.331+0000 7f51b4afc700 1 --2- 172.21.15.16:0/3791299035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51b0101ce0 0x7f51b0196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:26.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.331+0000 7f51b4afc700 1 -- 172.21.15.16:0/3791299035 >> 172.21.15.16:0/3791299035 conn(0x7f51b00fc0a0 msgr2=0x7f51b0105280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:26.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.331+0000 7f51b4afc700 1 -- 172.21.15.16:0/3791299035 shutdown_connections 2024-04-02T04:39:26.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:26.331+0000 7f51b4afc700 1 -- 172.21.15.16:0/3791299035 wait complete. 2024-04-02T04:39:26.939 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-02T04:39:26.950 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi016.front.sepia.ceph.com 2024-04-02T04:39:26.950 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-04-02T04:39:27.159 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:39:28.082 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:27 smithi016 ceph-mon[28969]: from='client.14464 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:39:28.083 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:27 smithi016 ceph-mon[28969]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:39:28.083 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:27 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:39:28.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:27 smithi027 ceph-mon[37550]: from='client.14464 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:39:28.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:27 smithi027 ceph-mon[37550]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:39:28.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:27 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:39:28.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:28.551+0000 7fee50295700 1 -- 172.21.15.16:0/1219587691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee48100be0 msgr2=0x7fee48101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:28.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:28.551+0000 7fee50295700 1 --2- 172.21.15.16:0/1219587691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee48100be0 0x7fee48101040 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fee4000b0a0 tx=0x7fee4000b3b0 comp rx=0 tx=0).stop 2024-04-02T04:39:28.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:28.552+0000 7fee50295700 1 -- 172.21.15.16:0/1219587691 shutdown_connections 2024-04-02T04:39:28.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:28.552+0000 7fee50295700 1 --2- 172.21.15.16:0/1219587691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee48100be0 0x7fee48101040 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:28.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:28.552+0000 7fee50295700 1 --2- 172.21.15.16:0/1219587691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee480ff9e0 0x7fee480ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:28.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:28.552+0000 7fee50295700 1 -- 172.21.15.16:0/1219587691 >> 172.21.15.16:0/1219587691 conn(0x7fee480faf80 msgr2=0x7fee480fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:28.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:28.552+0000 7fee50295700 1 -- 172.21.15.16:0/1219587691 shutdown_connections 2024-04-02T04:39:28.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:28.552+0000 7fee50295700 1 -- 172.21.15.16:0/1219587691 wait complete. 2024-04-02T04:39:28.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:28.552+0000 7fee50295700 1 Processor -- start 2024-04-02T04:39:28.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:28.552+0000 7fee50295700 1 -- start start 2024-04-02T04:39:28.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:28.552+0000 7fee50295700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee480ff9e0 0x7fee48198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:28.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:28.552+0000 7fee50295700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee48100be0 0x7fee48198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:28.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:28.552+0000 7fee50295700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee48199220 con 0x7fee48100be0 2024-04-02T04:39:28.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:28.552+0000 7fee50295700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee48199360 con 0x7fee480ff9e0 2024-04-02T04:39:28.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:28.553+0000 7fee4d830700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee48100be0 0x7fee48198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:28.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:28.553+0000 7fee4d830700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee48100be0 0x7fee48198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52658/0 (socket says 172.21.15.16:52658) 2024-04-02T04:39:28.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:28.553+0000 7fee4d830700 1 -- 172.21.15.16:0/1437782603 learned_addr learned my addr 172.21.15.16:0/1437782603 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:39:28.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:28.553+0000 7fee4e031700 1 --2- 172.21.15.16:0/1437782603 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee480ff9e0 0x7fee48198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:28.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:28.553+0000 7fee4d830700 1 -- 172.21.15.16:0/1437782603 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee480ff9e0 msgr2=0x7fee48198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:28.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:28.553+0000 7fee4d830700 1 --2- 172.21.15.16:0/1437782603 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee480ff9e0 0x7fee48198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:28.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:28.553+0000 7fee4d830700 1 -- 172.21.15.16:0/1437782603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee40009d00 con 0x7fee48100be0 2024-04-02T04:39:28.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:28.553+0000 7fee4d830700 1 --2- 172.21.15.16:0/1437782603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee48100be0 0x7fee48198c90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fee40007950 tx=0x7fee40003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:28.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:28.553+0000 7fee3f7fe700 1 -- 172.21.15.16:0/1437782603 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee40021070 con 0x7fee48100be0 2024-04-02T04:39:28.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:28.553+0000 7fee3f7fe700 1 -- 172.21.15.16:0/1437782603 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fee4000ce60 con 0x7fee48100be0 2024-04-02T04:39:28.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:28.553+0000 7fee50295700 1 -- 172.21.15.16:0/1437782603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee4819dd70 con 0x7fee48100be0 2024-04-02T04:39:28.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:28.553+0000 7fee3f7fe700 1 -- 172.21.15.16:0/1437782603 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee4000e040 con 0x7fee48100be0 2024-04-02T04:39:28.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:28.553+0000 7fee50295700 1 -- 172.21.15.16:0/1437782603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee4819e2b0 con 0x7fee48100be0 2024-04-02T04:39:28.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:28.554+0000 7fee50295700 1 -- 172.21.15.16:0/1437782603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee4804ea80 con 0x7fee48100be0 2024-04-02T04:39:28.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:28.555+0000 7fee3f7fe700 1 -- 172.21.15.16:0/1437782603 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fee40004250 con 0x7fee48100be0 2024-04-02T04:39:28.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:28.555+0000 7fee3f7fe700 1 --2- 172.21.15.16:0/1437782603 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fee3406ccf0 0x7fee3406f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:28.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:28.555+0000 7fee4e031700 1 --2- 172.21.15.16:0/1437782603 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fee3406ccf0 0x7fee3406f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:28.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:28.555+0000 7fee3f7fe700 1 -- 172.21.15.16:0/1437782603 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fee4008cfe0 con 0x7fee48100be0 2024-04-02T04:39:28.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:28.556+0000 7fee4e031700 1 --2- 172.21.15.16:0/1437782603 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fee3406ccf0 0x7fee3406f1b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fee48100a40 tx=0x7fee4400a3b0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:28.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:28.560+0000 7fee3f7fe700 1 -- 172.21.15.16:0/1437782603 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fee4005b200 con 0x7fee48100be0 2024-04-02T04:39:28.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:28.789+0000 7fee50295700 1 -- 172.21.15.16:0/1437782603 --> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fee480696f0 con 0x7fee3406ccf0 2024-04-02T04:39:30.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:29 smithi016 ceph-mon[28969]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:39:30.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:29 smithi016 ceph-mon[28969]: from='client.14468 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:39:30.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:29 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-04-02T04:39:30.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:29 smithi027 ceph-mon[37550]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:39:30.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:29 smithi027 ceph-mon[37550]: from='client.14468 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:39:30.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:29 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-04-02T04:39:30.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:30.760+0000 7fee3f7fe700 1 -- 172.21.15.16:0/1437782603 <== mgr.14217 v2:172.21.15.16:6800/3643600900 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fee480696f0 con 0x7fee3406ccf0 2024-04-02T04:39:30.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:30.763+0000 7fee50295700 1 -- 172.21.15.16:0/1437782603 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fee3406ccf0 msgr2=0x7fee3406f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:30.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:30.763+0000 7fee50295700 1 --2- 172.21.15.16:0/1437782603 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fee3406ccf0 0x7fee3406f1b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fee48100a40 tx=0x7fee4400a3b0 comp rx=0 tx=0).stop 2024-04-02T04:39:30.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:30.763+0000 7fee50295700 1 -- 172.21.15.16:0/1437782603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee48100be0 msgr2=0x7fee48198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:30.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:30.763+0000 7fee50295700 1 --2- 172.21.15.16:0/1437782603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee48100be0 0x7fee48198c90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fee40007950 tx=0x7fee40003c30 comp rx=0 tx=0).stop 2024-04-02T04:39:30.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:30.763+0000 7fee50295700 1 -- 172.21.15.16:0/1437782603 shutdown_connections 2024-04-02T04:39:30.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:30.763+0000 7fee50295700 1 --2- 172.21.15.16:0/1437782603 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fee3406ccf0 0x7fee3406f1b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:30.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:30.763+0000 7fee50295700 1 --2- 172.21.15.16:0/1437782603 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee480ff9e0 0x7fee48198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:30.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:30.763+0000 7fee50295700 1 --2- 172.21.15.16:0/1437782603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee48100be0 0x7fee48198c90 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:30.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:30.763+0000 7fee50295700 1 -- 172.21.15.16:0/1437782603 >> 172.21.15.16:0/1437782603 conn(0x7fee480faf80 msgr2=0x7fee48068fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:30.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:30.763+0000 7fee50295700 1 -- 172.21.15.16:0/1437782603 shutdown_connections 2024-04-02T04:39:30.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:30.763+0000 7fee50295700 1 -- 172.21.15.16:0/1437782603 wait complete. 2024-04-02T04:39:31.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:30 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-04-02T04:39:31.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:30 smithi016 ceph-mon[28969]: osdmap e41: 6 total, 6 up, 6 in 2024-04-02T04:39:31.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:30 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-04-02T04:39:31.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:30 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi016[28965]: 2024-04-02T04:39:30.739+0000 7f053ffcd700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T04:39:31.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:30 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-04-02T04:39:31.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:30 smithi027 ceph-mon[37550]: osdmap e41: 6 total, 6 up, 6 in 2024-04-02T04:39:31.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:30 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-04-02T04:39:31.419 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- bash -c 'ceph fs dump' 2024-04-02T04:39:31.635 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:39:32.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:31 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-04-02T04:39:32.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:31 smithi016 ceph-mon[28969]: osdmap e42: 6 total, 6 up, 6 in 2024-04-02T04:39:32.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:31 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-04-02T04:39:32.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:31 smithi016 ceph-mon[28969]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T04:39:32.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:31 smithi016 ceph-mon[28969]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-02T04:39:32.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:31 smithi016 ceph-mon[28969]: osdmap e43: 6 total, 6 up, 6 in 2024-04-02T04:39:32.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:31 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-04-02T04:39:32.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:31 smithi016 ceph-mon[28969]: fsmap cephfs:0 2024-04-02T04:39:32.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:31 smithi016 ceph-mon[28969]: Saving service mds.cephfs spec with placement count:4 2024-04-02T04:39:32.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:31 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:32.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:31 smithi016 ceph-mon[28969]: pgmap v175: 65 pgs: 5 creating+peering, 1 active+clean, 59 unknown; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:39:32.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:31 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:39:32.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:31 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:39:32.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:31 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:39:32.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:31 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:32.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:31 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi016.uhgkgg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:39:32.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:31 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi016.uhgkgg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T04:39:32.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:31 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:39:32.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:31 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-04-02T04:39:32.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:31 smithi027 ceph-mon[37550]: osdmap e42: 6 total, 6 up, 6 in 2024-04-02T04:39:32.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:31 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-04-02T04:39:32.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:31 smithi027 ceph-mon[37550]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T04:39:32.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:31 smithi027 ceph-mon[37550]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-02T04:39:32.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:31 smithi027 ceph-mon[37550]: osdmap e43: 6 total, 6 up, 6 in 2024-04-02T04:39:32.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:31 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-04-02T04:39:32.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:31 smithi027 ceph-mon[37550]: fsmap cephfs:0 2024-04-02T04:39:32.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:31 smithi027 ceph-mon[37550]: Saving service mds.cephfs spec with placement count:4 2024-04-02T04:39:32.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:31 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:32.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:31 smithi027 ceph-mon[37550]: pgmap v175: 65 pgs: 5 creating+peering, 1 active+clean, 59 unknown; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:39:32.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:31 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:39:32.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:31 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:39:32.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:31 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:39:32.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:31 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:32.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:31 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi016.uhgkgg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:39:32.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:31 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi016.uhgkgg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T04:39:32.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:31 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:39:32.977 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:32 smithi016 ceph-mon[28969]: Deploying daemon mds.cephfs.smithi016.uhgkgg on smithi016 2024-04-02T04:39:32.977 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:32 smithi016 ceph-mon[28969]: osdmap e44: 6 total, 6 up, 6 in 2024-04-02T04:39:33.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:32 smithi027 ceph-mon[37550]: Deploying daemon mds.cephfs.smithi016.uhgkgg on smithi016 2024-04-02T04:39:33.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:32 smithi027 ceph-mon[37550]: osdmap e44: 6 total, 6 up, 6 in 2024-04-02T04:39:33.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.196+0000 7febd057d700 1 -- 172.21.15.16:0/3208375442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc8100ae0 msgr2=0x7febc8100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:33.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.196+0000 7febd057d700 1 --2- 172.21.15.16:0/3208375442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc8100ae0 0x7febc8100f00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7febc4009910 tx=0x7febc4009c20 comp rx=0 tx=0).stop 2024-04-02T04:39:33.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.196+0000 7febd057d700 1 -- 172.21.15.16:0/3208375442 shutdown_connections 2024-04-02T04:39:33.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.196+0000 7febd057d700 1 --2- 172.21.15.16:0/3208375442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febc8101ce0 0x7febc8102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:33.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.196+0000 7febd057d700 1 --2- 172.21.15.16:0/3208375442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc8100ae0 0x7febc8100f00 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:33.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.196+0000 7febd057d700 1 -- 172.21.15.16:0/3208375442 >> 172.21.15.16:0/3208375442 conn(0x7febc80fc0a0 msgr2=0x7febc80fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:33.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.196+0000 7febd057d700 1 -- 172.21.15.16:0/3208375442 shutdown_connections 2024-04-02T04:39:33.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.196+0000 7febd057d700 1 -- 172.21.15.16:0/3208375442 wait complete. 2024-04-02T04:39:33.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.196+0000 7febd057d700 1 Processor -- start 2024-04-02T04:39:33.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.196+0000 7febd057d700 1 -- start start 2024-04-02T04:39:33.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.197+0000 7febd057d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc8100ae0 0x7febc8074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:33.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.197+0000 7febd057d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febc8101ce0 0x7febc80731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:33.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.197+0000 7febd057d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc80736e0 con 0x7febc8100ae0 2024-04-02T04:39:33.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.197+0000 7febd057d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc8073820 con 0x7febc8101ce0 2024-04-02T04:39:33.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.197+0000 7febce319700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc8100ae0 0x7febc8074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:33.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.197+0000 7febce319700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc8100ae0 0x7febc8074b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42292/0 (socket says 172.21.15.16:42292) 2024-04-02T04:39:33.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.197+0000 7febce319700 1 -- 172.21.15.16:0/2713095627 learned_addr learned my addr 172.21.15.16:0/2713095627 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:39:33.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.197+0000 7febcdb18700 1 --2- 172.21.15.16:0/2713095627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febc8101ce0 0x7febc80731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:33.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.197+0000 7febce319700 1 -- 172.21.15.16:0/2713095627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febc8101ce0 msgr2=0x7febc80731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:33.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.197+0000 7febce319700 1 --2- 172.21.15.16:0/2713095627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febc8101ce0 0x7febc80731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:33.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.197+0000 7febce319700 1 -- 172.21.15.16:0/2713095627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc40095f0 con 0x7febc8100ae0 2024-04-02T04:39:33.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.197+0000 7febce319700 1 --2- 172.21.15.16:0/2713095627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc8100ae0 0x7febc8074b00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7febc400b620 tx=0x7febc4005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:33.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.197+0000 7febbf7fe700 1 -- 172.21.15.16:0/2713095627 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febc401d070 con 0x7febc8100ae0 2024-04-02T04:39:33.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.198+0000 7febbf7fe700 1 -- 172.21.15.16:0/2713095627 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7febc40052a0 con 0x7febc8100ae0 2024-04-02T04:39:33.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.198+0000 7febd057d700 1 -- 172.21.15.16:0/2713095627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febc8073aa0 con 0x7febc8100ae0 2024-04-02T04:39:33.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.198+0000 7febbf7fe700 1 -- 172.21.15.16:0/2713095627 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febc4017400 con 0x7febc8100ae0 2024-04-02T04:39:33.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.198+0000 7febd057d700 1 -- 172.21.15.16:0/2713095627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc8074010 con 0x7febc8100ae0 2024-04-02T04:39:33.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.198+0000 7febd057d700 1 -- 172.21.15.16:0/2713095627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febc8190660 con 0x7febc8100ae0 2024-04-02T04:39:33.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.199+0000 7febbf7fe700 1 -- 172.21.15.16:0/2713095627 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7febc4003680 con 0x7febc8100ae0 2024-04-02T04:39:33.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.200+0000 7febbf7fe700 1 --2- 172.21.15.16:0/2713095627 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7febb406ccf0 0x7febb406f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:33.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.200+0000 7febbf7fe700 1 -- 172.21.15.16:0/2713095627 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7febc408d5c0 con 0x7febc8100ae0 2024-04-02T04:39:33.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.200+0000 7febcdb18700 1 --2- 172.21.15.16:0/2713095627 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7febb406ccf0 0x7febb406f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:33.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.200+0000 7febcdb18700 1 --2- 172.21.15.16:0/2713095627 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7febb406ccf0 0x7febb406f1b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7febb8009c60 tx=0x7febb8009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:33.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.204+0000 7febbf7fe700 1 -- 172.21.15.16:0/2713095627 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7febc405b4a0 con 0x7febc8100ae0 2024-04-02T04:39:33.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.456+0000 7febd057d700 1 -- 172.21.15.16:0/2713095627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7febc8074010 con 0x7febc8100ae0 2024-04-02T04:39:33.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.457+0000 7febbf7fe700 1 -- 172.21.15.16:0/2713095627 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1114 (secure 0 0 0) 0x7febc8074010 con 0x7febc8100ae0 2024-04-02T04:39:33.462 INFO:teuthology.orchestra.run.smithi016.stdout:e2 2024-04-02T04:39:33.463 INFO:teuthology.orchestra.run.smithi016.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:39:33.463 INFO:teuthology.orchestra.run.smithi016.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:39:33.463 INFO:teuthology.orchestra.run.smithi016.stdout:legacy client fscid: 1 2024-04-02T04:39:33.463 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:39:33.463 INFO:teuthology.orchestra.run.smithi016.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:39:33.463 INFO:teuthology.orchestra.run.smithi016.stdout:fs_name cephfs 2024-04-02T04:39:33.463 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 2 2024-04-02T04:39:33.463 INFO:teuthology.orchestra.run.smithi016.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:39:33.463 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-04-02T04:39:30.740168+0000 2024-04-02T04:39:33.463 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-04-02T04:39:30.740229+0000 2024-04-02T04:39:33.463 INFO:teuthology.orchestra.run.smithi016.stdout:tableserver 0 2024-04-02T04:39:33.463 INFO:teuthology.orchestra.run.smithi016.stdout:root 0 2024-04-02T04:39:33.463 INFO:teuthology.orchestra.run.smithi016.stdout:session_timeout 60 2024-04-02T04:39:33.464 INFO:teuthology.orchestra.run.smithi016.stdout:session_autoclose 300 2024-04-02T04:39:33.464 INFO:teuthology.orchestra.run.smithi016.stdout:max_file_size 1099511627776 2024-04-02T04:39:33.464 INFO:teuthology.orchestra.run.smithi016.stdout:max_xattr_size 65536 2024-04-02T04:39:33.464 INFO:teuthology.orchestra.run.smithi016.stdout:required_client_features {} 2024-04-02T04:39:33.464 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure 0 2024-04-02T04:39:33.464 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure_osd_epoch 0 2024-04-02T04:39:33.464 INFO:teuthology.orchestra.run.smithi016.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:39:33.464 INFO:teuthology.orchestra.run.smithi016.stdout:max_mds 1 2024-04-02T04:39:33.464 INFO:teuthology.orchestra.run.smithi016.stdout:in 2024-04-02T04:39:33.464 INFO:teuthology.orchestra.run.smithi016.stdout:up {} 2024-04-02T04:39:33.464 INFO:teuthology.orchestra.run.smithi016.stdout:failed 2024-04-02T04:39:33.464 INFO:teuthology.orchestra.run.smithi016.stdout:damaged 2024-04-02T04:39:33.464 INFO:teuthology.orchestra.run.smithi016.stdout:stopped 2024-04-02T04:39:33.464 INFO:teuthology.orchestra.run.smithi016.stdout:data_pools [3] 2024-04-02T04:39:33.464 INFO:teuthology.orchestra.run.smithi016.stdout:metadata_pool 2 2024-04-02T04:39:33.465 INFO:teuthology.orchestra.run.smithi016.stdout:inline_data disabled 2024-04-02T04:39:33.465 INFO:teuthology.orchestra.run.smithi016.stdout:balancer 2024-04-02T04:39:33.465 INFO:teuthology.orchestra.run.smithi016.stdout:bal_rank_mask -1 2024-04-02T04:39:33.465 INFO:teuthology.orchestra.run.smithi016.stdout:standby_count_wanted 0 2024-04-02T04:39:33.465 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:39:33.465 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:39:33.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.459+0000 7febd057d700 1 -- 172.21.15.16:0/2713095627 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7febb406ccf0 msgr2=0x7febb406f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:33.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.459+0000 7febd057d700 1 --2- 172.21.15.16:0/2713095627 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7febb406ccf0 0x7febb406f1b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7febb8009c60 tx=0x7febb8009260 comp rx=0 tx=0).stop 2024-04-02T04:39:33.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.459+0000 7febd057d700 1 -- 172.21.15.16:0/2713095627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc8100ae0 msgr2=0x7febc8074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:33.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.459+0000 7febd057d700 1 --2- 172.21.15.16:0/2713095627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc8100ae0 0x7febc8074b00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7febc400b620 tx=0x7febc4005f00 comp rx=0 tx=0).stop 2024-04-02T04:39:33.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.459+0000 7febd057d700 1 -- 172.21.15.16:0/2713095627 shutdown_connections 2024-04-02T04:39:33.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.459+0000 7febd057d700 1 --2- 172.21.15.16:0/2713095627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc8100ae0 0x7febc8074b00 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:33.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.459+0000 7febd057d700 1 --2- 172.21.15.16:0/2713095627 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7febb406ccf0 0x7febb406f1b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:33.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.459+0000 7febd057d700 1 --2- 172.21.15.16:0/2713095627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febc8101ce0 0x7febc80731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:33.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.459+0000 7febd057d700 1 -- 172.21.15.16:0/2713095627 >> 172.21.15.16:0/2713095627 conn(0x7febc80fc0a0 msgr2=0x7febc81052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:33.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.459+0000 7febd057d700 1 -- 172.21.15.16:0/2713095627 shutdown_connections 2024-04-02T04:39:33.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:33.459+0000 7febd057d700 1 -- 172.21.15.16:0/2713095627 wait complete. 2024-04-02T04:39:33.466 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 2 2024-04-02T04:39:33.988 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:33 smithi016 ceph-mon[28969]: osdmap e45: 6 total, 6 up, 6 in 2024-04-02T04:39:33.989 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:33 smithi016 ceph-mon[28969]: pgmap v178: 65 pgs: 10 creating+peering, 8 active+clean, 47 unknown; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:39:33.989 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:33 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/2713095627' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:39:34.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:33 smithi027 ceph-mon[37550]: osdmap e45: 6 total, 6 up, 6 in 2024-04-02T04:39:34.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:33 smithi027 ceph-mon[37550]: pgmap v178: 65 pgs: 10 creating+peering, 8 active+clean, 47 unknown; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:39:34.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:33 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/2713095627' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:39:34.919 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-02T04:39:34.930 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi016.front.sepia.ceph.com 2024-04-02T04:39:34.930 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- bash -c 'ceph fs set cephfs max_mds 1' 2024-04-02T04:39:35.446 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:39:35.902 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:35 smithi016 ceph-mon[28969]: pgmap v179: 65 pgs: 10 creating+peering, 51 active+clean, 4 unknown; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:39:35.902 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:35 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:35.902 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:35 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:35.902 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:35 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:35.902 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:35 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi027.zwrfoo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:39:35.902 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:35 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi027.zwrfoo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T04:39:35.903 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:35 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:39:36.120 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:35 smithi027 ceph-mon[37550]: pgmap v179: 65 pgs: 10 creating+peering, 51 active+clean, 4 unknown; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:39:36.120 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:35 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:36.120 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:35 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:36.120 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:35 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:36.120 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:35 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi027.zwrfoo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:39:36.120 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:35 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi027.zwrfoo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T04:39:36.120 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:35 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:39:37.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.087+0000 7f2628f9f700 1 -- 172.21.15.16:0/3470809692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2624101b30 msgr2=0x7f2624101f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:37.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.087+0000 7f2628f9f700 1 --2- 172.21.15.16:0/3470809692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2624101b30 0x7f2624101f90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2614009910 tx=0x7f2614009c20 comp rx=0 tx=0).stop 2024-04-02T04:39:37.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.087+0000 7f2628f9f700 1 -- 172.21.15.16:0/3470809692 shutdown_connections 2024-04-02T04:39:37.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.087+0000 7f2628f9f700 1 --2- 172.21.15.16:0/3470809692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2624101b30 0x7f2624101f90 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:37.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.087+0000 7f2628f9f700 1 --2- 172.21.15.16:0/3470809692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2624100930 0x7f2624100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:37.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.087+0000 7f2628f9f700 1 -- 172.21.15.16:0/3470809692 >> 172.21.15.16:0/3470809692 conn(0x7f26240fbed0 msgr2=0x7f26240fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:37.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.087+0000 7f2628f9f700 1 -- 172.21.15.16:0/3470809692 shutdown_connections 2024-04-02T04:39:37.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.087+0000 7f2628f9f700 1 -- 172.21.15.16:0/3470809692 wait complete. 2024-04-02T04:39:37.092 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:36 smithi016 ceph-mon[28969]: Deploying daemon mds.cephfs.smithi027.zwrfoo on smithi027 2024-04-02T04:39:37.092 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:36 smithi016 ceph-mon[28969]: mds.? [v2:172.21.15.16:6826/71509234,v1:172.21.15.16:6827/71509234] up:boot 2024-04-02T04:39:37.092 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:36 smithi016 ceph-mon[28969]: daemon mds.cephfs.smithi016.uhgkgg assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-04-02T04:39:37.092 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:36 smithi016 ceph-mon[28969]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-02T04:39:37.092 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:36 smithi016 ceph-mon[28969]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-02T04:39:37.092 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:36 smithi016 ceph-mon[28969]: Cluster is now healthy 2024-04-02T04:39:37.093 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:36 smithi016 ceph-mon[28969]: fsmap cephfs:0 1 up:standby 2024-04-02T04:39:37.093 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:36 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi016.uhgkgg"}]: dispatch 2024-04-02T04:39:37.093 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:36 smithi016 ceph-mon[28969]: fsmap cephfs:1 {0=cephfs.smithi016.uhgkgg=up:creating} 2024-04-02T04:39:37.093 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:36 smithi016 ceph-mon[28969]: daemon mds.cephfs.smithi016.uhgkgg is now active in filesystem cephfs as rank 0 2024-04-02T04:39:37.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.088+0000 7f2628f9f700 1 Processor -- start 2024-04-02T04:39:37.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.088+0000 7f2628f9f700 1 -- start start 2024-04-02T04:39:37.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.088+0000 7f2628f9f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2624100930 0x7f2624196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:37.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.088+0000 7f2628f9f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2624101b30 0x7f2624196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:37.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.088+0000 7f2628f9f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2624196ea0 con 0x7f2624101b30 2024-04-02T04:39:37.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.088+0000 7f2628f9f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2624196fe0 con 0x7f2624100930 2024-04-02T04:39:37.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.088+0000 7f2622ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2624101b30 0x7f2624196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:37.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.088+0000 7f2622ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2624101b30 0x7f2624196880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42338/0 (socket says 172.21.15.16:42338) 2024-04-02T04:39:37.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.088+0000 7f2622ffd700 1 -- 172.21.15.16:0/345626630 learned_addr learned my addr 172.21.15.16:0/345626630 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:39:37.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.088+0000 7f26237fe700 1 --2- 172.21.15.16:0/345626630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2624100930 0x7f2624196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:37.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.088+0000 7f2622ffd700 1 -- 172.21.15.16:0/345626630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2624100930 msgr2=0x7f2624196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:37.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.088+0000 7f2622ffd700 1 --2- 172.21.15.16:0/345626630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2624100930 0x7f2624196340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:37.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.088+0000 7f2622ffd700 1 -- 172.21.15.16:0/345626630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26140095f0 con 0x7f2624101b30 2024-04-02T04:39:37.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.089+0000 7f2622ffd700 1 --2- 172.21.15.16:0/345626630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2624101b30 0x7f2624196880 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f261400b800 tx=0x7f2614005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:37.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.089+0000 7f2620ff9700 1 -- 172.21.15.16:0/345626630 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f261401d070 con 0x7f2624101b30 2024-04-02T04:39:37.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.089+0000 7f2628f9f700 1 -- 172.21.15.16:0/345626630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f262419b9e0 con 0x7f2624101b30 2024-04-02T04:39:37.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.089+0000 7f2620ff9700 1 -- 172.21.15.16:0/345626630 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f26140052a0 con 0x7f2624101b30 2024-04-02T04:39:37.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.089+0000 7f2620ff9700 1 -- 172.21.15.16:0/345626630 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2614017400 con 0x7f2624101b30 2024-04-02T04:39:37.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.089+0000 7f2628f9f700 1 -- 172.21.15.16:0/345626630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2624072fe0 con 0x7f2624101b30 2024-04-02T04:39:37.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.090+0000 7f2628f9f700 1 -- 172.21.15.16:0/345626630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26241904b0 con 0x7f2624101b30 2024-04-02T04:39:37.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.090+0000 7f2620ff9700 1 -- 172.21.15.16:0/345626630 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f2614003780 con 0x7f2624101b30 2024-04-02T04:39:37.096 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:36 smithi027 ceph-mon[37550]: Deploying daemon mds.cephfs.smithi027.zwrfoo on smithi027 2024-04-02T04:39:37.096 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:36 smithi027 ceph-mon[37550]: mds.? [v2:172.21.15.16:6826/71509234,v1:172.21.15.16:6827/71509234] up:boot 2024-04-02T04:39:37.097 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:36 smithi027 ceph-mon[37550]: daemon mds.cephfs.smithi016.uhgkgg assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-04-02T04:39:37.097 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:36 smithi027 ceph-mon[37550]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-02T04:39:37.097 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:36 smithi027 ceph-mon[37550]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-02T04:39:37.097 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:36 smithi027 ceph-mon[37550]: Cluster is now healthy 2024-04-02T04:39:37.097 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:36 smithi027 ceph-mon[37550]: fsmap cephfs:0 1 up:standby 2024-04-02T04:39:37.097 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:36 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi016.uhgkgg"}]: dispatch 2024-04-02T04:39:37.097 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:36 smithi027 ceph-mon[37550]: fsmap cephfs:1 {0=cephfs.smithi016.uhgkgg=up:creating} 2024-04-02T04:39:37.097 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:36 smithi027 ceph-mon[37550]: daemon mds.cephfs.smithi016.uhgkgg is now active in filesystem cephfs as rank 0 2024-04-02T04:39:37.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.091+0000 7f2620ff9700 1 --2- 172.21.15.16:0/345626630 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f26100710f0 0x7f26100735b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:37.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.091+0000 7f2620ff9700 1 -- 172.21.15.16:0/345626630 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f261408d6b0 con 0x7f2624101b30 2024-04-02T04:39:37.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.091+0000 7f26237fe700 1 --2- 172.21.15.16:0/345626630 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f26100710f0 0x7f26100735b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:37.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.091+0000 7f26237fe700 1 --2- 172.21.15.16:0/345626630 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f26100710f0 0x7f26100735b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2624101990 tx=0x7f260c009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:37.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.095+0000 7f2620ff9700 1 -- 172.21.15.16:0/345626630 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f261405b510 con 0x7f2624101b30 2024-04-02T04:39:37.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.341+0000 7f2628f9f700 1 -- 172.21.15.16:0/345626630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7f262402cda0 con 0x7f2624101b30 2024-04-02T04:39:37.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.829+0000 7f2620ff9700 1 -- 172.21.15.16:0/345626630 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v6) v1 ==== 105+0+0 (secure 0 0 0) 0x7f261405b0a0 con 0x7f2624101b30 2024-04-02T04:39:37.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.832+0000 7f2628f9f700 1 -- 172.21.15.16:0/345626630 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f26100710f0 msgr2=0x7f26100735b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:37.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.832+0000 7f2628f9f700 1 --2- 172.21.15.16:0/345626630 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f26100710f0 0x7f26100735b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2624101990 tx=0x7f260c009260 comp rx=0 tx=0).stop 2024-04-02T04:39:37.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.832+0000 7f2628f9f700 1 -- 172.21.15.16:0/345626630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2624101b30 msgr2=0x7f2624196880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:37.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.832+0000 7f2628f9f700 1 --2- 172.21.15.16:0/345626630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2624101b30 0x7f2624196880 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f261400b800 tx=0x7f2614005f00 comp rx=0 tx=0).stop 2024-04-02T04:39:37.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.832+0000 7f2628f9f700 1 -- 172.21.15.16:0/345626630 shutdown_connections 2024-04-02T04:39:37.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.832+0000 7f2628f9f700 1 --2- 172.21.15.16:0/345626630 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f26100710f0 0x7f26100735b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:37.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.832+0000 7f2628f9f700 1 --2- 172.21.15.16:0/345626630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2624100930 0x7f2624196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:37.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.832+0000 7f2628f9f700 1 --2- 172.21.15.16:0/345626630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2624101b30 0x7f2624196880 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:37.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.832+0000 7f2628f9f700 1 -- 172.21.15.16:0/345626630 >> 172.21.15.16:0/345626630 conn(0x7f26240fbed0 msgr2=0x7f26241050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:37.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.832+0000 7f2628f9f700 1 -- 172.21.15.16:0/345626630 shutdown_connections 2024-04-02T04:39:37.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:37.832+0000 7f2628f9f700 1 -- 172.21.15.16:0/345626630 wait complete. 2024-04-02T04:39:38.139 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:37 smithi016 ceph-mon[28969]: pgmap v180: 65 pgs: 65 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail; 340 B/s wr, 0 op/s 2024-04-02T04:39:38.139 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:37 smithi016 ceph-mon[28969]: mds.? [v2:172.21.15.16:6826/71509234,v1:172.21.15.16:6827/71509234] up:active 2024-04-02T04:39:38.139 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:37 smithi016 ceph-mon[28969]: fsmap cephfs:1 {0=cephfs.smithi016.uhgkgg=up:active} 2024-04-02T04:39:38.139 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:37 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/345626630' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-04-02T04:39:38.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:37 smithi027 ceph-mon[37550]: pgmap v180: 65 pgs: 65 active+clean; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail; 340 B/s wr, 0 op/s 2024-04-02T04:39:38.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:37 smithi027 ceph-mon[37550]: mds.? [v2:172.21.15.16:6826/71509234,v1:172.21.15.16:6827/71509234] up:active 2024-04-02T04:39:38.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:37 smithi027 ceph-mon[37550]: fsmap cephfs:1 {0=cephfs.smithi016.uhgkgg=up:active} 2024-04-02T04:39:38.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:37 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/345626630' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-04-02T04:39:38.474 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-02T04:39:38.484 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi016.front.sepia.ceph.com 2024-04-02T04:39:38.484 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- bash -c 'ceph fs set cephfs allow_standby_replay false' 2024-04-02T04:39:38.704 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:39:39.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:38 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/345626630' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-04-02T04:39:39.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:38 smithi027 ceph-mon[37550]: fsmap cephfs:1 {0=cephfs.smithi016.uhgkgg=up:active} 2024-04-02T04:39:39.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:38 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:39.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:38 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:39.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:38 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:39.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:38 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi016.fyzrnm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:39:39.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:38 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi016.fyzrnm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T04:39:39.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:38 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:39:39.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:38 smithi027 ceph-mon[37550]: mds.? [v2:172.21.15.27:6824/1461455976,v1:172.21.15.27:6825/1461455976] up:boot 2024-04-02T04:39:39.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:38 smithi027 ceph-mon[37550]: fsmap cephfs:1 {0=cephfs.smithi016.uhgkgg=up:active} 1 up:standby 2024-04-02T04:39:39.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:38 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.zwrfoo"}]: dispatch 2024-04-02T04:39:39.150 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:38 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/345626630' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-04-02T04:39:39.150 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:38 smithi016 ceph-mon[28969]: fsmap cephfs:1 {0=cephfs.smithi016.uhgkgg=up:active} 2024-04-02T04:39:39.150 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:38 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:39.150 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:38 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:39.150 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:38 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:39.150 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:38 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi016.fyzrnm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:39:39.150 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:38 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi016.fyzrnm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T04:39:39.150 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:38 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:39:39.150 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:38 smithi016 ceph-mon[28969]: mds.? [v2:172.21.15.27:6824/1461455976,v1:172.21.15.27:6825/1461455976] up:boot 2024-04-02T04:39:39.150 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:38 smithi016 ceph-mon[28969]: fsmap cephfs:1 {0=cephfs.smithi016.uhgkgg=up:active} 1 up:standby 2024-04-02T04:39:39.150 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:38 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.zwrfoo"}]: dispatch 2024-04-02T04:39:39.895 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:39 smithi027 ceph-mon[37550]: Deploying daemon mds.cephfs.smithi016.fyzrnm on smithi016 2024-04-02T04:39:39.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:39 smithi027 ceph-mon[37550]: pgmap v181: 65 pgs: 65 active+clean; 578 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 639 B/s wr, 2 op/s 2024-04-02T04:39:40.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:39 smithi016 ceph-mon[28969]: Deploying daemon mds.cephfs.smithi016.fyzrnm on smithi016 2024-04-02T04:39:40.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:39 smithi016 ceph-mon[28969]: pgmap v181: 65 pgs: 65 active+clean; 578 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 639 B/s wr, 2 op/s 2024-04-02T04:39:40.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:40.933+0000 7f87bb956700 1 -- 172.21.15.16:0/3756139666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b4100bd0 msgr2=0x7f87b4101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:40.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:40.933+0000 7f87bb956700 1 --2- 172.21.15.16:0/3756139666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b4100bd0 0x7f87b4101030 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f87a4009910 tx=0x7f87a4009c20 comp rx=0 tx=0).stop 2024-04-02T04:39:40.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:40.933+0000 7f87bb956700 1 -- 172.21.15.16:0/3756139666 shutdown_connections 2024-04-02T04:39:40.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:40.933+0000 7f87bb956700 1 --2- 172.21.15.16:0/3756139666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b4100bd0 0x7f87b4101030 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:40.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:40.933+0000 7f87bb956700 1 --2- 172.21.15.16:0/3756139666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87b40ff9d0 0x7f87b40ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:40.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:40.933+0000 7f87bb956700 1 -- 172.21.15.16:0/3756139666 >> 172.21.15.16:0/3756139666 conn(0x7f87b40faf70 msgr2=0x7f87b40fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:40.933+0000 7f87bb956700 1 -- 172.21.15.16:0/3756139666 shutdown_connections 2024-04-02T04:39:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:40.933+0000 7f87bb956700 1 -- 172.21.15.16:0/3756139666 wait complete. 2024-04-02T04:39:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:40.933+0000 7f87bb956700 1 Processor -- start 2024-04-02T04:39:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:40.934+0000 7f87bb956700 1 -- start start 2024-04-02T04:39:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:40.934+0000 7f87bb956700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b40ff9d0 0x7f87b4103910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:40.934+0000 7f87bb956700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87b4100bd0 0x7f87b4101fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:40.934+0000 7f87bb956700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b41024f0 con 0x7f87b40ff9d0 2024-04-02T04:39:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:40.934+0000 7f87bb956700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b4102630 con 0x7f87b4100bd0 2024-04-02T04:39:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:40.934+0000 7f87b96f2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b40ff9d0 0x7f87b4103910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:40.934+0000 7f87b96f2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b40ff9d0 0x7f87b4103910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42354/0 (socket says 172.21.15.16:42354) 2024-04-02T04:39:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:40.934+0000 7f87b96f2700 1 -- 172.21.15.16:0/2680569061 learned_addr learned my addr 172.21.15.16:0/2680569061 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:39:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:40.934+0000 7f87b8ef1700 1 --2- 172.21.15.16:0/2680569061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87b4100bd0 0x7f87b4101fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:40.934+0000 7f87b96f2700 1 -- 172.21.15.16:0/2680569061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87b4100bd0 msgr2=0x7f87b4101fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:40.934+0000 7f87b96f2700 1 --2- 172.21.15.16:0/2680569061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87b4100bd0 0x7f87b4101fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:40.934+0000 7f87b96f2700 1 -- 172.21.15.16:0/2680569061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a40095f0 con 0x7f87b40ff9d0 2024-04-02T04:39:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:40.934+0000 7f87b8ef1700 1 --2- 172.21.15.16:0/2680569061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87b4100bd0 0x7f87b4101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T04:39:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:40.934+0000 7f87b96f2700 1 --2- 172.21.15.16:0/2680569061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b40ff9d0 0x7f87b4103910 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f87b000e8f0 tx=0x7f87b000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:40.934+0000 7f87aa7fc700 1 -- 172.21.15.16:0/2680569061 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87b0009790 con 0x7f87b40ff9d0 2024-04-02T04:39:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:40.934+0000 7f87bb956700 1 -- 172.21.15.16:0/2680569061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b4102910 con 0x7f87b40ff9d0 2024-04-02T04:39:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:40.935+0000 7f87aa7fc700 1 -- 172.21.15.16:0/2680569061 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f87b00044e0 con 0x7f87b40ff9d0 2024-04-02T04:39:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:40.935+0000 7f87aa7fc700 1 -- 172.21.15.16:0/2680569061 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87b0003e00 con 0x7f87b40ff9d0 2024-04-02T04:39:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:40.935+0000 7f87bb956700 1 -- 172.21.15.16:0/2680569061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b419e580 con 0x7f87b40ff9d0 2024-04-02T04:39:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:40.936+0000 7f87bb956700 1 -- 172.21.15.16:0/2680569061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b404ea80 con 0x7f87b40ff9d0 2024-04-02T04:39:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:40.936+0000 7f87aa7fc700 1 -- 172.21.15.16:0/2680569061 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f87b000cba0 con 0x7f87b40ff9d0 2024-04-02T04:39:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:40.936+0000 7f87aa7fc700 1 --2- 172.21.15.16:0/2680569061 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f87a006cc80 0x7f87a006f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:40.936+0000 7f87aa7fc700 1 -- 172.21.15.16:0/2680569061 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f87b0014070 con 0x7f87b40ff9d0 2024-04-02T04:39:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:40.937+0000 7f87b8ef1700 1 --2- 172.21.15.16:0/2680569061 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f87a006cc80 0x7f87a006f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:40.937+0000 7f87b8ef1700 1 --2- 172.21.15.16:0/2680569061 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f87a006cc80 0x7f87a006f140 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f87a400b650 tx=0x7f87a400b5c0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:40.941+0000 7f87aa7fc700 1 -- 172.21.15.16:0/2680569061 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f87b005a470 con 0x7f87b40ff9d0 2024-04-02T04:39:41.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:41.190+0000 7f87bb956700 1 -- 172.21.15.16:0/2680569061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f87b4108530 con 0x7f87b40ff9d0 2024-04-02T04:39:41.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:41.834+0000 7f87aa7fc700 1 -- 172.21.15.16:0/2680569061 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f87b005a000 con 0x7f87b40ff9d0 2024-04-02T04:39:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:41.836+0000 7f87bb956700 1 -- 172.21.15.16:0/2680569061 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f87a006cc80 msgr2=0x7f87a006f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:41.837+0000 7f87bb956700 1 --2- 172.21.15.16:0/2680569061 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f87a006cc80 0x7f87a006f140 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f87a400b650 tx=0x7f87a400b5c0 comp rx=0 tx=0).stop 2024-04-02T04:39:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:41.837+0000 7f87bb956700 1 -- 172.21.15.16:0/2680569061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b40ff9d0 msgr2=0x7f87b4103910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:41.837+0000 7f87bb956700 1 --2- 172.21.15.16:0/2680569061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b40ff9d0 0x7f87b4103910 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f87b000e8f0 tx=0x7f87b000ecb0 comp rx=0 tx=0).stop 2024-04-02T04:39:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:41.837+0000 7f87bb956700 1 -- 172.21.15.16:0/2680569061 shutdown_connections 2024-04-02T04:39:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:41.837+0000 7f87bb956700 1 --2- 172.21.15.16:0/2680569061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b40ff9d0 0x7f87b4103910 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:41.837+0000 7f87bb956700 1 --2- 172.21.15.16:0/2680569061 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f87a006cc80 0x7f87a006f140 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:41.837+0000 7f87bb956700 1 --2- 172.21.15.16:0/2680569061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87b4100bd0 0x7f87b4101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:41.837+0000 7f87bb956700 1 -- 172.21.15.16:0/2680569061 >> 172.21.15.16:0/2680569061 conn(0x7f87b40faf70 msgr2=0x7f87b4068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:41.837+0000 7f87bb956700 1 -- 172.21.15.16:0/2680569061 shutdown_connections 2024-04-02T04:39:41.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:41.837+0000 7f87bb956700 1 -- 172.21.15.16:0/2680569061 wait complete. 2024-04-02T04:39:42.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:41 smithi016 ceph-mon[28969]: pgmap v182: 65 pgs: 65 active+clean; 579 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s wr, 4 op/s 2024-04-02T04:39:42.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:41 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/2680569061' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-04-02T04:39:42.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:41 smithi027 ceph-mon[37550]: pgmap v182: 65 pgs: 65 active+clean; 579 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s wr, 4 op/s 2024-04-02T04:39:42.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:41 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/2680569061' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-04-02T04:39:42.581 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-02T04:39:42.591 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi016.front.sepia.ceph.com 2024-04-02T04:39:42.592 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- bash -c 'ceph fs set cephfs inline_data true --yes-i-really-really-mean-it' 2024-04-02T04:39:43.114 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:42 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/2680569061' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-04-02T04:39:43.114 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:42 smithi016 ceph-mon[28969]: fsmap cephfs:1 {0=cephfs.smithi016.uhgkgg=up:active} 1 up:standby 2024-04-02T04:39:43.114 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:42 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:39:43.114 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:42 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:43.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:42 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/2680569061' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-04-02T04:39:43.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:42 smithi027 ceph-mon[37550]: fsmap cephfs:1 {0=cephfs.smithi016.uhgkgg=up:active} 1 up:standby 2024-04-02T04:39:43.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:42 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:39:43.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:42 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:43.153 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:39:43.962 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:43 smithi027 ceph-mon[37550]: pgmap v183: 65 pgs: 65 active+clean; 579 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-02T04:39:43.962 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:43 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:43.962 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:43 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:43.962 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:43 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:43.962 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:43 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi027.sdublg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:39:43.962 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:43 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi027.sdublg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T04:39:43.962 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:43 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:39:44.145 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:43 smithi016 ceph-mon[28969]: pgmap v183: 65 pgs: 65 active+clean; 579 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-02T04:39:44.145 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:43 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:44.146 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:43 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:44.146 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:43 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:44.146 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:43 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi027.sdublg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:39:44.146 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:43 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi027.sdublg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T04:39:44.146 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:43 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:39:44.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:44.629+0000 7f21ec865700 1 -- 172.21.15.16:0/1403220701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e8101ce0 msgr2=0x7f21e8102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:44.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:44.629+0000 7f21ec865700 1 --2- 172.21.15.16:0/1403220701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e8101ce0 0x7f21e8102140 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f21dc009910 tx=0x7f21dc009c20 comp rx=0 tx=0).stop 2024-04-02T04:39:44.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:44.629+0000 7f21ec865700 1 -- 172.21.15.16:0/1403220701 shutdown_connections 2024-04-02T04:39:44.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:44.629+0000 7f21ec865700 1 --2- 172.21.15.16:0/1403220701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e8101ce0 0x7f21e8102140 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:44.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:44.629+0000 7f21ec865700 1 --2- 172.21.15.16:0/1403220701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21e8100ae0 0x7f21e8100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:44.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:44.629+0000 7f21ec865700 1 -- 172.21.15.16:0/1403220701 >> 172.21.15.16:0/1403220701 conn(0x7f21e80fc0a0 msgr2=0x7f21e80fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:44.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:44.629+0000 7f21ec865700 1 -- 172.21.15.16:0/1403220701 shutdown_connections 2024-04-02T04:39:44.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:44.629+0000 7f21ec865700 1 -- 172.21.15.16:0/1403220701 wait complete. 2024-04-02T04:39:44.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:44.630+0000 7f21ec865700 1 Processor -- start 2024-04-02T04:39:44.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:44.630+0000 7f21ec865700 1 -- start start 2024-04-02T04:39:44.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:44.630+0000 7f21ec865700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21e8100ae0 0x7f21e81964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:44.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:44.630+0000 7f21ec865700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e8101ce0 0x7f21e8196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:44.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:44.630+0000 7f21ec865700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e8197020 con 0x7f21e8101ce0 2024-04-02T04:39:44.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:44.630+0000 7f21ec865700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e8197160 con 0x7f21e8100ae0 2024-04-02T04:39:44.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:44.630+0000 7f21e5d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e8101ce0 0x7f21e8196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:44.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:44.630+0000 7f21e5d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e8101ce0 0x7f21e8196a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50970/0 (socket says 172.21.15.16:50970) 2024-04-02T04:39:44.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:44.630+0000 7f21e5d9b700 1 -- 172.21.15.16:0/3485429139 learned_addr learned my addr 172.21.15.16:0/3485429139 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:39:44.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:44.630+0000 7f21e659c700 1 --2- 172.21.15.16:0/3485429139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21e8100ae0 0x7f21e81964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:44.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:44.631+0000 7f21e5d9b700 1 -- 172.21.15.16:0/3485429139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21e8100ae0 msgr2=0x7f21e81964c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:44.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:44.631+0000 7f21e5d9b700 1 --2- 172.21.15.16:0/3485429139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21e8100ae0 0x7f21e81964c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:44.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:44.631+0000 7f21e5d9b700 1 -- 172.21.15.16:0/3485429139 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21dc0095f0 con 0x7f21e8101ce0 2024-04-02T04:39:44.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:44.631+0000 7f21e5d9b700 1 --2- 172.21.15.16:0/3485429139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e8101ce0 0x7f21e8196a00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f21dc0092e0 tx=0x7f21dc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:44.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:44.631+0000 7f21d77fe700 1 -- 172.21.15.16:0/3485429139 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21dc01d070 con 0x7f21e8101ce0 2024-04-02T04:39:44.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:44.631+0000 7f21d77fe700 1 -- 172.21.15.16:0/3485429139 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f21dc00f460 con 0x7f21e8101ce0 2024-04-02T04:39:44.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:44.631+0000 7f21ec865700 1 -- 172.21.15.16:0/3485429139 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e819bb60 con 0x7f21e8101ce0 2024-04-02T04:39:44.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:44.631+0000 7f21d77fe700 1 -- 172.21.15.16:0/3485429139 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21dc021640 con 0x7f21e8101ce0 2024-04-02T04:39:44.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:44.631+0000 7f21ec865700 1 -- 172.21.15.16:0/3485429139 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e819c0d0 con 0x7f21e8101ce0 2024-04-02T04:39:44.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:44.632+0000 7f21ec865700 1 -- 172.21.15.16:0/3485429139 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e8190660 con 0x7f21e8101ce0 2024-04-02T04:39:44.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:44.633+0000 7f21d77fe700 1 -- 172.21.15.16:0/3485429139 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f21dc00f5d0 con 0x7f21e8101ce0 2024-04-02T04:39:44.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:44.633+0000 7f21d77fe700 1 --2- 172.21.15.16:0/3485429139 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f21d006cce0 0x7f21d006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:44.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:44.633+0000 7f21d77fe700 1 -- 172.21.15.16:0/3485429139 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f21dc08d540 con 0x7f21e8101ce0 2024-04-02T04:39:44.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:44.633+0000 7f21e659c700 1 --2- 172.21.15.16:0/3485429139 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f21d006cce0 0x7f21d006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:44.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:44.634+0000 7f21e659c700 1 --2- 172.21.15.16:0/3485429139 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f21d006cce0 0x7f21d006f1a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f21e8101b40 tx=0x7f21d800a380 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:44.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:44.638+0000 7f21d77fe700 1 -- 172.21.15.16:0/3485429139 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f21dc05b3a0 con 0x7f21e8101ce0 2024-04-02T04:39:44.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:44.890+0000 7f21ec865700 1 -- 172.21.15.16:0/3485429139 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f21e804ead0 con 0x7f21e8101ce0 2024-04-02T04:39:44.980 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:44 smithi027 ceph-mon[37550]: Deploying daemon mds.cephfs.smithi027.sdublg on smithi027 2024-04-02T04:39:44.980 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:44 smithi027 ceph-mon[37550]: mds.? [v2:172.21.15.16:6828/2578719859,v1:172.21.15.16:6829/2578719859] up:boot 2024-04-02T04:39:44.980 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:44 smithi027 ceph-mon[37550]: fsmap cephfs:1 {0=cephfs.smithi016.uhgkgg=up:active} 2 up:standby 2024-04-02T04:39:44.980 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:44 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi016.fyzrnm"}]: dispatch 2024-04-02T04:39:45.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:44 smithi016 ceph-mon[28969]: Deploying daemon mds.cephfs.smithi027.sdublg on smithi027 2024-04-02T04:39:45.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:44 smithi016 ceph-mon[28969]: mds.? [v2:172.21.15.16:6828/2578719859,v1:172.21.15.16:6829/2578719859] up:boot 2024-04-02T04:39:45.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:44 smithi016 ceph-mon[28969]: fsmap cephfs:1 {0=cephfs.smithi016.uhgkgg=up:active} 2 up:standby 2024-04-02T04:39:45.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:44 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi016.fyzrnm"}]: dispatch 2024-04-02T04:39:45.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:45.857+0000 7f21d77fe700 1 -- 172.21.15.16:0/3485429139 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]=0 inline data enabled v10) v1 ==== 168+0+0 (secure 0 0 0) 0x7f21dc05af30 con 0x7f21e8101ce0 2024-04-02T04:39:45.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:45.860+0000 7f21ec865700 1 -- 172.21.15.16:0/3485429139 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f21d006cce0 msgr2=0x7f21d006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:45.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:45.860+0000 7f21ec865700 1 --2- 172.21.15.16:0/3485429139 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f21d006cce0 0x7f21d006f1a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f21e8101b40 tx=0x7f21d800a380 comp rx=0 tx=0).stop 2024-04-02T04:39:45.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:45.860+0000 7f21ec865700 1 -- 172.21.15.16:0/3485429139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e8101ce0 msgr2=0x7f21e8196a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:45.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:45.860+0000 7f21ec865700 1 --2- 172.21.15.16:0/3485429139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e8101ce0 0x7f21e8196a00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f21dc0092e0 tx=0x7f21dc00bd70 comp rx=0 tx=0).stop 2024-04-02T04:39:45.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:45.860+0000 7f21ec865700 1 -- 172.21.15.16:0/3485429139 shutdown_connections 2024-04-02T04:39:45.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:45.860+0000 7f21ec865700 1 --2- 172.21.15.16:0/3485429139 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f21d006cce0 0x7f21d006f1a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:45.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:45.860+0000 7f21ec865700 1 --2- 172.21.15.16:0/3485429139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21e8100ae0 0x7f21e81964c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:45.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:45.860+0000 7f21ec865700 1 --2- 172.21.15.16:0/3485429139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e8101ce0 0x7f21e8196a00 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:45.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:45.860+0000 7f21ec865700 1 -- 172.21.15.16:0/3485429139 >> 172.21.15.16:0/3485429139 conn(0x7f21e80fc0a0 msgr2=0x7f21e8105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:45.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:45.860+0000 7f21ec865700 1 -- 172.21.15.16:0/3485429139 shutdown_connections 2024-04-02T04:39:45.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:45.860+0000 7f21ec865700 1 -- 172.21.15.16:0/3485429139 wait complete. 2024-04-02T04:39:45.865 INFO:teuthology.orchestra.run.smithi016.stderr:inline data enabled 2024-04-02T04:39:45.895 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:45 smithi027 ceph-mon[37550]: pgmap v184: 65 pgs: 65 active+clean; 579 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-02T04:39:45.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:45 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/3485429139' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T04:39:46.211 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:45 smithi016 ceph-mon[28969]: pgmap v184: 65 pgs: 65 active+clean; 579 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-02T04:39:46.211 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:45 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/3485429139' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T04:39:46.557 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-02T04:39:46.568 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi016.front.sepia.ceph.com 2024-04-02T04:39:46.568 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- bash -c 'ceph fs dump' 2024-04-02T04:39:47.115 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:46 smithi027 ceph-mon[37550]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-04-02T04:39:47.115 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:46 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/3485429139' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-04-02T04:39:47.116 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:46 smithi027 ceph-mon[37550]: fsmap cephfs:1 {0=cephfs.smithi016.uhgkgg=up:active} 2 up:standby 2024-04-02T04:39:47.116 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:46 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:47.116 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:46 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:47.116 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:46 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:47.116 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:46 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:47.116 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:46 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:47.116 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:46 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:39:47.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:46 smithi016 ceph-mon[28969]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-04-02T04:39:47.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:46 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/3485429139' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-04-02T04:39:47.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:46 smithi016 ceph-mon[28969]: fsmap cephfs:1 {0=cephfs.smithi016.uhgkgg=up:active} 2 up:standby 2024-04-02T04:39:47.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:46 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:47.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:46 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:47.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:46 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:47.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:46 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:47.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:46 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:47.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:46 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:39:47.400 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:39:47.966 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:47 smithi027 ceph-mon[37550]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-02T04:39:47.967 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:47 smithi027 ceph-mon[37550]: mds.? [v2:172.21.15.27:6826/4177192740,v1:172.21.15.27:6827/4177192740] up:boot 2024-04-02T04:39:47.967 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:47 smithi027 ceph-mon[37550]: mds.? [v2:172.21.15.27:6824/1461455976,v1:172.21.15.27:6825/1461455976] up:standby 2024-04-02T04:39:47.967 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:47 smithi027 ceph-mon[37550]: Dropping low affinity active daemon mds.cephfs.smithi016.uhgkgg in favor of higher affinity standby. 2024-04-02T04:39:47.967 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:47 smithi027 ceph-mon[37550]: Replacing daemon mds.cephfs.smithi016.uhgkgg as rank 0 with standby daemon mds.cephfs.smithi027.zwrfoo 2024-04-02T04:39:47.967 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:47 smithi027 ceph-mon[37550]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-02T04:39:47.967 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:47 smithi027 ceph-mon[37550]: fsmap cephfs:1 {0=cephfs.smithi016.uhgkgg=up:active} 3 up:standby 2024-04-02T04:39:47.967 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:47 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.sdublg"}]: dispatch 2024-04-02T04:39:47.967 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:47 smithi027 ceph-mon[37550]: osdmap e46: 6 total, 6 up, 6 in 2024-04-02T04:39:47.967 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:47 smithi027 ceph-mon[37550]: fsmap cephfs:1/1 {0=cephfs.smithi027.zwrfoo=up:replay} 2 up:standby 2024-04-02T04:39:47.967 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:47 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:48.046 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:47 smithi016 ceph-mon[28969]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-02T04:39:48.046 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:47 smithi016 ceph-mon[28969]: mds.? [v2:172.21.15.27:6826/4177192740,v1:172.21.15.27:6827/4177192740] up:boot 2024-04-02T04:39:48.046 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:47 smithi016 ceph-mon[28969]: mds.? [v2:172.21.15.27:6824/1461455976,v1:172.21.15.27:6825/1461455976] up:standby 2024-04-02T04:39:48.046 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:47 smithi016 ceph-mon[28969]: Dropping low affinity active daemon mds.cephfs.smithi016.uhgkgg in favor of higher affinity standby. 2024-04-02T04:39:48.047 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:47 smithi016 ceph-mon[28969]: Replacing daemon mds.cephfs.smithi016.uhgkgg as rank 0 with standby daemon mds.cephfs.smithi027.zwrfoo 2024-04-02T04:39:48.047 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:47 smithi016 ceph-mon[28969]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-02T04:39:48.047 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:47 smithi016 ceph-mon[28969]: fsmap cephfs:1 {0=cephfs.smithi016.uhgkgg=up:active} 3 up:standby 2024-04-02T04:39:48.047 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:47 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.sdublg"}]: dispatch 2024-04-02T04:39:48.047 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:47 smithi016 ceph-mon[28969]: osdmap e46: 6 total, 6 up, 6 in 2024-04-02T04:39:48.047 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:47 smithi016 ceph-mon[28969]: fsmap cephfs:1/1 {0=cephfs.smithi027.zwrfoo=up:replay} 2 up:standby 2024-04-02T04:39:48.047 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:47 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:48.971 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:48 smithi016 ceph-mon[28969]: mds.? [v2:172.21.15.27:6824/1461455976,v1:172.21.15.27:6825/1461455976] up:reconnect 2024-04-02T04:39:48.971 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:48 smithi016 ceph-mon[28969]: mds.? [v2:172.21.15.16:6826/601263579,v1:172.21.15.16:6827/601263579] up:boot 2024-04-02T04:39:48.971 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:48 smithi016 ceph-mon[28969]: mds.? [v2:172.21.15.16:6828/2578719859,v1:172.21.15.16:6829/2578719859] up:standby 2024-04-02T04:39:48.971 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:48 smithi016 ceph-mon[28969]: fsmap cephfs:1/1 {0=cephfs.smithi027.zwrfoo=up:reconnect} 3 up:standby 2024-04-02T04:39:48.971 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:48 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi016.uhgkgg"}]: dispatch 2024-04-02T04:39:48.972 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:48 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:48.972 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:48 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:49.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:48 smithi027 ceph-mon[37550]: mds.? [v2:172.21.15.27:6824/1461455976,v1:172.21.15.27:6825/1461455976] up:reconnect 2024-04-02T04:39:49.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:48 smithi027 ceph-mon[37550]: mds.? [v2:172.21.15.16:6826/601263579,v1:172.21.15.16:6827/601263579] up:boot 2024-04-02T04:39:49.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:48 smithi027 ceph-mon[37550]: mds.? [v2:172.21.15.16:6828/2578719859,v1:172.21.15.16:6829/2578719859] up:standby 2024-04-02T04:39:49.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:48 smithi027 ceph-mon[37550]: fsmap cephfs:1/1 {0=cephfs.smithi027.zwrfoo=up:reconnect} 3 up:standby 2024-04-02T04:39:49.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:48 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi016.uhgkgg"}]: dispatch 2024-04-02T04:39:49.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:48 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:49.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:48 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:49.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.512+0000 7f3b8cdc2700 1 -- 172.21.15.16:0/3752600594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b88101ce0 msgr2=0x7f3b88102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:49.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.512+0000 7f3b8cdc2700 1 --2- 172.21.15.16:0/3752600594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b88101ce0 0x7f3b88102140 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3b78009910 tx=0x7f3b78009c20 comp rx=0 tx=0).stop 2024-04-02T04:39:49.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.512+0000 7f3b8cdc2700 1 -- 172.21.15.16:0/3752600594 shutdown_connections 2024-04-02T04:39:49.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.512+0000 7f3b8cdc2700 1 --2- 172.21.15.16:0/3752600594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b88101ce0 0x7f3b88102140 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:49.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.512+0000 7f3b8cdc2700 1 --2- 172.21.15.16:0/3752600594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b88100ae0 0x7f3b88100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:49.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.512+0000 7f3b8cdc2700 1 -- 172.21.15.16:0/3752600594 >> 172.21.15.16:0/3752600594 conn(0x7f3b880fc0a0 msgr2=0x7f3b880fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:49.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.512+0000 7f3b8cdc2700 1 -- 172.21.15.16:0/3752600594 shutdown_connections 2024-04-02T04:39:49.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.512+0000 7f3b8cdc2700 1 -- 172.21.15.16:0/3752600594 wait complete. 2024-04-02T04:39:49.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.513+0000 7f3b8cdc2700 1 Processor -- start 2024-04-02T04:39:49.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.513+0000 7f3b8cdc2700 1 -- start start 2024-04-02T04:39:49.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.513+0000 7f3b8cdc2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b88100ae0 0x7f3b881964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:49.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.513+0000 7f3b8cdc2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b88101ce0 0x7f3b88196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:49.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.513+0000 7f3b8cdc2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b88197020 con 0x7f3b88100ae0 2024-04-02T04:39:49.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.513+0000 7f3b8cdc2700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b88197160 con 0x7f3b88101ce0 2024-04-02T04:39:49.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.513+0000 7f3b8659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b88100ae0 0x7f3b881964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:49.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.513+0000 7f3b8659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b88100ae0 0x7f3b881964c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51004/0 (socket says 172.21.15.16:51004) 2024-04-02T04:39:49.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.513+0000 7f3b8659c700 1 -- 172.21.15.16:0/1097602012 learned_addr learned my addr 172.21.15.16:0/1097602012 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:39:49.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.513+0000 7f3b85d9b700 1 --2- 172.21.15.16:0/1097602012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b88101ce0 0x7f3b88196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:49.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.514+0000 7f3b8659c700 1 -- 172.21.15.16:0/1097602012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b88101ce0 msgr2=0x7f3b88196a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:49.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.514+0000 7f3b8659c700 1 --2- 172.21.15.16:0/1097602012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b88101ce0 0x7f3b88196a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:49.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.514+0000 7f3b8659c700 1 -- 172.21.15.16:0/1097602012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b780095f0 con 0x7f3b88100ae0 2024-04-02T04:39:49.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.514+0000 7f3b8659c700 1 --2- 172.21.15.16:0/1097602012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b88100ae0 0x7f3b881964c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3b7000d900 tx=0x7f3b7000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:49.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.514+0000 7f3b7f7fe700 1 -- 172.21.15.16:0/1097602012 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b7000b920 con 0x7f3b88100ae0 2024-04-02T04:39:49.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.514+0000 7f3b7f7fe700 1 -- 172.21.15.16:0/1097602012 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3b70010460 con 0x7f3b88100ae0 2024-04-02T04:39:49.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.514+0000 7f3b8cdc2700 1 -- 172.21.15.16:0/1097602012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b8819bbc0 con 0x7f3b88100ae0 2024-04-02T04:39:49.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.514+0000 7f3b7f7fe700 1 -- 172.21.15.16:0/1097602012 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b7000f590 con 0x7f3b88100ae0 2024-04-02T04:39:49.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.514+0000 7f3b8cdc2700 1 -- 172.21.15.16:0/1097602012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b8819c110 con 0x7f3b88100ae0 2024-04-02T04:39:49.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.515+0000 7f3b7d7fa700 1 -- 172.21.15.16:0/1097602012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b6c0052f0 con 0x7f3b88100ae0 2024-04-02T04:39:49.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.516+0000 7f3b7f7fe700 1 -- 172.21.15.16:0/1097602012 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f3b700105d0 con 0x7f3b88100ae0 2024-04-02T04:39:49.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.516+0000 7f3b7f7fe700 1 --2- 172.21.15.16:0/1097602012 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f3b7406cce0 0x7f3b7406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:49.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.517+0000 7f3b7f7fe700 1 -- 172.21.15.16:0/1097602012 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f3b7008bfd0 con 0x7f3b88100ae0 2024-04-02T04:39:49.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.517+0000 7f3b85d9b700 1 --2- 172.21.15.16:0/1097602012 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f3b7406cce0 0x7f3b7406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:49.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.517+0000 7f3b85d9b700 1 --2- 172.21.15.16:0/1097602012 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f3b7406cce0 0x7f3b7406f1a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3b78009fd0 tx=0x7f3b78005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:49.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.525+0000 7f3b7f7fe700 1 -- 172.21.15.16:0/1097602012 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3b70059d30 con 0x7f3b88100ae0 2024-04-02T04:39:49.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.780+0000 7f3b7d7fa700 1 -- 172.21.15.16:0/1097602012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3b6c005db0 con 0x7f3b88100ae0 2024-04-02T04:39:49.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.781+0000 7f3b7f7fe700 1 -- 172.21.15.16:0/1097602012 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1830 (secure 0 0 0) 0x7f3b70016740 con 0x7f3b88100ae0 2024-04-02T04:39:49.784 INFO:teuthology.orchestra.run.smithi016.stdout:e14 2024-04-02T04:39:49.784 INFO:teuthology.orchestra.run.smithi016.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:39:49.784 INFO:teuthology.orchestra.run.smithi016.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:39:49.785 INFO:teuthology.orchestra.run.smithi016.stdout:legacy client fscid: 1 2024-04-02T04:39:49.785 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:39:49.785 INFO:teuthology.orchestra.run.smithi016.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:39:49.785 INFO:teuthology.orchestra.run.smithi016.stdout:fs_name cephfs 2024-04-02T04:39:49.785 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 14 2024-04-02T04:39:49.785 INFO:teuthology.orchestra.run.smithi016.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:39:49.785 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-04-02T04:39:30.740168+0000 2024-04-02T04:39:49.785 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-04-02T04:39:48.869311+0000 2024-04-02T04:39:49.785 INFO:teuthology.orchestra.run.smithi016.stdout:tableserver 0 2024-04-02T04:39:49.785 INFO:teuthology.orchestra.run.smithi016.stdout:root 0 2024-04-02T04:39:49.785 INFO:teuthology.orchestra.run.smithi016.stdout:session_timeout 60 2024-04-02T04:39:49.785 INFO:teuthology.orchestra.run.smithi016.stdout:session_autoclose 300 2024-04-02T04:39:49.785 INFO:teuthology.orchestra.run.smithi016.stdout:max_file_size 1099511627776 2024-04-02T04:39:49.785 INFO:teuthology.orchestra.run.smithi016.stdout:max_xattr_size 65536 2024-04-02T04:39:49.785 INFO:teuthology.orchestra.run.smithi016.stdout:required_client_features {} 2024-04-02T04:39:49.786 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure 0 2024-04-02T04:39:49.786 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure_osd_epoch 46 2024-04-02T04:39:49.786 INFO:teuthology.orchestra.run.smithi016.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:39:49.786 INFO:teuthology.orchestra.run.smithi016.stdout:max_mds 1 2024-04-02T04:39:49.786 INFO:teuthology.orchestra.run.smithi016.stdout:in 0 2024-04-02T04:39:49.786 INFO:teuthology.orchestra.run.smithi016.stdout:up {0=24285} 2024-04-02T04:39:49.786 INFO:teuthology.orchestra.run.smithi016.stdout:failed 2024-04-02T04:39:49.786 INFO:teuthology.orchestra.run.smithi016.stdout:damaged 2024-04-02T04:39:49.786 INFO:teuthology.orchestra.run.smithi016.stdout:stopped 2024-04-02T04:39:49.786 INFO:teuthology.orchestra.run.smithi016.stdout:data_pools [3] 2024-04-02T04:39:49.786 INFO:teuthology.orchestra.run.smithi016.stdout:metadata_pool 2 2024-04-02T04:39:49.786 INFO:teuthology.orchestra.run.smithi016.stdout:inline_data enabled 2024-04-02T04:39:49.786 INFO:teuthology.orchestra.run.smithi016.stdout:balancer 2024-04-02T04:39:49.786 INFO:teuthology.orchestra.run.smithi016.stdout:bal_rank_mask -1 2024-04-02T04:39:49.787 INFO:teuthology.orchestra.run.smithi016.stdout:standby_count_wanted 1 2024-04-02T04:39:49.787 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.zwrfoo{0:24285} state up:rejoin seq 5 join_fscid=1 addr [v2:172.21.15.27:6824/1461455976,v1:172.21.15.27:6825/1461455976] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:39:49.787 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:39:49.787 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:39:49.787 INFO:teuthology.orchestra.run.smithi016.stdout:Standby daemons: 2024-04-02T04:39:49.787 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:39:49.787 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.fyzrnm{-1:14488} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.16:6828/2578719859,v1:172.21.15.16:6829/2578719859] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:39:49.787 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.uhgkgg{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.16:6826/601263579,v1:172.21.15.16:6827/601263579] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:39:49.787 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.sdublg{-1:24295} state up:standby seq 1 addr [v2:172.21.15.27:6826/4177192740,v1:172.21.15.27:6827/4177192740] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:39:49.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.784+0000 7f3b7d7fa700 1 -- 172.21.15.16:0/1097602012 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f3b7406cce0 msgr2=0x7f3b7406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:49.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.784+0000 7f3b7d7fa700 1 --2- 172.21.15.16:0/1097602012 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f3b7406cce0 0x7f3b7406f1a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3b78009fd0 tx=0x7f3b78005960 comp rx=0 tx=0).stop 2024-04-02T04:39:49.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.784+0000 7f3b7d7fa700 1 -- 172.21.15.16:0/1097602012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b88100ae0 msgr2=0x7f3b881964c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:49.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.784+0000 7f3b7d7fa700 1 --2- 172.21.15.16:0/1097602012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b88100ae0 0x7f3b881964c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3b7000d900 tx=0x7f3b7000dc10 comp rx=0 tx=0).stop 2024-04-02T04:39:49.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.784+0000 7f3b7d7fa700 1 -- 172.21.15.16:0/1097602012 shutdown_connections 2024-04-02T04:39:49.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.784+0000 7f3b7d7fa700 1 --2- 172.21.15.16:0/1097602012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b88100ae0 0x7f3b881964c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:49.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.784+0000 7f3b7d7fa700 1 --2- 172.21.15.16:0/1097602012 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f3b7406cce0 0x7f3b7406f1a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:49.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.784+0000 7f3b7d7fa700 1 --2- 172.21.15.16:0/1097602012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b88101ce0 0x7f3b88196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:49.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.784+0000 7f3b7d7fa700 1 -- 172.21.15.16:0/1097602012 >> 172.21.15.16:0/1097602012 conn(0x7f3b880fc0a0 msgr2=0x7f3b88105280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:49.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.784+0000 7f3b7d7fa700 1 -- 172.21.15.16:0/1097602012 shutdown_connections 2024-04-02T04:39:49.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:49.784+0000 7f3b7d7fa700 1 -- 172.21.15.16:0/1097602012 wait complete. 2024-04-02T04:39:49.789 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 14 2024-04-02T04:39:50.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:49 smithi016 ceph-mon[28969]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 511 B/s rd, 921 B/s wr, 3 op/s 2024-04-02T04:39:50.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:49 smithi016 ceph-mon[28969]: mds.? [v2:172.21.15.27:6824/1461455976,v1:172.21.15.27:6825/1461455976] up:rejoin 2024-04-02T04:39:50.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:49 smithi016 ceph-mon[28969]: fsmap cephfs:1/1 {0=cephfs.smithi027.zwrfoo=up:rejoin} 3 up:standby 2024-04-02T04:39:50.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:49 smithi016 ceph-mon[28969]: daemon mds.cephfs.smithi027.zwrfoo is now active in filesystem cephfs as rank 0 2024-04-02T04:39:50.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:49 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/1097602012' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:39:50.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:49 smithi027 ceph-mon[37550]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 511 B/s rd, 921 B/s wr, 3 op/s 2024-04-02T04:39:50.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:49 smithi027 ceph-mon[37550]: mds.? [v2:172.21.15.27:6824/1461455976,v1:172.21.15.27:6825/1461455976] up:rejoin 2024-04-02T04:39:50.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:49 smithi027 ceph-mon[37550]: fsmap cephfs:1/1 {0=cephfs.smithi027.zwrfoo=up:rejoin} 3 up:standby 2024-04-02T04:39:50.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:49 smithi027 ceph-mon[37550]: daemon mds.cephfs.smithi027.zwrfoo is now active in filesystem cephfs as rank 0 2024-04-02T04:39:50.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:49 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/1097602012' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:39:50.414 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-04-02T04:39:50.811 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:39:51.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:50 smithi027 ceph-mon[37550]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-02T04:39:51.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:50 smithi027 ceph-mon[37550]: mds.? [v2:172.21.15.27:6824/1461455976,v1:172.21.15.27:6825/1461455976] up:active 2024-04-02T04:39:51.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:50 smithi027 ceph-mon[37550]: mds.? [v2:172.21.15.27:6826/4177192740,v1:172.21.15.27:6827/4177192740] up:standby 2024-04-02T04:39:51.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:50 smithi027 ceph-mon[37550]: fsmap cephfs:1 {0=cephfs.smithi027.zwrfoo=up:active} 3 up:standby 2024-04-02T04:39:51.159 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:50 smithi016 ceph-mon[28969]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-02T04:39:51.159 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:50 smithi016 ceph-mon[28969]: mds.? [v2:172.21.15.27:6824/1461455976,v1:172.21.15.27:6825/1461455976] up:active 2024-04-02T04:39:51.159 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:50 smithi016 ceph-mon[28969]: mds.? [v2:172.21.15.27:6826/4177192740,v1:172.21.15.27:6827/4177192740] up:standby 2024-04-02T04:39:51.159 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:50 smithi016 ceph-mon[28969]: fsmap cephfs:1 {0=cephfs.smithi027.zwrfoo=up:active} 3 up:standby 2024-04-02T04:39:51.949 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:51 smithi016 ceph-mon[28969]: pgmap v188: 65 pgs: 65 active+clean; 579 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 819 B/s rd, 0 B/s wr, 0 op/s 2024-04-02T04:39:52.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:51 smithi027 ceph-mon[37550]: pgmap v188: 65 pgs: 65 active+clean; 579 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 819 B/s rd, 0 B/s wr, 0 op/s 2024-04-02T04:39:53.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.686+0000 7f2a36368700 1 -- 172.21.15.16:0/3712187227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a30103eb0 msgr2=0x7f2a301062a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:53.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.686+0000 7f2a36368700 1 --2- 172.21.15.16:0/3712187227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a30103eb0 0x7f2a301062a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2a24009910 tx=0x7f2a24009c20 comp rx=0 tx=0).stop 2024-04-02T04:39:53.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.687+0000 7f2a36368700 1 -- 172.21.15.16:0/3712187227 shutdown_connections 2024-04-02T04:39:53.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.687+0000 7f2a36368700 1 --2- 172.21.15.16:0/3712187227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a30103eb0 0x7f2a301062a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:53.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.687+0000 7f2a36368700 1 --2- 172.21.15.16:0/3712187227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a30101580 0x7f2a30103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:53.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.687+0000 7f2a36368700 1 -- 172.21.15.16:0/3712187227 >> 172.21.15.16:0/3712187227 conn(0x7f2a300faf90 msgr2=0x7f2a300fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:53.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.687+0000 7f2a36368700 1 -- 172.21.15.16:0/3712187227 shutdown_connections 2024-04-02T04:39:53.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.687+0000 7f2a36368700 1 -- 172.21.15.16:0/3712187227 wait complete. 2024-04-02T04:39:53.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.687+0000 7f2a36368700 1 Processor -- start 2024-04-02T04:39:53.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.687+0000 7f2a36368700 1 -- start start 2024-04-02T04:39:53.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.688+0000 7f2a36368700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a30101580 0x7f2a30100e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:53.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.688+0000 7f2a36368700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a30103eb0 0x7f2a300ff530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:53.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.688+0000 7f2a36368700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a300ffa70 con 0x7f2a30103eb0 2024-04-02T04:39:53.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.688+0000 7f2a36368700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a300ffbb0 con 0x7f2a30101580 2024-04-02T04:39:53.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.688+0000 7f2a2f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a30103eb0 0x7f2a300ff530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:53.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.688+0000 7f2a2f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a30103eb0 0x7f2a300ff530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54540/0 (socket says 172.21.15.16:54540) 2024-04-02T04:39:53.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.688+0000 7f2a2f7fe700 1 -- 172.21.15.16:0/894179329 learned_addr learned my addr 172.21.15.16:0/894179329 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:39:53.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.688+0000 7f2a2ffff700 1 --2- 172.21.15.16:0/894179329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a30101580 0x7f2a30100e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:53.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.688+0000 7f2a2f7fe700 1 -- 172.21.15.16:0/894179329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a30101580 msgr2=0x7f2a30100e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:53.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.688+0000 7f2a2f7fe700 1 --2- 172.21.15.16:0/894179329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a30101580 0x7f2a30100e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:53.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.688+0000 7f2a2f7fe700 1 -- 172.21.15.16:0/894179329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a240095f0 con 0x7f2a30103eb0 2024-04-02T04:39:53.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.688+0000 7f2a2f7fe700 1 --2- 172.21.15.16:0/894179329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a30103eb0 0x7f2a300ff530 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2a24005b40 tx=0x7f2a2400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:53.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.688+0000 7f2a2d7fa700 1 -- 172.21.15.16:0/894179329 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a2401d070 con 0x7f2a30103eb0 2024-04-02T04:39:53.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.688+0000 7f2a36368700 1 -- 172.21.15.16:0/894179329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a300ffe30 con 0x7f2a30103eb0 2024-04-02T04:39:53.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.688+0000 7f2a2d7fa700 1 -- 172.21.15.16:0/894179329 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a24005dc0 con 0x7f2a30103eb0 2024-04-02T04:39:53.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.688+0000 7f2a2d7fa700 1 -- 172.21.15.16:0/894179329 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a24017400 con 0x7f2a30103eb0 2024-04-02T04:39:53.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.688+0000 7f2a36368700 1 -- 172.21.15.16:0/894179329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a301003a0 con 0x7f2a30103eb0 2024-04-02T04:39:53.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.689+0000 7f2a36368700 1 -- 172.21.15.16:0/894179329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a3004ea80 con 0x7f2a30103eb0 2024-04-02T04:39:53.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.690+0000 7f2a2d7fa700 1 -- 172.21.15.16:0/894179329 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f2a2400f6d0 con 0x7f2a30103eb0 2024-04-02T04:39:53.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.690+0000 7f2a2d7fa700 1 --2- 172.21.15.16:0/894179329 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f2a1806ccf0 0x7f2a1806f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:53.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.690+0000 7f2a2d7fa700 1 -- 172.21.15.16:0/894179329 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f2a2408d540 con 0x7f2a30103eb0 2024-04-02T04:39:53.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.690+0000 7f2a2ffff700 1 --2- 172.21.15.16:0/894179329 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f2a1806ccf0 0x7f2a1806f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:53.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.691+0000 7f2a2ffff700 1 --2- 172.21.15.16:0/894179329 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f2a1806ccf0 0x7f2a1806f1b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2a20005950 tx=0x7f2a2000b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:53.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.695+0000 7f2a2d7fa700 1 -- 172.21.15.16:0/894179329 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2a2405b340 con 0x7f2a30103eb0 2024-04-02T04:39:53.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.953+0000 7f2a36368700 1 -- 172.21.15.16:0/894179329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a30109650 con 0x7f2a30103eb0 2024-04-02T04:39:53.957 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:53 smithi016 ceph-mon[28969]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 1 op/s 2024-04-02T04:39:53.957 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:53 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:53.957 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:53 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:53.957 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:53 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:39:53.957 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:53 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:39:53.957 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:53 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:53.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.954+0000 7f2a2d7fa700 1 -- 172.21.15.16:0/894179329 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4822 (secure 0 0 0) 0x7f2a2400f460 con 0x7f2a30103eb0 2024-04-02T04:39:53.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.956+0000 7f2a36368700 1 -- 172.21.15.16:0/894179329 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f2a1806ccf0 msgr2=0x7f2a1806f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:53.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.956+0000 7f2a36368700 1 --2- 172.21.15.16:0/894179329 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f2a1806ccf0 0x7f2a1806f1b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2a20005950 tx=0x7f2a2000b410 comp rx=0 tx=0).stop 2024-04-02T04:39:53.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.956+0000 7f2a36368700 1 -- 172.21.15.16:0/894179329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a30103eb0 msgr2=0x7f2a300ff530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:53.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.956+0000 7f2a36368700 1 --2- 172.21.15.16:0/894179329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a30103eb0 0x7f2a300ff530 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2a24005b40 tx=0x7f2a2400bd70 comp rx=0 tx=0).stop 2024-04-02T04:39:53.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.956+0000 7f2a36368700 1 -- 172.21.15.16:0/894179329 shutdown_connections 2024-04-02T04:39:53.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.956+0000 7f2a36368700 1 --2- 172.21.15.16:0/894179329 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f2a1806ccf0 0x7f2a1806f1b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:53.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.956+0000 7f2a36368700 1 --2- 172.21.15.16:0/894179329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a30101580 0x7f2a30100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:53.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.956+0000 7f2a36368700 1 --2- 172.21.15.16:0/894179329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a30103eb0 0x7f2a300ff530 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:53.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.956+0000 7f2a36368700 1 -- 172.21.15.16:0/894179329 >> 172.21.15.16:0/894179329 conn(0x7f2a300faf90 msgr2=0x7f2a300fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:53.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.956+0000 7f2a36368700 1 -- 172.21.15.16:0/894179329 shutdown_connections 2024-04-02T04:39:53.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:53.956+0000 7f2a36368700 1 -- 172.21.15.16:0/894179329 wait complete. 2024-04-02T04:39:53.960 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 15 2024-04-02T04:39:53.973 INFO:teuthology.orchestra.run.smithi016.stdout:true 2024-04-02T04:39:54.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:53 smithi027 ceph-mon[37550]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 1 op/s 2024-04-02T04:39:54.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:53 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:54.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:53 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:54.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:53 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:39:54.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:53 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:39:54.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:53 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:54.570 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-04-02T04:39:54.798 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:39:55.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:54 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/894179329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T04:39:55.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:54 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:39:55.210 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:54 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/894179329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T04:39:55.210 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:54 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:39:55.956 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:55 smithi016 ceph-mon[28969]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 5.2 KiB/s rd, 4 op/s 2024-04-02T04:39:56.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:55 smithi027 ceph-mon[37550]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 5.2 KiB/s rd, 4 op/s 2024-04-02T04:39:57.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.128+0000 7fcd5b979700 1 -- 172.21.15.16:0/2858590215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd54103ef0 msgr2=0x7fcd541062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:57.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.128+0000 7fcd5b979700 1 --2- 172.21.15.16:0/2858590215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd54103ef0 0x7fcd541062e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcd44009910 tx=0x7fcd44009c20 comp rx=0 tx=0).stop 2024-04-02T04:39:57.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.128+0000 7fcd5b979700 1 -- 172.21.15.16:0/2858590215 shutdown_connections 2024-04-02T04:39:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.128+0000 7fcd5b979700 1 --2- 172.21.15.16:0/2858590215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd54103ef0 0x7fcd541062e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.128+0000 7fcd5b979700 1 --2- 172.21.15.16:0/2858590215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd541015c0 0x7fcd541039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.128+0000 7fcd5b979700 1 -- 172.21.15.16:0/2858590215 >> 172.21.15.16:0/2858590215 conn(0x7fcd540faf70 msgr2=0x7fcd540fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.128+0000 7fcd5b979700 1 -- 172.21.15.16:0/2858590215 shutdown_connections 2024-04-02T04:39:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.128+0000 7fcd5b979700 1 -- 172.21.15.16:0/2858590215 wait complete. 2024-04-02T04:39:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.129+0000 7fcd5b979700 1 Processor -- start 2024-04-02T04:39:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.129+0000 7fcd5b979700 1 -- start start 2024-04-02T04:39:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.129+0000 7fcd5b979700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd541015c0 0x7fcd54196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.129+0000 7fcd5b979700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd54103ef0 0x7fcd541969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.129+0000 7fcd5b979700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd54196fd0 con 0x7fcd54103ef0 2024-04-02T04:39:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.129+0000 7fcd5b979700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd54197110 con 0x7fcd541015c0 2024-04-02T04:39:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.129+0000 7fcd58f14700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd54103ef0 0x7fcd541969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.129+0000 7fcd58f14700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd54103ef0 0x7fcd541969b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54566/0 (socket says 172.21.15.16:54566) 2024-04-02T04:39:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.129+0000 7fcd58f14700 1 -- 172.21.15.16:0/203075663 learned_addr learned my addr 172.21.15.16:0/203075663 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:39:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.129+0000 7fcd59715700 1 --2- 172.21.15.16:0/203075663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd541015c0 0x7fcd54196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.129+0000 7fcd58f14700 1 -- 172.21.15.16:0/203075663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd541015c0 msgr2=0x7fcd54196470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.129+0000 7fcd58f14700 1 --2- 172.21.15.16:0/203075663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd541015c0 0x7fcd54196470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.129+0000 7fcd58f14700 1 -- 172.21.15.16:0/203075663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd440095f0 con 0x7fcd54103ef0 2024-04-02T04:39:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.130+0000 7fcd58f14700 1 --2- 172.21.15.16:0/203075663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd54103ef0 0x7fcd541969b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcd4400b800 tx=0x7fcd4400b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.130+0000 7fcd4a7fc700 1 -- 172.21.15.16:0/203075663 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd4401d070 con 0x7fcd54103ef0 2024-04-02T04:39:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.130+0000 7fcd5b979700 1 -- 172.21.15.16:0/203075663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd5419bb10 con 0x7fcd54103ef0 2024-04-02T04:39:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.130+0000 7fcd4a7fc700 1 -- 172.21.15.16:0/203075663 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcd440052a0 con 0x7fcd54103ef0 2024-04-02T04:39:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.130+0000 7fcd4a7fc700 1 -- 172.21.15.16:0/203075663 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd44022470 con 0x7fcd54103ef0 2024-04-02T04:39:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.130+0000 7fcd5b979700 1 -- 172.21.15.16:0/203075663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd5419c080 con 0x7fcd54103ef0 2024-04-02T04:39:57.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.131+0000 7fcd5b979700 1 -- 172.21.15.16:0/203075663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd5404ea80 con 0x7fcd54103ef0 2024-04-02T04:39:57.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.132+0000 7fcd4a7fc700 1 -- 172.21.15.16:0/203075663 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fcd4400f810 con 0x7fcd54103ef0 2024-04-02T04:39:57.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.133+0000 7fcd4a7fc700 1 --2- 172.21.15.16:0/203075663 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fcd4006cce0 0x7fcd4006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:57.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.133+0000 7fcd4a7fc700 1 -- 172.21.15.16:0/203075663 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fcd4408d700 con 0x7fcd54103ef0 2024-04-02T04:39:57.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.133+0000 7fcd59715700 1 --2- 172.21.15.16:0/203075663 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fcd4006cce0 0x7fcd4006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:57.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.133+0000 7fcd59715700 1 --2- 172.21.15.16:0/203075663 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fcd4006cce0 0x7fcd4006f1a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcd50009c60 tx=0x7fcd50009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:57.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.137+0000 7fcd4a7fc700 1 -- 172.21.15.16:0/203075663 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fcd4405b500 con 0x7fcd54103ef0 2024-04-02T04:39:57.140 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:57 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:57.141 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:57 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:57.141 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:57 smithi016 ceph-mon[28969]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 8.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-02T04:39:57.395 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:57 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:57.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:57 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:57.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:57 smithi027 ceph-mon[37550]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 8.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-02T04:39:57.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.428+0000 7fcd5b979700 1 -- 172.21.15.16:0/203075663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7fcd541097e0 con 0x7fcd54103ef0 2024-04-02T04:39:57.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.428+0000 7fcd4a7fc700 1 -- 172.21.15.16:0/203075663 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v15) v1 ==== 78+0+98 (secure 0 0 0) 0x7fcd4405b090 con 0x7fcd54103ef0 2024-04-02T04:39:57.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.430+0000 7fcd5b979700 1 -- 172.21.15.16:0/203075663 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fcd4006cce0 msgr2=0x7fcd4006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:57.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.430+0000 7fcd5b979700 1 --2- 172.21.15.16:0/203075663 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fcd4006cce0 0x7fcd4006f1a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcd50009c60 tx=0x7fcd50009260 comp rx=0 tx=0).stop 2024-04-02T04:39:57.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.430+0000 7fcd5b979700 1 -- 172.21.15.16:0/203075663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd54103ef0 msgr2=0x7fcd541969b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:57.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.430+0000 7fcd5b979700 1 --2- 172.21.15.16:0/203075663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd54103ef0 0x7fcd541969b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcd4400b800 tx=0x7fcd4400b600 comp rx=0 tx=0).stop 2024-04-02T04:39:57.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.431+0000 7fcd5b979700 1 -- 172.21.15.16:0/203075663 shutdown_connections 2024-04-02T04:39:57.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.431+0000 7fcd5b979700 1 --2- 172.21.15.16:0/203075663 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fcd4006cce0 0x7fcd4006f1a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:57.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.431+0000 7fcd5b979700 1 --2- 172.21.15.16:0/203075663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd541015c0 0x7fcd54196470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:57.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.431+0000 7fcd5b979700 1 --2- 172.21.15.16:0/203075663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd54103ef0 0x7fcd541969b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:57.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.431+0000 7fcd5b979700 1 -- 172.21.15.16:0/203075663 >> 172.21.15.16:0/203075663 conn(0x7fcd540faf70 msgr2=0x7fcd540fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:57.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.431+0000 7fcd5b979700 1 -- 172.21.15.16:0/203075663 shutdown_connections 2024-04-02T04:39:57.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:57.431+0000 7fcd5b979700 1 -- 172.21.15.16:0/203075663 wait complete. 2024-04-02T04:39:57.443 INFO:teuthology.orchestra.run.smithi016.stdout:true 2024-04-02T04:39:58.031 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-04-02T04:39:58.060 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T04:39:58.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.194+0000 7f6d6e950640 1 -- 172.21.15.16:0/3325782776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d680fee70 msgr2=0x7f6d680ff290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:58.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.194+0000 7f6d6e950640 1 --2- 172.21.15.16:0/3325782776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d680fee70 0x7f6d680ff290 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c009880 tx=0x7f6d5c02bce0 comp rx=0 tx=0).stop 2024-04-02T04:39:58.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.194+0000 7f6d6e950640 1 -- 172.21.15.16:0/3325782776 shutdown_connections 2024-04-02T04:39:58.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.194+0000 7f6d6e950640 1 --2- 172.21.15.16:0/3325782776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d68100050 0x7f6d681004d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:58.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.194+0000 7f6d6e950640 1 --2- 172.21.15.16:0/3325782776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d680fee70 0x7f6d680ff290 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:58.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.194+0000 7f6d6e950640 1 -- 172.21.15.16:0/3325782776 >> 172.21.15.16:0/3325782776 conn(0x7f6d680fa450 msgr2=0x7f6d680fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:58.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.194+0000 7f6d6e950640 1 -- 172.21.15.16:0/3325782776 shutdown_connections 2024-04-02T04:39:58.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.194+0000 7f6d6e950640 1 -- 172.21.15.16:0/3325782776 wait complete. 2024-04-02T04:39:58.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.195+0000 7f6d6e950640 1 Processor -- start 2024-04-02T04:39:58.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.195+0000 7f6d6e950640 1 -- start start 2024-04-02T04:39:58.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.195+0000 7f6d6e950640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d680fee70 0x7f6d68068eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:58.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.195+0000 7f6d6e950640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d68100050 0x7f6d6819a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:58.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.195+0000 7f6d6e950640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6819ad80 con 0x7f6d68100050 2024-04-02T04:39:58.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.195+0000 7f6d6e950640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6819aec0 con 0x7f6d680fee70 2024-04-02T04:39:58.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.195+0000 7f6d677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d68100050 0x7f6d6819a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:58.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.195+0000 7f6d677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d68100050 0x7f6d6819a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54596/0 (socket says 172.21.15.16:54596) 2024-04-02T04:39:58.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.195+0000 7f6d677fe640 1 -- 172.21.15.16:0/4268662204 learned_addr learned my addr 172.21.15.16:0/4268662204 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:39:58.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.195+0000 7f6d677fe640 1 -- 172.21.15.16:0/4268662204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d680fee70 msgr2=0x7f6d68068eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:58.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.195+0000 7f6d67fff640 1 --2- 172.21.15.16:0/4268662204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d680fee70 0x7f6d68068eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:58.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.195+0000 7f6d677fe640 1 --2- 172.21.15.16:0/4268662204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d680fee70 0x7f6d68068eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:58.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.195+0000 7f6d677fe640 1 -- 172.21.15.16:0/4268662204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d5c009560 con 0x7f6d68100050 2024-04-02T04:39:58.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.195+0000 7f6d677fe640 1 --2- 172.21.15.16:0/4268662204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d68100050 0x7f6d6819a840 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6d5800e9a0 tx=0x7f6d5800eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:58.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.196+0000 7f6d657fa640 1 -- 172.21.15.16:0/4268662204 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d5800cd50 con 0x7f6d68100050 2024-04-02T04:39:58.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.196+0000 7f6d657fa640 1 -- 172.21.15.16:0/4268662204 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6d5800ceb0 con 0x7f6d68100050 2024-04-02T04:39:58.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.196+0000 7f6d6e950640 1 -- 172.21.15.16:0/4268662204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d6819b170 con 0x7f6d68100050 2024-04-02T04:39:58.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.196+0000 7f6d657fa640 1 -- 172.21.15.16:0/4268662204 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d58010660 con 0x7f6d68100050 2024-04-02T04:39:58.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.196+0000 7f6d6e950640 1 -- 172.21.15.16:0/4268662204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d6819b750 con 0x7f6d68100050 2024-04-02T04:39:58.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.197+0000 7f6d6e950640 1 -- 172.21.15.16:0/4268662204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d6818e9b0 con 0x7f6d68100050 2024-04-02T04:39:58.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.197+0000 7f6d657fa640 1 -- 172.21.15.16:0/4268662204 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f6d580048a0 con 0x7f6d68100050 2024-04-02T04:39:58.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.197+0000 7f6d657fa640 1 --2- 172.21.15.16:0/4268662204 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f6d5006cc50 0x7f6d5006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:58.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.198+0000 7f6d657fa640 1 -- 172.21.15.16:0/4268662204 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f6d58014070 con 0x7f6d68100050 2024-04-02T04:39:58.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.198+0000 7f6d67fff640 1 --2- 172.21.15.16:0/4268662204 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f6d5006cc50 0x7f6d5006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:58.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.198+0000 7f6d67fff640 1 --2- 172.21.15.16:0/4268662204 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f6d5006cc50 0x7f6d5006f110 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c02c7c0 tx=0x7f6d5c005fd0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:58.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.202+0000 7f6d657fa640 1 -- 172.21.15.16:0/4268662204 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6d5805a650 con 0x7f6d68100050 2024-04-02T04:39:58.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:58 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/203075663' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-04-02T04:39:58.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:58 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:58.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:58 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:39:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.405+0000 7f6d6e950640 1 -- 172.21.15.16:0/4268662204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6d68107a20 con 0x7f6d68100050 2024-04-02T04:39:58.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.406+0000 7f6d657fa640 1 -- 172.21.15.16:0/4268662204 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+15072 (secure 0 0 0) 0x7f6d5805a1e0 con 0x7f6d68100050 2024-04-02T04:39:58.410 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:39:58.410 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":46,"fsid":"dc5403b4-f0a9-11ee-b647-cb9ed24678a4","created":"2024-04-02T04:32:23.014546+0000","modified":"2024-04-02T04:39:46.861360+0000","last_up_change":"2024-04-02T04:38:16.977561+0000","last_in_change":"2024-04-02T04:37:56.807214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T04:37:01.047338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-02T04:39:29.039519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-02T04:39:29.992892+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2b7be29b-0e2a-456b-b33e-c72abaad9136","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":906882291},{"type":"v1","addr":"172.21.15.16:6803","nonce":906882291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":906882291},{"type":"v1","addr":"172.21.15.16:6805","nonce":906882291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":906882291},{"type":"v1","addr":"172.21.15.16:6809","nonce":906882291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":906882291},{"type":"v1","addr":"172.21.15.16:6807","nonce":906882291}]},"public_addr":"172.21.15.16:6803/906882291","cluster_addr":"172.21.15.16:6805/906882291","heartbeat_back_addr":"172.21.15.16:6809/906882291","heartbeat_front_addr":"172.21.15.16:6807/906882291","state":["exists","up"]},{"osd":1,"uuid":"82578291-3c9c-4f3f-8de9-7a8710a4923e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3079548081},{"type":"v1","addr":"172.21.15.16:6811","nonce":3079548081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3079548081},{"type":"v1","addr":"172.21.15.16:6813","nonce":3079548081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3079548081},{"type":"v1","addr":"172.21.15.16:6817","nonce":3079548081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3079548081},{"type":"v1","addr":"172.21.15.16:6815","nonce":3079548081}]},"public_addr":"172.21.15.16:6811/3079548081","cluster_addr":"172.21.15.16:6813/3079548081","heartbeat_back_addr":"172.21.15.16:6817/3079548081","heartbeat_front_addr":"172.21.15.16:6815/3079548081","state":["exists","up"]},{"osd":2,"uuid":"046262b5-386e-4270-863c-e97a61a038b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3734332239},{"type":"v1","addr":"172.21.15.16:6819","nonce":3734332239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3734332239},{"type":"v1","addr":"172.21.15.16:6821","nonce":3734332239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3734332239},{"type":"v1","addr":"172.21.15.16:6825","nonce":3734332239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3734332239},{"type":"v1","addr":"172.21.15.16:6823","nonce":3734332239}]},"public_addr":"172.21.15.16:6819/3734332239","cluster_addr":"172.21.15.16:6821/3734332239","heartbeat_back_addr":"172.21.15.16:6825/3734332239","heartbeat_front_addr":"172.21.15.16:6823/3734332239","state":["exists","up"]},{"osd":3,"uuid":"637f2a77-59a2-44fb-b47e-fcbd70f4423c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":388124754},{"type":"v1","addr":"172.21.15.27:6801","nonce":388124754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":388124754},{"type":"v1","addr":"172.21.15.27:6803","nonce":388124754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":388124754},{"type":"v1","addr":"172.21.15.27:6807","nonce":388124754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":388124754},{"type":"v1","addr":"172.21.15.27:6805","nonce":388124754}]},"public_addr":"172.21.15.27:6801/388124754","cluster_addr":"172.21.15.27:6803/388124754","heartbeat_back_addr":"172.21.15.27:6807/388124754","heartbeat_front_addr":"172.21.15.27:6805/388124754","state":["exists","up"]},{"osd":4,"uuid":"334b7b3d-505f-45b6-85b0-59ec8eeb8e91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":935993381},{"type":"v1","addr":"172.21.15.27:6809","nonce":935993381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":935993381},{"type":"v1","addr":"172.21.15.27:6811","nonce":935993381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":935993381},{"type":"v1","addr":"172.21.15.27:6815","nonce":935993381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":935993381},{"type":"v1","addr":"172.21.15.27:6813","nonce":935993381}]},"public_addr":"172.21.15.27:6809/935993381","cluster_addr":"172.21.15.27:6811/935993381","heartbeat_back_addr":"172.21.15.27:6815/935993381","heartbeat_front_addr":"172.21.15.27:6813/935993381","state":["exists","up"]},{"osd":5,"uuid":"f83802ee-4f47-47de-91dd-a667f6cd22e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2454420135},{"type":"v1","addr":"172.21.15.27:6817","nonce":2454420135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":2454420135},{"type":"v1","addr":"172.21.15.27:6819","nonce":2454420135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":2454420135},{"type":"v1","addr":"172.21.15.27:6823","nonce":2454420135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":2454420135},{"type":"v1","addr":"172.21.15.27:6821","nonce":2454420135}]},"public_addr":"172.21.15.27:6817/2454420135","cluster_addr":"172.21.15.27:6819/2454420135","heartbeat_back_addr":"172.21.15.27:6823/2454420135","heartbeat_front_addr":"172.21.15.27:6821/2454420135","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:36:02.027852+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:36:29.393593+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:36:56.847224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:37:21.195876+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:37:47.992167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:38:15.012477+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6827/71509234":"2024-04-03T04:39:46.861330+0000","172.21.15.16:0/1884756010":"2024-04-03T04:32:56.644452+0000","172.21.15.16:6801/824107799":"2024-04-03T04:32:56.644452+0000","172.21.15.16:0/401549934":"2024-04-03T04:32:56.644452+0000","172.21.15.16:0/2255383579":"2024-04-03T04:32:56.644452+0000","172.21.15.16:6800/824107799":"2024-04-03T04:32:56.644452+0000","172.21.15.16:0/1228221358":"2024-04-03T04:33:28.907145+0000","172.21.15.16:6800/873531241":"2024-04-03T04:33:28.907145+0000","172.21.15.16:6801/873531241":"2024-04-03T04:33:28.907145+0000","172.21.15.16:0/4067493740":"2024-04-03T04:33:28.907145+0000","172.21.15.16:6801/3912038174":"2024-04-03T04:34:42.713541+0000","172.21.15.16:6800/3912038174":"2024-04-03T04:34:42.713541+0000","172.21.15.16:0/3405957790":"2024-04-03T04:34:42.713541+0000","172.21.15.16:0/3733852550":"2024-04-03T04:34:42.713541+0000","172.21.15.16:0/166410319":"2024-04-03T04:33:28.907145+0000","172.21.15.16:0/4209328561":"2024-04-03T04:34:42.713541+0000","172.21.15.16:6826/71509234":"2024-04-03T04:39:46.861330+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T04:39:58.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.409+0000 7f6d6e950640 1 -- 172.21.15.16:0/4268662204 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f6d5006cc50 msgr2=0x7f6d5006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:58.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.409+0000 7f6d6e950640 1 --2- 172.21.15.16:0/4268662204 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f6d5006cc50 0x7f6d5006f110 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c02c7c0 tx=0x7f6d5c005fd0 comp rx=0 tx=0).stop 2024-04-02T04:39:58.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.409+0000 7f6d6e950640 1 -- 172.21.15.16:0/4268662204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d68100050 msgr2=0x7f6d6819a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:58.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.409+0000 7f6d6e950640 1 --2- 172.21.15.16:0/4268662204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d68100050 0x7f6d6819a840 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6d5800e9a0 tx=0x7f6d5800eda0 comp rx=0 tx=0).stop 2024-04-02T04:39:58.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.409+0000 7f6d6e950640 1 -- 172.21.15.16:0/4268662204 shutdown_connections 2024-04-02T04:39:58.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.409+0000 7f6d6e950640 1 --2- 172.21.15.16:0/4268662204 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f6d5006cc50 0x7f6d5006f110 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:58.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.409+0000 7f6d6e950640 1 --2- 172.21.15.16:0/4268662204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d68100050 0x7f6d6819a840 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:58.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.409+0000 7f6d6e950640 1 --2- 172.21.15.16:0/4268662204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d680fee70 0x7f6d68068eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:58.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.409+0000 7f6d6e950640 1 -- 172.21.15.16:0/4268662204 >> 172.21.15.16:0/4268662204 conn(0x7f6d680fa450 msgr2=0x7f6d68103730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:58.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.409+0000 7f6d6e950640 1 -- 172.21.15.16:0/4268662204 shutdown_connections 2024-04-02T04:39:58.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.409+0000 7f6d6e950640 1 -- 172.21.15.16:0/4268662204 wait complete. 2024-04-02T04:39:58.423 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:58 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/203075663' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-04-02T04:39:58.423 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:58 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:39:58.423 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:58 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:39:58.424 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T04:37:01.047338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-02T04:39:29.039519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-02T04:39:29.992892+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T04:39:58.425 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T04:39:58.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.526+0000 7fb4f7f81640 1 -- 172.21.15.16:0/4180797097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4f0100040 msgr2=0x7fb4f01004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:58.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.526+0000 7fb4f7f81640 1 --2- 172.21.15.16:0/4180797097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4f0100040 0x7fb4f01004c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0009880 tx=0x7fb4e002bce0 comp rx=0 tx=0).stop 2024-04-02T04:39:58.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.527+0000 7fb4f7f81640 1 -- 172.21.15.16:0/4180797097 shutdown_connections 2024-04-02T04:39:58.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.527+0000 7fb4f7f81640 1 --2- 172.21.15.16:0/4180797097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4f0100040 0x7fb4f01004c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:58.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.527+0000 7fb4f7f81640 1 --2- 172.21.15.16:0/4180797097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4f00fee60 0x7fb4f00ff280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:58.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.527+0000 7fb4f7f81640 1 -- 172.21.15.16:0/4180797097 >> 172.21.15.16:0/4180797097 conn(0x7fb4f00fa440 msgr2=0x7fb4f00fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:58.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.527+0000 7fb4f7f81640 1 -- 172.21.15.16:0/4180797097 shutdown_connections 2024-04-02T04:39:58.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.527+0000 7fb4f7f81640 1 -- 172.21.15.16:0/4180797097 wait complete. 2024-04-02T04:39:58.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.527+0000 7fb4f7f81640 1 Processor -- start 2024-04-02T04:39:58.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.527+0000 7fb4f7f81640 1 -- start start 2024-04-02T04:39:58.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.527+0000 7fb4f7f81640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4f00fee60 0x7fb4f0102e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:58.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.527+0000 7fb4f7f81640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4f0100040 0x7fb4f01014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:58.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.527+0000 7fb4f7f81640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f01019f0 con 0x7fb4f0100040 2024-04-02T04:39:58.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.527+0000 7fb4f7f81640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f0101b60 con 0x7fb4f00fee60 2024-04-02T04:39:58.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.527+0000 7fb4f551c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4f0100040 0x7fb4f01014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:58.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.527+0000 7fb4f551c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4f0100040 0x7fb4f01014b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54604/0 (socket says 172.21.15.16:54604) 2024-04-02T04:39:58.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.527+0000 7fb4f551c640 1 -- 172.21.15.16:0/2526868325 learned_addr learned my addr 172.21.15.16:0/2526868325 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:39:58.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.528+0000 7fb4f5d1d640 1 --2- 172.21.15.16:0/2526868325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4f00fee60 0x7fb4f0102e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:58.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.528+0000 7fb4f551c640 1 -- 172.21.15.16:0/2526868325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4f00fee60 msgr2=0x7fb4f0102e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:58.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.528+0000 7fb4f551c640 1 --2- 172.21.15.16:0/2526868325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4f00fee60 0x7fb4f0102e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:58.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.528+0000 7fb4f551c640 1 -- 172.21.15.16:0/2526868325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e0009560 con 0x7fb4f0100040 2024-04-02T04:39:58.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.528+0000 7fb4f551c640 1 --2- 172.21.15.16:0/2526868325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4f0100040 0x7fb4f01014b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0036040 tx=0x7fb4e0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:58.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.528+0000 7fb4e6ffd640 1 -- 172.21.15.16:0/2526868325 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4e00035f0 con 0x7fb4f0100040 2024-04-02T04:39:58.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.528+0000 7fb4e6ffd640 1 -- 172.21.15.16:0/2526868325 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb4e0003c10 con 0x7fb4f0100040 2024-04-02T04:39:58.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.528+0000 7fb4f7f81640 1 -- 172.21.15.16:0/2526868325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4f0101de0 con 0x7fb4f0100040 2024-04-02T04:39:58.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.528+0000 7fb4e6ffd640 1 -- 172.21.15.16:0/2526868325 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4e0034440 con 0x7fb4f0100040 2024-04-02T04:39:58.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.528+0000 7fb4f7f81640 1 -- 172.21.15.16:0/2526868325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f019db40 con 0x7fb4f0100040 2024-04-02T04:39:58.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.529+0000 7fb4f7f81640 1 -- 172.21.15.16:0/2526868325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4f004e030 con 0x7fb4f0100040 2024-04-02T04:39:58.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.529+0000 7fb4e6ffd640 1 -- 172.21.15.16:0/2526868325 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fb4e0004ff0 con 0x7fb4f0100040 2024-04-02T04:39:58.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.530+0000 7fb4e6ffd640 1 --2- 172.21.15.16:0/2526868325 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fb4dc06cc60 0x7fb4dc06f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:58.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.530+0000 7fb4e6ffd640 1 -- 172.21.15.16:0/2526868325 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fb4e00ae640 con 0x7fb4f0100040 2024-04-02T04:39:58.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.530+0000 7fb4f5d1d640 1 --2- 172.21.15.16:0/2526868325 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fb4dc06cc60 0x7fb4dc06f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:58.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.530+0000 7fb4f5d1d640 1 --2- 172.21.15.16:0/2526868325 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fb4dc06cc60 0x7fb4dc06f120 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb4f00ffbb0 tx=0x7fb4ec00b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:58.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.534+0000 7fb4e6ffd640 1 -- 172.21.15.16:0/2526868325 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb4e0078a60 con 0x7fb4f0100040 2024-04-02T04:39:58.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.725+0000 7fb4f7f81640 1 -- 172.21.15.16:0/2526868325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb4f019dd70 con 0x7fb4f0100040 2024-04-02T04:39:58.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.727+0000 7fb4e6ffd640 1 -- 172.21.15.16:0/2526868325 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb4e007c080 con 0x7fb4f0100040 2024-04-02T04:39:58.730 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-02T04:39:58.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.729+0000 7fb4f7f81640 1 -- 172.21.15.16:0/2526868325 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fb4dc06cc60 msgr2=0x7fb4dc06f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:58.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.729+0000 7fb4f7f81640 1 --2- 172.21.15.16:0/2526868325 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fb4dc06cc60 0x7fb4dc06f120 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb4f00ffbb0 tx=0x7fb4ec00b390 comp rx=0 tx=0).stop 2024-04-02T04:39:58.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.729+0000 7fb4f7f81640 1 -- 172.21.15.16:0/2526868325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4f0100040 msgr2=0x7fb4f01014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:58.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.729+0000 7fb4f7f81640 1 --2- 172.21.15.16:0/2526868325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4f0100040 0x7fb4f01014b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0036040 tx=0x7fb4e0004040 comp rx=0 tx=0).stop 2024-04-02T04:39:58.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.729+0000 7fb4f7f81640 1 -- 172.21.15.16:0/2526868325 shutdown_connections 2024-04-02T04:39:58.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.729+0000 7fb4f7f81640 1 --2- 172.21.15.16:0/2526868325 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fb4dc06cc60 0x7fb4dc06f120 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:58.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.729+0000 7fb4f7f81640 1 --2- 172.21.15.16:0/2526868325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4f0100040 0x7fb4f01014b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:58.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.729+0000 7fb4f7f81640 1 --2- 172.21.15.16:0/2526868325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4f00fee60 0x7fb4f0102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:58.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.729+0000 7fb4f7f81640 1 -- 172.21.15.16:0/2526868325 >> 172.21.15.16:0/2526868325 conn(0x7fb4f00fa440 msgr2=0x7fb4f00672b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:58.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.729+0000 7fb4f7f81640 1 -- 172.21.15.16:0/2526868325 shutdown_connections 2024-04-02T04:39:58.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.729+0000 7fb4f7f81640 1 -- 172.21.15.16:0/2526868325 wait complete. 2024-04-02T04:39:58.743 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-02T04:39:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.845+0000 7f8d4a6d4640 1 -- 172.21.15.16:0/1488537274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d44103390 msgr2=0x7f8d44105780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.845+0000 7f8d4a6d4640 1 --2- 172.21.15.16:0/1488537274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d44103390 0x7f8d44105780 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c009880 tx=0x7f8d2c02bce0 comp rx=0 tx=0).stop 2024-04-02T04:39:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.845+0000 7f8d4a6d4640 1 -- 172.21.15.16:0/1488537274 shutdown_connections 2024-04-02T04:39:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.845+0000 7f8d4a6d4640 1 --2- 172.21.15.16:0/1488537274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d44103390 0x7f8d44105780 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.845+0000 7f8d4a6d4640 1 --2- 172.21.15.16:0/1488537274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d44100a60 0x7f8d44102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.845+0000 7f8d4a6d4640 1 -- 172.21.15.16:0/1488537274 >> 172.21.15.16:0/1488537274 conn(0x7f8d440fa430 msgr2=0x7f8d440fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.846+0000 7f8d4a6d4640 1 -- 172.21.15.16:0/1488537274 shutdown_connections 2024-04-02T04:39:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.846+0000 7f8d4a6d4640 1 -- 172.21.15.16:0/1488537274 wait complete. 2024-04-02T04:39:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.846+0000 7f8d4a6d4640 1 Processor -- start 2024-04-02T04:39:58.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.846+0000 7f8d4a6d4640 1 -- start start 2024-04-02T04:39:58.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.846+0000 7f8d4a6d4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d44100a60 0x7f8d44100330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:58.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.846+0000 7f8d4a6d4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d44103390 0x7f8d440fea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:58.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.846+0000 7f8d4a6d4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d44100900 con 0x7f8d44100a60 2024-04-02T04:39:58.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.846+0000 7f8d4a6d4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d440fef80 con 0x7f8d44103390 2024-04-02T04:39:58.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.847+0000 7f8d43fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d44100a60 0x7f8d44100330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:58.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.847+0000 7f8d43fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d44100a60 0x7f8d44100330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54612/0 (socket says 172.21.15.16:54612) 2024-04-02T04:39:58.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.847+0000 7f8d43fff640 1 -- 172.21.15.16:0/1126678427 learned_addr learned my addr 172.21.15.16:0/1126678427 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:39:58.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.847+0000 7f8d437fe640 1 --2- 172.21.15.16:0/1126678427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d44103390 0x7f8d440fea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:58.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.847+0000 7f8d43fff640 1 -- 172.21.15.16:0/1126678427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d44103390 msgr2=0x7f8d440fea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:58.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.847+0000 7f8d43fff640 1 --2- 172.21.15.16:0/1126678427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d44103390 0x7f8d440fea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:58.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.847+0000 7f8d43fff640 1 -- 172.21.15.16:0/1126678427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2c009560 con 0x7f8d44100a60 2024-04-02T04:39:58.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.847+0000 7f8d43fff640 1 --2- 172.21.15.16:0/1126678427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d44100a60 0x7f8d44100330 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400d6f0 tx=0x7f8d3400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:58.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.847+0000 7f8d417fa640 1 -- 172.21.15.16:0/1126678427 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d34009a30 con 0x7f8d44100a60 2024-04-02T04:39:58.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.847+0000 7f8d417fa640 1 -- 172.21.15.16:0/1126678427 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d340054e0 con 0x7f8d44100a60 2024-04-02T04:39:58.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.847+0000 7f8d4a6d4640 1 -- 172.21.15.16:0/1126678427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d440ff260 con 0x7f8d44100a60 2024-04-02T04:39:58.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.847+0000 7f8d417fa640 1 -- 172.21.15.16:0/1126678427 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d34010460 con 0x7f8d44100a60 2024-04-02T04:39:58.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.847+0000 7f8d4a6d4640 1 -- 172.21.15.16:0/1126678427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d440ff810 con 0x7f8d44100a60 2024-04-02T04:39:58.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.848+0000 7f8d4a6d4640 1 -- 172.21.15.16:0/1126678427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d4404e030 con 0x7f8d44100a60 2024-04-02T04:39:58.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.848+0000 7f8d417fa640 1 -- 172.21.15.16:0/1126678427 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f8d34004ec0 con 0x7f8d44100a60 2024-04-02T04:39:58.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.849+0000 7f8d417fa640 1 --2- 172.21.15.16:0/1126678427 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f8d3006cc50 0x7f8d3006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:58.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.849+0000 7f8d417fa640 1 -- 172.21.15.16:0/1126678427 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f8d3408be60 con 0x7f8d44100a60 2024-04-02T04:39:58.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.849+0000 7f8d437fe640 1 --2- 172.21.15.16:0/1126678427 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f8d3006cc50 0x7f8d3006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:58.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.849+0000 7f8d437fe640 1 --2- 172.21.15.16:0/1126678427 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f8d3006cc50 0x7f8d3006f110 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c006010 tx=0x7f8d2c02c920 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:58.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:58.854+0000 7f8d417fa640 1 -- 172.21.15.16:0/1126678427 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8d340561d0 con 0x7f8d44100a60 2024-04-02T04:39:59.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.034+0000 7f8d4a6d4640 1 -- 172.21.15.16:0/1126678427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f8d44108b90 con 0x7f8d44100a60 2024-04-02T04:39:59.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.034+0000 7f8d417fa640 1 -- 172.21.15.16:0/1126678427 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7f8d340597f0 con 0x7f8d44100a60 2024-04-02T04:39:59.038 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-02T04:39:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.036+0000 7f8d4a6d4640 1 -- 172.21.15.16:0/1126678427 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f8d3006cc50 msgr2=0x7f8d3006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.036+0000 7f8d4a6d4640 1 --2- 172.21.15.16:0/1126678427 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f8d3006cc50 0x7f8d3006f110 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c006010 tx=0x7f8d2c02c920 comp rx=0 tx=0).stop 2024-04-02T04:39:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.036+0000 7f8d4a6d4640 1 -- 172.21.15.16:0/1126678427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d44100a60 msgr2=0x7f8d44100330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.036+0000 7f8d4a6d4640 1 --2- 172.21.15.16:0/1126678427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d44100a60 0x7f8d44100330 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400d6f0 tx=0x7f8d3400dbb0 comp rx=0 tx=0).stop 2024-04-02T04:39:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.037+0000 7f8d4a6d4640 1 -- 172.21.15.16:0/1126678427 shutdown_connections 2024-04-02T04:39:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.037+0000 7f8d4a6d4640 1 --2- 172.21.15.16:0/1126678427 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f8d3006cc50 0x7f8d3006f110 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.037+0000 7f8d4a6d4640 1 --2- 172.21.15.16:0/1126678427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d44103390 0x7f8d440fea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.037+0000 7f8d4a6d4640 1 --2- 172.21.15.16:0/1126678427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d44100a60 0x7f8d44100330 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.037+0000 7f8d4a6d4640 1 -- 172.21.15.16:0/1126678427 >> 172.21.15.16:0/1126678427 conn(0x7f8d440fa430 msgr2=0x7f8d440fc820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.037+0000 7f8d4a6d4640 1 -- 172.21.15.16:0/1126678427 shutdown_connections 2024-04-02T04:39:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.037+0000 7f8d4a6d4640 1 -- 172.21.15.16:0/1126678427 wait complete. 2024-04-02T04:39:59.051 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-02T04:39:59.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.164+0000 7f97dd077640 1 -- 172.21.15.16:0/4181119553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97d80fe890 msgr2=0x7f97d80fecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:59.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.164+0000 7f97dd077640 1 --2- 172.21.15.16:0/4181119553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97d80fe890 0x7f97d80fecb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f97d0009880 tx=0x7f97d002bce0 comp rx=0 tx=0).stop 2024-04-02T04:39:59.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.164+0000 7f97dd077640 1 -- 172.21.15.16:0/4181119553 shutdown_connections 2024-04-02T04:39:59.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.164+0000 7f97dd077640 1 --2- 172.21.15.16:0/4181119553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97d80ff1f0 0x7f97d80ff650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:59.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.164+0000 7f97dd077640 1 --2- 172.21.15.16:0/4181119553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97d80fe890 0x7f97d80fecb0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:59.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.164+0000 7f97dd077640 1 -- 172.21.15.16:0/4181119553 >> 172.21.15.16:0/4181119553 conn(0x7f97d80fa430 msgr2=0x7f97d80fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:59.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.164+0000 7f97dd077640 1 -- 172.21.15.16:0/4181119553 shutdown_connections 2024-04-02T04:39:59.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.164+0000 7f97dd077640 1 -- 172.21.15.16:0/4181119553 wait complete. 2024-04-02T04:39:59.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.165+0000 7f97dd077640 1 Processor -- start 2024-04-02T04:39:59.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.165+0000 7f97dd077640 1 -- start start 2024-04-02T04:39:59.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.166+0000 7f97dd077640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97d80fe890 0x7f97d8197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:59.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.166+0000 7f97dd077640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97d80ff1f0 0x7f97d8198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:59.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.166+0000 7f97dd077640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d8198840 con 0x7f97d80ff1f0 2024-04-02T04:39:59.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.166+0000 7f97dd077640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d81989b0 con 0x7f97d80fe890 2024-04-02T04:39:59.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.166+0000 7f97d659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97d80ff1f0 0x7f97d8198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:59.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.166+0000 7f97d659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97d80ff1f0 0x7f97d8198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54626/0 (socket says 172.21.15.16:54626) 2024-04-02T04:39:59.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.166+0000 7f97d659c640 1 -- 172.21.15.16:0/1913049637 learned_addr learned my addr 172.21.15.16:0/1913049637 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:39:59.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.166+0000 7f97d6d9d640 1 --2- 172.21.15.16:0/1913049637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97d80fe890 0x7f97d8197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:59.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.166+0000 7f97d659c640 1 -- 172.21.15.16:0/1913049637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97d80fe890 msgr2=0x7f97d8197d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:59.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.166+0000 7f97d659c640 1 --2- 172.21.15.16:0/1913049637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97d80fe890 0x7f97d8197d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:59.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.166+0000 7f97d659c640 1 -- 172.21.15.16:0/1913049637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d0009560 con 0x7f97d80ff1f0 2024-04-02T04:39:59.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.166+0000 7f97d659c640 1 --2- 172.21.15.16:0/1913049637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97d80ff1f0 0x7f97d8198240 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f97c000e970 tx=0x7f97c000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:59.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.166+0000 7f97cffff640 1 -- 172.21.15.16:0/1913049637 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97c000cc90 con 0x7f97d80ff1f0 2024-04-02T04:39:59.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.166+0000 7f97dd077640 1 -- 172.21.15.16:0/1913049637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97d819d3e0 con 0x7f97d80ff1f0 2024-04-02T04:39:59.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.166+0000 7f97dd077640 1 -- 172.21.15.16:0/1913049637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97d819d9c0 con 0x7f97d80ff1f0 2024-04-02T04:39:59.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.166+0000 7f97cffff640 1 -- 172.21.15.16:0/1913049637 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f97c000cdf0 con 0x7f97d80ff1f0 2024-04-02T04:39:59.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.166+0000 7f97cffff640 1 -- 172.21.15.16:0/1913049637 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97c0010430 con 0x7f97d80ff1f0 2024-04-02T04:39:59.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.167+0000 7f97dd077640 1 -- 172.21.15.16:0/1913049637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97b8005280 con 0x7f97d80ff1f0 2024-04-02T04:39:59.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.168+0000 7f97cffff640 1 -- 172.21.15.16:0/1913049637 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f97c00046c0 con 0x7f97d80ff1f0 2024-04-02T04:39:59.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.168+0000 7f97cffff640 1 --2- 172.21.15.16:0/1913049637 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f97c406cc50 0x7f97c406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:59.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.168+0000 7f97d6d9d640 1 --2- 172.21.15.16:0/1913049637 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f97c406cc50 0x7f97c406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:59.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.168+0000 7f97cffff640 1 -- 172.21.15.16:0/1913049637 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f97c0014070 con 0x7f97d80ff1f0 2024-04-02T04:39:59.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.169+0000 7f97d6d9d640 1 --2- 172.21.15.16:0/1913049637 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f97c406cc50 0x7f97c406f110 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f97d0006010 tx=0x7f97d002cb40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:59.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.173+0000 7f97cffff640 1 -- 172.21.15.16:0/1913049637 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f97c0056a80 con 0x7f97d80ff1f0 2024-04-02T04:39:59.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:59 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/4268662204' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T04:39:59.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:59 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/2526868325' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T04:39:59.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:59 smithi016 ceph-mon[28969]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 6.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-02T04:39:59.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:39:59 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/1126678427' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-02T04:39:59.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.351+0000 7f97dd077640 1 -- 172.21.15.16:0/1913049637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f97b8005ab0 con 0x7f97d80ff1f0 2024-04-02T04:39:59.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.352+0000 7f97cffff640 1 -- 172.21.15.16:0/1913049637 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7f97c005a0a0 con 0x7f97d80ff1f0 2024-04-02T04:39:59.355 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-02T04:39:59.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.354+0000 7f97dd077640 1 -- 172.21.15.16:0/1913049637 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f97c406cc50 msgr2=0x7f97c406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:59.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.354+0000 7f97dd077640 1 --2- 172.21.15.16:0/1913049637 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f97c406cc50 0x7f97c406f110 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f97d0006010 tx=0x7f97d002cb40 comp rx=0 tx=0).stop 2024-04-02T04:39:59.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.354+0000 7f97dd077640 1 -- 172.21.15.16:0/1913049637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97d80ff1f0 msgr2=0x7f97d8198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:59.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.354+0000 7f97dd077640 1 --2- 172.21.15.16:0/1913049637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97d80ff1f0 0x7f97d8198240 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f97c000e970 tx=0x7f97c000ed70 comp rx=0 tx=0).stop 2024-04-02T04:39:59.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.354+0000 7f97dd077640 1 -- 172.21.15.16:0/1913049637 shutdown_connections 2024-04-02T04:39:59.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.354+0000 7f97dd077640 1 --2- 172.21.15.16:0/1913049637 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f97c406cc50 0x7f97c406f110 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:59.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.354+0000 7f97dd077640 1 --2- 172.21.15.16:0/1913049637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97d80ff1f0 0x7f97d8198240 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:59.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.354+0000 7f97dd077640 1 --2- 172.21.15.16:0/1913049637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97d80fe890 0x7f97d8197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:59.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.354+0000 7f97dd077640 1 -- 172.21.15.16:0/1913049637 >> 172.21.15.16:0/1913049637 conn(0x7f97d80fa430 msgr2=0x7f97d81072e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:59.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.355+0000 7f97dd077640 1 -- 172.21.15.16:0/1913049637 shutdown_connections 2024-04-02T04:39:59.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.355+0000 7f97dd077640 1 -- 172.21.15.16:0/1913049637 wait complete. 2024-04-02T04:39:59.372 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T04:39:59.395 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:59 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/4268662204' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T04:39:59.395 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:59 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/2526868325' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T04:39:59.395 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:59 smithi027 ceph-mon[37550]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 6.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-02T04:39:59.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:39:59 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/1126678427' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-02T04:39:59.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.476+0000 7ff950b47640 1 -- 172.21.15.16:0/3229632417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff94c100ab0 msgr2=0x7ff94c102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:59.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.476+0000 7ff950b47640 1 --2- 172.21.15.16:0/3229632417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff94c100ab0 0x7ff94c102ea0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff93c009c20 tx=0x7ff93c02c1b0 comp rx=0 tx=0).stop 2024-04-02T04:39:59.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.476+0000 7ff950b47640 1 -- 172.21.15.16:0/3229632417 shutdown_connections 2024-04-02T04:39:59.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.476+0000 7ff950b47640 1 --2- 172.21.15.16:0/3229632417 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff94c1033e0 0x7ff94c1057d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:59.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.476+0000 7ff950b47640 1 --2- 172.21.15.16:0/3229632417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff94c100ab0 0x7ff94c102ea0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:59.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.476+0000 7ff950b47640 1 -- 172.21.15.16:0/3229632417 >> 172.21.15.16:0/3229632417 conn(0x7ff94c0fa440 msgr2=0x7ff94c0fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:59.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.476+0000 7ff950b47640 1 -- 172.21.15.16:0/3229632417 shutdown_connections 2024-04-02T04:39:59.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.476+0000 7ff950b47640 1 -- 172.21.15.16:0/3229632417 wait complete. 2024-04-02T04:39:59.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.476+0000 7ff950b47640 1 Processor -- start 2024-04-02T04:39:59.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.476+0000 7ff950b47640 1 -- start start 2024-04-02T04:39:59.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.477+0000 7ff950b47640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff94c100ab0 0x7ff94c195b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:59.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.477+0000 7ff950b47640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff94c1033e0 0x7ff94c196040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:59.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.477+0000 7ff950b47640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94c196610 con 0x7ff94c1033e0 2024-04-02T04:39:59.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.477+0000 7ff950b47640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94c196780 con 0x7ff94c100ab0 2024-04-02T04:39:59.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.477+0000 7ff949d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff94c1033e0 0x7ff94c196040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:59.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.477+0000 7ff949d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff94c1033e0 0x7ff94c196040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54644/0 (socket says 172.21.15.16:54644) 2024-04-02T04:39:59.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.477+0000 7ff949d9b640 1 -- 172.21.15.16:0/3237542618 learned_addr learned my addr 172.21.15.16:0/3237542618 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:39:59.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.477+0000 7ff94a59c640 1 --2- 172.21.15.16:0/3237542618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff94c100ab0 0x7ff94c195b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:59.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.477+0000 7ff949d9b640 1 -- 172.21.15.16:0/3237542618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff94c100ab0 msgr2=0x7ff94c195b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:59.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.477+0000 7ff949d9b640 1 --2- 172.21.15.16:0/3237542618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff94c100ab0 0x7ff94c195b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:59.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.477+0000 7ff949d9b640 1 -- 172.21.15.16:0/3237542618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff93c009900 con 0x7ff94c1033e0 2024-04-02T04:39:59.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.477+0000 7ff949d9b640 1 --2- 172.21.15.16:0/3237542618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff94c1033e0 0x7ff94c196040 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff93400d6f0 tx=0x7ff93400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:59.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.477+0000 7ff9437fe640 1 -- 172.21.15.16:0/3237542618 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff934009a30 con 0x7ff94c1033e0 2024-04-02T04:39:59.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.477+0000 7ff9437fe640 1 -- 172.21.15.16:0/3237542618 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff934009b90 con 0x7ff94c1033e0 2024-04-02T04:39:59.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.477+0000 7ff950b47640 1 -- 172.21.15.16:0/3237542618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff94c19b1c0 con 0x7ff94c1033e0 2024-04-02T04:39:59.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.477+0000 7ff9437fe640 1 -- 172.21.15.16:0/3237542618 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff93400f680 con 0x7ff94c1033e0 2024-04-02T04:39:59.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.478+0000 7ff950b47640 1 -- 172.21.15.16:0/3237542618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff94c19b7a0 con 0x7ff94c1033e0 2024-04-02T04:39:59.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.479+0000 7ff950b47640 1 -- 172.21.15.16:0/3237542618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff94c108ba0 con 0x7ff94c1033e0 2024-04-02T04:39:59.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.479+0000 7ff9437fe640 1 -- 172.21.15.16:0/3237542618 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7ff934010460 con 0x7ff94c1033e0 2024-04-02T04:39:59.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.480+0000 7ff9437fe640 1 --2- 172.21.15.16:0/3237542618 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7ff93806cc50 0x7ff93806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:39:59.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.480+0000 7ff9437fe640 1 -- 172.21.15.16:0/3237542618 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7ff93408c130 con 0x7ff94c1033e0 2024-04-02T04:39:59.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.480+0000 7ff94a59c640 1 --2- 172.21.15.16:0/3237542618 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7ff93806cc50 0x7ff93806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:39:59.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.480+0000 7ff94a59c640 1 --2- 172.21.15.16:0/3237542618 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7ff93806cc50 0x7ff93806f110 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff93c032830 tx=0x7ff93c009360 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:39:59.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.485+0000 7ff9437fe640 1 -- 172.21.15.16:0/3237542618 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff9340564a0 con 0x7ff94c1033e0 2024-04-02T04:39:59.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.706+0000 7ff950b47640 1 -- 172.21.15.16:0/3237542618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff94c19b9c0 con 0x7ff94c1033e0 2024-04-02T04:39:59.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.706+0000 7ff9437fe640 1 -- 172.21.15.16:0/3237542618 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4822 (secure 0 0 0) 0x7ff934059ac0 con 0x7ff94c1033e0 2024-04-02T04:39:59.710 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:39:59.710 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14488,"name":"cephfs.smithi016.fyzrnm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.16:6829/2578719859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2578719859},{"type":"v1","addr":"172.21.15.16:6829","nonce":2578719859}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":14500,"name":"cephfs.smithi016.uhgkgg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6827/601263579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":601263579},{"type":"v1","addr":"172.21.15.16:6827","nonce":601263579}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24295,"name":"cephfs.smithi027.sdublg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.27:6827/4177192740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":4177192740},{"type":"v1","addr":"172.21.15.27:6827","nonce":4177192740}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:39:30.740168+0000","modified":"2024-04-02T04:39:49.875319+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24285":{"gid":24285,"name":"cephfs.smithi027.zwrfoo","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6825/1461455976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1461455976},{"type":"v1","addr":"172.21.15.27:6825","nonce":1461455976}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-02T04:39:59.710 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 15 2024-04-02T04:39:59.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.708+0000 7ff950b47640 1 -- 172.21.15.16:0/3237542618 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7ff93806cc50 msgr2=0x7ff93806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:59.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.708+0000 7ff950b47640 1 --2- 172.21.15.16:0/3237542618 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7ff93806cc50 0x7ff93806f110 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff93c032830 tx=0x7ff93c009360 comp rx=0 tx=0).stop 2024-04-02T04:39:59.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.708+0000 7ff950b47640 1 -- 172.21.15.16:0/3237542618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff94c1033e0 msgr2=0x7ff94c196040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:39:59.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.708+0000 7ff950b47640 1 --2- 172.21.15.16:0/3237542618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff94c1033e0 0x7ff94c196040 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff93400d6f0 tx=0x7ff93400dbb0 comp rx=0 tx=0).stop 2024-04-02T04:39:59.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.708+0000 7ff950b47640 1 -- 172.21.15.16:0/3237542618 shutdown_connections 2024-04-02T04:39:59.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.708+0000 7ff950b47640 1 --2- 172.21.15.16:0/3237542618 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7ff93806cc50 0x7ff93806f110 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:59.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.708+0000 7ff950b47640 1 --2- 172.21.15.16:0/3237542618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff94c1033e0 0x7ff94c196040 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:59.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.708+0000 7ff950b47640 1 --2- 172.21.15.16:0/3237542618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff94c100ab0 0x7ff94c195b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:39:59.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.708+0000 7ff950b47640 1 -- 172.21.15.16:0/3237542618 >> 172.21.15.16:0/3237542618 conn(0x7ff94c0fa440 msgr2=0x7ff94c0fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:39:59.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.709+0000 7ff950b47640 1 -- 172.21.15.16:0/3237542618 shutdown_connections 2024-04-02T04:39:59.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:39:59.709+0000 7ff950b47640 1 -- 172.21.15.16:0/3237542618 wait complete. 2024-04-02T04:39:59.722 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 15, 'max_mds': 1, 'flags': 18} 2024-04-02T04:39:59.722 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-02T04:39:59.792 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-02T04:39:59.792 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-04-02T04:39:59.792 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-02T04:39:59.792 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-02T04:39:59.792 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-02T04:39:59.792 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-02T04:39:59.793 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-02T04:39:59.793 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-02T04:39:59.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:39:59.793 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-04-02T04:39:59.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:39:59.808 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-04-02T04:39:59.883 INFO:teuthology.orchestra.run.smithi016.stderr:Cannot find device "ceph-brx" 2024-04-02T04:39:59.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T04:39:59.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:39:59.885 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-04-02T04:39:59.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:39:59.906 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link delete ceph-brx 2024-04-02T04:39:59.988 INFO:teuthology.orchestra.run.smithi027.stderr:Cannot find device "ceph-brx" 2024-04-02T04:39:59.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T04:39:59.990 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-02T04:39:59.990 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph fs ls 2024-04-02T04:40:00.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:00 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/1913049637' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-02T04:40:00.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:00 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/3237542618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T04:40:00.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:00 smithi016 ceph-mon[28969]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-02T04:40:00.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:00 smithi016 ceph-mon[28969]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-02T04:40:00.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:00 smithi016 ceph-mon[28969]: fs cephfs has deprecated feature inline_data enabled. 2024-04-02T04:40:00.395 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:00 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/1913049637' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-02T04:40:00.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:00 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/3237542618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T04:40:00.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:00 smithi027 ceph-mon[37550]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-02T04:40:00.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:00 smithi027 ceph-mon[37550]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-02T04:40:00.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:00 smithi027 ceph-mon[37550]: fs cephfs has deprecated feature inline_data enabled. 2024-04-02T04:40:00.623 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:40:01.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:01 smithi016 ceph-mon[28969]: pgmap v193: 65 pgs: 65 active+clean; 579 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 6.4 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-02T04:40:01.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:01 smithi027 ceph-mon[37550]: pgmap v193: 65 pgs: 65 active+clean; 579 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 6.4 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-02T04:40:02.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.182+0000 7fbc1d4fd700 1 -- 172.21.15.16:0/2607774843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc18105950 msgr2=0x7fbc18105d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:02.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.182+0000 7fbc1d4fd700 1 --2- 172.21.15.16:0/2607774843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc18105950 0x7fbc18105d30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbc0800b0a0 tx=0x7fbc0800b3b0 comp rx=0 tx=0).stop 2024-04-02T04:40:02.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.182+0000 7fbc1d4fd700 1 -- 172.21.15.16:0/2607774843 shutdown_connections 2024-04-02T04:40:02.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.182+0000 7fbc1d4fd700 1 --2- 172.21.15.16:0/2607774843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc180ff950 0x7fbc180ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:02.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.182+0000 7fbc1d4fd700 1 --2- 172.21.15.16:0/2607774843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc18105950 0x7fbc18105d30 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:02.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.182+0000 7fbc1d4fd700 1 -- 172.21.15.16:0/2607774843 >> 172.21.15.16:0/2607774843 conn(0x7fbc18076640 msgr2=0x7fbc180fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:02.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.182+0000 7fbc1d4fd700 1 -- 172.21.15.16:0/2607774843 shutdown_connections 2024-04-02T04:40:02.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.182+0000 7fbc1d4fd700 1 -- 172.21.15.16:0/2607774843 wait complete. 2024-04-02T04:40:02.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.182+0000 7fbc1d4fd700 1 Processor -- start 2024-04-02T04:40:02.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.183+0000 7fbc1d4fd700 1 -- start start 2024-04-02T04:40:02.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.183+0000 7fbc1d4fd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc180ff950 0x7fbc18196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:02.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.183+0000 7fbc1d4fd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc18105950 0x7fbc18196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:02.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.183+0000 7fbc1d4fd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc18197680 con 0x7fbc18105950 2024-04-02T04:40:02.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.183+0000 7fbc1d4fd700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1819b380 con 0x7fbc180ff950 2024-04-02T04:40:02.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.183+0000 7fbc167fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc18105950 0x7fbc18196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:02.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.183+0000 7fbc167fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc18105950 0x7fbc18196ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39818/0 (socket says 172.21.15.16:39818) 2024-04-02T04:40:02.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.183+0000 7fbc167fc700 1 -- 172.21.15.16:0/4082503786 learned_addr learned my addr 172.21.15.16:0/4082503786 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:40:02.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.183+0000 7fbc16ffd700 1 --2- 172.21.15.16:0/4082503786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc180ff950 0x7fbc18196ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:02.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.183+0000 7fbc167fc700 1 -- 172.21.15.16:0/4082503786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc180ff950 msgr2=0x7fbc18196ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:02.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.183+0000 7fbc167fc700 1 --2- 172.21.15.16:0/4082503786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc180ff950 0x7fbc18196ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:02.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.183+0000 7fbc167fc700 1 -- 172.21.15.16:0/4082503786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc08009d00 con 0x7fbc18105950 2024-04-02T04:40:02.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.184+0000 7fbc167fc700 1 --2- 172.21.15.16:0/4082503786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc18105950 0x7fbc18196ff0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbc0000b4b0 tx=0x7fbc0000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:02.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.184+0000 7fbc0ffff700 1 -- 172.21.15.16:0/4082503786 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc00011800 con 0x7fbc18105950 2024-04-02T04:40:02.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.184+0000 7fbc0ffff700 1 -- 172.21.15.16:0/4082503786 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc00011e20 con 0x7fbc18105950 2024-04-02T04:40:02.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.184+0000 7fbc0ffff700 1 -- 172.21.15.16:0/4082503786 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc0000f2e0 con 0x7fbc18105950 2024-04-02T04:40:02.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.184+0000 7fbc1d4fd700 1 -- 172.21.15.16:0/4082503786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc1819b660 con 0x7fbc18105950 2024-04-02T04:40:02.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.184+0000 7fbc1d4fd700 1 -- 172.21.15.16:0/4082503786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc1819bbb0 con 0x7fbc18105950 2024-04-02T04:40:02.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.185+0000 7fbc1d4fd700 1 -- 172.21.15.16:0/4082503786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc18101090 con 0x7fbc18105950 2024-04-02T04:40:02.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.185+0000 7fbc0ffff700 1 -- 172.21.15.16:0/4082503786 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fbc00010380 con 0x7fbc18105950 2024-04-02T04:40:02.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.186+0000 7fbc0ffff700 1 --2- 172.21.15.16:0/4082503786 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fbc0406cce0 0x7fbc0406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:02.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.186+0000 7fbc0ffff700 1 -- 172.21.15.16:0/4082503786 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fbc0008bcb0 con 0x7fbc18105950 2024-04-02T04:40:02.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.186+0000 7fbc16ffd700 1 --2- 172.21.15.16:0/4082503786 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fbc0406cce0 0x7fbc0406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:02.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.186+0000 7fbc16ffd700 1 --2- 172.21.15.16:0/4082503786 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fbc0406cce0 0x7fbc0406f1a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbc08000f80 tx=0x7fbc080060c0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:02.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.191+0000 7fbc0ffff700 1 -- 172.21.15.16:0/4082503786 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbc00059be0 con 0x7fbc18105950 2024-04-02T04:40:02.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.445+0000 7fbc1d4fd700 1 -- 172.21.15.16:0/4082503786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbc1810cbd0 con 0x7fbc18105950 2024-04-02T04:40:02.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.445+0000 7fbc0ffff700 1 -- 172.21.15.16:0/4082503786 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+83 (secure 0 0 0) 0x7fbc00015070 con 0x7fbc18105950 2024-04-02T04:40:02.448 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-04-02T04:40:02.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.447+0000 7fbc1d4fd700 1 -- 172.21.15.16:0/4082503786 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fbc0406cce0 msgr2=0x7fbc0406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:02.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.447+0000 7fbc1d4fd700 1 --2- 172.21.15.16:0/4082503786 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fbc0406cce0 0x7fbc0406f1a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbc08000f80 tx=0x7fbc080060c0 comp rx=0 tx=0).stop 2024-04-02T04:40:02.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.447+0000 7fbc1d4fd700 1 -- 172.21.15.16:0/4082503786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc18105950 msgr2=0x7fbc18196ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:02.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.447+0000 7fbc1d4fd700 1 --2- 172.21.15.16:0/4082503786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc18105950 0x7fbc18196ff0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbc0000b4b0 tx=0x7fbc0000b7c0 comp rx=0 tx=0).stop 2024-04-02T04:40:02.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.447+0000 7fbc1d4fd700 1 -- 172.21.15.16:0/4082503786 shutdown_connections 2024-04-02T04:40:02.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.447+0000 7fbc1d4fd700 1 --2- 172.21.15.16:0/4082503786 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fbc0406cce0 0x7fbc0406f1a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:02.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.447+0000 7fbc1d4fd700 1 --2- 172.21.15.16:0/4082503786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc180ff950 0x7fbc18196ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:02.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.447+0000 7fbc1d4fd700 1 --2- 172.21.15.16:0/4082503786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc18105950 0x7fbc18196ff0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:02.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.447+0000 7fbc1d4fd700 1 -- 172.21.15.16:0/4082503786 >> 172.21.15.16:0/4082503786 conn(0x7fbc18076640 msgr2=0x7fbc180fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:02.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.447+0000 7fbc1d4fd700 1 -- 172.21.15.16:0/4082503786 shutdown_connections 2024-04-02T04:40:02.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:02.447+0000 7fbc1d4fd700 1 -- 172.21.15.16:0/4082503786 wait complete. 2024-04-02T04:40:03.125 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-02T04:40:03.125 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-02T04:40:03.125 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-04-02T04:40:03.126 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-02T04:40:03.126 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-02T04:40:03.126 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-02T04:40:03.126 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-02T04:40:03.126 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-02T04:40:03.126 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-02T04:40:03.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:40:03.126 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-04-02T04:40:03.142 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-02T04:40:03.142 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-02T04:40:03.142 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-04-02T04:40:03.142 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-02T04:40:03.142 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-04-02T04:40:03.143 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-02T04:40:03.143 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T04:40:03.143 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-04-02T04:40:03.143 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-04-02T04:40:03.143 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T04:40:03.143 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-04-02T04:40:03.143 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-04-02T04:40:03.143 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-02T04:40:03.143 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-04-02T04:40:03.143 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-04-02T04:40:03.143 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-02T04:40:03.143 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42253sec preferred_lft 42253sec 2024-04-02T04:40:03.143 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2024-04-02T04:40:03.143 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-04-02T04:40:03.144 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T04:40:03.144 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-04-02T04:40:03.144 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-04-02T04:40:03.144 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-02T04:40:03.144 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T04:40:03.144 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-04-02T04:40:03.144 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add name ceph-brx type bridge 2024-04-02T04:40:03.144 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr flush dev ceph-brx 2024-04-02T04:40:03.144 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set ceph-brx up 2024-04-02T04:40:03.145 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-02T04:40:03.145 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-04-02T04:40:03.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T04:40:03.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T04:40:03.353 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:03 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:40:03.353 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:03 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:40:03.354 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:03 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:40:03.354 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:03 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:40:03.354 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:03 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:40:03.354 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:03 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/4082503786' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-02T04:40:03.354 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:03 smithi016 ceph-mon[28969]: pgmap v194: 65 pgs: 65 active+clean; 579 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-02T04:40:03.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:40:03.359 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-02T04:40:03.386 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-04-02T04:40:03.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:40:03.387 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-04-02T04:40:03.456 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-04-02T04:40:03.456 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-02T04:40:03.456 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-02T04:40:03.456 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-02T04:40:03.456 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-02T04:40:03.457 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T04:40:03.457 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-04-02T04:40:03.457 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-02T04:40:03.457 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-02T04:40:03.457 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-02T04:40:03.457 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-04-02T04:40:03.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T04:40:03.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:03 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:40:03.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:03 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:40:03.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:03 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:40:03.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:03 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:40:03.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:03 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:40:03.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:03 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/4082503786' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-02T04:40:03.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:03 smithi027 ceph-mon[37550]: pgmap v194: 65 pgs: 65 active+clean; 579 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-02T04:40:03.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T04:40:03.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:40:03.766 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-04-02T04:40:03.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:40:03.783 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2024-04-02T04:40:03.837 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T04:40:03.838 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-04-02T04:40:03.838 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T04:40:03.838 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-02T04:40:03.838 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-04-02T04:40:03.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T04:40:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T04:40:03.961 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-02T04:40:03.961 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T04:40:03.961 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-04-02T04:40:03.961 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-02T04:40:03.961 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-02T04:40:03.961 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-02T04:40:03.961 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-02T04:40:03.961 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-02T04:40:03.961 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-04-02T04:40:04.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T04:40:04.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T04:40:04.097 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T04:40:04.097 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-04-02T04:40:04.097 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-04-02T04:40:04.097 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.0 master ceph-brx 2024-04-02T04:40:04.097 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-04-02T04:40:04.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T04:40:04.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T04:40:04.178 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-02T04:40:04.178 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T04:40:04.178 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-02T04:40:04.192 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-02T04:40:04.193 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T04:40:04.193 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-02T04:40:04.247 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-04-02T04:40:04.315 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-04-02T04:40:04.371 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-04-02T04:40:04.371 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-04-02T04:40:04.371 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-04-02T04:40:04.371 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-04-02T04:40:04.371 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-04-02T04:40:04.371 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-04-02T04:40:04.372 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-04-02T04:40:04.372 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-04-02T04:40:04.372 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-04-02T04:40:04.372 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-04-02T04:40:04.372 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-04-02T04:40:04.372 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-04-02T04:40:04.372 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T04:40:04.372 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-04-02T04:40:04.372 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-04-02T04:40:04.372 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-04-02T04:40:04.372 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-04-02T04:40:04.372 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-02T04:40:04.372 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-04-02T04:40:04.373 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-04-02T04:40:04.373 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-04-02T04:40:04.373 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-02T04:40:04.373 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-04-02T04:40:04.373 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-04-02T04:40:04.373 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-04-02T04:40:04.373 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay 2024-04-02T04:40:04.373 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/b420ff4c58b813333614cb27f34ae98b21103186d1cbf33d5876218ffb6a37fc/merged 2024-04-02T04:40:04.373 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/ae5fc8d62dc6a93d1da59931796118ed84249deb46b24ea923adfb2128b30b47/merged 2024-04-02T04:40:04.373 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/0 2024-04-02T04:40:04.373 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T04:40:04.373 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/270ede68cbdb0dc9dcda9b8d5cd6d6725c56e1a786b551f21f444543294be15e/merged 2024-04-02T04:40:04.373 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/6048cadaaf406a147450a326a2a685830225b465be80f8a065cd242679395035/merged 2024-04-02T04:40:04.373 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/349d73fc93ad8a7ad0cbea183c0f51396ef7f8a04eb5f54ccccbeee93730a3c7/merged 2024-04-02T04:40:04.374 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/a99bf6f1d5ac9d80ba73c3eeb1b6c945a88eb266a9e3f144c3d72c7a87150a3e/merged 2024-04-02T04:40:04.374 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/057c5d05854cf568d8900459c751314c4288f7887edd1eb10df059df2d0bb9f3/merged 2024-04-02T04:40:04.374 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/be232eaf311841e932a10fee3d933287223735caafa5f0290e361734f313f9aa/merged 2024-04-02T04:40:04.374 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/77af707ed1786ec9c99841b5d61a615c2babcfe1b83f043c7d44472a7a8fd5de/merged 2024-04-02T04:40:04.374 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/eef9d4b9f776c3e4998e17ab8046864ae8f5718ab5a30f7e4979ca45ffb7b6ae/merged 2024-04-02T04:40:04.374 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/85c377920a3c2d3f7567a82e644f8f5a5f15edf4e45e6484710996e61f95734e/merged 2024-04-02T04:40:04.374 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/ca91c416b1cbc0f4d865d2899332d415dfcecaed5318d5b763e879e3ccd0698a/merged 2024-04-02T04:40:04.374 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/e30eca122cf4a83be6d3bcdaeb3041c5098cf5a8a303d01805131ea6514c60fd/merged 2024-04-02T04:40:04.374 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-04-02T04:40:04.374 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T04:40:04.374 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T04:40:04.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:40:04.375 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-04-02T04:40:04.426 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-02T04:40:04.426 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-02T04:40:04.469 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-04-02T04:40:04.496 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-04-02T04:40:04.552 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-04-02T04:40:04.552 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-04-02T04:40:04.552 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-04-02T04:40:04.552 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-04-02T04:40:04.552 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-04-02T04:40:04.552 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-04-02T04:40:04.552 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-04-02T04:40:04.553 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-04-02T04:40:04.553 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-04-02T04:40:04.553 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-04-02T04:40:04.553 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-04-02T04:40:04.553 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-04-02T04:40:04.553 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T04:40:04.553 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-04-02T04:40:04.553 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-04-02T04:40:04.553 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-04-02T04:40:04.553 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-04-02T04:40:04.553 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-02T04:40:04.553 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-04-02T04:40:04.553 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-04-02T04:40:04.553 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-04-02T04:40:04.554 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-02T04:40:04.554 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-04-02T04:40:04.554 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-04-02T04:40:04.554 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-04-02T04:40:04.554 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay 2024-04-02T04:40:04.554 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/b420ff4c58b813333614cb27f34ae98b21103186d1cbf33d5876218ffb6a37fc/merged 2024-04-02T04:40:04.554 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/ae5fc8d62dc6a93d1da59931796118ed84249deb46b24ea923adfb2128b30b47/merged 2024-04-02T04:40:04.554 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/0 2024-04-02T04:40:04.554 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T04:40:04.554 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/270ede68cbdb0dc9dcda9b8d5cd6d6725c56e1a786b551f21f444543294be15e/merged 2024-04-02T04:40:04.554 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/6048cadaaf406a147450a326a2a685830225b465be80f8a065cd242679395035/merged 2024-04-02T04:40:04.554 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/349d73fc93ad8a7ad0cbea183c0f51396ef7f8a04eb5f54ccccbeee93730a3c7/merged 2024-04-02T04:40:04.554 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/a99bf6f1d5ac9d80ba73c3eeb1b6c945a88eb266a9e3f144c3d72c7a87150a3e/merged 2024-04-02T04:40:04.554 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/057c5d05854cf568d8900459c751314c4288f7887edd1eb10df059df2d0bb9f3/merged 2024-04-02T04:40:04.555 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/be232eaf311841e932a10fee3d933287223735caafa5f0290e361734f313f9aa/merged 2024-04-02T04:40:04.555 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/77af707ed1786ec9c99841b5d61a615c2babcfe1b83f043c7d44472a7a8fd5de/merged 2024-04-02T04:40:04.555 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/eef9d4b9f776c3e4998e17ab8046864ae8f5718ab5a30f7e4979ca45ffb7b6ae/merged 2024-04-02T04:40:04.555 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/85c377920a3c2d3f7567a82e644f8f5a5f15edf4e45e6484710996e61f95734e/merged 2024-04-02T04:40:04.555 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/ca91c416b1cbc0f4d865d2899332d415dfcecaed5318d5b763e879e3ccd0698a/merged 2024-04-02T04:40:04.555 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/e30eca122cf4a83be6d3bcdaeb3041c5098cf5a8a303d01805131ea6514c60fd/merged 2024-04-02T04:40:04.555 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-04-02T04:40:04.555 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T04:40:04.555 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T04:40:04.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:40:04.555 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-04-02T04:40:04.575 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:2024-04-02T04:40:04.572+0000 7fec6643e480 -1 init, newargv = 0x5590320e3b00 newargc=15 2024-04-02T04:40:04.575 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[76979]: starting ceph client 2024-04-02T04:40:04.580 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[76979]: starting fuse 2024-04-02T04:40:04.589 INFO:teuthology.orchestra.run.smithi016.stdout:159 2024-04-02T04:40:04.590 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [159] 2024-04-02T04:40:04.590 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-04-02T04:40:04.590 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-04-02T04:40:04.590 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-04-02T04:40:04.590 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-04-02T04:40:04.590 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-04-02T04:40:04.590 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-02T04:40:04.590 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-04-02T04:40:04.590 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-02T04:40:04.591 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-04-02T04:40:04.591 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-02T04:40:04.591 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-02T04:40:04.591 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-04-02T04:40:04.591 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-04-02T04:40:04.591 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-04-02T04:40:04.591 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-04-02T04:40:04.591 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-02T04:40:04.591 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-04-02T04:40:04.591 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-02T04:40:04.591 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-02T04:40:04.591 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-02T04:40:04.591 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-04-02T04:40:04.591 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-04-02T04:40:04.591 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-04-02T04:40:04.591 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-02T04:40:04.591 DEBUG:teuthology.orchestra.run.smithi016:> 2024-04-02T04:40:04.592 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2024-04-02T04:40:04.592 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-04-02T04:40:04.710 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.76979.asok 2024-04-02T04:40:04.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T04:40:04.720 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.76979.asok 2024-04-02T04:40:04.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:40:04.721 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.76979.asok status 2024-04-02T04:40:04.858 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:40:04.858 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2024-04-02T04:40:04.858 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "4d05f4cad464de0fcdd11a2856643e533aea34e8", 2024-04-02T04:40:04.858 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)", 2024-04-02T04:40:04.858 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2024-04-02T04:40:04.859 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2024-04-02T04:40:04.859 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-02T04:40:04.859 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "76979", 2024-04-02T04:40:04.859 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2024-04-02T04:40:04.859 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:40:04.859 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2024-04-02T04:40:04.859 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2024-04-02T04:40:04.859 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 14540, 2024-04-02T04:40:04.859 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-04-02T04:40:04.859 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-04-02T04:40:04.859 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-04-02T04:40:04.859 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 14540 2024-04-02T04:40:04.860 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:40:04.860 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-04-02T04:40:04.860 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-04-02T04:40:04.860 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-04-02T04:40:04.860 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2963519639 2024-04-02T04:40:04.860 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-02T04:40:04.860 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:40:04.860 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-04-02T04:40:04.860 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-04-02T04:40:04.860 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-04-02T04:40:04.860 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2963519639 2024-04-02T04:40:04.860 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:40:04.860 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.14540 192.168.0.1:0/2963519639", 2024-04-02T04:40:04.860 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/2963519639", 2024-04-02T04:40:04.861 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2024-04-02T04:40:04.861 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 15, 2024-04-02T04:40:04.861 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 46, 2024-04-02T04:40:04.861 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2024-04-02T04:40:04.861 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2024-04-02T04:40:04.861 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2024-04-02T04:40:04.861 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:40:04.867 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- ceph fs ls 2024-04-02T04:40:05.076 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:40:06.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:05 smithi027 ceph-mon[37550]: pgmap v195: 65 pgs: 65 active+clean; 579 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 5.3 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-02T04:40:06.270 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:05 smithi016 ceph-mon[28969]: pgmap v195: 65 pgs: 65 active+clean; 579 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 5.3 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-02T04:40:06.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:06.799+0000 7fcf24245700 1 -- 172.21.15.16:0/3246188406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf1c0ff910 msgr2=0x7fcf1c0ffd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:06.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:06.799+0000 7fcf24245700 1 --2- 172.21.15.16:0/3246188406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf1c0ff910 0x7fcf1c0ffd90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c009910 tx=0x7fcf0c009c20 comp rx=0 tx=0).stop 2024-04-02T04:40:06.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:06.799+0000 7fcf24245700 1 -- 172.21.15.16:0/3246188406 shutdown_connections 2024-04-02T04:40:06.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:06.799+0000 7fcf24245700 1 --2- 172.21.15.16:0/3246188406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf1c0ff910 0x7fcf1c0ffd90 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:06.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:06.799+0000 7fcf24245700 1 --2- 172.21.15.16:0/3246188406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf1c105910 0x7fcf1c105cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:06.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:06.799+0000 7fcf24245700 1 -- 172.21.15.16:0/3246188406 >> 172.21.15.16:0/3246188406 conn(0x7fcf1c0fb440 msgr2=0x7fcf1c0fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:06.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:06.799+0000 7fcf24245700 1 -- 172.21.15.16:0/3246188406 shutdown_connections 2024-04-02T04:40:06.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:06.799+0000 7fcf24245700 1 -- 172.21.15.16:0/3246188406 wait complete. 2024-04-02T04:40:06.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:06.800+0000 7fcf24245700 1 Processor -- start 2024-04-02T04:40:06.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:06.800+0000 7fcf24245700 1 -- start start 2024-04-02T04:40:06.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:06.800+0000 7fcf24245700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf1c0ff910 0x7fcf1c10bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:06.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:06.800+0000 7fcf24245700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf1c105910 0x7fcf1c10c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:06.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:06.800+0000 7fcf24245700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1c10c580 con 0x7fcf1c105910 2024-04-02T04:40:06.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:06.800+0000 7fcf24245700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1c10c6c0 con 0x7fcf1c0ff910 2024-04-02T04:40:06.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:06.800+0000 7fcf217e0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf1c105910 0x7fcf1c10c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:06.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:06.800+0000 7fcf217e0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf1c105910 0x7fcf1c10c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39834/0 (socket says 172.21.15.16:39834) 2024-04-02T04:40:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:06.800+0000 7fcf217e0700 1 -- 172.21.15.16:0/856824600 learned_addr learned my addr 172.21.15.16:0/856824600 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:40:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:06.800+0000 7fcf21fe1700 1 --2- 172.21.15.16:0/856824600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf1c0ff910 0x7fcf1c10bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:06.801+0000 7fcf217e0700 1 -- 172.21.15.16:0/856824600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf1c0ff910 msgr2=0x7fcf1c10bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:06.801+0000 7fcf217e0700 1 --2- 172.21.15.16:0/856824600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf1c0ff910 0x7fcf1c10bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:06.801+0000 7fcf217e0700 1 -- 172.21.15.16:0/856824600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf0c0095f0 con 0x7fcf1c105910 2024-04-02T04:40:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:06.801+0000 7fcf217e0700 1 --2- 172.21.15.16:0/856824600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf1c105910 0x7fcf1c10c040 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c006010 tx=0x7fcf0c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:06.801+0000 7fcf12ffd700 1 -- 172.21.15.16:0/856824600 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf0c01d070 con 0x7fcf1c105910 2024-04-02T04:40:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:06.801+0000 7fcf12ffd700 1 -- 172.21.15.16:0/856824600 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcf0c00f460 con 0x7fcf1c105910 2024-04-02T04:40:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:06.801+0000 7fcf24245700 1 -- 172.21.15.16:0/856824600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf1c10c940 con 0x7fcf1c105910 2024-04-02T04:40:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:06.801+0000 7fcf12ffd700 1 -- 172.21.15.16:0/856824600 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf0c0175f0 con 0x7fcf1c105910 2024-04-02T04:40:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:06.801+0000 7fcf24245700 1 -- 172.21.15.16:0/856824600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf1c1a2e80 con 0x7fcf1c105910 2024-04-02T04:40:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:06.802+0000 7fcf24245700 1 -- 172.21.15.16:0/856824600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf1c107f30 con 0x7fcf1c105910 2024-04-02T04:40:06.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:06.803+0000 7fcf12ffd700 1 -- 172.21.15.16:0/856824600 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fcf0c003a60 con 0x7fcf1c105910 2024-04-02T04:40:06.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:06.803+0000 7fcf12ffd700 1 --2- 172.21.15.16:0/856824600 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fcf0806cce0 0x7fcf0806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:06.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:06.803+0000 7fcf12ffd700 1 -- 172.21.15.16:0/856824600 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fcf0c08d720 con 0x7fcf1c105910 2024-04-02T04:40:06.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:06.803+0000 7fcf21fe1700 1 --2- 172.21.15.16:0/856824600 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fcf0806cce0 0x7fcf0806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:06.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:06.804+0000 7fcf21fe1700 1 --2- 172.21.15.16:0/856824600 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fcf0806cce0 0x7fcf0806f1a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcf1400a400 tx=0x7fcf1400d040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:06.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:06.808+0000 7fcf12ffd700 1 -- 172.21.15.16:0/856824600 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fcf0c057a90 con 0x7fcf1c105910 2024-04-02T04:40:07.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:07.054+0000 7fcf24245700 1 -- 172.21.15.16:0/856824600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fcf1c101050 con 0x7fcf1c105910 2024-04-02T04:40:07.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:07.055+0000 7fcf12ffd700 1 -- 172.21.15.16:0/856824600 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+83 (secure 0 0 0) 0x7fcf0c005740 con 0x7fcf1c105910 2024-04-02T04:40:07.058 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-04-02T04:40:07.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:07.057+0000 7fcf24245700 1 -- 172.21.15.16:0/856824600 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fcf0806cce0 msgr2=0x7fcf0806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:07.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:07.057+0000 7fcf24245700 1 --2- 172.21.15.16:0/856824600 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fcf0806cce0 0x7fcf0806f1a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcf1400a400 tx=0x7fcf1400d040 comp rx=0 tx=0).stop 2024-04-02T04:40:07.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:07.057+0000 7fcf24245700 1 -- 172.21.15.16:0/856824600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf1c105910 msgr2=0x7fcf1c10c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:07.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:07.057+0000 7fcf24245700 1 --2- 172.21.15.16:0/856824600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf1c105910 0x7fcf1c10c040 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c006010 tx=0x7fcf0c00bd70 comp rx=0 tx=0).stop 2024-04-02T04:40:07.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:07.057+0000 7fcf24245700 1 -- 172.21.15.16:0/856824600 shutdown_connections 2024-04-02T04:40:07.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:07.057+0000 7fcf24245700 1 --2- 172.21.15.16:0/856824600 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fcf0806cce0 0x7fcf0806f1a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:07.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:07.057+0000 7fcf24245700 1 --2- 172.21.15.16:0/856824600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf1c0ff910 0x7fcf1c10bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:07.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:07.057+0000 7fcf24245700 1 --2- 172.21.15.16:0/856824600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf1c105910 0x7fcf1c10c040 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:07.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:07.057+0000 7fcf24245700 1 -- 172.21.15.16:0/856824600 >> 172.21.15.16:0/856824600 conn(0x7fcf1c0fb440 msgr2=0x7fcf1c0fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:07.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:07.057+0000 7fcf24245700 1 -- 172.21.15.16:0/856824600 shutdown_connections 2024-04-02T04:40:07.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:07.057+0000 7fcf24245700 1 -- 172.21.15.16:0/856824600 wait complete. 2024-04-02T04:40:07.733 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-02T04:40:07.733 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-02T04:40:07.733 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2024-04-02T04:40:07.733 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-02T04:40:07.733 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-02T04:40:07.733 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-02T04:40:07.733 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-02T04:40:07.733 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-02T04:40:07.734 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-02T04:40:07.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:40:07.734 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2024-04-02T04:40:07.753 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-02T04:40:07.753 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-02T04:40:07.753 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2024-04-02T04:40:07.754 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-04-02T04:40:07.754 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2024-04-02T04:40:07.754 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-04-02T04:40:07.754 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T04:40:07.754 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2024-04-02T04:40:07.754 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2024-04-02T04:40:07.754 INFO:teuthology.orchestra.run.smithi027.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-02T04:40:07.754 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2024-04-02T04:40:07.754 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2024-04-02T04:40:07.754 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-02T04:40:07.754 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 42260sec preferred_lft 42260sec 2024-04-02T04:40:07.754 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link noprefixroute 2024-04-02T04:40:07.754 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-04-02T04:40:07.754 INFO:teuthology.orchestra.run.smithi027.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T04:40:07.755 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2024-04-02T04:40:07.755 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2024-04-02T04:40:07.755 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T04:40:07.755 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2024-04-02T04:40:07.755 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2024-04-02T04:40:07.755 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-02T04:40:07.755 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T04:40:07.755 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-04-02T04:40:07.755 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add name ceph-brx type bridge 2024-04-02T04:40:07.755 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr flush dev ceph-brx 2024-04-02T04:40:07.755 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set ceph-brx up 2024-04-02T04:40:07.756 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-02T04:40:07.756 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-04-02T04:40:07.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:40:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T04:40:07.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:40:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T04:40:08.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:40:08.001 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-02T04:40:08.033 INFO:teuthology.orchestra.run.smithi027.stdout:1 2024-04-02T04:40:08.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:40:08.035 DEBUG:teuthology.orchestra.run.smithi027:> route 2024-04-02T04:40:08.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:07 smithi016 ceph-mon[28969]: pgmap v196: 65 pgs: 65 active+clean; 579 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-02T04:40:08.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:07 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/856824600' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-02T04:40:08.101 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2024-04-02T04:40:08.101 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-02T04:40:08.102 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-02T04:40:08.102 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-02T04:40:08.102 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-02T04:40:08.102 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T04:40:08.102 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-04-02T04:40:08.102 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-02T04:40:08.102 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-02T04:40:08.102 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-02T04:40:08.103 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-04-02T04:40:08.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:07 smithi027 ceph-mon[37550]: pgmap v196: 65 pgs: 65 active+clean; 579 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-02T04:40:08.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:07 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/856824600' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-02T04:40:08.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:40:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T04:40:08.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:40:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T04:40:08.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:40:08.432 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-04-02T04:40:08.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:40:08.489 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list-id 2024-04-02T04:40:08.545 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T04:40:08.546 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-04-02T04:40:08.546 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-02T04:40:08.546 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-02T04:40:08.546 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-04-02T04:40:08.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:40:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T04:40:08.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:40:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T04:40:08.680 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-02T04:40:08.680 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T04:40:08.680 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-04-02T04:40:08.680 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-02T04:40:08.680 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-02T04:40:08.680 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-02T04:40:08.680 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-02T04:40:08.680 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-02T04:40:08.681 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-04-02T04:40:08.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:40:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T04:40:08.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:40:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T04:40:08.841 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T04:40:08.841 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-04-02T04:40:08.841 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2024-04-02T04:40:08.841 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set dev brx.0 master ceph-brx 2024-04-02T04:40:08.841 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-04-02T04:40:08.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:40:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T04:40:08.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:40:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T04:40:08.931 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-02T04:40:08.931 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T04:40:08.931 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-02T04:40:08.946 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-02T04:40:08.947 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T04:40:08.947 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-02T04:40:09.002 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2024-04-02T04:40:09.073 DEBUG:teuthology.orchestra.run.smithi027:> cat /proc/self/mounts | awk '{print $2}' 2024-04-02T04:40:09.129 INFO:teuthology.orchestra.run.smithi027.stdout:/proc 2024-04-02T04:40:09.130 INFO:teuthology.orchestra.run.smithi027.stdout:/sys 2024-04-02T04:40:09.130 INFO:teuthology.orchestra.run.smithi027.stdout:/dev 2024-04-02T04:40:09.130 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/security 2024-04-02T04:40:09.130 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/shm 2024-04-02T04:40:09.130 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/pts 2024-04-02T04:40:09.130 INFO:teuthology.orchestra.run.smithi027.stdout:/run 2024-04-02T04:40:09.130 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup 2024-04-02T04:40:09.130 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/pstore 2024-04-02T04:40:09.130 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/bpf 2024-04-02T04:40:09.130 INFO:teuthology.orchestra.run.smithi027.stdout:/ 2024-04-02T04:40:09.130 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/selinux 2024-04-02T04:40:09.130 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T04:40:09.131 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/hugepages 2024-04-02T04:40:09.131 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/mqueue 2024-04-02T04:40:09.131 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/debug 2024-04-02T04:40:09.131 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/tracing 2024-04-02T04:40:09.131 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/fuse/connections 2024-04-02T04:40:09.131 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/config 2024-04-02T04:40:09.131 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-sysctl.service 2024-04-02T04:40:09.131 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-02T04:40:09.131 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-02T04:40:09.131 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/nfs/rpc_pipefs 2024-04-02T04:40:09.131 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/1000 2024-04-02T04:40:09.131 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/ceph 2024-04-02T04:40:09.131 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/0 2024-04-02T04:40:09.131 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T04:40:09.132 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay 2024-04-02T04:40:09.132 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/fd0a0e643c44c128f52d5e985eb8336a5dacf5fd3fa2b03c674c9a679fcc7105/merged 2024-04-02T04:40:09.132 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/15fae42bdc508147c81e5501ed93536e0a95552f775cac77cd90532ebf18f03d/merged 2024-04-02T04:40:09.132 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/d2ad4e9e329c34bebb3924572bebdf9503703290a8654a6c0263f87e276eab5d/merged 2024-04-02T04:40:09.132 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/36bb51636d9ff322a705b7fa4e7c19dab5dbe4f5cabde813401694d70e9a54b9/merged 2024-04-02T04:40:09.132 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/5f9966fc33fada609de7640ecd3f40200486e072b9d80a2aca30524b7320752e/merged 2024-04-02T04:40:09.132 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/f60c6fda4e019331d692f71df86940133b81edead3dd089742319f4001226a96/merged 2024-04-02T04:40:09.132 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/84c0d996857b2c6224e9f46fe57c6ebce52ed6c005b48f74b757f10da6ed760e/merged 2024-04-02T04:40:09.132 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/00c0efd02b790c8d4b66d4aea07b42dd34ffcc83e42238d2b1bdb871a32d2487/merged 2024-04-02T04:40:09.132 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/5a8cd9d19c62ce7404d711e961fe85a5127e2047e334830c7edc6c48f8219ed8/merged 2024-04-02T04:40:09.132 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/a5bf8f9aa84ef05ac26b36e40041f94ae155bb90cf4db1f2c6a45ee55b293ac9/merged 2024-04-02T04:40:09.132 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns 2024-04-02T04:40:09.132 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-02T04:40:09.132 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-02T04:40:09.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:40:09.133 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2024-04-02T04:40:09.186 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-02T04:40:09.186 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-02T04:40:09.228 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2024-04-02T04:40:09.259 DEBUG:teuthology.orchestra.run.smithi027:> cat /proc/self/mounts | awk '{print $2}' 2024-04-02T04:40:09.315 INFO:teuthology.orchestra.run.smithi027.stdout:/proc 2024-04-02T04:40:09.315 INFO:teuthology.orchestra.run.smithi027.stdout:/sys 2024-04-02T04:40:09.315 INFO:teuthology.orchestra.run.smithi027.stdout:/dev 2024-04-02T04:40:09.315 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/security 2024-04-02T04:40:09.316 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/shm 2024-04-02T04:40:09.316 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/pts 2024-04-02T04:40:09.316 INFO:teuthology.orchestra.run.smithi027.stdout:/run 2024-04-02T04:40:09.316 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup 2024-04-02T04:40:09.316 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/pstore 2024-04-02T04:40:09.316 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/bpf 2024-04-02T04:40:09.316 INFO:teuthology.orchestra.run.smithi027.stdout:/ 2024-04-02T04:40:09.316 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/selinux 2024-04-02T04:40:09.316 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T04:40:09.316 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/hugepages 2024-04-02T04:40:09.316 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/mqueue 2024-04-02T04:40:09.316 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/debug 2024-04-02T04:40:09.316 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/tracing 2024-04-02T04:40:09.316 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/fuse/connections 2024-04-02T04:40:09.316 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/config 2024-04-02T04:40:09.317 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-sysctl.service 2024-04-02T04:40:09.317 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-02T04:40:09.317 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-02T04:40:09.317 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/nfs/rpc_pipefs 2024-04-02T04:40:09.317 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/1000 2024-04-02T04:40:09.317 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/ceph 2024-04-02T04:40:09.317 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/0 2024-04-02T04:40:09.317 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T04:40:09.317 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay 2024-04-02T04:40:09.317 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/fd0a0e643c44c128f52d5e985eb8336a5dacf5fd3fa2b03c674c9a679fcc7105/merged 2024-04-02T04:40:09.317 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/15fae42bdc508147c81e5501ed93536e0a95552f775cac77cd90532ebf18f03d/merged 2024-04-02T04:40:09.317 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/d2ad4e9e329c34bebb3924572bebdf9503703290a8654a6c0263f87e276eab5d/merged 2024-04-02T04:40:09.317 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/36bb51636d9ff322a705b7fa4e7c19dab5dbe4f5cabde813401694d70e9a54b9/merged 2024-04-02T04:40:09.317 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/5f9966fc33fada609de7640ecd3f40200486e072b9d80a2aca30524b7320752e/merged 2024-04-02T04:40:09.317 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/f60c6fda4e019331d692f71df86940133b81edead3dd089742319f4001226a96/merged 2024-04-02T04:40:09.318 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/84c0d996857b2c6224e9f46fe57c6ebce52ed6c005b48f74b757f10da6ed760e/merged 2024-04-02T04:40:09.318 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/00c0efd02b790c8d4b66d4aea07b42dd34ffcc83e42238d2b1bdb871a32d2487/merged 2024-04-02T04:40:09.318 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/5a8cd9d19c62ce7404d711e961fe85a5127e2047e334830c7edc6c48f8219ed8/merged 2024-04-02T04:40:09.318 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/a5bf8f9aa84ef05ac26b36e40041f94ae155bb90cf4db1f2c6a45ee55b293ac9/merged 2024-04-02T04:40:09.318 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns 2024-04-02T04:40:09.318 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-02T04:40:09.318 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-02T04:40:09.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:40:09.318 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2024-04-02T04:40:09.348 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi027.stderr:2024-04-02T04:40:09.346+0000 7f77ef5f8480 -1 init, newargv = 0x555bf7f4cb30 newargc=15 2024-04-02T04:40:09.348 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi027.stderr:ceph-fuse[63139]: starting ceph client 2024-04-02T04:40:09.353 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi027.stderr:ceph-fuse[63139]: starting fuse 2024-04-02T04:40:09.364 INFO:teuthology.orchestra.run.smithi027.stdout:126 2024-04-02T04:40:09.365 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [126] 2024-04-02T04:40:09.365 DEBUG:teuthology.orchestra.run.smithi027:> sudo stdin-killer -- python3 -c ' 2024-04-02T04:40:09.365 DEBUG:teuthology.orchestra.run.smithi027:> import glob 2024-04-02T04:40:09.365 DEBUG:teuthology.orchestra.run.smithi027:> import re 2024-04-02T04:40:09.365 DEBUG:teuthology.orchestra.run.smithi027:> import os 2024-04-02T04:40:09.365 DEBUG:teuthology.orchestra.run.smithi027:> import subprocess 2024-04-02T04:40:09.365 DEBUG:teuthology.orchestra.run.smithi027:> 2024-04-02T04:40:09.365 DEBUG:teuthology.orchestra.run.smithi027:> def _find_admin_socket(client_name): 2024-04-02T04:40:09.365 DEBUG:teuthology.orchestra.run.smithi027:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-02T04:40:09.365 DEBUG:teuthology.orchestra.run.smithi027:> files = glob.glob(asok_path) 2024-04-02T04:40:09.366 DEBUG:teuthology.orchestra.run.smithi027:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-02T04:40:09.366 DEBUG:teuthology.orchestra.run.smithi027:> 2024-04-02T04:40:09.366 DEBUG:teuthology.orchestra.run.smithi027:> # Given a non-glob path, it better be there 2024-04-02T04:40:09.366 DEBUG:teuthology.orchestra.run.smithi027:> if "*" not in asok_path: 2024-04-02T04:40:09.366 DEBUG:teuthology.orchestra.run.smithi027:> assert(len(files) == 1) 2024-04-02T04:40:09.366 DEBUG:teuthology.orchestra.run.smithi027:> return files[0] 2024-04-02T04:40:09.366 DEBUG:teuthology.orchestra.run.smithi027:> 2024-04-02T04:40:09.366 DEBUG:teuthology.orchestra.run.smithi027:> for f in files: 2024-04-02T04:40:09.366 DEBUG:teuthology.orchestra.run.smithi027:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-02T04:40:09.366 DEBUG:teuthology.orchestra.run.smithi027:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-02T04:40:09.366 DEBUG:teuthology.orchestra.run.smithi027:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-02T04:40:09.366 DEBUG:teuthology.orchestra.run.smithi027:> contents = proc_f.read() 2024-04-02T04:40:09.366 DEBUG:teuthology.orchestra.run.smithi027:> if mountpoint in contents: 2024-04-02T04:40:09.366 DEBUG:teuthology.orchestra.run.smithi027:> return f 2024-04-02T04:40:09.366 DEBUG:teuthology.orchestra.run.smithi027:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-02T04:40:09.366 DEBUG:teuthology.orchestra.run.smithi027:> 2024-04-02T04:40:09.367 DEBUG:teuthology.orchestra.run.smithi027:> print(_find_admin_socket("client.1")) 2024-04-02T04:40:09.367 DEBUG:teuthology.orchestra.run.smithi027:> ' 2024-04-02T04:40:09.491 INFO:teuthology.orchestra.run.smithi027.stdout:/var/run/ceph/ceph-client.1.63139.asok 2024-04-02T04:40:09.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-02T04:40:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T04:40:09.502 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.63139.asok 2024-04-02T04:40:09.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:40:09.502 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.63139.asok status 2024-04-02T04:40:09.644 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-02T04:40:09.644 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": { 2024-04-02T04:40:09.644 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_sha1": "4d05f4cad464de0fcdd11a2856643e533aea34e8", 2024-04-02T04:40:09.644 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)", 2024-04-02T04:40:09.644 INFO:teuthology.orchestra.run.smithi027.stdout: "entity_id": "1", 2024-04-02T04:40:09.644 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": "smithi027", 2024-04-02T04:40:09.644 INFO:teuthology.orchestra.run.smithi027.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-02T04:40:09.644 INFO:teuthology.orchestra.run.smithi027.stdout: "pid": "63139", 2024-04-02T04:40:09.644 INFO:teuthology.orchestra.run.smithi027.stdout: "root": "/" 2024-04-02T04:40:09.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-02T04:40:09.645 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_count": 0, 2024-04-02T04:40:09.645 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_pinned_count": 0, 2024-04-02T04:40:09.645 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24337, 2024-04-02T04:40:09.645 INFO:teuthology.orchestra.run.smithi027.stdout: "inst": { 2024-04-02T04:40:09.645 INFO:teuthology.orchestra.run.smithi027.stdout: "name": { 2024-04-02T04:40:09.645 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "client", 2024-04-02T04:40:09.645 INFO:teuthology.orchestra.run.smithi027.stdout: "num": 24337 2024-04-02T04:40:09.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-02T04:40:09.645 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2024-04-02T04:40:09.645 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-04-02T04:40:09.645 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2024-04-02T04:40:09.645 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 392542449 2024-04-02T04:40:09.645 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-02T04:40:09.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-02T04:40:09.646 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2024-04-02T04:40:09.646 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-04-02T04:40:09.646 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2024-04-02T04:40:09.646 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 392542449 2024-04-02T04:40:09.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-04-02T04:40:09.646 INFO:teuthology.orchestra.run.smithi027.stdout: "inst_str": "client.24337 192.168.0.1:0/392542449", 2024-04-02T04:40:09.646 INFO:teuthology.orchestra.run.smithi027.stdout: "addr_str": "192.168.0.1:0/392542449", 2024-04-02T04:40:09.646 INFO:teuthology.orchestra.run.smithi027.stdout: "inode_count": 1, 2024-04-02T04:40:09.646 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_epoch": 15, 2024-04-02T04:40:09.646 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch": 46, 2024-04-02T04:40:09.646 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch_barrier": 0, 2024-04-02T04:40:09.646 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklisted": false, 2024-04-02T04:40:09.646 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs" 2024-04-02T04:40:09.646 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-02T04:40:09.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:40:09.652 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2024-04-02T04:40:09.652 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-02T04:40:09.672 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2024-04-02T04:40:09.672 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-02T04:40:09.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:40:09.673 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-02T04:40:09.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:40:09.747 DEBUG:teuthology.orchestra.run.smithi027:> stat --file-system '--printf=%T 2024-04-02T04:40:09.748 DEBUG:teuthology.orchestra.run.smithi027:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-02T04:40:09.763 INFO:teuthology.orchestra.run.smithi027.stdout:fuseblk 2024-04-02T04:40:09.763 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-02T04:40:09.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:40:09.764 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-02T04:40:09.837 INFO:teuthology.run_tasks:Running task print... 2024-04-02T04:40:09.848 INFO:teuthology.task.print:**** done client 2024-04-02T04:40:09.848 INFO:teuthology.run_tasks:Running task parallel... 2024-04-02T04:40:09.884 INFO:teuthology.task.parallel:starting parallel... 2024-04-02T04:40:09.884 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-04-02T04:40:09.885 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-02T04:40:09.885 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi016.front.sepia.ceph.com 2024-04-02T04:40:09.886 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs false || true' 2024-04-02T04:40:09.886 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-04-02T04:40:09.886 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-04-02T04:40:09.890 INFO:tasks.workunit:Pulling workunits from ref 6cec793d9073f0594e2364b9b687ecaf22424cca 2024-04-02T04:40:09.890 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-02T04:40:09.890 INFO:tasks.workunit:timeout=3h 2024-04-02T04:40:09.890 INFO:tasks.workunit:cleanup=True 2024-04-02T04:40:09.890 DEBUG:teuthology.orchestra.run.smithi016:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-02T04:40:09.909 INFO:teuthology.orchestra.run.smithi016.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-02T04:40:09.909 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-02T04:40:09.909 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 9fh/159d Inode: 1 Links: 2 2024-04-02T04:40:09.909 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T04:40:09.909 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-02T04:40:09.910 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-02T04:40:09.910 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-04-02 04:39:35.822209421 +0000 2024-04-02T04:40:09.910 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-04-02 04:40:09.742714107 +0000 2024-04-02T04:40:09.910 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: - 2024-04-02T04:40:09.910 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-02T04:40:09.910 DEBUG:teuthology.orchestra.run.smithi016:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-02T04:40:09.982 DEBUG:teuthology.orchestra.run.smithi027:> stat -- /home/ubuntu/cephtest/mnt.1 2024-04-02T04:40:09.999 INFO:teuthology.orchestra.run.smithi027.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-04-02T04:40:09.999 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-02T04:40:09.999 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 7eh/126d Inode: 1 Links: 3 2024-04-02T04:40:09.999 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T04:40:09.999 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-02T04:40:09.999 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-02T04:40:09.999 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-04-02 04:40:09.975352822 +0000 2024-04-02T04:40:09.999 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-04-02 04:40:09.975352822 +0000 2024-04-02T04:40:09.999 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: - 2024-04-02T04:40:10.000 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-04-02T04:40:10.000 DEBUG:teuthology.orchestra.run.smithi027:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-04-02T04:40:10.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:09 smithi016 ceph-mon[28969]: pgmap v197: 65 pgs: 65 active+clean; 579 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-02T04:40:10.067 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:09 smithi027 ceph-mon[37550]: pgmap v197: 65 pgs: 65 active+clean; 579 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-02T04:40:10.075 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 6cec793d9073f0594e2364b9b687ecaf22424cca 2024-04-02T04:40:10.075 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout 6cec793d9073f0594e2364b9b687ecaf22424cca 2024-04-02T04:40:10.090 INFO:tasks.workunit.client.0.smithi016.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-02T04:40:10.111 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:40:10.131 INFO:tasks.workunit.client.1.smithi027.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-04-02T04:40:11.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.582+0000 7f0b37173700 1 -- 172.21.15.16:0/4131392935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b300ff9f0 msgr2=0x7f0b300ffe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:11.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.582+0000 7f0b37173700 1 --2- 172.21.15.16:0/4131392935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b300ff9f0 0x7f0b300ffe10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0b20009910 tx=0x7f0b20009c20 comp rx=0 tx=0).stop 2024-04-02T04:40:11.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.582+0000 7f0b37173700 1 -- 172.21.15.16:0/4131392935 shutdown_connections 2024-04-02T04:40:11.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.582+0000 7f0b37173700 1 --2- 172.21.15.16:0/4131392935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b30100bf0 0x7f0b30101050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:11.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.582+0000 7f0b37173700 1 --2- 172.21.15.16:0/4131392935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b300ff9f0 0x7f0b300ffe10 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:11.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.582+0000 7f0b37173700 1 -- 172.21.15.16:0/4131392935 >> 172.21.15.16:0/4131392935 conn(0x7f0b300faf90 msgr2=0x7f0b300fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:11.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.582+0000 7f0b37173700 1 -- 172.21.15.16:0/4131392935 shutdown_connections 2024-04-02T04:40:11.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.582+0000 7f0b37173700 1 -- 172.21.15.16:0/4131392935 wait complete. 2024-04-02T04:40:11.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.582+0000 7f0b37173700 1 Processor -- start 2024-04-02T04:40:11.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.583+0000 7f0b37173700 1 -- start start 2024-04-02T04:40:11.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.583+0000 7f0b37173700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b300ff9f0 0x7f0b30069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:11.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.583+0000 7f0b37173700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b30100bf0 0x7f0b3019b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:11.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.583+0000 7f0b37173700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3019b840 con 0x7f0b30100bf0 2024-04-02T04:40:11.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.583+0000 7f0b37173700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3019b980 con 0x7f0b300ff9f0 2024-04-02T04:40:11.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.583+0000 7f0b2ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b30100bf0 0x7f0b3019b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:11.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.583+0000 7f0b2ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b30100bf0 0x7f0b3019b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39846/0 (socket says 172.21.15.16:39846) 2024-04-02T04:40:11.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.583+0000 7f0b2ffff700 1 -- 172.21.15.16:0/3645772481 learned_addr learned my addr 172.21.15.16:0/3645772481 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:40:11.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.583+0000 7f0b34f0f700 1 --2- 172.21.15.16:0/3645772481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b300ff9f0 0x7f0b30069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:11.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.583+0000 7f0b2ffff700 1 -- 172.21.15.16:0/3645772481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b300ff9f0 msgr2=0x7f0b30069ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:11.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.583+0000 7f0b2ffff700 1 --2- 172.21.15.16:0/3645772481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b300ff9f0 0x7f0b30069ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:11.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.583+0000 7f0b2ffff700 1 -- 172.21.15.16:0/3645772481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b200095f0 con 0x7f0b30100bf0 2024-04-02T04:40:11.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.583+0000 7f0b34f0f700 1 --2- 172.21.15.16:0/3645772481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b300ff9f0 0x7f0b30069ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T04:40:11.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.583+0000 7f0b2ffff700 1 --2- 172.21.15.16:0/3645772481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b30100bf0 0x7f0b3019b300 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0b2400d900 tx=0x7f0b2400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:11.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.584+0000 7f0b2dffb700 1 -- 172.21.15.16:0/3645772481 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b240041f0 con 0x7f0b30100bf0 2024-04-02T04:40:11.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.584+0000 7f0b2dffb700 1 -- 172.21.15.16:0/3645772481 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0b24004cf0 con 0x7f0b30100bf0 2024-04-02T04:40:11.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.584+0000 7f0b37173700 1 -- 172.21.15.16:0/3645772481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b3019bc30 con 0x7f0b30100bf0 2024-04-02T04:40:11.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.584+0000 7f0b2dffb700 1 -- 172.21.15.16:0/3645772481 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b2400b750 con 0x7f0b30100bf0 2024-04-02T04:40:11.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.584+0000 7f0b37173700 1 -- 172.21.15.16:0/3645772481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b3019c180 con 0x7f0b30100bf0 2024-04-02T04:40:11.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.585+0000 7f0b37173700 1 -- 172.21.15.16:0/3645772481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b3018f570 con 0x7f0b30100bf0 2024-04-02T04:40:11.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.585+0000 7f0b2dffb700 1 -- 172.21.15.16:0/3645772481 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f0b24004350 con 0x7f0b30100bf0 2024-04-02T04:40:11.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.585+0000 7f0b2dffb700 1 --2- 172.21.15.16:0/3645772481 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f0b1806cce0 0x7f0b1806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:11.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.586+0000 7f0b2dffb700 1 -- 172.21.15.16:0/3645772481 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f0b2408c460 con 0x7f0b30100bf0 2024-04-02T04:40:11.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.586+0000 7f0b34f0f700 1 --2- 172.21.15.16:0/3645772481 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f0b1806cce0 0x7f0b1806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:11.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.586+0000 7f0b34f0f700 1 --2- 172.21.15.16:0/3645772481 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f0b1806cce0 0x7f0b1806f1a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0b20005090 tx=0x7f0b2001a040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:11.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.590+0000 7f0b2dffb700 1 -- 172.21.15.16:0/3645772481 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0b2405a260 con 0x7f0b30100bf0 2024-04-02T04:40:11.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.795+0000 7f0b37173700 1 -- 172.21.15.16:0/3645772481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7f0b3004ea80 con 0x7f0b30100bf0 2024-04-02T04:40:11.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.799+0000 7f0b2dffb700 1 -- 172.21.15.16:0/3645772481 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v26)=0 v26) v1 ==== 126+0+0 (secure 0 0 0) 0x7f0b24059df0 con 0x7f0b30100bf0 2024-04-02T04:40:11.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.801+0000 7f0b37173700 1 -- 172.21.15.16:0/3645772481 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f0b1806cce0 msgr2=0x7f0b1806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:11.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.801+0000 7f0b37173700 1 --2- 172.21.15.16:0/3645772481 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f0b1806cce0 0x7f0b1806f1a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0b20005090 tx=0x7f0b2001a040 comp rx=0 tx=0).stop 2024-04-02T04:40:11.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.801+0000 7f0b37173700 1 -- 172.21.15.16:0/3645772481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b30100bf0 msgr2=0x7f0b3019b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:11.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.801+0000 7f0b37173700 1 --2- 172.21.15.16:0/3645772481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b30100bf0 0x7f0b3019b300 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0b2400d900 tx=0x7f0b2400dcc0 comp rx=0 tx=0).stop 2024-04-02T04:40:11.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.801+0000 7f0b37173700 1 -- 172.21.15.16:0/3645772481 shutdown_connections 2024-04-02T04:40:11.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.801+0000 7f0b37173700 1 --2- 172.21.15.16:0/3645772481 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f0b1806cce0 0x7f0b1806f1a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:11.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.801+0000 7f0b37173700 1 --2- 172.21.15.16:0/3645772481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b300ff9f0 0x7f0b30069ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:11.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.801+0000 7f0b37173700 1 --2- 172.21.15.16:0/3645772481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b30100bf0 0x7f0b3019b300 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:11.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.801+0000 7f0b37173700 1 -- 172.21.15.16:0/3645772481 >> 172.21.15.16:0/3645772481 conn(0x7f0b300faf90 msgr2=0x7f0b30104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:11.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.801+0000 7f0b37173700 1 -- 172.21.15.16:0/3645772481 shutdown_connections 2024-04-02T04:40:11.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:11.802+0000 7f0b37173700 1 -- 172.21.15.16:0/3645772481 wait complete. 2024-04-02T04:40:12.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:11 smithi016 ceph-mon[28969]: pgmap v198: 65 pgs: 65 active+clean; 579 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-02T04:40:12.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:11 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/3645772481' entity='client.admin' 2024-04-02T04:40:12.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:11 smithi027 ceph-mon[37550]: pgmap v198: 65 pgs: 65 active+clean; 579 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-02T04:40:12.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:11 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/3645772481' entity='client.admin' 2024-04-02T04:40:12.436 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-02T04:40:12.437 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi016.front.sepia.ceph.com 2024-04-02T04:40:12.437 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-04-02T04:40:12.688 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:40:13.078 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:12 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:40:13.078 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:12 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:40:13.078 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:12 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:40:13.078 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:12 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:40:13.078 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:12 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:40:13.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:12 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:40:13.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:12 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:40:13.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:12 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:40:13.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:12 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:40:13.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:12 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:40:14.044 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:13 smithi016 ceph-mon[28969]: pgmap v199: 65 pgs: 65 active+clean; 579 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-02T04:40:14.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:13 smithi027 ceph-mon[37550]: pgmap v199: 65 pgs: 65 active+clean; 579 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-02T04:40:14.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.271+0000 7fae0daf0700 1 -- 172.21.15.16:0/2246150296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae08100be0 msgr2=0x7fae08101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:14.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.271+0000 7fae0daf0700 1 --2- 172.21.15.16:0/2246150296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae08100be0 0x7fae08101040 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fadfc009910 tx=0x7fadfc009c20 comp rx=0 tx=0).stop 2024-04-02T04:40:14.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.271+0000 7fae0daf0700 1 -- 172.21.15.16:0/2246150296 shutdown_connections 2024-04-02T04:40:14.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.271+0000 7fae0daf0700 1 --2- 172.21.15.16:0/2246150296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae08100be0 0x7fae08101040 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:14.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.271+0000 7fae0daf0700 1 --2- 172.21.15.16:0/2246150296 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae080ff9e0 0x7fae080ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:14.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.271+0000 7fae0daf0700 1 -- 172.21.15.16:0/2246150296 >> 172.21.15.16:0/2246150296 conn(0x7fae080faf80 msgr2=0x7fae080fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:14.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.271+0000 7fae0daf0700 1 -- 172.21.15.16:0/2246150296 shutdown_connections 2024-04-02T04:40:14.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.271+0000 7fae0daf0700 1 -- 172.21.15.16:0/2246150296 wait complete. 2024-04-02T04:40:14.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.272+0000 7fae0daf0700 1 Processor -- start 2024-04-02T04:40:14.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.272+0000 7fae0daf0700 1 -- start start 2024-04-02T04:40:14.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.272+0000 7fae0daf0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae080ff9e0 0x7fae08069a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:14.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.272+0000 7fae0daf0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae08100be0 0x7fae0819b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:14.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.272+0000 7fae0daf0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0819b870 con 0x7fae08100be0 2024-04-02T04:40:14.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.272+0000 7fae0daf0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0819b9e0 con 0x7fae080ff9e0 2024-04-02T04:40:14.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.272+0000 7fae06ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae08100be0 0x7fae0819b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:14.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.272+0000 7fae06ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae08100be0 0x7fae0819b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46776/0 (socket says 172.21.15.16:46776) 2024-04-02T04:40:14.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.272+0000 7fae06ffd700 1 -- 172.21.15.16:0/2185846511 learned_addr learned my addr 172.21.15.16:0/2185846511 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:40:14.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.272+0000 7fae077fe700 1 --2- 172.21.15.16:0/2185846511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae080ff9e0 0x7fae08069a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:14.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.272+0000 7fae06ffd700 1 -- 172.21.15.16:0/2185846511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae080ff9e0 msgr2=0x7fae08069a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:14.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.272+0000 7fae06ffd700 1 --2- 172.21.15.16:0/2185846511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae080ff9e0 0x7fae08069a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:14.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.272+0000 7fae06ffd700 1 -- 172.21.15.16:0/2185846511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadfc0095f0 con 0x7fae08100be0 2024-04-02T04:40:14.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.272+0000 7fae077fe700 1 --2- 172.21.15.16:0/2185846511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae080ff9e0 0x7fae08069a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T04:40:14.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.273+0000 7fae06ffd700 1 --2- 172.21.15.16:0/2185846511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae08100be0 0x7fae0819b330 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fadfc0092e0 tx=0x7fadfc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:14.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.273+0000 7fae04ff9700 1 -- 172.21.15.16:0/2185846511 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadfc01d070 con 0x7fae08100be0 2024-04-02T04:40:14.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.273+0000 7fae04ff9700 1 -- 172.21.15.16:0/2185846511 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fadfc00f460 con 0x7fae08100be0 2024-04-02T04:40:14.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.273+0000 7fae0daf0700 1 -- 172.21.15.16:0/2185846511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae0819bc60 con 0x7fae08100be0 2024-04-02T04:40:14.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.273+0000 7fae04ff9700 1 -- 172.21.15.16:0/2185846511 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadfc021640 con 0x7fae08100be0 2024-04-02T04:40:14.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.273+0000 7fae0daf0700 1 -- 172.21.15.16:0/2185846511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae0819c1d0 con 0x7fae08100be0 2024-04-02T04:40:14.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.274+0000 7fae0daf0700 1 -- 172.21.15.16:0/2185846511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae0818f600 con 0x7fae08100be0 2024-04-02T04:40:14.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.274+0000 7fae04ff9700 1 -- 172.21.15.16:0/2185846511 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fadfc00f5d0 con 0x7fae08100be0 2024-04-02T04:40:14.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.275+0000 7fae04ff9700 1 --2- 172.21.15.16:0/2185846511 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fadf006cce0 0x7fadf006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:14.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.275+0000 7fae04ff9700 1 -- 172.21.15.16:0/2185846511 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fadfc08d620 con 0x7fae08100be0 2024-04-02T04:40:14.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.275+0000 7fae077fe700 1 --2- 172.21.15.16:0/2185846511 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fadf006cce0 0x7fadf006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:14.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.275+0000 7fae077fe700 1 --2- 172.21.15.16:0/2185846511 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fadf006cce0 0x7fadf006f1a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fae08100a40 tx=0x7fadf8008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:14.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.279+0000 7fae04ff9700 1 -- 172.21.15.16:0/2185846511 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fadfc05b420 con 0x7fae08100be0 2024-04-02T04:40:14.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.482+0000 7fae0daf0700 1 -- 172.21.15.16:0/2185846511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7fae0804ea80 con 0x7fae08100be0 2024-04-02T04:40:14.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.482+0000 7fae04ff9700 1 -- 172.21.15.16:0/2185846511 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v26)=0 v26) v1 ==== 155+0+0 (secure 0 0 0) 0x7fadfc05afb0 con 0x7fae08100be0 2024-04-02T04:40:14.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.484+0000 7fae0daf0700 1 -- 172.21.15.16:0/2185846511 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fadf006cce0 msgr2=0x7fadf006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:14.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.484+0000 7fae0daf0700 1 --2- 172.21.15.16:0/2185846511 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fadf006cce0 0x7fadf006f1a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fae08100a40 tx=0x7fadf8008040 comp rx=0 tx=0).stop 2024-04-02T04:40:14.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.484+0000 7fae0daf0700 1 -- 172.21.15.16:0/2185846511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae08100be0 msgr2=0x7fae0819b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:14.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.484+0000 7fae0daf0700 1 --2- 172.21.15.16:0/2185846511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae08100be0 0x7fae0819b330 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fadfc0092e0 tx=0x7fadfc00be20 comp rx=0 tx=0).stop 2024-04-02T04:40:14.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.484+0000 7fae0daf0700 1 -- 172.21.15.16:0/2185846511 shutdown_connections 2024-04-02T04:40:14.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.484+0000 7fae0daf0700 1 --2- 172.21.15.16:0/2185846511 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fadf006cce0 0x7fadf006f1a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:14.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.484+0000 7fae0daf0700 1 --2- 172.21.15.16:0/2185846511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae080ff9e0 0x7fae08069a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:14.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.484+0000 7fae0daf0700 1 --2- 172.21.15.16:0/2185846511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae08100be0 0x7fae0819b330 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:14.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.484+0000 7fae0daf0700 1 -- 172.21.15.16:0/2185846511 >> 172.21.15.16:0/2185846511 conn(0x7fae080faf80 msgr2=0x7fae081041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:14.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.484+0000 7fae0daf0700 1 -- 172.21.15.16:0/2185846511 shutdown_connections 2024-04-02T04:40:14.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:14.484+0000 7fae0daf0700 1 -- 172.21.15.16:0/2185846511 wait complete. 2024-04-02T04:40:15.127 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-04-02T04:40:15.337 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:40:16.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:15 smithi016 ceph-mon[28969]: pgmap v200: 65 pgs: 65 active+clean; 585 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-04-02T04:40:16.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:15 smithi027 ceph-mon[37550]: pgmap v200: 65 pgs: 65 active+clean; 585 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-04-02T04:40:16.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:16.898+0000 7f8382893700 1 -- 172.21.15.16:0/2072107576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f837c100ae0 msgr2=0x7f837c100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:16.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:16.898+0000 7f8382893700 1 --2- 172.21.15.16:0/2072107576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f837c100ae0 0x7f837c100f00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f836c009910 tx=0x7f836c009c20 comp rx=0 tx=0).stop 2024-04-02T04:40:16.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:16.898+0000 7f8382893700 1 -- 172.21.15.16:0/2072107576 shutdown_connections 2024-04-02T04:40:16.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:16.898+0000 7f8382893700 1 --2- 172.21.15.16:0/2072107576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f837c101ce0 0x7f837c102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:16.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:16.898+0000 7f8382893700 1 --2- 172.21.15.16:0/2072107576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f837c100ae0 0x7f837c100f00 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:16.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:16.898+0000 7f8382893700 1 -- 172.21.15.16:0/2072107576 >> 172.21.15.16:0/2072107576 conn(0x7f837c0fc0a0 msgr2=0x7f837c0fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:16.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:16.898+0000 7f8382893700 1 -- 172.21.15.16:0/2072107576 shutdown_connections 2024-04-02T04:40:16.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:16.898+0000 7f8382893700 1 -- 172.21.15.16:0/2072107576 wait complete. 2024-04-02T04:40:16.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:16.898+0000 7f8382893700 1 Processor -- start 2024-04-02T04:40:16.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:16.898+0000 7f8382893700 1 -- start start 2024-04-02T04:40:16.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:16.899+0000 7f8382893700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f837c100ae0 0x7f837c196600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:16.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:16.899+0000 7f8382893700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f837c101ce0 0x7f837c196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:16.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:16.899+0000 7f8382893700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f837c197160 con 0x7f837c101ce0 2024-04-02T04:40:16.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:16.899+0000 7f8382893700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f837c1972a0 con 0x7f837c100ae0 2024-04-02T04:40:16.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:16.899+0000 7f8373fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f837c101ce0 0x7f837c196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:16.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:16.899+0000 7f8373fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f837c101ce0 0x7f837c196b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46782/0 (socket says 172.21.15.16:46782) 2024-04-02T04:40:16.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:16.899+0000 7f8373fff700 1 -- 172.21.15.16:0/637828518 learned_addr learned my addr 172.21.15.16:0/637828518 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:40:16.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:16.899+0000 7f837bfff700 1 --2- 172.21.15.16:0/637828518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f837c100ae0 0x7f837c196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:16.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:16.899+0000 7f8373fff700 1 -- 172.21.15.16:0/637828518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f837c100ae0 msgr2=0x7f837c196600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:16.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:16.899+0000 7f8373fff700 1 --2- 172.21.15.16:0/637828518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f837c100ae0 0x7f837c196600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:16.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:16.899+0000 7f8373fff700 1 -- 172.21.15.16:0/637828518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f836c0095f0 con 0x7f837c101ce0 2024-04-02T04:40:16.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:16.899+0000 7f8373fff700 1 --2- 172.21.15.16:0/637828518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f837c101ce0 0x7f837c196b40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f836400b4b0 tx=0x7f836400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:16.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:16.899+0000 7f8379ffb700 1 -- 172.21.15.16:0/637828518 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8364011800 con 0x7f837c101ce0 2024-04-02T04:40:16.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:16.899+0000 7f8379ffb700 1 -- 172.21.15.16:0/637828518 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8364011e20 con 0x7f837c101ce0 2024-04-02T04:40:16.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:16.899+0000 7f8382893700 1 -- 172.21.15.16:0/637828518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f837c19bd00 con 0x7f837c101ce0 2024-04-02T04:40:16.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:16.899+0000 7f8379ffb700 1 -- 172.21.15.16:0/637828518 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f836400f2e0 con 0x7f837c101ce0 2024-04-02T04:40:16.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:16.899+0000 7f8382893700 1 -- 172.21.15.16:0/637828518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f837c19c2d0 con 0x7f837c101ce0 2024-04-02T04:40:16.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:16.900+0000 7f8382893700 1 -- 172.21.15.16:0/637828518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f837c1906e0 con 0x7f837c101ce0 2024-04-02T04:40:16.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:16.901+0000 7f8379ffb700 1 -- 172.21.15.16:0/637828518 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f8364010380 con 0x7f837c101ce0 2024-04-02T04:40:16.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:16.901+0000 7f8379ffb700 1 --2- 172.21.15.16:0/637828518 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f836806cce0 0x7f836806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:16.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:16.902+0000 7f837bfff700 1 --2- 172.21.15.16:0/637828518 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f836806cce0 0x7f836806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:16.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:16.902+0000 7f8379ffb700 1 -- 172.21.15.16:0/637828518 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f836408be60 con 0x7f837c101ce0 2024-04-02T04:40:16.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:16.902+0000 7f837bfff700 1 --2- 172.21.15.16:0/637828518 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f836806cce0 0x7f836806f1a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f836c00b5c0 tx=0x7f836c005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:16.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:16.907+0000 7f8379ffb700 1 -- 172.21.15.16:0/637828518 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8364059bc0 con 0x7f837c101ce0 2024-04-02T04:40:17.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:17.109+0000 7f8382893700 1 -- 172.21.15.16:0/637828518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f837c04ea80 con 0x7f837c101ce0 2024-04-02T04:40:17.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:17.109+0000 7f8379ffb700 1 -- 172.21.15.16:0/637828518 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v26)=0 v26) v1 ==== 163+0+0 (secure 0 0 0) 0x7f8364059750 con 0x7f837c101ce0 2024-04-02T04:40:17.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:17.111+0000 7f8382893700 1 -- 172.21.15.16:0/637828518 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f836806cce0 msgr2=0x7f836806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:17.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:17.111+0000 7f8382893700 1 --2- 172.21.15.16:0/637828518 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f836806cce0 0x7f836806f1a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f836c00b5c0 tx=0x7f836c005960 comp rx=0 tx=0).stop 2024-04-02T04:40:17.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:17.111+0000 7f8382893700 1 -- 172.21.15.16:0/637828518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f837c101ce0 msgr2=0x7f837c196b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:17.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:17.111+0000 7f8382893700 1 --2- 172.21.15.16:0/637828518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f837c101ce0 0x7f837c196b40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f836400b4b0 tx=0x7f836400b7c0 comp rx=0 tx=0).stop 2024-04-02T04:40:17.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:17.111+0000 7f8382893700 1 -- 172.21.15.16:0/637828518 shutdown_connections 2024-04-02T04:40:17.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:17.111+0000 7f8382893700 1 --2- 172.21.15.16:0/637828518 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f836806cce0 0x7f836806f1a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:17.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:17.111+0000 7f8382893700 1 --2- 172.21.15.16:0/637828518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f837c100ae0 0x7f837c196600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:17.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:17.111+0000 7f8382893700 1 --2- 172.21.15.16:0/637828518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f837c101ce0 0x7f837c196b40 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:17.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:17.111+0000 7f8382893700 1 -- 172.21.15.16:0/637828518 >> 172.21.15.16:0/637828518 conn(0x7f837c0fc0a0 msgr2=0x7f837c105330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:17.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:17.111+0000 7f8382893700 1 -- 172.21.15.16:0/637828518 shutdown_connections 2024-04-02T04:40:17.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:17.111+0000 7f8382893700 1 -- 172.21.15.16:0/637828518 wait complete. 2024-04-02T04:40:17.790 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'ceph config set global log_to_journald false --force' 2024-04-02T04:40:18.016 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:40:18.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:17 smithi016 ceph-mon[28969]: pgmap v201: 65 pgs: 65 active+clean; 585 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-04-02T04:40:18.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:17 smithi027 ceph-mon[37550]: pgmap v201: 65 pgs: 65 active+clean; 585 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-04-02T04:40:19.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.593+0000 7fd0cc2eb700 1 -- 172.21.15.16:0/3441624157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0c4100be0 msgr2=0x7fd0c4101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:19.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.593+0000 7fd0cc2eb700 1 --2- 172.21.15.16:0/3441624157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0c4100be0 0x7fd0c4101040 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4009910 tx=0x7fd0b4009c20 comp rx=0 tx=0).stop 2024-04-02T04:40:19.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.593+0000 7fd0cc2eb700 1 -- 172.21.15.16:0/3441624157 shutdown_connections 2024-04-02T04:40:19.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.593+0000 7fd0cc2eb700 1 --2- 172.21.15.16:0/3441624157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0c4100be0 0x7fd0c4101040 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:19.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.593+0000 7fd0cc2eb700 1 --2- 172.21.15.16:0/3441624157 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0c40ff9e0 0x7fd0c40ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:19.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.593+0000 7fd0cc2eb700 1 -- 172.21.15.16:0/3441624157 >> 172.21.15.16:0/3441624157 conn(0x7fd0c40faf80 msgr2=0x7fd0c40fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:19.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.593+0000 7fd0cc2eb700 1 -- 172.21.15.16:0/3441624157 shutdown_connections 2024-04-02T04:40:19.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.593+0000 7fd0cc2eb700 1 -- 172.21.15.16:0/3441624157 wait complete. 2024-04-02T04:40:19.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.593+0000 7fd0cc2eb700 1 Processor -- start 2024-04-02T04:40:19.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.593+0000 7fd0cc2eb700 1 -- start start 2024-04-02T04:40:19.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.594+0000 7fd0cc2eb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0c40ff9e0 0x7fd0c41965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:19.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.594+0000 7fd0cc2eb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0c4100be0 0x7fd0c4196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:19.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.594+0000 7fd0cc2eb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c4197140 con 0x7fd0c4100be0 2024-04-02T04:40:19.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.594+0000 7fd0cc2eb700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c4197280 con 0x7fd0c40ff9e0 2024-04-02T04:40:19.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.594+0000 7fd0c9886700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0c4100be0 0x7fd0c4196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:19.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.594+0000 7fd0c9886700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0c4100be0 0x7fd0c4196b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46800/0 (socket says 172.21.15.16:46800) 2024-04-02T04:40:19.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.594+0000 7fd0c9886700 1 -- 172.21.15.16:0/2054993159 learned_addr learned my addr 172.21.15.16:0/2054993159 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:40:19.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.594+0000 7fd0ca087700 1 --2- 172.21.15.16:0/2054993159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0c40ff9e0 0x7fd0c41965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:19.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.594+0000 7fd0c9886700 1 -- 172.21.15.16:0/2054993159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0c40ff9e0 msgr2=0x7fd0c41965e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:19.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.594+0000 7fd0c9886700 1 --2- 172.21.15.16:0/2054993159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0c40ff9e0 0x7fd0c41965e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:19.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.594+0000 7fd0c9886700 1 -- 172.21.15.16:0/2054993159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b40095f0 con 0x7fd0c4100be0 2024-04-02T04:40:19.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.594+0000 7fd0c9886700 1 --2- 172.21.15.16:0/2054993159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0c4100be0 0x7fd0c4196b20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4006010 tx=0x7fd0b400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:19.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.594+0000 7fd0bb7fe700 1 -- 172.21.15.16:0/2054993159 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0b401d070 con 0x7fd0c4100be0 2024-04-02T04:40:19.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.594+0000 7fd0cc2eb700 1 -- 172.21.15.16:0/2054993159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0c419bc80 con 0x7fd0c4100be0 2024-04-02T04:40:19.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.594+0000 7fd0bb7fe700 1 -- 172.21.15.16:0/2054993159 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd0b400f460 con 0x7fd0c4100be0 2024-04-02T04:40:19.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.594+0000 7fd0bb7fe700 1 -- 172.21.15.16:0/2054993159 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0b4021620 con 0x7fd0c4100be0 2024-04-02T04:40:19.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.595+0000 7fd0cc2eb700 1 -- 172.21.15.16:0/2054993159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0c419c1f0 con 0x7fd0c4100be0 2024-04-02T04:40:19.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.595+0000 7fd0cc2eb700 1 -- 172.21.15.16:0/2054993159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0c404ea80 con 0x7fd0c4100be0 2024-04-02T04:40:19.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.596+0000 7fd0bb7fe700 1 -- 172.21.15.16:0/2054993159 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fd0b400f5d0 con 0x7fd0c4100be0 2024-04-02T04:40:19.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.596+0000 7fd0bb7fe700 1 --2- 172.21.15.16:0/2054993159 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fd0b006cce0 0x7fd0b006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:19.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.596+0000 7fd0bb7fe700 1 -- 172.21.15.16:0/2054993159 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fd0b408d5c0 con 0x7fd0c4100be0 2024-04-02T04:40:19.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.597+0000 7fd0ca087700 1 --2- 172.21.15.16:0/2054993159 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fd0b006cce0 0x7fd0b006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:19.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.597+0000 7fd0ca087700 1 --2- 172.21.15.16:0/2054993159 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fd0b006cce0 0x7fd0b006f1a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4100a40 tx=0x7fd0c0009190 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:19.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.601+0000 7fd0bb7fe700 1 -- 172.21.15.16:0/2054993159 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd0b405b3c0 con 0x7fd0c4100be0 2024-04-02T04:40:19.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.806+0000 7fd0cc2eb700 1 -- 172.21.15.16:0/2054993159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7fd0c419c630 con 0x7fd0c4100be0 2024-04-02T04:40:19.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.807+0000 7fd0bb7fe700 1 -- 172.21.15.16:0/2054993159 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v26)=0 v26) v1 ==== 135+0+0 (secure 0 0 0) 0x7fd0b405af50 con 0x7fd0c4100be0 2024-04-02T04:40:19.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.809+0000 7fd0cc2eb700 1 -- 172.21.15.16:0/2054993159 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fd0b006cce0 msgr2=0x7fd0b006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:19.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.809+0000 7fd0cc2eb700 1 --2- 172.21.15.16:0/2054993159 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fd0b006cce0 0x7fd0b006f1a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4100a40 tx=0x7fd0c0009190 comp rx=0 tx=0).stop 2024-04-02T04:40:19.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.809+0000 7fd0cc2eb700 1 -- 172.21.15.16:0/2054993159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0c4100be0 msgr2=0x7fd0c4196b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:19.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.809+0000 7fd0cc2eb700 1 --2- 172.21.15.16:0/2054993159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0c4100be0 0x7fd0c4196b20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4006010 tx=0x7fd0b400bd70 comp rx=0 tx=0).stop 2024-04-02T04:40:19.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.809+0000 7fd0cc2eb700 1 -- 172.21.15.16:0/2054993159 shutdown_connections 2024-04-02T04:40:19.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.809+0000 7fd0cc2eb700 1 --2- 172.21.15.16:0/2054993159 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fd0b006cce0 0x7fd0b006f1a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:19.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.809+0000 7fd0cc2eb700 1 --2- 172.21.15.16:0/2054993159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0c40ff9e0 0x7fd0c41965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:19.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.809+0000 7fd0cc2eb700 1 --2- 172.21.15.16:0/2054993159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0c4100be0 0x7fd0c4196b20 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:19.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.809+0000 7fd0cc2eb700 1 -- 172.21.15.16:0/2054993159 >> 172.21.15.16:0/2054993159 conn(0x7fd0c40faf80 msgr2=0x7fd0c4068fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:19.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.809+0000 7fd0cc2eb700 1 -- 172.21.15.16:0/2054993159 shutdown_connections 2024-04-02T04:40:19.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:19.809+0000 7fd0cc2eb700 1 -- 172.21.15.16:0/2054993159 wait complete. 2024-04-02T04:40:19.899 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:19 smithi027 ceph-mon[37550]: pgmap v202: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 1.1 KiB/s wr, 0 op/s 2024-04-02T04:40:20.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:19 smithi016 ceph-mon[28969]: pgmap v202: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 1.1 KiB/s wr, 0 op/s 2024-04-02T04:40:20.524 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-04-02T04:40:20.740 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:40:22.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:21 smithi027 ceph-mon[37550]: pgmap v203: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 938 B/s wr, 0 op/s 2024-04-02T04:40:22.171 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:21 smithi016 ceph-mon[28969]: pgmap v203: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 938 B/s wr, 0 op/s 2024-04-02T04:40:22.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.387+0000 7fc1faf19700 1 -- 172.21.15.16:0/4277986774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1f4100390 msgr2=0x7fc1f41007f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:22.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.387+0000 7fc1faf19700 1 --2- 172.21.15.16:0/4277986774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1f4100390 0x7fc1f41007f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0009910 tx=0x7fc1e0009c20 comp rx=0 tx=0).stop 2024-04-02T04:40:22.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.387+0000 7fc1faf19700 1 -- 172.21.15.16:0/4277986774 shutdown_connections 2024-04-02T04:40:22.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.387+0000 7fc1faf19700 1 --2- 172.21.15.16:0/4277986774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1f4100390 0x7fc1f41007f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:22.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.387+0000 7fc1faf19700 1 --2- 172.21.15.16:0/4277986774 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1f40ff190 0x7fc1f40ff5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:22.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.387+0000 7fc1faf19700 1 -- 172.21.15.16:0/4277986774 >> 172.21.15.16:0/4277986774 conn(0x7fc1f40fa730 msgr2=0x7fc1f40fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:22.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.387+0000 7fc1faf19700 1 -- 172.21.15.16:0/4277986774 shutdown_connections 2024-04-02T04:40:22.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.387+0000 7fc1faf19700 1 -- 172.21.15.16:0/4277986774 wait complete. 2024-04-02T04:40:22.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.388+0000 7fc1faf19700 1 Processor -- start 2024-04-02T04:40:22.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.388+0000 7fc1faf19700 1 -- start start 2024-04-02T04:40:22.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.388+0000 7fc1faf19700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1f40ff190 0x7fc1f41955a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:22.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.388+0000 7fc1faf19700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1f4100390 0x7fc1f4195ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:22.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.388+0000 7fc1faf19700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f4196100 con 0x7fc1f40ff190 2024-04-02T04:40:22.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.388+0000 7fc1faf19700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f4196240 con 0x7fc1f4100390 2024-04-02T04:40:22.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.388+0000 7fc1f8cb5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1f40ff190 0x7fc1f41955a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:22.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.388+0000 7fc1f8cb5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1f40ff190 0x7fc1f41955a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43938/0 (socket says 172.21.15.16:43938) 2024-04-02T04:40:22.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.388+0000 7fc1f8cb5700 1 -- 172.21.15.16:0/879047321 learned_addr learned my addr 172.21.15.16:0/879047321 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:40:22.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.388+0000 7fc1f3fff700 1 --2- 172.21.15.16:0/879047321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1f4100390 0x7fc1f4195ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:22.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.389+0000 7fc1f8cb5700 1 -- 172.21.15.16:0/879047321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1f4100390 msgr2=0x7fc1f4195ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:22.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.389+0000 7fc1f8cb5700 1 --2- 172.21.15.16:0/879047321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1f4100390 0x7fc1f4195ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:22.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.389+0000 7fc1f8cb5700 1 -- 172.21.15.16:0/879047321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e00095f0 con 0x7fc1f40ff190 2024-04-02T04:40:22.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.389+0000 7fc1f8cb5700 1 --2- 172.21.15.16:0/879047321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1f40ff190 0x7fc1f41955a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc1e800b480 tx=0x7fc1e800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:22.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.389+0000 7fc1f1ffb700 1 -- 172.21.15.16:0/879047321 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1e8011800 con 0x7fc1f40ff190 2024-04-02T04:40:22.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.389+0000 7fc1f1ffb700 1 -- 172.21.15.16:0/879047321 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc1e8011e20 con 0x7fc1f40ff190 2024-04-02T04:40:22.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.389+0000 7fc1faf19700 1 -- 172.21.15.16:0/879047321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1f419aca0 con 0x7fc1f40ff190 2024-04-02T04:40:22.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.389+0000 7fc1f1ffb700 1 -- 172.21.15.16:0/879047321 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1e8010ea0 con 0x7fc1f40ff190 2024-04-02T04:40:22.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.389+0000 7fc1faf19700 1 -- 172.21.15.16:0/879047321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f419b220 con 0x7fc1f40ff190 2024-04-02T04:40:22.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.390+0000 7fc1faf19700 1 -- 172.21.15.16:0/879047321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1f418edb0 con 0x7fc1f40ff190 2024-04-02T04:40:22.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.391+0000 7fc1f1ffb700 1 -- 172.21.15.16:0/879047321 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fc1e8010380 con 0x7fc1f40ff190 2024-04-02T04:40:22.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.391+0000 7fc1f1ffb700 1 --2- 172.21.15.16:0/879047321 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc1e406cce0 0x7fc1e406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:22.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.391+0000 7fc1f1ffb700 1 -- 172.21.15.16:0/879047321 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fc1e808bd50 con 0x7fc1f40ff190 2024-04-02T04:40:22.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.391+0000 7fc1f3fff700 1 --2- 172.21.15.16:0/879047321 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc1e406cce0 0x7fc1e406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:22.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.392+0000 7fc1f3fff700 1 --2- 172.21.15.16:0/879047321 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc1e406cce0 0x7fc1e406f1a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc1e00051c0 tx=0x7fc1e001a040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:22.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.396+0000 7fc1f1ffb700 1 -- 172.21.15.16:0/879047321 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc1e8059bd0 con 0x7fc1f40ff190 2024-04-02T04:40:22.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.616+0000 7fc1faf19700 1 -- 172.21.15.16:0/879047321 --> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:6cec793d9073f0594e2364b9b687ecaf22424cca", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1f402ce10 con 0x7fc1e406cce0 2024-04-02T04:40:22.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.621+0000 7fc1f1ffb700 1 -- 172.21.15.16:0/879047321 <== mgr.14217 v2:172.21.15.16:6800/3643600900 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7fc1f402ce10 con 0x7fc1e406cce0 2024-04-02T04:40:22.624 INFO:teuthology.orchestra.run.smithi016.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:6cec793d9073f0594e2364b9b687ecaf22424cca 2024-04-02T04:40:22.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.623+0000 7fc1faf19700 1 -- 172.21.15.16:0/879047321 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc1e406cce0 msgr2=0x7fc1e406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:22.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.623+0000 7fc1faf19700 1 --2- 172.21.15.16:0/879047321 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc1e406cce0 0x7fc1e406f1a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc1e00051c0 tx=0x7fc1e001a040 comp rx=0 tx=0).stop 2024-04-02T04:40:22.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.623+0000 7fc1faf19700 1 -- 172.21.15.16:0/879047321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1f40ff190 msgr2=0x7fc1f41955a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:22.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.623+0000 7fc1faf19700 1 --2- 172.21.15.16:0/879047321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1f40ff190 0x7fc1f41955a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc1e800b480 tx=0x7fc1e800b790 comp rx=0 tx=0).stop 2024-04-02T04:40:22.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.623+0000 7fc1faf19700 1 -- 172.21.15.16:0/879047321 shutdown_connections 2024-04-02T04:40:22.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.623+0000 7fc1faf19700 1 --2- 172.21.15.16:0/879047321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1f40ff190 0x7fc1f41955a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:22.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.623+0000 7fc1faf19700 1 --2- 172.21.15.16:0/879047321 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc1e406cce0 0x7fc1e406f1a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:22.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.623+0000 7fc1faf19700 1 --2- 172.21.15.16:0/879047321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1f4100390 0x7fc1f4195ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:22.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.624+0000 7fc1faf19700 1 -- 172.21.15.16:0/879047321 >> 172.21.15.16:0/879047321 conn(0x7fc1f40fa730 msgr2=0x7fc1f4103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:22.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.624+0000 7fc1faf19700 1 -- 172.21.15.16:0/879047321 shutdown_connections 2024-04-02T04:40:22.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:22.624+0000 7fc1faf19700 1 -- 172.21.15.16:0/879047321 wait complete. 2024-04-02T04:40:23.352 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-02T04:40:23.352 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi016.front.sepia.ceph.com 2024-04-02T04:40:23.353 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-04-02T04:40:23.586 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:40:23.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:23 smithi027 ceph-mon[37550]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:6cec793d9073f0594e2364b9b687ecaf22424cca", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:40:23.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:23 smithi027 ceph-mon[37550]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:6cec793d9073f0594e2364b9b687ecaf22424cca 2024-04-02T04:40:23.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:23 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:40:23.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:23 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:40:23.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:23 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:40:23.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:23 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:40:23.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:23 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:40:23.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:23 smithi027 ceph-mon[37550]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:6cec793d9073f0594e2364b9b687ecaf22424cca 2024-04-02T04:40:23.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:23 smithi027 ceph-mon[37550]: pgmap v204: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-04-02T04:40:23.949 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:23 smithi016 ceph-mon[28969]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:6cec793d9073f0594e2364b9b687ecaf22424cca", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:40:23.949 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:23 smithi016 ceph-mon[28969]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:6cec793d9073f0594e2364b9b687ecaf22424cca 2024-04-02T04:40:23.949 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:23 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:40:23.950 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:23 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:40:23.950 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:23 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:40:23.950 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:23 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:40:23.950 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:23 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:40:23.950 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:23 smithi016 ceph-mon[28969]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:6cec793d9073f0594e2364b9b687ecaf22424cca 2024-04-02T04:40:23.950 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:23 smithi016 ceph-mon[28969]: pgmap v204: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-04-02T04:40:25.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.356+0000 7f94d8ed9700 1 -- 172.21.15.16:0/4061849114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94d4100ae0 msgr2=0x7f94d4100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:25.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.356+0000 7f94d8ed9700 1 --2- 172.21.15.16:0/4061849114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94d4100ae0 0x7f94d4100f00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f94c4009910 tx=0x7f94c4009c20 comp rx=0 tx=0).stop 2024-04-02T04:40:25.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.356+0000 7f94d8ed9700 1 -- 172.21.15.16:0/4061849114 shutdown_connections 2024-04-02T04:40:25.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.356+0000 7f94d8ed9700 1 --2- 172.21.15.16:0/4061849114 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94d4101ce0 0x7f94d4102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:25.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.356+0000 7f94d8ed9700 1 --2- 172.21.15.16:0/4061849114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94d4100ae0 0x7f94d4100f00 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:25.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.356+0000 7f94d8ed9700 1 -- 172.21.15.16:0/4061849114 >> 172.21.15.16:0/4061849114 conn(0x7f94d40fc0a0 msgr2=0x7f94d40fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:25.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.356+0000 7f94d8ed9700 1 -- 172.21.15.16:0/4061849114 shutdown_connections 2024-04-02T04:40:25.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.356+0000 7f94d8ed9700 1 -- 172.21.15.16:0/4061849114 wait complete. 2024-04-02T04:40:25.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.357+0000 7f94d8ed9700 1 Processor -- start 2024-04-02T04:40:25.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.357+0000 7f94d8ed9700 1 -- start start 2024-04-02T04:40:25.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.357+0000 7f94d8ed9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94d4100ae0 0x7f94d4074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:25.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.357+0000 7f94d8ed9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94d4101ce0 0x7f94d40731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:25.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.357+0000 7f94d8ed9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d40736e0 con 0x7f94d4101ce0 2024-04-02T04:40:25.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.357+0000 7f94d8ed9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d4073820 con 0x7f94d4100ae0 2024-04-02T04:40:25.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.357+0000 7f94cbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94d4101ce0 0x7f94d40731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:25.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.357+0000 7f94cbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94d4101ce0 0x7f94d40731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43968/0 (socket says 172.21.15.16:43968) 2024-04-02T04:40:25.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.357+0000 7f94cbfff700 1 -- 172.21.15.16:0/911212504 learned_addr learned my addr 172.21.15.16:0/911212504 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:40:25.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.357+0000 7f94d259c700 1 --2- 172.21.15.16:0/911212504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94d4100ae0 0x7f94d4074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:25.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.357+0000 7f94cbfff700 1 -- 172.21.15.16:0/911212504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94d4100ae0 msgr2=0x7f94d4074b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:25.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.357+0000 7f94cbfff700 1 --2- 172.21.15.16:0/911212504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94d4100ae0 0x7f94d4074b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:25.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.357+0000 7f94cbfff700 1 -- 172.21.15.16:0/911212504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c40095f0 con 0x7f94d4101ce0 2024-04-02T04:40:25.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.358+0000 7f94cbfff700 1 --2- 172.21.15.16:0/911212504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94d4101ce0 0x7f94d40731a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f94bc00b4b0 tx=0x7f94bc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:25.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.358+0000 7f94cb7fe700 1 -- 172.21.15.16:0/911212504 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94bc011800 con 0x7f94d4101ce0 2024-04-02T04:40:25.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.358+0000 7f94d8ed9700 1 -- 172.21.15.16:0/911212504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d4073b00 con 0x7f94d4101ce0 2024-04-02T04:40:25.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.358+0000 7f94cb7fe700 1 -- 172.21.15.16:0/911212504 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f94bc011e20 con 0x7f94d4101ce0 2024-04-02T04:40:25.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.358+0000 7f94cb7fe700 1 -- 172.21.15.16:0/911212504 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94bc00f2e0 con 0x7f94d4101ce0 2024-04-02T04:40:25.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.358+0000 7f94d8ed9700 1 -- 172.21.15.16:0/911212504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d419e620 con 0x7f94d4101ce0 2024-04-02T04:40:25.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.359+0000 7f94d8ed9700 1 -- 172.21.15.16:0/911212504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94d41906e0 con 0x7f94d4101ce0 2024-04-02T04:40:25.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.359+0000 7f94cb7fe700 1 -- 172.21.15.16:0/911212504 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f94bc010380 con 0x7f94d4101ce0 2024-04-02T04:40:25.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.360+0000 7f94cb7fe700 1 --2- 172.21.15.16:0/911212504 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f94c006cce0 0x7f94c006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:25.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.360+0000 7f94cb7fe700 1 -- 172.21.15.16:0/911212504 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f94bc08bd80 con 0x7f94d4101ce0 2024-04-02T04:40:25.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.360+0000 7f94d259c700 1 --2- 172.21.15.16:0/911212504 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f94c006cce0 0x7f94c006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:25.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.360+0000 7f94d259c700 1 --2- 172.21.15.16:0/911212504 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f94c006cce0 0x7f94c006f1a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f94c400b5c0 tx=0x7f94c4005160 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:25.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.365+0000 7f94cb7fe700 1 -- 172.21.15.16:0/911212504 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f94bc059c00 con 0x7f94d4101ce0 2024-04-02T04:40:25.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.604+0000 7f94d8ed9700 1 -- 172.21.15.16:0/911212504 --> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f94d402cd20 con 0x7f94c006cce0 2024-04-02T04:40:25.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.605+0000 7f94cb7fe700 1 -- 172.21.15.16:0/911212504 <== mgr.14217 v2:172.21.15.16:6800/3643600900 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f94d402cd20 con 0x7f94c006cce0 2024-04-02T04:40:25.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.607+0000 7f94d8ed9700 1 -- 172.21.15.16:0/911212504 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f94c006cce0 msgr2=0x7f94c006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:25.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.607+0000 7f94d8ed9700 1 --2- 172.21.15.16:0/911212504 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f94c006cce0 0x7f94c006f1a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f94c400b5c0 tx=0x7f94c4005160 comp rx=0 tx=0).stop 2024-04-02T04:40:25.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.607+0000 7f94d8ed9700 1 -- 172.21.15.16:0/911212504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94d4101ce0 msgr2=0x7f94d40731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:25.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.607+0000 7f94d8ed9700 1 --2- 172.21.15.16:0/911212504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94d4101ce0 0x7f94d40731a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f94bc00b4b0 tx=0x7f94bc00b870 comp rx=0 tx=0).stop 2024-04-02T04:40:25.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.607+0000 7f94d8ed9700 1 -- 172.21.15.16:0/911212504 shutdown_connections 2024-04-02T04:40:25.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.607+0000 7f94d8ed9700 1 --2- 172.21.15.16:0/911212504 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f94c006cce0 0x7f94c006f1a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:25.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.607+0000 7f94d8ed9700 1 --2- 172.21.15.16:0/911212504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94d4100ae0 0x7f94d4074b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:25.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.607+0000 7f94d8ed9700 1 --2- 172.21.15.16:0/911212504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94d4101ce0 0x7f94d40731a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:25.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.607+0000 7f94d8ed9700 1 -- 172.21.15.16:0/911212504 >> 172.21.15.16:0/911212504 conn(0x7f94d40fc0a0 msgr2=0x7f94d4105330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:25.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.607+0000 7f94d8ed9700 1 -- 172.21.15.16:0/911212504 shutdown_connections 2024-04-02T04:40:25.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.607+0000 7f94d8ed9700 1 -- 172.21.15.16:0/911212504 wait complete. 2024-04-02T04:40:25.621 INFO:teuthology.orchestra.run.smithi016.stdout:true 2024-04-02T04:40:25.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.727+0000 7fe879175700 1 -- 172.21.15.16:0/1326224922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe874103ef0 msgr2=0x7fe8741062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:25.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.727+0000 7fe879175700 1 --2- 172.21.15.16:0/1326224922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe874103ef0 0x7fe8741062e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe868009910 tx=0x7fe868009c20 comp rx=0 tx=0).stop 2024-04-02T04:40:25.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.727+0000 7fe879175700 1 -- 172.21.15.16:0/1326224922 shutdown_connections 2024-04-02T04:40:25.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.727+0000 7fe879175700 1 --2- 172.21.15.16:0/1326224922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe874103ef0 0x7fe8741062e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:25.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.727+0000 7fe879175700 1 --2- 172.21.15.16:0/1326224922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8741015c0 0x7fe8741039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:25.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.727+0000 7fe879175700 1 -- 172.21.15.16:0/1326224922 >> 172.21.15.16:0/1326224922 conn(0x7fe8740faf70 msgr2=0x7fe8740fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:25.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.727+0000 7fe879175700 1 -- 172.21.15.16:0/1326224922 shutdown_connections 2024-04-02T04:40:25.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.727+0000 7fe879175700 1 -- 172.21.15.16:0/1326224922 wait complete. 2024-04-02T04:40:25.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.728+0000 7fe879175700 1 Processor -- start 2024-04-02T04:40:25.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.728+0000 7fe879175700 1 -- start start 2024-04-02T04:40:25.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.729+0000 7fe879175700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8741015c0 0x7fe874198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:25.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.729+0000 7fe879175700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe874103ef0 0x7fe874198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:25.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.729+0000 7fe879175700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8741993b0 con 0x7fe8741015c0 2024-04-02T04:40:25.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.729+0000 7fe879175700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8741994f0 con 0x7fe874103ef0 2024-04-02T04:40:25.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.729+0000 7fe872d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8741015c0 0x7fe874198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:25.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.729+0000 7fe872d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8741015c0 0x7fe874198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43990/0 (socket says 172.21.15.16:43990) 2024-04-02T04:40:25.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.729+0000 7fe872d9d700 1 -- 172.21.15.16:0/2628685011 learned_addr learned my addr 172.21.15.16:0/2628685011 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:40:25.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.729+0000 7fe87259c700 1 --2- 172.21.15.16:0/2628685011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe874103ef0 0x7fe874198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:25.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.729+0000 7fe872d9d700 1 -- 172.21.15.16:0/2628685011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe874103ef0 msgr2=0x7fe874198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:25.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.729+0000 7fe872d9d700 1 --2- 172.21.15.16:0/2628685011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe874103ef0 0x7fe874198d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:25.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.729+0000 7fe872d9d700 1 -- 172.21.15.16:0/2628685011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8680095f0 con 0x7fe8741015c0 2024-04-02T04:40:25.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.729+0000 7fe872d9d700 1 --2- 172.21.15.16:0/2628685011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8741015c0 0x7fe874198850 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe86400b4b0 tx=0x7fe86400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:25.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.729+0000 7fe85bfff700 1 -- 172.21.15.16:0/2628685011 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe864011800 con 0x7fe8741015c0 2024-04-02T04:40:25.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.730+0000 7fe879175700 1 -- 172.21.15.16:0/2628685011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe87419df50 con 0x7fe8741015c0 2024-04-02T04:40:25.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.730+0000 7fe85bfff700 1 -- 172.21.15.16:0/2628685011 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe864011e20 con 0x7fe8741015c0 2024-04-02T04:40:25.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.730+0000 7fe85bfff700 1 -- 172.21.15.16:0/2628685011 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe86400f2e0 con 0x7fe8741015c0 2024-04-02T04:40:25.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.730+0000 7fe879175700 1 -- 172.21.15.16:0/2628685011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe87419e410 con 0x7fe8741015c0 2024-04-02T04:40:25.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.731+0000 7fe879175700 1 -- 172.21.15.16:0/2628685011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe87404ea80 con 0x7fe8741015c0 2024-04-02T04:40:25.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.731+0000 7fe85bfff700 1 -- 172.21.15.16:0/2628685011 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fe864010380 con 0x7fe8741015c0 2024-04-02T04:40:25.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.732+0000 7fe85bfff700 1 --2- 172.21.15.16:0/2628685011 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fe85c06cce0 0x7fe85c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:25.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.732+0000 7fe85bfff700 1 -- 172.21.15.16:0/2628685011 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fe86408be50 con 0x7fe8741015c0 2024-04-02T04:40:25.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.732+0000 7fe87259c700 1 --2- 172.21.15.16:0/2628685011 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fe85c06cce0 0x7fe85c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:25.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.732+0000 7fe87259c700 1 --2- 172.21.15.16:0/2628685011 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fe85c06cce0 0x7fe85c06f1a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe86800b5c0 tx=0x7fe868005160 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:25.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.736+0000 7fe85bfff700 1 -- 172.21.15.16:0/2628685011 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe864059c50 con 0x7fe8741015c0 2024-04-02T04:40:25.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.979+0000 7fe879175700 1 -- 172.21.15.16:0/2628685011 --> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe8740fff00 con 0x7fe85c06cce0 2024-04-02T04:40:25.983 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:25 smithi016 ceph-mon[28969]: pgmap v205: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-04-02T04:40:25.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.981+0000 7fe85bfff700 1 -- 172.21.15.16:0/2628685011 <== mgr.14217 v2:172.21.15.16:6800/3643600900 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7fe8740fff00 con 0x7fe85c06cce0 2024-04-02T04:40:25.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.982+0000 7fe879175700 1 -- 172.21.15.16:0/2628685011 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fe85c06cce0 msgr2=0x7fe85c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:25.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.982+0000 7fe879175700 1 --2- 172.21.15.16:0/2628685011 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fe85c06cce0 0x7fe85c06f1a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe86800b5c0 tx=0x7fe868005160 comp rx=0 tx=0).stop 2024-04-02T04:40:25.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.983+0000 7fe879175700 1 -- 172.21.15.16:0/2628685011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8741015c0 msgr2=0x7fe874198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:25.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.983+0000 7fe879175700 1 --2- 172.21.15.16:0/2628685011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8741015c0 0x7fe874198850 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe86400b4b0 tx=0x7fe86400b870 comp rx=0 tx=0).stop 2024-04-02T04:40:25.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.983+0000 7fe879175700 1 -- 172.21.15.16:0/2628685011 shutdown_connections 2024-04-02T04:40:25.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.983+0000 7fe879175700 1 --2- 172.21.15.16:0/2628685011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8741015c0 0x7fe874198850 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:25.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.983+0000 7fe879175700 1 --2- 172.21.15.16:0/2628685011 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fe85c06cce0 0x7fe85c06f1a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:25.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.983+0000 7fe879175700 1 --2- 172.21.15.16:0/2628685011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe874103ef0 0x7fe874198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:25.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.983+0000 7fe879175700 1 -- 172.21.15.16:0/2628685011 >> 172.21.15.16:0/2628685011 conn(0x7fe8740faf70 msgr2=0x7fe8740ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:25.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.983+0000 7fe879175700 1 -- 172.21.15.16:0/2628685011 shutdown_connections 2024-04-02T04:40:25.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:25.983+0000 7fe879175700 1 -- 172.21.15.16:0/2628685011 wait complete. 2024-04-02T04:40:26.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.092+0000 7f586d79d700 1 -- 172.21.15.16:0/3819853367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5868101420 msgr2=0x7f5868103810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:26.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.092+0000 7f586d79d700 1 --2- 172.21.15.16:0/3819853367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5868101420 0x7f5868103810 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5858009910 tx=0x7f5858009c20 comp rx=0 tx=0).stop 2024-04-02T04:40:26.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.092+0000 7f586d79d700 1 -- 172.21.15.16:0/3819853367 shutdown_connections 2024-04-02T04:40:26.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.092+0000 7f586d79d700 1 --2- 172.21.15.16:0/3819853367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5868103d50 0x7f5868106140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:26.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.092+0000 7f586d79d700 1 --2- 172.21.15.16:0/3819853367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5868101420 0x7f5868103810 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:26.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.092+0000 7f586d79d700 1 -- 172.21.15.16:0/3819853367 >> 172.21.15.16:0/3819853367 conn(0x7f58680fadf0 msgr2=0x7f58680fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:26.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.092+0000 7f586d79d700 1 -- 172.21.15.16:0/3819853367 shutdown_connections 2024-04-02T04:40:26.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.092+0000 7f586d79d700 1 -- 172.21.15.16:0/3819853367 wait complete. 2024-04-02T04:40:26.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.093+0000 7f586d79d700 1 Processor -- start 2024-04-02T04:40:26.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.093+0000 7f586d79d700 1 -- start start 2024-04-02T04:40:26.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.093+0000 7f586d79d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5868101420 0x7f5868196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:26.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.093+0000 7f586d79d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5868103d50 0x7f58681969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:26.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.093+0000 7f586d79d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5868196fd0 con 0x7f5868103d50 2024-04-02T04:40:26.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.093+0000 7f586d79d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5868197110 con 0x7f5868101420 2024-04-02T04:40:26.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.094+0000 7f58677fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5868103d50 0x7f58681969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:26.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.094+0000 7f58677fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5868103d50 0x7f58681969b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44008/0 (socket says 172.21.15.16:44008) 2024-04-02T04:40:26.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.094+0000 7f58677fe700 1 -- 172.21.15.16:0/3574818120 learned_addr learned my addr 172.21.15.16:0/3574818120 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:40:26.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.094+0000 7f5867fff700 1 --2- 172.21.15.16:0/3574818120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5868101420 0x7f5868196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:26.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.094+0000 7f58677fe700 1 -- 172.21.15.16:0/3574818120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5868101420 msgr2=0x7f5868196470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:26.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.094+0000 7f58677fe700 1 --2- 172.21.15.16:0/3574818120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5868101420 0x7f5868196470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:26.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.094+0000 7f58677fe700 1 -- 172.21.15.16:0/3574818120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58580095f0 con 0x7f5868103d50 2024-04-02T04:40:26.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.094+0000 7f58677fe700 1 --2- 172.21.15.16:0/3574818120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5868103d50 0x7f58681969b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f585c00b4b0 tx=0x7f585c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:26.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.094+0000 7f58657fa700 1 -- 172.21.15.16:0/3574818120 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f585c011800 con 0x7f5868103d50 2024-04-02T04:40:26.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.094+0000 7f586d79d700 1 -- 172.21.15.16:0/3574818120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f586819bb20 con 0x7f5868103d50 2024-04-02T04:40:26.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.094+0000 7f58657fa700 1 -- 172.21.15.16:0/3574818120 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f585c011e20 con 0x7f5868103d50 2024-04-02T04:40:26.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.094+0000 7f58657fa700 1 -- 172.21.15.16:0/3574818120 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f585c00f2e0 con 0x7f5868103d50 2024-04-02T04:40:26.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.094+0000 7f586d79d700 1 -- 172.21.15.16:0/3574818120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f586819c0f0 con 0x7f5868103d50 2024-04-02T04:40:26.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.095+0000 7f586d79d700 1 -- 172.21.15.16:0/3574818120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f586804f2d0 con 0x7f5868103d50 2024-04-02T04:40:26.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.096+0000 7f58657fa700 1 -- 172.21.15.16:0/3574818120 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f585c010380 con 0x7f5868103d50 2024-04-02T04:40:26.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.097+0000 7f58657fa700 1 --2- 172.21.15.16:0/3574818120 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f585006cce0 0x7f585006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:26.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.097+0000 7f58657fa700 1 -- 172.21.15.16:0/3574818120 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f585c08bdc0 con 0x7f5868103d50 2024-04-02T04:40:26.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.097+0000 7f5867fff700 1 --2- 172.21.15.16:0/3574818120 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f585006cce0 0x7f585006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:26.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.097+0000 7f5867fff700 1 --2- 172.21.15.16:0/3574818120 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f585006cce0 0x7f585006f1a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f585800b5c0 tx=0x7f5858005260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:26.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.101+0000 7f58657fa700 1 -- 172.21.15.16:0/3574818120 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f585c059bc0 con 0x7f5868103d50 2024-04-02T04:40:26.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:25 smithi027 ceph-mon[37550]: pgmap v205: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-04-02T04:40:26.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.330+0000 7f586d79d700 1 -- 172.21.15.16:0/3574818120 --> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5868104810 con 0x7f585006cce0 2024-04-02T04:40:26.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.350+0000 7f58657fa700 1 -- 172.21.15.16:0/3574818120 <== mgr.14217 v2:172.21.15.16:6800/3643600900 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f5868104810 con 0x7f585006cce0 2024-04-02T04:40:26.353 INFO:teuthology.orchestra.run.smithi016.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:40:26.353 INFO:teuthology.orchestra.run.smithi016.stdout:alertmanager.smithi016 smithi016 *:9093,9094 running (4m) 24s ago 6m 24.5M - 0.25.0 c8568f914cd2 b95d8354b016 2024-04-02T04:40:26.353 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi016 smithi016 running (6m) 24s ago 6m 7465k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 9de6c16a487f 2024-04-02T04:40:26.353 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi027 smithi027 running (5m) 30s ago 5m 7398k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 99956592471a 2024-04-02T04:40:26.353 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi016 smithi016 running (6m) 24s ago 6m 6656k - 18.2.2-1059-g4d05f4ca 909dce4fe53d bd73919b0711 2024-04-02T04:40:26.353 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi027 smithi027 running (5m) 30s ago 5m 6660k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 7a54c37f202e 2024-04-02T04:40:26.354 INFO:teuthology.orchestra.run.smithi016.stdout:grafana.smithi016 smithi016 *:3000 running (4m) 24s ago 6m 84.9M - 9.4.7 954c08fa6188 7c43d133569c 2024-04-02T04:40:26.354 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.fyzrnm smithi016 running (43s) 24s ago 43s 13.4M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 70f28b9572a6 2024-04-02T04:40:26.354 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.uhgkgg smithi016 running (51s) 24s ago 50s 11.5M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 0aa4d16562a6 2024-04-02T04:40:26.354 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.sdublg smithi027 running (40s) 30s ago 40s 15.6M - 18.2.2-1059-g4d05f4ca 909dce4fe53d a538451099db 2024-04-02T04:40:26.354 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.zwrfoo smithi027 running (48s) 30s ago 48s 18.5M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 4cb3fa428d71 2024-04-02T04:40:26.354 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi016.eqnebx smithi016 *:9283,8765,8443 running (7m) 24s ago 7m 497M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 7d7ad6d79d01 2024-04-02T04:40:26.354 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi027.kjzcsc smithi027 *:8443,9283,8765 running (5m) 30s ago 5m 451M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 1ced128585bc 2024-04-02T04:40:26.354 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi016 smithi016 running (7m) 24s ago 8m 46.2M 2048M 18.2.2-1059-g4d05f4ca 909dce4fe53d 24d5820dc994 2024-04-02T04:40:26.354 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi027 smithi027 running (5m) 30s ago 5m 35.4M 2048M 18.2.2-1059-g4d05f4ca 909dce4fe53d b77563184a1e 2024-04-02T04:40:26.354 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi016 smithi016 *:9100 running (6m) 24s ago 6m 14.6M - 1.5.0 0da6a335fe13 09f54c65b53c 2024-04-02T04:40:26.354 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi027 smithi027 *:9100 running (5m) 30s ago 5m 14.8M - 1.5.0 0da6a335fe13 e41a657c7443 2024-04-02T04:40:26.354 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 smithi016 running (4m) 24s ago 4m 55.9M 3948M 18.2.2-1059-g4d05f4ca 909dce4fe53d 3f40108debd6 2024-04-02T04:40:26.354 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 smithi016 running (4m) 24s ago 4m 54.4M 3948M 18.2.2-1059-g4d05f4ca 909dce4fe53d 7361abc9e1b6 2024-04-02T04:40:26.354 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 smithi016 running (3m) 24s ago 3m 51.5M 3948M 18.2.2-1059-g4d05f4ca 909dce4fe53d 52e592b4d1f2 2024-04-02T04:40:26.354 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 smithi027 running (3m) 30s ago 3m 48.8M 4970M 18.2.2-1059-g4d05f4ca 909dce4fe53d cf7c7fe022f0 2024-04-02T04:40:26.355 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 smithi027 running (2m) 30s ago 2m 50.1M 4970M 18.2.2-1059-g4d05f4ca 909dce4fe53d 8cbbedf8cac9 2024-04-02T04:40:26.355 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 smithi027 running (2m) 30s ago 2m 45.4M 4970M 18.2.2-1059-g4d05f4ca 909dce4fe53d ac6094415529 2024-04-02T04:40:26.355 INFO:teuthology.orchestra.run.smithi016.stdout:prometheus.smithi016 smithi016 *:9095 running (4m) 24s ago 5m 53.9M - 2.43.0 a07b618ecd1d 232d70bae1ba 2024-04-02T04:40:26.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.352+0000 7f586d79d700 1 -- 172.21.15.16:0/3574818120 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f585006cce0 msgr2=0x7f585006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:26.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.352+0000 7f586d79d700 1 --2- 172.21.15.16:0/3574818120 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f585006cce0 0x7f585006f1a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f585800b5c0 tx=0x7f5858005260 comp rx=0 tx=0).stop 2024-04-02T04:40:26.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.352+0000 7f586d79d700 1 -- 172.21.15.16:0/3574818120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5868103d50 msgr2=0x7f58681969b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:26.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.352+0000 7f586d79d700 1 --2- 172.21.15.16:0/3574818120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5868103d50 0x7f58681969b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f585c00b4b0 tx=0x7f585c00b7c0 comp rx=0 tx=0).stop 2024-04-02T04:40:26.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.352+0000 7f586d79d700 1 -- 172.21.15.16:0/3574818120 shutdown_connections 2024-04-02T04:40:26.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.352+0000 7f586d79d700 1 --2- 172.21.15.16:0/3574818120 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f585006cce0 0x7f585006f1a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:26.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.352+0000 7f586d79d700 1 --2- 172.21.15.16:0/3574818120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5868101420 0x7f5868196470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:26.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.352+0000 7f586d79d700 1 --2- 172.21.15.16:0/3574818120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5868103d50 0x7f58681969b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:26.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.352+0000 7f586d79d700 1 -- 172.21.15.16:0/3574818120 >> 172.21.15.16:0/3574818120 conn(0x7f58680fadf0 msgr2=0x7f58680fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:26.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.352+0000 7f586d79d700 1 -- 172.21.15.16:0/3574818120 shutdown_connections 2024-04-02T04:40:26.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.352+0000 7f586d79d700 1 -- 172.21.15.16:0/3574818120 wait complete. 2024-04-02T04:40:26.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.460+0000 7fa8b74d0700 1 -- 172.21.15.16:0/1786133958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8b0100bd0 msgr2=0x7fa8b0101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:26.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.460+0000 7fa8b74d0700 1 --2- 172.21.15.16:0/1786133958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8b0100bd0 0x7fa8b0101030 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0009910 tx=0x7fa8a0009c20 comp rx=0 tx=0).stop 2024-04-02T04:40:26.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.460+0000 7fa8b74d0700 1 -- 172.21.15.16:0/1786133958 shutdown_connections 2024-04-02T04:40:26.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.460+0000 7fa8b74d0700 1 --2- 172.21.15.16:0/1786133958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8b0100bd0 0x7fa8b0101030 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:26.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.460+0000 7fa8b74d0700 1 --2- 172.21.15.16:0/1786133958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8b00ff9d0 0x7fa8b00ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:26.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.460+0000 7fa8b74d0700 1 -- 172.21.15.16:0/1786133958 >> 172.21.15.16:0/1786133958 conn(0x7fa8b00faf70 msgr2=0x7fa8b00fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:26.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.461+0000 7fa8b74d0700 1 -- 172.21.15.16:0/1786133958 shutdown_connections 2024-04-02T04:40:26.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.461+0000 7fa8b74d0700 1 -- 172.21.15.16:0/1786133958 wait complete. 2024-04-02T04:40:26.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.461+0000 7fa8b74d0700 1 Processor -- start 2024-04-02T04:40:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.461+0000 7fa8b74d0700 1 -- start start 2024-04-02T04:40:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.461+0000 7fa8b74d0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8b00ff9d0 0x7fa8b01966a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.461+0000 7fa8b74d0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8b0100bd0 0x7fa8b0196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.461+0000 7fa8b74d0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b0197200 con 0x7fa8b00ff9d0 2024-04-02T04:40:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.461+0000 7fa8b74d0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b0197340 con 0x7fa8b0100bd0 2024-04-02T04:40:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.462+0000 7fa8b526c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8b00ff9d0 0x7fa8b01966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.462+0000 7fa8b526c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8b00ff9d0 0x7fa8b01966a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44024/0 (socket says 172.21.15.16:44024) 2024-04-02T04:40:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.462+0000 7fa8b526c700 1 -- 172.21.15.16:0/2758331106 learned_addr learned my addr 172.21.15.16:0/2758331106 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:40:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.462+0000 7fa8b4a6b700 1 --2- 172.21.15.16:0/2758331106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8b0100bd0 0x7fa8b0196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.462+0000 7fa8b526c700 1 -- 172.21.15.16:0/2758331106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8b0100bd0 msgr2=0x7fa8b0196be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.462+0000 7fa8b526c700 1 --2- 172.21.15.16:0/2758331106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8b0100bd0 0x7fa8b0196be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.462+0000 7fa8b526c700 1 -- 172.21.15.16:0/2758331106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a00095f0 con 0x7fa8b00ff9d0 2024-04-02T04:40:26.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.462+0000 7fa8b526c700 1 --2- 172.21.15.16:0/2758331106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8b00ff9d0 0x7fa8b01966a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac00b480 tx=0x7fa8ac00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:26.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.462+0000 7fa8a67fc700 1 -- 172.21.15.16:0/2758331106 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8ac011800 con 0x7fa8b00ff9d0 2024-04-02T04:40:26.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.462+0000 7fa8b74d0700 1 -- 172.21.15.16:0/2758331106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8b019bd00 con 0x7fa8b00ff9d0 2024-04-02T04:40:26.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.462+0000 7fa8a67fc700 1 -- 172.21.15.16:0/2758331106 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa8ac011e20 con 0x7fa8b00ff9d0 2024-04-02T04:40:26.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.462+0000 7fa8a67fc700 1 -- 172.21.15.16:0/2758331106 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8ac010ea0 con 0x7fa8b00ff9d0 2024-04-02T04:40:26.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.462+0000 7fa8b74d0700 1 -- 172.21.15.16:0/2758331106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8b019c270 con 0x7fa8b00ff9d0 2024-04-02T04:40:26.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.463+0000 7fa8b74d0700 1 -- 172.21.15.16:0/2758331106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8b004eaf0 con 0x7fa8b00ff9d0 2024-04-02T04:40:26.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.464+0000 7fa8a67fc700 1 -- 172.21.15.16:0/2758331106 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fa8ac010380 con 0x7fa8b00ff9d0 2024-04-02T04:40:26.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.464+0000 7fa8a67fc700 1 --2- 172.21.15.16:0/2758331106 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fa89c06cce0 0x7fa89c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:26.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.464+0000 7fa8a67fc700 1 -- 172.21.15.16:0/2758331106 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fa8ac08b9d0 con 0x7fa8b00ff9d0 2024-04-02T04:40:26.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.464+0000 7fa8b4a6b700 1 --2- 172.21.15.16:0/2758331106 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fa89c06cce0 0x7fa89c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:26.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.465+0000 7fa8b4a6b700 1 --2- 172.21.15.16:0/2758331106 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fa89c06cce0 0x7fa89c06f1a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0009fd0 tx=0x7fa8a0005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:26.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.469+0000 7fa8a67fc700 1 -- 172.21.15.16:0/2758331106 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa8b004eaf0 con 0x7fa8b00ff9d0 2024-04-02T04:40:26.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.773+0000 7fa8b74d0700 1 -- 172.21.15.16:0/2758331106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa8b004eaf0 con 0x7fa8b00ff9d0 2024-04-02T04:40:26.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.774+0000 7fa8a67fc700 1 -- 172.21.15.16:0/2758331106 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fa8b004eaf0 con 0x7fa8b00ff9d0 2024-04-02T04:40:26.777 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:40:26.777 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": { 2024-04-02T04:40:26.777 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 2 2024-04-02T04:40:26.777 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:40:26.777 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": { 2024-04-02T04:40:26.778 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 2 2024-04-02T04:40:26.778 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:40:26.778 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": { 2024-04-02T04:40:26.778 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 6 2024-04-02T04:40:26.778 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:40:26.778 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-04-02T04:40:26.778 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 4 2024-04-02T04:40:26.778 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:40:26.778 INFO:teuthology.orchestra.run.smithi016.stdout: "overall": { 2024-04-02T04:40:26.778 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 14 2024-04-02T04:40:26.778 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-02T04:40:26.778 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:40:26.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.777+0000 7fa8b74d0700 1 -- 172.21.15.16:0/2758331106 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fa89c06cce0 msgr2=0x7fa89c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:26.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.777+0000 7fa8b74d0700 1 --2- 172.21.15.16:0/2758331106 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fa89c06cce0 0x7fa89c06f1a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0009fd0 tx=0x7fa8a0005960 comp rx=0 tx=0).stop 2024-04-02T04:40:26.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.777+0000 7fa8b74d0700 1 -- 172.21.15.16:0/2758331106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8b00ff9d0 msgr2=0x7fa8b01966a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:26.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.777+0000 7fa8b74d0700 1 --2- 172.21.15.16:0/2758331106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8b00ff9d0 0x7fa8b01966a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac00b480 tx=0x7fa8ac00b790 comp rx=0 tx=0).stop 2024-04-02T04:40:26.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.777+0000 7fa8b74d0700 1 -- 172.21.15.16:0/2758331106 shutdown_connections 2024-04-02T04:40:26.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.777+0000 7fa8b74d0700 1 --2- 172.21.15.16:0/2758331106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8b00ff9d0 0x7fa8b01966a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:26.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.777+0000 7fa8b74d0700 1 --2- 172.21.15.16:0/2758331106 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fa89c06cce0 0x7fa89c06f1a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:26.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.777+0000 7fa8b74d0700 1 --2- 172.21.15.16:0/2758331106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8b0100bd0 0x7fa8b0196be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:26.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.777+0000 7fa8b74d0700 1 -- 172.21.15.16:0/2758331106 >> 172.21.15.16:0/2758331106 conn(0x7fa8b00faf70 msgr2=0x7fa8b00684b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:26.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.777+0000 7fa8b74d0700 1 -- 172.21.15.16:0/2758331106 shutdown_connections 2024-04-02T04:40:26.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.777+0000 7fa8b74d0700 1 -- 172.21.15.16:0/2758331106 wait complete. 2024-04-02T04:40:26.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.890+0000 7fee25b9b700 1 -- 172.21.15.16:0/4052084102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee200ff3f0 msgr2=0x7fee200ff810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:26.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.890+0000 7fee25b9b700 1 --2- 172.21.15.16:0/4052084102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee200ff3f0 0x7fee200ff810 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fee10009910 tx=0x7fee10009c20 comp rx=0 tx=0).stop 2024-04-02T04:40:26.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.890+0000 7fee25b9b700 1 -- 172.21.15.16:0/4052084102 shutdown_connections 2024-04-02T04:40:26.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.890+0000 7fee25b9b700 1 --2- 172.21.15.16:0/4052084102 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee200ffd50 0x7fee201001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:26.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.890+0000 7fee25b9b700 1 --2- 172.21.15.16:0/4052084102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee200ff3f0 0x7fee200ff810 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:26.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.890+0000 7fee25b9b700 1 -- 172.21.15.16:0/4052084102 >> 172.21.15.16:0/4052084102 conn(0x7fee200faf70 msgr2=0x7fee200fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:26.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.890+0000 7fee25b9b700 1 -- 172.21.15.16:0/4052084102 shutdown_connections 2024-04-02T04:40:26.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.890+0000 7fee25b9b700 1 -- 172.21.15.16:0/4052084102 wait complete. 2024-04-02T04:40:26.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.890+0000 7fee25b9b700 1 Processor -- start 2024-04-02T04:40:26.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.891+0000 7fee25b9b700 1 -- start start 2024-04-02T04:40:26.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.891+0000 7fee25b9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee200ff3f0 0x7fee201986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:26.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.891+0000 7fee25b9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee200ffd50 0x7fee20198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:26.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.891+0000 7fee25b9b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee20199240 con 0x7fee200ff3f0 2024-04-02T04:40:26.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.891+0000 7fee25b9b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee20199380 con 0x7fee200ffd50 2024-04-02T04:40:26.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.891+0000 7fee1f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee200ff3f0 0x7fee201986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:26.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.891+0000 7fee1f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee200ff3f0 0x7fee201986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44038/0 (socket says 172.21.15.16:44038) 2024-04-02T04:40:26.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.891+0000 7fee1f7fe700 1 -- 172.21.15.16:0/2499337321 learned_addr learned my addr 172.21.15.16:0/2499337321 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:40:26.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.891+0000 7fee1f7fe700 1 -- 172.21.15.16:0/2499337321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee200ffd50 msgr2=0x7fee20198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:26.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.891+0000 7fee1f7fe700 1 --2- 172.21.15.16:0/2499337321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee200ffd50 0x7fee20198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:26.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.891+0000 7fee1f7fe700 1 -- 172.21.15.16:0/2499337321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee100095f0 con 0x7fee200ff3f0 2024-04-02T04:40:26.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.891+0000 7fee1f7fe700 1 --2- 172.21.15.16:0/2499337321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee200ff3f0 0x7fee201986e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fee10006010 tx=0x7fee1000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:26.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.892+0000 7fee1cff9700 1 -- 172.21.15.16:0/2499337321 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee1001d070 con 0x7fee200ff3f0 2024-04-02T04:40:26.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.892+0000 7fee1cff9700 1 -- 172.21.15.16:0/2499337321 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fee1000f460 con 0x7fee200ff3f0 2024-04-02T04:40:26.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.892+0000 7fee25b9b700 1 -- 172.21.15.16:0/2499337321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee2019dd80 con 0x7fee200ff3f0 2024-04-02T04:40:26.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.892+0000 7fee1cff9700 1 -- 172.21.15.16:0/2499337321 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee10021640 con 0x7fee200ff3f0 2024-04-02T04:40:26.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.892+0000 7fee25b9b700 1 -- 172.21.15.16:0/2499337321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee2019e290 con 0x7fee200ff3f0 2024-04-02T04:40:26.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.893+0000 7fee25b9b700 1 -- 172.21.15.16:0/2499337321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee2004eaf0 con 0x7fee200ff3f0 2024-04-02T04:40:26.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.893+0000 7fee1cff9700 1 -- 172.21.15.16:0/2499337321 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fee1000fab0 con 0x7fee200ff3f0 2024-04-02T04:40:26.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.894+0000 7fee1cff9700 1 --2- 172.21.15.16:0/2499337321 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fee0806cce0 0x7fee0806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:26.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.894+0000 7fee1cff9700 1 -- 172.21.15.16:0/2499337321 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fee1008d4f0 con 0x7fee200ff3f0 2024-04-02T04:40:26.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.894+0000 7fee1effd700 1 --2- 172.21.15.16:0/2499337321 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fee0806cce0 0x7fee0806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:26.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.895+0000 7fee1effd700 1 --2- 172.21.15.16:0/2499337321 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fee0806cce0 0x7fee0806f1a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fee14005950 tx=0x7fee140058e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:26.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:26.899+0000 7fee1cff9700 1 -- 172.21.15.16:0/2499337321 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fee2004eaf0 con 0x7fee200ff3f0 2024-04-02T04:40:27.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:26 smithi016 ceph-mon[28969]: from='client.14572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:40:27.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:26 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/2758331106' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:40:27.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:26 smithi027 ceph-mon[37550]: from='client.14572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:40:27.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:26 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/2758331106' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:40:27.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.188+0000 7fee25b9b700 1 -- 172.21.15.16:0/2499337321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fee2004ea80 con 0x7fee200ff3f0 2024-04-02T04:40:27.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.188+0000 7fee1cff9700 1 -- 172.21.15.16:0/2499337321 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1843 (secure 0 0 0) 0x7fee1005b370 con 0x7fee200ff3f0 2024-04-02T04:40:27.192 INFO:teuthology.orchestra.run.smithi016.stdout:e15 2024-04-02T04:40:27.192 INFO:teuthology.orchestra.run.smithi016.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:40:27.192 INFO:teuthology.orchestra.run.smithi016.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:40:27.192 INFO:teuthology.orchestra.run.smithi016.stdout:legacy client fscid: 1 2024-04-02T04:40:27.192 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:40:27.192 INFO:teuthology.orchestra.run.smithi016.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:40:27.192 INFO:teuthology.orchestra.run.smithi016.stdout:fs_name cephfs 2024-04-02T04:40:27.192 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 15 2024-04-02T04:40:27.192 INFO:teuthology.orchestra.run.smithi016.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:40:27.192 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-04-02T04:39:30.740168+0000 2024-04-02T04:40:27.192 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-04-02T04:39:49.875319+0000 2024-04-02T04:40:27.192 INFO:teuthology.orchestra.run.smithi016.stdout:tableserver 0 2024-04-02T04:40:27.193 INFO:teuthology.orchestra.run.smithi016.stdout:root 0 2024-04-02T04:40:27.193 INFO:teuthology.orchestra.run.smithi016.stdout:session_timeout 60 2024-04-02T04:40:27.193 INFO:teuthology.orchestra.run.smithi016.stdout:session_autoclose 300 2024-04-02T04:40:27.193 INFO:teuthology.orchestra.run.smithi016.stdout:max_file_size 1099511627776 2024-04-02T04:40:27.193 INFO:teuthology.orchestra.run.smithi016.stdout:max_xattr_size 65536 2024-04-02T04:40:27.193 INFO:teuthology.orchestra.run.smithi016.stdout:required_client_features {} 2024-04-02T04:40:27.193 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure 0 2024-04-02T04:40:27.193 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure_osd_epoch 46 2024-04-02T04:40:27.193 INFO:teuthology.orchestra.run.smithi016.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:40:27.193 INFO:teuthology.orchestra.run.smithi016.stdout:max_mds 1 2024-04-02T04:40:27.193 INFO:teuthology.orchestra.run.smithi016.stdout:in 0 2024-04-02T04:40:27.193 INFO:teuthology.orchestra.run.smithi016.stdout:up {0=24285} 2024-04-02T04:40:27.194 INFO:teuthology.orchestra.run.smithi016.stdout:failed 2024-04-02T04:40:27.194 INFO:teuthology.orchestra.run.smithi016.stdout:damaged 2024-04-02T04:40:27.194 INFO:teuthology.orchestra.run.smithi016.stdout:stopped 2024-04-02T04:40:27.194 INFO:teuthology.orchestra.run.smithi016.stdout:data_pools [3] 2024-04-02T04:40:27.194 INFO:teuthology.orchestra.run.smithi016.stdout:metadata_pool 2 2024-04-02T04:40:27.194 INFO:teuthology.orchestra.run.smithi016.stdout:inline_data enabled 2024-04-02T04:40:27.194 INFO:teuthology.orchestra.run.smithi016.stdout:balancer 2024-04-02T04:40:27.194 INFO:teuthology.orchestra.run.smithi016.stdout:bal_rank_mask -1 2024-04-02T04:40:27.194 INFO:teuthology.orchestra.run.smithi016.stdout:standby_count_wanted 1 2024-04-02T04:40:27.194 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.zwrfoo{0:24285} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.27:6824/1461455976,v1:172.21.15.27:6825/1461455976] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:40:27.194 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:40:27.194 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:40:27.194 INFO:teuthology.orchestra.run.smithi016.stdout:Standby daemons: 2024-04-02T04:40:27.194 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:40:27.195 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.fyzrnm{-1:14488} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.16:6828/2578719859,v1:172.21.15.16:6829/2578719859] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:40:27.195 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.uhgkgg{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.16:6826/601263579,v1:172.21.15.16:6827/601263579] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:40:27.195 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.sdublg{-1:24295} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.27:6826/4177192740,v1:172.21.15.27:6827/4177192740] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:40:27.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.191+0000 7fee25b9b700 1 -- 172.21.15.16:0/2499337321 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fee0806cce0 msgr2=0x7fee0806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:27.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.191+0000 7fee25b9b700 1 --2- 172.21.15.16:0/2499337321 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fee0806cce0 0x7fee0806f1a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fee14005950 tx=0x7fee140058e0 comp rx=0 tx=0).stop 2024-04-02T04:40:27.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.191+0000 7fee25b9b700 1 -- 172.21.15.16:0/2499337321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee200ff3f0 msgr2=0x7fee201986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:27.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.191+0000 7fee25b9b700 1 --2- 172.21.15.16:0/2499337321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee200ff3f0 0x7fee201986e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fee10006010 tx=0x7fee1000be20 comp rx=0 tx=0).stop 2024-04-02T04:40:27.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.191+0000 7fee25b9b700 1 -- 172.21.15.16:0/2499337321 shutdown_connections 2024-04-02T04:40:27.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.191+0000 7fee25b9b700 1 --2- 172.21.15.16:0/2499337321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee200ff3f0 0x7fee201986e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:27.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.191+0000 7fee25b9b700 1 --2- 172.21.15.16:0/2499337321 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fee0806cce0 0x7fee0806f1a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:27.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.191+0000 7fee25b9b700 1 --2- 172.21.15.16:0/2499337321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee200ffd50 0x7fee20198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:27.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.191+0000 7fee25b9b700 1 -- 172.21.15.16:0/2499337321 >> 172.21.15.16:0/2499337321 conn(0x7fee200faf70 msgr2=0x7fee20107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:27.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.191+0000 7fee25b9b700 1 -- 172.21.15.16:0/2499337321 shutdown_connections 2024-04-02T04:40:27.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.191+0000 7fee25b9b700 1 -- 172.21.15.16:0/2499337321 wait complete. 2024-04-02T04:40:27.196 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 15 2024-04-02T04:40:27.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.309+0000 7f000f1c4700 1 -- 172.21.15.16:0/660556775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00080ff9d0 msgr2=0x7f00080ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:27.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.309+0000 7f000f1c4700 1 --2- 172.21.15.16:0/660556775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00080ff9d0 0x7f00080ffdf0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7efffc009910 tx=0x7efffc009c20 comp rx=0 tx=0).stop 2024-04-02T04:40:27.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.310+0000 7f000f1c4700 1 -- 172.21.15.16:0/660556775 shutdown_connections 2024-04-02T04:40:27.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.310+0000 7f000f1c4700 1 --2- 172.21.15.16:0/660556775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0008100bd0 0x7f0008101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:27.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.310+0000 7f000f1c4700 1 --2- 172.21.15.16:0/660556775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00080ff9d0 0x7f00080ffdf0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:27.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.310+0000 7f000f1c4700 1 -- 172.21.15.16:0/660556775 >> 172.21.15.16:0/660556775 conn(0x7f00080faf70 msgr2=0x7f00080fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:27.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.310+0000 7f000f1c4700 1 -- 172.21.15.16:0/660556775 shutdown_connections 2024-04-02T04:40:27.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.310+0000 7f000f1c4700 1 -- 172.21.15.16:0/660556775 wait complete. 2024-04-02T04:40:27.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.310+0000 7f000f1c4700 1 Processor -- start 2024-04-02T04:40:27.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.310+0000 7f000f1c4700 1 -- start start 2024-04-02T04:40:27.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.311+0000 7f000f1c4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00080ff9d0 0x7f00081954d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:27.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.311+0000 7f000f1c4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0008100bd0 0x7f0008195a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:27.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.311+0000 7f000f1c4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0008196030 con 0x7f0008100bd0 2024-04-02T04:40:27.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.311+0000 7f000f1c4700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0008196170 con 0x7f00080ff9d0 2024-04-02T04:40:27.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.311+0000 7f0007fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0008100bd0 0x7f0008195a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:27.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.311+0000 7f0007fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0008100bd0 0x7f0008195a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44056/0 (socket says 172.21.15.16:44056) 2024-04-02T04:40:27.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.311+0000 7f0007fff700 1 -- 172.21.15.16:0/624437631 learned_addr learned my addr 172.21.15.16:0/624437631 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:40:27.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.311+0000 7f000cf60700 1 --2- 172.21.15.16:0/624437631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00080ff9d0 0x7f00081954d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:27.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.311+0000 7f0007fff700 1 -- 172.21.15.16:0/624437631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00080ff9d0 msgr2=0x7f00081954d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:27.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.311+0000 7f0007fff700 1 --2- 172.21.15.16:0/624437631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00080ff9d0 0x7f00081954d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:27.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.311+0000 7f0007fff700 1 -- 172.21.15.16:0/624437631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efffc0095f0 con 0x7f0008100bd0 2024-04-02T04:40:27.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.311+0000 7f0007fff700 1 --2- 172.21.15.16:0/624437631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0008100bd0 0x7f0008195a10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7efff400b4b0 tx=0x7efff400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:27.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.311+0000 7f0005ffb700 1 -- 172.21.15.16:0/624437631 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efff4011800 con 0x7f0008100bd0 2024-04-02T04:40:27.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.312+0000 7f000f1c4700 1 -- 172.21.15.16:0/624437631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0008068620 con 0x7f0008100bd0 2024-04-02T04:40:27.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.312+0000 7f0005ffb700 1 -- 172.21.15.16:0/624437631 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efff4011e20 con 0x7f0008100bd0 2024-04-02T04:40:27.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.312+0000 7f0005ffb700 1 -- 172.21.15.16:0/624437631 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efff400f2e0 con 0x7f0008100bd0 2024-04-02T04:40:27.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.312+0000 7f000f1c4700 1 -- 172.21.15.16:0/624437631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0008068bf0 con 0x7f0008100bd0 2024-04-02T04:40:27.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.313+0000 7f000f1c4700 1 -- 172.21.15.16:0/624437631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f000818f5d0 con 0x7f0008100bd0 2024-04-02T04:40:27.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.314+0000 7f0005ffb700 1 -- 172.21.15.16:0/624437631 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7efff4010380 con 0x7f0008100bd0 2024-04-02T04:40:27.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.314+0000 7f0005ffb700 1 --2- 172.21.15.16:0/624437631 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7efff806cce0 0x7efff806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:27.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.314+0000 7f0005ffb700 1 -- 172.21.15.16:0/624437631 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7efff408bd60 con 0x7f0008100bd0 2024-04-02T04:40:27.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.314+0000 7f000cf60700 1 --2- 172.21.15.16:0/624437631 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7efff806cce0 0x7efff806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:27.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.315+0000 7f000cf60700 1 --2- 172.21.15.16:0/624437631 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7efff806cce0 0x7efff806f1a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efffc00b5c0 tx=0x7efffc005160 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:27.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.321+0000 7f0005ffb700 1 -- 172.21.15.16:0/624437631 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7efff4059be0 con 0x7f0008100bd0 2024-04-02T04:40:27.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.552+0000 7f000f1c4700 1 -- 172.21.15.16:0/624437631 --> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f000802cd20 con 0x7efff806cce0 2024-04-02T04:40:27.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.553+0000 7f0005ffb700 1 -- 172.21.15.16:0/624437631 <== mgr.14217 v2:172.21.15.16:6800/3643600900 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f000802cd20 con 0x7efff806cce0 2024-04-02T04:40:27.556 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:40:27.556 INFO:teuthology.orchestra.run.smithi016.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:6cec793d9073f0594e2364b9b687ecaf22424cca", 2024-04-02T04:40:27.556 INFO:teuthology.orchestra.run.smithi016.stdout: "in_progress": true, 2024-04-02T04:40:27.556 INFO:teuthology.orchestra.run.smithi016.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-02T04:40:27.556 INFO:teuthology.orchestra.run.smithi016.stdout: "services_complete": [], 2024-04-02T04:40:27.557 INFO:teuthology.orchestra.run.smithi016.stdout: "progress": "", 2024-04-02T04:40:27.557 INFO:teuthology.orchestra.run.smithi016.stdout: "message": "Doing first pull of quay.ceph.io/ceph-ci/ceph:6cec793d9073f0594e2364b9b687ecaf22424cca image", 2024-04-02T04:40:27.557 INFO:teuthology.orchestra.run.smithi016.stdout: "is_paused": false 2024-04-02T04:40:27.557 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:40:27.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.555+0000 7f000f1c4700 1 -- 172.21.15.16:0/624437631 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7efff806cce0 msgr2=0x7efff806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:27.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.555+0000 7f000f1c4700 1 --2- 172.21.15.16:0/624437631 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7efff806cce0 0x7efff806f1a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efffc00b5c0 tx=0x7efffc005160 comp rx=0 tx=0).stop 2024-04-02T04:40:27.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.555+0000 7f000f1c4700 1 -- 172.21.15.16:0/624437631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0008100bd0 msgr2=0x7f0008195a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:27.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.555+0000 7f000f1c4700 1 --2- 172.21.15.16:0/624437631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0008100bd0 0x7f0008195a10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7efff400b4b0 tx=0x7efff400b870 comp rx=0 tx=0).stop 2024-04-02T04:40:27.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.555+0000 7f000f1c4700 1 -- 172.21.15.16:0/624437631 shutdown_connections 2024-04-02T04:40:27.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.555+0000 7f000f1c4700 1 --2- 172.21.15.16:0/624437631 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7efff806cce0 0x7efff806f1a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:27.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.555+0000 7f000f1c4700 1 --2- 172.21.15.16:0/624437631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00080ff9d0 0x7f00081954d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:27.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.555+0000 7f000f1c4700 1 --2- 172.21.15.16:0/624437631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0008100bd0 0x7f0008195a10 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:27.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.555+0000 7f000f1c4700 1 -- 172.21.15.16:0/624437631 >> 172.21.15.16:0/624437631 conn(0x7f00080faf70 msgr2=0x7f00081042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:27.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.556+0000 7f000f1c4700 1 -- 172.21.15.16:0/624437631 shutdown_connections 2024-04-02T04:40:27.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.556+0000 7f000f1c4700 1 -- 172.21.15.16:0/624437631 wait complete. 2024-04-02T04:40:27.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.667+0000 7ff6bbae8700 1 -- 172.21.15.16:0/1054289554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6b4101cd0 msgr2=0x7ff6b4102130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:27.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.667+0000 7ff6bbae8700 1 --2- 172.21.15.16:0/1054289554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6b4101cd0 0x7ff6b4102130 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4009910 tx=0x7ff6a4009c20 comp rx=0 tx=0).stop 2024-04-02T04:40:27.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.667+0000 7ff6bbae8700 1 -- 172.21.15.16:0/1054289554 shutdown_connections 2024-04-02T04:40:27.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.667+0000 7ff6bbae8700 1 --2- 172.21.15.16:0/1054289554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6b4101cd0 0x7ff6b4102130 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:27.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.667+0000 7ff6bbae8700 1 --2- 172.21.15.16:0/1054289554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6b4100ad0 0x7ff6b4100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:27.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.667+0000 7ff6bbae8700 1 -- 172.21.15.16:0/1054289554 >> 172.21.15.16:0/1054289554 conn(0x7ff6b40fc090 msgr2=0x7ff6b40fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:27.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.667+0000 7ff6bbae8700 1 -- 172.21.15.16:0/1054289554 shutdown_connections 2024-04-02T04:40:27.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.667+0000 7ff6bbae8700 1 -- 172.21.15.16:0/1054289554 wait complete. 2024-04-02T04:40:27.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.667+0000 7ff6bbae8700 1 Processor -- start 2024-04-02T04:40:27.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.667+0000 7ff6bbae8700 1 -- start start 2024-04-02T04:40:27.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.668+0000 7ff6bbae8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6b4100ad0 0x7ff6b41987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:27.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.668+0000 7ff6bbae8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6b4101cd0 0x7ff6b4198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:27.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.668+0000 7ff6bbae8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b4199340 con 0x7ff6b4100ad0 2024-04-02T04:40:27.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.668+0000 7ff6bbae8700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b4199480 con 0x7ff6b4101cd0 2024-04-02T04:40:27.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.668+0000 7ff6b9884700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6b4100ad0 0x7ff6b41987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:27.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.668+0000 7ff6b9884700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6b4100ad0 0x7ff6b41987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44080/0 (socket says 172.21.15.16:44080) 2024-04-02T04:40:27.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.668+0000 7ff6b9884700 1 -- 172.21.15.16:0/3443266587 learned_addr learned my addr 172.21.15.16:0/3443266587 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:40:27.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.668+0000 7ff6b9083700 1 --2- 172.21.15.16:0/3443266587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6b4101cd0 0x7ff6b4198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:27.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.668+0000 7ff6b9884700 1 -- 172.21.15.16:0/3443266587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6b4101cd0 msgr2=0x7ff6b4198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:27.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.668+0000 7ff6b9884700 1 --2- 172.21.15.16:0/3443266587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6b4101cd0 0x7ff6b4198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:27.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.668+0000 7ff6b9884700 1 -- 172.21.15.16:0/3443266587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a40095f0 con 0x7ff6b4100ad0 2024-04-02T04:40:27.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.668+0000 7ff6b9884700 1 --2- 172.21.15.16:0/3443266587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6b4100ad0 0x7ff6b41987e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff6b000b480 tx=0x7ff6b000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:27.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.668+0000 7ff6aaffd700 1 -- 172.21.15.16:0/3443266587 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6b0011800 con 0x7ff6b4100ad0 2024-04-02T04:40:27.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.669+0000 7ff6aaffd700 1 -- 172.21.15.16:0/3443266587 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff6b0011e20 con 0x7ff6b4100ad0 2024-04-02T04:40:27.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.669+0000 7ff6aaffd700 1 -- 172.21.15.16:0/3443266587 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6b0010ea0 con 0x7ff6b4100ad0 2024-04-02T04:40:27.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.669+0000 7ff6bbae8700 1 -- 172.21.15.16:0/3443266587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6b419dee0 con 0x7ff6b4100ad0 2024-04-02T04:40:27.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.669+0000 7ff6bbae8700 1 -- 172.21.15.16:0/3443266587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6b419e3d0 con 0x7ff6b4100ad0 2024-04-02T04:40:27.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.670+0000 7ff6bbae8700 1 -- 172.21.15.16:0/3443266587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6b41066a0 con 0x7ff6b4100ad0 2024-04-02T04:40:27.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.670+0000 7ff6aaffd700 1 -- 172.21.15.16:0/3443266587 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7ff6b0010380 con 0x7ff6b4100ad0 2024-04-02T04:40:27.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.671+0000 7ff6aaffd700 1 --2- 172.21.15.16:0/3443266587 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7ff6a006cce0 0x7ff6a006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:27.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.671+0000 7ff6aaffd700 1 -- 172.21.15.16:0/3443266587 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7ff6b008bd30 con 0x7ff6b4100ad0 2024-04-02T04:40:27.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.671+0000 7ff6b9083700 1 --2- 172.21.15.16:0/3443266587 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7ff6a006cce0 0x7ff6a006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:27.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.671+0000 7ff6b9083700 1 --2- 172.21.15.16:0/3443266587 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7ff6a006cce0 0x7ff6a006f1a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4005090 tx=0x7ff6a4005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:27.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.675+0000 7ff6aaffd700 1 -- 172.21.15.16:0/3443266587 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff6b0059bb0 con 0x7ff6b4100ad0 2024-04-02T04:40:27.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.993+0000 7ff6bbae8700 1 -- 172.21.15.16:0/3443266587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff6b404ea80 con 0x7ff6b4100ad0 2024-04-02T04:40:27.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.994+0000 7ff6aaffd700 1 -- 172.21.15.16:0/3443266587 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7ff6b00150e0 con 0x7ff6b4100ad0 2024-04-02T04:40:27.997 INFO:teuthology.orchestra.run.smithi016.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-02T04:40:27.997 INFO:teuthology.orchestra.run.smithi016.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-02T04:40:27.997 INFO:teuthology.orchestra.run.smithi016.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-02T04:40:27.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.996+0000 7ff6bbae8700 1 -- 172.21.15.16:0/3443266587 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7ff6a006cce0 msgr2=0x7ff6a006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:27.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.996+0000 7ff6bbae8700 1 --2- 172.21.15.16:0/3443266587 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7ff6a006cce0 0x7ff6a006f1a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4005090 tx=0x7ff6a4005960 comp rx=0 tx=0).stop 2024-04-02T04:40:27.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.996+0000 7ff6bbae8700 1 -- 172.21.15.16:0/3443266587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6b4100ad0 msgr2=0x7ff6b41987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:27.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.996+0000 7ff6bbae8700 1 --2- 172.21.15.16:0/3443266587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6b4100ad0 0x7ff6b41987e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff6b000b480 tx=0x7ff6b000b790 comp rx=0 tx=0).stop 2024-04-02T04:40:27.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.996+0000 7ff6bbae8700 1 -- 172.21.15.16:0/3443266587 shutdown_connections 2024-04-02T04:40:27.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.996+0000 7ff6bbae8700 1 --2- 172.21.15.16:0/3443266587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6b4100ad0 0x7ff6b41987e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:27.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.996+0000 7ff6bbae8700 1 --2- 172.21.15.16:0/3443266587 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7ff6a006cce0 0x7ff6a006f1a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:27.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.996+0000 7ff6bbae8700 1 --2- 172.21.15.16:0/3443266587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6b4101cd0 0x7ff6b4198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:28.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.996+0000 7ff6bbae8700 1 -- 172.21.15.16:0/3443266587 >> 172.21.15.16:0/3443266587 conn(0x7ff6b40fc090 msgr2=0x7ff6b4073400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:28.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.996+0000 7ff6bbae8700 1 -- 172.21.15.16:0/3443266587 shutdown_connections 2024-04-02T04:40:28.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:27.996+0000 7ff6bbae8700 1 -- 172.21.15.16:0/3443266587 wait complete. 2024-04-02T04:40:28.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:27 smithi027 ceph-mon[37550]: from='client.14576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:40:28.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:27 smithi027 ceph-mon[37550]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:40:28.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:27 smithi027 ceph-mon[37550]: pgmap v206: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-02T04:40:28.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:27 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/2499337321' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:40:28.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:27 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:40:28.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:27 smithi016 ceph-mon[28969]: from='client.14576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:40:28.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:27 smithi016 ceph-mon[28969]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:40:28.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:27 smithi016 ceph-mon[28969]: pgmap v206: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-02T04:40:28.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:27 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/2499337321' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:40:28.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:27 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:40:29.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:28 smithi027 ceph-mon[37550]: from='client.14592 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:40:29.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:28 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/3443266587' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:40:29.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:28 smithi016 ceph-mon[28969]: from='client.14592 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:40:29.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:28 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/3443266587' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:40:30.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:29 smithi027 ceph-mon[37550]: pgmap v207: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-04-02T04:40:30.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:29 smithi016 ceph-mon[28969]: pgmap v207: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-04-02T04:40:32.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:31 smithi027 ceph-mon[37550]: pgmap v208: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-02T04:40:32.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:31 smithi016 ceph-mon[28969]: pgmap v208: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-02T04:40:34.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:33 smithi027 ceph-mon[37550]: pgmap v209: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-02T04:40:34.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:33 smithi016 ceph-mon[28969]: pgmap v209: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-02T04:40:36.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:35 smithi027 ceph-mon[37550]: pgmap v210: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:40:36.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:35 smithi016 ceph-mon[28969]: pgmap v210: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:40:38.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:37 smithi027 ceph-mon[37550]: pgmap v211: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:40:38.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:37 smithi016 ceph-mon[28969]: pgmap v211: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:40:40.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:39 smithi016 ceph-mon[28969]: pgmap v212: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:40:40.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:39 smithi027 ceph-mon[37550]: pgmap v212: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:40:42.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:41 smithi027 ceph-mon[37550]: pgmap v213: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:40:42.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:41 smithi016 ceph-mon[28969]: pgmap v213: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:40:43.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:42 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:40:43.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:42 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:40:44.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:43 smithi027 ceph-mon[37550]: pgmap v214: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:40:44.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:43 smithi016 ceph-mon[28969]: pgmap v214: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:40:46.111 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:45 smithi016 ceph-mon[28969]: pgmap v215: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:40:46.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:45 smithi027 ceph-mon[37550]: pgmap v215: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:40:48.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:47 smithi027 ceph-mon[37550]: pgmap v216: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:40:48.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:47 smithi016 ceph-mon[28969]: pgmap v216: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:40:50.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:49 smithi016 ceph-mon[28969]: pgmap v217: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:40:50.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:49 smithi027 ceph-mon[37550]: pgmap v217: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:40:52.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:51 smithi027 ceph-mon[37550]: pgmap v218: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:40:52.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:51 smithi016 ceph-mon[28969]: pgmap v218: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:40:54.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:53 smithi027 ceph-mon[37550]: pgmap v219: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:40:54.199 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:53 smithi016 ceph-mon[28969]: pgmap v219: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:40:56.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:55 smithi027 ceph-mon[37550]: pgmap v220: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:40:56.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:55 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:40:56.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:55 smithi027 ceph-mon[37550]: Upgrade: Target is version 19.0.0-1979-g6cec793d (unknown) 2024-04-02T04:40:56.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:55 smithi027 ceph-mon[37550]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, digests ['quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae'] 2024-04-02T04:40:56.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:55 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:40:56.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:55 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:40:56.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:55 smithi027 ceph-mon[37550]: Upgrade: Need to upgrade myself (mgr.smithi016.eqnebx) 2024-04-02T04:40:56.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:55 smithi016 ceph-mon[28969]: pgmap v220: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:40:56.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:55 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:40:56.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:55 smithi016 ceph-mon[28969]: Upgrade: Target is version 19.0.0-1979-g6cec793d (unknown) 2024-04-02T04:40:56.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:55 smithi016 ceph-mon[28969]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, digests ['quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae'] 2024-04-02T04:40:56.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:55 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:40:56.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:55 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:40:56.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:55 smithi016 ceph-mon[28969]: Upgrade: Need to upgrade myself (mgr.smithi016.eqnebx) 2024-04-02T04:40:57.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:56 smithi027 ceph-mon[37550]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae on smithi027 2024-04-02T04:40:57.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:56 smithi016 ceph-mon[28969]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae on smithi027 2024-04-02T04:40:58.112 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:57 smithi016 ceph-mon[28969]: pgmap v221: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:40:58.112 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:57 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:40:58.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.108+0000 7fc3da805700 1 -- 172.21.15.16:0/3102980254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3d4100bd0 msgr2=0x7fc3d4101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:58.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.108+0000 7fc3da805700 1 --2- 172.21.15.16:0/3102980254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3d4100bd0 0x7fc3d4101030 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009910 tx=0x7fc3c8009c20 comp rx=0 tx=0).stop 2024-04-02T04:40:58.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.108+0000 7fc3da805700 1 -- 172.21.15.16:0/3102980254 shutdown_connections 2024-04-02T04:40:58.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.108+0000 7fc3da805700 1 --2- 172.21.15.16:0/3102980254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3d4100bd0 0x7fc3d4101030 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:58.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.108+0000 7fc3da805700 1 --2- 172.21.15.16:0/3102980254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3d40ff9d0 0x7fc3d40ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:58.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.108+0000 7fc3da805700 1 -- 172.21.15.16:0/3102980254 >> 172.21.15.16:0/3102980254 conn(0x7fc3d40faf70 msgr2=0x7fc3d40fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:58.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.108+0000 7fc3da805700 1 -- 172.21.15.16:0/3102980254 shutdown_connections 2024-04-02T04:40:58.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.108+0000 7fc3da805700 1 -- 172.21.15.16:0/3102980254 wait complete. 2024-04-02T04:40:58.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.109+0000 7fc3da805700 1 Processor -- start 2024-04-02T04:40:58.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.109+0000 7fc3da805700 1 -- start start 2024-04-02T04:40:58.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.109+0000 7fc3da805700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3d40ff9d0 0x7fc3d4068ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:58.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.109+0000 7fc3da805700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3d4100bd0 0x7fc3d419b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:58.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.109+0000 7fc3da805700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d419b7e0 con 0x7fc3d40ff9d0 2024-04-02T04:40:58.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.109+0000 7fc3da805700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d419b920 con 0x7fc3d4100bd0 2024-04-02T04:40:58.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.109+0000 7fc3d3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3d40ff9d0 0x7fc3d4068ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:58.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.109+0000 7fc3d3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3d40ff9d0 0x7fc3d4068ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58626/0 (socket says 172.21.15.16:58626) 2024-04-02T04:40:58.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.109+0000 7fc3d3fff700 1 -- 172.21.15.16:0/1119337574 learned_addr learned my addr 172.21.15.16:0/1119337574 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:40:58.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.109+0000 7fc3d37fe700 1 --2- 172.21.15.16:0/1119337574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3d4100bd0 0x7fc3d419b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:58.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.110+0000 7fc3d3fff700 1 -- 172.21.15.16:0/1119337574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3d4100bd0 msgr2=0x7fc3d419b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:58.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.110+0000 7fc3d3fff700 1 --2- 172.21.15.16:0/1119337574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3d4100bd0 0x7fc3d419b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:58.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.110+0000 7fc3d3fff700 1 -- 172.21.15.16:0/1119337574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c80095f0 con 0x7fc3d40ff9d0 2024-04-02T04:40:58.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.110+0000 7fc3d3fff700 1 --2- 172.21.15.16:0/1119337574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3d40ff9d0 0x7fc3d4068ff0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc3c400d900 tx=0x7fc3c400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:58.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.110+0000 7fc3d17fa700 1 -- 172.21.15.16:0/1119337574 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3c40041f0 con 0x7fc3d40ff9d0 2024-04-02T04:40:58.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.110+0000 7fc3d17fa700 1 -- 172.21.15.16:0/1119337574 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc3c4004350 con 0x7fc3d40ff9d0 2024-04-02T04:40:58.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.110+0000 7fc3da805700 1 -- 172.21.15.16:0/1119337574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d419bc00 con 0x7fc3d40ff9d0 2024-04-02T04:40:58.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.110+0000 7fc3d17fa700 1 -- 172.21.15.16:0/1119337574 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3c4003d20 con 0x7fc3d40ff9d0 2024-04-02T04:40:58.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.110+0000 7fc3da805700 1 -- 172.21.15.16:0/1119337574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d419c1d0 con 0x7fc3d40ff9d0 2024-04-02T04:40:58.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.111+0000 7fc3da805700 1 -- 172.21.15.16:0/1119337574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3d418f550 con 0x7fc3d40ff9d0 2024-04-02T04:40:58.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.112+0000 7fc3d17fa700 1 -- 172.21.15.16:0/1119337574 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fc3c4010460 con 0x7fc3d40ff9d0 2024-04-02T04:40:58.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.112+0000 7fc3d17fa700 1 --2- 172.21.15.16:0/1119337574 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc3bc06cce0 0x7fc3bc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:58.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.112+0000 7fc3d17fa700 1 -- 172.21.15.16:0/1119337574 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fc3c408c140 con 0x7fc3d40ff9d0 2024-04-02T04:40:58.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.112+0000 7fc3d37fe700 1 --2- 172.21.15.16:0/1119337574 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc3bc06cce0 0x7fc3bc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:58.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.113+0000 7fc3d37fe700 1 --2- 172.21.15.16:0/1119337574 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc3bc06cce0 0x7fc3bc06f1a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009fd0 tx=0x7fc3c8005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:58.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.117+0000 7fc3d17fa700 1 -- 172.21.15.16:0/1119337574 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc3c4059fb0 con 0x7fc3d40ff9d0 2024-04-02T04:40:58.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:57 smithi027 ceph-mon[37550]: pgmap v221: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:40:58.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:57 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:40:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.350+0000 7fc3da805700 1 -- 172.21.15.16:0/1119337574 --> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc3d402ce60 con 0x7fc3bc06cce0 2024-04-02T04:40:58.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.351+0000 7fc3d17fa700 1 -- 172.21.15.16:0/1119337574 <== mgr.14217 v2:172.21.15.16:6800/3643600900 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7fc3d402ce60 con 0x7fc3bc06cce0 2024-04-02T04:40:58.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.353+0000 7fc3da805700 1 -- 172.21.15.16:0/1119337574 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc3bc06cce0 msgr2=0x7fc3bc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:58.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.353+0000 7fc3da805700 1 --2- 172.21.15.16:0/1119337574 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc3bc06cce0 0x7fc3bc06f1a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009fd0 tx=0x7fc3c8005960 comp rx=0 tx=0).stop 2024-04-02T04:40:58.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.353+0000 7fc3da805700 1 -- 172.21.15.16:0/1119337574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3d40ff9d0 msgr2=0x7fc3d4068ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:58.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.353+0000 7fc3da805700 1 --2- 172.21.15.16:0/1119337574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3d40ff9d0 0x7fc3d4068ff0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc3c400d900 tx=0x7fc3c400dc10 comp rx=0 tx=0).stop 2024-04-02T04:40:58.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.353+0000 7fc3da805700 1 -- 172.21.15.16:0/1119337574 shutdown_connections 2024-04-02T04:40:58.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.353+0000 7fc3da805700 1 --2- 172.21.15.16:0/1119337574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3d40ff9d0 0x7fc3d4068ff0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:58.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.353+0000 7fc3da805700 1 --2- 172.21.15.16:0/1119337574 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc3bc06cce0 0x7fc3bc06f1a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:58.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.353+0000 7fc3da805700 1 --2- 172.21.15.16:0/1119337574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3d4100bd0 0x7fc3d419b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:58.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.353+0000 7fc3da805700 1 -- 172.21.15.16:0/1119337574 >> 172.21.15.16:0/1119337574 conn(0x7fc3d40faf70 msgr2=0x7fc3d4104120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:58.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.354+0000 7fc3da805700 1 -- 172.21.15.16:0/1119337574 shutdown_connections 2024-04-02T04:40:58.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.354+0000 7fc3da805700 1 -- 172.21.15.16:0/1119337574 wait complete. 2024-04-02T04:40:58.368 INFO:teuthology.orchestra.run.smithi016.stdout:true 2024-04-02T04:40:58.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.466+0000 7f7b8446a700 1 -- 172.21.15.16:0/1899314994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b7c103e90 msgr2=0x7f7b7c106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:58.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.466+0000 7f7b8446a700 1 --2- 172.21.15.16:0/1899314994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b7c103e90 0x7f7b7c106280 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009910 tx=0x7f7b6c009c20 comp rx=0 tx=0).stop 2024-04-02T04:40:58.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.466+0000 7f7b8446a700 1 -- 172.21.15.16:0/1899314994 shutdown_connections 2024-04-02T04:40:58.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.466+0000 7f7b8446a700 1 --2- 172.21.15.16:0/1899314994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b7c103e90 0x7f7b7c106280 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:58.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.466+0000 7f7b8446a700 1 --2- 172.21.15.16:0/1899314994 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b7c101560 0x7f7b7c103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:58.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.466+0000 7f7b8446a700 1 -- 172.21.15.16:0/1899314994 >> 172.21.15.16:0/1899314994 conn(0x7f7b7c0faf70 msgr2=0x7f7b7c0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:58.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.466+0000 7f7b8446a700 1 -- 172.21.15.16:0/1899314994 shutdown_connections 2024-04-02T04:40:58.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.466+0000 7f7b8446a700 1 -- 172.21.15.16:0/1899314994 wait complete. 2024-04-02T04:40:58.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.467+0000 7f7b8446a700 1 Processor -- start 2024-04-02T04:40:58.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.467+0000 7f7b8446a700 1 -- start start 2024-04-02T04:40:58.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.467+0000 7f7b8446a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b7c101560 0x7f7b7c196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:58.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.467+0000 7f7b8446a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b7c103e90 0x7f7b7c1969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:58.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.467+0000 7f7b8446a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7c196ff0 con 0x7f7b7c103e90 2024-04-02T04:40:58.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.467+0000 7f7b8446a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7c197130 con 0x7f7b7c101560 2024-04-02T04:40:58.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.467+0000 7f7b81a05700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b7c103e90 0x7f7b7c1969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:58.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.467+0000 7f7b81a05700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b7c103e90 0x7f7b7c1969d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58634/0 (socket says 172.21.15.16:58634) 2024-04-02T04:40:58.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.467+0000 7f7b81a05700 1 -- 172.21.15.16:0/2872006405 learned_addr learned my addr 172.21.15.16:0/2872006405 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:40:58.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.467+0000 7f7b82206700 1 --2- 172.21.15.16:0/2872006405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b7c101560 0x7f7b7c196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:58.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.468+0000 7f7b81a05700 1 -- 172.21.15.16:0/2872006405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b7c101560 msgr2=0x7f7b7c196490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:58.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.468+0000 7f7b81a05700 1 --2- 172.21.15.16:0/2872006405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b7c101560 0x7f7b7c196490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:58.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.468+0000 7f7b81a05700 1 -- 172.21.15.16:0/2872006405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b6c0095f0 con 0x7f7b7c103e90 2024-04-02T04:40:58.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.468+0000 7f7b81a05700 1 --2- 172.21.15.16:0/2872006405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b7c103e90 0x7f7b7c1969d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c0092e0 tx=0x7f7b6c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:58.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.468+0000 7f7b737fe700 1 -- 172.21.15.16:0/2872006405 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b6c01d070 con 0x7f7b7c103e90 2024-04-02T04:40:58.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.468+0000 7f7b8446a700 1 -- 172.21.15.16:0/2872006405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b7c19bac0 con 0x7f7b7c103e90 2024-04-02T04:40:58.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.468+0000 7f7b8446a700 1 -- 172.21.15.16:0/2872006405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b7c19c030 con 0x7f7b7c103e90 2024-04-02T04:40:58.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.468+0000 7f7b737fe700 1 -- 172.21.15.16:0/2872006405 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b6c00f460 con 0x7f7b7c103e90 2024-04-02T04:40:58.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.468+0000 7f7b737fe700 1 -- 172.21.15.16:0/2872006405 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b6c021620 con 0x7f7b7c103e90 2024-04-02T04:40:58.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.469+0000 7f7b8446a700 1 -- 172.21.15.16:0/2872006405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b60005320 con 0x7f7b7c103e90 2024-04-02T04:40:58.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.470+0000 7f7b737fe700 1 -- 172.21.15.16:0/2872006405 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f7b6c003a60 con 0x7f7b7c103e90 2024-04-02T04:40:58.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.470+0000 7f7b737fe700 1 --2- 172.21.15.16:0/2872006405 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f7b6806cce0 0x7f7b6806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:58.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.470+0000 7f7b737fe700 1 -- 172.21.15.16:0/2872006405 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f7b6c08d3e0 con 0x7f7b7c103e90 2024-04-02T04:40:58.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.470+0000 7f7b82206700 1 --2- 172.21.15.16:0/2872006405 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f7b6806cce0 0x7f7b6806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:58.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.471+0000 7f7b82206700 1 --2- 172.21.15.16:0/2872006405 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f7b6806cce0 0x7f7b6806f1a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7b78005950 tx=0x7f7b7800b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:58.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.475+0000 7f7b737fe700 1 -- 172.21.15.16:0/2872006405 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7b6c05b1c0 con 0x7f7b7c103e90 2024-04-02T04:40:58.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.720+0000 7f7b8446a700 1 -- 172.21.15.16:0/2872006405 --> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b60000bf0 con 0x7f7b6806cce0 2024-04-02T04:40:58.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.721+0000 7f7b737fe700 1 -- 172.21.15.16:0/2872006405 <== mgr.14217 v2:172.21.15.16:6800/3643600900 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f7b60000bf0 con 0x7f7b6806cce0 2024-04-02T04:40:58.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.723+0000 7f7b8446a700 1 -- 172.21.15.16:0/2872006405 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f7b6806cce0 msgr2=0x7f7b6806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:58.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.723+0000 7f7b8446a700 1 --2- 172.21.15.16:0/2872006405 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f7b6806cce0 0x7f7b6806f1a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7b78005950 tx=0x7f7b7800b410 comp rx=0 tx=0).stop 2024-04-02T04:40:58.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.723+0000 7f7b8446a700 1 -- 172.21.15.16:0/2872006405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b7c103e90 msgr2=0x7f7b7c1969d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:58.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.723+0000 7f7b8446a700 1 --2- 172.21.15.16:0/2872006405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b7c103e90 0x7f7b7c1969d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c0092e0 tx=0x7f7b6c00bd70 comp rx=0 tx=0).stop 2024-04-02T04:40:58.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.723+0000 7f7b8446a700 1 -- 172.21.15.16:0/2872006405 shutdown_connections 2024-04-02T04:40:58.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.723+0000 7f7b8446a700 1 --2- 172.21.15.16:0/2872006405 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f7b6806cce0 0x7f7b6806f1a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:58.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.723+0000 7f7b8446a700 1 --2- 172.21.15.16:0/2872006405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b7c101560 0x7f7b7c196490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:58.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.723+0000 7f7b8446a700 1 --2- 172.21.15.16:0/2872006405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b7c103e90 0x7f7b7c1969d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:58.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.723+0000 7f7b8446a700 1 -- 172.21.15.16:0/2872006405 >> 172.21.15.16:0/2872006405 conn(0x7f7b7c0faf70 msgr2=0x7f7b7c0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:58.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.724+0000 7f7b8446a700 1 -- 172.21.15.16:0/2872006405 shutdown_connections 2024-04-02T04:40:58.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.724+0000 7f7b8446a700 1 -- 172.21.15.16:0/2872006405 wait complete. 2024-04-02T04:40:58.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.839+0000 7f8e5c95d700 1 -- 172.21.15.16:0/4013089674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e580ff160 msgr2=0x7f8e580ff580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:58.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.839+0000 7f8e5c95d700 1 --2- 172.21.15.16:0/4013089674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e580ff160 0x7f8e580ff580 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8e48009910 tx=0x7f8e48009c20 comp rx=0 tx=0).stop 2024-04-02T04:40:58.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.839+0000 7f8e5c95d700 1 -- 172.21.15.16:0/4013089674 shutdown_connections 2024-04-02T04:40:58.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.839+0000 7f8e5c95d700 1 --2- 172.21.15.16:0/4013089674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e58100360 0x7f8e581007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:58.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.839+0000 7f8e5c95d700 1 --2- 172.21.15.16:0/4013089674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e580ff160 0x7f8e580ff580 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:58.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.839+0000 7f8e5c95d700 1 -- 172.21.15.16:0/4013089674 >> 172.21.15.16:0/4013089674 conn(0x7f8e580fa700 msgr2=0x7f8e580fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:58.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.839+0000 7f8e5c95d700 1 -- 172.21.15.16:0/4013089674 shutdown_connections 2024-04-02T04:40:58.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.840+0000 7f8e5c95d700 1 -- 172.21.15.16:0/4013089674 wait complete. 2024-04-02T04:40:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.840+0000 7f8e5c95d700 1 Processor -- start 2024-04-02T04:40:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.840+0000 7f8e5c95d700 1 -- start start 2024-04-02T04:40:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.841+0000 7f8e5c95d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e580ff160 0x7f8e58196530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.841+0000 7f8e5c95d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e58100360 0x7f8e58196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.841+0000 7f8e5c95d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e58197090 con 0x7f8e580ff160 2024-04-02T04:40:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.841+0000 7f8e5c95d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e581971d0 con 0x7f8e58100360 2024-04-02T04:40:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.841+0000 7f8e5659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e580ff160 0x7f8e58196530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.841+0000 7f8e5659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e580ff160 0x7f8e58196530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58646/0 (socket says 172.21.15.16:58646) 2024-04-02T04:40:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.841+0000 7f8e55d9b700 1 --2- 172.21.15.16:0/3980392927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e58100360 0x7f8e58196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.841+0000 7f8e5659c700 1 -- 172.21.15.16:0/3980392927 learned_addr learned my addr 172.21.15.16:0/3980392927 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:40:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.841+0000 7f8e5659c700 1 -- 172.21.15.16:0/3980392927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e58100360 msgr2=0x7f8e58196a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.841+0000 7f8e5659c700 1 --2- 172.21.15.16:0/3980392927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e58100360 0x7f8e58196a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.841+0000 7f8e5659c700 1 -- 172.21.15.16:0/3980392927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e480095f0 con 0x7f8e580ff160 2024-04-02T04:40:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.841+0000 7f8e5659c700 1 --2- 172.21.15.16:0/3980392927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e580ff160 0x7f8e58196530 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8e4800bd70 tx=0x7f8e4800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.841+0000 7f8e477fe700 1 -- 172.21.15.16:0/3980392927 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e4801d070 con 0x7f8e580ff160 2024-04-02T04:40:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.841+0000 7f8e477fe700 1 -- 172.21.15.16:0/3980392927 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e48005dc0 con 0x7f8e580ff160 2024-04-02T04:40:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.841+0000 7f8e5c95d700 1 -- 172.21.15.16:0/3980392927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e5819bbd0 con 0x7f8e580ff160 2024-04-02T04:40:58.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.842+0000 7f8e477fe700 1 -- 172.21.15.16:0/3980392927 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e48017400 con 0x7f8e580ff160 2024-04-02T04:40:58.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.842+0000 7f8e5c95d700 1 -- 172.21.15.16:0/3980392927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e5819c0c0 con 0x7f8e580ff160 2024-04-02T04:40:58.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.843+0000 7f8e5c95d700 1 -- 172.21.15.16:0/3980392927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e5818e450 con 0x7f8e580ff160 2024-04-02T04:40:58.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.843+0000 7f8e477fe700 1 -- 172.21.15.16:0/3980392927 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f8e4800f460 con 0x7f8e580ff160 2024-04-02T04:40:58.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.843+0000 7f8e477fe700 1 --2- 172.21.15.16:0/3980392927 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f8e4006ccf0 0x7f8e4006f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:58.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.844+0000 7f8e477fe700 1 -- 172.21.15.16:0/3980392927 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f8e4808d610 con 0x7f8e580ff160 2024-04-02T04:40:58.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.844+0000 7f8e55d9b700 1 --2- 172.21.15.16:0/3980392927 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f8e4006ccf0 0x7f8e4006f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:58.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.844+0000 7f8e55d9b700 1 --2- 172.21.15.16:0/3980392927 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f8e4006ccf0 0x7f8e4006f1b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c00bd20 tx=0x7f8e4c00b490 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:58.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:58.848+0000 7f8e477fe700 1 -- 172.21.15.16:0/3980392927 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8e4805b410 con 0x7f8e580ff160 2024-04-02T04:40:59.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.083+0000 7f8e5c95d700 1 -- 172.21.15.16:0/3980392927 --> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e5802ce70 con 0x7f8e4006ccf0 2024-04-02T04:40:59.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.094+0000 7f8e477fe700 1 -- 172.21.15.16:0/3980392927 <== mgr.14217 v2:172.21.15.16:6800/3643600900 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f8e5802ce70 con 0x7f8e4006ccf0 2024-04-02T04:40:59.097 INFO:teuthology.orchestra.run.smithi016.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:40:59.097 INFO:teuthology.orchestra.run.smithi016.stdout:alertmanager.smithi016 smithi016 *:9093,9094 running (5m) 56s ago 6m 24.5M - 0.25.0 c8568f914cd2 b95d8354b016 2024-04-02T04:40:59.097 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi016 smithi016 running (7m) 56s ago 7m 7465k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 9de6c16a487f 2024-04-02T04:40:59.097 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi027 smithi027 running (6m) 62s ago 6m 7398k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 99956592471a 2024-04-02T04:40:59.097 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi016 smithi016 running (7m) 56s ago 7m 6656k - 18.2.2-1059-g4d05f4ca 909dce4fe53d bd73919b0711 2024-04-02T04:40:59.097 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi027 smithi027 running (6m) 62s ago 5m 6660k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 7a54c37f202e 2024-04-02T04:40:59.097 INFO:teuthology.orchestra.run.smithi016.stdout:grafana.smithi016 smithi016 *:3000 running (5m) 56s ago 6m 84.9M - 9.4.7 954c08fa6188 7c43d133569c 2024-04-02T04:40:59.098 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.fyzrnm smithi016 running (76s) 56s ago 75s 13.4M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 70f28b9572a6 2024-04-02T04:40:59.098 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.uhgkgg smithi016 running (84s) 56s ago 83s 11.5M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 0aa4d16562a6 2024-04-02T04:40:59.098 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.sdublg smithi027 running (73s) 62s ago 73s 15.6M - 18.2.2-1059-g4d05f4ca 909dce4fe53d a538451099db 2024-04-02T04:40:59.098 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.zwrfoo smithi027 running (81s) 62s ago 80s 18.5M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 4cb3fa428d71 2024-04-02T04:40:59.098 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi016.eqnebx smithi016 *:9283,8765,8443 running (8m) 56s ago 8m 497M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 7d7ad6d79d01 2024-04-02T04:40:59.098 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi027.kjzcsc smithi027 *:8443,9283,8765 running (5m) 62s ago 5m 451M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 1ced128585bc 2024-04-02T04:40:59.098 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi016 smithi016 running (8m) 56s ago 8m 46.2M 2048M 18.2.2-1059-g4d05f4ca 909dce4fe53d 24d5820dc994 2024-04-02T04:40:59.098 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi027 smithi027 running (5m) 62s ago 5m 35.4M 2048M 18.2.2-1059-g4d05f4ca 909dce4fe53d b77563184a1e 2024-04-02T04:40:59.098 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi016 smithi016 *:9100 running (7m) 56s ago 7m 14.6M - 1.5.0 0da6a335fe13 09f54c65b53c 2024-04-02T04:40:59.098 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi027 smithi027 *:9100 running (5m) 62s ago 5m 14.8M - 1.5.0 0da6a335fe13 e41a657c7443 2024-04-02T04:40:59.098 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 smithi016 running (5m) 56s ago 5m 55.9M 3948M 18.2.2-1059-g4d05f4ca 909dce4fe53d 3f40108debd6 2024-04-02T04:40:59.098 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 smithi016 running (4m) 56s ago 4m 54.4M 3948M 18.2.2-1059-g4d05f4ca 909dce4fe53d 7361abc9e1b6 2024-04-02T04:40:59.098 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 smithi016 running (4m) 56s ago 4m 51.5M 3948M 18.2.2-1059-g4d05f4ca 909dce4fe53d 52e592b4d1f2 2024-04-02T04:40:59.098 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 smithi027 running (3m) 62s ago 3m 48.8M 4970M 18.2.2-1059-g4d05f4ca 909dce4fe53d cf7c7fe022f0 2024-04-02T04:40:59.099 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 smithi027 running (3m) 62s ago 3m 50.1M 4970M 18.2.2-1059-g4d05f4ca 909dce4fe53d 8cbbedf8cac9 2024-04-02T04:40:59.099 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 smithi027 running (2m) 62s ago 2m 45.4M 4970M 18.2.2-1059-g4d05f4ca 909dce4fe53d ac6094415529 2024-04-02T04:40:59.099 INFO:teuthology.orchestra.run.smithi016.stdout:prometheus.smithi016 smithi016 *:9095 running (5m) 56s ago 6m 53.9M - 2.43.0 a07b618ecd1d 232d70bae1ba 2024-04-02T04:40:59.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.096+0000 7f8e5c95d700 1 -- 172.21.15.16:0/3980392927 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f8e4006ccf0 msgr2=0x7f8e4006f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:59.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.096+0000 7f8e5c95d700 1 --2- 172.21.15.16:0/3980392927 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f8e4006ccf0 0x7f8e4006f1b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c00bd20 tx=0x7f8e4c00b490 comp rx=0 tx=0).stop 2024-04-02T04:40:59.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.096+0000 7f8e5c95d700 1 -- 172.21.15.16:0/3980392927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e580ff160 msgr2=0x7f8e58196530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:59.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.096+0000 7f8e5c95d700 1 --2- 172.21.15.16:0/3980392927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e580ff160 0x7f8e58196530 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8e4800bd70 tx=0x7f8e4800bda0 comp rx=0 tx=0).stop 2024-04-02T04:40:59.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.096+0000 7f8e5c95d700 1 -- 172.21.15.16:0/3980392927 shutdown_connections 2024-04-02T04:40:59.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.096+0000 7f8e5c95d700 1 --2- 172.21.15.16:0/3980392927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e580ff160 0x7f8e58196530 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:59.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.096+0000 7f8e5c95d700 1 --2- 172.21.15.16:0/3980392927 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f8e4006ccf0 0x7f8e4006f1b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:59.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.096+0000 7f8e5c95d700 1 --2- 172.21.15.16:0/3980392927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e58100360 0x7f8e58196a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:59.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.096+0000 7f8e5c95d700 1 -- 172.21.15.16:0/3980392927 >> 172.21.15.16:0/3980392927 conn(0x7f8e580fa700 msgr2=0x7f8e58103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:59.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.096+0000 7f8e5c95d700 1 -- 172.21.15.16:0/3980392927 shutdown_connections 2024-04-02T04:40:59.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.096+0000 7f8e5c95d700 1 -- 172.21.15.16:0/3980392927 wait complete. 2024-04-02T04:40:59.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.207+0000 7f10e6328700 1 -- 172.21.15.16:0/1194218390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10e0100ae0 msgr2=0x7f10e0100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:59.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.207+0000 7f10e6328700 1 --2- 172.21.15.16:0/1194218390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10e0100ae0 0x7f10e0100f00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f10d0009910 tx=0x7f10d0009c20 comp rx=0 tx=0).stop 2024-04-02T04:40:59.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.207+0000 7f10e6328700 1 -- 172.21.15.16:0/1194218390 shutdown_connections 2024-04-02T04:40:59.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.207+0000 7f10e6328700 1 --2- 172.21.15.16:0/1194218390 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10e0101ce0 0x7f10e0102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:59.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.207+0000 7f10e6328700 1 --2- 172.21.15.16:0/1194218390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10e0100ae0 0x7f10e0100f00 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:59.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.207+0000 7f10e6328700 1 -- 172.21.15.16:0/1194218390 >> 172.21.15.16:0/1194218390 conn(0x7f10e00fc0a0 msgr2=0x7f10e00fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:59.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.207+0000 7f10e6328700 1 -- 172.21.15.16:0/1194218390 shutdown_connections 2024-04-02T04:40:59.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.207+0000 7f10e6328700 1 -- 172.21.15.16:0/1194218390 wait complete. 2024-04-02T04:40:59.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.208+0000 7f10e6328700 1 Processor -- start 2024-04-02T04:40:59.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.208+0000 7f10e6328700 1 -- start start 2024-04-02T04:40:59.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.208+0000 7f10e6328700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10e0100ae0 0x7f10e0074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:59.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.208+0000 7f10e6328700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10e0101ce0 0x7f10e00731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:59.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.208+0000 7f10e6328700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e00736e0 con 0x7f10e0100ae0 2024-04-02T04:40:59.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.208+0000 7f10e6328700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e0073820 con 0x7f10e0101ce0 2024-04-02T04:40:59.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.208+0000 7f10dffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10e0100ae0 0x7f10e0074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:59.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.208+0000 7f10df7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10e0101ce0 0x7f10e00731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:59.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.208+0000 7f10dffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10e0100ae0 0x7f10e0074b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58654/0 (socket says 172.21.15.16:58654) 2024-04-02T04:40:59.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.208+0000 7f10dffff700 1 -- 172.21.15.16:0/1951647714 learned_addr learned my addr 172.21.15.16:0/1951647714 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:40:59.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.209+0000 7f10dffff700 1 -- 172.21.15.16:0/1951647714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10e0101ce0 msgr2=0x7f10e00731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:59.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.209+0000 7f10dffff700 1 --2- 172.21.15.16:0/1951647714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10e0101ce0 0x7f10e00731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:59.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.209+0000 7f10dffff700 1 -- 172.21.15.16:0/1951647714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d00095f0 con 0x7f10e0100ae0 2024-04-02T04:40:59.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.209+0000 7f10df7fe700 1 --2- 172.21.15.16:0/1951647714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10e0101ce0 0x7f10e00731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T04:40:59.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.209+0000 7f10dffff700 1 --2- 172.21.15.16:0/1951647714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10e0100ae0 0x7f10e0074b00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f10d000b620 tx=0x7f10d0005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:59.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.209+0000 7f10dd7fa700 1 -- 172.21.15.16:0/1951647714 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10d001d070 con 0x7f10e0100ae0 2024-04-02T04:40:59.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.209+0000 7f10e6328700 1 -- 172.21.15.16:0/1951647714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10e0073aa0 con 0x7f10e0100ae0 2024-04-02T04:40:59.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.209+0000 7f10dd7fa700 1 -- 172.21.15.16:0/1951647714 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f10d00052a0 con 0x7f10e0100ae0 2024-04-02T04:40:59.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.209+0000 7f10dd7fa700 1 -- 172.21.15.16:0/1951647714 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10d0017400 con 0x7f10e0100ae0 2024-04-02T04:40:59.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.209+0000 7f10e6328700 1 -- 172.21.15.16:0/1951647714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10e0074010 con 0x7f10e0100ae0 2024-04-02T04:40:59.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.211+0000 7f10e6328700 1 -- 172.21.15.16:0/1951647714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10e0190660 con 0x7f10e0100ae0 2024-04-02T04:40:59.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.211+0000 7f10dd7fa700 1 -- 172.21.15.16:0/1951647714 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f10d0003680 con 0x7f10e0100ae0 2024-04-02T04:40:59.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.211+0000 7f10dd7fa700 1 --2- 172.21.15.16:0/1951647714 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f10c806ccf0 0x7f10c806f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:59.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.211+0000 7f10dd7fa700 1 -- 172.21.15.16:0/1951647714 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f10d008d490 con 0x7f10e0100ae0 2024-04-02T04:40:59.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.211+0000 7f10df7fe700 1 --2- 172.21.15.16:0/1951647714 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f10c806ccf0 0x7f10c806f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:59.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.212+0000 7f10df7fe700 1 --2- 172.21.15.16:0/1951647714 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f10c806ccf0 0x7f10c806f1b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f10d4011490 tx=0x7f10d4009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:59.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.216+0000 7f10dd7fa700 1 -- 172.21.15.16:0/1951647714 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f10d005b290 con 0x7f10e0100ae0 2024-04-02T04:40:59.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.535+0000 7f10e6328700 1 -- 172.21.15.16:0/1951647714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f10e0074010 con 0x7f10e0100ae0 2024-04-02T04:40:59.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.535+0000 7f10dd7fa700 1 -- 172.21.15.16:0/1951647714 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f10e0074010 con 0x7f10e0100ae0 2024-04-02T04:40:59.539 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:40:59.539 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": { 2024-04-02T04:40:59.539 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 2 2024-04-02T04:40:59.539 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:40:59.539 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": { 2024-04-02T04:40:59.539 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 2 2024-04-02T04:40:59.539 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:40:59.539 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": { 2024-04-02T04:40:59.540 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 6 2024-04-02T04:40:59.540 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:40:59.540 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-04-02T04:40:59.540 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 4 2024-04-02T04:40:59.540 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:40:59.540 INFO:teuthology.orchestra.run.smithi016.stdout: "overall": { 2024-04-02T04:40:59.540 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 14 2024-04-02T04:40:59.540 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-02T04:40:59.540 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:40:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.538+0000 7f10e6328700 1 -- 172.21.15.16:0/1951647714 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f10c806ccf0 msgr2=0x7f10c806f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.538+0000 7f10e6328700 1 --2- 172.21.15.16:0/1951647714 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f10c806ccf0 0x7f10c806f1b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f10d4011490 tx=0x7f10d4009260 comp rx=0 tx=0).stop 2024-04-02T04:40:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.538+0000 7f10e6328700 1 -- 172.21.15.16:0/1951647714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10e0100ae0 msgr2=0x7f10e0074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.538+0000 7f10e6328700 1 --2- 172.21.15.16:0/1951647714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10e0100ae0 0x7f10e0074b00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f10d000b620 tx=0x7f10d0005f00 comp rx=0 tx=0).stop 2024-04-02T04:40:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.539+0000 7f10e6328700 1 -- 172.21.15.16:0/1951647714 shutdown_connections 2024-04-02T04:40:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.539+0000 7f10e6328700 1 --2- 172.21.15.16:0/1951647714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10e0100ae0 0x7f10e0074b00 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.539+0000 7f10e6328700 1 --2- 172.21.15.16:0/1951647714 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f10c806ccf0 0x7f10c806f1b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.539+0000 7f10e6328700 1 --2- 172.21.15.16:0/1951647714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10e0101ce0 0x7f10e00731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.539+0000 7f10e6328700 1 -- 172.21.15.16:0/1951647714 >> 172.21.15.16:0/1951647714 conn(0x7f10e00fc0a0 msgr2=0x7f10e01052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.539+0000 7f10e6328700 1 -- 172.21.15.16:0/1951647714 shutdown_connections 2024-04-02T04:40:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.539+0000 7f10e6328700 1 -- 172.21.15.16:0/1951647714 wait complete. 2024-04-02T04:40:59.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.657+0000 7f279307f700 1 -- 172.21.15.16:0/2843095933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f278c103e90 msgr2=0x7f278c106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:59.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.657+0000 7f279307f700 1 --2- 172.21.15.16:0/2843095933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f278c103e90 0x7f278c106280 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2778009910 tx=0x7f2778009c20 comp rx=0 tx=0).stop 2024-04-02T04:40:59.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.657+0000 7f279307f700 1 -- 172.21.15.16:0/2843095933 shutdown_connections 2024-04-02T04:40:59.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.657+0000 7f279307f700 1 --2- 172.21.15.16:0/2843095933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f278c103e90 0x7f278c106280 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:59.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.657+0000 7f279307f700 1 --2- 172.21.15.16:0/2843095933 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f278c101560 0x7f278c103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:59.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.657+0000 7f279307f700 1 -- 172.21.15.16:0/2843095933 >> 172.21.15.16:0/2843095933 conn(0x7f278c0faf70 msgr2=0x7f278c0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:59.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.657+0000 7f279307f700 1 -- 172.21.15.16:0/2843095933 shutdown_connections 2024-04-02T04:40:59.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.657+0000 7f279307f700 1 -- 172.21.15.16:0/2843095933 wait complete. 2024-04-02T04:40:59.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.658+0000 7f279307f700 1 Processor -- start 2024-04-02T04:40:59.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.658+0000 7f279307f700 1 -- start start 2024-04-02T04:40:59.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.658+0000 7f279307f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f278c101560 0x7f278c196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:59.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.658+0000 7f279307f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f278c103e90 0x7f278c1969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:59.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.658+0000 7f279307f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278c196fd0 con 0x7f278c103e90 2024-04-02T04:40:59.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.658+0000 7f279307f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278c197110 con 0x7f278c101560 2024-04-02T04:40:59.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.658+0000 7f278bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f278c103e90 0x7f278c1969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:59.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.658+0000 7f278bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f278c103e90 0x7f278c1969b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58670/0 (socket says 172.21.15.16:58670) 2024-04-02T04:40:59.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.658+0000 7f2790e1b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f278c101560 0x7f278c196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:59.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.658+0000 7f278bfff700 1 -- 172.21.15.16:0/2044880475 learned_addr learned my addr 172.21.15.16:0/2044880475 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:40:59.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.659+0000 7f278bfff700 1 -- 172.21.15.16:0/2044880475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f278c101560 msgr2=0x7f278c196470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:59.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.659+0000 7f278bfff700 1 --2- 172.21.15.16:0/2044880475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f278c101560 0x7f278c196470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:59.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.659+0000 7f278bfff700 1 -- 172.21.15.16:0/2044880475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27780095f0 con 0x7f278c103e90 2024-04-02T04:40:59.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.659+0000 7f278bfff700 1 --2- 172.21.15.16:0/2044880475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f278c103e90 0x7f278c1969b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2778006010 tx=0x7f277800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:59.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.659+0000 7f2789ffb700 1 -- 172.21.15.16:0/2044880475 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f277801d070 con 0x7f278c103e90 2024-04-02T04:40:59.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.659+0000 7f2789ffb700 1 -- 172.21.15.16:0/2044880475 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f277800f460 con 0x7f278c103e90 2024-04-02T04:40:59.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.659+0000 7f279307f700 1 -- 172.21.15.16:0/2044880475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f278c19bb10 con 0x7f278c103e90 2024-04-02T04:40:59.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.659+0000 7f2789ffb700 1 -- 172.21.15.16:0/2044880475 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2778021620 con 0x7f278c103e90 2024-04-02T04:40:59.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.659+0000 7f279307f700 1 -- 172.21.15.16:0/2044880475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f278c19c000 con 0x7f278c103e90 2024-04-02T04:40:59.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.660+0000 7f279307f700 1 -- 172.21.15.16:0/2044880475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f278c04ea80 con 0x7f278c103e90 2024-04-02T04:40:59.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.661+0000 7f2789ffb700 1 -- 172.21.15.16:0/2044880475 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f277800f5d0 con 0x7f278c103e90 2024-04-02T04:40:59.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.661+0000 7f2789ffb700 1 --2- 172.21.15.16:0/2044880475 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f277c06cce0 0x7f277c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:40:59.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.661+0000 7f2789ffb700 1 -- 172.21.15.16:0/2044880475 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f277808d570 con 0x7f278c103e90 2024-04-02T04:40:59.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.661+0000 7f2790e1b700 1 --2- 172.21.15.16:0/2044880475 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f277c06cce0 0x7f277c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:40:59.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.662+0000 7f2790e1b700 1 --2- 172.21.15.16:0/2044880475 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f277c06cce0 0x7f277c06f1a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2780007c60 tx=0x7f27800073d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:40:59.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.670+0000 7f2789ffb700 1 -- 172.21.15.16:0/2044880475 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f277805b2d0 con 0x7f278c103e90 2024-04-02T04:40:59.809 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:59 smithi016 ceph-mon[28969]: from='client.14600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:40:59.809 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:59 smithi016 ceph-mon[28969]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:40:59.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:59 smithi016 ceph-mon[28969]: pgmap v222: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:40:59.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:40:59 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/1951647714' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:40:59.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.952+0000 7f279307f700 1 -- 172.21.15.16:0/2044880475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f278c19c400 con 0x7f278c103e90 2024-04-02T04:40:59.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.952+0000 7f2789ffb700 1 -- 172.21.15.16:0/2044880475 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1843 (secure 0 0 0) 0x7f277805ae60 con 0x7f278c103e90 2024-04-02T04:40:59.956 INFO:teuthology.orchestra.run.smithi016.stdout:e15 2024-04-02T04:40:59.956 INFO:teuthology.orchestra.run.smithi016.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:40:59.956 INFO:teuthology.orchestra.run.smithi016.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:40:59.957 INFO:teuthology.orchestra.run.smithi016.stdout:legacy client fscid: 1 2024-04-02T04:40:59.957 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:40:59.957 INFO:teuthology.orchestra.run.smithi016.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:40:59.957 INFO:teuthology.orchestra.run.smithi016.stdout:fs_name cephfs 2024-04-02T04:40:59.957 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 15 2024-04-02T04:40:59.957 INFO:teuthology.orchestra.run.smithi016.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:40:59.957 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-04-02T04:39:30.740168+0000 2024-04-02T04:40:59.958 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-04-02T04:39:49.875319+0000 2024-04-02T04:40:59.958 INFO:teuthology.orchestra.run.smithi016.stdout:tableserver 0 2024-04-02T04:40:59.958 INFO:teuthology.orchestra.run.smithi016.stdout:root 0 2024-04-02T04:40:59.958 INFO:teuthology.orchestra.run.smithi016.stdout:session_timeout 60 2024-04-02T04:40:59.958 INFO:teuthology.orchestra.run.smithi016.stdout:session_autoclose 300 2024-04-02T04:40:59.958 INFO:teuthology.orchestra.run.smithi016.stdout:max_file_size 1099511627776 2024-04-02T04:40:59.958 INFO:teuthology.orchestra.run.smithi016.stdout:max_xattr_size 65536 2024-04-02T04:40:59.958 INFO:teuthology.orchestra.run.smithi016.stdout:required_client_features {} 2024-04-02T04:40:59.959 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure 0 2024-04-02T04:40:59.959 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure_osd_epoch 46 2024-04-02T04:40:59.959 INFO:teuthology.orchestra.run.smithi016.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:40:59.959 INFO:teuthology.orchestra.run.smithi016.stdout:max_mds 1 2024-04-02T04:40:59.959 INFO:teuthology.orchestra.run.smithi016.stdout:in 0 2024-04-02T04:40:59.959 INFO:teuthology.orchestra.run.smithi016.stdout:up {0=24285} 2024-04-02T04:40:59.960 INFO:teuthology.orchestra.run.smithi016.stdout:failed 2024-04-02T04:40:59.960 INFO:teuthology.orchestra.run.smithi016.stdout:damaged 2024-04-02T04:40:59.960 INFO:teuthology.orchestra.run.smithi016.stdout:stopped 2024-04-02T04:40:59.960 INFO:teuthology.orchestra.run.smithi016.stdout:data_pools [3] 2024-04-02T04:40:59.960 INFO:teuthology.orchestra.run.smithi016.stdout:metadata_pool 2 2024-04-02T04:40:59.960 INFO:teuthology.orchestra.run.smithi016.stdout:inline_data enabled 2024-04-02T04:40:59.960 INFO:teuthology.orchestra.run.smithi016.stdout:balancer 2024-04-02T04:40:59.961 INFO:teuthology.orchestra.run.smithi016.stdout:bal_rank_mask -1 2024-04-02T04:40:59.961 INFO:teuthology.orchestra.run.smithi016.stdout:standby_count_wanted 1 2024-04-02T04:40:59.961 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.zwrfoo{0:24285} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.27:6824/1461455976,v1:172.21.15.27:6825/1461455976] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:40:59.961 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:40:59.961 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:40:59.961 INFO:teuthology.orchestra.run.smithi016.stdout:Standby daemons: 2024-04-02T04:40:59.961 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:40:59.961 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.fyzrnm{-1:14488} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.16:6828/2578719859,v1:172.21.15.16:6829/2578719859] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:40:59.962 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.uhgkgg{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.16:6826/601263579,v1:172.21.15.16:6827/601263579] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:40:59.962 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.sdublg{-1:24295} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.27:6826/4177192740,v1:172.21.15.27:6827/4177192740] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:40:59.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.955+0000 7f279307f700 1 -- 172.21.15.16:0/2044880475 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f277c06cce0 msgr2=0x7f277c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:59.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.955+0000 7f279307f700 1 --2- 172.21.15.16:0/2044880475 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f277c06cce0 0x7f277c06f1a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2780007c60 tx=0x7f27800073d0 comp rx=0 tx=0).stop 2024-04-02T04:40:59.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.955+0000 7f279307f700 1 -- 172.21.15.16:0/2044880475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f278c103e90 msgr2=0x7f278c1969b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:40:59.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.955+0000 7f279307f700 1 --2- 172.21.15.16:0/2044880475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f278c103e90 0x7f278c1969b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2778006010 tx=0x7f277800bd70 comp rx=0 tx=0).stop 2024-04-02T04:40:59.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.955+0000 7f279307f700 1 -- 172.21.15.16:0/2044880475 shutdown_connections 2024-04-02T04:40:59.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.955+0000 7f279307f700 1 --2- 172.21.15.16:0/2044880475 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f277c06cce0 0x7f277c06f1a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:59.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.955+0000 7f279307f700 1 --2- 172.21.15.16:0/2044880475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f278c101560 0x7f278c196470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:59.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.955+0000 7f279307f700 1 --2- 172.21.15.16:0/2044880475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f278c103e90 0x7f278c1969b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:40:59.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.955+0000 7f279307f700 1 -- 172.21.15.16:0/2044880475 >> 172.21.15.16:0/2044880475 conn(0x7f278c0faf70 msgr2=0x7f278c0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:40:59.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.955+0000 7f279307f700 1 -- 172.21.15.16:0/2044880475 shutdown_connections 2024-04-02T04:40:59.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:40:59.955+0000 7f279307f700 1 -- 172.21.15.16:0/2044880475 wait complete. 2024-04-02T04:40:59.965 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 15 2024-04-02T04:41:00.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.064+0000 7f659ca5b700 1 -- 172.21.15.16:0/3996019813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65980ff9d0 msgr2=0x7f65980ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:00.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.064+0000 7f659ca5b700 1 --2- 172.21.15.16:0/3996019813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65980ff9d0 0x7f65980ffdf0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6588009910 tx=0x7f6588009c20 comp rx=0 tx=0).stop 2024-04-02T04:41:00.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.065+0000 7f659ca5b700 1 -- 172.21.15.16:0/3996019813 shutdown_connections 2024-04-02T04:41:00.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.065+0000 7f659ca5b700 1 --2- 172.21.15.16:0/3996019813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6598100bd0 0x7f6598101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:00.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.065+0000 7f659ca5b700 1 --2- 172.21.15.16:0/3996019813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65980ff9d0 0x7f65980ffdf0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:00.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.065+0000 7f659ca5b700 1 -- 172.21.15.16:0/3996019813 >> 172.21.15.16:0/3996019813 conn(0x7f65980faf70 msgr2=0x7f65980fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:41:00.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.065+0000 7f659ca5b700 1 -- 172.21.15.16:0/3996019813 shutdown_connections 2024-04-02T04:41:00.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.065+0000 7f659ca5b700 1 -- 172.21.15.16:0/3996019813 wait complete. 2024-04-02T04:41:00.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.066+0000 7f659ca5b700 1 Processor -- start 2024-04-02T04:41:00.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.066+0000 7f659ca5b700 1 -- start start 2024-04-02T04:41:00.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.066+0000 7f659ca5b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65980ff9d0 0x7f6598069840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:41:00.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.066+0000 7f659ca5b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6598100bd0 0x7f659819b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:41:00.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.066+0000 7f659ca5b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659819b7e0 con 0x7f65980ff9d0 2024-04-02T04:41:00.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.066+0000 7f659ca5b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659819b920 con 0x7f6598100bd0 2024-04-02T04:41:00.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.066+0000 7f659659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65980ff9d0 0x7f6598069840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:41:00.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.066+0000 7f659659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65980ff9d0 0x7f6598069840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58682/0 (socket says 172.21.15.16:58682) 2024-04-02T04:41:00.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.066+0000 7f659659c700 1 -- 172.21.15.16:0/3059722887 learned_addr learned my addr 172.21.15.16:0/3059722887 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:41:00.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.066+0000 7f6595d9b700 1 --2- 172.21.15.16:0/3059722887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6598100bd0 0x7f659819b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:41:00.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.066+0000 7f659659c700 1 -- 172.21.15.16:0/3059722887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6598100bd0 msgr2=0x7f659819b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:00.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.066+0000 7f659659c700 1 --2- 172.21.15.16:0/3059722887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6598100bd0 0x7f659819b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:00.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.066+0000 7f659659c700 1 -- 172.21.15.16:0/3059722887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65880095f0 con 0x7f65980ff9d0 2024-04-02T04:41:00.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.067+0000 7f659659c700 1 --2- 172.21.15.16:0/3059722887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65980ff9d0 0x7f6598069840 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f658800b800 tx=0x7f658800b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:41:00.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.067+0000 7f65877fe700 1 -- 172.21.15.16:0/3059722887 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f658801d070 con 0x7f65980ff9d0 2024-04-02T04:41:00.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.067+0000 7f659ca5b700 1 -- 172.21.15.16:0/3059722887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f659819bb50 con 0x7f65980ff9d0 2024-04-02T04:41:00.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.067+0000 7f65877fe700 1 -- 172.21.15.16:0/3059722887 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f65880052a0 con 0x7f65980ff9d0 2024-04-02T04:41:00.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.067+0000 7f65877fe700 1 -- 172.21.15.16:0/3059722887 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6588017400 con 0x7f65980ff9d0 2024-04-02T04:41:00.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.067+0000 7f659ca5b700 1 -- 172.21.15.16:0/3059722887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f659819c040 con 0x7f65980ff9d0 2024-04-02T04:41:00.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.068+0000 7f659ca5b700 1 -- 172.21.15.16:0/3059722887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f659818f570 con 0x7f65980ff9d0 2024-04-02T04:41:00.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.069+0000 7f65877fe700 1 -- 172.21.15.16:0/3059722887 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f6588003680 con 0x7f65980ff9d0 2024-04-02T04:41:00.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.069+0000 7f65877fe700 1 --2- 172.21.15.16:0/3059722887 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f658006cce0 0x7f658006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:41:00.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.069+0000 7f65877fe700 1 -- 172.21.15.16:0/3059722887 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f658808d570 con 0x7f65980ff9d0 2024-04-02T04:41:00.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.069+0000 7f6595d9b700 1 --2- 172.21.15.16:0/3059722887 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f658006cce0 0x7f658006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:41:00.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.070+0000 7f6595d9b700 1 --2- 172.21.15.16:0/3059722887 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f658006cce0 0x7f658006f1a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f658c009c60 tx=0x7f658c009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:41:00.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.074+0000 7f65877fe700 1 -- 172.21.15.16:0/3059722887 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f658805b3f0 con 0x7f65980ff9d0 2024-04-02T04:41:00.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:59 smithi027 ceph-mon[37550]: from='client.14600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:41:00.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:59 smithi027 ceph-mon[37550]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:41:00.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:59 smithi027 ceph-mon[37550]: pgmap v222: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:41:00.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:40:59 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/1951647714' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:41:00.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.348+0000 7f659ca5b700 1 -- 172.21.15.16:0/3059722887 --> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f659802cd70 con 0x7f658006cce0 2024-04-02T04:41:00.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.350+0000 7f65877fe700 1 -- 172.21.15.16:0/3059722887 <== mgr.14217 v2:172.21.15.16:6800/3643600900 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f659802cd70 con 0x7f658006cce0 2024-04-02T04:41:00.353 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:41:00.353 INFO:teuthology.orchestra.run.smithi016.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:41:00.353 INFO:teuthology.orchestra.run.smithi016.stdout: "in_progress": true, 2024-04-02T04:41:00.353 INFO:teuthology.orchestra.run.smithi016.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-02T04:41:00.354 INFO:teuthology.orchestra.run.smithi016.stdout: "services_complete": [], 2024-04-02T04:41:00.354 INFO:teuthology.orchestra.run.smithi016.stdout: "progress": "0/23 daemons upgraded", 2024-04-02T04:41:00.354 INFO:teuthology.orchestra.run.smithi016.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae image on host smithi027", 2024-04-02T04:41:00.354 INFO:teuthology.orchestra.run.smithi016.stdout: "is_paused": false 2024-04-02T04:41:00.354 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:41:00.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.353+0000 7f659ca5b700 1 -- 172.21.15.16:0/3059722887 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f658006cce0 msgr2=0x7f658006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:00.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.353+0000 7f659ca5b700 1 --2- 172.21.15.16:0/3059722887 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f658006cce0 0x7f658006f1a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f658c009c60 tx=0x7f658c009260 comp rx=0 tx=0).stop 2024-04-02T04:41:00.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.353+0000 7f659ca5b700 1 -- 172.21.15.16:0/3059722887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65980ff9d0 msgr2=0x7f6598069840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:00.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.353+0000 7f659ca5b700 1 --2- 172.21.15.16:0/3059722887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65980ff9d0 0x7f6598069840 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f658800b800 tx=0x7f658800b600 comp rx=0 tx=0).stop 2024-04-02T04:41:00.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.353+0000 7f659ca5b700 1 -- 172.21.15.16:0/3059722887 shutdown_connections 2024-04-02T04:41:00.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.353+0000 7f659ca5b700 1 --2- 172.21.15.16:0/3059722887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65980ff9d0 0x7f6598069840 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:00.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.353+0000 7f659ca5b700 1 --2- 172.21.15.16:0/3059722887 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f658006cce0 0x7f658006f1a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:00.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.353+0000 7f659ca5b700 1 --2- 172.21.15.16:0/3059722887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6598100bd0 0x7f659819b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:00.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.353+0000 7f659ca5b700 1 -- 172.21.15.16:0/3059722887 >> 172.21.15.16:0/3059722887 conn(0x7f65980faf70 msgr2=0x7f65981041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:41:00.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.353+0000 7f659ca5b700 1 -- 172.21.15.16:0/3059722887 shutdown_connections 2024-04-02T04:41:00.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.353+0000 7f659ca5b700 1 -- 172.21.15.16:0/3059722887 wait complete. 2024-04-02T04:41:00.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.460+0000 7f14aed36700 1 -- 172.21.15.16:0/3629454898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14a8100bd0 msgr2=0x7f14a8101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:00.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.460+0000 7f14aed36700 1 --2- 172.21.15.16:0/3629454898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14a8100bd0 0x7f14a8101030 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1494009910 tx=0x7f1494009c20 comp rx=0 tx=0).stop 2024-04-02T04:41:00.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.460+0000 7f14aed36700 1 -- 172.21.15.16:0/3629454898 shutdown_connections 2024-04-02T04:41:00.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.460+0000 7f14aed36700 1 --2- 172.21.15.16:0/3629454898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14a8100bd0 0x7f14a8101030 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:00.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.460+0000 7f14aed36700 1 --2- 172.21.15.16:0/3629454898 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14a80ff9d0 0x7f14a80ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:00.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.460+0000 7f14aed36700 1 -- 172.21.15.16:0/3629454898 >> 172.21.15.16:0/3629454898 conn(0x7f14a80faf70 msgr2=0x7f14a80fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:41:00.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.460+0000 7f14aed36700 1 -- 172.21.15.16:0/3629454898 shutdown_connections 2024-04-02T04:41:00.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.460+0000 7f14aed36700 1 -- 172.21.15.16:0/3629454898 wait complete. 2024-04-02T04:41:00.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.461+0000 7f14aed36700 1 Processor -- start 2024-04-02T04:41:00.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.461+0000 7f14aed36700 1 -- start start 2024-04-02T04:41:00.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.461+0000 7f14aed36700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14a80ff9d0 0x7f14a8195410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:41:00.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.461+0000 7f14aed36700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14a8100bd0 0x7f14a8195950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:41:00.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.461+0000 7f14aed36700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a8195f70 con 0x7f14a8100bd0 2024-04-02T04:41:00.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.461+0000 7f14aed36700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a81960b0 con 0x7f14a80ff9d0 2024-04-02T04:41:00.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.461+0000 7f14acad2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14a80ff9d0 0x7f14a8195410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:41:00.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.461+0000 7f14acad2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14a80ff9d0 0x7f14a8195410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.16:48092/0 (socket says 172.21.15.16:48092) 2024-04-02T04:41:00.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.461+0000 7f14acad2700 1 -- 172.21.15.16:0/657976549 learned_addr learned my addr 172.21.15.16:0/657976549 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:41:00.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.461+0000 7f14a7fff700 1 --2- 172.21.15.16:0/657976549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14a8100bd0 0x7f14a8195950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:41:00.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.461+0000 7f14acad2700 1 -- 172.21.15.16:0/657976549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14a8100bd0 msgr2=0x7f14a8195950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:00.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.461+0000 7f14acad2700 1 --2- 172.21.15.16:0/657976549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14a8100bd0 0x7f14a8195950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:00.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.461+0000 7f14acad2700 1 -- 172.21.15.16:0/657976549 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14940095f0 con 0x7f14a80ff9d0 2024-04-02T04:41:00.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.461+0000 7f14a7fff700 1 --2- 172.21.15.16:0/657976549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14a8100bd0 0x7f14a8195950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T04:41:00.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.462+0000 7f14acad2700 1 --2- 172.21.15.16:0/657976549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14a80ff9d0 0x7f14a8195410 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f149c00e8f0 tx=0x7f149c00ec00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:41:00.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.462+0000 7f14a5ffb700 1 -- 172.21.15.16:0/657976549 <== mon.1 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f149c009790 con 0x7f14a80ff9d0 2024-04-02T04:41:00.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.462+0000 7f14aed36700 1 -- 172.21.15.16:0/657976549 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14a80684e0 con 0x7f14a80ff9d0 2024-04-02T04:41:00.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.462+0000 7f14a5ffb700 1 -- 172.21.15.16:0/657976549 <== mon.1 v2:172.21.15.27:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f149c00c810 con 0x7f14a80ff9d0 2024-04-02T04:41:00.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.462+0000 7f14a5ffb700 1 -- 172.21.15.16:0/657976549 <== mon.1 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f149c003d20 con 0x7f14a80ff9d0 2024-04-02T04:41:00.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.462+0000 7f14aed36700 1 -- 172.21.15.16:0/657976549 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a8068ab0 con 0x7f14a80ff9d0 2024-04-02T04:41:00.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.463+0000 7f14aed36700 1 -- 172.21.15.16:0/657976549 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14a804ea80 con 0x7f14a80ff9d0 2024-04-02T04:41:00.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.464+0000 7f14a5ffb700 1 -- 172.21.15.16:0/657976549 <== mon.1 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f149c0041f0 con 0x7f14a80ff9d0 2024-04-02T04:41:00.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.464+0000 7f14a5ffb700 1 --2- 172.21.15.16:0/657976549 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f149806cce0 0x7f149806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:41:00.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.464+0000 7f14a5ffb700 1 -- 172.21.15.16:0/657976549 <== mon.1 v2:172.21.15.27:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f149c014070 con 0x7f14a80ff9d0 2024-04-02T04:41:00.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.464+0000 7f14a7fff700 1 --2- 172.21.15.16:0/657976549 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f149806cce0 0x7f149806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:41:00.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.465+0000 7f14a7fff700 1 --2- 172.21.15.16:0/657976549 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f149806cce0 0x7f149806f1a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f149400b800 tx=0x7f1494009f70 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:41:00.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.469+0000 7f14a5ffb700 1 -- 172.21.15.16:0/657976549 <== mon.1 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f149c05a3a0 con 0x7f14a80ff9d0 2024-04-02T04:41:00.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.767+0000 7f14aed36700 1 -- 172.21.15.16:0/657976549 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f14a8108530 con 0x7f14a80ff9d0 2024-04-02T04:41:00.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.767+0000 7f14a5ffb700 1 -- 172.21.15.16:0/657976549 <== mon.1 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f149c059f30 con 0x7f14a80ff9d0 2024-04-02T04:41:00.771 INFO:teuthology.orchestra.run.smithi016.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-02T04:41:00.771 INFO:teuthology.orchestra.run.smithi016.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-02T04:41:00.771 INFO:teuthology.orchestra.run.smithi016.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-02T04:41:00.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.769+0000 7f14aed36700 1 -- 172.21.15.16:0/657976549 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f149806cce0 msgr2=0x7f149806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:00.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.769+0000 7f14aed36700 1 --2- 172.21.15.16:0/657976549 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f149806cce0 0x7f149806f1a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f149400b800 tx=0x7f1494009f70 comp rx=0 tx=0).stop 2024-04-02T04:41:00.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.769+0000 7f14aed36700 1 -- 172.21.15.16:0/657976549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14a80ff9d0 msgr2=0x7f14a8195410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:00.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.769+0000 7f14aed36700 1 --2- 172.21.15.16:0/657976549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14a80ff9d0 0x7f14a8195410 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f149c00e8f0 tx=0x7f149c00ec00 comp rx=0 tx=0).stop 2024-04-02T04:41:00.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.770+0000 7f14aed36700 1 -- 172.21.15.16:0/657976549 shutdown_connections 2024-04-02T04:41:00.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.770+0000 7f14aed36700 1 --2- 172.21.15.16:0/657976549 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f149806cce0 0x7f149806f1a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:00.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.770+0000 7f14aed36700 1 --2- 172.21.15.16:0/657976549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14a80ff9d0 0x7f14a8195410 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:00.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.770+0000 7f14aed36700 1 --2- 172.21.15.16:0/657976549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14a8100bd0 0x7f14a8195950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:00.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.770+0000 7f14aed36700 1 -- 172.21.15.16:0/657976549 >> 172.21.15.16:0/657976549 conn(0x7f14a80faf70 msgr2=0x7f14a8104120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:41:00.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.770+0000 7f14aed36700 1 -- 172.21.15.16:0/657976549 shutdown_connections 2024-04-02T04:41:00.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:00.770+0000 7f14aed36700 1 -- 172.21.15.16:0/657976549 wait complete. 2024-04-02T04:41:01.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:00 smithi016 ceph-mon[28969]: from='client.14608 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:41:01.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:00 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/2044880475' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:41:01.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:00 smithi027 ceph-mon[37550]: from='client.14608 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:41:01.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:00 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/2044880475' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:41:02.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:01 smithi016 ceph-mon[28969]: from='client.14620 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:41:02.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:01 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/657976549' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:41:02.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:01 smithi016 ceph-mon[28969]: pgmap v223: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:41:02.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:01 smithi027 ceph-mon[37550]: from='client.14620 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:41:02.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:01 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/657976549' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:41:02.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:01 smithi027 ceph-mon[37550]: pgmap v223: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:41:04.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:03 smithi027 ceph-mon[37550]: pgmap v224: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:41:04.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:03 smithi016 ceph-mon[28969]: pgmap v224: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:41:06.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:05 smithi027 ceph-mon[37550]: pgmap v225: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:41:06.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:05 smithi016 ceph-mon[28969]: pgmap v225: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:41:08.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:07 smithi016 ceph-mon[28969]: pgmap v226: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:41:08.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:07 smithi027 ceph-mon[37550]: pgmap v226: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:41:10.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:09 smithi016 ceph-mon[28969]: pgmap v227: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:41:10.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:09 smithi027 ceph-mon[37550]: pgmap v227: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:41:12.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:11 smithi027 ceph-mon[37550]: pgmap v228: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:41:12.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:11 smithi016 ceph-mon[28969]: pgmap v228: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:41:13.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:12 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:41:13.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:12 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:41:14.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:13 smithi027 ceph-mon[37550]: pgmap v229: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:41:14.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:13 smithi016 ceph-mon[28969]: pgmap v229: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:41:16.111 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:15 smithi016 ceph-mon[28969]: pgmap v230: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:41:16.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:15 smithi027 ceph-mon[37550]: pgmap v230: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:41:18.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:17 smithi016 ceph-mon[28969]: pgmap v231: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:41:18.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:17 smithi027 ceph-mon[37550]: pgmap v231: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:41:20.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:19 smithi016 ceph-mon[28969]: pgmap v232: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:41:20.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:19 smithi027 ceph-mon[37550]: pgmap v232: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:41:22.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:21 smithi027 ceph-mon[37550]: pgmap v233: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:41:22.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:21 smithi016 ceph-mon[28969]: pgmap v233: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:41:24.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:23 smithi027 ceph-mon[37550]: pgmap v234: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:41:24.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:23 smithi016 ceph-mon[28969]: pgmap v234: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:41:26.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:25 smithi027 ceph-mon[37550]: pgmap v235: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:41:26.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:25 smithi016 ceph-mon[28969]: pgmap v235: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:41:27.408 INFO:tasks.workunit.client.0.smithi016.stderr:Updating files: 99% (12429/12435) Updating files: 100% (12435/12435) Updating files: 100% (12435/12435), done. 2024-04-02T04:41:27.974 INFO:tasks.workunit.client.0.smithi016.stderr:Note: switching to '6cec793d9073f0594e2364b9b687ecaf22424cca'. 2024-04-02T04:41:27.974 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-04-02T04:41:27.974 INFO:tasks.workunit.client.0.smithi016.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-02T04:41:27.975 INFO:tasks.workunit.client.0.smithi016.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-02T04:41:27.975 INFO:tasks.workunit.client.0.smithi016.stderr:state without impacting any branches by switching back to a branch. 2024-04-02T04:41:27.975 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-04-02T04:41:27.975 INFO:tasks.workunit.client.0.smithi016.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-02T04:41:27.975 INFO:tasks.workunit.client.0.smithi016.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-02T04:41:27.975 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-04-02T04:41:27.975 INFO:tasks.workunit.client.0.smithi016.stderr: git switch -c 2024-04-02T04:41:27.975 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-04-02T04:41:27.975 INFO:tasks.workunit.client.0.smithi016.stderr:Or undo this operation with: 2024-04-02T04:41:27.975 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-04-02T04:41:27.975 INFO:tasks.workunit.client.0.smithi016.stderr: git switch - 2024-04-02T04:41:27.975 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-04-02T04:41:27.975 INFO:tasks.workunit.client.0.smithi016.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-02T04:41:27.975 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-04-02T04:41:27.975 INFO:tasks.workunit.client.0.smithi016.stderr:HEAD is now at 6cec793d907 Merge pull request #56601 from zdover23/wip-doc-2024-04-01-backport-56600-to-squid 2024-04-02T04:41:27.979 DEBUG:teuthology.orchestra.run.smithi016:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-02T04:41:28.015 INFO:tasks.workunit.client.0.smithi016.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-02T04:41:28.017 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-02T04:41:28.017 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-02T04:41:28.136 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:27 smithi027 ceph-mon[37550]: pgmap v236: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:41:28.136 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:27 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:41:28.203 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:27 smithi016 ceph-mon[28969]: pgmap v236: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:41:28.203 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:27 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:41:28.797 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-02T04:41:28.854 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-02T04:41:28.894 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-02T04:41:28.895 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-02T04:41:28.895 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-02T04:41:28.934 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-02T04:41:28.936 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-02T04:41:28.936 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-02T04:41:28.992 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-04-02T04:41:28.995 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-04-02T04:41:28.995 DEBUG:teuthology.orchestra.run.smithi016:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6cec793d9073f0594e2364b9b687ecaf22424cca TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-04-02T04:41:29.058 INFO:tasks.workunit.client.0.smithi016.stderr:+ mkdir -p fsstress 2024-04-02T04:41:29.060 INFO:tasks.workunit.client.0.smithi016.stderr:+ pushd fsstress 2024-04-02T04:41:29.061 INFO:tasks.workunit.client.0.smithi016.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-02T04:41:29.062 INFO:tasks.workunit.client.0.smithi016.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-04-02T04:41:30.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:29 smithi016 ceph-mon[28969]: pgmap v237: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:41:30.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:29 smithi027 ceph-mon[37550]: pgmap v237: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:41:30.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:30.885+0000 7f7fdf0a7700 1 -- 172.21.15.16:0/747346338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd8100be0 msgr2=0x7f7fd8101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:30.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:30.885+0000 7f7fdf0a7700 1 --2- 172.21.15.16:0/747346338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd8100be0 0x7f7fd8101040 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4009910 tx=0x7f7fc4009c20 comp rx=0 tx=0).stop 2024-04-02T04:41:30.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:30.886+0000 7f7fdf0a7700 1 -- 172.21.15.16:0/747346338 shutdown_connections 2024-04-02T04:41:30.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:30.886+0000 7f7fdf0a7700 1 --2- 172.21.15.16:0/747346338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd8100be0 0x7f7fd8101040 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:30.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:30.886+0000 7f7fdf0a7700 1 --2- 172.21.15.16:0/747346338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7fd80ff9e0 0x7f7fd80ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:30.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:30.886+0000 7f7fdf0a7700 1 -- 172.21.15.16:0/747346338 >> 172.21.15.16:0/747346338 conn(0x7f7fd80faf80 msgr2=0x7f7fd80fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:41:30.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:30.886+0000 7f7fdf0a7700 1 -- 172.21.15.16:0/747346338 shutdown_connections 2024-04-02T04:41:30.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:30.886+0000 7f7fdf0a7700 1 -- 172.21.15.16:0/747346338 wait complete. 2024-04-02T04:41:30.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:30.887+0000 7f7fdf0a7700 1 Processor -- start 2024-04-02T04:41:30.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:30.887+0000 7f7fdf0a7700 1 -- start start 2024-04-02T04:41:30.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:30.887+0000 7f7fdf0a7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7fd80ff9e0 0x7f7fd8103c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:41:30.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:30.887+0000 7f7fdf0a7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd8100be0 0x7f7fd8102310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:41:30.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:30.887+0000 7f7fdf0a7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd8102850 con 0x7f7fd8100be0 2024-04-02T04:41:30.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:30.887+0000 7f7fdf0a7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd8102990 con 0x7f7fd80ff9e0 2024-04-02T04:41:30.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:30.887+0000 7f7fd7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd8100be0 0x7f7fd8102310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:41:30.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:30.887+0000 7f7fd7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd8100be0 0x7f7fd8102310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40570/0 (socket says 172.21.15.16:40570) 2024-04-02T04:41:30.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:30.887+0000 7f7fd7fff700 1 -- 172.21.15.16:0/3683622314 learned_addr learned my addr 172.21.15.16:0/3683622314 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:41:30.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:30.887+0000 7f7fdce43700 1 --2- 172.21.15.16:0/3683622314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7fd80ff9e0 0x7f7fd8103c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:41:30.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:30.887+0000 7f7fd7fff700 1 -- 172.21.15.16:0/3683622314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7fd80ff9e0 msgr2=0x7f7fd8103c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:30.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:30.887+0000 7f7fd7fff700 1 --2- 172.21.15.16:0/3683622314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7fd80ff9e0 0x7f7fd8103c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:30.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:30.887+0000 7f7fd7fff700 1 -- 172.21.15.16:0/3683622314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc40095f0 con 0x7f7fd8100be0 2024-04-02T04:41:30.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:30.887+0000 7f7fd7fff700 1 --2- 172.21.15.16:0/3683622314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd8100be0 0x7f7fd8102310 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40092e0 tx=0x7f7fc400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:41:30.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:30.887+0000 7f7fd5ffb700 1 -- 172.21.15.16:0/3683622314 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fc401d070 con 0x7f7fd8100be0 2024-04-02T04:41:30.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:30.888+0000 7f7fdf0a7700 1 -- 172.21.15.16:0/3683622314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fd8102c10 con 0x7f7fd8100be0 2024-04-02T04:41:30.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:30.888+0000 7f7fd5ffb700 1 -- 172.21.15.16:0/3683622314 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7fc400f460 con 0x7f7fd8100be0 2024-04-02T04:41:30.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:30.888+0000 7f7fd5ffb700 1 -- 172.21.15.16:0/3683622314 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fc4021620 con 0x7f7fd8100be0 2024-04-02T04:41:30.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:30.888+0000 7f7fdf0a7700 1 -- 172.21.15.16:0/3683622314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fd8103180 con 0x7f7fd8100be0 2024-04-02T04:41:30.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:30.889+0000 7f7fdf0a7700 1 -- 172.21.15.16:0/3683622314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fd804ea80 con 0x7f7fd8100be0 2024-04-02T04:41:30.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:30.889+0000 7f7fd5ffb700 1 -- 172.21.15.16:0/3683622314 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f7fc400f5d0 con 0x7f7fd8100be0 2024-04-02T04:41:30.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:30.890+0000 7f7fd5ffb700 1 --2- 172.21.15.16:0/3683622314 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f7fc806cce0 0x7f7fc806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:41:30.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:30.890+0000 7f7fd5ffb700 1 -- 172.21.15.16:0/3683622314 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f7fc408d340 con 0x7f7fd8100be0 2024-04-02T04:41:30.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:30.890+0000 7f7fdce43700 1 --2- 172.21.15.16:0/3683622314 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f7fc806cce0 0x7f7fc806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:41:30.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:30.890+0000 7f7fdce43700 1 --2- 172.21.15.16:0/3683622314 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f7fc806cce0 0x7f7fc806f1a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8100a40 tx=0x7f7fcc00b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:41:30.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:30.894+0000 7f7fd5ffb700 1 -- 172.21.15.16:0/3683622314 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7fc405b140 con 0x7f7fd8100be0 2024-04-02T04:41:31.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.119+0000 7f7fdf0a7700 1 -- 172.21.15.16:0/3683622314 --> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7fd8103180 con 0x7f7fc806cce0 2024-04-02T04:41:31.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.120+0000 7f7fd5ffb700 1 -- 172.21.15.16:0/3683622314 <== mgr.14217 v2:172.21.15.16:6800/3643600900 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f7fd8103180 con 0x7f7fc806cce0 2024-04-02T04:41:31.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.122+0000 7f7fdf0a7700 1 -- 172.21.15.16:0/3683622314 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f7fc806cce0 msgr2=0x7f7fc806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:31.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.122+0000 7f7fdf0a7700 1 --2- 172.21.15.16:0/3683622314 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f7fc806cce0 0x7f7fc806f1a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8100a40 tx=0x7f7fcc00b410 comp rx=0 tx=0).stop 2024-04-02T04:41:31.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.122+0000 7f7fdf0a7700 1 -- 172.21.15.16:0/3683622314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd8100be0 msgr2=0x7f7fd8102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:31.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.122+0000 7f7fdf0a7700 1 --2- 172.21.15.16:0/3683622314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd8100be0 0x7f7fd8102310 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40092e0 tx=0x7f7fc400bd70 comp rx=0 tx=0).stop 2024-04-02T04:41:31.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.123+0000 7f7fdf0a7700 1 -- 172.21.15.16:0/3683622314 shutdown_connections 2024-04-02T04:41:31.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.123+0000 7f7fdf0a7700 1 --2- 172.21.15.16:0/3683622314 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f7fc806cce0 0x7f7fc806f1a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:31.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.123+0000 7f7fdf0a7700 1 --2- 172.21.15.16:0/3683622314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7fd80ff9e0 0x7f7fd8103c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:31.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.123+0000 7f7fdf0a7700 1 --2- 172.21.15.16:0/3683622314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd8100be0 0x7f7fd8102310 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:31.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.123+0000 7f7fdf0a7700 1 -- 172.21.15.16:0/3683622314 >> 172.21.15.16:0/3683622314 conn(0x7f7fd80faf80 msgr2=0x7f7fd8068fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:41:31.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.123+0000 7f7fdf0a7700 1 -- 172.21.15.16:0/3683622314 shutdown_connections 2024-04-02T04:41:31.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.123+0000 7f7fdf0a7700 1 -- 172.21.15.16:0/3683622314 wait complete. 2024-04-02T04:41:31.136 INFO:teuthology.orchestra.run.smithi016.stdout:true 2024-04-02T04:41:31.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.237+0000 7f9bdbe50700 1 -- 172.21.15.16:0/2753863170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bd40feda0 msgr2=0x7f9bd40ff1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:31.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.237+0000 7f9bdbe50700 1 --2- 172.21.15.16:0/2753863170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bd40feda0 0x7f9bd40ff1c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0009910 tx=0x7f9bd0009c20 comp rx=0 tx=0).stop 2024-04-02T04:41:31.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.237+0000 7f9bdbe50700 1 -- 172.21.15.16:0/2753863170 shutdown_connections 2024-04-02T04:41:31.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.237+0000 7f9bdbe50700 1 --2- 172.21.15.16:0/2753863170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bd4101d80 0x7f9bd41021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:31.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.237+0000 7f9bdbe50700 1 --2- 172.21.15.16:0/2753863170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bd40feda0 0x7f9bd40ff1c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:31.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.237+0000 7f9bdbe50700 1 -- 172.21.15.16:0/2753863170 >> 172.21.15.16:0/2753863170 conn(0x7f9bd40fa700 msgr2=0x7f9bd40fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:41:31.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.238+0000 7f9bdbe50700 1 -- 172.21.15.16:0/2753863170 shutdown_connections 2024-04-02T04:41:31.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.238+0000 7f9bdbe50700 1 -- 172.21.15.16:0/2753863170 wait complete. 2024-04-02T04:41:31.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.238+0000 7f9bdbe50700 1 Processor -- start 2024-04-02T04:41:31.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.238+0000 7f9bdbe50700 1 -- start start 2024-04-02T04:41:31.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.238+0000 7f9bdbe50700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bd40feda0 0x7f9bd4101640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:41:31.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.238+0000 7f9bdbe50700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bd4101d80 0x7f9bd40ffd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:41:31.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.238+0000 7f9bdbe50700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd4100270 con 0x7f9bd40feda0 2024-04-02T04:41:31.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.238+0000 7f9bdbe50700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd41003b0 con 0x7f9bd4101d80 2024-04-02T04:41:31.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.239+0000 7f9bd9bec700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bd40feda0 0x7f9bd4101640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:41:31.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.239+0000 7f9bd9bec700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bd40feda0 0x7f9bd4101640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40594/0 (socket says 172.21.15.16:40594) 2024-04-02T04:41:31.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.239+0000 7f9bd9bec700 1 -- 172.21.15.16:0/1153686247 learned_addr learned my addr 172.21.15.16:0/1153686247 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:41:31.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.239+0000 7f9bd93eb700 1 --2- 172.21.15.16:0/1153686247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bd4101d80 0x7f9bd40ffd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:41:31.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.239+0000 7f9bd9bec700 1 -- 172.21.15.16:0/1153686247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bd4101d80 msgr2=0x7f9bd40ffd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:31.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.239+0000 7f9bd9bec700 1 --2- 172.21.15.16:0/1153686247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bd4101d80 0x7f9bd40ffd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:31.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.239+0000 7f9bd9bec700 1 -- 172.21.15.16:0/1153686247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd00095f0 con 0x7f9bd40feda0 2024-04-02T04:41:31.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.239+0000 7f9bd9bec700 1 --2- 172.21.15.16:0/1153686247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bd40feda0 0x7f9bd4101640 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9bd00092e0 tx=0x7f9bd000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:41:31.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.239+0000 7f9bcaffd700 1 -- 172.21.15.16:0/1153686247 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bd001d070 con 0x7f9bd40feda0 2024-04-02T04:41:31.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.239+0000 7f9bdbe50700 1 -- 172.21.15.16:0/1153686247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bd4100630 con 0x7f9bd40feda0 2024-04-02T04:41:31.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.239+0000 7f9bdbe50700 1 -- 172.21.15.16:0/1153686247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bd4100b20 con 0x7f9bd40feda0 2024-04-02T04:41:31.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.239+0000 7f9bcaffd700 1 -- 172.21.15.16:0/1153686247 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9bd0005dc0 con 0x7f9bd40feda0 2024-04-02T04:41:31.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.240+0000 7f9bcaffd700 1 -- 172.21.15.16:0/1153686247 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bd0017400 con 0x7f9bd40feda0 2024-04-02T04:41:31.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.241+0000 7f9bdbe50700 1 -- 172.21.15.16:0/1153686247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bb8005320 con 0x7f9bd40feda0 2024-04-02T04:41:31.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.241+0000 7f9bcaffd700 1 -- 172.21.15.16:0/1153686247 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f9bd0003a60 con 0x7f9bd40feda0 2024-04-02T04:41:31.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.241+0000 7f9bcaffd700 1 --2- 172.21.15.16:0/1153686247 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f9bc006cce0 0x7f9bc006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:41:31.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.242+0000 7f9bcaffd700 1 -- 172.21.15.16:0/1153686247 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f9bd008d530 con 0x7f9bd40feda0 2024-04-02T04:41:31.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.242+0000 7f9bd93eb700 1 --2- 172.21.15.16:0/1153686247 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f9bc006cce0 0x7f9bc006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:41:31.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.242+0000 7f9bd93eb700 1 --2- 172.21.15.16:0/1153686247 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f9bc006cce0 0x7f9bc006f1a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4007c60 tx=0x7f9bc40073d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:41:31.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.247+0000 7f9bcaffd700 1 -- 172.21.15.16:0/1153686247 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9bd005b3b0 con 0x7f9bd40feda0 2024-04-02T04:41:31.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.485+0000 7f9bdbe50700 1 -- 172.21.15.16:0/1153686247 --> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9bb8000bf0 con 0x7f9bc006cce0 2024-04-02T04:41:31.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.486+0000 7f9bcaffd700 1 -- 172.21.15.16:0/1153686247 <== mgr.14217 v2:172.21.15.16:6800/3643600900 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f9bb8000bf0 con 0x7f9bc006cce0 2024-04-02T04:41:31.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.488+0000 7f9bdbe50700 1 -- 172.21.15.16:0/1153686247 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f9bc006cce0 msgr2=0x7f9bc006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:31.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.488+0000 7f9bdbe50700 1 --2- 172.21.15.16:0/1153686247 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f9bc006cce0 0x7f9bc006f1a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4007c60 tx=0x7f9bc40073d0 comp rx=0 tx=0).stop 2024-04-02T04:41:31.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.488+0000 7f9bdbe50700 1 -- 172.21.15.16:0/1153686247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bd40feda0 msgr2=0x7f9bd4101640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:31.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.488+0000 7f9bdbe50700 1 --2- 172.21.15.16:0/1153686247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bd40feda0 0x7f9bd4101640 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9bd00092e0 tx=0x7f9bd000bd70 comp rx=0 tx=0).stop 2024-04-02T04:41:31.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.488+0000 7f9bdbe50700 1 -- 172.21.15.16:0/1153686247 shutdown_connections 2024-04-02T04:41:31.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.488+0000 7f9bdbe50700 1 --2- 172.21.15.16:0/1153686247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bd40feda0 0x7f9bd4101640 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:31.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.488+0000 7f9bdbe50700 1 --2- 172.21.15.16:0/1153686247 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f9bc006cce0 0x7f9bc006f1a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:31.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.488+0000 7f9bdbe50700 1 --2- 172.21.15.16:0/1153686247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bd4101d80 0x7f9bd40ffd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:31.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.488+0000 7f9bdbe50700 1 -- 172.21.15.16:0/1153686247 >> 172.21.15.16:0/1153686247 conn(0x7f9bd40fa700 msgr2=0x7f9bd41053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:41:31.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.488+0000 7f9bdbe50700 1 -- 172.21.15.16:0/1153686247 shutdown_connections 2024-04-02T04:41:31.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.488+0000 7f9bdbe50700 1 -- 172.21.15.16:0/1153686247 wait complete. 2024-04-02T04:41:31.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.595+0000 7f1207ad5700 1 -- 172.21.15.16:0/888145526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1200100bd0 msgr2=0x7f1200101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:31.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.595+0000 7f1207ad5700 1 --2- 172.21.15.16:0/888145526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1200100bd0 0x7f1200101030 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f11f0009910 tx=0x7f11f0009c20 comp rx=0 tx=0).stop 2024-04-02T04:41:31.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.595+0000 7f1207ad5700 1 -- 172.21.15.16:0/888145526 shutdown_connections 2024-04-02T04:41:31.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.595+0000 7f1207ad5700 1 --2- 172.21.15.16:0/888145526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1200100bd0 0x7f1200101030 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:31.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.595+0000 7f1207ad5700 1 --2- 172.21.15.16:0/888145526 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12000ff9d0 0x7f12000ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:31.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.595+0000 7f1207ad5700 1 -- 172.21.15.16:0/888145526 >> 172.21.15.16:0/888145526 conn(0x7f12000faf70 msgr2=0x7f12000fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:41:31.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.595+0000 7f1207ad5700 1 -- 172.21.15.16:0/888145526 shutdown_connections 2024-04-02T04:41:31.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.595+0000 7f1207ad5700 1 -- 172.21.15.16:0/888145526 wait complete. 2024-04-02T04:41:31.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.596+0000 7f1207ad5700 1 Processor -- start 2024-04-02T04:41:31.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.596+0000 7f1207ad5700 1 -- start start 2024-04-02T04:41:31.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.596+0000 7f1207ad5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12000ff9d0 0x7f12001965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:41:31.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.596+0000 7f1207ad5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1200100bd0 0x7f1200196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:41:31.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.596+0000 7f1207ad5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1200197130 con 0x7f12000ff9d0 2024-04-02T04:41:31.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.596+0000 7f1207ad5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1200197270 con 0x7f1200100bd0 2024-04-02T04:41:31.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.596+0000 7f1205871700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12000ff9d0 0x7f12001965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:41:31.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.596+0000 7f1205871700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12000ff9d0 0x7f12001965d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40620/0 (socket says 172.21.15.16:40620) 2024-04-02T04:41:31.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.596+0000 7f1205871700 1 -- 172.21.15.16:0/86049686 learned_addr learned my addr 172.21.15.16:0/86049686 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:41:31.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.596+0000 7f1205070700 1 --2- 172.21.15.16:0/86049686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1200100bd0 0x7f1200196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:41:31.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.597+0000 7f1205871700 1 -- 172.21.15.16:0/86049686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1200100bd0 msgr2=0x7f1200196b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:31.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.597+0000 7f1205871700 1 --2- 172.21.15.16:0/86049686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1200100bd0 0x7f1200196b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:31.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.597+0000 7f1205871700 1 -- 172.21.15.16:0/86049686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f00095f0 con 0x7f12000ff9d0 2024-04-02T04:41:31.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.597+0000 7f1205070700 1 --2- 172.21.15.16:0/86049686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1200100bd0 0x7f1200196b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T04:41:31.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.597+0000 7f1205871700 1 --2- 172.21.15.16:0/86049686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12000ff9d0 0x7f12001965d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00efc0 tx=0x7f11fc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:41:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.597+0000 7f11f6ffd700 1 -- 172.21.15.16:0/86049686 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11fc018070 con 0x7f12000ff9d0 2024-04-02T04:41:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.597+0000 7f11f6ffd700 1 -- 172.21.15.16:0/86049686 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f11fc00f040 con 0x7f12000ff9d0 2024-04-02T04:41:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.597+0000 7f1207ad5700 1 -- 172.21.15.16:0/86049686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f120019bc20 con 0x7f12000ff9d0 2024-04-02T04:41:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.597+0000 7f11f6ffd700 1 -- 172.21.15.16:0/86049686 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11fc0044b0 con 0x7f12000ff9d0 2024-04-02T04:41:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.597+0000 7f1207ad5700 1 -- 172.21.15.16:0/86049686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f120019c170 con 0x7f12000ff9d0 2024-04-02T04:41:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.598+0000 7f1207ad5700 1 -- 172.21.15.16:0/86049686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f120004ea80 con 0x7f12000ff9d0 2024-04-02T04:41:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.599+0000 7f11f6ffd700 1 -- 172.21.15.16:0/86049686 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f11fc004730 con 0x7f12000ff9d0 2024-04-02T04:41:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.599+0000 7f11f6ffd700 1 --2- 172.21.15.16:0/86049686 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f11ec06cce0 0x7f11ec06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:41:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.599+0000 7f11f6ffd700 1 -- 172.21.15.16:0/86049686 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f11fc08c4a0 con 0x7f12000ff9d0 2024-04-02T04:41:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.599+0000 7f1205070700 1 --2- 172.21.15.16:0/86049686 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f11ec06cce0 0x7f11ec06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:41:31.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.600+0000 7f1205070700 1 --2- 172.21.15.16:0/86049686 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f11ec06cce0 0x7f11ec06f1a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f11f000b800 tx=0x7f11f0009f70 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:41:31.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.604+0000 7f11f6ffd700 1 -- 172.21.15.16:0/86049686 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f11fc05a2a0 con 0x7f12000ff9d0 2024-04-02T04:41:31.809 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:31 smithi016 ceph-mon[28969]: Upgrade: Updating mgr.smithi027.kjzcsc 2024-04-02T04:41:31.809 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:31 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:41:31.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:31 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi027.kjzcsc", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T04:41:31.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:31 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T04:41:31.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:31 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:41:31.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:31 smithi016 ceph-mon[28969]: Deploying daemon mgr.smithi027.kjzcsc on smithi027 2024-04-02T04:41:31.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:31 smithi016 ceph-mon[28969]: pgmap v238: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:41:31.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.825+0000 7f1207ad5700 1 -- 172.21.15.16:0/86049686 --> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1200068c20 con 0x7f11ec06cce0 2024-04-02T04:41:31.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.836+0000 7f11f6ffd700 1 -- 172.21.15.16:0/86049686 <== mgr.14217 v2:172.21.15.16:6800/3643600900 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3864 (secure 0 0 0) 0x7f1200068c20 con 0x7f11ec06cce0 2024-04-02T04:41:31.839 INFO:teuthology.orchestra.run.smithi016.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:41:31.839 INFO:teuthology.orchestra.run.smithi016.stdout:alertmanager.smithi016 smithi016 *:9093,9094 running (5m) 89s ago 7m 24.5M - 0.25.0 c8568f914cd2 b95d8354b016 2024-04-02T04:41:31.839 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi016 smithi016 running (7m) 89s ago 7m 7465k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 9de6c16a487f 2024-04-02T04:41:31.839 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi027 smithi027 running (6m) 95s ago 6m 7398k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 99956592471a 2024-04-02T04:41:31.839 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi016 smithi016 running (7m) 89s ago 7m 6656k - 18.2.2-1059-g4d05f4ca 909dce4fe53d bd73919b0711 2024-04-02T04:41:31.839 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi027 smithi027 running (6m) 95s ago 6m 6660k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 7a54c37f202e 2024-04-02T04:41:31.839 INFO:teuthology.orchestra.run.smithi016.stdout:grafana.smithi016 smithi016 *:3000 running (5m) 89s ago 7m 84.9M - 9.4.7 954c08fa6188 7c43d133569c 2024-04-02T04:41:31.839 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.fyzrnm smithi016 running (109s) 89s ago 108s 13.4M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 70f28b9572a6 2024-04-02T04:41:31.839 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.uhgkgg smithi016 running (116s) 89s ago 116s 11.5M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 0aa4d16562a6 2024-04-02T04:41:31.839 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.sdublg smithi027 running (106s) 95s ago 105s 15.6M - 18.2.2-1059-g4d05f4ca 909dce4fe53d a538451099db 2024-04-02T04:41:31.840 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.zwrfoo smithi027 running (114s) 95s ago 113s 18.5M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 4cb3fa428d71 2024-04-02T04:41:31.840 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi016.eqnebx smithi016 *:9283,8765,8443 running (8m) 89s ago 8m 497M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 7d7ad6d79d01 2024-04-02T04:41:31.840 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi027.kjzcsc smithi027 *:8443,9283,8765 running (6m) 95s ago 6m 451M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 1ced128585bc 2024-04-02T04:41:31.840 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi016 smithi016 running (9m) 89s ago 9m 46.2M 2048M 18.2.2-1059-g4d05f4ca 909dce4fe53d 24d5820dc994 2024-04-02T04:41:31.840 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi027 smithi027 running (6m) 95s ago 6m 35.4M 2048M 18.2.2-1059-g4d05f4ca 909dce4fe53d b77563184a1e 2024-04-02T04:41:31.840 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi016 smithi016 *:9100 running (7m) 89s ago 7m 14.6M - 1.5.0 0da6a335fe13 09f54c65b53c 2024-04-02T04:41:31.840 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi027 smithi027 *:9100 running (6m) 95s ago 6m 14.8M - 1.5.0 0da6a335fe13 e41a657c7443 2024-04-02T04:41:31.840 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 smithi016 running (5m) 89s ago 5m 55.9M 3948M 18.2.2-1059-g4d05f4ca 909dce4fe53d 3f40108debd6 2024-04-02T04:41:31.840 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 smithi016 running (5m) 89s ago 5m 54.4M 3948M 18.2.2-1059-g4d05f4ca 909dce4fe53d 7361abc9e1b6 2024-04-02T04:41:31.840 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 smithi016 running (4m) 89s ago 4m 51.5M 3948M 18.2.2-1059-g4d05f4ca 909dce4fe53d 52e592b4d1f2 2024-04-02T04:41:31.840 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 smithi027 running (4m) 95s ago 4m 48.8M 4970M 18.2.2-1059-g4d05f4ca 909dce4fe53d cf7c7fe022f0 2024-04-02T04:41:31.840 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 smithi027 running (3m) 95s ago 3m 50.1M 4970M 18.2.2-1059-g4d05f4ca 909dce4fe53d 8cbbedf8cac9 2024-04-02T04:41:31.840 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 smithi027 running (3m) 95s ago 3m 45.4M 4970M 18.2.2-1059-g4d05f4ca 909dce4fe53d ac6094415529 2024-04-02T04:41:31.840 INFO:teuthology.orchestra.run.smithi016.stdout:prometheus.smithi016 smithi016 *:9095 running (5m) 89s ago 6m 53.9M - 2.43.0 a07b618ecd1d 232d70bae1ba 2024-04-02T04:41:31.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.838+0000 7f1207ad5700 1 -- 172.21.15.16:0/86049686 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f11ec06cce0 msgr2=0x7f11ec06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:31.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.838+0000 7f1207ad5700 1 --2- 172.21.15.16:0/86049686 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f11ec06cce0 0x7f11ec06f1a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f11f000b800 tx=0x7f11f0009f70 comp rx=0 tx=0).stop 2024-04-02T04:41:31.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.838+0000 7f1207ad5700 1 -- 172.21.15.16:0/86049686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12000ff9d0 msgr2=0x7f12001965d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:31.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.838+0000 7f1207ad5700 1 --2- 172.21.15.16:0/86049686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12000ff9d0 0x7f12001965d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00efc0 tx=0x7f11fc00c3a0 comp rx=0 tx=0).stop 2024-04-02T04:41:31.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.838+0000 7f1207ad5700 1 -- 172.21.15.16:0/86049686 shutdown_connections 2024-04-02T04:41:31.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.838+0000 7f1207ad5700 1 --2- 172.21.15.16:0/86049686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12000ff9d0 0x7f12001965d0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:31.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.838+0000 7f1207ad5700 1 --2- 172.21.15.16:0/86049686 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f11ec06cce0 0x7f11ec06f1a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:31.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.838+0000 7f1207ad5700 1 --2- 172.21.15.16:0/86049686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1200100bd0 0x7f1200196b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:31.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.838+0000 7f1207ad5700 1 -- 172.21.15.16:0/86049686 >> 172.21.15.16:0/86049686 conn(0x7f12000faf70 msgr2=0x7f1200068510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:41:31.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.838+0000 7f1207ad5700 1 -- 172.21.15.16:0/86049686 shutdown_connections 2024-04-02T04:41:31.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.838+0000 7f1207ad5700 1 -- 172.21.15.16:0/86049686 wait complete. 2024-04-02T04:41:31.895 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:31 smithi027 ceph-mon[37550]: Upgrade: Updating mgr.smithi027.kjzcsc 2024-04-02T04:41:31.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:31 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:41:31.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:31 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi027.kjzcsc", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T04:41:31.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:31 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T04:41:31.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:31 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:41:31.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:31 smithi027 ceph-mon[37550]: Deploying daemon mgr.smithi027.kjzcsc on smithi027 2024-04-02T04:41:31.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:31 smithi027 ceph-mon[37550]: pgmap v238: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:41:31.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.949+0000 7f301fb4c700 1 -- 172.21.15.16:0/2622943812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3018101ce0 msgr2=0x7f3018102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:31.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.949+0000 7f301fb4c700 1 --2- 172.21.15.16:0/2622943812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3018101ce0 0x7f3018102140 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3008009910 tx=0x7f3008009c20 comp rx=0 tx=0).stop 2024-04-02T04:41:31.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.949+0000 7f301fb4c700 1 -- 172.21.15.16:0/2622943812 shutdown_connections 2024-04-02T04:41:31.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.949+0000 7f301fb4c700 1 --2- 172.21.15.16:0/2622943812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3018101ce0 0x7f3018102140 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:31.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.949+0000 7f301fb4c700 1 --2- 172.21.15.16:0/2622943812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3018100ae0 0x7f3018100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:31.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.949+0000 7f301fb4c700 1 -- 172.21.15.16:0/2622943812 >> 172.21.15.16:0/2622943812 conn(0x7f30180fc0a0 msgr2=0x7f30180fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:41:31.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.950+0000 7f301fb4c700 1 -- 172.21.15.16:0/2622943812 shutdown_connections 2024-04-02T04:41:31.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.950+0000 7f301fb4c700 1 -- 172.21.15.16:0/2622943812 wait complete. 2024-04-02T04:41:31.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.950+0000 7f301fb4c700 1 Processor -- start 2024-04-02T04:41:31.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.950+0000 7f301fb4c700 1 -- start start 2024-04-02T04:41:31.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.950+0000 7f301fb4c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3018100ae0 0x7f3018074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:41:31.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.950+0000 7f301fb4c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3018101ce0 0x7f30180731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:41:31.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.950+0000 7f301fb4c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30180736e0 con 0x7f3018100ae0 2024-04-02T04:41:31.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.950+0000 7f301fb4c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3018073820 con 0x7f3018101ce0 2024-04-02T04:41:31.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.951+0000 7f301d8e8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3018100ae0 0x7f3018074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:41:31.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.951+0000 7f301d8e8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3018100ae0 0x7f3018074b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39850/0 (socket says 172.21.15.16:39850) 2024-04-02T04:41:31.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.951+0000 7f301d8e8700 1 -- 172.21.15.16:0/4077883968 learned_addr learned my addr 172.21.15.16:0/4077883968 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:41:31.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.951+0000 7f301d0e7700 1 --2- 172.21.15.16:0/4077883968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3018101ce0 0x7f30180731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:41:31.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.951+0000 7f301d8e8700 1 -- 172.21.15.16:0/4077883968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3018101ce0 msgr2=0x7f30180731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:31.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.951+0000 7f301d8e8700 1 --2- 172.21.15.16:0/4077883968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3018101ce0 0x7f30180731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:31.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.951+0000 7f301d8e8700 1 -- 172.21.15.16:0/4077883968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30080095f0 con 0x7f3018100ae0 2024-04-02T04:41:31.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.951+0000 7f301d8e8700 1 --2- 172.21.15.16:0/4077883968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3018100ae0 0x7f3018074b00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f301400d900 tx=0x7f301400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:41:31.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.951+0000 7f300effd700 1 -- 172.21.15.16:0/4077883968 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30140041f0 con 0x7f3018100ae0 2024-04-02T04:41:31.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.951+0000 7f300effd700 1 -- 172.21.15.16:0/4077883968 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3014004350 con 0x7f3018100ae0 2024-04-02T04:41:31.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.951+0000 7f301fb4c700 1 -- 172.21.15.16:0/4077883968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3018073b00 con 0x7f3018100ae0 2024-04-02T04:41:31.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.951+0000 7f300effd700 1 -- 172.21.15.16:0/4077883968 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3014003cf0 con 0x7f3018100ae0 2024-04-02T04:41:31.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.951+0000 7f301fb4c700 1 -- 172.21.15.16:0/4077883968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f301819e520 con 0x7f3018100ae0 2024-04-02T04:41:31.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.953+0000 7f301fb4c700 1 -- 172.21.15.16:0/4077883968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3018190660 con 0x7f3018100ae0 2024-04-02T04:41:31.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.953+0000 7f300effd700 1 -- 172.21.15.16:0/4077883968 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f3014009730 con 0x7f3018100ae0 2024-04-02T04:41:31.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.954+0000 7f300effd700 1 --2- 172.21.15.16:0/4077883968 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f300406cce0 0x7f300406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:41:31.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.954+0000 7f300effd700 1 -- 172.21.15.16:0/4077883968 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f301408c4d0 con 0x7f3018100ae0 2024-04-02T04:41:31.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.954+0000 7f301d0e7700 1 --2- 172.21.15.16:0/4077883968 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f300406cce0 0x7f300406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:41:31.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.955+0000 7f301d0e7700 1 --2- 172.21.15.16:0/4077883968 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f300406cce0 0x7f300406f1a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3008005090 tx=0x7f3008005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:41:31.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:31.960+0000 7f300effd700 1 -- 172.21.15.16:0/4077883968 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f301405a3f0 con 0x7f3018100ae0 2024-04-02T04:41:32.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.278+0000 7f301fb4c700 1 -- 172.21.15.16:0/4077883968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3018109640 con 0x7f3018100ae0 2024-04-02T04:41:32.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.279+0000 7f300effd700 1 -- 172.21.15.16:0/4077883968 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f3014059f80 con 0x7f3018100ae0 2024-04-02T04:41:32.282 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:41:32.282 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": { 2024-04-02T04:41:32.282 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 2 2024-04-02T04:41:32.282 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:41:32.282 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": { 2024-04-02T04:41:32.282 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 2 2024-04-02T04:41:32.282 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:41:32.282 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": { 2024-04-02T04:41:32.282 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 6 2024-04-02T04:41:32.282 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:41:32.283 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-04-02T04:41:32.283 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 4 2024-04-02T04:41:32.283 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:41:32.283 INFO:teuthology.orchestra.run.smithi016.stdout: "overall": { 2024-04-02T04:41:32.283 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 14 2024-04-02T04:41:32.283 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-02T04:41:32.283 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:41:32.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.281+0000 7f301fb4c700 1 -- 172.21.15.16:0/4077883968 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f300406cce0 msgr2=0x7f300406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:32.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.281+0000 7f301fb4c700 1 --2- 172.21.15.16:0/4077883968 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f300406cce0 0x7f300406f1a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3008005090 tx=0x7f3008005960 comp rx=0 tx=0).stop 2024-04-02T04:41:32.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.281+0000 7f301fb4c700 1 -- 172.21.15.16:0/4077883968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3018100ae0 msgr2=0x7f3018074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:32.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.281+0000 7f301fb4c700 1 --2- 172.21.15.16:0/4077883968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3018100ae0 0x7f3018074b00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f301400d900 tx=0x7f301400dc10 comp rx=0 tx=0).stop 2024-04-02T04:41:32.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.281+0000 7f301fb4c700 1 -- 172.21.15.16:0/4077883968 shutdown_connections 2024-04-02T04:41:32.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.281+0000 7f301fb4c700 1 --2- 172.21.15.16:0/4077883968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3018100ae0 0x7f3018074b00 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:32.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.281+0000 7f301fb4c700 1 --2- 172.21.15.16:0/4077883968 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f300406cce0 0x7f300406f1a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:32.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.281+0000 7f301fb4c700 1 --2- 172.21.15.16:0/4077883968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3018101ce0 0x7f30180731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:32.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.281+0000 7f301fb4c700 1 -- 172.21.15.16:0/4077883968 >> 172.21.15.16:0/4077883968 conn(0x7f30180fc0a0 msgr2=0x7f3018105280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:41:32.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.281+0000 7f301fb4c700 1 -- 172.21.15.16:0/4077883968 shutdown_connections 2024-04-02T04:41:32.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.281+0000 7f301fb4c700 1 -- 172.21.15.16:0/4077883968 wait complete. 2024-04-02T04:41:32.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.390+0000 7f7cefc4d700 1 -- 172.21.15.16:0/1888025132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ce8102cf0 msgr2=0x7f7ce8103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:32.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.390+0000 7f7cefc4d700 1 --2- 172.21.15.16:0/1888025132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ce8102cf0 0x7f7ce8103110 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4009910 tx=0x7f7ce4009c20 comp rx=0 tx=0).stop 2024-04-02T04:41:32.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.391+0000 7f7cefc4d700 1 -- 172.21.15.16:0/1888025132 shutdown_connections 2024-04-02T04:41:32.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.391+0000 7f7cefc4d700 1 --2- 172.21.15.16:0/1888025132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ce8103ef0 0x7f7ce8104350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:32.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.391+0000 7f7cefc4d700 1 --2- 172.21.15.16:0/1888025132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ce8102cf0 0x7f7ce8103110 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:32.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.391+0000 7f7cefc4d700 1 -- 172.21.15.16:0/1888025132 >> 172.21.15.16:0/1888025132 conn(0x7f7ce80fe2f0 msgr2=0x7f7ce8100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:41:32.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.391+0000 7f7cefc4d700 1 -- 172.21.15.16:0/1888025132 shutdown_connections 2024-04-02T04:41:32.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.391+0000 7f7cefc4d700 1 -- 172.21.15.16:0/1888025132 wait complete. 2024-04-02T04:41:32.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.391+0000 7f7cefc4d700 1 Processor -- start 2024-04-02T04:41:32.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.391+0000 7f7cefc4d700 1 -- start start 2024-04-02T04:41:32.396 INFO:tasks.workunit.client.0.smithi016.stderr:+ tar xzf ltp-full.tgz 2024-04-02T04:41:32.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.392+0000 7f7cefc4d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ce8102cf0 0x7f7ce81986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:41:32.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.392+0000 7f7cefc4d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ce8103ef0 0x7f7ce8198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:41:32.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.392+0000 7f7cefc4d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce8199250 con 0x7f7ce8103ef0 2024-04-02T04:41:32.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.392+0000 7f7cefc4d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce8199390 con 0x7f7ce8102cf0 2024-04-02T04:41:32.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.392+0000 7f7ced1e8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ce8103ef0 0x7f7ce8198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:41:32.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.392+0000 7f7ced1e8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ce8103ef0 0x7f7ce8198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39870/0 (socket says 172.21.15.16:39870) 2024-04-02T04:41:32.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.392+0000 7f7ced9e9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ce8102cf0 0x7f7ce81986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:41:32.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.392+0000 7f7ced1e8700 1 -- 172.21.15.16:0/4006177120 learned_addr learned my addr 172.21.15.16:0/4006177120 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:41:32.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.392+0000 7f7ced1e8700 1 -- 172.21.15.16:0/4006177120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ce8102cf0 msgr2=0x7f7ce81986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:32.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.392+0000 7f7ced1e8700 1 --2- 172.21.15.16:0/4006177120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ce8102cf0 0x7f7ce81986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:32.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.392+0000 7f7ced1e8700 1 -- 172.21.15.16:0/4006177120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce40095f0 con 0x7f7ce8103ef0 2024-04-02T04:41:32.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.392+0000 7f7ced1e8700 1 --2- 172.21.15.16:0/4006177120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ce8103ef0 0x7f7ce8198c30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7cd800d900 tx=0x7f7cd800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:41:32.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.392+0000 7f7cdeffd700 1 -- 172.21.15.16:0/4006177120 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cd80041f0 con 0x7f7ce8103ef0 2024-04-02T04:41:32.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.393+0000 7f7cefc4d700 1 -- 172.21.15.16:0/4006177120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ce819ddf0 con 0x7f7ce8103ef0 2024-04-02T04:41:32.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.393+0000 7f7cdeffd700 1 -- 172.21.15.16:0/4006177120 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7cd8004cf0 con 0x7f7ce8103ef0 2024-04-02T04:41:32.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.393+0000 7f7cefc4d700 1 -- 172.21.15.16:0/4006177120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce819e310 con 0x7f7ce8103ef0 2024-04-02T04:41:32.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.394+0000 7f7cdeffd700 1 -- 172.21.15.16:0/4006177120 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cd800b750 con 0x7f7ce8103ef0 2024-04-02T04:41:32.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.394+0000 7f7cefc4d700 1 -- 172.21.15.16:0/4006177120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ccc005320 con 0x7f7ce8103ef0 2024-04-02T04:41:32.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.395+0000 7f7cdeffd700 1 -- 172.21.15.16:0/4006177120 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f7cd800b8b0 con 0x7f7ce8103ef0 2024-04-02T04:41:32.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.396+0000 7f7cdeffd700 1 --2- 172.21.15.16:0/4006177120 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f7cd406cce0 0x7f7cd406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:41:32.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.396+0000 7f7ced9e9700 1 --2- 172.21.15.16:0/4006177120 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f7cd406cce0 0x7f7cd406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:41:32.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.396+0000 7f7cdeffd700 1 -- 172.21.15.16:0/4006177120 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f7cd808c100 con 0x7f7ce8103ef0 2024-04-02T04:41:32.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.396+0000 7f7ced9e9700 1 --2- 172.21.15.16:0/4006177120 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f7cd406cce0 0x7f7cd406f1a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7ce400b5c0 tx=0x7f7ce4005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:41:32.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.401+0000 7f7cdeffd700 1 -- 172.21.15.16:0/4006177120 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7cd8059f20 con 0x7f7ce8103ef0 2024-04-02T04:41:32.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.699+0000 7f7cefc4d700 1 -- 172.21.15.16:0/4006177120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7ccc005de0 con 0x7f7ce8103ef0 2024-04-02T04:41:32.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.699+0000 7f7cdeffd700 1 -- 172.21.15.16:0/4006177120 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1843 (secure 0 0 0) 0x7f7cd8016750 con 0x7f7ce8103ef0 2024-04-02T04:41:32.702 INFO:teuthology.orchestra.run.smithi016.stdout:e15 2024-04-02T04:41:32.703 INFO:teuthology.orchestra.run.smithi016.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:41:32.703 INFO:teuthology.orchestra.run.smithi016.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:41:32.703 INFO:teuthology.orchestra.run.smithi016.stdout:legacy client fscid: 1 2024-04-02T04:41:32.703 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:41:32.703 INFO:teuthology.orchestra.run.smithi016.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:41:32.703 INFO:teuthology.orchestra.run.smithi016.stdout:fs_name cephfs 2024-04-02T04:41:32.703 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 15 2024-04-02T04:41:32.703 INFO:teuthology.orchestra.run.smithi016.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:41:32.703 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-04-02T04:39:30.740168+0000 2024-04-02T04:41:32.704 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-04-02T04:39:49.875319+0000 2024-04-02T04:41:32.704 INFO:teuthology.orchestra.run.smithi016.stdout:tableserver 0 2024-04-02T04:41:32.704 INFO:teuthology.orchestra.run.smithi016.stdout:root 0 2024-04-02T04:41:32.704 INFO:teuthology.orchestra.run.smithi016.stdout:session_timeout 60 2024-04-02T04:41:32.704 INFO:teuthology.orchestra.run.smithi016.stdout:session_autoclose 300 2024-04-02T04:41:32.704 INFO:teuthology.orchestra.run.smithi016.stdout:max_file_size 1099511627776 2024-04-02T04:41:32.704 INFO:teuthology.orchestra.run.smithi016.stdout:max_xattr_size 65536 2024-04-02T04:41:32.704 INFO:teuthology.orchestra.run.smithi016.stdout:required_client_features {} 2024-04-02T04:41:32.704 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure 0 2024-04-02T04:41:32.704 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure_osd_epoch 46 2024-04-02T04:41:32.704 INFO:teuthology.orchestra.run.smithi016.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:41:32.704 INFO:teuthology.orchestra.run.smithi016.stdout:max_mds 1 2024-04-02T04:41:32.704 INFO:teuthology.orchestra.run.smithi016.stdout:in 0 2024-04-02T04:41:32.704 INFO:teuthology.orchestra.run.smithi016.stdout:up {0=24285} 2024-04-02T04:41:32.705 INFO:teuthology.orchestra.run.smithi016.stdout:failed 2024-04-02T04:41:32.705 INFO:teuthology.orchestra.run.smithi016.stdout:damaged 2024-04-02T04:41:32.705 INFO:teuthology.orchestra.run.smithi016.stdout:stopped 2024-04-02T04:41:32.705 INFO:teuthology.orchestra.run.smithi016.stdout:data_pools [3] 2024-04-02T04:41:32.705 INFO:teuthology.orchestra.run.smithi016.stdout:metadata_pool 2 2024-04-02T04:41:32.705 INFO:teuthology.orchestra.run.smithi016.stdout:inline_data enabled 2024-04-02T04:41:32.705 INFO:teuthology.orchestra.run.smithi016.stdout:balancer 2024-04-02T04:41:32.705 INFO:teuthology.orchestra.run.smithi016.stdout:bal_rank_mask -1 2024-04-02T04:41:32.705 INFO:teuthology.orchestra.run.smithi016.stdout:standby_count_wanted 1 2024-04-02T04:41:32.705 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.zwrfoo{0:24285} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.27:6824/1461455976,v1:172.21.15.27:6825/1461455976] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:41:32.705 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:41:32.705 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:41:32.705 INFO:teuthology.orchestra.run.smithi016.stdout:Standby daemons: 2024-04-02T04:41:32.705 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:41:32.705 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.fyzrnm{-1:14488} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.16:6828/2578719859,v1:172.21.15.16:6829/2578719859] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:41:32.705 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.uhgkgg{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.16:6826/601263579,v1:172.21.15.16:6827/601263579] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:41:32.706 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.sdublg{-1:24295} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.27:6826/4177192740,v1:172.21.15.27:6827/4177192740] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:41:32.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.702+0000 7f7cefc4d700 1 -- 172.21.15.16:0/4006177120 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f7cd406cce0 msgr2=0x7f7cd406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:32.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.702+0000 7f7cefc4d700 1 --2- 172.21.15.16:0/4006177120 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f7cd406cce0 0x7f7cd406f1a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7ce400b5c0 tx=0x7f7ce4005960 comp rx=0 tx=0).stop 2024-04-02T04:41:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.702+0000 7f7cefc4d700 1 -- 172.21.15.16:0/4006177120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ce8103ef0 msgr2=0x7f7ce8198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.702+0000 7f7cefc4d700 1 --2- 172.21.15.16:0/4006177120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ce8103ef0 0x7f7ce8198c30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7cd800d900 tx=0x7f7cd800dc10 comp rx=0 tx=0).stop 2024-04-02T04:41:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.702+0000 7f7cefc4d700 1 -- 172.21.15.16:0/4006177120 shutdown_connections 2024-04-02T04:41:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.702+0000 7f7cefc4d700 1 --2- 172.21.15.16:0/4006177120 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f7cd406cce0 0x7f7cd406f1a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.702+0000 7f7cefc4d700 1 --2- 172.21.15.16:0/4006177120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ce8102cf0 0x7f7ce81986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.702+0000 7f7cefc4d700 1 --2- 172.21.15.16:0/4006177120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ce8103ef0 0x7f7ce8198c30 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.702+0000 7f7cefc4d700 1 -- 172.21.15.16:0/4006177120 >> 172.21.15.16:0/4006177120 conn(0x7f7ce80fe2f0 msgr2=0x7f7ce8107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:41:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.702+0000 7f7cefc4d700 1 -- 172.21.15.16:0/4006177120 shutdown_connections 2024-04-02T04:41:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.702+0000 7f7cefc4d700 1 -- 172.21.15.16:0/4006177120 wait complete. 2024-04-02T04:41:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 15 2024-04-02T04:41:32.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.816+0000 7f9d63aaa700 1 -- 172.21.15.16:0/337886778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d5c101d00 msgr2=0x7f9d5c102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:32.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.816+0000 7f9d63aaa700 1 --2- 172.21.15.16:0/337886778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d5c101d00 0x7f9d5c102160 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9d50009910 tx=0x7f9d50009c20 comp rx=0 tx=0).stop 2024-04-02T04:41:32.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.816+0000 7f9d63aaa700 1 -- 172.21.15.16:0/337886778 shutdown_connections 2024-04-02T04:41:32.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.816+0000 7f9d63aaa700 1 --2- 172.21.15.16:0/337886778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d5c101d00 0x7f9d5c102160 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:32.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.816+0000 7f9d63aaa700 1 --2- 172.21.15.16:0/337886778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d5c100b00 0x7f9d5c100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:32.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.816+0000 7f9d63aaa700 1 -- 172.21.15.16:0/337886778 >> 172.21.15.16:0/337886778 conn(0x7f9d5c0fc0a0 msgr2=0x7f9d5c0fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:41:32.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.816+0000 7f9d63aaa700 1 -- 172.21.15.16:0/337886778 shutdown_connections 2024-04-02T04:41:32.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.816+0000 7f9d63aaa700 1 -- 172.21.15.16:0/337886778 wait complete. 2024-04-02T04:41:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.817+0000 7f9d63aaa700 1 Processor -- start 2024-04-02T04:41:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.817+0000 7f9d63aaa700 1 -- start start 2024-04-02T04:41:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.817+0000 7f9d63aaa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d5c100b00 0x7f9d5c1964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:41:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.817+0000 7f9d63aaa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d5c101d00 0x7f9d5c196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:41:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.817+0000 7f9d63aaa700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c197040 con 0x7f9d5c100b00 2024-04-02T04:41:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.817+0000 7f9d63aaa700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c197180 con 0x7f9d5c101d00 2024-04-02T04:41:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.817+0000 7f9d61846700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d5c100b00 0x7f9d5c1964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:41:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.817+0000 7f9d61846700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d5c100b00 0x7f9d5c1964e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39878/0 (socket says 172.21.15.16:39878) 2024-04-02T04:41:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.817+0000 7f9d61846700 1 -- 172.21.15.16:0/164401681 learned_addr learned my addr 172.21.15.16:0/164401681 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:41:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.817+0000 7f9d61045700 1 --2- 172.21.15.16:0/164401681 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d5c101d00 0x7f9d5c196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:41:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.817+0000 7f9d61846700 1 -- 172.21.15.16:0/164401681 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d5c101d00 msgr2=0x7f9d5c196a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.817+0000 7f9d61846700 1 --2- 172.21.15.16:0/164401681 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d5c101d00 0x7f9d5c196a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.817+0000 7f9d61846700 1 -- 172.21.15.16:0/164401681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d500095f0 con 0x7f9d5c100b00 2024-04-02T04:41:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.818+0000 7f9d61846700 1 --2- 172.21.15.16:0/164401681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d5c100b00 0x7f9d5c1964e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9d54007e90 tx=0x7f9d5400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:41:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.818+0000 7f9d5affd700 1 -- 172.21.15.16:0/164401681 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d54017070 con 0x7f9d5c100b00 2024-04-02T04:41:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.818+0000 7f9d5affd700 1 -- 172.21.15.16:0/164401681 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d5400f040 con 0x7f9d5c100b00 2024-04-02T04:41:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.818+0000 7f9d63aaa700 1 -- 172.21.15.16:0/164401681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d5c19bbe0 con 0x7f9d5c100b00 2024-04-02T04:41:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.818+0000 7f9d5affd700 1 -- 172.21.15.16:0/164401681 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d54004850 con 0x7f9d5c100b00 2024-04-02T04:41:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.818+0000 7f9d63aaa700 1 -- 172.21.15.16:0/164401681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d5c19c130 con 0x7f9d5c100b00 2024-04-02T04:41:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.819+0000 7f9d63aaa700 1 -- 172.21.15.16:0/164401681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d5c190680 con 0x7f9d5c100b00 2024-04-02T04:41:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.819+0000 7f9d5affd700 1 -- 172.21.15.16:0/164401681 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f9d54003a00 con 0x7f9d5c100b00 2024-04-02T04:41:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.820+0000 7f9d5affd700 1 --2- 172.21.15.16:0/164401681 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f9d4406cc80 0x7f9d4406f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:41:32.824 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:32 smithi016 ceph-mon[28969]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:41:32.824 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:32 smithi016 ceph-mon[28969]: from='client.14632 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:41:32.824 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:32 smithi016 ceph-mon[28969]: from='client.14636 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:41:32.824 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:32 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/4077883968' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:41:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.820+0000 7f9d5affd700 1 -- 172.21.15.16:0/164401681 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f9d5408bcb0 con 0x7f9d5c100b00 2024-04-02T04:41:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.820+0000 7f9d61045700 1 --2- 172.21.15.16:0/164401681 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f9d4406cc80 0x7f9d4406f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:41:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.820+0000 7f9d61045700 1 --2- 172.21.15.16:0/164401681 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f9d4406cc80 0x7f9d4406f140 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9d50005090 tx=0x7f9d50005960 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:41:32.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:32.825+0000 7f9d5affd700 1 -- 172.21.15.16:0/164401681 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9d54059ab0 con 0x7f9d5c100b00 2024-04-02T04:41:32.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:32 smithi027 ceph-mon[37550]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:41:32.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:32 smithi027 ceph-mon[37550]: from='client.14632 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:41:32.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:32 smithi027 ceph-mon[37550]: from='client.14636 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:41:32.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:32 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/4077883968' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:41:33.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.097+0000 7f9d63aaa700 1 -- 172.21.15.16:0/164401681 --> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d5c02cdf0 con 0x7f9d4406cc80 2024-04-02T04:41:33.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.098+0000 7f9d5affd700 1 -- 172.21.15.16:0/164401681 <== mgr.14217 v2:172.21.15.16:6800/3643600900 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f9d5c02cdf0 con 0x7f9d4406cc80 2024-04-02T04:41:33.101 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:41:33.102 INFO:teuthology.orchestra.run.smithi016.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:41:33.102 INFO:teuthology.orchestra.run.smithi016.stdout: "in_progress": true, 2024-04-02T04:41:33.102 INFO:teuthology.orchestra.run.smithi016.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-02T04:41:33.102 INFO:teuthology.orchestra.run.smithi016.stdout: "services_complete": [], 2024-04-02T04:41:33.102 INFO:teuthology.orchestra.run.smithi016.stdout: "progress": "0/23 daemons upgraded", 2024-04-02T04:41:33.102 INFO:teuthology.orchestra.run.smithi016.stdout: "message": "Currently upgrading mgr daemons", 2024-04-02T04:41:33.102 INFO:teuthology.orchestra.run.smithi016.stdout: "is_paused": false 2024-04-02T04:41:33.102 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:41:33.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.100+0000 7f9d63aaa700 1 -- 172.21.15.16:0/164401681 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f9d4406cc80 msgr2=0x7f9d4406f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:33.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.101+0000 7f9d63aaa700 1 --2- 172.21.15.16:0/164401681 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f9d4406cc80 0x7f9d4406f140 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9d50005090 tx=0x7f9d50005960 comp rx=0 tx=0).stop 2024-04-02T04:41:33.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.101+0000 7f9d63aaa700 1 -- 172.21.15.16:0/164401681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d5c100b00 msgr2=0x7f9d5c1964e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:33.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.101+0000 7f9d63aaa700 1 --2- 172.21.15.16:0/164401681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d5c100b00 0x7f9d5c1964e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9d54007e90 tx=0x7f9d5400d3b0 comp rx=0 tx=0).stop 2024-04-02T04:41:33.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.101+0000 7f9d63aaa700 1 -- 172.21.15.16:0/164401681 shutdown_connections 2024-04-02T04:41:33.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.101+0000 7f9d63aaa700 1 --2- 172.21.15.16:0/164401681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d5c100b00 0x7f9d5c1964e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:33.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.101+0000 7f9d63aaa700 1 --2- 172.21.15.16:0/164401681 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f9d4406cc80 0x7f9d4406f140 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:33.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.101+0000 7f9d63aaa700 1 --2- 172.21.15.16:0/164401681 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d5c101d00 0x7f9d5c196a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:33.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.101+0000 7f9d63aaa700 1 -- 172.21.15.16:0/164401681 >> 172.21.15.16:0/164401681 conn(0x7f9d5c0fc0a0 msgr2=0x7f9d5c105250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:41:33.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.101+0000 7f9d63aaa700 1 -- 172.21.15.16:0/164401681 shutdown_connections 2024-04-02T04:41:33.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.101+0000 7f9d63aaa700 1 -- 172.21.15.16:0/164401681 wait complete. 2024-04-02T04:41:33.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.215+0000 7f28d17ba700 1 -- 172.21.15.16:0/433454251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28cc103ed0 msgr2=0x7f28cc1062c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:33.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.215+0000 7f28d17ba700 1 --2- 172.21.15.16:0/433454251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28cc103ed0 0x7f28cc1062c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f28c0009910 tx=0x7f28c0009c20 comp rx=0 tx=0).stop 2024-04-02T04:41:33.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.215+0000 7f28d17ba700 1 -- 172.21.15.16:0/433454251 shutdown_connections 2024-04-02T04:41:33.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.215+0000 7f28d17ba700 1 --2- 172.21.15.16:0/433454251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28cc103ed0 0x7f28cc1062c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:33.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.215+0000 7f28d17ba700 1 --2- 172.21.15.16:0/433454251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28cc1015a0 0x7f28cc103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:33.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.215+0000 7f28d17ba700 1 -- 172.21.15.16:0/433454251 >> 172.21.15.16:0/433454251 conn(0x7f28cc0faf70 msgr2=0x7f28cc0fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:41:33.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.215+0000 7f28d17ba700 1 -- 172.21.15.16:0/433454251 shutdown_connections 2024-04-02T04:41:33.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.216+0000 7f28d17ba700 1 -- 172.21.15.16:0/433454251 wait complete. 2024-04-02T04:41:33.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.216+0000 7f28d17ba700 1 Processor -- start 2024-04-02T04:41:33.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.216+0000 7f28d17ba700 1 -- start start 2024-04-02T04:41:33.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.216+0000 7f28d17ba700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28cc1015a0 0x7f28cc1964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:41:33.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.216+0000 7f28d17ba700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28cc103ed0 0x7f28cc1969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:41:33.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.216+0000 7f28d17ba700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc197010 con 0x7f28cc1015a0 2024-04-02T04:41:33.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.216+0000 7f28d17ba700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc197150 con 0x7f28cc103ed0 2024-04-02T04:41:33.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.217+0000 7f28caffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28cc1015a0 0x7f28cc1964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:41:33.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.217+0000 7f28caffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28cc1015a0 0x7f28cc1964b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39898/0 (socket says 172.21.15.16:39898) 2024-04-02T04:41:33.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.217+0000 7f28caffd700 1 -- 172.21.15.16:0/1014581140 learned_addr learned my addr 172.21.15.16:0/1014581140 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:41:33.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.217+0000 7f28ca7fc700 1 --2- 172.21.15.16:0/1014581140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28cc103ed0 0x7f28cc1969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:41:33.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.217+0000 7f28caffd700 1 -- 172.21.15.16:0/1014581140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28cc103ed0 msgr2=0x7f28cc1969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:33.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.217+0000 7f28caffd700 1 --2- 172.21.15.16:0/1014581140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28cc103ed0 0x7f28cc1969f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:33.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.217+0000 7f28caffd700 1 -- 172.21.15.16:0/1014581140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c00095f0 con 0x7f28cc1015a0 2024-04-02T04:41:33.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.217+0000 7f28caffd700 1 --2- 172.21.15.16:0/1014581140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28cc1015a0 0x7f28cc1964b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f28bc00e920 tx=0x7f28bc00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:41:33.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.217+0000 7f28b3fff700 1 -- 172.21.15.16:0/1014581140 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28bc00c9d0 con 0x7f28cc1015a0 2024-04-02T04:41:33.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.217+0000 7f28d17ba700 1 -- 172.21.15.16:0/1014581140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28cc19bbb0 con 0x7f28cc1015a0 2024-04-02T04:41:33.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.217+0000 7f28b3fff700 1 -- 172.21.15.16:0/1014581140 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f28bc010430 con 0x7f28cc1015a0 2024-04-02T04:41:33.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.217+0000 7f28b3fff700 1 -- 172.21.15.16:0/1014581140 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28bc018630 con 0x7f28cc1015a0 2024-04-02T04:41:33.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.217+0000 7f28d17ba700 1 -- 172.21.15.16:0/1014581140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28cc19c100 con 0x7f28cc1015a0 2024-04-02T04:41:33.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.219+0000 7f28d17ba700 1 -- 172.21.15.16:0/1014581140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28cc04ea80 con 0x7f28cc1015a0 2024-04-02T04:41:33.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.219+0000 7f28b3fff700 1 -- 172.21.15.16:0/1014581140 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f28bc0105a0 con 0x7f28cc1015a0 2024-04-02T04:41:33.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.220+0000 7f28b3fff700 1 --2- 172.21.15.16:0/1014581140 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f28b406ccf0 0x7f28b406f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:41:33.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.220+0000 7f28b3fff700 1 -- 172.21.15.16:0/1014581140 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f28bc014070 con 0x7f28cc1015a0 2024-04-02T04:41:33.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.220+0000 7f28ca7fc700 1 --2- 172.21.15.16:0/1014581140 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f28b406ccf0 0x7f28b406f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:41:33.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.220+0000 7f28ca7fc700 1 --2- 172.21.15.16:0/1014581140 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f28b406ccf0 0x7f28b406f1b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f28c000b800 tx=0x7f28c0009f70 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:41:33.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.225+0000 7f28b3fff700 1 -- 172.21.15.16:0/1014581140 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f28bc0529f0 con 0x7f28cc1015a0 2024-04-02T04:41:33.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.576+0000 7f28d17ba700 1 -- 172.21.15.16:0/1014581140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f28cc109670 con 0x7f28cc1015a0 2024-04-02T04:41:33.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.577+0000 7f28b3fff700 1 -- 172.21.15.16:0/1014581140 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f28bc056030 con 0x7f28cc1015a0 2024-04-02T04:41:33.580 INFO:teuthology.orchestra.run.smithi016.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-02T04:41:33.580 INFO:teuthology.orchestra.run.smithi016.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-02T04:41:33.580 INFO:teuthology.orchestra.run.smithi016.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-02T04:41:33.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.579+0000 7f28d17ba700 1 -- 172.21.15.16:0/1014581140 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f28b406ccf0 msgr2=0x7f28b406f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:33.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.579+0000 7f28d17ba700 1 --2- 172.21.15.16:0/1014581140 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f28b406ccf0 0x7f28b406f1b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f28c000b800 tx=0x7f28c0009f70 comp rx=0 tx=0).stop 2024-04-02T04:41:33.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.579+0000 7f28d17ba700 1 -- 172.21.15.16:0/1014581140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28cc1015a0 msgr2=0x7f28cc1964b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:41:33.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.579+0000 7f28d17ba700 1 --2- 172.21.15.16:0/1014581140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28cc1015a0 0x7f28cc1964b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f28bc00e920 tx=0x7f28bc00ec30 comp rx=0 tx=0).stop 2024-04-02T04:41:33.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.579+0000 7f28d17ba700 1 -- 172.21.15.16:0/1014581140 shutdown_connections 2024-04-02T04:41:33.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.579+0000 7f28d17ba700 1 --2- 172.21.15.16:0/1014581140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28cc1015a0 0x7f28cc1964b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:33.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.579+0000 7f28d17ba700 1 --2- 172.21.15.16:0/1014581140 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f28b406ccf0 0x7f28b406f1b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:33.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.579+0000 7f28d17ba700 1 --2- 172.21.15.16:0/1014581140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28cc103ed0 0x7f28cc1969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:41:33.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.579+0000 7f28d17ba700 1 -- 172.21.15.16:0/1014581140 >> 172.21.15.16:0/1014581140 conn(0x7f28cc0faf70 msgr2=0x7f28cc0fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:41:33.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.580+0000 7f28d17ba700 1 -- 172.21.15.16:0/1014581140 shutdown_connections 2024-04-02T04:41:33.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:41:33.580+0000 7f28d17ba700 1 -- 172.21.15.16:0/1014581140 wait complete. 2024-04-02T04:41:33.895 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:33 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/4006177120' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:41:33.895 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:33 smithi027 ceph-mon[37550]: pgmap v239: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:41:34.035 INFO:tasks.workunit.client.1.smithi027.stderr:Updating files: 82% (10271/12435) Updating files: 83% (10322/12435) Updating files: 84% (10446/12435) Updating files: 85% (10570/12435) Updating files: 86% (10695/12435) Updating files: 87% (10819/12435) Updating files: 88% (10943/12435) Updating files: 89% (11068/12435) Updating files: 90% (11192/12435) Updating files: 91% (11316/12435) Updating files: 92% (11441/12435) Updating files: 93% (11565/12435) Updating files: 94% (11689/12435) Updating files: 95% (11814/12435) Updating files: 96% (11938/12435) Updating files: 97% (12062/12435) Updating files: 98% (12187/12435) Updating files: 99% (12311/12435) Updating files: 100% (12435/12435) Updating files: 100% (12435/12435), done. 2024-04-02T04:41:34.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:33 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/4006177120' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:41:34.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:33 smithi016 ceph-mon[28969]: pgmap v239: 65 pgs: 65 active+clean; 588 KiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:41:34.258 INFO:tasks.workunit.client.1.smithi027.stderr:Note: switching to '6cec793d9073f0594e2364b9b687ecaf22424cca'. 2024-04-02T04:41:34.258 INFO:tasks.workunit.client.1.smithi027.stderr: 2024-04-02T04:41:34.258 INFO:tasks.workunit.client.1.smithi027.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-02T04:41:34.258 INFO:tasks.workunit.client.1.smithi027.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-02T04:41:34.258 INFO:tasks.workunit.client.1.smithi027.stderr:state without impacting any branches by switching back to a branch. 2024-04-02T04:41:34.258 INFO:tasks.workunit.client.1.smithi027.stderr: 2024-04-02T04:41:34.258 INFO:tasks.workunit.client.1.smithi027.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-02T04:41:34.258 INFO:tasks.workunit.client.1.smithi027.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-02T04:41:34.258 INFO:tasks.workunit.client.1.smithi027.stderr: 2024-04-02T04:41:34.258 INFO:tasks.workunit.client.1.smithi027.stderr: git switch -c 2024-04-02T04:41:34.259 INFO:tasks.workunit.client.1.smithi027.stderr: 2024-04-02T04:41:34.259 INFO:tasks.workunit.client.1.smithi027.stderr:Or undo this operation with: 2024-04-02T04:41:34.259 INFO:tasks.workunit.client.1.smithi027.stderr: 2024-04-02T04:41:34.259 INFO:tasks.workunit.client.1.smithi027.stderr: git switch - 2024-04-02T04:41:34.259 INFO:tasks.workunit.client.1.smithi027.stderr: 2024-04-02T04:41:34.259 INFO:tasks.workunit.client.1.smithi027.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-02T04:41:34.259 INFO:tasks.workunit.client.1.smithi027.stderr: 2024-04-02T04:41:34.259 INFO:tasks.workunit.client.1.smithi027.stderr:HEAD is now at 6cec793d907 Merge pull request #56601 from zdover23/wip-doc-2024-04-01-backport-56600-to-squid 2024-04-02T04:41:34.263 DEBUG:teuthology.orchestra.run.smithi027:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-04-02T04:41:34.367 INFO:tasks.workunit.client.1.smithi027.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-02T04:41:34.369 INFO:tasks.workunit.client.1.smithi027.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-02T04:41:34.369 INFO:tasks.workunit.client.1.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-02T04:41:34.829 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:34 smithi027 ceph-mon[37550]: from='client.14648 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:41:34.829 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:34 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/1014581140' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:41:35.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:34 smithi016 ceph-mon[28969]: from='client.14648 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:41:35.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:34 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/1014581140' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:41:35.895 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:35 smithi027 ceph-mon[37550]: pgmap v240: 65 pgs: 65 active+clean; 13 MiB data, 220 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.0 MiB/s wr, 14 op/s 2024-04-02T04:41:36.022 INFO:tasks.workunit.client.1.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-02T04:41:36.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:35 smithi016 ceph-mon[28969]: pgmap v240: 65 pgs: 65 active+clean; 13 MiB data, 220 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.0 MiB/s wr, 14 op/s 2024-04-02T04:41:36.111 INFO:tasks.workunit.client.1.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-02T04:41:36.154 INFO:tasks.workunit.client.1.smithi027.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-02T04:41:36.155 INFO:tasks.workunit.client.1.smithi027.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-02T04:41:36.155 INFO:tasks.workunit.client.1.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-02T04:41:36.202 INFO:tasks.workunit.client.1.smithi027.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-02T04:41:36.206 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-02T04:41:36.206 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-04-02T04:41:36.225 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-04-02T04:41:36.226 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-04-02T04:41:36.226 DEBUG:teuthology.orchestra.run.smithi027:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6cec793d9073f0594e2364b9b687ecaf22424cca TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-04-02T04:41:36.295 INFO:tasks.workunit.client.1.smithi027.stderr:+ mkdir -p fsstress 2024-04-02T04:41:36.298 INFO:tasks.workunit.client.1.smithi027.stderr:+ pushd fsstress 2024-04-02T04:41:36.300 INFO:tasks.workunit.client.1.smithi027.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-02T04:41:36.300 INFO:tasks.workunit.client.1.smithi027.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-04-02T04:41:37.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:36 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:41:37.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:36 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:41:37.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:36 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:41:37.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:36 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:41:37.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:36 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:41:37.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:36 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:41:37.534 INFO:tasks.workunit.client.1.smithi027.stderr:+ tar xzf ltp-full.tgz 2024-04-02T04:41:38.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:37 smithi027 ceph-mon[37550]: pgmap v241: 65 pgs: 65 active+clean; 13 MiB data, 233 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.1 MiB/s wr, 24 op/s 2024-04-02T04:41:38.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:37 smithi016 ceph-mon[28969]: pgmap v241: 65 pgs: 65 active+clean; 13 MiB data, 233 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.1 MiB/s wr, 24 op/s 2024-04-02T04:41:40.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:39 smithi016 ceph-mon[28969]: pgmap v242: 65 pgs: 65 active+clean; 22 MiB data, 252 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.8 MiB/s wr, 45 op/s 2024-04-02T04:41:40.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:39 smithi027 ceph-mon[37550]: pgmap v242: 65 pgs: 65 active+clean; 22 MiB data, 252 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.8 MiB/s wr, 45 op/s 2024-04-02T04:41:41.895 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:41 smithi027 ceph-mon[37550]: pgmap v243: 65 pgs: 65 active+clean; 23 MiB data, 278 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.9 MiB/s wr, 70 op/s 2024-04-02T04:41:42.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:41 smithi016 ceph-mon[28969]: pgmap v243: 65 pgs: 65 active+clean; 23 MiB data, 278 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.9 MiB/s wr, 70 op/s 2024-04-02T04:41:43.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:42 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:41:43.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:42 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:41:43.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:42 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:41:43.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:42 smithi027 ceph-mon[37550]: pgmap v244: 65 pgs: 65 active+clean; 34 MiB data, 295 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.8 MiB/s wr, 84 op/s 2024-04-02T04:41:43.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:42 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:41:43.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:42 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:41:43.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:42 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:41:43.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:42 smithi016 ceph-mon[28969]: pgmap v244: 65 pgs: 65 active+clean; 34 MiB data, 295 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.8 MiB/s wr, 84 op/s 2024-04-02T04:41:44.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:44 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:41:44.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:44 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:41:44.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:44 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:41:44.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:44 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:41:45.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:45 smithi027 ceph-mon[37550]: pgmap v245: 65 pgs: 65 active+clean; 45 MiB data, 343 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.8 MiB/s wr, 122 op/s 2024-04-02T04:41:45.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:45 smithi016 ceph-mon[28969]: pgmap v245: 65 pgs: 65 active+clean; 45 MiB data, 343 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.8 MiB/s wr, 122 op/s 2024-04-02T04:41:46.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:46 smithi027 ceph-mon[37550]: Standby manager daemon smithi027.kjzcsc restarted 2024-04-02T04:41:46.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:46 smithi027 ceph-mon[37550]: Standby manager daemon smithi027.kjzcsc started 2024-04-02T04:41:46.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:46 smithi027 ceph-mon[37550]: from='mgr.? 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi027.kjzcsc/crt"}]: dispatch 2024-04-02T04:41:46.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:46 smithi027 ceph-mon[37550]: from='mgr.? 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T04:41:46.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:46 smithi027 ceph-mon[37550]: from='mgr.? 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi027.kjzcsc/key"}]: dispatch 2024-04-02T04:41:46.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:46 smithi027 ceph-mon[37550]: from='mgr.? 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T04:41:46.460 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:46 smithi016 ceph-mon[28969]: Standby manager daemon smithi027.kjzcsc restarted 2024-04-02T04:41:46.460 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:46 smithi016 ceph-mon[28969]: Standby manager daemon smithi027.kjzcsc started 2024-04-02T04:41:46.460 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:46 smithi016 ceph-mon[28969]: from='mgr.? 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi027.kjzcsc/crt"}]: dispatch 2024-04-02T04:41:46.460 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:46 smithi016 ceph-mon[28969]: from='mgr.? 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T04:41:46.461 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:46 smithi016 ceph-mon[28969]: from='mgr.? 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi027.kjzcsc/key"}]: dispatch 2024-04-02T04:41:46.461 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:46 smithi016 ceph-mon[28969]: from='mgr.? 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T04:41:47.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:47 smithi027 ceph-mon[37550]: mgrmap e20: smithi016.eqnebx(active, since 7m), standbys: smithi027.kjzcsc 2024-04-02T04:41:47.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:47 smithi027 ceph-mon[37550]: pgmap v246: 65 pgs: 65 active+clean; 46 MiB data, 352 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.9 MiB/s wr, 119 op/s 2024-04-02T04:41:47.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:47 smithi016 ceph-mon[28969]: mgrmap e20: smithi016.eqnebx(active, since 7m), standbys: smithi027.kjzcsc 2024-04-02T04:41:47.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:47 smithi016 ceph-mon[28969]: pgmap v246: 65 pgs: 65 active+clean; 46 MiB data, 352 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.9 MiB/s wr, 119 op/s 2024-04-02T04:41:50.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:49 smithi016 ceph-mon[28969]: pgmap v247: 65 pgs: 65 active+clean; 58 MiB data, 377 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.8 MiB/s wr, 142 op/s 2024-04-02T04:41:50.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:49 smithi027 ceph-mon[37550]: pgmap v247: 65 pgs: 65 active+clean; 58 MiB data, 377 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.8 MiB/s wr, 142 op/s 2024-04-02T04:41:52.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:51 smithi027 ceph-mon[37550]: pgmap v248: 65 pgs: 65 active+clean; 67 MiB data, 399 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.8 MiB/s wr, 158 op/s 2024-04-02T04:41:52.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:51 smithi016 ceph-mon[28969]: pgmap v248: 65 pgs: 65 active+clean; 67 MiB data, 399 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.8 MiB/s wr, 158 op/s 2024-04-02T04:41:54.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:53 smithi027 ceph-mon[37550]: pgmap v249: 65 pgs: 65 active+clean; 68 MiB data, 388 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.8 MiB/s wr, 142 op/s 2024-04-02T04:41:54.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:53 smithi016 ceph-mon[28969]: pgmap v249: 65 pgs: 65 active+clean; 68 MiB data, 388 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.8 MiB/s wr, 142 op/s 2024-04-02T04:41:56.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:55 smithi027 ceph-mon[37550]: pgmap v250: 65 pgs: 65 active+clean; 88 MiB data, 452 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.5 MiB/s wr, 181 op/s 2024-04-02T04:41:56.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:55 smithi016 ceph-mon[28969]: pgmap v250: 65 pgs: 65 active+clean; 88 MiB data, 452 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.5 MiB/s wr, 181 op/s 2024-04-02T04:41:58.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:57 smithi027 ceph-mon[37550]: pgmap v251: 65 pgs: 65 active+clean; 91 MiB data, 480 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.9 MiB/s wr, 166 op/s 2024-04-02T04:41:58.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:57 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:41:58.253 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:57 smithi016 ceph-mon[28969]: pgmap v251: 65 pgs: 65 active+clean; 91 MiB data, 480 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.9 MiB/s wr, 166 op/s 2024-04-02T04:41:58.253 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:57 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:42:00.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:41:59 smithi016 ceph-mon[28969]: pgmap v252: 65 pgs: 65 active+clean; 102 MiB data, 501 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.7 MiB/s wr, 188 op/s 2024-04-02T04:42:00.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:41:59 smithi027 ceph-mon[37550]: pgmap v252: 65 pgs: 65 active+clean; 102 MiB data, 501 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.7 MiB/s wr, 188 op/s 2024-04-02T04:42:02.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:01 smithi027 ceph-mon[37550]: pgmap v253: 65 pgs: 65 active+clean; 106 MiB data, 511 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.1 MiB/s wr, 173 op/s 2024-04-02T04:42:02.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:01 smithi016 ceph-mon[28969]: pgmap v253: 65 pgs: 65 active+clean; 106 MiB data, 511 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.1 MiB/s wr, 173 op/s 2024-04-02T04:42:03.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.695+0000 7fd36ca99700 1 -- 172.21.15.16:0/2109452041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd368100ae0 msgr2=0x7fd368100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:03.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.695+0000 7fd36ca99700 1 --2- 172.21.15.16:0/2109452041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd368100ae0 0x7fd368100f00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd358009910 tx=0x7fd358009c20 comp rx=0 tx=0).stop 2024-04-02T04:42:03.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.695+0000 7fd36ca99700 1 -- 172.21.15.16:0/2109452041 shutdown_connections 2024-04-02T04:42:03.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.695+0000 7fd36ca99700 1 --2- 172.21.15.16:0/2109452041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd368101ce0 0x7fd368102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:03.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.695+0000 7fd36ca99700 1 --2- 172.21.15.16:0/2109452041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd368100ae0 0x7fd368100f00 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:03.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.695+0000 7fd36ca99700 1 -- 172.21.15.16:0/2109452041 >> 172.21.15.16:0/2109452041 conn(0x7fd3680fc0a0 msgr2=0x7fd3680fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:42:03.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.695+0000 7fd36ca99700 1 -- 172.21.15.16:0/2109452041 shutdown_connections 2024-04-02T04:42:03.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.695+0000 7fd36ca99700 1 -- 172.21.15.16:0/2109452041 wait complete. 2024-04-02T04:42:03.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.696+0000 7fd36ca99700 1 Processor -- start 2024-04-02T04:42:03.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.696+0000 7fd36ca99700 1 -- start start 2024-04-02T04:42:03.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.697+0000 7fd36ca99700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd368100ae0 0x7fd3681964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:03.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.697+0000 7fd36ca99700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd368101ce0 0x7fd368196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:03.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.697+0000 7fd36ca99700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd368197020 con 0x7fd368101ce0 2024-04-02T04:42:03.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.697+0000 7fd36ca99700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd368197160 con 0x7fd368100ae0 2024-04-02T04:42:03.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.697+0000 7fd365d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd368101ce0 0x7fd368196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:03.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.697+0000 7fd365d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd368101ce0 0x7fd368196a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34466/0 (socket says 172.21.15.16:34466) 2024-04-02T04:42:03.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.697+0000 7fd365d9b700 1 -- 172.21.15.16:0/3781065038 learned_addr learned my addr 172.21.15.16:0/3781065038 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:42:03.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.697+0000 7fd36659c700 1 --2- 172.21.15.16:0/3781065038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd368100ae0 0x7fd3681964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:03.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.697+0000 7fd365d9b700 1 -- 172.21.15.16:0/3781065038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd368100ae0 msgr2=0x7fd3681964c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:03.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.697+0000 7fd365d9b700 1 --2- 172.21.15.16:0/3781065038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd368100ae0 0x7fd3681964c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:03.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.697+0000 7fd365d9b700 1 -- 172.21.15.16:0/3781065038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3580095f0 con 0x7fd368101ce0 2024-04-02T04:42:03.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.697+0000 7fd365d9b700 1 --2- 172.21.15.16:0/3781065038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd368101ce0 0x7fd368196a00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00e920 tx=0x7fd35c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:42:03.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.698+0000 7fd3577fe700 1 -- 172.21.15.16:0/3781065038 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd35c00c9d0 con 0x7fd368101ce0 2024-04-02T04:42:03.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.698+0000 7fd3577fe700 1 -- 172.21.15.16:0/3781065038 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd35c010430 con 0x7fd368101ce0 2024-04-02T04:42:03.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.698+0000 7fd3577fe700 1 -- 172.21.15.16:0/3781065038 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd35c018630 con 0x7fd368101ce0 2024-04-02T04:42:03.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.698+0000 7fd36ca99700 1 -- 172.21.15.16:0/3781065038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd36819bbc0 con 0x7fd368101ce0 2024-04-02T04:42:03.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.698+0000 7fd36ca99700 1 -- 172.21.15.16:0/3781065038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd36819c190 con 0x7fd368101ce0 2024-04-02T04:42:03.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.699+0000 7fd36ca99700 1 -- 172.21.15.16:0/3781065038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd368190660 con 0x7fd368101ce0 2024-04-02T04:42:03.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.700+0000 7fd3577fe700 1 -- 172.21.15.16:0/3781065038 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 91278+0+0 (secure 0 0 0) 0x7fd35c0105a0 con 0x7fd368101ce0 2024-04-02T04:42:03.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.700+0000 7fd3577fe700 1 --2- 172.21.15.16:0/3781065038 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fd35006d3c0 0x7fd35006f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:03.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.700+0000 7fd3577fe700 1 -- 172.21.15.16:0/3781065038 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fd35c014070 con 0x7fd368101ce0 2024-04-02T04:42:03.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.700+0000 7fd36659c700 1 --2- 172.21.15.16:0/3781065038 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fd35006d3c0 0x7fd35006f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:03.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.701+0000 7fd36659c700 1 --2- 172.21.15.16:0/3781065038 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fd35006d3c0 0x7fd35006f880 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd35800b800 tx=0x7fd358009f70 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:42:03.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.705+0000 7fd3577fe700 1 -- 172.21.15.16:0/3781065038 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd35c0529f0 con 0x7fd368101ce0 2024-04-02T04:42:03.895 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:03 smithi027 ceph-mon[37550]: pgmap v254: 65 pgs: 65 active+clean; 107 MiB data, 520 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.4 MiB/s wr, 140 op/s 2024-04-02T04:42:03.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.971+0000 7fd36ca99700 1 -- 172.21.15.16:0/3781065038 --> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd36802ce10 con 0x7fd35006d3c0 2024-04-02T04:42:03.975 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:03 smithi016 ceph-mon[28969]: pgmap v254: 65 pgs: 65 active+clean; 107 MiB data, 520 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.4 MiB/s wr, 140 op/s 2024-04-02T04:42:03.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.973+0000 7fd3577fe700 1 -- 172.21.15.16:0/3781065038 <== mgr.14217 v2:172.21.15.16:6800/3643600900 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7fd36802ce10 con 0x7fd35006d3c0 2024-04-02T04:42:03.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.975+0000 7fd36ca99700 1 -- 172.21.15.16:0/3781065038 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fd35006d3c0 msgr2=0x7fd35006f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:03.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.975+0000 7fd36ca99700 1 --2- 172.21.15.16:0/3781065038 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fd35006d3c0 0x7fd35006f880 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd35800b800 tx=0x7fd358009f70 comp rx=0 tx=0).stop 2024-04-02T04:42:03.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.975+0000 7fd36ca99700 1 -- 172.21.15.16:0/3781065038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd368101ce0 msgr2=0x7fd368196a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:03.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.975+0000 7fd36ca99700 1 --2- 172.21.15.16:0/3781065038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd368101ce0 0x7fd368196a00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00e920 tx=0x7fd35c00ec30 comp rx=0 tx=0).stop 2024-04-02T04:42:03.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.975+0000 7fd36ca99700 1 -- 172.21.15.16:0/3781065038 shutdown_connections 2024-04-02T04:42:03.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.975+0000 7fd36ca99700 1 --2- 172.21.15.16:0/3781065038 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fd35006d3c0 0x7fd35006f880 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:03.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.975+0000 7fd36ca99700 1 --2- 172.21.15.16:0/3781065038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd368100ae0 0x7fd3681964c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:03.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.975+0000 7fd36ca99700 1 --2- 172.21.15.16:0/3781065038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd368101ce0 0x7fd368196a00 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:03.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.975+0000 7fd36ca99700 1 -- 172.21.15.16:0/3781065038 >> 172.21.15.16:0/3781065038 conn(0x7fd3680fc0a0 msgr2=0x7fd3681052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:42:03.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.975+0000 7fd36ca99700 1 -- 172.21.15.16:0/3781065038 shutdown_connections 2024-04-02T04:42:03.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:03.975+0000 7fd36ca99700 1 -- 172.21.15.16:0/3781065038 wait complete. 2024-04-02T04:42:03.989 INFO:teuthology.orchestra.run.smithi016.stdout:true 2024-04-02T04:42:04.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.096+0000 7fc72e811700 1 -- 172.21.15.16:0/1898229996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc728101cd0 msgr2=0x7fc728102130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:04.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.096+0000 7fc72e811700 1 --2- 172.21.15.16:0/1898229996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc728101cd0 0x7fc728102130 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc71c009910 tx=0x7fc71c009c20 comp rx=0 tx=0).stop 2024-04-02T04:42:04.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.096+0000 7fc72e811700 1 -- 172.21.15.16:0/1898229996 shutdown_connections 2024-04-02T04:42:04.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.096+0000 7fc72e811700 1 --2- 172.21.15.16:0/1898229996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc728101cd0 0x7fc728102130 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:04.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.096+0000 7fc72e811700 1 --2- 172.21.15.16:0/1898229996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc728100ad0 0x7fc728100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:04.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.096+0000 7fc72e811700 1 -- 172.21.15.16:0/1898229996 >> 172.21.15.16:0/1898229996 conn(0x7fc7280fc090 msgr2=0x7fc7280fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:42:04.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.096+0000 7fc72e811700 1 -- 172.21.15.16:0/1898229996 shutdown_connections 2024-04-02T04:42:04.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.097+0000 7fc72e811700 1 -- 172.21.15.16:0/1898229996 wait complete. 2024-04-02T04:42:04.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.097+0000 7fc72e811700 1 Processor -- start 2024-04-02T04:42:04.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.097+0000 7fc72e811700 1 -- start start 2024-04-02T04:42:04.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.097+0000 7fc72e811700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc728100ad0 0x7fc728196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:04.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.097+0000 7fc72e811700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc728101cd0 0x7fc728196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:04.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.097+0000 7fc72e811700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc728197080 con 0x7fc728101cd0 2024-04-02T04:42:04.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.097+0000 7fc72e811700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7281971c0 con 0x7fc728100ad0 2024-04-02T04:42:04.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.098+0000 7fc7277fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc728101cd0 0x7fc728196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:04.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.098+0000 7fc727fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc728100ad0 0x7fc728196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:04.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.098+0000 7fc7277fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc728101cd0 0x7fc728196a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34480/0 (socket says 172.21.15.16:34480) 2024-04-02T04:42:04.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.098+0000 7fc7277fe700 1 -- 172.21.15.16:0/2612727374 learned_addr learned my addr 172.21.15.16:0/2612727374 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:42:04.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.098+0000 7fc727fff700 1 --2- 172.21.15.16:0/2612727374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc728100ad0 0x7fc728196520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.16:52710/0 (socket says 172.21.15.16:52710) 2024-04-02T04:42:04.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.098+0000 7fc727fff700 1 -- 172.21.15.16:0/2612727374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc728101cd0 msgr2=0x7fc728196a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:04.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.098+0000 7fc727fff700 1 --2- 172.21.15.16:0/2612727374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc728101cd0 0x7fc728196a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:04.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.098+0000 7fc727fff700 1 -- 172.21.15.16:0/2612727374 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc71c0095f0 con 0x7fc728100ad0 2024-04-02T04:42:04.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.098+0000 7fc7277fe700 1 --2- 172.21.15.16:0/2612727374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc728101cd0 0x7fc728196a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T04:42:04.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.098+0000 7fc727fff700 1 --2- 172.21.15.16:0/2612727374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc728100ad0 0x7fc728196520 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc71800e940 tx=0x7fc71800ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:42:04.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.098+0000 7fc7257fa700 1 -- 172.21.15.16:0/2612727374 <== mon.1 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7180041f0 con 0x7fc728100ad0 2024-04-02T04:42:04.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.098+0000 7fc72e811700 1 -- 172.21.15.16:0/2612727374 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc72819bc20 con 0x7fc728100ad0 2024-04-02T04:42:04.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.099+0000 7fc7257fa700 1 -- 172.21.15.16:0/2612727374 <== mon.1 v2:172.21.15.27:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc718004350 con 0x7fc728100ad0 2024-04-02T04:42:04.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.099+0000 7fc7257fa700 1 -- 172.21.15.16:0/2612727374 <== mon.1 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc718003c90 con 0x7fc728100ad0 2024-04-02T04:42:04.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.099+0000 7fc72e811700 1 -- 172.21.15.16:0/2612727374 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc72819c170 con 0x7fc728100ad0 2024-04-02T04:42:04.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.100+0000 7fc72e811700 1 -- 172.21.15.16:0/2612727374 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc728190650 con 0x7fc728100ad0 2024-04-02T04:42:04.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.100+0000 7fc7257fa700 1 -- 172.21.15.16:0/2612727374 <== mon.1 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 20) v1 ==== 91278+0+0 (secure 0 0 0) 0x7fc71800c7c0 con 0x7fc728100ad0 2024-04-02T04:42:04.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.101+0000 7fc7257fa700 1 --2- 172.21.15.16:0/2612727374 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc71006d370 0x7fc71006f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:04.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.101+0000 7fc7257fa700 1 -- 172.21.15.16:0/2612727374 <== mon.1 v2:172.21.15.27:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fc718014070 con 0x7fc728100ad0 2024-04-02T04:42:04.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.101+0000 7fc7277fe700 1 --2- 172.21.15.16:0/2612727374 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc71006d370 0x7fc71006f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:04.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.101+0000 7fc7277fe700 1 --2- 172.21.15.16:0/2612727374 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc71006d370 0x7fc71006f830 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc71c00b6f0 tx=0x7fc71c005a90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:42:04.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.106+0000 7fc7257fa700 1 -- 172.21.15.16:0/2612727374 <== mon.1 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc718092020 con 0x7fc728100ad0 2024-04-02T04:42:04.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.359+0000 7fc72e811700 1 -- 172.21.15.16:0/2612727374 --> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc72802ce10 con 0x7fc71006d370 2024-04-02T04:42:04.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.361+0000 7fc7257fa700 1 -- 172.21.15.16:0/2612727374 <== mgr.14217 v2:172.21.15.16:6800/3643600900 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7fc72802ce10 con 0x7fc71006d370 2024-04-02T04:42:04.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.363+0000 7fc72e811700 1 -- 172.21.15.16:0/2612727374 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc71006d370 msgr2=0x7fc71006f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:04.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.363+0000 7fc72e811700 1 --2- 172.21.15.16:0/2612727374 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc71006d370 0x7fc71006f830 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc71c00b6f0 tx=0x7fc71c005a90 comp rx=0 tx=0).stop 2024-04-02T04:42:04.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.363+0000 7fc72e811700 1 -- 172.21.15.16:0/2612727374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc728100ad0 msgr2=0x7fc728196520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:04.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.363+0000 7fc72e811700 1 --2- 172.21.15.16:0/2612727374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc728100ad0 0x7fc728196520 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc71800e940 tx=0x7fc71800ec50 comp rx=0 tx=0).stop 2024-04-02T04:42:04.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.363+0000 7fc72e811700 1 -- 172.21.15.16:0/2612727374 shutdown_connections 2024-04-02T04:42:04.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.363+0000 7fc72e811700 1 --2- 172.21.15.16:0/2612727374 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc71006d370 0x7fc71006f830 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:04.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.363+0000 7fc72e811700 1 --2- 172.21.15.16:0/2612727374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc728100ad0 0x7fc728196520 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:04.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.363+0000 7fc72e811700 1 --2- 172.21.15.16:0/2612727374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc728101cd0 0x7fc728196a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:04.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.363+0000 7fc72e811700 1 -- 172.21.15.16:0/2612727374 >> 172.21.15.16:0/2612727374 conn(0x7fc7280fc090 msgr2=0x7fc728105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:42:04.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.363+0000 7fc72e811700 1 -- 172.21.15.16:0/2612727374 shutdown_connections 2024-04-02T04:42:04.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.363+0000 7fc72e811700 1 -- 172.21.15.16:0/2612727374 wait complete. 2024-04-02T04:42:04.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.479+0000 7f84c8fcc700 1 -- 172.21.15.16:0/629908505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c4103e90 msgr2=0x7f84c4106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:04.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.479+0000 7f84c8fcc700 1 --2- 172.21.15.16:0/629908505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c4103e90 0x7f84c4106280 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f84b8009910 tx=0x7f84b8009c20 comp rx=0 tx=0).stop 2024-04-02T04:42:04.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.479+0000 7f84c8fcc700 1 -- 172.21.15.16:0/629908505 shutdown_connections 2024-04-02T04:42:04.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.479+0000 7f84c8fcc700 1 --2- 172.21.15.16:0/629908505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c4103e90 0x7f84c4106280 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:04.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.479+0000 7f84c8fcc700 1 --2- 172.21.15.16:0/629908505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84c4101560 0x7f84c4103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:04.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.479+0000 7f84c8fcc700 1 -- 172.21.15.16:0/629908505 >> 172.21.15.16:0/629908505 conn(0x7f84c40faf70 msgr2=0x7f84c40fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:42:04.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.479+0000 7f84c8fcc700 1 -- 172.21.15.16:0/629908505 shutdown_connections 2024-04-02T04:42:04.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.479+0000 7f84c8fcc700 1 -- 172.21.15.16:0/629908505 wait complete. 2024-04-02T04:42:04.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.480+0000 7f84c8fcc700 1 Processor -- start 2024-04-02T04:42:04.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.480+0000 7f84c8fcc700 1 -- start start 2024-04-02T04:42:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.480+0000 7f84c8fcc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c4101560 0x7f84c4196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.480+0000 7f84c8fcc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84c4103e90 0x7f84c41969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.480+0000 7f84c8fcc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c4196ff0 con 0x7f84c4101560 2024-04-02T04:42:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.480+0000 7f84c8fcc700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c4197130 con 0x7f84c4103e90 2024-04-02T04:42:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.480+0000 7f84c259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c4101560 0x7f84c4196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.480+0000 7f84c1d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84c4103e90 0x7f84c41969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.480+0000 7f84c259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c4101560 0x7f84c4196490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34500/0 (socket says 172.21.15.16:34500) 2024-04-02T04:42:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.480+0000 7f84c259c700 1 -- 172.21.15.16:0/778905325 learned_addr learned my addr 172.21.15.16:0/778905325 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:42:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.481+0000 7f84c259c700 1 -- 172.21.15.16:0/778905325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84c4103e90 msgr2=0x7f84c41969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.481+0000 7f84c259c700 1 --2- 172.21.15.16:0/778905325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84c4103e90 0x7f84c41969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.481+0000 7f84c259c700 1 -- 172.21.15.16:0/778905325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b80095f0 con 0x7f84c4101560 2024-04-02T04:42:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.481+0000 7f84c1d9b700 1 --2- 172.21.15.16:0/778905325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84c4103e90 0x7f84c41969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T04:42:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.481+0000 7f84c259c700 1 --2- 172.21.15.16:0/778905325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c4101560 0x7f84c4196490 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f84b400e920 tx=0x7f84b400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:42:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.481+0000 7f84b37fe700 1 -- 172.21.15.16:0/778905325 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84b400c9d0 con 0x7f84c4101560 2024-04-02T04:42:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.481+0000 7f84b37fe700 1 -- 172.21.15.16:0/778905325 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f84b4010430 con 0x7f84c4101560 2024-04-02T04:42:04.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.481+0000 7f84c8fcc700 1 -- 172.21.15.16:0/778905325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c419bb90 con 0x7f84c4101560 2024-04-02T04:42:04.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.481+0000 7f84b37fe700 1 -- 172.21.15.16:0/778905325 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84b4018630 con 0x7f84c4101560 2024-04-02T04:42:04.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.481+0000 7f84c8fcc700 1 -- 172.21.15.16:0/778905325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c419c0e0 con 0x7f84c4101560 2024-04-02T04:42:04.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.482+0000 7f84c8fcc700 1 -- 172.21.15.16:0/778905325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c404ea80 con 0x7f84c4101560 2024-04-02T04:42:04.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.483+0000 7f84b37fe700 1 -- 172.21.15.16:0/778905325 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 91278+0+0 (secure 0 0 0) 0x7f84b40188f0 con 0x7f84c4101560 2024-04-02T04:42:04.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.483+0000 7f84b37fe700 1 --2- 172.21.15.16:0/778905325 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f84ac06d360 0x7f84ac06f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:04.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.483+0000 7f84b37fe700 1 -- 172.21.15.16:0/778905325 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f84b4014070 con 0x7f84c4101560 2024-04-02T04:42:04.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.483+0000 7f84c1d9b700 1 --2- 172.21.15.16:0/778905325 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f84ac06d360 0x7f84ac06f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:04.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.484+0000 7f84c1d9b700 1 --2- 172.21.15.16:0/778905325 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f84ac06d360 0x7f84ac06f820 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f84b800b6f0 tx=0x7f84b8005fb0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:42:04.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.489+0000 7f84b37fe700 1 -- 172.21.15.16:0/778905325 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f84b40529f0 con 0x7f84c4101560 2024-04-02T04:42:04.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.739+0000 7f84c8fcc700 1 -- 172.21.15.16:0/778905325 --> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f84c4105d70 con 0x7f84ac06d360 2024-04-02T04:42:04.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.751+0000 7f84b37fe700 1 -- 172.21.15.16:0/778905325 <== mgr.14217 v2:172.21.15.16:6800/3643600900 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f84c4105d70 con 0x7f84ac06d360 2024-04-02T04:42:04.755 INFO:teuthology.orchestra.run.smithi016.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:42:04.755 INFO:teuthology.orchestra.run.smithi016.stdout:alertmanager.smithi016 smithi016 *:9093,9094 running (6m) 2m ago 8m 24.5M - 0.25.0 c8568f914cd2 b95d8354b016 2024-04-02T04:42:04.755 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi016 smithi016 running (8m) 2m ago 8m 7465k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 9de6c16a487f 2024-04-02T04:42:04.756 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi027 smithi027 running (7m) 22s ago 7m 7578k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 99956592471a 2024-04-02T04:42:04.756 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi016 smithi016 running (8m) 2m ago 8m 6656k - 18.2.2-1059-g4d05f4ca 909dce4fe53d bd73919b0711 2024-04-02T04:42:04.756 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi027 smithi027 running (7m) 22s ago 7m 6660k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 7a54c37f202e 2024-04-02T04:42:04.756 INFO:teuthology.orchestra.run.smithi016.stdout:grafana.smithi016 smithi016 *:3000 running (6m) 2m ago 7m 84.9M - 9.4.7 954c08fa6188 7c43d133569c 2024-04-02T04:42:04.756 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.fyzrnm smithi016 running (2m) 2m ago 2m 13.4M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 70f28b9572a6 2024-04-02T04:42:04.756 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.uhgkgg smithi016 running (2m) 2m ago 2m 11.5M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 0aa4d16562a6 2024-04-02T04:42:04.757 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.sdublg smithi027 running (2m) 22s ago 2m 16.4M - 18.2.2-1059-g4d05f4ca 909dce4fe53d a538451099db 2024-04-02T04:42:04.757 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.zwrfoo smithi027 running (2m) 22s ago 2m 48.2M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 4cb3fa428d71 2024-04-02T04:42:04.757 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi016.eqnebx smithi016 *:9283,8765,8443 running (9m) 2m ago 9m 497M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 7d7ad6d79d01 2024-04-02T04:42:04.757 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi027.kjzcsc smithi027 *:8443,9283,8765 running (29s) 22s ago 6m 56.6M - 19.0.0-1979-g6cec793d 78f830190839 a1e2b7522dcb 2024-04-02T04:42:04.758 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi016 smithi016 running (9m) 2m ago 9m 46.2M 2048M 18.2.2-1059-g4d05f4ca 909dce4fe53d 24d5820dc994 2024-04-02T04:42:04.758 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi027 smithi027 running (6m) 22s ago 6m 36.4M 2048M 18.2.2-1059-g4d05f4ca 909dce4fe53d b77563184a1e 2024-04-02T04:42:04.758 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi016 smithi016 *:9100 running (8m) 2m ago 8m 14.6M - 1.5.0 0da6a335fe13 09f54c65b53c 2024-04-02T04:42:04.758 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi027 smithi027 *:9100 running (7m) 22s ago 7m 15.3M - 1.5.0 0da6a335fe13 e41a657c7443 2024-04-02T04:42:04.758 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 smithi016 running (6m) 2m ago 6m 55.9M 3948M 18.2.2-1059-g4d05f4ca 909dce4fe53d 3f40108debd6 2024-04-02T04:42:04.758 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 smithi016 running (5m) 2m ago 5m 54.4M 3948M 18.2.2-1059-g4d05f4ca 909dce4fe53d 7361abc9e1b6 2024-04-02T04:42:04.759 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 smithi016 running (5m) 2m ago 5m 51.5M 3948M 18.2.2-1059-g4d05f4ca 909dce4fe53d 52e592b4d1f2 2024-04-02T04:42:04.759 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 smithi027 running (4m) 22s ago 4m 56.2M 4970M 18.2.2-1059-g4d05f4ca 909dce4fe53d cf7c7fe022f0 2024-04-02T04:42:04.759 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 smithi027 running (4m) 22s ago 4m 58.3M 4970M 18.2.2-1059-g4d05f4ca 909dce4fe53d 8cbbedf8cac9 2024-04-02T04:42:04.759 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 smithi027 running (3m) 22s ago 3m 52.0M 4970M 18.2.2-1059-g4d05f4ca 909dce4fe53d ac6094415529 2024-04-02T04:42:04.759 INFO:teuthology.orchestra.run.smithi016.stdout:prometheus.smithi016 smithi016 *:9095 running (6m) 2m ago 7m 53.9M - 2.43.0 a07b618ecd1d 232d70bae1ba 2024-04-02T04:42:04.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.753+0000 7f84c8fcc700 1 -- 172.21.15.16:0/778905325 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f84ac06d360 msgr2=0x7f84ac06f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.753+0000 7f84c8fcc700 1 --2- 172.21.15.16:0/778905325 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f84ac06d360 0x7f84ac06f820 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f84b800b6f0 tx=0x7f84b8005fb0 comp rx=0 tx=0).stop 2024-04-02T04:42:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.753+0000 7f84c8fcc700 1 -- 172.21.15.16:0/778905325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c4101560 msgr2=0x7f84c4196490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.753+0000 7f84c8fcc700 1 --2- 172.21.15.16:0/778905325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c4101560 0x7f84c4196490 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f84b400e920 tx=0x7f84b400ec30 comp rx=0 tx=0).stop 2024-04-02T04:42:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.753+0000 7f84c8fcc700 1 -- 172.21.15.16:0/778905325 shutdown_connections 2024-04-02T04:42:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.753+0000 7f84c8fcc700 1 --2- 172.21.15.16:0/778905325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c4101560 0x7f84c4196490 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.753+0000 7f84c8fcc700 1 --2- 172.21.15.16:0/778905325 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f84ac06d360 0x7f84ac06f820 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.753+0000 7f84c8fcc700 1 --2- 172.21.15.16:0/778905325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84c4103e90 0x7f84c41969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.753+0000 7f84c8fcc700 1 -- 172.21.15.16:0/778905325 >> 172.21.15.16:0/778905325 conn(0x7f84c40faf70 msgr2=0x7f84c40fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:42:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.753+0000 7f84c8fcc700 1 -- 172.21.15.16:0/778905325 shutdown_connections 2024-04-02T04:42:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.753+0000 7f84c8fcc700 1 -- 172.21.15.16:0/778905325 wait complete. 2024-04-02T04:42:04.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.872+0000 7fc06fa9e700 1 -- 172.21.15.16:0/690935944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0681015a0 msgr2=0x7fc068103990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:04.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.872+0000 7fc06fa9e700 1 --2- 172.21.15.16:0/690935944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0681015a0 0x7fc068103990 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc064009910 tx=0x7fc064009c20 comp rx=0 tx=0).stop 2024-04-02T04:42:04.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.872+0000 7fc06fa9e700 1 -- 172.21.15.16:0/690935944 shutdown_connections 2024-04-02T04:42:04.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.872+0000 7fc06fa9e700 1 --2- 172.21.15.16:0/690935944 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc068103ed0 0x7fc0681062c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:04.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.873+0000 7fc06fa9e700 1 --2- 172.21.15.16:0/690935944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0681015a0 0x7fc068103990 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:04.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.873+0000 7fc06fa9e700 1 -- 172.21.15.16:0/690935944 >> 172.21.15.16:0/690935944 conn(0x7fc0680faf70 msgr2=0x7fc0680fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:42:04.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.873+0000 7fc06fa9e700 1 -- 172.21.15.16:0/690935944 shutdown_connections 2024-04-02T04:42:04.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.873+0000 7fc06fa9e700 1 -- 172.21.15.16:0/690935944 wait complete. 2024-04-02T04:42:04.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.873+0000 7fc06fa9e700 1 Processor -- start 2024-04-02T04:42:04.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.873+0000 7fc06fa9e700 1 -- start start 2024-04-02T04:42:04.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.874+0000 7fc06fa9e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0681015a0 0x7fc068196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:04.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.874+0000 7fc06fa9e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc068103ed0 0x7fc0681969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:04.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.874+0000 7fc06fa9e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc068196fe0 con 0x7fc068103ed0 2024-04-02T04:42:04.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.874+0000 7fc06fa9e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc068197120 con 0x7fc0681015a0 2024-04-02T04:42:04.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.874+0000 7fc06d039700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc068103ed0 0x7fc0681969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.874+0000 7fc06d039700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc068103ed0 0x7fc0681969c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34520/0 (socket says 172.21.15.16:34520) 2024-04-02T04:42:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.874+0000 7fc06d039700 1 -- 172.21.15.16:0/3835356154 learned_addr learned my addr 172.21.15.16:0/3835356154 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:42:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.874+0000 7fc06d83a700 1 --2- 172.21.15.16:0/3835356154 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0681015a0 0x7fc068196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.874+0000 7fc06d039700 1 -- 172.21.15.16:0/3835356154 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0681015a0 msgr2=0x7fc068196480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.874+0000 7fc06d039700 1 --2- 172.21.15.16:0/3835356154 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0681015a0 0x7fc068196480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.874+0000 7fc06d039700 1 -- 172.21.15.16:0/3835356154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0640095f0 con 0x7fc068103ed0 2024-04-02T04:42:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.874+0000 7fc06d039700 1 --2- 172.21.15.16:0/3835356154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc068103ed0 0x7fc0681969c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc05800e920 tx=0x7fc05800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:42:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.875+0000 7fc05effd700 1 -- 172.21.15.16:0/3835356154 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc05800c9d0 con 0x7fc068103ed0 2024-04-02T04:42:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.875+0000 7fc05effd700 1 -- 172.21.15.16:0/3835356154 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc058010430 con 0x7fc068103ed0 2024-04-02T04:42:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.875+0000 7fc06fa9e700 1 -- 172.21.15.16:0/3835356154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc06819bb80 con 0x7fc068103ed0 2024-04-02T04:42:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.875+0000 7fc05effd700 1 -- 172.21.15.16:0/3835356154 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc058018630 con 0x7fc068103ed0 2024-04-02T04:42:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.875+0000 7fc06fa9e700 1 -- 172.21.15.16:0/3835356154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc06819c150 con 0x7fc068103ed0 2024-04-02T04:42:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.876+0000 7fc06fa9e700 1 -- 172.21.15.16:0/3835356154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc04c005320 con 0x7fc068103ed0 2024-04-02T04:42:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.876+0000 7fc05effd700 1 -- 172.21.15.16:0/3835356154 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 91278+0+0 (secure 0 0 0) 0x7fc05800cb30 con 0x7fc068103ed0 2024-04-02T04:42:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.877+0000 7fc05effd700 1 --2- 172.21.15.16:0/3835356154 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc05406d360 0x7fc05406f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.877+0000 7fc06d83a700 1 --2- 172.21.15.16:0/3835356154 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc05406d360 0x7fc05406f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.877+0000 7fc05effd700 1 -- 172.21.15.16:0/3835356154 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fc058014070 con 0x7fc068103ed0 2024-04-02T04:42:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.877+0000 7fc06d83a700 1 --2- 172.21.15.16:0/3835356154 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc05406d360 0x7fc05406f820 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc06400b6f0 tx=0x7fc064005fb0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:42:04.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:04.882+0000 7fc05effd700 1 -- 172.21.15.16:0/3835356154 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc0580529f0 con 0x7fc068103ed0 2024-04-02T04:42:05.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.217+0000 7fc06fa9e700 1 -- 172.21.15.16:0/3835356154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc04c005de0 con 0x7fc068103ed0 2024-04-02T04:42:05.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.217+0000 7fc05effd700 1 -- 172.21.15.16:0/3835356154 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+841 (secure 0 0 0) 0x7fc05805aa90 con 0x7fc068103ed0 2024-04-02T04:42:05.220 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:42:05.221 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": { 2024-04-02T04:42:05.221 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 2 2024-04-02T04:42:05.221 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:42:05.221 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": { 2024-04-02T04:42:05.221 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 1, 2024-04-02T04:42:05.221 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 1 2024-04-02T04:42:05.221 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:42:05.221 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": { 2024-04-02T04:42:05.221 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 6 2024-04-02T04:42:05.221 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:42:05.221 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-04-02T04:42:05.221 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 4 2024-04-02T04:42:05.222 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:42:05.222 INFO:teuthology.orchestra.run.smithi016.stdout: "overall": { 2024-04-02T04:42:05.222 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 13, 2024-04-02T04:42:05.222 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 1 2024-04-02T04:42:05.222 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-02T04:42:05.222 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:42:05.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.220+0000 7fc06fa9e700 1 -- 172.21.15.16:0/3835356154 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc05406d360 msgr2=0x7fc05406f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:05.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.220+0000 7fc06fa9e700 1 --2- 172.21.15.16:0/3835356154 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc05406d360 0x7fc05406f820 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc06400b6f0 tx=0x7fc064005fb0 comp rx=0 tx=0).stop 2024-04-02T04:42:05.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.220+0000 7fc06fa9e700 1 -- 172.21.15.16:0/3835356154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc068103ed0 msgr2=0x7fc0681969c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:05.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.220+0000 7fc06fa9e700 1 --2- 172.21.15.16:0/3835356154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc068103ed0 0x7fc0681969c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc05800e920 tx=0x7fc05800ec30 comp rx=0 tx=0).stop 2024-04-02T04:42:05.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.220+0000 7fc06fa9e700 1 -- 172.21.15.16:0/3835356154 shutdown_connections 2024-04-02T04:42:05.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.220+0000 7fc06fa9e700 1 --2- 172.21.15.16:0/3835356154 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fc05406d360 0x7fc05406f820 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:05.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.220+0000 7fc06fa9e700 1 --2- 172.21.15.16:0/3835356154 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0681015a0 0x7fc068196480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:05.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.220+0000 7fc06fa9e700 1 --2- 172.21.15.16:0/3835356154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc068103ed0 0x7fc0681969c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:05.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.220+0000 7fc06fa9e700 1 -- 172.21.15.16:0/3835356154 >> 172.21.15.16:0/3835356154 conn(0x7fc0680faf70 msgr2=0x7fc0680fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:42:05.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.220+0000 7fc06fa9e700 1 -- 172.21.15.16:0/3835356154 shutdown_connections 2024-04-02T04:42:05.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.220+0000 7fc06fa9e700 1 -- 172.21.15.16:0/3835356154 wait complete. 2024-04-02T04:42:05.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.344+0000 7fd23469f700 1 -- 172.21.15.16:0/3083741605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd22c0ff9d0 msgr2=0x7fd22c0ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:05.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.344+0000 7fd23469f700 1 --2- 172.21.15.16:0/3083741605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd22c0ff9d0 0x7fd22c0ffdf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd228009910 tx=0x7fd228009c20 comp rx=0 tx=0).stop 2024-04-02T04:42:05.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.344+0000 7fd23469f700 1 -- 172.21.15.16:0/3083741605 shutdown_connections 2024-04-02T04:42:05.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.344+0000 7fd23469f700 1 --2- 172.21.15.16:0/3083741605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd22c100bd0 0x7fd22c101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:05.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.344+0000 7fd23469f700 1 --2- 172.21.15.16:0/3083741605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd22c0ff9d0 0x7fd22c0ffdf0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:05.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.344+0000 7fd23469f700 1 -- 172.21.15.16:0/3083741605 >> 172.21.15.16:0/3083741605 conn(0x7fd22c0faf70 msgr2=0x7fd22c0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:42:05.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.344+0000 7fd23469f700 1 -- 172.21.15.16:0/3083741605 shutdown_connections 2024-04-02T04:42:05.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.344+0000 7fd23469f700 1 -- 172.21.15.16:0/3083741605 wait complete. 2024-04-02T04:42:05.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.345+0000 7fd23469f700 1 Processor -- start 2024-04-02T04:42:05.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.345+0000 7fd23469f700 1 -- start start 2024-04-02T04:42:05.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.345+0000 7fd23469f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd22c0ff9d0 0x7fd22c196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:05.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.345+0000 7fd23469f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd22c100bd0 0x7fd22c196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:05.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.345+0000 7fd23469f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22c1970e0 con 0x7fd22c100bd0 2024-04-02T04:42:05.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.345+0000 7fd23469f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22c197220 con 0x7fd22c0ff9d0 2024-04-02T04:42:05.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.346+0000 7fd23243b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd22c0ff9d0 0x7fd22c196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:05.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.346+0000 7fd23243b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd22c0ff9d0 0x7fd22c196580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.16:52764/0 (socket says 172.21.15.16:52764) 2024-04-02T04:42:05.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.346+0000 7fd23243b700 1 -- 172.21.15.16:0/3079017767 learned_addr learned my addr 172.21.15.16:0/3079017767 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:42:05.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.346+0000 7fd231c3a700 1 --2- 172.21.15.16:0/3079017767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd22c100bd0 0x7fd22c196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:05.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.346+0000 7fd23243b700 1 -- 172.21.15.16:0/3079017767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd22c100bd0 msgr2=0x7fd22c196ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:05.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.346+0000 7fd23243b700 1 --2- 172.21.15.16:0/3079017767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd22c100bd0 0x7fd22c196ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:05.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.346+0000 7fd23243b700 1 -- 172.21.15.16:0/3079017767 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2280095f0 con 0x7fd22c0ff9d0 2024-04-02T04:42:05.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.346+0000 7fd23243b700 1 --2- 172.21.15.16:0/3079017767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd22c0ff9d0 0x7fd22c196580 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd228005f00 tx=0x7fd22800b5e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:42:05.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.346+0000 7fd2237fe700 1 -- 172.21.15.16:0/3079017767 <== mon.1 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd22801d070 con 0x7fd22c0ff9d0 2024-04-02T04:42:05.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.346+0000 7fd2237fe700 1 -- 172.21.15.16:0/3079017767 <== mon.1 v2:172.21.15.27:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd2280052a0 con 0x7fd22c0ff9d0 2024-04-02T04:42:05.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.346+0000 7fd23469f700 1 -- 172.21.15.16:0/3079017767 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd22c19bbd0 con 0x7fd22c0ff9d0 2024-04-02T04:42:05.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.346+0000 7fd2237fe700 1 -- 172.21.15.16:0/3079017767 <== mon.1 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd228017400 con 0x7fd22c0ff9d0 2024-04-02T04:42:05.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.347+0000 7fd23469f700 1 -- 172.21.15.16:0/3079017767 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd22c19c0c0 con 0x7fd22c0ff9d0 2024-04-02T04:42:05.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.347+0000 7fd23469f700 1 -- 172.21.15.16:0/3079017767 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd22c04ea80 con 0x7fd22c0ff9d0 2024-04-02T04:42:05.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.348+0000 7fd2237fe700 1 -- 172.21.15.16:0/3079017767 <== mon.1 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 20) v1 ==== 91278+0+0 (secure 0 0 0) 0x7fd228003680 con 0x7fd22c0ff9d0 2024-04-02T04:42:05.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.349+0000 7fd2237fe700 1 --2- 172.21.15.16:0/3079017767 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fd21806d360 0x7fd21806f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:05.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.349+0000 7fd2237fe700 1 -- 172.21.15.16:0/3079017767 <== mon.1 v2:172.21.15.27:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fd22808df00 con 0x7fd22c0ff9d0 2024-04-02T04:42:05.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.349+0000 7fd231c3a700 1 --2- 172.21.15.16:0/3079017767 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fd21806d360 0x7fd21806f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:05.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.349+0000 7fd231c3a700 1 --2- 172.21.15.16:0/3079017767 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fd21806d360 0x7fd21806f820 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd21c009c60 tx=0x7fd21c009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:42:05.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.356+0000 7fd2237fe700 1 -- 172.21.15.16:0/3079017767 <== mon.1 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd22805bca0 con 0x7fd22c0ff9d0 2024-04-02T04:42:05.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.641+0000 7fd23469f700 1 -- 172.21.15.16:0/3079017767 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd22c19c5d0 con 0x7fd22c0ff9d0 2024-04-02T04:42:05.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.642+0000 7fd2237fe700 1 -- 172.21.15.16:0/3079017767 <== mon.1 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1843 (secure 0 0 0) 0x7fd22805b830 con 0x7fd22c0ff9d0 2024-04-02T04:42:05.645 INFO:teuthology.orchestra.run.smithi016.stdout:e15 2024-04-02T04:42:05.645 INFO:teuthology.orchestra.run.smithi016.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:42:05.645 INFO:teuthology.orchestra.run.smithi016.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:42:05.646 INFO:teuthology.orchestra.run.smithi016.stdout:legacy client fscid: 1 2024-04-02T04:42:05.646 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:42:05.646 INFO:teuthology.orchestra.run.smithi016.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:42:05.646 INFO:teuthology.orchestra.run.smithi016.stdout:fs_name cephfs 2024-04-02T04:42:05.646 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 15 2024-04-02T04:42:05.646 INFO:teuthology.orchestra.run.smithi016.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:42:05.646 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-04-02T04:39:30.740168+0000 2024-04-02T04:42:05.646 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-04-02T04:39:49.875319+0000 2024-04-02T04:42:05.646 INFO:teuthology.orchestra.run.smithi016.stdout:tableserver 0 2024-04-02T04:42:05.646 INFO:teuthology.orchestra.run.smithi016.stdout:root 0 2024-04-02T04:42:05.646 INFO:teuthology.orchestra.run.smithi016.stdout:session_timeout 60 2024-04-02T04:42:05.646 INFO:teuthology.orchestra.run.smithi016.stdout:session_autoclose 300 2024-04-02T04:42:05.646 INFO:teuthology.orchestra.run.smithi016.stdout:max_file_size 1099511627776 2024-04-02T04:42:05.646 INFO:teuthology.orchestra.run.smithi016.stdout:max_xattr_size 65536 2024-04-02T04:42:05.647 INFO:teuthology.orchestra.run.smithi016.stdout:required_client_features {} 2024-04-02T04:42:05.647 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure 0 2024-04-02T04:42:05.647 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure_osd_epoch 46 2024-04-02T04:42:05.647 INFO:teuthology.orchestra.run.smithi016.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:42:05.647 INFO:teuthology.orchestra.run.smithi016.stdout:max_mds 1 2024-04-02T04:42:05.647 INFO:teuthology.orchestra.run.smithi016.stdout:in 0 2024-04-02T04:42:05.647 INFO:teuthology.orchestra.run.smithi016.stdout:up {0=24285} 2024-04-02T04:42:05.647 INFO:teuthology.orchestra.run.smithi016.stdout:failed 2024-04-02T04:42:05.647 INFO:teuthology.orchestra.run.smithi016.stdout:damaged 2024-04-02T04:42:05.647 INFO:teuthology.orchestra.run.smithi016.stdout:stopped 2024-04-02T04:42:05.647 INFO:teuthology.orchestra.run.smithi016.stdout:data_pools [3] 2024-04-02T04:42:05.647 INFO:teuthology.orchestra.run.smithi016.stdout:metadata_pool 2 2024-04-02T04:42:05.647 INFO:teuthology.orchestra.run.smithi016.stdout:inline_data enabled 2024-04-02T04:42:05.647 INFO:teuthology.orchestra.run.smithi016.stdout:balancer 2024-04-02T04:42:05.648 INFO:teuthology.orchestra.run.smithi016.stdout:bal_rank_mask -1 2024-04-02T04:42:05.648 INFO:teuthology.orchestra.run.smithi016.stdout:standby_count_wanted 1 2024-04-02T04:42:05.648 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.zwrfoo{0:24285} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.27:6824/1461455976,v1:172.21.15.27:6825/1461455976] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:42:05.648 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:42:05.648 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:42:05.648 INFO:teuthology.orchestra.run.smithi016.stdout:Standby daemons: 2024-04-02T04:42:05.648 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:42:05.648 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.fyzrnm{-1:14488} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.16:6828/2578719859,v1:172.21.15.16:6829/2578719859] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:42:05.648 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.uhgkgg{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.16:6826/601263579,v1:172.21.15.16:6827/601263579] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:42:05.648 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.sdublg{-1:24295} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.27:6826/4177192740,v1:172.21.15.27:6827/4177192740] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:42:05.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.644+0000 7fd23469f700 1 -- 172.21.15.16:0/3079017767 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fd21806d360 msgr2=0x7fd21806f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:05.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.644+0000 7fd23469f700 1 --2- 172.21.15.16:0/3079017767 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fd21806d360 0x7fd21806f820 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd21c009c60 tx=0x7fd21c009260 comp rx=0 tx=0).stop 2024-04-02T04:42:05.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.644+0000 7fd23469f700 1 -- 172.21.15.16:0/3079017767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd22c0ff9d0 msgr2=0x7fd22c196580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:05.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.644+0000 7fd23469f700 1 --2- 172.21.15.16:0/3079017767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd22c0ff9d0 0x7fd22c196580 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd228005f00 tx=0x7fd22800b5e0 comp rx=0 tx=0).stop 2024-04-02T04:42:05.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.644+0000 7fd23469f700 1 -- 172.21.15.16:0/3079017767 shutdown_connections 2024-04-02T04:42:05.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.644+0000 7fd23469f700 1 --2- 172.21.15.16:0/3079017767 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7fd21806d360 0x7fd21806f820 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:05.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.644+0000 7fd23469f700 1 --2- 172.21.15.16:0/3079017767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd22c0ff9d0 0x7fd22c196580 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:05.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.644+0000 7fd23469f700 1 --2- 172.21.15.16:0/3079017767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd22c100bd0 0x7fd22c196ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:05.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.644+0000 7fd23469f700 1 -- 172.21.15.16:0/3079017767 >> 172.21.15.16:0/3079017767 conn(0x7fd22c0faf70 msgr2=0x7fd22c068530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:42:05.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.645+0000 7fd23469f700 1 -- 172.21.15.16:0/3079017767 shutdown_connections 2024-04-02T04:42:05.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.645+0000 7fd23469f700 1 -- 172.21.15.16:0/3079017767 wait complete. 2024-04-02T04:42:05.650 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 15 2024-04-02T04:42:05.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.759+0000 7f4f4ee22700 1 -- 172.21.15.16:0/635072972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f480ff9d0 msgr2=0x7f4f480ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:05.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.759+0000 7f4f4ee22700 1 --2- 172.21.15.16:0/635072972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f480ff9d0 0x7f4f480ffdf0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c009910 tx=0x7f4f3c009c20 comp rx=0 tx=0).stop 2024-04-02T04:42:05.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.759+0000 7f4f4ee22700 1 -- 172.21.15.16:0/635072972 shutdown_connections 2024-04-02T04:42:05.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.759+0000 7f4f4ee22700 1 --2- 172.21.15.16:0/635072972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f48100bd0 0x7f4f48101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:05.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.759+0000 7f4f4ee22700 1 --2- 172.21.15.16:0/635072972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f480ff9d0 0x7f4f480ffdf0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:05.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.759+0000 7f4f4ee22700 1 -- 172.21.15.16:0/635072972 >> 172.21.15.16:0/635072972 conn(0x7f4f480faf70 msgr2=0x7f4f480fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:42:05.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.759+0000 7f4f4ee22700 1 -- 172.21.15.16:0/635072972 shutdown_connections 2024-04-02T04:42:05.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.759+0000 7f4f4ee22700 1 -- 172.21.15.16:0/635072972 wait complete. 2024-04-02T04:42:05.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.760+0000 7f4f4ee22700 1 Processor -- start 2024-04-02T04:42:05.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.760+0000 7f4f4ee22700 1 -- start start 2024-04-02T04:42:05.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.760+0000 7f4f4ee22700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f480ff9d0 0x7f4f48195440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:05.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.760+0000 7f4f4ee22700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f48100bd0 0x7f4f48195980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:05.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.760+0000 7f4f4ee22700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f48195fa0 con 0x7f4f48100bd0 2024-04-02T04:42:05.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.760+0000 7f4f4ee22700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f481960e0 con 0x7f4f480ff9d0 2024-04-02T04:42:05.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.760+0000 7f4f47fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f48100bd0 0x7f4f48195980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:05.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.760+0000 7f4f47fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f48100bd0 0x7f4f48195980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34544/0 (socket says 172.21.15.16:34544) 2024-04-02T04:42:05.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.760+0000 7f4f4cbbe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f480ff9d0 0x7f4f48195440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:05.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.760+0000 7f4f47fff700 1 -- 172.21.15.16:0/1819757685 learned_addr learned my addr 172.21.15.16:0/1819757685 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:42:05.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.760+0000 7f4f47fff700 1 -- 172.21.15.16:0/1819757685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f480ff9d0 msgr2=0x7f4f48195440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:05.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.760+0000 7f4f47fff700 1 --2- 172.21.15.16:0/1819757685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f480ff9d0 0x7f4f48195440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:05.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.760+0000 7f4f47fff700 1 -- 172.21.15.16:0/1819757685 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f3c0095f0 con 0x7f4f48100bd0 2024-04-02T04:42:05.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.761+0000 7f4f47fff700 1 --2- 172.21.15.16:0/1819757685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f48100bd0 0x7f4f48195980 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4f3400e920 tx=0x7f4f3400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:42:05.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.761+0000 7f4f45ffb700 1 -- 172.21.15.16:0/1819757685 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f3400c9d0 con 0x7f4f48100bd0 2024-04-02T04:42:05.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.761+0000 7f4f45ffb700 1 -- 172.21.15.16:0/1819757685 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f34010430 con 0x7f4f48100bd0 2024-04-02T04:42:05.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.761+0000 7f4f45ffb700 1 -- 172.21.15.16:0/1819757685 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f34018630 con 0x7f4f48100bd0 2024-04-02T04:42:05.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.761+0000 7f4f4ee22700 1 -- 172.21.15.16:0/1819757685 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f4819aaf0 con 0x7f4f48100bd0 2024-04-02T04:42:05.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.761+0000 7f4f4ee22700 1 -- 172.21.15.16:0/1819757685 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f4819b070 con 0x7f4f48100bd0 2024-04-02T04:42:05.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.762+0000 7f4f4ee22700 1 -- 172.21.15.16:0/1819757685 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f4818f550 con 0x7f4f48100bd0 2024-04-02T04:42:05.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.763+0000 7f4f45ffb700 1 -- 172.21.15.16:0/1819757685 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 91278+0+0 (secure 0 0 0) 0x7f4f340105a0 con 0x7f4f48100bd0 2024-04-02T04:42:05.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.763+0000 7f4f45ffb700 1 --2- 172.21.15.16:0/1819757685 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f4f3806d360 0x7f4f3806f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:05.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.763+0000 7f4f45ffb700 1 -- 172.21.15.16:0/1819757685 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f4f34014070 con 0x7f4f48100bd0 2024-04-02T04:42:05.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.763+0000 7f4f4cbbe700 1 --2- 172.21.15.16:0/1819757685 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f4f3806d360 0x7f4f3806f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:05.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.763+0000 7f4f4cbbe700 1 --2- 172.21.15.16:0/1819757685 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f4f3806d360 0x7f4f3806f820 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c006010 tx=0x7f4f3c009f70 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:42:05.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:05.768+0000 7f4f45ffb700 1 -- 172.21.15.16:0/1819757685 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4f340529f0 con 0x7f4f48100bd0 2024-04-02T04:42:06.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.008+0000 7f4f4ee22700 1 -- 172.21.15.16:0/1819757685 --> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f4802cdc0 con 0x7f4f3806d360 2024-04-02T04:42:06.012 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:05 smithi016 ceph-mon[28969]: from='client.14660 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:42:06.012 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:05 smithi016 ceph-mon[28969]: from='client.24423 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:42:06.012 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:05 smithi016 ceph-mon[28969]: from='client.14668 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:42:06.012 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:05 smithi016 ceph-mon[28969]: pgmap v255: 65 pgs: 65 active+clean; 116 MiB data, 544 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.1 MiB/s wr, 161 op/s 2024-04-02T04:42:06.012 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:05 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/3835356154' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:42:06.012 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:05 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/3079017767' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:42:06.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.009+0000 7f4f45ffb700 1 -- 172.21.15.16:0/1819757685 <== mgr.14217 v2:172.21.15.16:6800/3643600900 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f4f4802cdc0 con 0x7f4f3806d360 2024-04-02T04:42:06.013 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:42:06.013 INFO:teuthology.orchestra.run.smithi016.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:42:06.013 INFO:teuthology.orchestra.run.smithi016.stdout: "in_progress": true, 2024-04-02T04:42:06.013 INFO:teuthology.orchestra.run.smithi016.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-02T04:42:06.013 INFO:teuthology.orchestra.run.smithi016.stdout: "services_complete": [], 2024-04-02T04:42:06.013 INFO:teuthology.orchestra.run.smithi016.stdout: "progress": "1/23 daemons upgraded", 2024-04-02T04:42:06.013 INFO:teuthology.orchestra.run.smithi016.stdout: "message": "Currently upgrading mgr daemons", 2024-04-02T04:42:06.013 INFO:teuthology.orchestra.run.smithi016.stdout: "is_paused": false 2024-04-02T04:42:06.013 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:42:06.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.011+0000 7f4f4ee22700 1 -- 172.21.15.16:0/1819757685 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f4f3806d360 msgr2=0x7f4f3806f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:06.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.011+0000 7f4f4ee22700 1 --2- 172.21.15.16:0/1819757685 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f4f3806d360 0x7f4f3806f820 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c006010 tx=0x7f4f3c009f70 comp rx=0 tx=0).stop 2024-04-02T04:42:06.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.011+0000 7f4f4ee22700 1 -- 172.21.15.16:0/1819757685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f48100bd0 msgr2=0x7f4f48195980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:06.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.011+0000 7f4f4ee22700 1 --2- 172.21.15.16:0/1819757685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f48100bd0 0x7f4f48195980 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4f3400e920 tx=0x7f4f3400ec30 comp rx=0 tx=0).stop 2024-04-02T04:42:06.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.012+0000 7f4f4ee22700 1 -- 172.21.15.16:0/1819757685 shutdown_connections 2024-04-02T04:42:06.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.012+0000 7f4f4ee22700 1 --2- 172.21.15.16:0/1819757685 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7f4f3806d360 0x7f4f3806f820 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:06.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.012+0000 7f4f4ee22700 1 --2- 172.21.15.16:0/1819757685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f480ff9d0 0x7f4f48195440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:06.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.012+0000 7f4f4ee22700 1 --2- 172.21.15.16:0/1819757685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f48100bd0 0x7f4f48195980 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:06.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.012+0000 7f4f4ee22700 1 -- 172.21.15.16:0/1819757685 >> 172.21.15.16:0/1819757685 conn(0x7f4f480faf70 msgr2=0x7f4f48104220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:42:06.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.012+0000 7f4f4ee22700 1 -- 172.21.15.16:0/1819757685 shutdown_connections 2024-04-02T04:42:06.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.012+0000 7f4f4ee22700 1 -- 172.21.15.16:0/1819757685 wait complete. 2024-04-02T04:42:06.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.120+0000 7ffb9011e700 1 -- 172.21.15.16:0/3853950439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb88101560 msgr2=0x7ffb88103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:06.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.120+0000 7ffb9011e700 1 --2- 172.21.15.16:0/3853950439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb88101560 0x7ffb88103950 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ffb7800b0a0 tx=0x7ffb7800b3b0 comp rx=0 tx=0).stop 2024-04-02T04:42:06.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.120+0000 7ffb9011e700 1 -- 172.21.15.16:0/3853950439 shutdown_connections 2024-04-02T04:42:06.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.120+0000 7ffb9011e700 1 --2- 172.21.15.16:0/3853950439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb88103e90 0x7ffb88106280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:06.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.120+0000 7ffb9011e700 1 --2- 172.21.15.16:0/3853950439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb88101560 0x7ffb88103950 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:06.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.120+0000 7ffb9011e700 1 -- 172.21.15.16:0/3853950439 >> 172.21.15.16:0/3853950439 conn(0x7ffb880faf70 msgr2=0x7ffb880fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:42:06.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.120+0000 7ffb9011e700 1 -- 172.21.15.16:0/3853950439 shutdown_connections 2024-04-02T04:42:06.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.120+0000 7ffb9011e700 1 -- 172.21.15.16:0/3853950439 wait complete. 2024-04-02T04:42:06.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.121+0000 7ffb9011e700 1 Processor -- start 2024-04-02T04:42:06.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.121+0000 7ffb9011e700 1 -- start start 2024-04-02T04:42:06.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.121+0000 7ffb9011e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb88101560 0x7ffb881964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:06.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.121+0000 7ffb9011e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb88103e90 0x7ffb88196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:06.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.121+0000 7ffb9011e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb88197050 con 0x7ffb88103e90 2024-04-02T04:42:06.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.121+0000 7ffb9011e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb88197190 con 0x7ffb88101560 2024-04-02T04:42:06.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.121+0000 7ffb8d6b9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb88103e90 0x7ffb88196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:06.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.121+0000 7ffb8d6b9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb88103e90 0x7ffb88196a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34566/0 (socket says 172.21.15.16:34566) 2024-04-02T04:42:06.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.121+0000 7ffb8d6b9700 1 -- 172.21.15.16:0/2678575465 learned_addr learned my addr 172.21.15.16:0/2678575465 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:42:06.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.121+0000 7ffb8deba700 1 --2- 172.21.15.16:0/2678575465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb88101560 0x7ffb881964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:06.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.122+0000 7ffb8d6b9700 1 -- 172.21.15.16:0/2678575465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb88101560 msgr2=0x7ffb881964f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:06.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.122+0000 7ffb8d6b9700 1 --2- 172.21.15.16:0/2678575465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb88101560 0x7ffb881964f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:06.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.122+0000 7ffb8d6b9700 1 -- 172.21.15.16:0/2678575465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb78009d00 con 0x7ffb88103e90 2024-04-02T04:42:06.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.122+0000 7ffb8deba700 1 --2- 172.21.15.16:0/2678575465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb88101560 0x7ffb881964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T04:42:06.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.122+0000 7ffb8d6b9700 1 --2- 172.21.15.16:0/2678575465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb88103e90 0x7ffb88196a30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c00d8d0 tx=0x7ffb7c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:42:06.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.122+0000 7ffb86ffd700 1 -- 172.21.15.16:0/2678575465 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb7c0098a0 con 0x7ffb88103e90 2024-04-02T04:42:06.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.122+0000 7ffb9011e700 1 -- 172.21.15.16:0/2678575465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb8819bba0 con 0x7ffb88103e90 2024-04-02T04:42:06.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.122+0000 7ffb86ffd700 1 -- 172.21.15.16:0/2678575465 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffb7c010460 con 0x7ffb88103e90 2024-04-02T04:42:06.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.122+0000 7ffb86ffd700 1 -- 172.21.15.16:0/2678575465 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb7c00f5f0 con 0x7ffb88103e90 2024-04-02T04:42:06.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.122+0000 7ffb9011e700 1 -- 172.21.15.16:0/2678575465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb8819c170 con 0x7ffb88103e90 2024-04-02T04:42:06.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.123+0000 7ffb9011e700 1 -- 172.21.15.16:0/2678575465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb8804ea80 con 0x7ffb88103e90 2024-04-02T04:42:06.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.124+0000 7ffb86ffd700 1 -- 172.21.15.16:0/2678575465 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 91278+0+0 (secure 0 0 0) 0x7ffb7c009a00 con 0x7ffb88103e90 2024-04-02T04:42:06.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.124+0000 7ffb86ffd700 1 --2- 172.21.15.16:0/2678575465 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7ffb7006d370 0x7ffb7006f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:06.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.124+0000 7ffb86ffd700 1 -- 172.21.15.16:0/2678575465 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7ffb7c08c9f0 con 0x7ffb88103e90 2024-04-02T04:42:06.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.124+0000 7ffb8deba700 1 --2- 172.21.15.16:0/2678575465 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7ffb7006d370 0x7ffb7006f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:06.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.125+0000 7ffb8deba700 1 --2- 172.21.15.16:0/2678575465 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7ffb7006d370 0x7ffb7006f830 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ffb78000f80 tx=0x7ffb78007910 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:42:06.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.129+0000 7ffb86ffd700 1 -- 172.21.15.16:0/2678575465 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ffb7c08ee60 con 0x7ffb88103e90 2024-04-02T04:42:06.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:05 smithi027 ceph-mon[37550]: from='client.14660 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:42:06.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:05 smithi027 ceph-mon[37550]: from='client.24423 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:42:06.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:05 smithi027 ceph-mon[37550]: from='client.14668 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:42:06.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:05 smithi027 ceph-mon[37550]: pgmap v255: 65 pgs: 65 active+clean; 116 MiB data, 544 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.1 MiB/s wr, 161 op/s 2024-04-02T04:42:06.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:05 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/3835356154' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:42:06.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:05 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/3079017767' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:42:06.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.455+0000 7ffb9011e700 1 -- 172.21.15.16:0/2678575465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ffb88109830 con 0x7ffb88103e90 2024-04-02T04:42:06.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.456+0000 7ffb86ffd700 1 -- 172.21.15.16:0/2678575465 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7ffb7c08ee60 con 0x7ffb88103e90 2024-04-02T04:42:06.459 INFO:teuthology.orchestra.run.smithi016.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-02T04:42:06.459 INFO:teuthology.orchestra.run.smithi016.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-02T04:42:06.459 INFO:teuthology.orchestra.run.smithi016.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-02T04:42:06.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.458+0000 7ffb9011e700 1 -- 172.21.15.16:0/2678575465 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7ffb7006d370 msgr2=0x7ffb7006f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.458+0000 7ffb9011e700 1 --2- 172.21.15.16:0/2678575465 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7ffb7006d370 0x7ffb7006f830 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ffb78000f80 tx=0x7ffb78007910 comp rx=0 tx=0).stop 2024-04-02T04:42:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.458+0000 7ffb9011e700 1 -- 172.21.15.16:0/2678575465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb88103e90 msgr2=0x7ffb88196a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.458+0000 7ffb9011e700 1 --2- 172.21.15.16:0/2678575465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb88103e90 0x7ffb88196a30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c00d8d0 tx=0x7ffb7c00dbe0 comp rx=0 tx=0).stop 2024-04-02T04:42:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.458+0000 7ffb9011e700 1 -- 172.21.15.16:0/2678575465 shutdown_connections 2024-04-02T04:42:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.458+0000 7ffb9011e700 1 --2- 172.21.15.16:0/2678575465 >> [v2:172.21.15.16:6800/3643600900,v1:172.21.15.16:6801/3643600900] conn(0x7ffb7006d370 0x7ffb7006f830 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.458+0000 7ffb9011e700 1 --2- 172.21.15.16:0/2678575465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb88101560 0x7ffb881964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.458+0000 7ffb9011e700 1 --2- 172.21.15.16:0/2678575465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb88103e90 0x7ffb88196a30 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.458+0000 7ffb9011e700 1 -- 172.21.15.16:0/2678575465 >> 172.21.15.16:0/2678575465 conn(0x7ffb880faf70 msgr2=0x7ffb880fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:42:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.459+0000 7ffb9011e700 1 -- 172.21.15.16:0/2678575465 shutdown_connections 2024-04-02T04:42:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:06.459+0000 7ffb9011e700 1 -- 172.21.15.16:0/2678575465 wait complete. 2024-04-02T04:42:07.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:06 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/2678575465' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:42:07.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:06 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/2678575465' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:42:07.984 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:07 smithi027 ceph-mon[37550]: from='client.14680 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:42:07.984 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:07 smithi027 ceph-mon[37550]: pgmap v256: 65 pgs: 65 active+clean; 121 MiB data, 553 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.8 MiB/s wr, 130 op/s 2024-04-02T04:42:07.984 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:07 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:42:07.984 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:07 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:42:07.984 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:07 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:42:07.984 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:07 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:42:07.985 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:07 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:42:08.113 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:07 smithi016 ceph-mon[28969]: from='client.14680 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:42:08.113 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:07 smithi016 ceph-mon[28969]: pgmap v256: 65 pgs: 65 active+clean; 121 MiB data, 553 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.8 MiB/s wr, 130 op/s 2024-04-02T04:42:08.113 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:07 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:42:08.113 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:07 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:42:08.113 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:07 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:42:08.113 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:07 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:42:08.113 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:07 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' 2024-04-02T04:42:09.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:08 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:42:09.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:08 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mgr fail", "who": "smithi016.eqnebx"}]: dispatch 2024-04-02T04:42:09.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:08 smithi027 ceph-mon[37550]: osdmap e47: 6 total, 6 up, 6 in 2024-04-02T04:42:09.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:08 smithi027 ceph-mon[37550]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "mgr fail", "who": "smithi016.eqnebx"}]': finished 2024-04-02T04:42:09.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:08 smithi027 ceph-mon[37550]: mgrmap e21: smithi027.kjzcsc(active, starting, since 0.00663543s) 2024-04-02T04:42:09.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:08 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "mon metadata", "id": "smithi016"}]: dispatch 2024-04-02T04:42:09.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:08 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2024-04-02T04:42:09.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:08 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi016.fyzrnm"}]: dispatch 2024-04-02T04:42:09.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:08 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi016.uhgkgg"}]: dispatch 2024-04-02T04:42:09.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:08 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.zwrfoo"}]: dispatch 2024-04-02T04:42:09.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:08 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.sdublg"}]: dispatch 2024-04-02T04:42:09.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:08 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "mgr metadata", "who": "smithi027.kjzcsc", "id": "smithi027.kjzcsc"}]: dispatch 2024-04-02T04:42:09.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:08 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T04:42:09.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:08 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T04:42:09.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:08 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T04:42:09.148 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:08 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T04:42:09.148 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:08 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T04:42:09.148 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:08 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T04:42:09.148 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:08 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T04:42:09.148 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:08 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T04:42:09.148 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:08 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T04:42:09.148 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:08 smithi027 ceph-mon[37550]: Manager daemon smithi027.kjzcsc is now available 2024-04-02T04:42:09.148 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:08 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:42:09.148 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:08 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:42:09.148 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:08 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.kjzcsc/mirror_snapshot_schedule"}]: dispatch 2024-04-02T04:42:09.148 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:08 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.kjzcsc/trash_purge_schedule"}]: dispatch 2024-04-02T04:42:09.148 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:08 smithi027 ceph-mon[37550]: Deploying cephadm binary to smithi016 2024-04-02T04:42:09.148 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:08 smithi027 ceph-mon[37550]: Deploying cephadm binary to smithi027 2024-04-02T04:42:09.148 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:08 smithi027 ceph-mon[37550]: [02/Apr/2024:04:42:08] ENGINE Bus STARTING 2024-04-02T04:42:09.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:08 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:42:09.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:08 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mgr fail", "who": "smithi016.eqnebx"}]: dispatch 2024-04-02T04:42:09.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:08 smithi016 ceph-mon[28969]: osdmap e47: 6 total, 6 up, 6 in 2024-04-02T04:42:09.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:08 smithi016 ceph-mon[28969]: from='mgr.14217 172.21.15.16:0/1148542194' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "mgr fail", "who": "smithi016.eqnebx"}]': finished 2024-04-02T04:42:09.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:08 smithi016 ceph-mon[28969]: mgrmap e21: smithi027.kjzcsc(active, starting, since 0.00663543s) 2024-04-02T04:42:09.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:08 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "mon metadata", "id": "smithi016"}]: dispatch 2024-04-02T04:42:09.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:08 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2024-04-02T04:42:09.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:08 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi016.fyzrnm"}]: dispatch 2024-04-02T04:42:09.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:08 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi016.uhgkgg"}]: dispatch 2024-04-02T04:42:09.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:08 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.zwrfoo"}]: dispatch 2024-04-02T04:42:09.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:08 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.sdublg"}]: dispatch 2024-04-02T04:42:09.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:08 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "mgr metadata", "who": "smithi027.kjzcsc", "id": "smithi027.kjzcsc"}]: dispatch 2024-04-02T04:42:09.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:08 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T04:42:09.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:08 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T04:42:09.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:08 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T04:42:09.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:08 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T04:42:09.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:08 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T04:42:09.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:08 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T04:42:09.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:08 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T04:42:09.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:08 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T04:42:09.312 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:08 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T04:42:09.312 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:08 smithi016 ceph-mon[28969]: Manager daemon smithi027.kjzcsc is now available 2024-04-02T04:42:09.312 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:08 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:42:09.312 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:08 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:42:09.312 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:08 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.kjzcsc/mirror_snapshot_schedule"}]: dispatch 2024-04-02T04:42:09.312 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:08 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.kjzcsc/trash_purge_schedule"}]: dispatch 2024-04-02T04:42:09.312 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:08 smithi016 ceph-mon[28969]: Deploying cephadm binary to smithi016 2024-04-02T04:42:09.312 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:08 smithi016 ceph-mon[28969]: Deploying cephadm binary to smithi027 2024-04-02T04:42:09.312 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:08 smithi016 ceph-mon[28969]: [02/Apr/2024:04:42:08] ENGINE Bus STARTING 2024-04-02T04:42:10.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:09 smithi016 ceph-mon[28969]: [02/Apr/2024:04:42:08] ENGINE Serving on http://172.21.15.27:8765 2024-04-02T04:42:10.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:09 smithi016 ceph-mon[28969]: [02/Apr/2024:04:42:08] ENGINE Serving on https://172.21.15.27:7150 2024-04-02T04:42:10.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:09 smithi016 ceph-mon[28969]: [02/Apr/2024:04:42:08] ENGINE Bus STARTED 2024-04-02T04:42:10.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:09 smithi016 ceph-mon[28969]: mgrmap e22: smithi027.kjzcsc(active, since 1.00938s) 2024-04-02T04:42:10.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:09 smithi016 ceph-mon[28969]: pgmap v3: 65 pgs: 65 active+clean; 134 MiB data, 601 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:42:10.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:09 smithi027 ceph-mon[37550]: [02/Apr/2024:04:42:08] ENGINE Serving on http://172.21.15.27:8765 2024-04-02T04:42:10.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:09 smithi027 ceph-mon[37550]: [02/Apr/2024:04:42:08] ENGINE Serving on https://172.21.15.27:7150 2024-04-02T04:42:10.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:09 smithi027 ceph-mon[37550]: [02/Apr/2024:04:42:08] ENGINE Bus STARTED 2024-04-02T04:42:10.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:09 smithi027 ceph-mon[37550]: mgrmap e22: smithi027.kjzcsc(active, since 1.00938s) 2024-04-02T04:42:10.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:09 smithi027 ceph-mon[37550]: pgmap v3: 65 pgs: 65 active+clean; 134 MiB data, 601 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:42:11.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:10 smithi027 ceph-mon[37550]: mgrmap e23: smithi027.kjzcsc(active, since 2s) 2024-04-02T04:42:11.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:10 smithi027 ceph-mon[37550]: pgmap v4: 65 pgs: 65 active+clean; 134 MiB data, 601 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:42:11.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:10 smithi016 ceph-mon[28969]: mgrmap e23: smithi027.kjzcsc(active, since 2s) 2024-04-02T04:42:11.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:10 smithi016 ceph-mon[28969]: pgmap v4: 65 pgs: 65 active+clean; 134 MiB data, 601 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:42:13.234 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:12 smithi016 ceph-mon[28969]: pgmap v5: 65 pgs: 65 active+clean; 134 MiB data, 601 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:42:13.234 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:12 smithi016 ceph-mon[28969]: mgrmap e24: smithi027.kjzcsc(active, since 4s) 2024-04-02T04:42:13.395 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:12 smithi027 ceph-mon[37550]: pgmap v5: 65 pgs: 65 active+clean; 134 MiB data, 601 MiB used, 536 GiB / 536 GiB avail 2024-04-02T04:42:13.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:12 smithi027 ceph-mon[37550]: mgrmap e24: smithi027.kjzcsc(active, since 4s) 2024-04-02T04:42:15.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:14 smithi016 ceph-mon[28969]: pgmap v6: 65 pgs: 65 active+clean; 140 MiB data, 619 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.1 MiB/s wr, 56 op/s 2024-04-02T04:42:15.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:14 smithi027 ceph-mon[37550]: pgmap v6: 65 pgs: 65 active+clean; 140 MiB data, 619 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.1 MiB/s wr, 56 op/s 2024-04-02T04:42:16.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:15 smithi016 ceph-mon[28969]: Standby manager daemon smithi016.eqnebx started 2024-04-02T04:42:16.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:15 smithi016 ceph-mon[28969]: from='mgr.? 172.21.15.16:0/1178518492' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi016.eqnebx/crt"}]: dispatch 2024-04-02T04:42:16.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:15 smithi016 ceph-mon[28969]: from='mgr.? 172.21.15.16:0/1178518492' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T04:42:16.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:15 smithi016 ceph-mon[28969]: from='mgr.? 172.21.15.16:0/1178518492' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi016.eqnebx/key"}]: dispatch 2024-04-02T04:42:16.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:15 smithi016 ceph-mon[28969]: from='mgr.? 172.21.15.16:0/1178518492' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T04:42:16.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:15 smithi027 ceph-mon[37550]: Standby manager daemon smithi016.eqnebx started 2024-04-02T04:42:16.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:15 smithi027 ceph-mon[37550]: from='mgr.? 172.21.15.16:0/1178518492' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi016.eqnebx/crt"}]: dispatch 2024-04-02T04:42:16.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:15 smithi027 ceph-mon[37550]: from='mgr.? 172.21.15.16:0/1178518492' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T04:42:16.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:15 smithi027 ceph-mon[37550]: from='mgr.? 172.21.15.16:0/1178518492' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi016.eqnebx/key"}]: dispatch 2024-04-02T04:42:16.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:15 smithi027 ceph-mon[37550]: from='mgr.? 172.21.15.16:0/1178518492' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T04:42:17.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:16 smithi027 ceph-mon[37550]: mgrmap e25: smithi027.kjzcsc(active, since 7s), standbys: smithi016.eqnebx 2024-04-02T04:42:17.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:16 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "mgr metadata", "who": "smithi016.eqnebx", "id": "smithi016.eqnebx"}]: dispatch 2024-04-02T04:42:17.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:16 smithi027 ceph-mon[37550]: pgmap v7: 65 pgs: 65 active+clean; 152 MiB data, 664 MiB used, 536 GiB / 536 GiB avail; 29 KiB/s rd, 2.6 MiB/s wr, 180 op/s 2024-04-02T04:42:17.157 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:16 smithi016 ceph-mon[28969]: mgrmap e25: smithi027.kjzcsc(active, since 7s), standbys: smithi016.eqnebx 2024-04-02T04:42:17.157 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:16 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "mgr metadata", "who": "smithi016.eqnebx", "id": "smithi016.eqnebx"}]: dispatch 2024-04-02T04:42:17.157 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:16 smithi016 ceph-mon[28969]: pgmap v7: 65 pgs: 65 active+clean; 152 MiB data, 664 MiB used, 536 GiB / 536 GiB avail; 29 KiB/s rd, 2.6 MiB/s wr, 180 op/s 2024-04-02T04:42:19.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:19 smithi016 ceph-mon[28969]: pgmap v8: 65 pgs: 65 active+clean; 152 MiB data, 664 MiB used, 536 GiB / 536 GiB avail; 23 KiB/s rd, 2.0 MiB/s wr, 140 op/s 2024-04-02T04:42:19.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:19 smithi027 ceph-mon[37550]: pgmap v8: 65 pgs: 65 active+clean; 152 MiB data, 664 MiB used, 536 GiB / 536 GiB avail; 23 KiB/s rd, 2.0 MiB/s wr, 140 op/s 2024-04-02T04:42:20.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:20 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:20.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:20 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:20.645 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:20 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:20.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:20 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:21.448 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:21 smithi027 ceph-mon[37550]: pgmap v9: 65 pgs: 65 active+clean; 170 MiB data, 730 MiB used, 536 GiB / 536 GiB avail; 19 KiB/s rd, 3.4 MiB/s wr, 278 op/s 2024-04-02T04:42:21.448 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:21 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:21.448 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:21 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:21.448 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:21 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:21.448 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:21 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:21.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:21 smithi016 ceph-mon[28969]: pgmap v9: 65 pgs: 65 active+clean; 170 MiB data, 730 MiB used, 536 GiB / 536 GiB avail; 19 KiB/s rd, 3.4 MiB/s wr, 278 op/s 2024-04-02T04:42:21.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:21 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:21.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:21 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:21.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:21 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:21.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:21 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:22.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:22 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:22.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:22 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:22.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:22 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:42:22.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:22 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:42:22.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:22 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:42:22.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:22 smithi027 ceph-mon[37550]: Adjusting osd_memory_target on smithi016 to 1217M 2024-04-02T04:42:22.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:22 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:23.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:22 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:23.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:22 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:23.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:22 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:42:23.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:22 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:42:23.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:22 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:42:23.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:22 smithi016 ceph-mon[28969]: Adjusting osd_memory_target on smithi016 to 1217M 2024-04-02T04:42:23.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:22 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:23.809 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:23 smithi016 ceph-mon[28969]: pgmap v10: 65 pgs: 65 active+clean; 170 MiB data, 730 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 3.1 MiB/s wr, 255 op/s 2024-04-02T04:42:23.809 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:23 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:42:23.895 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:23 smithi027 ceph-mon[37550]: pgmap v10: 65 pgs: 65 active+clean; 170 MiB data, 730 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 3.1 MiB/s wr, 255 op/s 2024-04-02T04:42:23.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:23 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:42:25.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:24 smithi016 ceph-mon[28969]: pgmap v11: 65 pgs: 65 active+clean; 173 MiB data, 749 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 3.4 MiB/s wr, 289 op/s 2024-04-02T04:42:25.140 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:24 smithi027 ceph-mon[37550]: pgmap v11: 65 pgs: 65 active+clean; 173 MiB data, 749 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 3.4 MiB/s wr, 289 op/s 2024-04-02T04:42:27.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:27 smithi016 ceph-mon[28969]: pgmap v12: 65 pgs: 65 active+clean; 189 MiB data, 800 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 4.2 MiB/s wr, 353 op/s 2024-04-02T04:42:27.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:27 smithi027 ceph-mon[37550]: pgmap v12: 65 pgs: 65 active+clean; 189 MiB data, 800 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 4.2 MiB/s wr, 353 op/s 2024-04-02T04:42:27.471 INFO:tasks.workunit.client.0.smithi016.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-04-02T04:42:27.476 INFO:tasks.workunit.client.0.smithi016.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-02T04:42:27.476 INFO:tasks.workunit.client.0.smithi016.stderr:+ make 2024-04-02T04:42:27.987 INFO:tasks.workunit.client.0.smithi016.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-04-02T04:42:28.398 INFO:tasks.workunit.client.0.smithi016.stderr:++ readlink -f fsstress 2024-04-02T04:42:28.400 INFO:tasks.workunit.client.0.smithi016.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-04-02T04:42:28.400 INFO:tasks.workunit.client.0.smithi016.stderr:+ popd 2024-04-02T04:42:28.401 INFO:tasks.workunit.client.0.smithi016.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-02T04:42:28.401 INFO:tasks.workunit.client.0.smithi016.stderr:+ popd 2024-04-02T04:42:28.401 INFO:tasks.workunit.client.0.smithi016.stdout:~/cephtest/mnt.0/client.0/tmp 2024-04-02T04:42:28.402 INFO:tasks.workunit.client.0.smithi016.stderr:++ mktemp -d -p . 2024-04-02T04:42:28.648 INFO:tasks.workunit.client.0.smithi016.stderr:+ T=./tmp.4MHuDQnisZ 2024-04-02T04:42:28.648 INFO:tasks.workunit.client.0.smithi016.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.4MHuDQnisZ -l 1 -n 1000 -p 10 -v 2024-04-02T04:42:28.652 INFO:tasks.workunit.client.0.smithi016.stdout:seed = 1711432380 2024-04-02T04:42:28.661 INFO:tasks.workunit.client.0.smithi016.stdout:1/0: rmdir - no directory 2024-04-02T04:42:28.661 INFO:tasks.workunit.client.0.smithi016.stdout:1/1: chown . 3404997 1 2024-04-02T04:42:28.661 INFO:tasks.workunit.client.0.smithi016.stdout:1/2: write - no filename 2024-04-02T04:42:28.661 INFO:tasks.workunit.client.0.smithi016.stdout:1/3: dread - no filename 2024-04-02T04:42:28.662 INFO:tasks.workunit.client.0.smithi016.stdout:0/0: mkdir d0 0 2024-04-02T04:42:28.663 INFO:tasks.workunit.client.0.smithi016.stdout:2/0: readlink - no filename 2024-04-02T04:42:28.663 INFO:tasks.workunit.client.0.smithi016.stdout:2/1: read - no filename 2024-04-02T04:42:28.663 INFO:tasks.workunit.client.0.smithi016.stdout:2/2: dwrite - no filename 2024-04-02T04:42:28.664 INFO:tasks.workunit.client.0.smithi016.stdout:1/4: mknod c0 0 2024-04-02T04:42:28.664 INFO:tasks.workunit.client.0.smithi016.stdout:1/5: dread - no filename 2024-04-02T04:42:28.666 INFO:tasks.workunit.client.0.smithi016.stdout:3/0: write - no filename 2024-04-02T04:42:28.666 INFO:tasks.workunit.client.0.smithi016.stdout:3/1: dwrite - no filename 2024-04-02T04:42:28.667 INFO:tasks.workunit.client.0.smithi016.stdout:3/2: dwrite - no filename 2024-04-02T04:42:28.667 INFO:tasks.workunit.client.0.smithi016.stdout:3/3: dwrite - no filename 2024-04-02T04:42:28.667 INFO:tasks.workunit.client.0.smithi016.stdout:3/4: fdatasync - no filename 2024-04-02T04:42:28.667 INFO:tasks.workunit.client.0.smithi016.stdout:3/5: write - no filename 2024-04-02T04:42:28.667 INFO:tasks.workunit.client.0.smithi016.stdout:3/6: fdatasync - no filename 2024-04-02T04:42:28.667 INFO:tasks.workunit.client.0.smithi016.stdout:0/1: symlink d0/l1 0 2024-04-02T04:42:28.667 INFO:tasks.workunit.client.0.smithi016.stdout:0/2: dwrite - no filename 2024-04-02T04:42:28.667 INFO:tasks.workunit.client.0.smithi016.stdout:0/3: dwrite - no filename 2024-04-02T04:42:28.667 INFO:tasks.workunit.client.0.smithi016.stdout:0/4: rename d0 to d0/d2 22 2024-04-02T04:42:28.667 INFO:tasks.workunit.client.0.smithi016.stdout:0/5: write - no filename 2024-04-02T04:42:28.669 INFO:tasks.workunit.client.0.smithi016.stdout:1/6: symlink l1 0 2024-04-02T04:42:28.670 INFO:tasks.workunit.client.0.smithi016.stdout:1/7: chown c0 1206946133 1 2024-04-02T04:42:28.671 INFO:tasks.workunit.client.0.smithi016.stdout:1/8: dread - no filename 2024-04-02T04:42:28.671 INFO:tasks.workunit.client.0.smithi016.stdout:1/9: dwrite - no filename 2024-04-02T04:42:28.671 INFO:tasks.workunit.client.0.smithi016.stdout:1/10: write - no filename 2024-04-02T04:42:28.671 INFO:tasks.workunit.client.0.smithi016.stdout:4/0: dwrite - no filename 2024-04-02T04:42:28.671 INFO:tasks.workunit.client.0.smithi016.stdout:4/1: dread - no filename 2024-04-02T04:42:28.671 INFO:tasks.workunit.client.0.smithi016.stdout:4/2: unlink - no file 2024-04-02T04:42:28.671 INFO:tasks.workunit.client.0.smithi016.stdout:4/3: dwrite - no filename 2024-04-02T04:42:28.671 INFO:tasks.workunit.client.0.smithi016.stdout:4/4: dread - no filename 2024-04-02T04:42:28.671 INFO:tasks.workunit.client.0.smithi016.stdout:4/5: readlink - no filename 2024-04-02T04:42:28.671 INFO:tasks.workunit.client.0.smithi016.stdout:4/6: stat - no entries 2024-04-02T04:42:28.671 INFO:tasks.workunit.client.0.smithi016.stdout:4/7: dread - no filename 2024-04-02T04:42:28.671 INFO:tasks.workunit.client.0.smithi016.stdout:4/8: dread - no filename 2024-04-02T04:42:28.671 INFO:tasks.workunit.client.0.smithi016.stdout:4/9: fdatasync - no filename 2024-04-02T04:42:28.671 INFO:tasks.workunit.client.0.smithi016.stdout:2/3: mkdir d0 0 2024-04-02T04:42:28.672 INFO:tasks.workunit.client.0.smithi016.stdout:2/4: dread - no filename 2024-04-02T04:42:28.672 INFO:tasks.workunit.client.0.smithi016.stdout:2/5: write - no filename 2024-04-02T04:42:28.672 INFO:tasks.workunit.client.0.smithi016.stdout:2/6: write - no filename 2024-04-02T04:42:28.672 INFO:tasks.workunit.client.0.smithi016.stdout:1/11: chown l1 14 1 2024-04-02T04:42:28.672 INFO:tasks.workunit.client.0.smithi016.stdout:2/7: chown d0 3473 1 2024-04-02T04:42:28.672 INFO:tasks.workunit.client.0.smithi016.stdout:3/7: creat f0 x:0 0 0 2024-04-02T04:42:28.673 INFO:tasks.workunit.client.0.smithi016.stdout:5/0: dwrite - no filename 2024-04-02T04:42:28.673 INFO:tasks.workunit.client.0.smithi016.stdout:3/8: write f0 [657612,72348] 0 2024-04-02T04:42:28.673 INFO:tasks.workunit.client.0.smithi016.stdout:3/9: chown f0 10614049 1 2024-04-02T04:42:28.673 INFO:tasks.workunit.client.0.smithi016.stdout:1/12: creat f2 x:0 0 0 2024-04-02T04:42:28.674 INFO:tasks.workunit.client.0.smithi016.stdout:1/13: dread - f2 zero size 2024-04-02T04:42:28.674 INFO:tasks.workunit.client.0.smithi016.stdout:4/10: symlink l0 0 2024-04-02T04:42:28.676 INFO:tasks.workunit.client.0.smithi016.stdout:6/0: dwrite - no filename 2024-04-02T04:42:28.677 INFO:tasks.workunit.client.0.smithi016.stdout:6/1: dwrite - no filename 2024-04-02T04:42:28.678 INFO:tasks.workunit.client.0.smithi016.stdout:1/14: mknod c3 0 2024-04-02T04:42:28.681 INFO:tasks.workunit.client.0.smithi016.stdout:4/11: creat f1 x:0 0 0 2024-04-02T04:42:28.681 INFO:tasks.workunit.client.0.smithi016.stdout:4/12: readlink l0 0 2024-04-02T04:42:28.681 INFO:tasks.workunit.client.0.smithi016.stdout:4/13: stat l0 0 2024-04-02T04:42:28.682 INFO:tasks.workunit.client.0.smithi016.stdout:5/1: creat f0 x:0 0 0 2024-04-02T04:42:28.682 INFO:tasks.workunit.client.0.smithi016.stdout:5/2: truncate f0 159229 0 2024-04-02T04:42:28.682 INFO:tasks.workunit.client.0.smithi016.stdout:7/0: chown . 7826738 1 2024-04-02T04:42:28.683 INFO:tasks.workunit.client.0.smithi016.stdout:7/1: dread - no filename 2024-04-02T04:42:28.683 INFO:tasks.workunit.client.0.smithi016.stdout:7/2: chown . 1 1 2024-04-02T04:42:28.683 INFO:tasks.workunit.client.0.smithi016.stdout:7/3: dwrite - no filename 2024-04-02T04:42:28.683 INFO:tasks.workunit.client.0.smithi016.stdout:7/4: dwrite - no filename 2024-04-02T04:42:28.683 INFO:tasks.workunit.client.0.smithi016.stdout:3/10: link f0 f1 0 2024-04-02T04:42:28.683 INFO:tasks.workunit.client.0.smithi016.stdout:5/3: dread f0 [0,4194304] 0 2024-04-02T04:42:28.683 INFO:tasks.workunit.client.0.smithi016.stdout:5/4: readlink - no filename 2024-04-02T04:42:28.683 INFO:tasks.workunit.client.0.smithi016.stdout:2/8: rmdir d0 0 2024-04-02T04:42:28.683 INFO:tasks.workunit.client.0.smithi016.stdout:2/9: dwrite - no filename 2024-04-02T04:42:28.683 INFO:tasks.workunit.client.0.smithi016.stdout:2/10: truncate - no filename 2024-04-02T04:42:28.685 INFO:tasks.workunit.client.0.smithi016.stdout:4/14: unlink l0 0 2024-04-02T04:42:28.686 INFO:tasks.workunit.client.0.smithi016.stdout:8/0: dread - no filename 2024-04-02T04:42:28.686 INFO:tasks.workunit.client.0.smithi016.stdout:8/1: unlink - no file 2024-04-02T04:42:28.686 INFO:tasks.workunit.client.0.smithi016.stdout:8/2: fsync - no filename 2024-04-02T04:42:28.688 INFO:tasks.workunit.client.0.smithi016.stdout:2/11: mknod c1 0 2024-04-02T04:42:28.690 INFO:tasks.workunit.client.0.smithi016.stdout:4/15: creat f2 x:0 0 0 2024-04-02T04:42:28.692 INFO:tasks.workunit.client.0.smithi016.stdout:7/5: getdents . 0 2024-04-02T04:42:28.692 INFO:tasks.workunit.client.0.smithi016.stdout:7/6: chown . 3 1 2024-04-02T04:42:28.692 INFO:tasks.workunit.client.0.smithi016.stdout:7/7: read - no filename 2024-04-02T04:42:28.693 INFO:tasks.workunit.client.0.smithi016.stdout:5/5: link f0 f1 0 2024-04-02T04:42:28.693 INFO:tasks.workunit.client.0.smithi016.stdout:5/6: truncate f1 354679 0 2024-04-02T04:42:28.696 INFO:tasks.workunit.client.0.smithi016.stdout:2/12: unlink c1 0 2024-04-02T04:42:28.696 INFO:tasks.workunit.client.0.smithi016.stdout:2/13: truncate - no filename 2024-04-02T04:42:28.696 INFO:tasks.workunit.client.0.smithi016.stdout:1/15: link c0 c4 0 2024-04-02T04:42:28.698 INFO:tasks.workunit.client.0.smithi016.stdout:4/16: mknod c3 0 2024-04-02T04:42:28.701 INFO:tasks.workunit.client.0.smithi016.stdout:8/3: symlink l0 0 2024-04-02T04:42:28.703 INFO:tasks.workunit.client.0.smithi016.stdout:5/7: creat f2 x:0 0 0 2024-04-02T04:42:28.703 INFO:tasks.workunit.client.0.smithi016.stdout:2/14: symlink l2 0 2024-04-02T04:42:28.705 INFO:tasks.workunit.client.0.smithi016.stdout:2/15: chown l2 1 1 2024-04-02T04:42:28.706 INFO:tasks.workunit.client.0.smithi016.stdout:9/0: mkdir d0 0 2024-04-02T04:42:28.706 INFO:tasks.workunit.client.0.smithi016.stdout:1/16: dwrite f2 [0,4194304] 0 2024-04-02T04:42:28.707 INFO:tasks.workunit.client.0.smithi016.stdout:7/8: mkdir d0 0 2024-04-02T04:42:28.708 INFO:tasks.workunit.client.0.smithi016.stdout:7/9: read - no filename 2024-04-02T04:42:28.708 INFO:tasks.workunit.client.0.smithi016.stdout:7/10: dread - no filename 2024-04-02T04:42:28.708 INFO:tasks.workunit.client.0.smithi016.stdout:7/11: write - no filename 2024-04-02T04:42:28.708 INFO:tasks.workunit.client.0.smithi016.stdout:9/1: chown d0 106 1 2024-04-02T04:42:28.710 INFO:tasks.workunit.client.0.smithi016.stdout:8/4: mkdir d1 0 2024-04-02T04:42:28.710 INFO:tasks.workunit.client.0.smithi016.stdout:8/5: dread - no filename 2024-04-02T04:42:28.711 INFO:tasks.workunit.client.0.smithi016.stdout:4/17: dwrite f2 [0,4194304] 0 2024-04-02T04:42:28.712 INFO:tasks.workunit.client.0.smithi016.stdout:5/8: mkdir d3 0 2024-04-02T04:42:28.713 INFO:tasks.workunit.client.0.smithi016.stdout:5/9: chown f2 0 1 2024-04-02T04:42:28.717 INFO:tasks.workunit.client.0.smithi016.stdout:2/16: creat f3 x:0 0 0 2024-04-02T04:42:28.720 INFO:tasks.workunit.client.0.smithi016.stdout:1/17: mknod c5 0 2024-04-02T04:42:28.724 INFO:tasks.workunit.client.0.smithi016.stdout:2/17: dwrite f3 [0,4194304] 0 2024-04-02T04:42:28.724 INFO:tasks.workunit.client.0.smithi016.stdout:2/18: stat f3 0 2024-04-02T04:42:28.725 INFO:tasks.workunit.client.0.smithi016.stdout:2/19: write f3 [152984,120080] 0 2024-04-02T04:42:28.726 INFO:tasks.workunit.client.0.smithi016.stdout:4/18: creat f4 x:0 0 0 2024-04-02T04:42:28.728 INFO:tasks.workunit.client.0.smithi016.stdout:4/19: write f2 [3679417,51151] 0 2024-04-02T04:42:28.729 INFO:tasks.workunit.client.0.smithi016.stdout:5/10: unlink f0 0 2024-04-02T04:42:28.729 INFO:tasks.workunit.client.0.smithi016.stdout:4/20: dread - f4 zero size 2024-04-02T04:42:28.733 INFO:tasks.workunit.client.0.smithi016.stdout:7/12: mknod d0/c1 0 2024-04-02T04:42:28.733 INFO:tasks.workunit.client.0.smithi016.stdout:7/13: truncate - no filename 2024-04-02T04:42:28.734 INFO:tasks.workunit.client.0.smithi016.stdout:9/2: mknod d0/c1 0 2024-04-02T04:42:28.735 INFO:tasks.workunit.client.0.smithi016.stdout:9/3: dwrite - no filename 2024-04-02T04:42:28.736 INFO:tasks.workunit.client.0.smithi016.stdout:8/6: symlink d1/l2 0 2024-04-02T04:42:28.736 INFO:tasks.workunit.client.0.smithi016.stdout:8/7: fdatasync - no filename 2024-04-02T04:42:28.736 INFO:tasks.workunit.client.0.smithi016.stdout:8/8: read - no filename 2024-04-02T04:42:28.736 INFO:tasks.workunit.client.0.smithi016.stdout:8/9: readlink l0 0 2024-04-02T04:42:28.736 INFO:tasks.workunit.client.0.smithi016.stdout:8/10: write - no filename 2024-04-02T04:42:28.738 INFO:tasks.workunit.client.0.smithi016.stdout:2/20: mknod c4 0 2024-04-02T04:42:28.741 INFO:tasks.workunit.client.0.smithi016.stdout:1/18: stat c4 0 2024-04-02T04:42:28.743 INFO:tasks.workunit.client.0.smithi016.stdout:9/4: symlink d0/l2 0 2024-04-02T04:42:28.743 INFO:tasks.workunit.client.0.smithi016.stdout:9/5: dwrite - no filename 2024-04-02T04:42:28.744 INFO:tasks.workunit.client.0.smithi016.stdout:9/6: readlink d0/l2 0 2024-04-02T04:42:28.744 INFO:tasks.workunit.client.0.smithi016.stdout:3/11: getdents . 0 2024-04-02T04:42:28.744 INFO:tasks.workunit.client.0.smithi016.stdout:9/7: readlink d0/l2 0 2024-04-02T04:42:28.744 INFO:tasks.workunit.client.0.smithi016.stdout:9/8: dwrite - no filename 2024-04-02T04:42:28.747 INFO:tasks.workunit.client.0.smithi016.stdout:8/11: creat d1/f3 x:0 0 0 2024-04-02T04:42:28.747 INFO:tasks.workunit.client.0.smithi016.stdout:3/12: dread f0 [0,4194304] 0 2024-04-02T04:42:28.748 INFO:tasks.workunit.client.0.smithi016.stdout:8/12: truncate d1/f3 761798 0 2024-04-02T04:42:28.752 INFO:tasks.workunit.client.0.smithi016.stdout:5/11: creat d3/f4 x:0 0 0 2024-04-02T04:42:28.752 INFO:tasks.workunit.client.0.smithi016.stdout:5/12: read - d3/f4 zero size 2024-04-02T04:42:28.753 INFO:tasks.workunit.client.0.smithi016.stdout:1/19: creat f6 x:0 0 0 2024-04-02T04:42:28.757 INFO:tasks.workunit.client.0.smithi016.stdout:9/9: mkdir d0/d3 0 2024-04-02T04:42:28.768 INFO:tasks.workunit.client.0.smithi016.stdout:1/20: creat f7 x:0 0 0 2024-04-02T04:42:28.770 INFO:tasks.workunit.client.0.smithi016.stdout:9/10: rename d0/c1 to d0/d3/c4 0 2024-04-02T04:42:28.770 INFO:tasks.workunit.client.0.smithi016.stdout:9/11: dwrite - no filename 2024-04-02T04:42:28.770 INFO:tasks.workunit.client.0.smithi016.stdout:9/12: dread - no filename 2024-04-02T04:42:28.770 INFO:tasks.workunit.client.0.smithi016.stdout:9/13: truncate - no filename 2024-04-02T04:42:28.774 INFO:tasks.workunit.client.0.smithi016.stdout:1/21: dwrite f6 [0,4194304] 0 2024-04-02T04:42:28.776 INFO:tasks.workunit.client.0.smithi016.stdout:9/14: creat d0/d3/f5 x:0 0 0 2024-04-02T04:42:28.776 INFO:tasks.workunit.client.0.smithi016.stdout:2/21: fsync f3 0 2024-04-02T04:42:28.776 INFO:tasks.workunit.client.0.smithi016.stdout:2/22: rmdir - no directory 2024-04-02T04:42:28.776 INFO:tasks.workunit.client.0.smithi016.stdout:9/15: dread - d0/d3/f5 zero size 2024-04-02T04:42:28.782 INFO:tasks.workunit.client.0.smithi016.stdout:1/22: stat c4 0 2024-04-02T04:42:28.784 INFO:tasks.workunit.client.0.smithi016.stdout:1/23: write f2 [1921532,89689] 0 2024-04-02T04:42:28.787 INFO:tasks.workunit.client.0.smithi016.stdout:2/23: dwrite f3 [0,4194304] 0 2024-04-02T04:42:28.788 INFO:tasks.workunit.client.0.smithi016.stdout:9/16: dwrite d0/d3/f5 [0,4194304] 0 2024-04-02T04:42:28.792 INFO:tasks.workunit.client.0.smithi016.stdout:1/24: creat f8 x:0 0 0 2024-04-02T04:42:28.798 INFO:tasks.workunit.client.0.smithi016.stdout:8/13: getdents d1 0 2024-04-02T04:42:28.800 INFO:tasks.workunit.client.0.smithi016.stdout:2/24: dwrite f3 [0,4194304] 0 2024-04-02T04:42:28.802 INFO:tasks.workunit.client.0.smithi016.stdout:5/13: getdents d3 0 2024-04-02T04:42:28.804 INFO:tasks.workunit.client.0.smithi016.stdout:1/25: link l1 l9 0 2024-04-02T04:42:28.806 INFO:tasks.workunit.client.0.smithi016.stdout:2/25: rename c4 to c5 0 2024-04-02T04:42:28.808 INFO:tasks.workunit.client.0.smithi016.stdout:3/13: truncate f0 793791 0 2024-04-02T04:42:28.810 INFO:tasks.workunit.client.0.smithi016.stdout:5/14: dread - f2 zero size 2024-04-02T04:42:28.811 INFO:tasks.workunit.client.0.smithi016.stdout:8/14: dwrite d1/f3 [0,4194304] 0 2024-04-02T04:42:28.813 INFO:tasks.workunit.client.0.smithi016.stdout:2/26: rename f3 to f6 0 2024-04-02T04:42:28.813 INFO:tasks.workunit.client.0.smithi016.stdout:1/26: dread f2 [0,4194304] 0 2024-04-02T04:42:28.815 INFO:tasks.workunit.client.0.smithi016.stdout:9/17: getdents d0 0 2024-04-02T04:42:28.817 INFO:tasks.workunit.client.0.smithi016.stdout:3/14: mkdir d2 0 2024-04-02T04:42:28.829 INFO:tasks.workunit.client.0.smithi016.stdout:2/27: symlink l7 0 2024-04-02T04:42:28.829 INFO:tasks.workunit.client.0.smithi016.stdout:2/28: readlink l7 0 2024-04-02T04:42:28.829 INFO:tasks.workunit.client.0.smithi016.stdout:1/27: readlink l9 0 2024-04-02T04:42:28.831 INFO:tasks.workunit.client.0.smithi016.stdout:8/15: fsync d1/f3 0 2024-04-02T04:42:28.836 INFO:tasks.workunit.client.0.smithi016.stdout:8/16: dwrite d1/f3 [0,4194304] 0 2024-04-02T04:42:28.836 INFO:tasks.workunit.client.0.smithi016.stdout:2/29: creat f8 x:0 0 0 2024-04-02T04:42:28.836 INFO:tasks.workunit.client.0.smithi016.stdout:2/30: fdatasync f8 0 2024-04-02T04:42:28.837 INFO:tasks.workunit.client.0.smithi016.stdout:1/28: creat fa x:0 0 0 2024-04-02T04:42:28.844 INFO:tasks.workunit.client.0.smithi016.stdout:5/15: link d3/f4 d3/f5 0 2024-04-02T04:42:28.847 INFO:tasks.workunit.client.0.smithi016.stdout:2/31: creat f9 x:0 0 0 2024-04-02T04:42:28.848 INFO:tasks.workunit.client.0.smithi016.stdout:1/29: creat fb x:0 0 0 2024-04-02T04:42:28.849 INFO:tasks.workunit.client.0.smithi016.stdout:9/18: link d0/l2 d0/l6 0 2024-04-02T04:42:28.853 INFO:tasks.workunit.client.0.smithi016.stdout:2/32: creat fa x:0 0 0 2024-04-02T04:42:28.853 INFO:tasks.workunit.client.0.smithi016.stdout:2/33: dread - f9 zero size 2024-04-02T04:42:28.853 INFO:tasks.workunit.client.0.smithi016.stdout:2/34: rmdir - no directory 2024-04-02T04:42:28.855 INFO:tasks.workunit.client.0.smithi016.stdout:2/35: dread - f8 zero size 2024-04-02T04:42:28.855 INFO:tasks.workunit.client.0.smithi016.stdout:2/36: readlink l2 0 2024-04-02T04:42:28.856 INFO:tasks.workunit.client.0.smithi016.stdout:1/30: mknod cc 0 2024-04-02T04:42:28.859 INFO:tasks.workunit.client.0.smithi016.stdout:3/15: link f0 d2/f3 0 2024-04-02T04:42:28.859 INFO:tasks.workunit.client.0.smithi016.stdout:5/16: dwrite d3/f5 [0,4194304] 0 2024-04-02T04:42:28.862 INFO:tasks.workunit.client.0.smithi016.stdout:2/37: rename l2 to lb 0 2024-04-02T04:42:28.866 INFO:tasks.workunit.client.0.smithi016.stdout:1/31: dwrite fa [0,4194304] 0 2024-04-02T04:42:28.867 INFO:tasks.workunit.client.0.smithi016.stdout:1/32: fsync f6 0 2024-04-02T04:42:28.868 INFO:tasks.workunit.client.0.smithi016.stdout:5/17: rename f2 to d3/f6 0 2024-04-02T04:42:28.868 INFO:tasks.workunit.client.0.smithi016.stdout:5/18: readlink - no filename 2024-04-02T04:42:28.869 INFO:tasks.workunit.client.0.smithi016.stdout:2/38: dread f6 [0,4194304] 0 2024-04-02T04:42:28.870 INFO:tasks.workunit.client.0.smithi016.stdout:3/16: creat d2/f4 x:0 0 0 2024-04-02T04:42:28.876 INFO:tasks.workunit.client.0.smithi016.stdout:8/17: truncate d1/f3 2150379 0 2024-04-02T04:42:28.877 INFO:tasks.workunit.client.0.smithi016.stdout:1/33: creat fd x:0 0 0 2024-04-02T04:42:28.880 INFO:tasks.workunit.client.0.smithi016.stdout:5/19: write d3/f4 [4928238,8914] 0 2024-04-02T04:42:28.880 INFO:tasks.workunit.client.0.smithi016.stdout:1/34: dread f2 [0,4194304] 0 2024-04-02T04:42:28.881 INFO:tasks.workunit.client.0.smithi016.stdout:1/35: write fd [111829,97160] 0 2024-04-02T04:42:28.881 INFO:tasks.workunit.client.0.smithi016.stdout:1/36: chown c0 0 1 2024-04-02T04:42:28.883 INFO:tasks.workunit.client.0.smithi016.stdout:2/39: rename f6 to fc 0 2024-04-02T04:42:28.883 INFO:tasks.workunit.client.0.smithi016.stdout:2/40: rmdir - no directory 2024-04-02T04:42:28.883 INFO:tasks.workunit.client.0.smithi016.stdout:2/41: fsync f8 0 2024-04-02T04:42:28.884 INFO:tasks.workunit.client.0.smithi016.stdout:3/17: stat d2/f3 0 2024-04-02T04:42:28.889 INFO:tasks.workunit.client.0.smithi016.stdout:1/37: mknod ce 0 2024-04-02T04:42:28.891 INFO:tasks.workunit.client.0.smithi016.stdout:2/42: creat fd x:0 0 0 2024-04-02T04:42:28.894 INFO:tasks.workunit.client.0.smithi016.stdout:3/18: truncate d2/f3 15132 0 2024-04-02T04:42:28.896 INFO:tasks.workunit.client.0.smithi016.stdout:1/38: rename c3 to cf 0 2024-04-02T04:42:28.897 INFO:tasks.workunit.client.0.smithi016.stdout:2/43: creat fe x:0 0 0 2024-04-02T04:42:28.902 INFO:tasks.workunit.client.0.smithi016.stdout:1/39: readlink l9 0 2024-04-02T04:42:28.903 INFO:tasks.workunit.client.0.smithi016.stdout:1/40: dread - f8 zero size 2024-04-02T04:42:28.905 INFO:tasks.workunit.client.0.smithi016.stdout:2/44: creat ff x:0 0 0 2024-04-02T04:42:28.906 INFO:tasks.workunit.client.0.smithi016.stdout:2/45: dread - fa zero size 2024-04-02T04:42:28.906 INFO:tasks.workunit.client.0.smithi016.stdout:2/46: dread - f9 zero size 2024-04-02T04:42:28.906 INFO:tasks.workunit.client.0.smithi016.stdout:2/47: dread - f8 zero size 2024-04-02T04:42:28.909 INFO:tasks.workunit.client.0.smithi016.stdout:5/20: getdents d3 0 2024-04-02T04:42:28.911 INFO:tasks.workunit.client.0.smithi016.stdout:1/41: mknod c10 0 2024-04-02T04:42:28.912 INFO:tasks.workunit.client.0.smithi016.stdout:8/18: truncate d1/f3 734574 0 2024-04-02T04:42:28.913 INFO:tasks.workunit.client.0.smithi016.stdout:2/48: creat f10 x:0 0 0 2024-04-02T04:42:28.916 INFO:tasks.workunit.client.0.smithi016.stdout:5/21: dwrite d3/f4 [4194304,4194304] 0 2024-04-02T04:42:28.917 INFO:tasks.workunit.client.0.smithi016.stdout:1/42: creat f11 x:0 0 0 2024-04-02T04:42:28.917 INFO:tasks.workunit.client.0.smithi016.stdout:5/22: rename d3 to d3/d7 22 2024-04-02T04:42:28.918 INFO:tasks.workunit.client.0.smithi016.stdout:8/19: creat d1/f4 x:0 0 0 2024-04-02T04:42:28.921 INFO:tasks.workunit.client.0.smithi016.stdout:2/49: unlink l7 0 2024-04-02T04:42:28.921 INFO:tasks.workunit.client.0.smithi016.stdout:3/19: dwrite f1 [0,4194304] 0 2024-04-02T04:42:28.922 INFO:tasks.workunit.client.0.smithi016.stdout:1/43: mknod c12 0 2024-04-02T04:42:28.922 INFO:tasks.workunit.client.0.smithi016.stdout:1/44: fsync fd 0 2024-04-02T04:42:28.924 INFO:tasks.workunit.client.0.smithi016.stdout:5/23: creat d3/f8 x:0 0 0 2024-04-02T04:42:28.925 INFO:tasks.workunit.client.0.smithi016.stdout:8/20: creat d1/f5 x:0 0 0 2024-04-02T04:42:28.929 INFO:tasks.workunit.client.0.smithi016.stdout:3/20: mknod d2/c5 0 2024-04-02T04:42:28.930 INFO:tasks.workunit.client.0.smithi016.stdout:3/21: fdatasync d2/f4 0 2024-04-02T04:42:28.934 INFO:tasks.workunit.client.0.smithi016.stdout:1/45: mknod c13 0 2024-04-02T04:42:28.940 INFO:tasks.workunit.client.0.smithi016.stdout:2/50: rename c5 to c11 0 2024-04-02T04:42:28.940 INFO:tasks.workunit.client.0.smithi016.stdout:2/51: write fe [480666,758] 0 2024-04-02T04:42:28.941 INFO:tasks.workunit.client.0.smithi016.stdout:2/52: dread - fd zero size 2024-04-02T04:42:28.941 INFO:tasks.workunit.client.0.smithi016.stdout:3/22: mknod d2/c6 0 2024-04-02T04:42:28.942 INFO:tasks.workunit.client.0.smithi016.stdout:3/23: write d2/f4 [849739,71084] 0 2024-04-02T04:42:28.942 INFO:tasks.workunit.client.0.smithi016.stdout:3/24: readlink - no filename 2024-04-02T04:42:28.944 INFO:tasks.workunit.client.0.smithi016.stdout:1/46: mkdir d14 0 2024-04-02T04:42:28.945 INFO:tasks.workunit.client.0.smithi016.stdout:1/47: fsync f11 0 2024-04-02T04:42:28.950 INFO:tasks.workunit.client.0.smithi016.stdout:2/53: mkdir d12 0 2024-04-02T04:42:28.952 INFO:tasks.workunit.client.0.smithi016.stdout:3/25: creat d2/f7 x:0 0 0 2024-04-02T04:42:28.956 INFO:tasks.workunit.client.0.smithi016.stdout:2/54: rename fe to d12/f13 0 2024-04-02T04:42:28.959 INFO:tasks.workunit.client.0.smithi016.stdout:8/21: dread d1/f3 [0,4194304] 0 2024-04-02T04:42:28.960 INFO:tasks.workunit.client.0.smithi016.stdout:8/22: fsync d1/f5 0 2024-04-02T04:42:28.961 INFO:tasks.workunit.client.0.smithi016.stdout:5/24: truncate d3/f4 2275171 0 2024-04-02T04:42:28.962 INFO:tasks.workunit.client.0.smithi016.stdout:3/26: symlink d2/l8 0 2024-04-02T04:42:28.962 INFO:tasks.workunit.client.0.smithi016.stdout:3/27: truncate d2/f7 557194 0 2024-04-02T04:42:28.963 INFO:tasks.workunit.client.0.smithi016.stdout:1/48: creat d14/f15 x:0 0 0 2024-04-02T04:42:28.964 INFO:tasks.workunit.client.0.smithi016.stdout:1/49: write fb [284946,66175] 0 2024-04-02T04:42:28.966 INFO:tasks.workunit.client.0.smithi016.stdout:8/23: mkdir d1/d6 0 2024-04-02T04:42:28.967 INFO:tasks.workunit.client.0.smithi016.stdout:8/24: stat d1/l2 0 2024-04-02T04:42:28.967 INFO:tasks.workunit.client.0.smithi016.stdout:8/25: truncate d1/f5 680614 0 2024-04-02T04:42:28.968 INFO:tasks.workunit.client.0.smithi016.stdout:3/28: creat d2/f9 x:0 0 0 2024-04-02T04:42:28.968 INFO:tasks.workunit.client.0.smithi016.stdout:3/29: truncate d2/f4 1273270 0 2024-04-02T04:42:28.969 INFO:tasks.workunit.client.0.smithi016.stdout:1/50: unlink c4 0 2024-04-02T04:42:28.970 INFO:tasks.workunit.client.0.smithi016.stdout:1/51: fsync fd 0 2024-04-02T04:42:28.970 INFO:tasks.workunit.client.0.smithi016.stdout:5/25: write f1 [315583,3869] 0 2024-04-02T04:42:28.971 INFO:tasks.workunit.client.0.smithi016.stdout:5/26: truncate d3/f8 809209 0 2024-04-02T04:42:28.971 INFO:tasks.workunit.client.0.smithi016.stdout:8/26: symlink d1/l7 0 2024-04-02T04:42:28.972 INFO:tasks.workunit.client.0.smithi016.stdout:8/27: stat d1/f4 0 2024-04-02T04:42:28.972 INFO:tasks.workunit.client.0.smithi016.stdout:3/30: creat d2/fa x:0 0 0 2024-04-02T04:42:28.973 INFO:tasks.workunit.client.0.smithi016.stdout:5/27: fsync d3/f6 0 2024-04-02T04:42:28.974 INFO:tasks.workunit.client.0.smithi016.stdout:8/28: creat d1/f8 x:0 0 0 2024-04-02T04:42:28.976 INFO:tasks.workunit.client.0.smithi016.stdout:5/28: creat d3/f9 x:0 0 0 2024-04-02T04:42:28.976 INFO:tasks.workunit.client.0.smithi016.stdout:3/31: dread f0 [0,4194304] 0 2024-04-02T04:42:28.981 INFO:tasks.workunit.client.0.smithi016.stdout:8/29: symlink d1/d6/l9 0 2024-04-02T04:42:28.982 INFO:tasks.workunit.client.0.smithi016.stdout:3/32: dwrite d2/fa [0,4194304] 0 2024-04-02T04:42:28.982 INFO:tasks.workunit.client.0.smithi016.stdout:8/30: write d1/f5 [536991,104694] 0 2024-04-02T04:42:28.983 INFO:tasks.workunit.client.0.smithi016.stdout:8/31: rename d1 to d1/d6/da 22 2024-04-02T04:42:28.986 INFO:tasks.workunit.client.0.smithi016.stdout:3/33: fsync f0 0 2024-04-02T04:42:28.989 INFO:tasks.workunit.client.0.smithi016.stdout:5/29: dwrite d3/f5 [0,4194304] 0 2024-04-02T04:42:28.990 INFO:tasks.workunit.client.0.smithi016.stdout:8/32: rename d1/f8 to d1/fb 0 2024-04-02T04:42:28.990 INFO:tasks.workunit.client.0.smithi016.stdout:5/30: stat d3/f6 0 2024-04-02T04:42:28.993 INFO:tasks.workunit.client.0.smithi016.stdout:3/34: creat d2/fb x:0 0 0 2024-04-02T04:42:28.997 INFO:tasks.workunit.client.0.smithi016.stdout:5/31: creat d3/fa x:0 0 0 2024-04-02T04:42:29.012 INFO:tasks.workunit.client.0.smithi016.stdout:3/35: getdents d2 0 2024-04-02T04:42:29.015 INFO:tasks.workunit.client.0.smithi016.stdout:3/36: dread d2/f3 [0,4194304] 0 2024-04-02T04:42:29.018 INFO:tasks.workunit.client.0.smithi016.stdout:3/37: rename d2/c5 to d2/cc 0 2024-04-02T04:42:29.025 INFO:tasks.workunit.client.0.smithi016.stdout:3/38: truncate d2/fa 3475762 0 2024-04-02T04:42:29.036 INFO:tasks.workunit.client.0.smithi016.stdout:3/39: write d2/fa [1045843,8800] 0 2024-04-02T04:42:29.038 INFO:tasks.workunit.client.0.smithi016.stdout:3/40: creat d2/fd x:0 0 0 2024-04-02T04:42:29.039 INFO:tasks.workunit.client.0.smithi016.stdout:3/41: read d2/f4 [1192919,93275] 0 2024-04-02T04:42:29.045 INFO:tasks.workunit.client.0.smithi016.stdout:3/42: dread d2/f4 [0,4194304] 0 2024-04-02T04:42:29.048 INFO:tasks.workunit.client.0.smithi016.stdout:3/43: symlink d2/le 0 2024-04-02T04:42:29.049 INFO:tasks.workunit.client.0.smithi016.stdout:3/44: creat d2/ff x:0 0 0 2024-04-02T04:42:29.049 INFO:tasks.workunit.client.0.smithi016.stdout:3/45: dread - d2/fb zero size 2024-04-02T04:42:29.052 INFO:tasks.workunit.client.0.smithi016.stdout:3/46: dread f1 [0,4194304] 0 2024-04-02T04:42:29.061 INFO:tasks.workunit.client.0.smithi016.stdout:3/47: truncate d2/fa 2012402 0 2024-04-02T04:42:29.064 INFO:tasks.workunit.client.0.smithi016.stdout:3/48: symlink d2/l10 0 2024-04-02T04:42:29.065 INFO:tasks.workunit.client.0.smithi016.stdout:3/49: truncate d2/f7 1041694 0 2024-04-02T04:42:29.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:29 smithi016 ceph-mon[28969]: pgmap v13: 65 pgs: 65 active+clean; 189 MiB data, 800 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.2 MiB/s wr, 271 op/s 2024-04-02T04:42:29.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:29 smithi027 ceph-mon[37550]: pgmap v13: 65 pgs: 65 active+clean; 189 MiB data, 800 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.2 MiB/s wr, 271 op/s 2024-04-02T04:42:29.675 INFO:tasks.workunit.client.0.smithi016.stdout:0/6: sync 2024-04-02T04:42:29.677 INFO:tasks.workunit.client.0.smithi016.stdout:0/7: creat d0/f3 x:0 0 0 2024-04-02T04:42:29.684 INFO:tasks.workunit.client.0.smithi016.stdout:0/8: dwrite d0/f3 [0,4194304] 0 2024-04-02T04:42:29.716 INFO:tasks.workunit.client.0.smithi016.stdout:1/52: sync 2024-04-02T04:42:29.716 INFO:tasks.workunit.client.0.smithi016.stdout:2/55: sync 2024-04-02T04:42:29.716 INFO:tasks.workunit.client.0.smithi016.stdout:3/50: sync 2024-04-02T04:42:29.717 INFO:tasks.workunit.client.0.smithi016.stdout:9/19: sync 2024-04-02T04:42:29.717 INFO:tasks.workunit.client.0.smithi016.stdout:7/14: sync 2024-04-02T04:42:29.717 INFO:tasks.workunit.client.0.smithi016.stdout:7/15: truncate - no filename 2024-04-02T04:42:29.717 INFO:tasks.workunit.client.0.smithi016.stdout:8/33: sync 2024-04-02T04:42:29.717 INFO:tasks.workunit.client.0.smithi016.stdout:7/16: fdatasync - no filename 2024-04-02T04:42:29.717 INFO:tasks.workunit.client.0.smithi016.stdout:7/17: write - no filename 2024-04-02T04:42:29.717 INFO:tasks.workunit.client.0.smithi016.stdout:7/18: read - no filename 2024-04-02T04:42:29.717 INFO:tasks.workunit.client.0.smithi016.stdout:5/32: sync 2024-04-02T04:42:29.717 INFO:tasks.workunit.client.0.smithi016.stdout:6/2: sync 2024-04-02T04:42:29.717 INFO:tasks.workunit.client.0.smithi016.stdout:6/3: write - no filename 2024-04-02T04:42:29.717 INFO:tasks.workunit.client.0.smithi016.stdout:6/4: rmdir - no directory 2024-04-02T04:42:29.717 INFO:tasks.workunit.client.0.smithi016.stdout:4/21: sync 2024-04-02T04:42:29.718 INFO:tasks.workunit.client.0.smithi016.stdout:8/34: stat d1/f3 0 2024-04-02T04:42:29.718 INFO:tasks.workunit.client.0.smithi016.stdout:1/53: read - f11 zero size 2024-04-02T04:42:29.718 INFO:tasks.workunit.client.0.smithi016.stdout:4/22: write f1 [595344,42237] 0 2024-04-02T04:42:29.718 INFO:tasks.workunit.client.0.smithi016.stdout:8/35: write d1/f5 [125547,76491] 0 2024-04-02T04:42:29.719 INFO:tasks.workunit.client.0.smithi016.stdout:5/33: mknod d3/cb 0 2024-04-02T04:42:29.719 INFO:tasks.workunit.client.0.smithi016.stdout:8/36: readlink d1/d6/l9 0 2024-04-02T04:42:29.721 INFO:tasks.workunit.client.0.smithi016.stdout:3/51: creat d2/f11 x:0 0 0 2024-04-02T04:42:29.721 INFO:tasks.workunit.client.0.smithi016.stdout:3/52: read - d2/ff zero size 2024-04-02T04:42:29.722 INFO:tasks.workunit.client.0.smithi016.stdout:3/53: chown d2/f11 77193430 1 2024-04-02T04:42:29.722 INFO:tasks.workunit.client.0.smithi016.stdout:4/23: dread f2 [0,4194304] 0 2024-04-02T04:42:29.723 INFO:tasks.workunit.client.0.smithi016.stdout:7/19: creat d0/f2 x:0 0 0 2024-04-02T04:42:29.728 INFO:tasks.workunit.client.0.smithi016.stdout:1/54: rmdir d14 39 2024-04-02T04:42:29.735 INFO:tasks.workunit.client.0.smithi016.stdout:8/37: creat d1/d6/fc x:0 0 0 2024-04-02T04:42:29.736 INFO:tasks.workunit.client.0.smithi016.stdout:3/54: mknod d2/c12 0 2024-04-02T04:42:29.737 INFO:tasks.workunit.client.0.smithi016.stdout:4/24: mkdir d5 0 2024-04-02T04:42:29.740 INFO:tasks.workunit.client.0.smithi016.stdout:7/20: rename d0/c1 to d0/c3 0 2024-04-02T04:42:29.741 INFO:tasks.workunit.client.0.smithi016.stdout:6/5: getdents . 0 2024-04-02T04:42:29.741 INFO:tasks.workunit.client.0.smithi016.stdout:6/6: fsync - no filename 2024-04-02T04:42:29.741 INFO:tasks.workunit.client.0.smithi016.stdout:6/7: dwrite - no filename 2024-04-02T04:42:29.741 INFO:tasks.workunit.client.0.smithi016.stdout:6/8: stat - no entries 2024-04-02T04:42:29.741 INFO:tasks.workunit.client.0.smithi016.stdout:6/9: dwrite - no filename 2024-04-02T04:42:29.742 INFO:tasks.workunit.client.0.smithi016.stdout:6/10: write - no filename 2024-04-02T04:42:29.742 INFO:tasks.workunit.client.0.smithi016.stdout:6/11: truncate - no filename 2024-04-02T04:42:29.743 INFO:tasks.workunit.client.0.smithi016.stdout:6/12: chown . 333201 1 2024-04-02T04:42:29.744 INFO:tasks.workunit.client.0.smithi016.stdout:8/38: dwrite d1/d6/fc [0,4194304] 0 2024-04-02T04:42:29.747 INFO:tasks.workunit.client.0.smithi016.stdout:5/34: link d3/fa d3/fc 0 2024-04-02T04:42:29.748 INFO:tasks.workunit.client.0.smithi016.stdout:3/55: creat d2/f13 x:0 0 0 2024-04-02T04:42:29.749 INFO:tasks.workunit.client.0.smithi016.stdout:4/25: dwrite f4 [0,4194304] 0 2024-04-02T04:42:29.754 INFO:tasks.workunit.client.0.smithi016.stdout:7/21: mkdir d0/d4 0 2024-04-02T04:42:29.756 INFO:tasks.workunit.client.0.smithi016.stdout:1/55: dwrite d14/f15 [0,4194304] 0 2024-04-02T04:42:29.762 INFO:tasks.workunit.client.0.smithi016.stdout:5/35: mknod d3/cd 0 2024-04-02T04:42:29.767 INFO:tasks.workunit.client.0.smithi016.stdout:5/36: dread d3/f4 [0,4194304] 0 2024-04-02T04:42:29.767 INFO:tasks.workunit.client.0.smithi016.stdout:5/37: readlink - no filename 2024-04-02T04:42:29.767 INFO:tasks.workunit.client.0.smithi016.stdout:3/56: rename d2/ff to d2/f14 0 2024-04-02T04:42:29.772 INFO:tasks.workunit.client.0.smithi016.stdout:7/22: rename d0/f2 to d0/f5 0 2024-04-02T04:42:29.773 INFO:tasks.workunit.client.0.smithi016.stdout:2/56: getdents d12 0 2024-04-02T04:42:29.780 INFO:tasks.workunit.client.0.smithi016.stdout:4/26: getdents d5 0 2024-04-02T04:42:29.781 INFO:tasks.workunit.client.0.smithi016.stdout:4/27: read f4 [1839584,77353] 0 2024-04-02T04:42:29.781 INFO:tasks.workunit.client.0.smithi016.stdout:4/28: chown f2 1832438 1 2024-04-02T04:42:29.783 INFO:tasks.workunit.client.0.smithi016.stdout:3/57: mkdir d2/d15 0 2024-04-02T04:42:29.784 INFO:tasks.workunit.client.0.smithi016.stdout:5/38: creat d3/fe x:0 0 0 2024-04-02T04:42:29.785 INFO:tasks.workunit.client.0.smithi016.stdout:9/20: getdents d0/d3 0 2024-04-02T04:42:29.788 INFO:tasks.workunit.client.0.smithi016.stdout:2/57: mkdir d12/d14 0 2024-04-02T04:42:29.790 INFO:tasks.workunit.client.0.smithi016.stdout:1/56: link f6 d14/f16 0 2024-04-02T04:42:29.790 INFO:tasks.workunit.client.0.smithi016.stdout:1/57: stat f11 0 2024-04-02T04:42:29.793 INFO:tasks.workunit.client.0.smithi016.stdout:1/58: write f7 [1047827,99946] 0 2024-04-02T04:42:29.793 INFO:tasks.workunit.client.0.smithi016.stdout:5/39: dwrite f1 [0,4194304] 0 2024-04-02T04:42:29.795 INFO:tasks.workunit.client.0.smithi016.stdout:9/21: dwrite d0/d3/f5 [0,4194304] 0 2024-04-02T04:42:29.798 INFO:tasks.workunit.client.0.smithi016.stdout:4/29: read f2 [3990629,71041] 0 2024-04-02T04:42:29.801 INFO:tasks.workunit.client.0.smithi016.stdout:3/58: unlink d2/c12 0 2024-04-02T04:42:29.806 INFO:tasks.workunit.client.0.smithi016.stdout:7/23: getdents d0/d4 0 2024-04-02T04:42:29.807 INFO:tasks.workunit.client.0.smithi016.stdout:2/58: write d12/f13 [45748,68435] 0 2024-04-02T04:42:29.808 INFO:tasks.workunit.client.0.smithi016.stdout:8/39: getdents d1 0 2024-04-02T04:42:29.808 INFO:tasks.workunit.client.0.smithi016.stdout:3/59: dwrite d2/fd [0,4194304] 0 2024-04-02T04:42:29.809 INFO:tasks.workunit.client.0.smithi016.stdout:2/59: dread - f10 zero size 2024-04-02T04:42:29.810 INFO:tasks.workunit.client.0.smithi016.stdout:2/60: dread - fa zero size 2024-04-02T04:42:29.810 INFO:tasks.workunit.client.0.smithi016.stdout:1/59: write f6 [3818923,19066] 0 2024-04-02T04:42:29.813 INFO:tasks.workunit.client.0.smithi016.stdout:5/40: mkdir d3/df 0 2024-04-02T04:42:29.814 INFO:tasks.workunit.client.0.smithi016.stdout:1/60: write f7 [1267835,130654] 0 2024-04-02T04:42:29.815 INFO:tasks.workunit.client.0.smithi016.stdout:5/41: write f1 [3162675,42351] 0 2024-04-02T04:42:29.820 INFO:tasks.workunit.client.0.smithi016.stdout:3/60: dwrite f0 [0,4194304] 0 2024-04-02T04:42:29.822 INFO:tasks.workunit.client.0.smithi016.stdout:3/61: truncate d2/f14 579178 0 2024-04-02T04:42:29.822 INFO:tasks.workunit.client.0.smithi016.stdout:1/61: dread fa [0,4194304] 0 2024-04-02T04:42:29.823 INFO:tasks.workunit.client.0.smithi016.stdout:2/61: unlink fa 0 2024-04-02T04:42:29.824 INFO:tasks.workunit.client.0.smithi016.stdout:3/62: truncate d2/f3 4647501 0 2024-04-02T04:42:29.826 INFO:tasks.workunit.client.0.smithi016.stdout:6/13: sync 2024-04-02T04:42:29.826 INFO:tasks.workunit.client.0.smithi016.stdout:6/14: fdatasync - no filename 2024-04-02T04:42:29.828 INFO:tasks.workunit.client.0.smithi016.stdout:8/40: dwrite d1/fb [0,4194304] 0 2024-04-02T04:42:29.830 INFO:tasks.workunit.client.0.smithi016.stdout:1/62: dread f6 [0,4194304] 0 2024-04-02T04:42:29.831 INFO:tasks.workunit.client.0.smithi016.stdout:9/22: unlink d0/l2 0 2024-04-02T04:42:29.832 INFO:tasks.workunit.client.0.smithi016.stdout:9/23: write d0/d3/f5 [1147427,3431] 0 2024-04-02T04:42:29.834 INFO:tasks.workunit.client.0.smithi016.stdout:5/42: creat d3/f10 x:0 0 0 2024-04-02T04:42:29.834 INFO:tasks.workunit.client.0.smithi016.stdout:9/24: write d0/d3/f5 [1626502,61662] 0 2024-04-02T04:42:29.844 INFO:tasks.workunit.client.0.smithi016.stdout:2/62: rmdir d12 39 2024-04-02T04:42:29.844 INFO:tasks.workunit.client.0.smithi016.stdout:8/41: rename d1/fb to d1/fd 0 2024-04-02T04:42:29.845 INFO:tasks.workunit.client.0.smithi016.stdout:8/42: chown d1 2 1 2024-04-02T04:42:29.846 INFO:tasks.workunit.client.0.smithi016.stdout:9/25: dread d0/d3/f5 [0,4194304] 0 2024-04-02T04:42:29.848 INFO:tasks.workunit.client.0.smithi016.stdout:1/63: unlink d14/f15 0 2024-04-02T04:42:29.848 INFO:tasks.workunit.client.0.smithi016.stdout:9/26: dread d0/d3/f5 [0,4194304] 0 2024-04-02T04:42:29.849 INFO:tasks.workunit.client.0.smithi016.stdout:1/64: write fd [761564,17216] 0 2024-04-02T04:42:29.849 INFO:tasks.workunit.client.0.smithi016.stdout:9/27: stat d0/d3 0 2024-04-02T04:42:29.856 INFO:tasks.workunit.client.0.smithi016.stdout:7/24: rmdir d0/d4 0 2024-04-02T04:42:29.859 INFO:tasks.workunit.client.0.smithi016.stdout:6/15: mkdir d0 0 2024-04-02T04:42:29.859 INFO:tasks.workunit.client.0.smithi016.stdout:6/16: dwrite - no filename 2024-04-02T04:42:29.859 INFO:tasks.workunit.client.0.smithi016.stdout:6/17: chown d0 10134 1 2024-04-02T04:42:29.859 INFO:tasks.workunit.client.0.smithi016.stdout:6/18: truncate - no filename 2024-04-02T04:42:29.931 INFO:tasks.workunit.client.0.smithi016.stdout:1/65: creat d14/f17 x:0 0 0 2024-04-02T04:42:29.931 INFO:tasks.workunit.client.0.smithi016.stdout:5/43: mknod d3/df/c11 0 2024-04-02T04:42:29.931 INFO:tasks.workunit.client.0.smithi016.stdout:7/25: fdatasync d0/f5 0 2024-04-02T04:42:29.931 INFO:tasks.workunit.client.0.smithi016.stdout:2/63: mknod d12/c15 0 2024-04-02T04:42:29.932 INFO:tasks.workunit.client.0.smithi016.stdout:9/28: getdents d0/d3 0 2024-04-02T04:42:29.932 INFO:tasks.workunit.client.0.smithi016.stdout:1/66: symlink d14/l18 0 2024-04-02T04:42:29.933 INFO:tasks.workunit.client.0.smithi016.stdout:6/19: creat d0/f1 x:0 0 0 2024-04-02T04:42:29.935 INFO:tasks.workunit.client.0.smithi016.stdout:5/44: mknod d3/c12 0 2024-04-02T04:42:29.936 INFO:tasks.workunit.client.0.smithi016.stdout:6/20: write d0/f1 [212763,129802] 0 2024-04-02T04:42:29.940 INFO:tasks.workunit.client.0.smithi016.stdout:9/29: creat d0/f7 x:0 0 0 2024-04-02T04:42:29.941 INFO:tasks.workunit.client.0.smithi016.stdout:1/67: mkdir d14/d19 0 2024-04-02T04:42:29.941 INFO:tasks.workunit.client.0.smithi016.stdout:7/26: dwrite d0/f5 [0,4194304] 0 2024-04-02T04:42:29.943 INFO:tasks.workunit.client.0.smithi016.stdout:2/64: dwrite d12/f13 [0,4194304] 0 2024-04-02T04:42:29.943 INFO:tasks.workunit.client.0.smithi016.stdout:7/27: write d0/f5 [867455,51699] 0 2024-04-02T04:42:29.944 INFO:tasks.workunit.client.0.smithi016.stdout:2/65: truncate f9 1027855 0 2024-04-02T04:42:29.944 INFO:tasks.workunit.client.0.smithi016.stdout:7/28: read d0/f5 [2431287,113358] 0 2024-04-02T04:42:29.945 INFO:tasks.workunit.client.0.smithi016.stdout:5/45: symlink d3/df/l13 0 2024-04-02T04:42:29.947 INFO:tasks.workunit.client.0.smithi016.stdout:6/21: unlink d0/f1 0 2024-04-02T04:42:29.947 INFO:tasks.workunit.client.0.smithi016.stdout:6/22: truncate - no filename 2024-04-02T04:42:29.947 INFO:tasks.workunit.client.0.smithi016.stdout:6/23: unlink - no file 2024-04-02T04:42:29.947 INFO:tasks.workunit.client.0.smithi016.stdout:6/24: dwrite - no filename 2024-04-02T04:42:29.955 INFO:tasks.workunit.client.0.smithi016.stdout:1/68: unlink d14/l18 0 2024-04-02T04:42:29.960 INFO:tasks.workunit.client.0.smithi016.stdout:2/66: unlink fc 0 2024-04-02T04:42:29.965 INFO:tasks.workunit.client.0.smithi016.stdout:5/46: creat d3/df/f14 x:0 0 0 2024-04-02T04:42:29.971 INFO:tasks.workunit.client.0.smithi016.stdout:9/30: chown d0/l6 199298609 1 2024-04-02T04:42:29.971 INFO:tasks.workunit.client.0.smithi016.stdout:9/31: read - d0/f7 zero size 2024-04-02T04:42:29.971 INFO:tasks.workunit.client.0.smithi016.stdout:3/63: sync 2024-04-02T04:42:29.973 INFO:tasks.workunit.client.0.smithi016.stdout:1/69: mknod d14/c1a 0 2024-04-02T04:42:29.973 INFO:tasks.workunit.client.0.smithi016.stdout:9/32: chown d0/d3/f5 5 1 2024-04-02T04:42:29.973 INFO:tasks.workunit.client.0.smithi016.stdout:3/64: dread d2/f7 [0,4194304] 0 2024-04-02T04:42:29.975 INFO:tasks.workunit.client.0.smithi016.stdout:2/67: unlink fd 0 2024-04-02T04:42:29.979 INFO:tasks.workunit.client.0.smithi016.stdout:5/47: dread d3/f4 [0,4194304] 0 2024-04-02T04:42:29.980 INFO:tasks.workunit.client.0.smithi016.stdout:1/70: mknod d14/c1b 0 2024-04-02T04:42:29.980 INFO:tasks.workunit.client.0.smithi016.stdout:5/48: read d3/f4 [2658159,42262] 0 2024-04-02T04:42:29.981 INFO:tasks.workunit.client.0.smithi016.stdout:5/49: stat d3/df/f14 0 2024-04-02T04:42:29.982 INFO:tasks.workunit.client.0.smithi016.stdout:9/33: write d0/d3/f5 [345767,54894] 0 2024-04-02T04:42:29.982 INFO:tasks.workunit.client.0.smithi016.stdout:9/34: chown d0/f7 6 1 2024-04-02T04:42:29.986 INFO:tasks.workunit.client.0.smithi016.stdout:9/35: dread d0/d3/f5 [0,4194304] 0 2024-04-02T04:42:29.986 INFO:tasks.workunit.client.0.smithi016.stdout:9/36: fsync d0/d3/f5 0 2024-04-02T04:42:29.988 INFO:tasks.workunit.client.0.smithi016.stdout:2/68: creat d12/f16 x:0 0 0 2024-04-02T04:42:29.993 INFO:tasks.workunit.client.0.smithi016.stdout:1/71: write f2 [4986117,34527] 0 2024-04-02T04:42:29.994 INFO:tasks.workunit.client.0.smithi016.stdout:1/72: dread - d14/f17 zero size 2024-04-02T04:42:29.998 INFO:tasks.workunit.client.0.smithi016.stdout:1/73: dread fd [0,4194304] 0 2024-04-02T04:42:29.999 INFO:tasks.workunit.client.0.smithi016.stdout:1/74: chown d14/f16 1017199998 1 2024-04-02T04:42:29.999 INFO:tasks.workunit.client.0.smithi016.stdout:1/75: dread - f11 zero size 2024-04-02T04:42:29.999 INFO:tasks.workunit.client.0.smithi016.stdout:9/37: rmdir d0/d3 39 2024-04-02T04:42:30.000 INFO:tasks.workunit.client.0.smithi016.stdout:9/38: write d0/f7 [411611,65953] 0 2024-04-02T04:42:30.010 INFO:tasks.workunit.client.0.smithi016.stdout:2/69: symlink d12/d14/l17 0 2024-04-02T04:42:30.015 INFO:tasks.workunit.client.0.smithi016.stdout:5/50: link d3/cb d3/df/c15 0 2024-04-02T04:42:30.018 INFO:tasks.workunit.client.0.smithi016.stdout:2/70: symlink d12/d14/l18 0 2024-04-02T04:42:30.022 INFO:tasks.workunit.client.0.smithi016.stdout:5/51: fdatasync d3/fa 0 2024-04-02T04:42:30.028 INFO:tasks.workunit.client.0.smithi016.stdout:5/52: creat d3/f16 x:0 0 0 2024-04-02T04:42:30.031 INFO:tasks.workunit.client.0.smithi016.stdout:9/39: rename d0/d3/c4 to d0/c8 0 2024-04-02T04:42:30.034 INFO:tasks.workunit.client.0.smithi016.stdout:5/53: rename d3/f16 to d3/f17 0 2024-04-02T04:42:30.038 INFO:tasks.workunit.client.0.smithi016.stdout:5/54: write d3/f4 [3364452,31107] 0 2024-04-02T04:42:30.040 INFO:tasks.workunit.client.0.smithi016.stdout:9/40: mkdir d0/d9 0 2024-04-02T04:42:30.042 INFO:tasks.workunit.client.0.smithi016.stdout:5/55: mkdir d3/df/d18 0 2024-04-02T04:42:30.043 INFO:tasks.workunit.client.0.smithi016.stdout:9/41: dread d0/f7 [0,4194304] 0 2024-04-02T04:42:30.043 INFO:tasks.workunit.client.0.smithi016.stdout:8/43: sync 2024-04-02T04:42:30.047 INFO:tasks.workunit.client.0.smithi016.stdout:8/44: symlink d1/le 0 2024-04-02T04:42:30.050 INFO:tasks.workunit.client.0.smithi016.stdout:5/56: creat d3/df/d18/f19 x:0 0 0 2024-04-02T04:42:30.050 INFO:tasks.workunit.client.0.smithi016.stdout:8/45: dread d1/f5 [0,4194304] 0 2024-04-02T04:42:30.052 INFO:tasks.workunit.client.0.smithi016.stdout:9/42: symlink d0/d9/la 0 2024-04-02T04:42:30.056 INFO:tasks.workunit.client.0.smithi016.stdout:5/57: rename d3/fc to d3/f1a 0 2024-04-02T04:42:30.062 INFO:tasks.workunit.client.0.smithi016.stdout:5/58: dwrite d3/f4 [0,4194304] 0 2024-04-02T04:42:30.063 INFO:tasks.workunit.client.0.smithi016.stdout:9/43: mkdir d0/d9/db 0 2024-04-02T04:42:30.064 INFO:tasks.workunit.client.0.smithi016.stdout:0/9: write d0/f3 [4926942,86973] 0 2024-04-02T04:42:30.068 INFO:tasks.workunit.client.0.smithi016.stdout:5/59: creat d3/df/d18/f1b x:0 0 0 2024-04-02T04:42:30.069 INFO:tasks.workunit.client.0.smithi016.stdout:0/10: write d0/f3 [2616097,43113] 0 2024-04-02T04:42:30.072 INFO:tasks.workunit.client.0.smithi016.stdout:5/60: write d3/f8 [761497,91548] 0 2024-04-02T04:42:30.074 INFO:tasks.workunit.client.0.smithi016.stdout:3/65: fsync d2/fa 0 2024-04-02T04:42:30.075 INFO:tasks.workunit.client.0.smithi016.stdout:8/46: dwrite d1/f5 [0,4194304] 0 2024-04-02T04:42:30.075 INFO:tasks.workunit.client.0.smithi016.stdout:3/66: stat d2/f7 0 2024-04-02T04:42:30.078 INFO:tasks.workunit.client.0.smithi016.stdout:9/44: mkdir d0/dc 0 2024-04-02T04:42:30.079 INFO:tasks.workunit.client.0.smithi016.stdout:5/61: fdatasync d3/f5 0 2024-04-02T04:42:30.080 INFO:tasks.workunit.client.0.smithi016.stdout:5/62: read - d3/f10 zero size 2024-04-02T04:42:30.083 INFO:tasks.workunit.client.0.smithi016.stdout:0/11: rename d0/f3 to d0/f4 0 2024-04-02T04:42:30.086 INFO:tasks.workunit.client.0.smithi016.stdout:5/63: dwrite d3/f8 [0,4194304] 0 2024-04-02T04:42:30.087 INFO:tasks.workunit.client.0.smithi016.stdout:5/64: chown d3/df/f14 17 1 2024-04-02T04:42:30.090 INFO:tasks.workunit.client.0.smithi016.stdout:8/47: rename d1/f5 to d1/ff 0 2024-04-02T04:42:30.094 INFO:tasks.workunit.client.0.smithi016.stdout:8/48: dread d1/fd [0,4194304] 0 2024-04-02T04:42:30.095 INFO:tasks.workunit.client.0.smithi016.stdout:9/45: fdatasync d0/d3/f5 0 2024-04-02T04:42:30.095 INFO:tasks.workunit.client.0.smithi016.stdout:8/49: read d1/d6/fc [1924336,130058] 0 2024-04-02T04:42:30.102 INFO:tasks.workunit.client.0.smithi016.stdout:0/12: rename d0/l1 to d0/l5 0 2024-04-02T04:42:30.108 INFO:tasks.workunit.client.0.smithi016.stdout:5/65: creat d3/df/f1c x:0 0 0 2024-04-02T04:42:30.109 INFO:tasks.workunit.client.0.smithi016.stdout:3/67: mkdir d2/d15/d16 0 2024-04-02T04:42:30.110 INFO:tasks.workunit.client.0.smithi016.stdout:4/30: truncate f2 3289470 0 2024-04-02T04:42:30.110 INFO:tasks.workunit.client.0.smithi016.stdout:4/31: chown d5 0 1 2024-04-02T04:42:30.117 INFO:tasks.workunit.client.0.smithi016.stdout:3/68: dwrite d2/f9 [0,4194304] 0 2024-04-02T04:42:30.117 INFO:tasks.workunit.client.0.smithi016.stdout:8/50: rename d1/f4 to d1/d6/f10 0 2024-04-02T04:42:30.121 INFO:tasks.workunit.client.0.smithi016.stdout:0/13: write d0/f4 [1823077,77171] 0 2024-04-02T04:42:30.122 INFO:tasks.workunit.client.0.smithi016.stdout:7/29: rename d0/c3 to d0/c6 0 2024-04-02T04:42:30.124 INFO:tasks.workunit.client.0.smithi016.stdout:5/66: symlink d3/l1d 0 2024-04-02T04:42:30.125 INFO:tasks.workunit.client.0.smithi016.stdout:7/30: read d0/f5 [2444974,111661] 0 2024-04-02T04:42:30.126 INFO:tasks.workunit.client.0.smithi016.stdout:3/69: dwrite f0 [0,4194304] 0 2024-04-02T04:42:30.127 INFO:tasks.workunit.client.0.smithi016.stdout:3/70: chown d2/l10 1517285202 1 2024-04-02T04:42:30.129 INFO:tasks.workunit.client.0.smithi016.stdout:0/14: dread d0/f4 [0,4194304] 0 2024-04-02T04:42:30.130 INFO:tasks.workunit.client.0.smithi016.stdout:0/15: write d0/f4 [3140747,32772] 0 2024-04-02T04:42:30.143 INFO:tasks.workunit.client.0.smithi016.stdout:7/31: write d0/f5 [2407630,19473] 0 2024-04-02T04:42:30.150 INFO:tasks.workunit.client.0.smithi016.stdout:7/32: rename d0/f5 to d0/f7 0 2024-04-02T04:42:30.158 INFO:tasks.workunit.client.0.smithi016.stdout:4/32: truncate f4 1515731 0 2024-04-02T04:42:30.162 INFO:tasks.workunit.client.0.smithi016.stdout:4/33: rename f1 to d5/f6 0 2024-04-02T04:42:30.168 INFO:tasks.workunit.client.0.smithi016.stdout:4/34: creat d5/f7 x:0 0 0 2024-04-02T04:42:30.185 INFO:tasks.workunit.client.0.smithi016.stdout:1/76: truncate d14/f16 4032631 0 2024-04-02T04:42:30.187 INFO:tasks.workunit.client.0.smithi016.stdout:1/77: creat d14/f1c x:0 0 0 2024-04-02T04:42:30.188 INFO:tasks.workunit.client.0.smithi016.stdout:1/78: read f7 [224760,45482] 0 2024-04-02T04:42:30.212 INFO:tasks.workunit.client.0.smithi016.stdout:6/25: getdents d0 0 2024-04-02T04:42:30.212 INFO:tasks.workunit.client.0.smithi016.stdout:6/26: fsync - no filename 2024-04-02T04:42:30.212 INFO:tasks.workunit.client.0.smithi016.stdout:6/27: write - no filename 2024-04-02T04:42:30.212 INFO:tasks.workunit.client.0.smithi016.stdout:6/28: dwrite - no filename 2024-04-02T04:42:30.212 INFO:tasks.workunit.client.0.smithi016.stdout:6/29: write - no filename 2024-04-02T04:42:30.212 INFO:tasks.workunit.client.0.smithi016.stdout:6/30: chown d0 13474 1 2024-04-02T04:42:30.212 INFO:tasks.workunit.client.0.smithi016.stdout:6/31: link - no file 2024-04-02T04:42:30.212 INFO:tasks.workunit.client.0.smithi016.stdout:6/32: fdatasync - no filename 2024-04-02T04:42:30.216 INFO:tasks.workunit.client.0.smithi016.stdout:6/33: getdents d0 0 2024-04-02T04:42:30.216 INFO:tasks.workunit.client.0.smithi016.stdout:6/34: rename d0 to d0/d2 22 2024-04-02T04:42:30.219 INFO:tasks.workunit.client.0.smithi016.stdout:6/35: mknod d0/c3 0 2024-04-02T04:42:30.219 INFO:tasks.workunit.client.0.smithi016.stdout:6/36: dwrite - no filename 2024-04-02T04:42:30.219 INFO:tasks.workunit.client.0.smithi016.stdout:6/37: dwrite - no filename 2024-04-02T04:42:30.219 INFO:tasks.workunit.client.0.smithi016.stdout:6/38: stat d0 0 2024-04-02T04:42:30.219 INFO:tasks.workunit.client.0.smithi016.stdout:6/39: chown d0/c3 2011584 1 2024-04-02T04:42:30.219 INFO:tasks.workunit.client.0.smithi016.stdout:6/40: truncate - no filename 2024-04-02T04:42:30.222 INFO:tasks.workunit.client.0.smithi016.stdout:6/41: creat d0/f4 x:0 0 0 2024-04-02T04:42:30.222 INFO:tasks.workunit.client.0.smithi016.stdout:6/42: stat d0 0 2024-04-02T04:42:30.222 INFO:tasks.workunit.client.0.smithi016.stdout:6/43: readlink - no filename 2024-04-02T04:42:30.222 INFO:tasks.workunit.client.0.smithi016.stdout:6/44: chown d0 125215 1 2024-04-02T04:42:30.245 INFO:tasks.workunit.client.0.smithi016.stdout:2/71: rmdir d12 39 2024-04-02T04:42:30.245 INFO:tasks.workunit.client.0.smithi016.stdout:2/72: write f10 [641431,62030] 0 2024-04-02T04:42:30.251 INFO:tasks.workunit.client.0.smithi016.stdout:0/16: sync 2024-04-02T04:42:30.287 INFO:tasks.workunit.client.0.smithi016.stdout:9/46: truncate d0/d3/f5 3203851 0 2024-04-02T04:42:30.295 INFO:tasks.workunit.client.0.smithi016.stdout:5/67: rmdir d3/df 39 2024-04-02T04:42:30.296 INFO:tasks.workunit.client.0.smithi016.stdout:5/68: chown d3/fe 4 1 2024-04-02T04:42:30.296 INFO:tasks.workunit.client.0.smithi016.stdout:3/71: getdents d2/d15 0 2024-04-02T04:42:30.297 INFO:tasks.workunit.client.0.smithi016.stdout:3/72: write d2/fb [585008,104084] 0 2024-04-02T04:42:30.297 INFO:tasks.workunit.client.0.smithi016.stdout:3/73: stat d2/fb 0 2024-04-02T04:42:30.301 INFO:tasks.workunit.client.0.smithi016.stdout:1/79: sync 2024-04-02T04:42:30.303 INFO:tasks.workunit.client.0.smithi016.stdout:8/51: getdents d1/d6 0 2024-04-02T04:42:30.305 INFO:tasks.workunit.client.0.smithi016.stdout:8/52: dread d1/d6/fc [0,4194304] 0 2024-04-02T04:42:30.318 INFO:tasks.workunit.client.0.smithi016.stdout:7/33: rename d0/f7 to d0/f8 0 2024-04-02T04:42:30.319 INFO:tasks.workunit.client.0.smithi016.stdout:7/34: truncate d0/f8 4928081 0 2024-04-02T04:42:30.319 INFO:tasks.workunit.client.0.smithi016.stdout:7/35: stat d0 0 2024-04-02T04:42:30.411 INFO:tasks.workunit.client.0.smithi016.stdout:7/36: sync 2024-04-02T04:42:30.424 INFO:tasks.workunit.client.0.smithi016.stdout:4/35: write f2 [3095202,102694] 0 2024-04-02T04:42:30.424 INFO:tasks.workunit.client.0.smithi016.stdout:4/36: readlink - no filename 2024-04-02T04:42:30.508 INFO:tasks.workunit.client.0.smithi016.stdout:6/45: rename d0/c3 to d0/c5 0 2024-04-02T04:42:30.508 INFO:tasks.workunit.client.0.smithi016.stdout:6/46: dread - d0/f4 zero size 2024-04-02T04:42:30.509 INFO:tasks.workunit.client.0.smithi016.stdout:2/73: chown d12/d14/l17 432703 1 2024-04-02T04:42:30.510 INFO:tasks.workunit.client.0.smithi016.stdout:2/74: dread f9 [0,4194304] 0 2024-04-02T04:42:30.511 INFO:tasks.workunit.client.0.smithi016.stdout:0/17: symlink d0/l6 0 2024-04-02T04:42:30.521 INFO:tasks.workunit.client.0.smithi016.stdout:8/53: rmdir d1 39 2024-04-02T04:42:30.524 INFO:tasks.workunit.client.0.smithi016.stdout:4/37: unlink d5/f6 0 2024-04-02T04:42:30.527 INFO:tasks.workunit.client.0.smithi016.stdout:7/37: dwrite d0/f8 [0,4194304] 0 2024-04-02T04:42:30.527 INFO:tasks.workunit.client.0.smithi016.stdout:7/38: readlink - no filename 2024-04-02T04:42:30.527 INFO:tasks.workunit.client.0.smithi016.stdout:6/47: creat d0/f6 x:0 0 0 2024-04-02T04:42:30.528 INFO:tasks.workunit.client.0.smithi016.stdout:7/39: read d0/f8 [551432,35189] 0 2024-04-02T04:42:30.528 INFO:tasks.workunit.client.0.smithi016.stdout:7/40: readlink - no filename 2024-04-02T04:42:30.528 INFO:tasks.workunit.client.0.smithi016.stdout:6/48: write d0/f4 [485351,38922] 0 2024-04-02T04:42:30.529 INFO:tasks.workunit.client.0.smithi016.stdout:2/75: mknod d12/d14/c19 0 2024-04-02T04:42:30.531 INFO:tasks.workunit.client.0.smithi016.stdout:0/18: fdatasync d0/f4 0 2024-04-02T04:42:30.535 INFO:tasks.workunit.client.0.smithi016.stdout:5/69: creat d3/df/f1e x:0 0 0 2024-04-02T04:42:30.536 INFO:tasks.workunit.client.0.smithi016.stdout:6/49: dwrite d0/f6 [0,4194304] 0 2024-04-02T04:42:30.542 INFO:tasks.workunit.client.0.smithi016.stdout:8/54: unlink l0 0 2024-04-02T04:42:30.553 INFO:tasks.workunit.client.0.smithi016.stdout:7/41: mkdir d0/d9 0 2024-04-02T04:42:30.555 INFO:tasks.workunit.client.0.smithi016.stdout:2/76: mknod d12/c1a 0 2024-04-02T04:42:30.556 INFO:tasks.workunit.client.0.smithi016.stdout:2/77: write f8 [652341,44759] 0 2024-04-02T04:42:30.556 INFO:tasks.workunit.client.0.smithi016.stdout:2/78: rename d12/d14 to d12/d14/d1b 22 2024-04-02T04:42:30.556 INFO:tasks.workunit.client.0.smithi016.stdout:2/79: fsync ff 0 2024-04-02T04:42:30.557 INFO:tasks.workunit.client.0.smithi016.stdout:2/80: fdatasync d12/f16 0 2024-04-02T04:42:30.560 INFO:tasks.workunit.client.0.smithi016.stdout:0/19: creat d0/f7 x:0 0 0 2024-04-02T04:42:30.562 INFO:tasks.workunit.client.0.smithi016.stdout:9/47: read d0/d3/f5 [2461549,21597] 0 2024-04-02T04:42:30.569 INFO:tasks.workunit.client.0.smithi016.stdout:5/70: creat d3/df/f1f x:0 0 0 2024-04-02T04:42:30.571 INFO:tasks.workunit.client.0.smithi016.stdout:6/50: creat d0/f7 x:0 0 0 2024-04-02T04:42:30.572 INFO:tasks.workunit.client.0.smithi016.stdout:6/51: chown d0/f4 890442104 1 2024-04-02T04:42:30.572 INFO:tasks.workunit.client.0.smithi016.stdout:6/52: read d0/f4 [513082,93331] 0 2024-04-02T04:42:30.582 INFO:tasks.workunit.client.0.smithi016.stdout:0/20: creat d0/f8 x:0 0 0 2024-04-02T04:42:30.587 INFO:tasks.workunit.client.0.smithi016.stdout:3/74: getdents d2 0 2024-04-02T04:42:30.588 INFO:tasks.workunit.client.0.smithi016.stdout:3/75: dread - d2/f13 zero size 2024-04-02T04:42:30.594 INFO:tasks.workunit.client.0.smithi016.stdout:3/76: dwrite d2/f13 [0,4194304] 0 2024-04-02T04:42:30.594 INFO:tasks.workunit.client.0.smithi016.stdout:6/53: mkdir d0/d8 0 2024-04-02T04:42:30.595 INFO:tasks.workunit.client.0.smithi016.stdout:3/77: chown d2/fb 760823194 1 2024-04-02T04:42:30.601 INFO:tasks.workunit.client.0.smithi016.stdout:7/42: creat d0/d9/fa x:0 0 0 2024-04-02T04:42:30.603 INFO:tasks.workunit.client.0.smithi016.stdout:9/48: truncate d0/f7 852743 0 2024-04-02T04:42:30.604 INFO:tasks.workunit.client.0.smithi016.stdout:8/55: dwrite d1/ff [0,4194304] 0 2024-04-02T04:42:30.606 INFO:tasks.workunit.client.0.smithi016.stdout:5/71: mknod d3/c20 0 2024-04-02T04:42:30.607 INFO:tasks.workunit.client.0.smithi016.stdout:9/49: dread d0/d3/f5 [0,4194304] 0 2024-04-02T04:42:30.614 INFO:tasks.workunit.client.0.smithi016.stdout:6/54: mkdir d0/d9 0 2024-04-02T04:42:30.617 INFO:tasks.workunit.client.0.smithi016.stdout:3/78: creat d2/d15/f17 x:0 0 0 2024-04-02T04:42:30.618 INFO:tasks.workunit.client.0.smithi016.stdout:3/79: write d2/fa [1839156,64652] 0 2024-04-02T04:42:30.619 INFO:tasks.workunit.client.0.smithi016.stdout:3/80: chown f0 62605 1 2024-04-02T04:42:30.620 INFO:tasks.workunit.client.0.smithi016.stdout:7/43: mkdir d0/d9/db 0 2024-04-02T04:42:30.622 INFO:tasks.workunit.client.0.smithi016.stdout:3/81: dread f0 [0,4194304] 0 2024-04-02T04:42:30.626 INFO:tasks.workunit.client.0.smithi016.stdout:5/72: unlink d3/fa 0 2024-04-02T04:42:30.628 INFO:tasks.workunit.client.0.smithi016.stdout:8/56: dwrite d1/d6/fc [0,4194304] 0 2024-04-02T04:42:30.630 INFO:tasks.workunit.client.0.smithi016.stdout:6/55: symlink d0/la 0 2024-04-02T04:42:30.639 INFO:tasks.workunit.client.0.smithi016.stdout:3/82: mknod d2/c18 0 2024-04-02T04:42:30.645 INFO:tasks.workunit.client.0.smithi016.stdout:8/57: unlink d1/d6/fc 0 2024-04-02T04:42:30.650 INFO:tasks.workunit.client.0.smithi016.stdout:7/44: unlink d0/c6 0 2024-04-02T04:42:30.651 INFO:tasks.workunit.client.0.smithi016.stdout:7/45: truncate d0/d9/fa 74293 0 2024-04-02T04:42:30.654 INFO:tasks.workunit.client.0.smithi016.stdout:8/58: mknod d1/c11 0 2024-04-02T04:42:30.655 INFO:tasks.workunit.client.0.smithi016.stdout:8/59: dread d1/f3 [0,4194304] 0 2024-04-02T04:42:30.660 INFO:tasks.workunit.client.0.smithi016.stdout:7/46: unlink d0/f8 0 2024-04-02T04:42:30.661 INFO:tasks.workunit.client.0.smithi016.stdout:7/47: write d0/d9/fa [520977,42415] 0 2024-04-02T04:42:30.662 INFO:tasks.workunit.client.0.smithi016.stdout:7/48: chown d0/d9 29 1 2024-04-02T04:42:30.663 INFO:tasks.workunit.client.0.smithi016.stdout:8/60: symlink d1/d6/l12 0 2024-04-02T04:42:30.664 INFO:tasks.workunit.client.0.smithi016.stdout:7/49: fdatasync d0/d9/fa 0 2024-04-02T04:42:30.669 INFO:tasks.workunit.client.0.smithi016.stdout:8/61: rename d1/c11 to d1/d6/c13 0 2024-04-02T04:42:30.670 INFO:tasks.workunit.client.0.smithi016.stdout:0/21: sync 2024-04-02T04:42:30.671 INFO:tasks.workunit.client.0.smithi016.stdout:0/22: dread d0/f4 [4194304,4194304] 0 2024-04-02T04:42:30.672 INFO:tasks.workunit.client.0.smithi016.stdout:7/50: creat d0/d9/db/fc x:0 0 0 2024-04-02T04:42:30.672 INFO:tasks.workunit.client.0.smithi016.stdout:7/51: fdatasync d0/d9/db/fc 0 2024-04-02T04:42:30.678 INFO:tasks.workunit.client.0.smithi016.stdout:7/52: symlink d0/d9/db/ld 0 2024-04-02T04:42:30.679 INFO:tasks.workunit.client.0.smithi016.stdout:7/53: write d0/d9/fa [364741,53578] 0 2024-04-02T04:42:30.679 INFO:tasks.workunit.client.0.smithi016.stdout:7/54: fdatasync d0/d9/db/fc 0 2024-04-02T04:42:30.682 INFO:tasks.workunit.client.0.smithi016.stdout:7/55: symlink d0/d9/db/le 0 2024-04-02T04:42:30.735 INFO:tasks.workunit.client.0.smithi016.stdout:4/38: rmdir d5 39 2024-04-02T04:42:30.740 INFO:tasks.workunit.client.0.smithi016.stdout:4/39: mkdir d5/d8 0 2024-04-02T04:42:30.740 INFO:tasks.workunit.client.0.smithi016.stdout:1/80: read d14/f16 [1499714,43463] 0 2024-04-02T04:42:30.743 INFO:tasks.workunit.client.0.smithi016.stdout:4/40: symlink d5/l9 0 2024-04-02T04:42:30.747 INFO:tasks.workunit.client.0.smithi016.stdout:1/81: mkdir d14/d19/d1d 0 2024-04-02T04:42:30.752 INFO:tasks.workunit.client.0.smithi016.stdout:1/82: truncate fa 2785621 0 2024-04-02T04:42:30.753 INFO:tasks.workunit.client.0.smithi016.stdout:1/83: fsync f2 0 2024-04-02T04:42:30.755 INFO:tasks.workunit.client.0.smithi016.stdout:1/84: dread fd [0,4194304] 0 2024-04-02T04:42:30.760 INFO:tasks.workunit.client.0.smithi016.stdout:2/81: truncate f9 1146048 0 2024-04-02T04:42:30.762 INFO:tasks.workunit.client.0.smithi016.stdout:2/82: rmdir d12/d14 39 2024-04-02T04:42:30.768 INFO:tasks.workunit.client.0.smithi016.stdout:0/23: truncate d0/f4 1020803 0 2024-04-02T04:42:30.771 INFO:tasks.workunit.client.0.smithi016.stdout:2/83: getdents d12 0 2024-04-02T04:42:30.772 INFO:tasks.workunit.client.0.smithi016.stdout:0/24: rename d0/f8 to d0/f9 0 2024-04-02T04:42:30.772 INFO:tasks.workunit.client.0.smithi016.stdout:0/25: chown d0/f7 950902 1 2024-04-02T04:42:30.778 INFO:tasks.workunit.client.0.smithi016.stdout:0/26: write d0/f9 [785434,8971] 0 2024-04-02T04:42:30.778 INFO:tasks.workunit.client.0.smithi016.stdout:5/73: rmdir d3/df 39 2024-04-02T04:42:30.781 INFO:tasks.workunit.client.0.smithi016.stdout:5/74: dread d3/f8 [0,4194304] 0 2024-04-02T04:42:30.790 INFO:tasks.workunit.client.0.smithi016.stdout:5/75: link d3/c20 d3/df/c21 0 2024-04-02T04:42:30.791 INFO:tasks.workunit.client.0.smithi016.stdout:5/76: fsync d3/df/f1c 0 2024-04-02T04:42:30.795 INFO:tasks.workunit.client.0.smithi016.stdout:5/77: unlink d3/c20 0 2024-04-02T04:42:30.797 INFO:tasks.workunit.client.0.smithi016.stdout:5/78: rmdir d3/df/d18 39 2024-04-02T04:42:30.803 INFO:tasks.workunit.client.0.smithi016.stdout:5/79: creat d3/df/f22 x:0 0 0 2024-04-02T04:42:30.814 INFO:tasks.workunit.client.0.smithi016.stdout:7/56: sync 2024-04-02T04:42:30.816 INFO:tasks.workunit.client.0.smithi016.stdout:5/80: chown d3/fe 426 1 2024-04-02T04:42:30.816 INFO:tasks.workunit.client.0.smithi016.stdout:9/50: write d0/d3/f5 [684485,80215] 0 2024-04-02T04:42:30.817 INFO:tasks.workunit.client.0.smithi016.stdout:7/57: write d0/d9/db/fc [444020,36101] 0 2024-04-02T04:42:30.819 INFO:tasks.workunit.client.0.smithi016.stdout:6/56: getdents d0 0 2024-04-02T04:42:30.824 INFO:tasks.workunit.client.0.smithi016.stdout:9/51: stat d0/l6 0 2024-04-02T04:42:30.825 INFO:tasks.workunit.client.0.smithi016.stdout:7/58: mknod d0/d9/db/cf 0 2024-04-02T04:42:30.826 INFO:tasks.workunit.client.0.smithi016.stdout:6/57: mknod d0/cb 0 2024-04-02T04:42:30.827 INFO:tasks.workunit.client.0.smithi016.stdout:6/58: dread - d0/f7 zero size 2024-04-02T04:42:30.831 INFO:tasks.workunit.client.0.smithi016.stdout:7/59: symlink d0/l10 0 2024-04-02T04:42:30.832 INFO:tasks.workunit.client.0.smithi016.stdout:7/60: write d0/d9/fa [27642,120032] 0 2024-04-02T04:42:30.836 INFO:tasks.workunit.client.0.smithi016.stdout:9/52: creat d0/d9/fd x:0 0 0 2024-04-02T04:42:30.839 INFO:tasks.workunit.client.0.smithi016.stdout:7/61: creat d0/d9/f11 x:0 0 0 2024-04-02T04:42:30.839 INFO:tasks.workunit.client.0.smithi016.stdout:3/83: truncate d2/fb 518899 0 2024-04-02T04:42:30.841 INFO:tasks.workunit.client.0.smithi016.stdout:7/62: dread d0/d9/db/fc [0,4194304] 0 2024-04-02T04:42:30.844 INFO:tasks.workunit.client.0.smithi016.stdout:3/84: mknod d2/d15/c19 0 2024-04-02T04:42:30.849 INFO:tasks.workunit.client.0.smithi016.stdout:7/63: rename d0/d9/db/ld to d0/d9/db/l12 0 2024-04-02T04:42:30.849 INFO:tasks.workunit.client.0.smithi016.stdout:7/64: chown d0 610 1 2024-04-02T04:42:30.850 INFO:tasks.workunit.client.0.smithi016.stdout:8/62: truncate d1/f3 107420 0 2024-04-02T04:42:30.852 INFO:tasks.workunit.client.0.smithi016.stdout:3/85: mkdir d2/d15/d1a 0 2024-04-02T04:42:30.853 INFO:tasks.workunit.client.0.smithi016.stdout:7/65: symlink d0/d9/l13 0 2024-04-02T04:42:30.854 INFO:tasks.workunit.client.0.smithi016.stdout:7/66: dread d0/d9/db/fc [0,4194304] 0 2024-04-02T04:42:30.855 INFO:tasks.workunit.client.0.smithi016.stdout:8/63: write d1/d6/f10 [44928,23959] 0 2024-04-02T04:42:30.858 INFO:tasks.workunit.client.0.smithi016.stdout:7/67: write d0/d9/fa [250066,101064] 0 2024-04-02T04:42:30.858 INFO:tasks.workunit.client.0.smithi016.stdout:3/86: creat d2/d15/f1b x:0 0 0 2024-04-02T04:42:30.858 INFO:tasks.workunit.client.0.smithi016.stdout:1/85: sync 2024-04-02T04:42:30.858 INFO:tasks.workunit.client.0.smithi016.stdout:1/86: dread - f11 zero size 2024-04-02T04:42:30.859 INFO:tasks.workunit.client.0.smithi016.stdout:3/87: dread d2/fa [0,4194304] 0 2024-04-02T04:42:30.860 INFO:tasks.workunit.client.0.smithi016.stdout:8/64: symlink d1/l14 0 2024-04-02T04:42:30.860 INFO:tasks.workunit.client.0.smithi016.stdout:8/65: chown d1/d6/l12 14468867 1 2024-04-02T04:42:30.861 INFO:tasks.workunit.client.0.smithi016.stdout:7/68: mknod d0/d9/db/c14 0 2024-04-02T04:42:30.862 INFO:tasks.workunit.client.0.smithi016.stdout:7/69: truncate d0/d9/f11 1027979 0 2024-04-02T04:42:30.866 INFO:tasks.workunit.client.0.smithi016.stdout:3/88: truncate d2/f7 1143845 0 2024-04-02T04:42:30.868 INFO:tasks.workunit.client.0.smithi016.stdout:8/66: creat d1/f15 x:0 0 0 2024-04-02T04:42:30.873 INFO:tasks.workunit.client.0.smithi016.stdout:8/67: mknod d1/c16 0 2024-04-02T04:42:30.877 INFO:tasks.workunit.client.0.smithi016.stdout:7/70: dwrite d0/d9/db/fc [0,4194304] 0 2024-04-02T04:42:30.881 INFO:tasks.workunit.client.0.smithi016.stdout:3/89: dwrite d2/f14 [0,4194304] 0 2024-04-02T04:42:30.889 INFO:tasks.workunit.client.0.smithi016.stdout:4/41: dwrite f4 [0,4194304] 0 2024-04-02T04:42:30.895 INFO:tasks.workunit.client.0.smithi016.stdout:4/42: dread f4 [0,4194304] 0 2024-04-02T04:42:30.896 INFO:tasks.workunit.client.0.smithi016.stdout:4/43: readlink d5/l9 0 2024-04-02T04:42:30.904 INFO:tasks.workunit.client.0.smithi016.stdout:4/44: mknod d5/d8/ca 0 2024-04-02T04:42:30.907 INFO:tasks.workunit.client.0.smithi016.stdout:4/45: creat d5/d8/fb x:0 0 0 2024-04-02T04:42:30.909 INFO:tasks.workunit.client.0.smithi016.stdout:4/46: symlink d5/d8/lc 0 2024-04-02T04:42:30.911 INFO:tasks.workunit.client.0.smithi016.stdout:4/47: creat d5/d8/fd x:0 0 0 2024-04-02T04:42:30.913 INFO:tasks.workunit.client.0.smithi016.stdout:4/48: mkdir d5/de 0 2024-04-02T04:42:30.916 INFO:tasks.workunit.client.0.smithi016.stdout:6/59: write d0/f4 [1383987,127601] 0 2024-04-02T04:42:30.919 INFO:tasks.workunit.client.0.smithi016.stdout:6/60: truncate d0/f7 888107 0 2024-04-02T04:42:30.921 INFO:tasks.workunit.client.0.smithi016.stdout:4/49: dwrite f2 [0,4194304] 0 2024-04-02T04:42:30.925 INFO:tasks.workunit.client.0.smithi016.stdout:4/50: creat d5/d8/ff x:0 0 0 2024-04-02T04:42:30.926 INFO:tasks.workunit.client.0.smithi016.stdout:7/71: readlink d0/d9/db/l12 0 2024-04-02T04:42:30.927 INFO:tasks.workunit.client.0.smithi016.stdout:6/61: dread d0/f6 [0,4194304] 0 2024-04-02T04:42:30.929 INFO:tasks.workunit.client.0.smithi016.stdout:4/51: stat d5/d8/fd 0 2024-04-02T04:42:30.930 INFO:tasks.workunit.client.0.smithi016.stdout:8/68: truncate d1/f3 170975 0 2024-04-02T04:42:30.930 INFO:tasks.workunit.client.0.smithi016.stdout:9/53: dwrite d0/f7 [0,4194304] 0 2024-04-02T04:42:30.931 INFO:tasks.workunit.client.0.smithi016.stdout:9/54: dread - d0/d9/fd zero size 2024-04-02T04:42:30.932 INFO:tasks.workunit.client.0.smithi016.stdout:9/55: write d0/d3/f5 [625771,127752] 0 2024-04-02T04:42:30.933 INFO:tasks.workunit.client.0.smithi016.stdout:9/56: readlink d0/d9/la 0 2024-04-02T04:42:30.935 INFO:tasks.workunit.client.0.smithi016.stdout:4/52: creat d5/f10 x:0 0 0 2024-04-02T04:42:30.936 INFO:tasks.workunit.client.0.smithi016.stdout:6/62: creat d0/fc x:0 0 0 2024-04-02T04:42:30.936 INFO:tasks.workunit.client.0.smithi016.stdout:6/63: dread - d0/fc zero size 2024-04-02T04:42:30.938 INFO:tasks.workunit.client.0.smithi016.stdout:8/69: mknod d1/c17 0 2024-04-02T04:42:30.943 INFO:tasks.workunit.client.0.smithi016.stdout:4/53: unlink f2 0 2024-04-02T04:42:30.945 INFO:tasks.workunit.client.0.smithi016.stdout:8/70: fsync d1/fd 0 2024-04-02T04:42:30.950 INFO:tasks.workunit.client.0.smithi016.stdout:1/87: dwrite f6 [0,4194304] 0 2024-04-02T04:42:30.951 INFO:tasks.workunit.client.0.smithi016.stdout:6/64: creat d0/d9/fd x:0 0 0 2024-04-02T04:42:30.953 INFO:tasks.workunit.client.0.smithi016.stdout:7/72: getdents d0 0 2024-04-02T04:42:30.953 INFO:tasks.workunit.client.0.smithi016.stdout:2/84: write f9 [1134527,100875] 0 2024-04-02T04:42:30.958 INFO:tasks.workunit.client.0.smithi016.stdout:8/71: dwrite d1/ff [0,4194304] 0 2024-04-02T04:42:30.963 INFO:tasks.workunit.client.0.smithi016.stdout:1/88: dwrite f6 [0,4194304] 0 2024-04-02T04:42:30.963 INFO:tasks.workunit.client.0.smithi016.stdout:4/54: mknod d5/de/c11 0 2024-04-02T04:42:30.964 INFO:tasks.workunit.client.0.smithi016.stdout:1/89: dread - f11 zero size 2024-04-02T04:42:30.971 INFO:tasks.workunit.client.0.smithi016.stdout:6/65: dwrite d0/f6 [0,4194304] 0 2024-04-02T04:42:30.971 INFO:tasks.workunit.client.0.smithi016.stdout:9/57: rmdir d0/dc 0 2024-04-02T04:42:30.973 INFO:tasks.workunit.client.0.smithi016.stdout:0/27: sync 2024-04-02T04:42:30.974 INFO:tasks.workunit.client.0.smithi016.stdout:6/66: read d0/f4 [1300551,99062] 0 2024-04-02T04:42:30.975 INFO:tasks.workunit.client.0.smithi016.stdout:6/67: stat d0/c5 0 2024-04-02T04:42:30.976 INFO:tasks.workunit.client.0.smithi016.stdout:6/68: write d0/d9/fd [452881,39216] 0 2024-04-02T04:42:30.978 INFO:tasks.workunit.client.0.smithi016.stdout:7/73: rename d0/d9/db to d0/d9/d15 0 2024-04-02T04:42:30.979 INFO:tasks.workunit.client.0.smithi016.stdout:8/72: creat d1/f18 x:0 0 0 2024-04-02T04:42:30.987 INFO:tasks.workunit.client.0.smithi016.stdout:2/85: link f8 d12/f1c 0 2024-04-02T04:42:30.990 INFO:tasks.workunit.client.0.smithi016.stdout:0/28: unlink d0/f9 0 2024-04-02T04:42:30.991 INFO:tasks.workunit.client.0.smithi016.stdout:5/81: fsync d3/df/d18/f19 0 2024-04-02T04:42:30.993 INFO:tasks.workunit.client.0.smithi016.stdout:6/69: mknod d0/d9/ce 0 2024-04-02T04:42:30.995 INFO:tasks.workunit.client.0.smithi016.stdout:6/70: dread d0/d9/fd [0,4194304] 0 2024-04-02T04:42:30.998 INFO:tasks.workunit.client.0.smithi016.stdout:9/58: mknod d0/d9/db/ce 0 2024-04-02T04:42:31.000 INFO:tasks.workunit.client.0.smithi016.stdout:1/90: dread fa [0,4194304] 0 2024-04-02T04:42:31.000 INFO:tasks.workunit.client.0.smithi016.stdout:1/91: readlink l9 0 2024-04-02T04:42:31.001 INFO:tasks.workunit.client.0.smithi016.stdout:1/92: dread fd [0,4194304] 0 2024-04-02T04:42:31.002 INFO:tasks.workunit.client.0.smithi016.stdout:1/93: write d14/f16 [5145285,29428] 0 2024-04-02T04:42:31.003 INFO:tasks.workunit.client.0.smithi016.stdout:1/94: write f8 [206701,100069] 0 2024-04-02T04:42:31.009 INFO:tasks.workunit.client.0.smithi016.stdout:8/73: fsync d1/f3 0 2024-04-02T04:42:31.010 INFO:tasks.workunit.client.0.smithi016.stdout:6/71: mkdir d0/d9/df 0 2024-04-02T04:42:31.011 INFO:tasks.workunit.client.0.smithi016.stdout:6/72: chown d0/d9 2114 1 2024-04-02T04:42:31.012 INFO:tasks.workunit.client.0.smithi016.stdout:9/59: symlink d0/d9/db/lf 0 2024-04-02T04:42:31.014 INFO:tasks.workunit.client.0.smithi016.stdout:2/86: chown d12/f1c 30 1 2024-04-02T04:42:31.014 INFO:tasks.workunit.client.0.smithi016.stdout:2/87: chown d12 762 1 2024-04-02T04:42:31.015 INFO:tasks.workunit.client.0.smithi016.stdout:9/60: dread d0/d3/f5 [0,4194304] 0 2024-04-02T04:42:31.017 INFO:tasks.workunit.client.0.smithi016.stdout:9/61: dread d0/d3/f5 [0,4194304] 0 2024-04-02T04:42:31.019 INFO:tasks.workunit.client.0.smithi016.stdout:5/82: symlink d3/l23 0 2024-04-02T04:42:31.020 INFO:tasks.workunit.client.0.smithi016.stdout:0/29: fsync d0/f4 0 2024-04-02T04:42:31.023 INFO:tasks.workunit.client.0.smithi016.stdout:1/95: mkdir d14/d19/d1e 0 2024-04-02T04:42:31.023 INFO:tasks.workunit.client.0.smithi016.stdout:9/62: dwrite d0/d3/f5 [0,4194304] 0 2024-04-02T04:42:31.023 INFO:tasks.workunit.client.0.smithi016.stdout:7/74: getdents d0 0 2024-04-02T04:42:31.025 INFO:tasks.workunit.client.0.smithi016.stdout:7/75: write d0/d9/f11 [29733,78807] 0 2024-04-02T04:42:31.025 INFO:tasks.workunit.client.0.smithi016.stdout:8/74: creat d1/f19 x:0 0 0 2024-04-02T04:42:31.027 INFO:tasks.workunit.client.0.smithi016.stdout:6/73: fsync d0/f4 0 2024-04-02T04:42:31.033 INFO:tasks.workunit.client.0.smithi016.stdout:0/30: mknod d0/ca 0 2024-04-02T04:42:31.033 INFO:tasks.workunit.client.0.smithi016.stdout:0/31: readlink d0/l6 0 2024-04-02T04:42:31.035 INFO:tasks.workunit.client.0.smithi016.stdout:6/74: dwrite d0/f6 [4194304,4194304] 0 2024-04-02T04:42:31.037 INFO:tasks.workunit.client.0.smithi016.stdout:6/75: dread d0/f7 [0,4194304] 0 2024-04-02T04:42:31.037 INFO:tasks.workunit.client.0.smithi016.stdout:9/63: rename d0/d9/la to d0/d3/l10 0 2024-04-02T04:42:31.037 INFO:tasks.workunit.client.0.smithi016.stdout:6/76: dread - d0/fc zero size 2024-04-02T04:42:31.038 INFO:tasks.workunit.client.0.smithi016.stdout:7/76: mkdir d0/d9/d15/d16 0 2024-04-02T04:42:31.042 INFO:tasks.workunit.client.0.smithi016.stdout:9/64: dread d0/f7 [0,4194304] 0 2024-04-02T04:42:31.044 INFO:tasks.workunit.client.0.smithi016.stdout:2/88: mknod d12/c1d 0 2024-04-02T04:42:31.046 INFO:tasks.workunit.client.0.smithi016.stdout:1/96: creat d14/d19/d1d/f1f x:0 0 0 2024-04-02T04:42:31.046 INFO:tasks.workunit.client.0.smithi016.stdout:7/77: dwrite d0/d9/f11 [0,4194304] 0 2024-04-02T04:42:31.047 INFO:tasks.workunit.client.0.smithi016.stdout:0/32: unlink d0/f7 0 2024-04-02T04:42:31.049 INFO:tasks.workunit.client.0.smithi016.stdout:7/78: write d0/d9/fa [920518,53474] 0 2024-04-02T04:42:31.053 INFO:tasks.workunit.client.0.smithi016.stdout:5/83: dwrite d3/df/f1f [0,4194304] 0 2024-04-02T04:42:31.056 INFO:tasks.workunit.client.0.smithi016.stdout:8/75: link d1/fd d1/d6/f1a 0 2024-04-02T04:42:31.062 INFO:tasks.workunit.client.0.smithi016.stdout:3/90: sync 2024-04-02T04:42:31.063 INFO:tasks.workunit.client.0.smithi016.stdout:9/65: rmdir d0 39 2024-04-02T04:42:31.064 INFO:tasks.workunit.client.0.smithi016.stdout:8/76: dwrite d1/d6/f10 [0,4194304] 0 2024-04-02T04:42:31.065 INFO:tasks.workunit.client.0.smithi016.stdout:8/77: truncate d1/f3 1039986 0 2024-04-02T04:42:31.066 INFO:tasks.workunit.client.0.smithi016.stdout:2/89: write d12/f13 [3077757,57190] 0 2024-04-02T04:42:31.068 INFO:tasks.workunit.client.0.smithi016.stdout:1/97: mkdir d14/d19/d20 0 2024-04-02T04:42:31.068 INFO:tasks.workunit.client.0.smithi016.stdout:2/90: read f10 [202560,65122] 0 2024-04-02T04:42:31.068 INFO:tasks.workunit.client.0.smithi016.stdout:1/98: stat f8 0 2024-04-02T04:42:31.069 INFO:tasks.workunit.client.0.smithi016.stdout:1/99: dread - d14/f17 zero size 2024-04-02T04:42:31.078 INFO:tasks.workunit.client.0.smithi016.stdout:5/84: rename d3/df/d18/f19 to d3/df/d18/f24 0 2024-04-02T04:42:31.079 INFO:tasks.workunit.client.0.smithi016.stdout:6/77: getdents d0/d9/df 0 2024-04-02T04:42:31.082 INFO:tasks.workunit.client.0.smithi016.stdout:3/91: mknod d2/c1c 0 2024-04-02T04:42:31.083 INFO:tasks.workunit.client.0.smithi016.stdout:3/92: chown d2/d15/f17 3 1 2024-04-02T04:42:31.084 INFO:tasks.workunit.client.0.smithi016.stdout:8/78: creat d1/d6/f1b x:0 0 0 2024-04-02T04:42:31.090 INFO:tasks.workunit.client.0.smithi016.stdout:3/93: dwrite d2/f14 [0,4194304] 0 2024-04-02T04:42:31.093 INFO:tasks.workunit.client.0.smithi016.stdout:7/79: symlink d0/d9/d15/d16/l17 0 2024-04-02T04:42:31.093 INFO:tasks.workunit.client.0.smithi016.stdout:0/33: write d0/f4 [1326914,21844] 0 2024-04-02T04:42:31.096 INFO:tasks.workunit.client.0.smithi016.stdout:5/85: rmdir d3/df 39 2024-04-02T04:42:31.100 INFO:tasks.workunit.client.0.smithi016.stdout:9/66: fsync d0/d3/f5 0 2024-04-02T04:42:31.103 INFO:tasks.workunit.client.0.smithi016.stdout:8/79: creat d1/f1c x:0 0 0 2024-04-02T04:42:31.105 INFO:tasks.workunit.client.0.smithi016.stdout:4/55: sync 2024-04-02T04:42:31.107 INFO:tasks.workunit.client.0.smithi016.stdout:2/91: creat d12/d14/f1e x:0 0 0 2024-04-02T04:42:31.109 INFO:tasks.workunit.client.0.smithi016.stdout:3/94: chown d2/fb 236898 1 2024-04-02T04:42:31.113 INFO:tasks.workunit.client.0.smithi016.stdout:7/80: rename d0/d9 to d0/d18 0 2024-04-02T04:42:31.114 INFO:tasks.workunit.client.0.smithi016.stdout:0/34: creat d0/fb x:0 0 0 2024-04-02T04:42:31.115 INFO:tasks.workunit.client.0.smithi016.stdout:4/56: dwrite d5/f10 [0,4194304] 0 2024-04-02T04:42:31.118 INFO:tasks.workunit.client.0.smithi016.stdout:5/86: chown d3/f1a 1268 1 2024-04-02T04:42:31.118 INFO:tasks.workunit.client.0.smithi016.stdout:1/100: dwrite f7 [0,4194304] 0 2024-04-02T04:42:31.126 INFO:tasks.workunit.client.0.smithi016.stdout:0/35: rename d0/fb to d0/fc 0 2024-04-02T04:42:31.129 INFO:tasks.workunit.client.0.smithi016.stdout:0/36: fdatasync d0/f4 0 2024-04-02T04:42:31.129 INFO:tasks.workunit.client.0.smithi016.stdout:6/78: dwrite d0/f4 [0,4194304] 0 2024-04-02T04:42:31.134 INFO:tasks.workunit.client.0.smithi016.stdout:0/37: dread - d0/fc zero size 2024-04-02T04:42:31.137 INFO:tasks.workunit.client.0.smithi016.stdout:1/101: dread d14/f16 [4194304,4194304] 0 2024-04-02T04:42:31.140 INFO:tasks.workunit.client.0.smithi016.stdout:9/67: dwrite d0/f7 [0,4194304] 0 2024-04-02T04:42:31.141 INFO:tasks.workunit.client.0.smithi016.stdout:0/38: creat d0/fd x:0 0 0 2024-04-02T04:42:31.142 INFO:tasks.workunit.client.0.smithi016.stdout:2/92: getdents d12/d14 0 2024-04-02T04:42:31.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:30 smithi027 ceph-mon[37550]: pgmap v14: 65 pgs: 65 active+clean; 245 MiB data, 854 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s rd, 9.0 MiB/s wr, 410 op/s 2024-04-02T04:42:31.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:30 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:31.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:30 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:31.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:30 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:42:31.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:30 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:42:31.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:30 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:42:31.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:30 smithi027 ceph-mon[37550]: Adjusting osd_memory_target on smithi027 to 2240M 2024-04-02T04:42:31.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:30 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:31.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:30 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:42:31.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:30 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:42:31.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:30 smithi027 ceph-mon[37550]: Updating smithi016:/etc/ceph/ceph.conf 2024-04-02T04:42:31.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:30 smithi027 ceph-mon[37550]: Updating smithi027:/etc/ceph/ceph.conf 2024-04-02T04:42:31.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:30 smithi027 ceph-mon[37550]: Updating smithi016:/var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:42:31.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:30 smithi027 ceph-mon[37550]: Updating smithi027:/var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:42:31.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:30 smithi027 ceph-mon[37550]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-04-02T04:42:31.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:30 smithi027 ceph-mon[37550]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-04-02T04:42:31.147 INFO:tasks.workunit.client.0.smithi016.stdout:2/93: fsync d12/f13 0 2024-04-02T04:42:31.149 INFO:tasks.workunit.client.0.smithi016.stdout:0/39: creat d0/fe x:0 0 0 2024-04-02T04:42:31.149 INFO:tasks.workunit.client.0.smithi016.stdout:3/95: dread f0 [0,4194304] 0 2024-04-02T04:42:31.151 INFO:tasks.workunit.client.0.smithi016.stdout:1/102: mknod d14/d19/d1e/c21 0 2024-04-02T04:42:31.152 INFO:tasks.workunit.client.0.smithi016.stdout:0/40: fsync d0/f4 0 2024-04-02T04:42:31.155 INFO:tasks.workunit.client.0.smithi016.stdout:8/80: dwrite d1/d6/f1a [0,4194304] 0 2024-04-02T04:42:31.155 INFO:tasks.workunit.client.0.smithi016.stdout:3/96: symlink d2/d15/l1d 0 2024-04-02T04:42:31.158 INFO:tasks.workunit.client.0.smithi016.stdout:1/103: chown d14/d19/d20 807666859 1 2024-04-02T04:42:31.161 INFO:tasks.workunit.client.0.smithi016.stdout:5/87: dwrite d3/df/d18/f1b [0,4194304] 0 2024-04-02T04:42:31.162 INFO:tasks.workunit.client.0.smithi016.stdout:6/79: dwrite d0/fc [0,4194304] 0 2024-04-02T04:42:31.163 INFO:tasks.workunit.client.0.smithi016.stdout:3/97: chown d2/fb 11 1 2024-04-02T04:42:31.167 INFO:tasks.workunit.client.0.smithi016.stdout:8/81: chown d1/le 317 1 2024-04-02T04:42:31.169 INFO:tasks.workunit.client.0.smithi016.stdout:9/68: dwrite d0/d3/f5 [0,4194304] 0 2024-04-02T04:42:31.172 INFO:tasks.workunit.client.0.smithi016.stdout:5/88: mkdir d3/d25 0 2024-04-02T04:42:31.178 INFO:tasks.workunit.client.0.smithi016.stdout:0/41: dwrite d0/fd [0,4194304] 0 2024-04-02T04:42:31.181 INFO:tasks.workunit.client.0.smithi016.stdout:6/80: dread d0/f4 [0,4194304] 0 2024-04-02T04:42:31.183 INFO:tasks.workunit.client.0.smithi016.stdout:1/104: write fd [1537485,32151] 0 2024-04-02T04:42:31.185 INFO:tasks.workunit.client.0.smithi016.stdout:5/89: mkdir d3/df/d18/d26 0 2024-04-02T04:42:31.185 INFO:tasks.workunit.client.0.smithi016.stdout:6/81: chown d0/d9/df 13517622 1 2024-04-02T04:42:31.185 INFO:tasks.workunit.client.0.smithi016.stdout:0/42: rename d0/ca to d0/cf 0 2024-04-02T04:42:31.187 INFO:tasks.workunit.client.0.smithi016.stdout:3/98: write d2/fa [1509424,22265] 0 2024-04-02T04:42:31.188 INFO:tasks.workunit.client.0.smithi016.stdout:8/82: dwrite d1/d6/f10 [0,4194304] 0 2024-04-02T04:42:31.190 INFO:tasks.workunit.client.0.smithi016.stdout:9/69: dwrite d0/d9/fd [0,4194304] 0 2024-04-02T04:42:31.193 INFO:tasks.workunit.client.0.smithi016.stdout:0/43: creat d0/f10 x:0 0 0 2024-04-02T04:42:31.198 INFO:tasks.workunit.client.0.smithi016.stdout:5/90: unlink d3/df/c11 0 2024-04-02T04:42:31.200 INFO:tasks.workunit.client.0.smithi016.stdout:6/82: truncate d0/d9/fd 1506826 0 2024-04-02T04:42:31.200 INFO:tasks.workunit.client.0.smithi016.stdout:9/70: dread d0/f7 [0,4194304] 0 2024-04-02T04:42:31.201 INFO:tasks.workunit.client.0.smithi016.stdout:8/83: dwrite d1/f1c [0,4194304] 0 2024-04-02T04:42:31.206 INFO:tasks.workunit.client.0.smithi016.stdout:0/44: stat d0/l5 0 2024-04-02T04:42:31.209 INFO:tasks.workunit.client.0.smithi016.stdout:5/91: fsync d3/df/d18/f24 0 2024-04-02T04:42:31.217 INFO:tasks.workunit.client.0.smithi016.stdout:8/84: rename d1/f18 to d1/f1d 0 2024-04-02T04:42:31.218 INFO:tasks.workunit.client.0.smithi016.stdout:8/85: chown d1/f19 166214 1 2024-04-02T04:42:31.218 INFO:tasks.workunit.client.0.smithi016.stdout:6/83: rmdir d0/d9 39 2024-04-02T04:42:31.225 INFO:tasks.workunit.client.0.smithi016.stdout:8/86: mkdir d1/d6/d1e 0 2024-04-02T04:42:31.226 INFO:tasks.workunit.client.0.smithi016.stdout:6/84: dwrite d0/f6 [0,4194304] 0 2024-04-02T04:42:31.228 INFO:tasks.workunit.client.0.smithi016.stdout:5/92: getdents d3/d25 0 2024-04-02T04:42:31.229 INFO:tasks.workunit.client.0.smithi016.stdout:3/99: dwrite d2/f7 [0,4194304] 0 2024-04-02T04:42:31.239 INFO:tasks.workunit.client.0.smithi016.stdout:8/87: symlink d1/d6/l1f 0 2024-04-02T04:42:31.239 INFO:tasks.workunit.client.0.smithi016.stdout:9/71: getdents d0/d3 0 2024-04-02T04:42:31.243 INFO:tasks.workunit.client.0.smithi016.stdout:6/85: dread d0/f4 [0,4194304] 0 2024-04-02T04:42:31.246 INFO:tasks.workunit.client.0.smithi016.stdout:9/72: dread d0/d9/fd [0,4194304] 0 2024-04-02T04:42:31.247 INFO:tasks.workunit.client.0.smithi016.stdout:5/93: dwrite d3/df/f22 [0,4194304] 0 2024-04-02T04:42:31.248 INFO:tasks.workunit.client.0.smithi016.stdout:9/73: write d0/d3/f5 [704103,10481] 0 2024-04-02T04:42:31.249 INFO:tasks.workunit.client.0.smithi016.stdout:9/74: truncate d0/d9/fd 4719414 0 2024-04-02T04:42:31.254 INFO:tasks.workunit.client.0.smithi016.stdout:9/75: dread d0/d3/f5 [0,4194304] 0 2024-04-02T04:42:31.257 INFO:tasks.workunit.client.0.smithi016.stdout:9/76: dread d0/d3/f5 [0,4194304] 0 2024-04-02T04:42:31.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:30 smithi016 ceph-mon[28969]: pgmap v14: 65 pgs: 65 active+clean; 245 MiB data, 854 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s rd, 9.0 MiB/s wr, 410 op/s 2024-04-02T04:42:31.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:30 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:31.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:30 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:31.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:30 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:42:31.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:30 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:42:31.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:30 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T04:42:31.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:30 smithi016 ceph-mon[28969]: Adjusting osd_memory_target on smithi027 to 2240M 2024-04-02T04:42:31.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:30 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:31.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:30 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:42:31.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:30 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:42:31.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:30 smithi016 ceph-mon[28969]: Updating smithi016:/etc/ceph/ceph.conf 2024-04-02T04:42:31.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:30 smithi016 ceph-mon[28969]: Updating smithi027:/etc/ceph/ceph.conf 2024-04-02T04:42:31.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:30 smithi016 ceph-mon[28969]: Updating smithi016:/var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:42:31.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:30 smithi016 ceph-mon[28969]: Updating smithi027:/var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:42:31.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:30 smithi016 ceph-mon[28969]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-04-02T04:42:31.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:30 smithi016 ceph-mon[28969]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-04-02T04:42:31.359 INFO:tasks.workunit.client.0.smithi016.stdout:2/94: sync 2024-04-02T04:42:31.359 INFO:tasks.workunit.client.0.smithi016.stdout:4/57: sync 2024-04-02T04:42:31.359 INFO:tasks.workunit.client.0.smithi016.stdout:7/81: sync 2024-04-02T04:42:31.364 INFO:tasks.workunit.client.0.smithi016.stdout:4/58: dwrite d5/f7 [0,4194304] 0 2024-04-02T04:42:31.378 INFO:tasks.workunit.client.0.smithi016.stdout:0/45: getdents d0 0 2024-04-02T04:42:31.388 INFO:tasks.workunit.client.0.smithi016.stdout:7/82: creat d0/d18/f19 x:0 0 0 2024-04-02T04:42:31.389 INFO:tasks.workunit.client.0.smithi016.stdout:5/94: dwrite d3/df/f1e [0,4194304] 0 2024-04-02T04:42:31.394 INFO:tasks.workunit.client.0.smithi016.stdout:3/100: creat d2/d15/d1a/f1e x:0 0 0 2024-04-02T04:42:31.394 INFO:tasks.workunit.client.0.smithi016.stdout:3/101: chown d2/f9 3 1 2024-04-02T04:42:31.396 INFO:tasks.workunit.client.0.smithi016.stdout:0/46: creat d0/f11 x:0 0 0 2024-04-02T04:42:31.396 INFO:tasks.workunit.client.0.smithi016.stdout:0/47: fdatasync d0/f11 0 2024-04-02T04:42:31.405 INFO:tasks.workunit.client.0.smithi016.stdout:5/95: fdatasync d3/fe 0 2024-04-02T04:42:31.406 INFO:tasks.workunit.client.0.smithi016.stdout:7/83: creat d0/d18/d15/f1a x:0 0 0 2024-04-02T04:42:31.408 INFO:tasks.workunit.client.0.smithi016.stdout:2/95: link d12/d14/f1e d12/d14/f1f 0 2024-04-02T04:42:31.412 INFO:tasks.workunit.client.0.smithi016.stdout:7/84: dwrite d0/d18/d15/f1a [0,4194304] 0 2024-04-02T04:42:31.412 INFO:tasks.workunit.client.0.smithi016.stdout:0/48: creat d0/f12 x:0 0 0 2024-04-02T04:42:31.415 INFO:tasks.workunit.client.0.smithi016.stdout:6/86: creat d0/d9/df/f10 x:0 0 0 2024-04-02T04:42:31.416 INFO:tasks.workunit.client.0.smithi016.stdout:6/87: read - d0/d9/df/f10 zero size 2024-04-02T04:42:31.419 INFO:tasks.workunit.client.0.smithi016.stdout:7/85: dwrite d0/d18/f19 [0,4194304] 0 2024-04-02T04:42:31.420 INFO:tasks.workunit.client.0.smithi016.stdout:9/77: creat d0/f11 x:0 0 0 2024-04-02T04:42:31.457 INFO:tasks.workunit.client.0.smithi016.stdout:5/96: symlink d3/df/l27 0 2024-04-02T04:42:31.459 INFO:tasks.workunit.client.0.smithi016.stdout:9/78: fdatasync d0/d9/fd 0 2024-04-02T04:42:31.461 INFO:tasks.workunit.client.0.smithi016.stdout:4/59: getdents d5 0 2024-04-02T04:42:31.463 INFO:tasks.workunit.client.0.smithi016.stdout:4/60: write d5/f10 [2363832,86547] 0 2024-04-02T04:42:31.463 INFO:tasks.workunit.client.0.smithi016.stdout:2/96: mkdir d12/d20 0 2024-04-02T04:42:31.475 INFO:tasks.workunit.client.0.smithi016.stdout:5/97: dread d3/f5 [0,4194304] 0 2024-04-02T04:42:31.478 INFO:tasks.workunit.client.0.smithi016.stdout:4/61: creat d5/d8/f12 x:0 0 0 2024-04-02T04:42:31.479 INFO:tasks.workunit.client.0.smithi016.stdout:4/62: write d5/d8/fd [262652,73615] 0 2024-04-02T04:42:31.481 INFO:tasks.workunit.client.0.smithi016.stdout:2/97: mkdir d12/d14/d21 0 2024-04-02T04:42:31.484 INFO:tasks.workunit.client.0.smithi016.stdout:9/79: mknod d0/c12 0 2024-04-02T04:42:31.484 INFO:tasks.workunit.client.0.smithi016.stdout:9/80: dread - d0/f11 zero size 2024-04-02T04:42:31.485 INFO:tasks.workunit.client.0.smithi016.stdout:9/81: dread - d0/f11 zero size 2024-04-02T04:42:31.487 INFO:tasks.workunit.client.0.smithi016.stdout:4/63: creat d5/d8/f13 x:0 0 0 2024-04-02T04:42:31.494 INFO:tasks.workunit.client.0.smithi016.stdout:4/64: dwrite d5/d8/ff [0,4194304] 0 2024-04-02T04:42:31.494 INFO:tasks.workunit.client.0.smithi016.stdout:5/98: creat d3/f28 x:0 0 0 2024-04-02T04:42:31.494 INFO:tasks.workunit.client.0.smithi016.stdout:4/65: readlink d5/d8/lc 0 2024-04-02T04:42:31.495 INFO:tasks.workunit.client.0.smithi016.stdout:4/66: read - d5/d8/f12 zero size 2024-04-02T04:42:31.495 INFO:tasks.workunit.client.0.smithi016.stdout:4/67: chown d5/de 2 1 2024-04-02T04:42:31.495 INFO:tasks.workunit.client.0.smithi016.stdout:4/68: chown c3 125409 1 2024-04-02T04:42:31.500 INFO:tasks.workunit.client.0.smithi016.stdout:5/99: mknod d3/df/d18/c29 0 2024-04-02T04:42:31.501 INFO:tasks.workunit.client.0.smithi016.stdout:4/69: creat d5/de/f14 x:0 0 0 2024-04-02T04:42:31.502 INFO:tasks.workunit.client.0.smithi016.stdout:4/70: write d5/f10 [5175122,106683] 0 2024-04-02T04:42:31.503 INFO:tasks.workunit.client.0.smithi016.stdout:4/71: rename d5/de to d5/de/d15 22 2024-04-02T04:42:31.504 INFO:tasks.workunit.client.0.smithi016.stdout:4/72: write d5/d8/f13 [919216,110284] 0 2024-04-02T04:42:31.506 INFO:tasks.workunit.client.0.smithi016.stdout:5/100: mkdir d3/d25/d2a 0 2024-04-02T04:42:31.508 INFO:tasks.workunit.client.0.smithi016.stdout:4/73: fsync f4 0 2024-04-02T04:42:31.511 INFO:tasks.workunit.client.0.smithi016.stdout:5/101: write d3/f4 [1456978,63598] 0 2024-04-02T04:42:31.512 INFO:tasks.workunit.client.0.smithi016.stdout:1/105: sync 2024-04-02T04:42:31.512 INFO:tasks.workunit.client.0.smithi016.stdout:4/74: mkdir d5/d16 0 2024-04-02T04:42:31.513 INFO:tasks.workunit.client.0.smithi016.stdout:1/106: write d14/d19/d1d/f1f [16914,20947] 0 2024-04-02T04:42:31.513 INFO:tasks.workunit.client.0.smithi016.stdout:4/75: stat d5/de/c11 0 2024-04-02T04:42:31.514 INFO:tasks.workunit.client.0.smithi016.stdout:1/107: read fa [361214,116195] 0 2024-04-02T04:42:31.516 INFO:tasks.workunit.client.0.smithi016.stdout:5/102: symlink d3/df/d18/l2b 0 2024-04-02T04:42:31.516 INFO:tasks.workunit.client.0.smithi016.stdout:5/103: chown d3/fe 42266 1 2024-04-02T04:42:31.521 INFO:tasks.workunit.client.0.smithi016.stdout:1/108: rename cf to d14/d19/d20/c22 0 2024-04-02T04:42:31.521 INFO:tasks.workunit.client.0.smithi016.stdout:1/109: readlink l1 0 2024-04-02T04:42:31.523 INFO:tasks.workunit.client.0.smithi016.stdout:5/104: mkdir d3/df/d18/d26/d2c 0 2024-04-02T04:42:31.526 INFO:tasks.workunit.client.0.smithi016.stdout:5/105: symlink d3/df/d18/l2d 0 2024-04-02T04:42:31.526 INFO:tasks.workunit.client.0.smithi016.stdout:5/106: dread - d3/df/d18/f24 zero size 2024-04-02T04:42:31.527 INFO:tasks.workunit.client.0.smithi016.stdout:5/107: write d3/fe [1038713,124889] 0 2024-04-02T04:42:31.530 INFO:tasks.workunit.client.0.smithi016.stdout:5/108: symlink d3/d25/l2e 0 2024-04-02T04:42:31.536 INFO:tasks.workunit.client.0.smithi016.stdout:5/109: dwrite d3/f4 [0,4194304] 0 2024-04-02T04:42:31.541 INFO:tasks.workunit.client.0.smithi016.stdout:5/110: mkdir d3/df/d18/d2f 0 2024-04-02T04:42:31.545 INFO:tasks.workunit.client.0.smithi016.stdout:5/111: mkdir d3/df/d18/d30 0 2024-04-02T04:42:31.548 INFO:tasks.workunit.client.0.smithi016.stdout:5/112: creat d3/df/d18/f31 x:0 0 0 2024-04-02T04:42:31.558 INFO:tasks.workunit.client.0.smithi016.stdout:5/113: link d3/df/c15 d3/df/d18/d2f/c32 0 2024-04-02T04:42:31.560 INFO:tasks.workunit.client.0.smithi016.stdout:5/114: readlink d3/l1d 0 2024-04-02T04:42:31.561 INFO:tasks.workunit.client.0.smithi016.stdout:5/115: dread - d3/df/d18/f31 zero size 2024-04-02T04:42:31.561 INFO:tasks.workunit.client.0.smithi016.stdout:5/116: readlink d3/l1d 0 2024-04-02T04:42:31.598 INFO:tasks.workunit.client.0.smithi016.stdout:8/88: truncate d1/f3 1022337 0 2024-04-02T04:42:31.598 INFO:tasks.workunit.client.0.smithi016.stdout:5/117: chown d3/f5 19795274 1 2024-04-02T04:42:31.603 INFO:tasks.workunit.client.0.smithi016.stdout:3/102: dwrite f1 [4194304,4194304] 0 2024-04-02T04:42:31.605 INFO:tasks.workunit.client.0.smithi016.stdout:6/88: write d0/d9/fd [85694,106279] 0 2024-04-02T04:42:31.606 INFO:tasks.workunit.client.0.smithi016.stdout:6/89: chown d0/d8 678144 1 2024-04-02T04:42:31.606 INFO:tasks.workunit.client.0.smithi016.stdout:5/118: creat d3/df/f33 x:0 0 0 2024-04-02T04:42:31.609 INFO:tasks.workunit.client.0.smithi016.stdout:7/86: truncate d0/d18/f19 1244304 0 2024-04-02T04:42:31.610 INFO:tasks.workunit.client.0.smithi016.stdout:3/103: rename d2/f4 to d2/f1f 0 2024-04-02T04:42:31.615 INFO:tasks.workunit.client.0.smithi016.stdout:3/104: creat d2/f20 x:0 0 0 2024-04-02T04:42:31.616 INFO:tasks.workunit.client.0.smithi016.stdout:3/105: write d2/fd [3576418,43165] 0 2024-04-02T04:42:31.617 INFO:tasks.workunit.client.0.smithi016.stdout:6/90: creat d0/d8/f11 x:0 0 0 2024-04-02T04:42:31.618 INFO:tasks.workunit.client.0.smithi016.stdout:3/106: write d2/d15/f17 [697012,10966] 0 2024-04-02T04:42:31.622 INFO:tasks.workunit.client.0.smithi016.stdout:3/107: dread d2/f14 [0,4194304] 0 2024-04-02T04:42:31.623 INFO:tasks.workunit.client.0.smithi016.stdout:5/119: rename d3/df/d18/d30 to d3/d34 0 2024-04-02T04:42:31.625 INFO:tasks.workunit.client.0.smithi016.stdout:6/91: fsync d0/f4 0 2024-04-02T04:42:31.625 INFO:tasks.workunit.client.0.smithi016.stdout:6/92: readlink d0/la 0 2024-04-02T04:42:31.628 INFO:tasks.workunit.client.0.smithi016.stdout:9/82: write d0/f7 [2181149,92167] 0 2024-04-02T04:42:31.630 INFO:tasks.workunit.client.0.smithi016.stdout:2/98: dwrite f8 [0,4194304] 0 2024-04-02T04:42:31.634 INFO:tasks.workunit.client.0.smithi016.stdout:5/120: creat d3/df/d18/f35 x:0 0 0 2024-04-02T04:42:31.635 INFO:tasks.workunit.client.0.smithi016.stdout:5/121: chown d3/f28 63424 1 2024-04-02T04:42:31.637 INFO:tasks.workunit.client.0.smithi016.stdout:9/83: dwrite d0/f7 [0,4194304] 0 2024-04-02T04:42:31.640 INFO:tasks.workunit.client.0.smithi016.stdout:2/99: dwrite ff [0,4194304] 0 2024-04-02T04:42:31.650 INFO:tasks.workunit.client.0.smithi016.stdout:6/93: link d0/f4 d0/d9/f12 0 2024-04-02T04:42:31.658 INFO:tasks.workunit.client.0.smithi016.stdout:9/84: creat d0/d3/f13 x:0 0 0 2024-04-02T04:42:31.666 INFO:tasks.workunit.client.0.smithi016.stdout:2/100: mknod d12/d20/c22 0 2024-04-02T04:42:31.666 INFO:tasks.workunit.client.0.smithi016.stdout:9/85: dwrite d0/f11 [0,4194304] 0 2024-04-02T04:42:31.667 INFO:tasks.workunit.client.0.smithi016.stdout:3/108: getdents d2/d15/d1a 0 2024-04-02T04:42:31.669 INFO:tasks.workunit.client.0.smithi016.stdout:0/49: sync 2024-04-02T04:42:31.671 INFO:tasks.workunit.client.0.smithi016.stdout:5/122: link d3/f28 d3/f36 0 2024-04-02T04:42:31.675 INFO:tasks.workunit.client.0.smithi016.stdout:3/109: creat d2/d15/d1a/f21 x:0 0 0 2024-04-02T04:42:31.676 INFO:tasks.workunit.client.0.smithi016.stdout:3/110: write d2/fa [2485004,106809] 0 2024-04-02T04:42:31.679 INFO:tasks.workunit.client.0.smithi016.stdout:0/50: mkdir d0/d13 0 2024-04-02T04:42:31.680 INFO:tasks.workunit.client.0.smithi016.stdout:0/51: dread - d0/fc zero size 2024-04-02T04:42:31.685 INFO:tasks.workunit.client.0.smithi016.stdout:9/86: link d0/f11 d0/d9/f14 0 2024-04-02T04:42:31.686 INFO:tasks.workunit.client.0.smithi016.stdout:0/52: dwrite d0/f4 [0,4194304] 0 2024-04-02T04:42:31.688 INFO:tasks.workunit.client.0.smithi016.stdout:6/94: getdents d0 0 2024-04-02T04:42:31.689 INFO:tasks.workunit.client.0.smithi016.stdout:3/111: creat d2/d15/d1a/f22 x:0 0 0 2024-04-02T04:42:31.690 INFO:tasks.workunit.client.0.smithi016.stdout:2/101: creat d12/f23 x:0 0 0 2024-04-02T04:42:31.690 INFO:tasks.workunit.client.0.smithi016.stdout:3/112: write d2/f9 [1952973,25864] 0 2024-04-02T04:42:31.699 INFO:tasks.workunit.client.0.smithi016.stdout:6/95: creat d0/f13 x:0 0 0 2024-04-02T04:42:31.703 INFO:tasks.workunit.client.0.smithi016.stdout:3/113: read d2/f14 [2532732,70690] 0 2024-04-02T04:42:31.707 INFO:tasks.workunit.client.0.smithi016.stdout:0/53: creat d0/d13/f14 x:0 0 0 2024-04-02T04:42:31.710 INFO:tasks.workunit.client.0.smithi016.stdout:3/114: creat d2/d15/f23 x:0 0 0 2024-04-02T04:42:31.711 INFO:tasks.workunit.client.0.smithi016.stdout:0/54: fsync d0/fc 0 2024-04-02T04:42:31.714 INFO:tasks.workunit.client.0.smithi016.stdout:3/115: creat d2/d15/f24 x:0 0 0 2024-04-02T04:42:31.716 INFO:tasks.workunit.client.0.smithi016.stdout:0/55: mknod d0/d13/c15 0 2024-04-02T04:42:31.716 INFO:tasks.workunit.client.0.smithi016.stdout:8/89: sync 2024-04-02T04:42:31.716 INFO:tasks.workunit.client.0.smithi016.stdout:4/76: sync 2024-04-02T04:42:31.716 INFO:tasks.workunit.client.0.smithi016.stdout:1/110: sync 2024-04-02T04:42:31.716 INFO:tasks.workunit.client.0.smithi016.stdout:0/56: write d0/f12 [399710,11901] 0 2024-04-02T04:42:31.716 INFO:tasks.workunit.client.0.smithi016.stdout:2/102: link d12/c15 d12/d20/c24 0 2024-04-02T04:42:31.717 INFO:tasks.workunit.client.0.smithi016.stdout:1/111: dread - d14/f1c zero size 2024-04-02T04:42:31.717 INFO:tasks.workunit.client.0.smithi016.stdout:1/112: stat c10 0 2024-04-02T04:42:31.720 INFO:tasks.workunit.client.0.smithi016.stdout:4/77: mkdir d5/d17 0 2024-04-02T04:42:31.721 INFO:tasks.workunit.client.0.smithi016.stdout:0/57: mkdir d0/d16 0 2024-04-02T04:42:31.723 INFO:tasks.workunit.client.0.smithi016.stdout:1/113: creat d14/d19/d1e/f23 x:0 0 0 2024-04-02T04:42:31.723 INFO:tasks.workunit.client.0.smithi016.stdout:0/58: write d0/d13/f14 [570112,96063] 0 2024-04-02T04:42:31.724 INFO:tasks.workunit.client.0.smithi016.stdout:2/103: dwrite d12/f23 [0,4194304] 0 2024-04-02T04:42:31.725 INFO:tasks.workunit.client.0.smithi016.stdout:1/114: chown d14/c1b 105960612 1 2024-04-02T04:42:31.726 INFO:tasks.workunit.client.0.smithi016.stdout:2/104: write d12/f1c [3186452,62876] 0 2024-04-02T04:42:31.727 INFO:tasks.workunit.client.0.smithi016.stdout:2/105: chown d12/d14/f1f 72 1 2024-04-02T04:42:31.728 INFO:tasks.workunit.client.0.smithi016.stdout:4/78: mkdir d5/d17/d18 0 2024-04-02T04:42:31.729 INFO:tasks.workunit.client.0.smithi016.stdout:4/79: dread - d5/d8/f12 zero size 2024-04-02T04:42:31.731 INFO:tasks.workunit.client.0.smithi016.stdout:2/106: creat d12/d20/f25 x:0 0 0 2024-04-02T04:42:31.736 INFO:tasks.workunit.client.0.smithi016.stdout:4/80: dwrite d5/d8/fd [0,4194304] 0 2024-04-02T04:42:31.740 INFO:tasks.workunit.client.0.smithi016.stdout:2/107: dwrite d12/f23 [0,4194304] 0 2024-04-02T04:42:31.742 INFO:tasks.workunit.client.0.smithi016.stdout:4/81: dread d5/d8/fd [0,4194304] 0 2024-04-02T04:42:31.749 INFO:tasks.workunit.client.0.smithi016.stdout:4/82: dwrite d5/de/f14 [0,4194304] 0 2024-04-02T04:42:31.755 INFO:tasks.workunit.client.0.smithi016.stdout:4/83: dwrite d5/f7 [0,4194304] 0 2024-04-02T04:42:31.756 INFO:tasks.workunit.client.0.smithi016.stdout:4/84: chown d5/de/f14 598884891 1 2024-04-02T04:42:31.756 INFO:tasks.workunit.client.0.smithi016.stdout:4/85: stat d5/d8/f13 0 2024-04-02T04:42:31.758 INFO:tasks.workunit.client.0.smithi016.stdout:4/86: write d5/d8/f12 [1006608,994] 0 2024-04-02T04:42:31.761 INFO:tasks.workunit.client.0.smithi016.stdout:7/87: rename d0/d18/f19 to d0/d18/d15/d16/f1b 0 2024-04-02T04:42:31.766 INFO:tasks.workunit.client.0.smithi016.stdout:0/59: rename d0/f11 to d0/d16/f17 0 2024-04-02T04:42:31.771 INFO:tasks.workunit.client.0.smithi016.stdout:1/115: rename f8 to d14/d19/d20/f24 0 2024-04-02T04:42:31.774 INFO:tasks.workunit.client.0.smithi016.stdout:4/87: fdatasync d5/f7 0 2024-04-02T04:42:31.774 INFO:tasks.workunit.client.0.smithi016.stdout:1/116: dread d14/d19/d1d/f1f [0,4194304] 0 2024-04-02T04:42:31.775 INFO:tasks.workunit.client.0.smithi016.stdout:4/88: dread - d5/d8/fb zero size 2024-04-02T04:42:31.775 INFO:tasks.workunit.client.0.smithi016.stdout:1/117: chown f2 25 1 2024-04-02T04:42:31.775 INFO:tasks.workunit.client.0.smithi016.stdout:1/118: chown f11 35583545 1 2024-04-02T04:42:31.783 INFO:tasks.workunit.client.0.smithi016.stdout:1/119: symlink d14/d19/l25 0 2024-04-02T04:42:31.786 INFO:tasks.workunit.client.0.smithi016.stdout:4/89: mknod d5/d17/d18/c19 0 2024-04-02T04:42:31.788 INFO:tasks.workunit.client.0.smithi016.stdout:5/123: rmdir d3/d34 0 2024-04-02T04:42:31.791 INFO:tasks.workunit.client.0.smithi016.stdout:1/120: dwrite fd [0,4194304] 0 2024-04-02T04:42:31.796 INFO:tasks.workunit.client.0.smithi016.stdout:5/124: dread d3/f4 [0,4194304] 0 2024-04-02T04:42:31.796 INFO:tasks.workunit.client.0.smithi016.stdout:4/90: dwrite d5/d8/f12 [0,4194304] 0 2024-04-02T04:42:31.796 INFO:tasks.workunit.client.0.smithi016.stdout:4/91: read f4 [813001,127042] 0 2024-04-02T04:42:31.799 INFO:tasks.workunit.client.0.smithi016.stdout:5/125: mknod d3/c37 0 2024-04-02T04:42:31.800 INFO:tasks.workunit.client.0.smithi016.stdout:5/126: chown d3/d25/l2e 11433 1 2024-04-02T04:42:31.803 INFO:tasks.workunit.client.0.smithi016.stdout:5/127: fsync d3/f1a 0 2024-04-02T04:42:31.812 INFO:tasks.workunit.client.0.smithi016.stdout:9/87: write d0/d9/f14 [4740325,103008] 0 2024-04-02T04:42:31.817 INFO:tasks.workunit.client.0.smithi016.stdout:5/128: unlink d3/df/d18/d2f/c32 0 2024-04-02T04:42:31.818 INFO:tasks.workunit.client.0.smithi016.stdout:5/129: chown d3/f36 4 1 2024-04-02T04:42:31.819 INFO:tasks.workunit.client.0.smithi016.stdout:9/88: dwrite d0/f7 [0,4194304] 0 2024-04-02T04:42:31.823 INFO:tasks.workunit.client.0.smithi016.stdout:9/89: symlink d0/d9/db/l15 0 2024-04-02T04:42:31.831 INFO:tasks.workunit.client.0.smithi016.stdout:9/90: dread d0/d3/f5 [0,4194304] 0 2024-04-02T04:42:31.836 INFO:tasks.workunit.client.0.smithi016.stdout:6/96: mknod d0/d9/c14 0 2024-04-02T04:42:31.839 INFO:tasks.workunit.client.0.smithi016.stdout:9/91: mkdir d0/d16 0 2024-04-02T04:42:31.842 INFO:tasks.workunit.client.0.smithi016.stdout:6/97: dread d0/f7 [0,4194304] 0 2024-04-02T04:42:31.842 INFO:tasks.workunit.client.0.smithi016.stdout:9/92: symlink d0/d3/l17 0 2024-04-02T04:42:31.843 INFO:tasks.workunit.client.0.smithi016.stdout:8/90: dwrite d1/f3 [0,4194304] 0 2024-04-02T04:42:31.846 INFO:tasks.workunit.client.0.smithi016.stdout:6/98: rmdir d0/d9 39 2024-04-02T04:42:31.847 INFO:tasks.workunit.client.0.smithi016.stdout:8/91: creat d1/d6/f20 x:0 0 0 2024-04-02T04:42:31.848 INFO:tasks.workunit.client.0.smithi016.stdout:2/108: getdents d12/d20 0 2024-04-02T04:42:31.852 INFO:tasks.workunit.client.0.smithi016.stdout:8/92: creat d1/d6/f21 x:0 0 0 2024-04-02T04:42:31.855 INFO:tasks.workunit.client.0.smithi016.stdout:8/93: write d1/ff [3646306,122449] 0 2024-04-02T04:42:31.855 INFO:tasks.workunit.client.0.smithi016.stdout:6/99: dread - d0/d9/df/f10 zero size 2024-04-02T04:42:31.856 INFO:tasks.workunit.client.0.smithi016.stdout:2/109: dwrite d12/d20/f25 [0,4194304] 0 2024-04-02T04:42:31.860 INFO:tasks.workunit.client.0.smithi016.stdout:6/100: unlink d0/d9/f12 0 2024-04-02T04:42:31.861 INFO:tasks.workunit.client.0.smithi016.stdout:2/110: fdatasync f10 0 2024-04-02T04:42:31.861 INFO:tasks.workunit.client.0.smithi016.stdout:2/111: chown f8 13524929 1 2024-04-02T04:42:31.863 INFO:tasks.workunit.client.0.smithi016.stdout:2/112: write d12/f13 [5205737,82389] 0 2024-04-02T04:42:31.870 INFO:tasks.workunit.client.0.smithi016.stdout:3/116: sync 2024-04-02T04:42:31.871 INFO:tasks.workunit.client.0.smithi016.stdout:6/101: dwrite d0/f6 [8388608,4194304] 0 2024-04-02T04:42:31.874 INFO:tasks.workunit.client.0.smithi016.stdout:7/88: rename d0/d18/d15/d16/f1b to d0/f1c 0 2024-04-02T04:42:31.878 INFO:tasks.workunit.client.0.smithi016.stdout:2/113: mkdir d12/d26 0 2024-04-02T04:42:31.880 INFO:tasks.workunit.client.0.smithi016.stdout:0/60: truncate d0/fd 1561059 0 2024-04-02T04:42:31.884 INFO:tasks.workunit.client.0.smithi016.stdout:0/61: dread d0/f12 [0,4194304] 0 2024-04-02T04:42:31.884 INFO:tasks.workunit.client.0.smithi016.stdout:4/92: rename d5/d8/f13 to d5/d17/d18/f1a 0 2024-04-02T04:42:31.884 INFO:tasks.workunit.client.0.smithi016.stdout:4/93: chown d5/d17/d18 244438639 1 2024-04-02T04:42:31.885 INFO:tasks.workunit.client.0.smithi016.stdout:3/117: creat d2/d15/d16/f25 x:0 0 0 2024-04-02T04:42:31.888 INFO:tasks.workunit.client.0.smithi016.stdout:6/102: symlink d0/d8/l15 0 2024-04-02T04:42:31.892 INFO:tasks.workunit.client.0.smithi016.stdout:0/62: fsync d0/d16/f17 0 2024-04-02T04:42:31.895 INFO:tasks.workunit.client.0.smithi016.stdout:3/118: creat d2/f26 x:0 0 0 2024-04-02T04:42:31.895 INFO:tasks.workunit.client.0.smithi016.stdout:2/114: getdents d12/d20 0 2024-04-02T04:42:31.896 INFO:tasks.workunit.client.0.smithi016.stdout:6/103: mkdir d0/d9/d16 0 2024-04-02T04:42:31.899 INFO:tasks.workunit.client.0.smithi016.stdout:0/63: creat d0/d16/f18 x:0 0 0 2024-04-02T04:42:31.900 INFO:tasks.workunit.client.0.smithi016.stdout:4/94: link d5/d17/d18/f1a d5/d16/f1b 0 2024-04-02T04:42:31.901 INFO:tasks.workunit.client.0.smithi016.stdout:2/115: dread f8 [0,4194304] 0 2024-04-02T04:42:31.902 INFO:tasks.workunit.client.0.smithi016.stdout:3/119: dwrite d2/d15/d1a/f1e [0,4194304] 0 2024-04-02T04:42:31.903 INFO:tasks.workunit.client.0.smithi016.stdout:3/120: write d2/f7 [684077,33610] 0 2024-04-02T04:42:31.910 INFO:tasks.workunit.client.0.smithi016.stdout:5/130: truncate d3/df/f1e 697552 0 2024-04-02T04:42:31.911 INFO:tasks.workunit.client.0.smithi016.stdout:3/121: dwrite d2/f26 [0,4194304] 0 2024-04-02T04:42:31.912 INFO:tasks.workunit.client.0.smithi016.stdout:3/122: chown d2/le 117887688 1 2024-04-02T04:42:31.913 INFO:tasks.workunit.client.0.smithi016.stdout:6/104: mknod d0/c17 0 2024-04-02T04:42:31.923 INFO:tasks.workunit.client.0.smithi016.stdout:5/131: rename d3/df/d18/f24 to d3/df/d18/f38 0 2024-04-02T04:42:31.924 INFO:tasks.workunit.client.0.smithi016.stdout:5/132: truncate d3/f4 4762442 0 2024-04-02T04:42:31.925 INFO:tasks.workunit.client.0.smithi016.stdout:3/123: unlink d2/fb 0 2024-04-02T04:42:31.927 INFO:tasks.workunit.client.0.smithi016.stdout:6/105: mknod d0/d9/df/c18 0 2024-04-02T04:42:31.929 INFO:tasks.workunit.client.0.smithi016.stdout:3/124: dread d2/f13 [0,4194304] 0 2024-04-02T04:42:31.929 INFO:tasks.workunit.client.0.smithi016.stdout:3/125: chown d2/d15/c19 4267 1 2024-04-02T04:42:31.930 INFO:tasks.workunit.client.0.smithi016.stdout:3/126: readlink d2/d15/l1d 0 2024-04-02T04:42:31.931 INFO:tasks.workunit.client.0.smithi016.stdout:5/133: chown d3/df/f1c 265691 1 2024-04-02T04:42:31.934 INFO:tasks.workunit.client.0.smithi016.stdout:4/95: getdents d5 0 2024-04-02T04:42:31.935 INFO:tasks.workunit.client.0.smithi016.stdout:4/96: chown d5/d17 1009 1 2024-04-02T04:42:31.937 INFO:tasks.workunit.client.0.smithi016.stdout:3/127: creat d2/d15/d16/f27 x:0 0 0 2024-04-02T04:42:31.938 INFO:tasks.workunit.client.0.smithi016.stdout:3/128: dread - d2/d15/d16/f25 zero size 2024-04-02T04:42:31.942 INFO:tasks.workunit.client.0.smithi016.stdout:4/97: fdatasync d5/d16/f1b 0 2024-04-02T04:42:31.944 INFO:tasks.workunit.client.0.smithi016.stdout:6/106: creat d0/d9/d16/f19 x:0 0 0 2024-04-02T04:42:31.944 INFO:tasks.workunit.client.0.smithi016.stdout:3/129: dwrite d2/d15/f1b [0,4194304] 0 2024-04-02T04:42:31.950 INFO:tasks.workunit.client.0.smithi016.stdout:5/134: rename d3/cb to d3/df/d18/c39 0 2024-04-02T04:42:31.951 INFO:tasks.workunit.client.0.smithi016.stdout:1/121: sync 2024-04-02T04:42:31.954 INFO:tasks.workunit.client.0.smithi016.stdout:6/107: dwrite d0/d9/d16/f19 [0,4194304] 0 2024-04-02T04:42:31.955 INFO:tasks.workunit.client.0.smithi016.stdout:7/89: dread d0/f1c [0,4194304] 0 2024-04-02T04:42:31.955 INFO:tasks.workunit.client.0.smithi016.stdout:4/98: unlink d5/d8/f12 0 2024-04-02T04:42:31.957 INFO:tasks.workunit.client.0.smithi016.stdout:7/90: truncate d0/d18/d15/fc 4991639 0 2024-04-02T04:42:31.960 INFO:tasks.workunit.client.0.smithi016.stdout:7/91: write d0/d18/d15/fc [1757044,66653] 0 2024-04-02T04:42:31.962 INFO:tasks.workunit.client.0.smithi016.stdout:5/135: rename d3/l23 to d3/df/d18/d2f/l3a 0 2024-04-02T04:42:31.962 INFO:tasks.workunit.client.0.smithi016.stdout:3/130: dread d2/f13 [0,4194304] 0 2024-04-02T04:42:31.962 INFO:tasks.workunit.client.0.smithi016.stdout:4/99: rename d5/d8 to d5/d8/d1c 22 2024-04-02T04:42:31.962 INFO:tasks.workunit.client.0.smithi016.stdout:1/122: rmdir d14/d19/d1d 39 2024-04-02T04:42:32.016 INFO:tasks.workunit.client.0.smithi016.stdout:4/100: read d5/d8/fd [4096247,13011] 0 2024-04-02T04:42:32.017 INFO:tasks.workunit.client.0.smithi016.stdout:7/92: creat d0/d18/d15/f1d x:0 0 0 2024-04-02T04:42:32.018 INFO:tasks.workunit.client.0.smithi016.stdout:5/136: write d3/df/d18/f38 [133950,109350] 0 2024-04-02T04:42:32.020 INFO:tasks.workunit.client.0.smithi016.stdout:0/64: write d0/fd [922413,130458] 0 2024-04-02T04:42:32.021 INFO:tasks.workunit.client.0.smithi016.stdout:0/65: dread - d0/d16/f18 zero size 2024-04-02T04:42:32.022 INFO:tasks.workunit.client.0.smithi016.stdout:4/101: mkdir d5/d1d 0 2024-04-02T04:42:32.022 INFO:tasks.workunit.client.0.smithi016.stdout:3/131: dread d2/f9 [0,4194304] 0 2024-04-02T04:42:32.024 INFO:tasks.workunit.client.0.smithi016.stdout:3/132: truncate d2/d15/f23 183575 0 2024-04-02T04:42:32.024 INFO:tasks.workunit.client.0.smithi016.stdout:3/133: write d2/d15/d1a/f1e [124133,96169] 0 2024-04-02T04:42:32.027 INFO:tasks.workunit.client.0.smithi016.stdout:0/66: creat d0/f19 x:0 0 0 2024-04-02T04:42:32.033 INFO:tasks.workunit.client.0.smithi016.stdout:3/134: creat d2/f28 x:0 0 0 2024-04-02T04:42:32.035 INFO:tasks.workunit.client.0.smithi016.stdout:5/137: creat d3/f3b x:0 0 0 2024-04-02T04:42:32.038 INFO:tasks.workunit.client.0.smithi016.stdout:3/135: dwrite d2/f3 [4194304,4194304] 0 2024-04-02T04:42:32.039 INFO:tasks.workunit.client.0.smithi016.stdout:1/123: write fa [1767274,105768] 0 2024-04-02T04:42:32.040 INFO:tasks.workunit.client.0.smithi016.stdout:4/102: getdents d5/d17/d18 0 2024-04-02T04:42:32.043 INFO:tasks.workunit.client.0.smithi016.stdout:1/124: rmdir d14/d19/d20 39 2024-04-02T04:42:32.044 INFO:tasks.workunit.client.0.smithi016.stdout:1/125: chown f11 1539448 1 2024-04-02T04:42:32.044 INFO:tasks.workunit.client.0.smithi016.stdout:5/138: creat d3/df/f3c x:0 0 0 2024-04-02T04:42:32.044 INFO:tasks.workunit.client.0.smithi016.stdout:7/93: truncate d0/d18/d15/f1a 3142327 0 2024-04-02T04:42:32.046 INFO:tasks.workunit.client.0.smithi016.stdout:3/136: dwrite d2/f11 [0,4194304] 0 2024-04-02T04:42:32.046 INFO:tasks.workunit.client.0.smithi016.stdout:1/126: chown d14/d19/l25 558386 1 2024-04-02T04:42:32.048 INFO:tasks.workunit.client.0.smithi016.stdout:4/103: mkdir d5/d1d/d1e 0 2024-04-02T04:42:32.049 INFO:tasks.workunit.client.0.smithi016.stdout:5/139: write d3/df/f1f [2681282,36775] 0 2024-04-02T04:42:32.050 INFO:tasks.workunit.client.0.smithi016.stdout:9/93: sync 2024-04-02T04:42:32.052 INFO:tasks.workunit.client.0.smithi016.stdout:7/94: symlink d0/d18/d15/l1e 0 2024-04-02T04:42:32.053 INFO:tasks.workunit.client.0.smithi016.stdout:3/137: creat d2/d15/d1a/f29 x:0 0 0 2024-04-02T04:42:32.055 INFO:tasks.workunit.client.0.smithi016.stdout:3/138: chown d2/fd 2507 1 2024-04-02T04:42:32.058 INFO:tasks.workunit.client.0.smithi016.stdout:4/104: dread d5/d17/d18/f1a [0,4194304] 0 2024-04-02T04:42:32.060 INFO:tasks.workunit.client.0.smithi016.stdout:5/140: mknod d3/df/d18/d26/d2c/c3d 0 2024-04-02T04:42:32.061 INFO:tasks.workunit.client.0.smithi016.stdout:9/94: dwrite d0/d9/f14 [0,4194304] 0 2024-04-02T04:42:32.062 INFO:tasks.workunit.client.0.smithi016.stdout:3/139: symlink d2/d15/d1a/l2a 0 2024-04-02T04:42:32.063 INFO:tasks.workunit.client.0.smithi016.stdout:5/141: write d3/df/f33 [996588,27783] 0 2024-04-02T04:42:32.066 INFO:tasks.workunit.client.0.smithi016.stdout:9/95: write d0/f11 [2378058,61799] 0 2024-04-02T04:42:32.068 INFO:tasks.workunit.client.0.smithi016.stdout:5/142: readlink d3/d25/l2e 0 2024-04-02T04:42:32.070 INFO:tasks.workunit.client.0.smithi016.stdout:1/127: getdents d14/d19/d1e 0 2024-04-02T04:42:32.070 INFO:tasks.workunit.client.0.smithi016.stdout:9/96: write d0/d3/f13 [375551,37484] 0 2024-04-02T04:42:32.072 INFO:tasks.workunit.client.0.smithi016.stdout:1/128: dread - d14/f1c zero size 2024-04-02T04:42:32.073 INFO:tasks.workunit.client.0.smithi016.stdout:4/105: dwrite d5/d17/d18/f1a [0,4194304] 0 2024-04-02T04:42:32.073 INFO:tasks.workunit.client.0.smithi016.stdout:7/95: write d0/f1c [2255631,54876] 0 2024-04-02T04:42:32.076 INFO:tasks.workunit.client.0.smithi016.stdout:1/129: write fd [1207371,33648] 0 2024-04-02T04:42:32.078 INFO:tasks.workunit.client.0.smithi016.stdout:9/97: mknod d0/d9/c18 0 2024-04-02T04:42:32.080 INFO:tasks.workunit.client.0.smithi016.stdout:9/98: chown d0/d9/db/lf 8352 1 2024-04-02T04:42:32.082 INFO:tasks.workunit.client.0.smithi016.stdout:3/140: truncate d2/f7 346717 0 2024-04-02T04:42:32.084 INFO:tasks.workunit.client.0.smithi016.stdout:4/106: dwrite d5/d16/f1b [0,4194304] 0 2024-04-02T04:42:32.085 INFO:tasks.workunit.client.0.smithi016.stdout:7/96: mknod d0/d18/c1f 0 2024-04-02T04:42:32.094 INFO:tasks.workunit.client.0.smithi016.stdout:1/130: creat d14/d19/d20/f26 x:0 0 0 2024-04-02T04:42:32.094 INFO:tasks.workunit.client.0.smithi016.stdout:1/131: readlink d14/d19/l25 0 2024-04-02T04:42:32.094 INFO:tasks.workunit.client.0.smithi016.stdout:1/132: dread - f11 zero size 2024-04-02T04:42:32.096 INFO:tasks.workunit.client.0.smithi016.stdout:7/97: dread d0/d18/fa [0,4194304] 0 2024-04-02T04:42:32.096 INFO:tasks.workunit.client.0.smithi016.stdout:7/98: chown d0/f1c 175795816 1 2024-04-02T04:42:32.098 INFO:tasks.workunit.client.0.smithi016.stdout:7/99: write d0/d18/d15/f1d [819842,43931] 0 2024-04-02T04:42:32.102 INFO:tasks.workunit.client.0.smithi016.stdout:1/133: link c12 d14/c27 0 2024-04-02T04:42:32.102 INFO:tasks.workunit.client.0.smithi016.stdout:1/134: chown d14/d19 200899108 1 2024-04-02T04:42:32.104 INFO:tasks.workunit.client.0.smithi016.stdout:5/143: truncate d3/df/d18/f1b 3771538 0 2024-04-02T04:42:32.106 INFO:tasks.workunit.client.0.smithi016.stdout:1/135: rename d14/f16 to d14/f28 0 2024-04-02T04:42:32.107 INFO:tasks.workunit.client.0.smithi016.stdout:1/136: chown fd 7 1 2024-04-02T04:42:32.107 INFO:tasks.workunit.client.0.smithi016.stdout:4/107: fdatasync d5/d16/f1b 0 2024-04-02T04:42:32.108 INFO:tasks.workunit.client.0.smithi016.stdout:4/108: chown d5/d8/ff 1799 1 2024-04-02T04:42:32.108 INFO:tasks.workunit.client.0.smithi016.stdout:8/94: sync 2024-04-02T04:42:32.108 INFO:tasks.workunit.client.0.smithi016.stdout:2/116: sync 2024-04-02T04:42:32.108 INFO:tasks.workunit.client.0.smithi016.stdout:7/100: link d0/f1c d0/d18/d15/d16/f20 0 2024-04-02T04:42:32.109 INFO:tasks.workunit.client.0.smithi016.stdout:4/109: fdatasync d5/d17/d18/f1a 0 2024-04-02T04:42:32.111 INFO:tasks.workunit.client.0.smithi016.stdout:7/101: chown d0/d18/d15/l1e 3241798 1 2024-04-02T04:42:32.111 INFO:tasks.workunit.client.0.smithi016.stdout:9/99: truncate d0/d3/f13 134994 0 2024-04-02T04:42:32.113 INFO:tasks.workunit.client.0.smithi016.stdout:8/95: dread d1/f3 [0,4194304] 0 2024-04-02T04:42:32.114 INFO:tasks.workunit.client.0.smithi016.stdout:4/110: dread d5/f7 [0,4194304] 0 2024-04-02T04:42:32.114 INFO:tasks.workunit.client.0.smithi016.stdout:3/141: truncate d2/fd 578500 0 2024-04-02T04:42:32.116 INFO:tasks.workunit.client.0.smithi016.stdout:7/102: mkdir d0/d21 0 2024-04-02T04:42:32.117 INFO:tasks.workunit.client.0.smithi016.stdout:7/103: fsync d0/d18/d15/fc 0 2024-04-02T04:42:32.119 INFO:tasks.workunit.client.0.smithi016.stdout:9/100: creat d0/d9/f19 x:0 0 0 2024-04-02T04:42:32.120 INFO:tasks.workunit.client.0.smithi016.stdout:4/111: dwrite d5/f7 [0,4194304] 0 2024-04-02T04:42:32.120 INFO:tasks.workunit.client.0.smithi016.stdout:9/101: fsync d0/f7 0 2024-04-02T04:42:32.121 INFO:tasks.workunit.client.0.smithi016.stdout:5/144: creat d3/f3e x:0 0 0 2024-04-02T04:42:32.121 INFO:tasks.workunit.client.0.smithi016.stdout:1/137: fsync f6 0 2024-04-02T04:42:32.123 INFO:tasks.workunit.client.0.smithi016.stdout:8/96: write d1/f3 [997224,66682] 0 2024-04-02T04:42:32.125 INFO:tasks.workunit.client.0.smithi016.stdout:5/145: stat d3/df/d18/d26/d2c/c3d 0 2024-04-02T04:42:32.126 INFO:tasks.workunit.client.0.smithi016.stdout:3/142: symlink d2/d15/d16/l2b 0 2024-04-02T04:42:32.128 INFO:tasks.workunit.client.0.smithi016.stdout:5/146: fsync f1 0 2024-04-02T04:42:32.128 INFO:tasks.workunit.client.0.smithi016.stdout:8/97: dread - d1/f1d zero size 2024-04-02T04:42:32.128 INFO:tasks.workunit.client.0.smithi016.stdout:7/104: dread d0/f1c [0,4194304] 0 2024-04-02T04:42:32.129 INFO:tasks.workunit.client.0.smithi016.stdout:8/98: stat d1/c16 0 2024-04-02T04:42:32.129 INFO:tasks.workunit.client.0.smithi016.stdout:1/138: truncate d14/d19/d1e/f23 326418 0 2024-04-02T04:42:32.133 INFO:tasks.workunit.client.0.smithi016.stdout:8/99: read - d1/d6/f21 zero size 2024-04-02T04:42:32.134 INFO:tasks.workunit.client.0.smithi016.stdout:4/112: dread d5/d17/d18/f1a [0,4194304] 0 2024-04-02T04:42:32.134 INFO:tasks.workunit.client.0.smithi016.stdout:9/102: dread d0/d3/f5 [0,4194304] 0 2024-04-02T04:42:32.136 INFO:tasks.workunit.client.0.smithi016.stdout:9/103: dread - d0/d9/f19 zero size 2024-04-02T04:42:32.137 INFO:tasks.workunit.client.0.smithi016.stdout:4/113: write d5/d8/ff [337642,15000] 0 2024-04-02T04:42:32.139 INFO:tasks.workunit.client.0.smithi016.stdout:9/104: write d0/f11 [4323706,26757] 0 2024-04-02T04:42:32.140 INFO:tasks.workunit.client.0.smithi016.stdout:5/147: creat d3/df/d18/d2f/f3f x:0 0 0 2024-04-02T04:42:32.140 INFO:tasks.workunit.client.0.smithi016.stdout:1/139: dwrite fb [0,4194304] 0 2024-04-02T04:42:32.142 INFO:tasks.workunit.client.0.smithi016.stdout:4/114: write d5/d8/ff [446653,47482] 0 2024-04-02T04:42:32.142 INFO:tasks.workunit.client.0.smithi016.stdout:9/105: chown d0/d9/db 161514532 1 2024-04-02T04:42:32.142 INFO:tasks.workunit.client.0.smithi016.stdout:3/143: link d2/d15/d1a/f29 d2/d15/d16/f2c 0 2024-04-02T04:42:32.147 INFO:tasks.workunit.client.0.smithi016.stdout:1/140: dread fd [0,4194304] 0 2024-04-02T04:42:32.154 INFO:tasks.workunit.client.0.smithi016.stdout:8/100: symlink d1/d6/d1e/l22 0 2024-04-02T04:42:32.157 INFO:tasks.workunit.client.0.smithi016.stdout:7/105: dwrite d0/f1c [0,4194304] 0 2024-04-02T04:42:32.157 INFO:tasks.workunit.client.0.smithi016.stdout:7/106: stat d0/d18/d15/fc 0 2024-04-02T04:42:32.159 INFO:tasks.workunit.client.0.smithi016.stdout:3/144: mknod d2/d15/c2d 0 2024-04-02T04:42:32.164 INFO:tasks.workunit.client.0.smithi016.stdout:5/148: mkdir d3/d40 0 2024-04-02T04:42:32.164 INFO:tasks.workunit.client.0.smithi016.stdout:5/149: readlink d3/d25/l2e 0 2024-04-02T04:42:32.169 INFO:tasks.workunit.client.0.smithi016.stdout:9/106: mkdir d0/d16/d1a 0 2024-04-02T04:42:32.170 INFO:tasks.workunit.client.0.smithi016.stdout:8/101: symlink d1/d6/l23 0 2024-04-02T04:42:32.173 INFO:tasks.workunit.client.0.smithi016.stdout:7/107: rename d0/f1c to d0/f22 0 2024-04-02T04:42:32.174 INFO:tasks.workunit.client.0.smithi016.stdout:3/145: mkdir d2/d2e 0 2024-04-02T04:42:32.178 INFO:tasks.workunit.client.0.smithi016.stdout:1/141: mkdir d14/d19/d1d/d29 0 2024-04-02T04:42:32.179 INFO:tasks.workunit.client.0.smithi016.stdout:3/146: write f0 [55928,98960] 0 2024-04-02T04:42:32.179 INFO:tasks.workunit.client.0.smithi016.stdout:9/107: rename d0/d9/fd to d0/d16/f1b 0 2024-04-02T04:42:32.182 INFO:tasks.workunit.client.0.smithi016.stdout:7/108: dwrite d0/d18/d15/f1d [0,4194304] 0 2024-04-02T04:42:32.190 INFO:tasks.workunit.client.0.smithi016.stdout:8/102: rename d1/d6/d1e to d1/d24 0 2024-04-02T04:42:32.190 INFO:tasks.workunit.client.0.smithi016.stdout:8/103: dread - d1/d6/f21 zero size 2024-04-02T04:42:32.191 INFO:tasks.workunit.client.0.smithi016.stdout:8/104: readlink d1/le 0 2024-04-02T04:42:32.192 INFO:tasks.workunit.client.0.smithi016.stdout:8/105: write d1/d6/f1a [2738089,91550] 0 2024-04-02T04:42:32.195 INFO:tasks.workunit.client.0.smithi016.stdout:1/142: rename d14/d19/l25 to d14/d19/d20/l2a 0 2024-04-02T04:42:32.200 INFO:tasks.workunit.client.0.smithi016.stdout:0/67: sync 2024-04-02T04:42:32.200 INFO:tasks.workunit.client.0.smithi016.stdout:6/108: sync 2024-04-02T04:42:32.200 INFO:tasks.workunit.client.0.smithi016.stdout:8/106: creat d1/d6/f25 x:0 0 0 2024-04-02T04:42:32.201 INFO:tasks.workunit.client.0.smithi016.stdout:3/147: rename d2/d15/d1a/f21 to d2/d2e/f2f 0 2024-04-02T04:42:32.206 INFO:tasks.workunit.client.0.smithi016.stdout:8/107: dwrite d1/f1c [4194304,4194304] 0 2024-04-02T04:42:32.207 INFO:tasks.workunit.client.0.smithi016.stdout:4/115: truncate d5/d8/fd 313431 0 2024-04-02T04:42:32.210 INFO:tasks.workunit.client.0.smithi016.stdout:0/68: mkdir d0/d16/d1a 0 2024-04-02T04:42:32.211 INFO:tasks.workunit.client.0.smithi016.stdout:0/69: truncate d0/d16/f18 414172 0 2024-04-02T04:42:32.211 INFO:tasks.workunit.client.0.smithi016.stdout:0/70: dread - d0/fe zero size 2024-04-02T04:42:32.212 INFO:tasks.workunit.client.0.smithi016.stdout:3/148: creat d2/d2e/f30 x:0 0 0 2024-04-02T04:42:32.217 INFO:tasks.workunit.client.0.smithi016.stdout:1/143: truncate f6 267192 0 2024-04-02T04:42:32.218 INFO:tasks.workunit.client.0.smithi016.stdout:7/109: getdents d0/d18/d15 0 2024-04-02T04:42:32.219 INFO:tasks.workunit.client.0.smithi016.stdout:6/109: unlink d0/la 0 2024-04-02T04:42:32.222 INFO:tasks.workunit.client.0.smithi016.stdout:8/108: symlink d1/d24/l26 0 2024-04-02T04:42:32.223 INFO:tasks.workunit.client.0.smithi016.stdout:6/110: dread d0/d9/fd [0,4194304] 0 2024-04-02T04:42:32.223 INFO:tasks.workunit.client.0.smithi016.stdout:8/109: dread - d1/d6/f21 zero size 2024-04-02T04:42:32.223 INFO:tasks.workunit.client.0.smithi016.stdout:4/116: mkdir d5/d17/d18/d1f 0 2024-04-02T04:42:32.224 INFO:tasks.workunit.client.0.smithi016.stdout:5/150: truncate d3/f5 3878065 0 2024-04-02T04:42:32.224 INFO:tasks.workunit.client.0.smithi016.stdout:0/71: symlink d0/d16/l1b 0 2024-04-02T04:42:32.225 INFO:tasks.workunit.client.0.smithi016.stdout:5/151: write d3/f1a [74407,109104] 0 2024-04-02T04:42:32.225 INFO:tasks.workunit.client.0.smithi016.stdout:3/149: creat d2/d15/f31 x:0 0 0 2024-04-02T04:42:32.232 INFO:tasks.workunit.client.0.smithi016.stdout:6/111: rmdir d0/d9 39 2024-04-02T04:42:32.234 INFO:tasks.workunit.client.0.smithi016.stdout:8/110: unlink d1/c16 0 2024-04-02T04:42:32.235 INFO:tasks.workunit.client.0.smithi016.stdout:3/150: dwrite d2/f3 [0,4194304] 0 2024-04-02T04:42:32.236 INFO:tasks.workunit.client.0.smithi016.stdout:8/111: fsync d1/ff 0 2024-04-02T04:42:32.237 INFO:tasks.workunit.client.0.smithi016.stdout:9/108: dwrite d0/d3/f5 [4194304,4194304] 0 2024-04-02T04:42:32.237 INFO:tasks.workunit.client.0.smithi016.stdout:8/112: stat d1/c17 0 2024-04-02T04:42:32.238 INFO:tasks.workunit.client.0.smithi016.stdout:8/113: read - d1/f15 zero size 2024-04-02T04:42:32.243 INFO:tasks.workunit.client.0.smithi016.stdout:2/117: sync 2024-04-02T04:42:32.261 INFO:tasks.workunit.client.0.smithi016.stdout:5/152: symlink d3/df/d18/d2f/l41 0 2024-04-02T04:42:32.261 INFO:tasks.workunit.client.0.smithi016.stdout:7/110: symlink d0/d21/l23 0 2024-04-02T04:42:32.262 INFO:tasks.workunit.client.0.smithi016.stdout:8/114: dread - d1/f15 zero size 2024-04-02T04:42:32.262 INFO:tasks.workunit.client.0.smithi016.stdout:6/112: stat d0/d9 0 2024-04-02T04:42:32.263 INFO:tasks.workunit.client.0.smithi016.stdout:4/117: creat d5/d16/f20 x:0 0 0 2024-04-02T04:42:32.264 INFO:tasks.workunit.client.0.smithi016.stdout:9/109: creat d0/d9/db/f1c x:0 0 0 2024-04-02T04:42:32.266 INFO:tasks.workunit.client.0.smithi016.stdout:0/72: write d0/f12 [364237,114956] 0 2024-04-02T04:42:32.266 INFO:tasks.workunit.client.0.smithi016.stdout:4/118: write d5/de/f14 [1325792,39906] 0 2024-04-02T04:42:32.274 INFO:tasks.workunit.client.0.smithi016.stdout:8/115: dwrite d1/d6/f10 [0,4194304] 0 2024-04-02T04:42:32.275 INFO:tasks.workunit.client.0.smithi016.stdout:1/144: dwrite d14/d19/d1d/f1f [0,4194304] 0 2024-04-02T04:42:32.277 INFO:tasks.workunit.client.0.smithi016.stdout:9/110: dread d0/d9/f14 [0,4194304] 0 2024-04-02T04:42:32.280 INFO:tasks.workunit.client.0.smithi016.stdout:0/73: dwrite d0/f12 [0,4194304] 0 2024-04-02T04:42:32.285 INFO:tasks.workunit.client.0.smithi016.stdout:8/116: dwrite d1/d6/f25 [0,4194304] 0 2024-04-02T04:42:32.287 INFO:tasks.workunit.client.0.smithi016.stdout:8/117: chown d1/d6/f1a 93716315 1 2024-04-02T04:42:32.289 INFO:tasks.workunit.client.0.smithi016.stdout:8/118: truncate d1/d6/f25 4953202 0 2024-04-02T04:42:32.343 INFO:tasks.workunit.client.0.smithi016.stdout:3/151: sync 2024-04-02T04:42:32.343 INFO:tasks.workunit.client.0.smithi016.stdout:3/152: readlink d2/d15/d16/l2b 0 2024-04-02T04:42:32.413 INFO:tasks.workunit.client.0.smithi016.stdout:5/153: creat d3/df/d18/f42 x:0 0 0 2024-04-02T04:42:32.420 INFO:tasks.workunit.client.0.smithi016.stdout:5/154: dwrite d3/df/d18/f38 [0,4194304] 0 2024-04-02T04:42:32.421 INFO:tasks.workunit.client.0.smithi016.stdout:4/119: rename d5/d17/d18/f1a to d5/d1d/f21 0 2024-04-02T04:42:32.431 INFO:tasks.workunit.client.0.smithi016.stdout:8/119: symlink d1/d24/l27 0 2024-04-02T04:42:32.431 INFO:tasks.workunit.client.0.smithi016.stdout:8/120: fdatasync d1/d6/f1b 0 2024-04-02T04:42:32.433 INFO:tasks.workunit.client.0.smithi016.stdout:3/153: symlink d2/d2e/l32 0 2024-04-02T04:42:32.434 INFO:tasks.workunit.client.0.smithi016.stdout:3/154: write d2/f26 [236075,30117] 0 2024-04-02T04:42:32.435 INFO:tasks.workunit.client.0.smithi016.stdout:8/121: dread d1/d6/f10 [0,4194304] 0 2024-04-02T04:42:32.440 INFO:tasks.workunit.client.0.smithi016.stdout:7/111: link d0/d18/d15/d16/f20 d0/d18/d15/f24 0 2024-04-02T04:42:32.440 INFO:tasks.workunit.client.0.smithi016.stdout:7/112: chown d0/d18/d15/cf 733 1 2024-04-02T04:42:32.444 INFO:tasks.workunit.client.0.smithi016.stdout:7/113: dread d0/f22 [0,4194304] 0 2024-04-02T04:42:32.448 INFO:tasks.workunit.client.0.smithi016.stdout:7/114: dread d0/d18/d15/f24 [0,4194304] 0 2024-04-02T04:42:32.449 INFO:tasks.workunit.client.0.smithi016.stdout:7/115: read d0/d18/d15/f1a [167257,91403] 0 2024-04-02T04:42:32.449 INFO:tasks.workunit.client.0.smithi016.stdout:9/111: symlink d0/d16/d1a/l1d 0 2024-04-02T04:42:32.451 INFO:tasks.workunit.client.0.smithi016.stdout:9/112: write d0/d3/f5 [6941683,128986] 0 2024-04-02T04:42:32.451 INFO:tasks.workunit.client.0.smithi016.stdout:7/116: write d0/d18/d15/fc [504368,65680] 0 2024-04-02T04:42:32.452 INFO:tasks.workunit.client.0.smithi016.stdout:9/113: write d0/d3/f5 [1406785,55453] 0 2024-04-02T04:42:32.453 INFO:tasks.workunit.client.0.smithi016.stdout:9/114: dread - d0/d9/db/f1c zero size 2024-04-02T04:42:32.455 INFO:tasks.workunit.client.0.smithi016.stdout:9/115: read d0/d16/f1b [2385383,46076] 0 2024-04-02T04:42:32.456 INFO:tasks.workunit.client.0.smithi016.stdout:3/155: rmdir d2/d15/d16 39 2024-04-02T04:42:32.464 INFO:tasks.workunit.client.0.smithi016.stdout:1/145: rename f6 to d14/d19/d1e/f2b 0 2024-04-02T04:42:32.464 INFO:tasks.workunit.client.0.smithi016.stdout:1/146: dread - d14/f1c zero size 2024-04-02T04:42:32.470 INFO:tasks.workunit.client.0.smithi016.stdout:1/147: dwrite d14/d19/d1e/f23 [0,4194304] 0 2024-04-02T04:42:32.474 INFO:tasks.workunit.client.0.smithi016.stdout:7/117: mknod d0/d18/c25 0 2024-04-02T04:42:32.474 INFO:tasks.workunit.client.0.smithi016.stdout:1/148: dread d14/d19/d1e/f23 [0,4194304] 0 2024-04-02T04:42:32.474 INFO:tasks.workunit.client.0.smithi016.stdout:1/149: read f7 [2361755,2671] 0 2024-04-02T04:42:32.475 INFO:tasks.workunit.client.0.smithi016.stdout:2/118: link d12/d20/c24 d12/d14/c27 0 2024-04-02T04:42:32.476 INFO:tasks.workunit.client.0.smithi016.stdout:7/118: dread d0/d18/fa [0,4194304] 0 2024-04-02T04:42:32.481 INFO:tasks.workunit.client.0.smithi016.stdout:3/156: symlink d2/l33 0 2024-04-02T04:42:32.482 INFO:tasks.workunit.client.0.smithi016.stdout:7/119: dwrite d0/d18/f11 [0,4194304] 0 2024-04-02T04:42:32.483 INFO:tasks.workunit.client.0.smithi016.stdout:2/119: dread f9 [0,4194304] 0 2024-04-02T04:42:32.486 INFO:tasks.workunit.client.0.smithi016.stdout:0/74: rename d0/d13/f14 to d0/f1c 0 2024-04-02T04:42:32.504 INFO:tasks.workunit.client.0.smithi016.stdout:5/155: sync 2024-04-02T04:42:32.517 INFO:tasks.workunit.client.0.smithi016.stdout:1/150: link fb d14/d19/d1d/f2c 0 2024-04-02T04:42:32.521 INFO:tasks.workunit.client.0.smithi016.stdout:8/122: getdents d1 0 2024-04-02T04:42:32.522 INFO:tasks.workunit.client.0.smithi016.stdout:8/123: dread - d1/f15 zero size 2024-04-02T04:42:32.522 INFO:tasks.workunit.client.0.smithi016.stdout:8/124: fdatasync d1/ff 0 2024-04-02T04:42:32.523 INFO:tasks.workunit.client.0.smithi016.stdout:5/156: dread - d3/df/f14 zero size 2024-04-02T04:42:32.527 INFO:tasks.workunit.client.0.smithi016.stdout:8/125: dwrite d1/f1c [8388608,4194304] 0 2024-04-02T04:42:32.529 INFO:tasks.workunit.client.0.smithi016.stdout:2/120: creat d12/d14/d21/f28 x:0 0 0 2024-04-02T04:42:32.529 INFO:tasks.workunit.client.0.smithi016.stdout:8/126: chown d1/d6 1613 1 2024-04-02T04:42:32.530 INFO:tasks.workunit.client.0.smithi016.stdout:0/75: symlink d0/d16/d1a/l1d 0 2024-04-02T04:42:32.532 INFO:tasks.workunit.client.0.smithi016.stdout:0/76: truncate d0/fe 849944 0 2024-04-02T04:42:32.533 INFO:tasks.workunit.client.0.smithi016.stdout:5/157: dwrite d3/df/d18/f42 [0,4194304] 0 2024-04-02T04:42:32.533 INFO:tasks.workunit.client.0.smithi016.stdout:1/151: symlink d14/l2d 0 2024-04-02T04:42:32.537 INFO:tasks.workunit.client.0.smithi016.stdout:7/120: link d0/d18/d15/c14 d0/d18/d15/c26 0 2024-04-02T04:42:32.538 INFO:tasks.workunit.client.0.smithi016.stdout:7/121: chown d0/d18/d15/d16 21819 1 2024-04-02T04:42:32.538 INFO:tasks.workunit.client.0.smithi016.stdout:7/122: chown d0/d21/l23 52940875 1 2024-04-02T04:42:32.543 INFO:tasks.workunit.client.0.smithi016.stdout:8/127: chown d1/d6/c13 0 1 2024-04-02T04:42:32.546 INFO:tasks.workunit.client.0.smithi016.stdout:0/77: creat d0/d16/d1a/f1e x:0 0 0 2024-04-02T04:42:32.550 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:32 smithi016 ceph-mon[28969]: Updating smithi016:/var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T04:42:32.551 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:32 smithi016 ceph-mon[28969]: Updating smithi027:/var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T04:42:32.551 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:32 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:32.551 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:32 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:32.551 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:32 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:32.551 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:32 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:32.551 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:32 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:32.551 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:32 smithi016 ceph-mon[28969]: Reconfiguring prometheus.smithi016 (dependencies changed)... 2024-04-02T04:42:32.551 INFO:tasks.workunit.client.0.smithi016.stdout:1/152: creat d14/d19/d1e/f2e x:0 0 0 2024-04-02T04:42:32.556 INFO:tasks.workunit.client.0.smithi016.stdout:2/121: creat d12/d26/f29 x:0 0 0 2024-04-02T04:42:32.558 INFO:tasks.workunit.client.0.smithi016.stdout:8/128: symlink d1/l28 0 2024-04-02T04:42:32.564 INFO:tasks.workunit.client.0.smithi016.stdout:1/153: dread f7 [0,4194304] 0 2024-04-02T04:42:32.565 INFO:tasks.workunit.client.0.smithi016.stdout:1/154: chown f2 1 1 2024-04-02T04:42:32.565 INFO:tasks.workunit.client.0.smithi016.stdout:1/155: write d14/f1c [425286,119736] 0 2024-04-02T04:42:32.568 INFO:tasks.workunit.client.0.smithi016.stdout:2/122: dread f9 [0,4194304] 0 2024-04-02T04:42:32.569 INFO:tasks.workunit.client.0.smithi016.stdout:2/123: chown d12/d14/d21 71864 1 2024-04-02T04:42:32.569 INFO:tasks.workunit.client.0.smithi016.stdout:8/129: rmdir d1/d24 39 2024-04-02T04:42:32.572 INFO:tasks.workunit.client.0.smithi016.stdout:7/123: link d0/d18/d15/c14 d0/c27 0 2024-04-02T04:42:32.573 INFO:tasks.workunit.client.0.smithi016.stdout:7/124: fdatasync d0/d18/d15/f1d 0 2024-04-02T04:42:32.573 INFO:tasks.workunit.client.0.smithi016.stdout:7/125: write d0/d18/d15/fc [1041493,106680] 0 2024-04-02T04:42:32.575 INFO:tasks.workunit.client.0.smithi016.stdout:1/156: symlink d14/l2f 0 2024-04-02T04:42:32.577 INFO:tasks.workunit.client.0.smithi016.stdout:8/130: mknod d1/d6/c29 0 2024-04-02T04:42:32.580 INFO:tasks.workunit.client.0.smithi016.stdout:1/157: dwrite d14/f17 [0,4194304] 0 2024-04-02T04:42:32.585 INFO:tasks.workunit.client.0.smithi016.stdout:7/126: mkdir d0/d28 0 2024-04-02T04:42:32.586 INFO:tasks.workunit.client.0.smithi016.stdout:1/158: dwrite d14/d19/d1e/f2e [0,4194304] 0 2024-04-02T04:42:32.586 INFO:tasks.workunit.client.0.smithi016.stdout:7/127: stat d0/d18/d15/le 0 2024-04-02T04:42:32.587 INFO:tasks.workunit.client.0.smithi016.stdout:7/128: readlink d0/l10 0 2024-04-02T04:42:32.602 INFO:tasks.workunit.client.0.smithi016.stdout:2/124: link d12/d26/f29 d12/f2a 0 2024-04-02T04:42:32.604 INFO:tasks.workunit.client.0.smithi016.stdout:8/131: rename d1/fd to d1/d24/f2a 0 2024-04-02T04:42:32.605 INFO:tasks.workunit.client.0.smithi016.stdout:8/132: stat d1/d6/f1a 0 2024-04-02T04:42:32.608 INFO:tasks.workunit.client.0.smithi016.stdout:7/129: symlink d0/d28/l29 0 2024-04-02T04:42:32.610 INFO:tasks.workunit.client.0.smithi016.stdout:8/133: dwrite d1/ff [0,4194304] 0 2024-04-02T04:42:32.610 INFO:tasks.workunit.client.0.smithi016.stdout:2/125: read - d12/f16 zero size 2024-04-02T04:42:32.610 INFO:tasks.workunit.client.0.smithi016.stdout:8/134: chown d1/f3 280696909 1 2024-04-02T04:42:32.610 INFO:tasks.workunit.client.0.smithi016.stdout:2/126: chown d12/f2a 19532725 1 2024-04-02T04:42:32.618 INFO:tasks.workunit.client.0.smithi016.stdout:2/127: unlink d12/c1d 0 2024-04-02T04:42:32.620 INFO:tasks.workunit.client.0.smithi016.stdout:1/159: getdents d14/d19 0 2024-04-02T04:42:32.621 INFO:tasks.workunit.client.0.smithi016.stdout:8/135: dread d1/d24/f2a [0,4194304] 0 2024-04-02T04:42:32.625 INFO:tasks.workunit.client.0.smithi016.stdout:1/160: write fb [4016230,103647] 0 2024-04-02T04:42:32.626 INFO:tasks.workunit.client.0.smithi016.stdout:8/136: dwrite d1/ff [4194304,4194304] 0 2024-04-02T04:42:32.632 INFO:tasks.workunit.client.0.smithi016.stdout:8/137: write d1/d24/f2a [3352923,83762] 0 2024-04-02T04:42:32.636 INFO:tasks.workunit.client.0.smithi016.stdout:8/138: creat d1/d24/f2b x:0 0 0 2024-04-02T04:42:32.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:32 smithi027 ceph-mon[37550]: Updating smithi016:/var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T04:42:32.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:32 smithi027 ceph-mon[37550]: Updating smithi027:/var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T04:42:32.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:32 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:32.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:32 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:32.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:32 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:32.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:32 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:32.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:32 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:32.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:32 smithi027 ceph-mon[37550]: Reconfiguring prometheus.smithi016 (dependencies changed)... 2024-04-02T04:42:32.674 INFO:tasks.workunit.client.0.smithi016.stdout:5/158: rmdir d3/df/d18 39 2024-04-02T04:42:32.677 INFO:tasks.workunit.client.0.smithi016.stdout:5/159: write d3/f6 [299935,122938] 0 2024-04-02T04:42:32.681 INFO:tasks.workunit.client.0.smithi016.stdout:6/113: dwrite d0/d9/fd [0,4194304] 0 2024-04-02T04:42:32.687 INFO:tasks.workunit.client.0.smithi016.stdout:6/114: mknod d0/d8/c1a 0 2024-04-02T04:42:32.687 INFO:tasks.workunit.client.0.smithi016.stdout:6/115: read - d0/f13 zero size 2024-04-02T04:42:32.689 INFO:tasks.workunit.client.0.smithi016.stdout:5/160: rename d3/f8 to d3/df/d18/f43 0 2024-04-02T04:42:32.689 INFO:tasks.workunit.client.0.smithi016.stdout:5/161: fdatasync d3/f3b 0 2024-04-02T04:42:32.691 INFO:tasks.workunit.client.0.smithi016.stdout:6/116: creat d0/d9/df/f1b x:0 0 0 2024-04-02T04:42:32.692 INFO:tasks.workunit.client.0.smithi016.stdout:6/117: fdatasync d0/d9/df/f10 0 2024-04-02T04:42:32.694 INFO:tasks.workunit.client.0.smithi016.stdout:6/118: creat d0/d9/d16/f1c x:0 0 0 2024-04-02T04:42:32.695 INFO:tasks.workunit.client.0.smithi016.stdout:6/119: fdatasync d0/d9/df/f1b 0 2024-04-02T04:42:32.696 INFO:tasks.workunit.client.0.smithi016.stdout:5/162: symlink d3/d40/l44 0 2024-04-02T04:42:32.697 INFO:tasks.workunit.client.0.smithi016.stdout:5/163: write f1 [1238850,58385] 0 2024-04-02T04:42:32.699 INFO:tasks.workunit.client.0.smithi016.stdout:6/120: chown d0/fc 44395557 1 2024-04-02T04:42:32.699 INFO:tasks.workunit.client.0.smithi016.stdout:6/121: write d0/f13 [731583,117761] 0 2024-04-02T04:42:32.700 INFO:tasks.workunit.client.0.smithi016.stdout:9/116: truncate d0/f11 385581 0 2024-04-02T04:42:32.701 INFO:tasks.workunit.client.0.smithi016.stdout:9/117: write d0/f7 [764892,125657] 0 2024-04-02T04:42:32.701 INFO:tasks.workunit.client.0.smithi016.stdout:5/164: mkdir d3/d25/d45 0 2024-04-02T04:42:32.704 INFO:tasks.workunit.client.0.smithi016.stdout:6/122: mknod d0/d8/c1d 0 2024-04-02T04:42:32.704 INFO:tasks.workunit.client.0.smithi016.stdout:6/123: chown d0/c17 78 1 2024-04-02T04:42:32.706 INFO:tasks.workunit.client.0.smithi016.stdout:6/124: write d0/f6 [6420627,35299] 0 2024-04-02T04:42:32.706 INFO:tasks.workunit.client.0.smithi016.stdout:6/125: dread - d0/d9/d16/f1c zero size 2024-04-02T04:42:32.708 INFO:tasks.workunit.client.0.smithi016.stdout:5/165: dread - d3/f17 zero size 2024-04-02T04:42:32.712 INFO:tasks.workunit.client.0.smithi016.stdout:9/118: rename d0/c8 to d0/d16/c1e 0 2024-04-02T04:42:32.714 INFO:tasks.workunit.client.0.smithi016.stdout:6/126: mknod d0/d9/df/c1e 0 2024-04-02T04:42:32.718 INFO:tasks.workunit.client.0.smithi016.stdout:9/119: creat d0/d16/d1a/f1f x:0 0 0 2024-04-02T04:42:32.719 INFO:tasks.workunit.client.0.smithi016.stdout:9/120: chown d0/d9/f19 14037869 1 2024-04-02T04:42:32.720 INFO:tasks.workunit.client.0.smithi016.stdout:6/127: dwrite d0/d8/f11 [0,4194304] 0 2024-04-02T04:42:32.725 INFO:tasks.workunit.client.0.smithi016.stdout:9/121: creat d0/d16/f20 x:0 0 0 2024-04-02T04:42:32.725 INFO:tasks.workunit.client.0.smithi016.stdout:4/120: truncate d5/de/f14 1335759 0 2024-04-02T04:42:32.726 INFO:tasks.workunit.client.0.smithi016.stdout:4/121: read f4 [1874871,12533] 0 2024-04-02T04:42:32.728 INFO:tasks.workunit.client.0.smithi016.stdout:9/122: creat d0/d3/f21 x:0 0 0 2024-04-02T04:42:32.728 INFO:tasks.workunit.client.0.smithi016.stdout:9/123: chown d0/d16 113749259 1 2024-04-02T04:42:32.731 INFO:tasks.workunit.client.0.smithi016.stdout:4/122: dread d5/d8/fd [0,4194304] 0 2024-04-02T04:42:32.734 INFO:tasks.workunit.client.0.smithi016.stdout:9/124: dwrite d0/d3/f5 [0,4194304] 0 2024-04-02T04:42:32.741 INFO:tasks.workunit.client.0.smithi016.stdout:0/78: getdents d0 0 2024-04-02T04:42:32.742 INFO:tasks.workunit.client.0.smithi016.stdout:4/123: dwrite d5/d16/f1b [4194304,4194304] 0 2024-04-02T04:42:32.742 INFO:tasks.workunit.client.0.smithi016.stdout:4/124: write d5/d1d/f21 [6520892,73730] 0 2024-04-02T04:42:32.746 INFO:tasks.workunit.client.0.smithi016.stdout:0/79: creat d0/d13/f1f x:0 0 0 2024-04-02T04:42:32.751 INFO:tasks.workunit.client.0.smithi016.stdout:0/80: rename d0/d13/f1f to d0/d16/d1a/f20 0 2024-04-02T04:42:32.753 INFO:tasks.workunit.client.0.smithi016.stdout:0/81: mknod d0/c21 0 2024-04-02T04:42:32.753 INFO:tasks.workunit.client.0.smithi016.stdout:0/82: stat d0/d16/d1a 0 2024-04-02T04:42:32.755 INFO:tasks.workunit.client.0.smithi016.stdout:3/157: write d2/d15/d1a/f29 [458827,118970] 0 2024-04-02T04:42:32.755 INFO:tasks.workunit.client.0.smithi016.stdout:0/83: dread - d0/d16/f17 zero size 2024-04-02T04:42:32.757 INFO:tasks.workunit.client.0.smithi016.stdout:0/84: unlink d0/fd 0 2024-04-02T04:42:32.757 INFO:tasks.workunit.client.0.smithi016.stdout:0/85: chown d0/d16/f17 398132 1 2024-04-02T04:42:32.758 INFO:tasks.workunit.client.0.smithi016.stdout:0/86: dread - d0/f19 zero size 2024-04-02T04:42:32.758 INFO:tasks.workunit.client.0.smithi016.stdout:0/87: read d0/f4 [885789,46603] 0 2024-04-02T04:42:32.758 INFO:tasks.workunit.client.0.smithi016.stdout:0/88: rename d0 to d0/d16/d22 22 2024-04-02T04:42:32.763 INFO:tasks.workunit.client.0.smithi016.stdout:2/128: truncate f9 1113678 0 2024-04-02T04:42:32.765 INFO:tasks.workunit.client.0.smithi016.stdout:0/89: mkdir d0/d16/d23 0 2024-04-02T04:42:32.765 INFO:tasks.workunit.client.0.smithi016.stdout:0/90: readlink d0/l5 0 2024-04-02T04:42:32.769 INFO:tasks.workunit.client.0.smithi016.stdout:3/158: getdents d2/d15/d1a 0 2024-04-02T04:42:32.770 INFO:tasks.workunit.client.0.smithi016.stdout:2/129: mkdir d12/d14/d21/d2b 0 2024-04-02T04:42:32.775 INFO:tasks.workunit.client.0.smithi016.stdout:3/159: rmdir d2/d15/d1a 39 2024-04-02T04:42:32.777 INFO:tasks.workunit.client.0.smithi016.stdout:2/130: rename ff to d12/d26/f2c 0 2024-04-02T04:42:32.782 INFO:tasks.workunit.client.0.smithi016.stdout:3/160: mkdir d2/d15/d1a/d34 0 2024-04-02T04:42:32.783 INFO:tasks.workunit.client.0.smithi016.stdout:3/161: fdatasync d2/f20 0 2024-04-02T04:42:32.783 INFO:tasks.workunit.client.0.smithi016.stdout:3/162: chown d2/d15/d1a/f22 19365443 1 2024-04-02T04:42:32.787 INFO:tasks.workunit.client.0.smithi016.stdout:3/163: mkdir d2/d15/d16/d35 0 2024-04-02T04:42:32.788 INFO:tasks.workunit.client.0.smithi016.stdout:3/164: write d2/d15/f17 [662344,18865] 0 2024-04-02T04:42:32.791 INFO:tasks.workunit.client.0.smithi016.stdout:3/165: mkdir d2/d2e/d36 0 2024-04-02T04:42:32.796 INFO:tasks.workunit.client.0.smithi016.stdout:3/166: dwrite d2/f11 [0,4194304] 0 2024-04-02T04:42:32.799 INFO:tasks.workunit.client.0.smithi016.stdout:3/167: mknod d2/c37 0 2024-04-02T04:42:32.805 INFO:tasks.workunit.client.0.smithi016.stdout:7/130: write d0/d18/d15/f1d [5040658,22733] 0 2024-04-02T04:42:32.809 INFO:tasks.workunit.client.0.smithi016.stdout:7/131: rename d0/d18/c25 to d0/d18/d15/c2a 0 2024-04-02T04:42:32.810 INFO:tasks.workunit.client.0.smithi016.stdout:7/132: rmdir d0/d28 39 2024-04-02T04:42:32.812 INFO:tasks.workunit.client.0.smithi016.stdout:7/133: write d0/d18/fa [833955,120270] 0 2024-04-02T04:42:32.812 INFO:tasks.workunit.client.0.smithi016.stdout:7/134: chown d0/d18/d15/l12 165837 1 2024-04-02T04:42:32.818 INFO:tasks.workunit.client.0.smithi016.stdout:7/135: link d0/d18/d15/cf d0/d18/c2b 0 2024-04-02T04:42:32.820 INFO:tasks.workunit.client.0.smithi016.stdout:7/136: chown d0/d18/d15/c26 30 1 2024-04-02T04:42:32.825 INFO:tasks.workunit.client.0.smithi016.stdout:7/137: dwrite d0/d18/fa [0,4194304] 0 2024-04-02T04:42:32.829 INFO:tasks.workunit.client.0.smithi016.stdout:7/138: stat d0/d18/c2b 0 2024-04-02T04:42:32.832 INFO:tasks.workunit.client.0.smithi016.stdout:7/139: creat d0/d18/d15/f2c x:0 0 0 2024-04-02T04:42:32.836 INFO:tasks.workunit.client.0.smithi016.stdout:7/140: dread d0/f22 [0,4194304] 0 2024-04-02T04:42:32.836 INFO:tasks.workunit.client.0.smithi016.stdout:7/141: chown d0/d18/d15/le 23 1 2024-04-02T04:42:32.839 INFO:tasks.workunit.client.0.smithi016.stdout:8/139: getdents d1/d24 0 2024-04-02T04:42:32.840 INFO:tasks.workunit.client.0.smithi016.stdout:7/142: dread d0/d18/fa [0,4194304] 0 2024-04-02T04:42:32.844 INFO:tasks.workunit.client.0.smithi016.stdout:7/143: rename d0/d18/l13 to d0/d28/l2d 0 2024-04-02T04:42:32.875 INFO:tasks.workunit.client.0.smithi016.stdout:6/128: getdents d0/d9/d16 0 2024-04-02T04:42:32.878 INFO:tasks.workunit.client.0.smithi016.stdout:6/129: mkdir d0/d1f 0 2024-04-02T04:42:32.881 INFO:tasks.workunit.client.0.smithi016.stdout:6/130: creat d0/d9/d16/f20 x:0 0 0 2024-04-02T04:42:32.881 INFO:tasks.workunit.client.0.smithi016.stdout:6/131: chown d0/d1f 15814157 1 2024-04-02T04:42:32.884 INFO:tasks.workunit.client.0.smithi016.stdout:6/132: rmdir d0/d8 39 2024-04-02T04:42:32.889 INFO:tasks.workunit.client.0.smithi016.stdout:6/133: dwrite d0/fc [0,4194304] 0 2024-04-02T04:42:32.891 INFO:tasks.workunit.client.0.smithi016.stdout:5/166: write d3/df/f1e [1018569,72540] 0 2024-04-02T04:42:32.891 INFO:tasks.workunit.client.0.smithi016.stdout:5/167: dread - d3/f28 zero size 2024-04-02T04:42:32.892 INFO:tasks.workunit.client.0.smithi016.stdout:5/168: fdatasync f1 0 2024-04-02T04:42:32.893 INFO:tasks.workunit.client.0.smithi016.stdout:6/134: unlink d0/d9/d16/f1c 0 2024-04-02T04:42:32.897 INFO:tasks.workunit.client.0.smithi016.stdout:9/125: rmdir d0/d16 39 2024-04-02T04:42:32.899 INFO:tasks.workunit.client.0.smithi016.stdout:6/135: dwrite d0/d9/df/f1b [0,4194304] 0 2024-04-02T04:42:32.911 INFO:tasks.workunit.client.0.smithi016.stdout:9/126: creat d0/d16/d1a/f22 x:0 0 0 2024-04-02T04:42:32.912 INFO:tasks.workunit.client.0.smithi016.stdout:6/136: mkdir d0/d8/d21 0 2024-04-02T04:42:32.914 INFO:tasks.workunit.client.0.smithi016.stdout:0/91: rename d0/d16/d1a to d0/d16/d24 0 2024-04-02T04:42:32.915 INFO:tasks.workunit.client.0.smithi016.stdout:6/137: mknod d0/d9/d16/c22 0 2024-04-02T04:42:32.916 INFO:tasks.workunit.client.0.smithi016.stdout:9/127: write d0/d16/f1b [1093883,116852] 0 2024-04-02T04:42:32.918 INFO:tasks.workunit.client.0.smithi016.stdout:4/125: dwrite d5/d8/fd [0,4194304] 0 2024-04-02T04:42:32.919 INFO:tasks.workunit.client.0.smithi016.stdout:9/128: write d0/d9/db/f1c [884045,58611] 0 2024-04-02T04:42:32.920 INFO:tasks.workunit.client.0.smithi016.stdout:1/161: sync 2024-04-02T04:42:32.921 INFO:tasks.workunit.client.0.smithi016.stdout:9/129: chown d0/d16/f1b 0 1 2024-04-02T04:42:32.926 INFO:tasks.workunit.client.0.smithi016.stdout:0/92: dwrite d0/d16/f17 [0,4194304] 0 2024-04-02T04:42:32.928 INFO:tasks.workunit.client.0.smithi016.stdout:1/162: dwrite f11 [0,4194304] 0 2024-04-02T04:42:32.930 INFO:tasks.workunit.client.0.smithi016.stdout:9/130: unlink d0/d3/f21 0 2024-04-02T04:42:32.931 INFO:tasks.workunit.client.0.smithi016.stdout:2/131: getdents d12/d26 0 2024-04-02T04:42:32.933 INFO:tasks.workunit.client.0.smithi016.stdout:9/131: dread - d0/d16/d1a/f1f zero size 2024-04-02T04:42:32.934 INFO:tasks.workunit.client.0.smithi016.stdout:9/132: dread - d0/d16/f20 zero size 2024-04-02T04:42:32.935 INFO:tasks.workunit.client.0.smithi016.stdout:0/93: dwrite d0/fe [0,4194304] 0 2024-04-02T04:42:32.935 INFO:tasks.workunit.client.0.smithi016.stdout:6/138: rmdir d0/d8/d21 0 2024-04-02T04:42:32.936 INFO:tasks.workunit.client.0.smithi016.stdout:9/133: chown d0/d9/db/l15 3 1 2024-04-02T04:42:32.939 INFO:tasks.workunit.client.0.smithi016.stdout:0/94: dread d0/fe [0,4194304] 0 2024-04-02T04:42:32.945 INFO:tasks.workunit.client.0.smithi016.stdout:4/126: symlink d5/d17/d18/d1f/l22 0 2024-04-02T04:42:32.951 INFO:tasks.workunit.client.0.smithi016.stdout:1/163: creat d14/d19/d1d/f30 x:0 0 0 2024-04-02T04:42:32.956 INFO:tasks.workunit.client.0.smithi016.stdout:6/139: rmdir d0/d9/df 39 2024-04-02T04:42:32.958 INFO:tasks.workunit.client.0.smithi016.stdout:9/134: mknod d0/d3/c23 0 2024-04-02T04:42:32.961 INFO:tasks.workunit.client.0.smithi016.stdout:6/140: dread d0/d9/d16/f19 [0,4194304] 0 2024-04-02T04:42:32.962 INFO:tasks.workunit.client.0.smithi016.stdout:1/164: rmdir d14/d19 39 2024-04-02T04:42:32.963 INFO:tasks.workunit.client.0.smithi016.stdout:3/168: dwrite d2/f13 [0,4194304] 0 2024-04-02T04:42:32.968 INFO:tasks.workunit.client.0.smithi016.stdout:9/135: dwrite d0/d16/f1b [0,4194304] 0 2024-04-02T04:42:32.972 INFO:tasks.workunit.client.0.smithi016.stdout:6/141: write d0/d9/df/f10 [127170,79365] 0 2024-04-02T04:42:32.977 INFO:tasks.workunit.client.0.smithi016.stdout:1/165: dwrite d14/d19/d1d/f1f [0,4194304] 0 2024-04-02T04:42:32.979 INFO:tasks.workunit.client.0.smithi016.stdout:8/140: sync 2024-04-02T04:42:32.980 INFO:tasks.workunit.client.0.smithi016.stdout:9/136: symlink d0/d3/l24 0 2024-04-02T04:42:32.986 INFO:tasks.workunit.client.0.smithi016.stdout:1/166: dread d14/d19/d1e/f2e [0,4194304] 0 2024-04-02T04:42:32.989 INFO:tasks.workunit.client.0.smithi016.stdout:0/95: getdents d0/d13 0 2024-04-02T04:42:32.990 INFO:tasks.workunit.client.0.smithi016.stdout:8/141: dwrite d1/d6/f21 [0,4194304] 0 2024-04-02T04:42:32.992 INFO:tasks.workunit.client.0.smithi016.stdout:7/144: dwrite d0/d18/fa [4194304,4194304] 0 2024-04-02T04:42:32.993 INFO:tasks.workunit.client.0.smithi016.stdout:8/142: chown d1/f1d 40 1 2024-04-02T04:42:32.993 INFO:tasks.workunit.client.0.smithi016.stdout:0/96: truncate d0/d16/f17 4965929 0 2024-04-02T04:42:33.006 INFO:tasks.workunit.client.0.smithi016.stdout:8/143: creat d1/d24/f2c x:0 0 0 2024-04-02T04:42:33.007 INFO:tasks.workunit.client.0.smithi016.stdout:8/144: stat d1/ff 0 2024-04-02T04:42:33.007 INFO:tasks.workunit.client.0.smithi016.stdout:7/145: symlink d0/d28/l2e 0 2024-04-02T04:42:33.010 INFO:tasks.workunit.client.0.smithi016.stdout:0/97: mknod d0/c25 0 2024-04-02T04:42:33.010 INFO:tasks.workunit.client.0.smithi016.stdout:0/98: readlink d0/l6 0 2024-04-02T04:42:33.013 INFO:tasks.workunit.client.0.smithi016.stdout:7/146: dwrite d0/d18/d15/f1d [4194304,4194304] 0 2024-04-02T04:42:33.018 INFO:tasks.workunit.client.0.smithi016.stdout:8/145: symlink d1/l2d 0 2024-04-02T04:42:33.021 INFO:tasks.workunit.client.0.smithi016.stdout:0/99: symlink d0/d16/d23/l26 0 2024-04-02T04:42:33.023 INFO:tasks.workunit.client.0.smithi016.stdout:8/146: creat d1/d6/f2e x:0 0 0 2024-04-02T04:42:33.025 INFO:tasks.workunit.client.0.smithi016.stdout:5/169: write d3/df/d18/f1b [3176070,10972] 0 2024-04-02T04:42:33.026 INFO:tasks.workunit.client.0.smithi016.stdout:0/100: fsync d0/f1c 0 2024-04-02T04:42:33.029 INFO:tasks.workunit.client.0.smithi016.stdout:5/170: mknod d3/df/d18/d26/d2c/c46 0 2024-04-02T04:42:33.032 INFO:tasks.workunit.client.0.smithi016.stdout:0/101: write d0/fe [3912086,23775] 0 2024-04-02T04:42:33.034 INFO:tasks.workunit.client.0.smithi016.stdout:0/102: read - d0/f10 zero size 2024-04-02T04:42:33.037 INFO:tasks.workunit.client.0.smithi016.stdout:5/171: dwrite d3/df/d18/f35 [0,4194304] 0 2024-04-02T04:42:33.038 INFO:tasks.workunit.client.0.smithi016.stdout:4/127: dread d5/de/f14 [0,4194304] 0 2024-04-02T04:42:33.038 INFO:tasks.workunit.client.0.smithi016.stdout:4/128: stat d5/d17/d18/c19 0 2024-04-02T04:42:33.039 INFO:tasks.workunit.client.0.smithi016.stdout:0/103: symlink d0/d16/d24/l27 0 2024-04-02T04:42:33.041 INFO:tasks.workunit.client.0.smithi016.stdout:5/172: creat d3/d25/f47 x:0 0 0 2024-04-02T04:42:33.042 INFO:tasks.workunit.client.0.smithi016.stdout:5/173: write d3/df/f1e [1302146,55584] 0 2024-04-02T04:42:33.047 INFO:tasks.workunit.client.0.smithi016.stdout:2/132: write d12/d14/f1e [61021,6428] 0 2024-04-02T04:42:33.050 INFO:tasks.workunit.client.0.smithi016.stdout:4/129: dread d5/f10 [0,4194304] 0 2024-04-02T04:42:33.050 INFO:tasks.workunit.client.0.smithi016.stdout:4/130: chown d5/d8 17 1 2024-04-02T04:42:33.051 INFO:tasks.workunit.client.0.smithi016.stdout:2/133: rename d12/d14/d21/f28 to d12/d14/d21/f2d 0 2024-04-02T04:42:33.052 INFO:tasks.workunit.client.0.smithi016.stdout:0/104: link d0/l5 d0/d16/l28 0 2024-04-02T04:42:33.055 INFO:tasks.workunit.client.0.smithi016.stdout:4/131: creat d5/d1d/f23 x:0 0 0 2024-04-02T04:42:33.072 INFO:tasks.workunit.client.0.smithi016.stdout:4/132: fdatasync d5/d8/fb 0 2024-04-02T04:42:33.074 INFO:tasks.workunit.client.0.smithi016.stdout:6/142: truncate d0/fc 3678771 0 2024-04-02T04:42:33.076 INFO:tasks.workunit.client.0.smithi016.stdout:8/147: getdents d1 0 2024-04-02T04:42:33.084 INFO:tasks.workunit.client.0.smithi016.stdout:3/169: dwrite d2/fd [0,4194304] 0 2024-04-02T04:42:33.085 INFO:tasks.workunit.client.0.smithi016.stdout:2/134: truncate d12/d26/f2c 2551118 0 2024-04-02T04:42:33.085 INFO:tasks.workunit.client.0.smithi016.stdout:8/148: write d1/f19 [101,9960] 0 2024-04-02T04:42:33.087 INFO:tasks.workunit.client.0.smithi016.stdout:0/105: dwrite d0/d16/d24/f20 [0,4194304] 0 2024-04-02T04:42:33.089 INFO:tasks.workunit.client.0.smithi016.stdout:1/167: dwrite fd [0,4194304] 0 2024-04-02T04:42:33.091 INFO:tasks.workunit.client.0.smithi016.stdout:4/133: dread d5/d16/f1b [4194304,4194304] 0 2024-04-02T04:42:33.092 INFO:tasks.workunit.client.0.smithi016.stdout:8/149: write d1/f19 [481339,40586] 0 2024-04-02T04:42:33.092 INFO:tasks.workunit.client.0.smithi016.stdout:2/135: symlink d12/d14/d21/l2e 0 2024-04-02T04:42:33.092 INFO:tasks.workunit.client.0.smithi016.stdout:3/170: chown d2/d15/d1a/f22 6 1 2024-04-02T04:42:33.094 INFO:tasks.workunit.client.0.smithi016.stdout:4/134: chown d5/d8/fb 24290659 1 2024-04-02T04:42:33.094 INFO:tasks.workunit.client.0.smithi016.stdout:3/171: fsync d2/d15/f31 0 2024-04-02T04:42:33.096 INFO:tasks.workunit.client.0.smithi016.stdout:4/135: write d5/d8/ff [3923441,58655] 0 2024-04-02T04:42:33.096 INFO:tasks.workunit.client.0.smithi016.stdout:2/136: truncate d12/d14/d21/f2d 270154 0 2024-04-02T04:42:33.098 INFO:tasks.workunit.client.0.smithi016.stdout:4/136: chown d5/l9 12 1 2024-04-02T04:42:33.098 INFO:tasks.workunit.client.0.smithi016.stdout:9/137: dwrite d0/f11 [0,4194304] 0 2024-04-02T04:42:33.100 INFO:tasks.workunit.client.0.smithi016.stdout:0/106: rename d0/d13 to d0/d29 0 2024-04-02T04:42:33.100 INFO:tasks.workunit.client.0.smithi016.stdout:4/137: write d5/d16/f20 [442508,8964] 0 2024-04-02T04:42:33.102 INFO:tasks.workunit.client.0.smithi016.stdout:1/168: mkdir d14/d19/d31 0 2024-04-02T04:42:33.103 INFO:tasks.workunit.client.0.smithi016.stdout:8/150: mknod d1/d24/c2f 0 2024-04-02T04:42:33.108 INFO:tasks.workunit.client.0.smithi016.stdout:3/172: fsync d2/f14 0 2024-04-02T04:42:33.110 INFO:tasks.workunit.client.0.smithi016.stdout:0/107: creat d0/d16/d24/f2a x:0 0 0 2024-04-02T04:42:33.112 INFO:tasks.workunit.client.0.smithi016.stdout:4/138: mknod d5/d16/c24 0 2024-04-02T04:42:33.114 INFO:tasks.workunit.client.0.smithi016.stdout:8/151: mknod d1/d24/c30 0 2024-04-02T04:42:33.118 INFO:tasks.workunit.client.0.smithi016.stdout:9/138: link d0/d9/db/f1c d0/d9/db/f25 0 2024-04-02T04:42:33.119 INFO:tasks.workunit.client.0.smithi016.stdout:8/152: dread d1/d6/f21 [0,4194304] 0 2024-04-02T04:42:33.119 INFO:tasks.workunit.client.0.smithi016.stdout:8/153: dread - d1/f15 zero size 2024-04-02T04:42:33.121 INFO:tasks.workunit.client.0.smithi016.stdout:0/108: creat d0/d16/d24/f2b x:0 0 0 2024-04-02T04:42:33.121 INFO:tasks.workunit.client.0.smithi016.stdout:8/154: write d1/d6/f2e [565743,130467] 0 2024-04-02T04:42:33.124 INFO:tasks.workunit.client.0.smithi016.stdout:0/109: chown d0/d16/d23 56 1 2024-04-02T04:42:33.127 INFO:tasks.workunit.client.0.smithi016.stdout:3/173: dread d2/d15/f1b [0,4194304] 0 2024-04-02T04:42:33.127 INFO:tasks.workunit.client.0.smithi016.stdout:4/139: dread d5/d8/ff [0,4194304] 0 2024-04-02T04:42:33.128 INFO:tasks.workunit.client.0.smithi016.stdout:7/147: sync 2024-04-02T04:42:33.128 INFO:tasks.workunit.client.0.smithi016.stdout:3/174: chown d2/f14 0 1 2024-04-02T04:42:33.130 INFO:tasks.workunit.client.0.smithi016.stdout:4/140: chown d5/d1d/d1e 36659278 1 2024-04-02T04:42:33.131 INFO:tasks.workunit.client.0.smithi016.stdout:7/148: readlink d0/d18/d15/le 0 2024-04-02T04:42:33.135 INFO:tasks.workunit.client.0.smithi016.stdout:1/169: mknod d14/d19/d1e/c32 0 2024-04-02T04:42:33.135 INFO:tasks.workunit.client.0.smithi016.stdout:0/110: dwrite d0/d16/d24/f2b [0,4194304] 0 2024-04-02T04:42:33.136 INFO:tasks.workunit.client.0.smithi016.stdout:9/139: dread d0/f7 [0,4194304] 0 2024-04-02T04:42:33.137 INFO:tasks.workunit.client.0.smithi016.stdout:2/137: dwrite f8 [0,4194304] 0 2024-04-02T04:42:33.137 INFO:tasks.workunit.client.0.smithi016.stdout:0/111: chown d0/f10 12861006 1 2024-04-02T04:42:33.140 INFO:tasks.workunit.client.0.smithi016.stdout:4/141: mkdir d5/d17/d18/d25 0 2024-04-02T04:42:33.141 INFO:tasks.workunit.client.0.smithi016.stdout:7/149: fdatasync d0/d18/d15/d16/f20 0 2024-04-02T04:42:33.142 INFO:tasks.workunit.client.0.smithi016.stdout:4/142: chown d5/d17/d18/c19 2402 1 2024-04-02T04:42:33.142 INFO:tasks.workunit.client.0.smithi016.stdout:4/143: stat d5/f7 0 2024-04-02T04:42:33.145 INFO:tasks.workunit.client.0.smithi016.stdout:1/170: dwrite fa [0,4194304] 0 2024-04-02T04:42:33.145 INFO:tasks.workunit.client.0.smithi016.stdout:8/155: write d1/d6/f10 [4342617,53864] 0 2024-04-02T04:42:33.149 INFO:tasks.workunit.client.0.smithi016.stdout:0/112: mknod d0/c2c 0 2024-04-02T04:42:33.152 INFO:tasks.workunit.client.0.smithi016.stdout:7/150: creat d0/d18/d15/f2f x:0 0 0 2024-04-02T04:42:33.154 INFO:tasks.workunit.client.0.smithi016.stdout:3/175: creat d2/d15/f38 x:0 0 0 2024-04-02T04:42:33.154 INFO:tasks.workunit.client.0.smithi016.stdout:7/151: chown d0/d18/d15/f1d 8708 1 2024-04-02T04:42:33.155 INFO:tasks.workunit.client.0.smithi016.stdout:9/140: dread d0/d9/db/f1c [0,4194304] 0 2024-04-02T04:42:33.156 INFO:tasks.workunit.client.0.smithi016.stdout:8/156: mkdir d1/d31 0 2024-04-02T04:42:33.156 INFO:tasks.workunit.client.0.smithi016.stdout:3/176: dread - d2/d15/f38 zero size 2024-04-02T04:42:33.158 INFO:tasks.workunit.client.0.smithi016.stdout:4/144: symlink d5/d1d/d1e/l26 0 2024-04-02T04:42:33.158 INFO:tasks.workunit.client.0.smithi016.stdout:2/138: dwrite d12/d14/d21/f2d [0,4194304] 0 2024-04-02T04:42:33.159 INFO:tasks.workunit.client.0.smithi016.stdout:3/177: write d2/d15/f31 [638373,65007] 0 2024-04-02T04:42:33.161 INFO:tasks.workunit.client.0.smithi016.stdout:8/157: write d1/ff [5964253,122568] 0 2024-04-02T04:42:33.162 INFO:tasks.workunit.client.0.smithi016.stdout:9/141: symlink d0/d16/d1a/l26 0 2024-04-02T04:42:33.162 INFO:tasks.workunit.client.0.smithi016.stdout:5/174: sync 2024-04-02T04:42:33.162 INFO:tasks.workunit.client.0.smithi016.stdout:6/143: sync 2024-04-02T04:42:33.163 INFO:tasks.workunit.client.0.smithi016.stdout:7/152: fsync d0/d18/d15/d16/f20 0 2024-04-02T04:42:33.163 INFO:tasks.workunit.client.0.smithi016.stdout:6/144: readlink d0/d8/l15 0 2024-04-02T04:42:33.164 INFO:tasks.workunit.client.0.smithi016.stdout:7/153: chown d0/d18 526280 1 2024-04-02T04:42:33.164 INFO:tasks.workunit.client.0.smithi016.stdout:4/145: symlink d5/l27 0 2024-04-02T04:42:33.167 INFO:tasks.workunit.client.0.smithi016.stdout:2/139: creat d12/d20/f2f x:0 0 0 2024-04-02T04:42:33.170 INFO:tasks.workunit.client.0.smithi016.stdout:3/178: rename d2/f13 to d2/d15/d1a/f39 0 2024-04-02T04:42:33.172 INFO:tasks.workunit.client.0.smithi016.stdout:3/179: chown d2/f11 98 1 2024-04-02T04:42:33.174 INFO:tasks.workunit.client.0.smithi016.stdout:5/175: dwrite d3/f3e [0,4194304] 0 2024-04-02T04:42:33.174 INFO:tasks.workunit.client.0.smithi016.stdout:3/180: readlink d2/d15/d16/l2b 0 2024-04-02T04:42:33.174 INFO:tasks.workunit.client.0.smithi016.stdout:9/142: dwrite d0/d3/f5 [0,4194304] 0 2024-04-02T04:42:33.176 INFO:tasks.workunit.client.0.smithi016.stdout:1/171: truncate fa 2836719 0 2024-04-02T04:42:33.177 INFO:tasks.workunit.client.0.smithi016.stdout:8/158: truncate d1/f19 1203230 0 2024-04-02T04:42:33.178 INFO:tasks.workunit.client.0.smithi016.stdout:1/172: chown d14/l2d 2432 1 2024-04-02T04:42:33.178 INFO:tasks.workunit.client.0.smithi016.stdout:7/154: creat d0/d18/d15/f30 x:0 0 0 2024-04-02T04:42:33.179 INFO:tasks.workunit.client.0.smithi016.stdout:0/113: truncate d0/fe 1471314 0 2024-04-02T04:42:33.180 INFO:tasks.workunit.client.0.smithi016.stdout:4/146: mkdir d5/d17/d28 0 2024-04-02T04:42:33.180 INFO:tasks.workunit.client.0.smithi016.stdout:7/155: dread - d0/d18/d15/f30 zero size 2024-04-02T04:42:33.187 INFO:tasks.workunit.client.0.smithi016.stdout:4/147: dwrite d5/d1d/f23 [0,4194304] 0 2024-04-02T04:42:33.192 INFO:tasks.workunit.client.0.smithi016.stdout:5/176: read - d3/f9 zero size 2024-04-02T04:42:33.193 INFO:tasks.workunit.client.0.smithi016.stdout:4/148: dread d5/f10 [0,4194304] 0 2024-04-02T04:42:33.195 INFO:tasks.workunit.client.0.smithi016.stdout:0/114: symlink d0/d29/l2d 0 2024-04-02T04:42:33.197 INFO:tasks.workunit.client.0.smithi016.stdout:7/156: mkdir d0/d18/d15/d16/d31 0 2024-04-02T04:42:33.197 INFO:tasks.workunit.client.0.smithi016.stdout:0/115: dread - d0/f19 zero size 2024-04-02T04:42:33.198 INFO:tasks.workunit.client.0.smithi016.stdout:2/140: symlink d12/l30 0 2024-04-02T04:42:33.200 INFO:tasks.workunit.client.0.smithi016.stdout:8/159: dwrite d1/d6/f2e [0,4194304] 0 2024-04-02T04:42:33.201 INFO:tasks.workunit.client.0.smithi016.stdout:6/145: link d0/d9/df/f1b d0/d9/d16/f23 0 2024-04-02T04:42:33.205 INFO:tasks.workunit.client.0.smithi016.stdout:5/177: mkdir d3/df/d18/d26/d48 0 2024-04-02T04:42:33.208 INFO:tasks.workunit.client.0.smithi016.stdout:7/157: dread d0/d18/d15/d16/f20 [0,4194304] 0 2024-04-02T04:42:33.209 INFO:tasks.workunit.client.0.smithi016.stdout:2/141: dwrite d12/d14/f1f [0,4194304] 0 2024-04-02T04:42:33.213 INFO:tasks.workunit.client.0.smithi016.stdout:0/116: mkdir d0/d16/d2e 0 2024-04-02T04:42:33.216 INFO:tasks.workunit.client.0.smithi016.stdout:8/160: stat d1/d24/l22 0 2024-04-02T04:42:33.218 INFO:tasks.workunit.client.0.smithi016.stdout:6/146: symlink d0/d9/df/l24 0 2024-04-02T04:42:33.219 INFO:tasks.workunit.client.0.smithi016.stdout:3/181: dread d2/f3 [0,4194304] 0 2024-04-02T04:42:33.222 INFO:tasks.workunit.client.0.smithi016.stdout:2/142: creat d12/d14/f31 x:0 0 0 2024-04-02T04:42:33.223 INFO:tasks.workunit.client.0.smithi016.stdout:7/158: unlink d0/c27 0 2024-04-02T04:42:33.223 INFO:tasks.workunit.client.0.smithi016.stdout:0/117: dwrite d0/d16/d24/f2a [0,4194304] 0 2024-04-02T04:42:33.223 INFO:tasks.workunit.client.0.smithi016.stdout:5/178: dread d3/df/d18/f1b [0,4194304] 0 2024-04-02T04:42:33.226 INFO:tasks.workunit.client.0.smithi016.stdout:0/118: dread d0/d16/f17 [4194304,4194304] 0 2024-04-02T04:42:33.228 INFO:tasks.workunit.client.0.smithi016.stdout:6/147: creat d0/d8/f25 x:0 0 0 2024-04-02T04:42:33.228 INFO:tasks.workunit.client.0.smithi016.stdout:3/182: dwrite d2/fa [0,4194304] 0 2024-04-02T04:42:33.229 INFO:tasks.workunit.client.0.smithi016.stdout:6/148: fdatasync d0/d8/f11 0 2024-04-02T04:42:33.230 INFO:tasks.workunit.client.0.smithi016.stdout:3/183: read - d2/d15/d16/f25 zero size 2024-04-02T04:42:33.233 INFO:tasks.workunit.client.0.smithi016.stdout:2/143: mknod d12/d20/c32 0 2024-04-02T04:42:33.235 INFO:tasks.workunit.client.0.smithi016.stdout:9/143: write d0/d3/f13 [851639,69818] 0 2024-04-02T04:42:33.236 INFO:tasks.workunit.client.0.smithi016.stdout:9/144: fsync d0/f11 0 2024-04-02T04:42:33.236 INFO:tasks.workunit.client.0.smithi016.stdout:5/179: chown d3/f10 8491227 1 2024-04-02T04:42:33.238 INFO:tasks.workunit.client.0.smithi016.stdout:8/161: getdents d1/d31 0 2024-04-02T04:42:33.240 INFO:tasks.workunit.client.0.smithi016.stdout:1/173: write d14/d19/d1e/f2b [1123074,77527] 0 2024-04-02T04:42:33.241 INFO:tasks.workunit.client.0.smithi016.stdout:0/119: symlink d0/d29/l2f 0 2024-04-02T04:42:33.243 INFO:tasks.workunit.client.0.smithi016.stdout:0/120: dread - d0/f10 zero size 2024-04-02T04:42:33.245 INFO:tasks.workunit.client.0.smithi016.stdout:2/144: readlink lb 0 2024-04-02T04:42:33.246 INFO:tasks.workunit.client.0.smithi016.stdout:9/145: dwrite d0/d16/f20 [0,4194304] 0 2024-04-02T04:42:33.247 INFO:tasks.workunit.client.0.smithi016.stdout:2/145: fsync d12/f23 0 2024-04-02T04:42:33.255 INFO:tasks.workunit.client.0.smithi016.stdout:1/174: dread - d14/d19/d1d/f30 zero size 2024-04-02T04:42:33.266 INFO:tasks.workunit.client.0.smithi016.stdout:7/159: creat d0/f32 x:0 0 0 2024-04-02T04:42:33.267 INFO:tasks.workunit.client.0.smithi016.stdout:2/146: unlink d12/d14/f31 0 2024-04-02T04:42:33.267 INFO:tasks.workunit.client.0.smithi016.stdout:5/180: creat d3/df/d18/d26/d48/f49 x:0 0 0 2024-04-02T04:42:33.268 INFO:tasks.workunit.client.0.smithi016.stdout:3/184: rmdir d2/d15/d1a/d34 0 2024-04-02T04:42:33.270 INFO:tasks.workunit.client.0.smithi016.stdout:7/160: write d0/d18/d15/f2c [118961,100413] 0 2024-04-02T04:42:33.270 INFO:tasks.workunit.client.0.smithi016.stdout:6/149: link d0/d9/df/f1b d0/d1f/f26 0 2024-04-02T04:42:33.270 INFO:tasks.workunit.client.0.smithi016.stdout:2/147: write d12/d20/f2f [994916,111947] 0 2024-04-02T04:42:33.275 INFO:tasks.workunit.client.0.smithi016.stdout:5/181: dread d3/f3e [0,4194304] 0 2024-04-02T04:42:33.275 INFO:tasks.workunit.client.0.smithi016.stdout:9/146: dwrite d0/d9/db/f1c [0,4194304] 0 2024-04-02T04:42:33.276 INFO:tasks.workunit.client.0.smithi016.stdout:5/182: stat d3/l1d 0 2024-04-02T04:42:33.277 INFO:tasks.workunit.client.0.smithi016.stdout:5/183: rename d3/df to d3/df/d18/d26/d4a 22 2024-04-02T04:42:33.278 INFO:tasks.workunit.client.0.smithi016.stdout:5/184: chown d3/df/d18/d2f/f3f 116426807 1 2024-04-02T04:42:33.302 INFO:tasks.workunit.client.0.smithi016.stdout:3/185: mknod d2/c3a 0 2024-04-02T04:42:33.303 INFO:tasks.workunit.client.0.smithi016.stdout:6/150: rmdir d0/d1f 39 2024-04-02T04:42:33.304 INFO:tasks.workunit.client.0.smithi016.stdout:7/161: creat d0/d18/d15/d16/f33 x:0 0 0 2024-04-02T04:42:33.309 INFO:tasks.workunit.client.0.smithi016.stdout:9/147: write d0/f7 [4122603,75218] 0 2024-04-02T04:42:33.311 INFO:tasks.workunit.client.0.smithi016.stdout:5/185: fdatasync d3/df/d18/f43 0 2024-04-02T04:42:33.347 INFO:tasks.workunit.client.0.smithi016.stdout:2/148: mknod d12/d14/d21/d2b/c33 0 2024-04-02T04:42:33.348 INFO:tasks.workunit.client.0.smithi016.stdout:7/162: creat d0/d18/d15/d16/f34 x:0 0 0 2024-04-02T04:42:33.353 INFO:tasks.workunit.client.0.smithi016.stdout:3/186: mknod d2/d2e/d36/c3b 0 2024-04-02T04:42:33.354 INFO:tasks.workunit.client.0.smithi016.stdout:6/151: mknod d0/c27 0 2024-04-02T04:42:33.356 INFO:tasks.workunit.client.0.smithi016.stdout:2/149: rename d12/f1c to d12/d14/d21/f34 0 2024-04-02T04:42:33.358 INFO:tasks.workunit.client.0.smithi016.stdout:1/175: getdents d14/d19/d20 0 2024-04-02T04:42:33.360 INFO:tasks.workunit.client.0.smithi016.stdout:6/152: stat d0/cb 0 2024-04-02T04:42:33.361 INFO:tasks.workunit.client.0.smithi016.stdout:6/153: chown d0/d9/c14 6312307 1 2024-04-02T04:42:33.364 INFO:tasks.workunit.client.0.smithi016.stdout:9/148: creat d0/f27 x:0 0 0 2024-04-02T04:42:33.366 INFO:tasks.workunit.client.0.smithi016.stdout:3/187: mknod d2/d15/d16/d35/c3c 0 2024-04-02T04:42:33.366 INFO:tasks.workunit.client.0.smithi016.stdout:9/149: write d0/d16/d1a/f1f [800749,122548] 0 2024-04-02T04:42:33.367 INFO:tasks.workunit.client.0.smithi016.stdout:3/188: chown d2/f9 13 1 2024-04-02T04:42:33.367 INFO:tasks.workunit.client.0.smithi016.stdout:9/150: truncate d0/d3/f13 922698 0 2024-04-02T04:42:33.369 INFO:tasks.workunit.client.0.smithi016.stdout:6/154: unlink d0/d9/ce 0 2024-04-02T04:42:33.370 INFO:tasks.workunit.client.0.smithi016.stdout:2/150: mkdir d12/d35 0 2024-04-02T04:42:33.373 INFO:tasks.workunit.client.0.smithi016.stdout:9/151: dread d0/d16/d1a/f1f [0,4194304] 0 2024-04-02T04:42:33.374 INFO:tasks.workunit.client.0.smithi016.stdout:9/152: write d0/f7 [901559,112136] 0 2024-04-02T04:42:33.375 INFO:tasks.workunit.client.0.smithi016.stdout:9/153: write d0/d9/db/f1c [2996613,6493] 0 2024-04-02T04:42:33.381 INFO:tasks.workunit.client.0.smithi016.stdout:2/151: mknod d12/d35/c36 0 2024-04-02T04:42:33.383 INFO:tasks.workunit.client.0.smithi016.stdout:0/121: truncate d0/d16/d24/f2a 420244 0 2024-04-02T04:42:33.384 INFO:tasks.workunit.client.0.smithi016.stdout:9/154: symlink d0/d16/l28 0 2024-04-02T04:42:33.384 INFO:tasks.workunit.client.0.smithi016.stdout:3/189: dread d2/f14 [0,4194304] 0 2024-04-02T04:42:33.386 INFO:tasks.workunit.client.0.smithi016.stdout:0/122: mknod d0/d16/d24/c30 0 2024-04-02T04:42:33.388 INFO:tasks.workunit.client.0.smithi016.stdout:9/155: symlink d0/d16/d1a/l29 0 2024-04-02T04:42:33.389 INFO:tasks.workunit.client.0.smithi016.stdout:3/190: mkdir d2/d15/d16/d3d 0 2024-04-02T04:42:33.389 INFO:tasks.workunit.client.0.smithi016.stdout:3/191: fdatasync d2/d15/d16/f2c 0 2024-04-02T04:42:33.389 INFO:tasks.workunit.client.0.smithi016.stdout:9/156: dread d0/d16/d1a/f1f [0,4194304] 0 2024-04-02T04:42:33.390 INFO:tasks.workunit.client.0.smithi016.stdout:9/157: chown d0/d9/db/l15 2123 1 2024-04-02T04:42:33.393 INFO:tasks.workunit.client.0.smithi016.stdout:0/123: unlink d0/f1c 0 2024-04-02T04:42:33.394 INFO:tasks.workunit.client.0.smithi016.stdout:0/124: read - d0/f10 zero size 2024-04-02T04:42:33.395 INFO:tasks.workunit.client.0.smithi016.stdout:3/192: dwrite d2/d15/f17 [0,4194304] 0 2024-04-02T04:42:33.397 INFO:tasks.workunit.client.0.smithi016.stdout:3/193: dread d2/d15/f23 [0,4194304] 0 2024-04-02T04:42:33.398 INFO:tasks.workunit.client.0.smithi016.stdout:9/158: symlink d0/d3/l2a 0 2024-04-02T04:42:33.403 INFO:tasks.workunit.client.0.smithi016.stdout:0/125: dwrite d0/d16/f18 [0,4194304] 0 2024-04-02T04:42:33.407 INFO:tasks.workunit.client.0.smithi016.stdout:9/159: dwrite d0/d9/f14 [0,4194304] 0 2024-04-02T04:42:33.407 INFO:tasks.workunit.client.0.smithi016.stdout:9/160: stat d0/d16/f20 0 2024-04-02T04:42:33.407 INFO:tasks.workunit.client.0.smithi016.stdout:3/194: unlink d2/f11 0 2024-04-02T04:42:33.412 INFO:tasks.workunit.client.0.smithi016.stdout:0/126: creat d0/d16/f31 x:0 0 0 2024-04-02T04:42:33.417 INFO:tasks.workunit.client.0.smithi016.stdout:9/161: dwrite d0/f7 [0,4194304] 0 2024-04-02T04:42:33.425 INFO:tasks.workunit.client.0.smithi016.stdout:9/162: dwrite d0/f27 [0,4194304] 0 2024-04-02T04:42:33.432 INFO:tasks.workunit.client.0.smithi016.stdout:9/163: mknod d0/c2b 0 2024-04-02T04:42:33.445 INFO:tasks.workunit.client.0.smithi016.stdout:5/186: dwrite d3/df/d18/f38 [4194304,4194304] 0 2024-04-02T04:42:33.449 INFO:tasks.workunit.client.0.smithi016.stdout:4/149: sync 2024-04-02T04:42:33.449 INFO:tasks.workunit.client.0.smithi016.stdout:8/162: sync 2024-04-02T04:42:33.450 INFO:tasks.workunit.client.0.smithi016.stdout:8/163: write d1/f19 [18399,96919] 0 2024-04-02T04:42:33.452 INFO:tasks.workunit.client.0.smithi016.stdout:5/187: creat d3/df/d18/d26/f4b x:0 0 0 2024-04-02T04:42:33.453 INFO:tasks.workunit.client.0.smithi016.stdout:5/188: write d3/df/d18/f35 [3118189,98218] 0 2024-04-02T04:42:33.454 INFO:tasks.workunit.client.0.smithi016.stdout:5/189: readlink d3/d40/l44 0 2024-04-02T04:42:33.462 INFO:tasks.workunit.client.0.smithi016.stdout:8/164: dwrite d1/f15 [0,4194304] 0 2024-04-02T04:42:33.463 INFO:tasks.workunit.client.0.smithi016.stdout:4/150: link d5/d8/fb d5/d17/d28/f29 0 2024-04-02T04:42:33.465 INFO:tasks.workunit.client.0.smithi016.stdout:6/155: dwrite d0/f7 [0,4194304] 0 2024-04-02T04:42:33.469 INFO:tasks.workunit.client.0.smithi016.stdout:5/190: mkdir d3/d4c 0 2024-04-02T04:42:33.471 INFO:tasks.workunit.client.0.smithi016.stdout:5/191: truncate d3/df/d18/f43 5128038 0 2024-04-02T04:42:33.472 INFO:tasks.workunit.client.0.smithi016.stdout:6/156: dwrite d0/f6 [8388608,4194304] 0 2024-04-02T04:42:33.479 INFO:tasks.workunit.client.0.smithi016.stdout:4/151: write d5/d17/d28/f29 [784702,103163] 0 2024-04-02T04:42:33.479 INFO:tasks.workunit.client.0.smithi016.stdout:2/152: dwrite d12/d26/f2c [0,4194304] 0 2024-04-02T04:42:33.482 INFO:tasks.workunit.client.0.smithi016.stdout:9/164: getdents d0/d3 0 2024-04-02T04:42:33.486 INFO:tasks.workunit.client.0.smithi016.stdout:5/192: rmdir d3/df/d18/d2f 39 2024-04-02T04:42:33.487 INFO:tasks.workunit.client.0.smithi016.stdout:5/193: truncate d3/d25/f47 984342 0 2024-04-02T04:42:33.491 INFO:tasks.workunit.client.0.smithi016.stdout:6/157: dread d0/d9/d16/f19 [0,4194304] 0 2024-04-02T04:42:33.492 INFO:tasks.workunit.client.0.smithi016.stdout:8/165: rename d1/l14 to d1/d6/l32 0 2024-04-02T04:42:33.495 INFO:tasks.workunit.client.0.smithi016.stdout:2/153: rmdir d12 39 2024-04-02T04:42:33.495 INFO:tasks.workunit.client.0.smithi016.stdout:8/166: truncate d1/d6/f1b 540393 0 2024-04-02T04:42:33.496 INFO:tasks.workunit.client.0.smithi016.stdout:8/167: chown d1 4895 1 2024-04-02T04:42:33.497 INFO:tasks.workunit.client.0.smithi016.stdout:4/152: creat d5/d17/d18/d1f/f2a x:0 0 0 2024-04-02T04:42:33.498 INFO:tasks.workunit.client.0.smithi016.stdout:8/168: write d1/f15 [3853666,108164] 0 2024-04-02T04:42:33.500 INFO:tasks.workunit.client.0.smithi016.stdout:6/158: dwrite d0/d9/d16/f20 [0,4194304] 0 2024-04-02T04:42:33.501 INFO:tasks.workunit.client.0.smithi016.stdout:5/194: dread d3/df/f1e [0,4194304] 0 2024-04-02T04:42:33.506 INFO:tasks.workunit.client.0.smithi016.stdout:3/195: write d2/f14 [816029,54753] 0 2024-04-02T04:42:33.509 INFO:tasks.workunit.client.0.smithi016.stdout:4/153: dwrite d5/d1d/f23 [0,4194304] 0 2024-04-02T04:42:33.512 INFO:tasks.workunit.client.0.smithi016.stdout:5/195: dwrite d3/f1a [0,4194304] 0 2024-04-02T04:42:33.512 INFO:tasks.workunit.client.0.smithi016.stdout:3/196: read d2/d15/d16/f2c [442705,35861] 0 2024-04-02T04:42:33.519 INFO:tasks.workunit.client.0.smithi016.stdout:8/169: rename d1/d6/f1b to d1/d24/f33 0 2024-04-02T04:42:33.529 INFO:tasks.workunit.client.0.smithi016.stdout:9/165: creat d0/f2c x:0 0 0 2024-04-02T04:42:33.533 INFO:tasks.workunit.client.0.smithi016.stdout:3/197: creat d2/f3e x:0 0 0 2024-04-02T04:42:33.534 INFO:tasks.workunit.client.0.smithi016.stdout:3/198: truncate d2/d15/d16/f27 742220 0 2024-04-02T04:42:33.538 INFO:tasks.workunit.client.0.smithi016.stdout:5/196: rename d3/l1d to d3/d25/l4d 0 2024-04-02T04:42:33.542 INFO:tasks.workunit.client.0.smithi016.stdout:2/154: link d12/d20/f25 d12/d14/f37 0 2024-04-02T04:42:33.542 INFO:tasks.workunit.client.0.smithi016.stdout:8/170: dread d1/d6/f21 [0,4194304] 0 2024-04-02T04:42:33.545 INFO:tasks.workunit.client.0.smithi016.stdout:9/166: unlink d0/d16/d1a/f22 0 2024-04-02T04:42:33.550 INFO:tasks.workunit.client.0.smithi016.stdout:6/159: creat d0/f28 x:0 0 0 2024-04-02T04:42:33.550 INFO:tasks.workunit.client.0.smithi016.stdout:0/127: dwrite d0/d16/d24/f2a [0,4194304] 0 2024-04-02T04:42:33.556 INFO:tasks.workunit.client.0.smithi016.stdout:2/155: symlink d12/d35/l38 0 2024-04-02T04:42:33.557 INFO:tasks.workunit.client.0.smithi016.stdout:0/128: dwrite d0/f12 [0,4194304] 0 2024-04-02T04:42:33.557 INFO:tasks.workunit.client.0.smithi016.stdout:3/199: dwrite d2/d15/d1a/f39 [0,4194304] 0 2024-04-02T04:42:33.558 INFO:tasks.workunit.client.0.smithi016.stdout:3/200: chown d2/d15/c2d 320933 1 2024-04-02T04:42:33.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:33 smithi016 ceph-mon[28969]: Reconfiguring daemon prometheus.smithi016 on smithi016 2024-04-02T04:42:33.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:33 smithi016 ceph-mon[28969]: pgmap v15: 65 pgs: 65 active+clean; 245 MiB data, 854 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s rd, 7.5 MiB/s wr, 260 op/s 2024-04-02T04:42:33.560 INFO:tasks.workunit.client.0.smithi016.stdout:8/171: mknod d1/d6/c34 0 2024-04-02T04:42:33.572 INFO:tasks.workunit.client.0.smithi016.stdout:6/160: creat d0/d9/d16/f29 x:0 0 0 2024-04-02T04:42:33.576 INFO:tasks.workunit.client.0.smithi016.stdout:5/197: rename d3/cd to d3/d40/c4e 0 2024-04-02T04:42:33.577 INFO:tasks.workunit.client.0.smithi016.stdout:2/156: creat d12/d14/d21/f39 x:0 0 0 2024-04-02T04:42:33.580 INFO:tasks.workunit.client.0.smithi016.stdout:0/129: creat d0/f32 x:0 0 0 2024-04-02T04:42:33.582 INFO:tasks.workunit.client.0.smithi016.stdout:0/130: dread d0/d16/f18 [0,4194304] 0 2024-04-02T04:42:33.583 INFO:tasks.workunit.client.0.smithi016.stdout:8/172: unlink d1/d24/l27 0 2024-04-02T04:42:33.583 INFO:tasks.workunit.client.0.smithi016.stdout:0/131: fdatasync d0/fc 0 2024-04-02T04:42:33.586 INFO:tasks.workunit.client.0.smithi016.stdout:8/173: dread d1/d6/f2e [0,4194304] 0 2024-04-02T04:42:33.591 INFO:tasks.workunit.client.0.smithi016.stdout:5/198: mkdir d3/d40/d4f 0 2024-04-02T04:42:33.593 INFO:tasks.workunit.client.0.smithi016.stdout:2/157: symlink d12/d14/l3a 0 2024-04-02T04:42:33.599 INFO:tasks.workunit.client.0.smithi016.stdout:3/201: fsync d2/f7 0 2024-04-02T04:42:33.600 INFO:tasks.workunit.client.0.smithi016.stdout:8/174: dread d1/d6/f10 [0,4194304] 0 2024-04-02T04:42:33.601 INFO:tasks.workunit.client.0.smithi016.stdout:0/132: unlink d0/cf 0 2024-04-02T04:42:33.601 INFO:tasks.workunit.client.0.smithi016.stdout:0/133: fdatasync d0/f12 0 2024-04-02T04:42:33.609 INFO:tasks.workunit.client.0.smithi016.stdout:4/154: truncate d5/d8/fb 782587 0 2024-04-02T04:42:33.613 INFO:tasks.workunit.client.0.smithi016.stdout:5/199: dwrite d3/f17 [0,4194304] 0 2024-04-02T04:42:33.618 INFO:tasks.workunit.client.0.smithi016.stdout:3/202: read d2/f1f [378806,2371] 0 2024-04-02T04:42:33.622 INFO:tasks.workunit.client.0.smithi016.stdout:0/134: rename d0/c21 to d0/d29/c33 0 2024-04-02T04:42:33.622 INFO:tasks.workunit.client.0.smithi016.stdout:0/135: dread - d0/d16/d24/f1e zero size 2024-04-02T04:42:33.635 INFO:tasks.workunit.client.0.smithi016.stdout:9/167: truncate d0/f7 93108 0 2024-04-02T04:42:33.636 INFO:tasks.workunit.client.0.smithi016.stdout:8/175: unlink d1/d6/l32 0 2024-04-02T04:42:33.639 INFO:tasks.workunit.client.0.smithi016.stdout:3/203: rename f0 to d2/f3f 0 2024-04-02T04:42:33.645 INFO:tasks.workunit.client.0.smithi016.stdout:5/200: mknod d3/df/d18/c50 0 2024-04-02T04:42:33.645 INFO:tasks.workunit.client.0.smithi016.stdout:5/201: fdatasync d3/df/f1c 0 2024-04-02T04:42:33.650 INFO:tasks.workunit.client.0.smithi016.stdout:3/204: rmdir d2/d2e 39 2024-04-02T04:42:33.652 INFO:tasks.workunit.client.0.smithi016.stdout:6/161: write d0/f4 [1160006,63240] 0 2024-04-02T04:42:33.655 INFO:tasks.workunit.client.0.smithi016.stdout:2/158: getdents d12/d14/d21 0 2024-04-02T04:42:33.662 INFO:tasks.workunit.client.0.smithi016.stdout:5/202: dread f1 [0,4194304] 0 2024-04-02T04:42:33.663 INFO:tasks.workunit.client.0.smithi016.stdout:8/176: dwrite d1/f1d [0,4194304] 0 2024-04-02T04:42:33.664 INFO:tasks.workunit.client.0.smithi016.stdout:2/159: chown c11 5 1 2024-04-02T04:42:33.665 INFO:tasks.workunit.client.0.smithi016.stdout:9/168: creat d0/f2d x:0 0 0 2024-04-02T04:42:33.668 INFO:tasks.workunit.client.0.smithi016.stdout:3/205: fdatasync d2/d2e/f2f 0 2024-04-02T04:42:33.670 INFO:tasks.workunit.client.0.smithi016.stdout:6/162: link d0/f6 d0/d9/d16/f2a 0 2024-04-02T04:42:33.672 INFO:tasks.workunit.client.0.smithi016.stdout:3/206: truncate d2/d2e/f30 145577 0 2024-04-02T04:42:33.674 INFO:tasks.workunit.client.0.smithi016.stdout:8/177: dwrite d1/f15 [0,4194304] 0 2024-04-02T04:42:33.674 INFO:tasks.workunit.client.0.smithi016.stdout:2/160: dwrite d12/f16 [0,4194304] 0 2024-04-02T04:42:33.674 INFO:tasks.workunit.client.0.smithi016.stdout:5/203: unlink d3/df/d18/f1b 0 2024-04-02T04:42:33.676 INFO:tasks.workunit.client.0.smithi016.stdout:8/178: chown d1/d31 92853192 1 2024-04-02T04:42:33.676 INFO:tasks.workunit.client.0.smithi016.stdout:7/163: sync 2024-04-02T04:42:33.676 INFO:tasks.workunit.client.0.smithi016.stdout:1/176: sync 2024-04-02T04:42:33.677 INFO:tasks.workunit.client.0.smithi016.stdout:1/177: stat d14/c1a 0 2024-04-02T04:42:33.678 INFO:tasks.workunit.client.0.smithi016.stdout:8/179: chown d1/d6/f1a 2555085 1 2024-04-02T04:42:33.683 INFO:tasks.workunit.client.0.smithi016.stdout:1/178: write d14/d19/d1e/f2b [973408,60052] 0 2024-04-02T04:42:33.683 INFO:tasks.workunit.client.0.smithi016.stdout:8/180: chown d1/d6/f1a 128 1 2024-04-02T04:42:33.683 INFO:tasks.workunit.client.0.smithi016.stdout:9/169: rename d0/d3/f13 to d0/d9/f2e 0 2024-04-02T04:42:33.685 INFO:tasks.workunit.client.0.smithi016.stdout:2/161: dread d12/d14/f1e [0,4194304] 0 2024-04-02T04:42:33.686 INFO:tasks.workunit.client.0.smithi016.stdout:9/170: readlink d0/d3/l10 0 2024-04-02T04:42:33.686 INFO:tasks.workunit.client.0.smithi016.stdout:1/179: fsync d14/f17 0 2024-04-02T04:42:33.686 INFO:tasks.workunit.client.0.smithi016.stdout:8/181: readlink d1/d24/l26 0 2024-04-02T04:42:33.686 INFO:tasks.workunit.client.0.smithi016.stdout:5/204: dwrite d3/df/d18/f43 [0,4194304] 0 2024-04-02T04:42:33.687 INFO:tasks.workunit.client.0.smithi016.stdout:2/162: chown f10 22983 1 2024-04-02T04:42:33.693 INFO:tasks.workunit.client.0.smithi016.stdout:3/207: rename d2/d15/d1a/l2a to d2/d15/d1a/l40 0 2024-04-02T04:42:33.695 INFO:tasks.workunit.client.0.smithi016.stdout:0/136: dwrite d0/f4 [4194304,4194304] 0 2024-04-02T04:42:33.697 INFO:tasks.workunit.client.0.smithi016.stdout:9/171: dwrite d0/f27 [0,4194304] 0 2024-04-02T04:42:33.697 INFO:tasks.workunit.client.0.smithi016.stdout:0/137: fsync d0/d16/d24/f1e 0 2024-04-02T04:42:33.703 INFO:tasks.workunit.client.0.smithi016.stdout:0/138: dwrite d0/f19 [0,4194304] 0 2024-04-02T04:42:33.713 INFO:tasks.workunit.client.0.smithi016.stdout:2/163: mkdir d12/d35/d3b 0 2024-04-02T04:42:33.716 INFO:tasks.workunit.client.0.smithi016.stdout:1/180: mknod d14/d19/d1d/c33 0 2024-04-02T04:42:33.723 INFO:tasks.workunit.client.0.smithi016.stdout:3/208: symlink d2/d15/d16/l41 0 2024-04-02T04:42:33.732 INFO:tasks.workunit.client.0.smithi016.stdout:7/164: creat d0/d18/d15/d16/d31/f35 x:0 0 0 2024-04-02T04:42:33.734 INFO:tasks.workunit.client.0.smithi016.stdout:0/139: creat d0/d16/f34 x:0 0 0 2024-04-02T04:42:33.739 INFO:tasks.workunit.client.0.smithi016.stdout:2/164: creat d12/d14/f3c x:0 0 0 2024-04-02T04:42:33.744 INFO:tasks.workunit.client.0.smithi016.stdout:6/163: creat d0/d9/f2b x:0 0 0 2024-04-02T04:42:33.745 INFO:tasks.workunit.client.0.smithi016.stdout:5/205: creat d3/d25/d45/f51 x:0 0 0 2024-04-02T04:42:33.748 INFO:tasks.workunit.client.0.smithi016.stdout:4/155: sync 2024-04-02T04:42:33.750 INFO:tasks.workunit.client.0.smithi016.stdout:5/206: dwrite d3/df/f1c [0,4194304] 0 2024-04-02T04:42:33.751 INFO:tasks.workunit.client.0.smithi016.stdout:7/165: creat d0/d21/f36 x:0 0 0 2024-04-02T04:42:33.753 INFO:tasks.workunit.client.0.smithi016.stdout:0/140: mkdir d0/d16/d24/d35 0 2024-04-02T04:42:33.759 INFO:tasks.workunit.client.0.smithi016.stdout:6/164: creat d0/d8/f2c x:0 0 0 2024-04-02T04:42:33.813 INFO:tasks.workunit.client.0.smithi016.stdout:7/166: dwrite d0/d18/d15/d16/f34 [0,4194304] 0 2024-04-02T04:42:33.814 INFO:tasks.workunit.client.0.smithi016.stdout:1/181: write d14/d19/d1e/f2e [3551339,6429] 0 2024-04-02T04:42:33.817 INFO:tasks.workunit.client.0.smithi016.stdout:8/182: getdents d1 0 2024-04-02T04:42:33.818 INFO:tasks.workunit.client.0.smithi016.stdout:0/141: write d0/d16/f17 [5050704,89599] 0 2024-04-02T04:42:33.820 INFO:tasks.workunit.client.0.smithi016.stdout:9/172: dread d0/f7 [0,4194304] 0 2024-04-02T04:42:33.820 INFO:tasks.workunit.client.0.smithi016.stdout:2/165: dwrite f8 [0,4194304] 0 2024-04-02T04:42:33.820 INFO:tasks.workunit.client.0.smithi016.stdout:3/209: getdents d2/d15/d1a 0 2024-04-02T04:42:33.823 INFO:tasks.workunit.client.0.smithi016.stdout:6/165: rename d0/d9/d16/f19 to d0/d1f/f2d 0 2024-04-02T04:42:33.824 INFO:tasks.workunit.client.0.smithi016.stdout:8/183: truncate d1/ff 8791610 0 2024-04-02T04:42:33.824 INFO:tasks.workunit.client.0.smithi016.stdout:0/142: chown d0/d16/d24/d35 8973734 1 2024-04-02T04:42:33.825 INFO:tasks.workunit.client.0.smithi016.stdout:8/184: chown d1/f3 100 1 2024-04-02T04:42:33.825 INFO:tasks.workunit.client.0.smithi016.stdout:0/143: read - d0/f10 zero size 2024-04-02T04:42:33.828 INFO:tasks.workunit.client.0.smithi016.stdout:4/156: dwrite d5/f10 [0,4194304] 0 2024-04-02T04:42:33.828 INFO:tasks.workunit.client.0.smithi016.stdout:2/166: creat d12/d14/d21/d2b/f3d x:0 0 0 2024-04-02T04:42:33.829 INFO:tasks.workunit.client.0.smithi016.stdout:0/144: write d0/d16/f17 [3430289,128307] 0 2024-04-02T04:42:33.830 INFO:tasks.workunit.client.0.smithi016.stdout:9/173: symlink d0/d3/l2f 0 2024-04-02T04:42:33.831 INFO:tasks.workunit.client.0.smithi016.stdout:4/157: read d5/d8/ff [227503,41931] 0 2024-04-02T04:42:33.832 INFO:tasks.workunit.client.0.smithi016.stdout:9/174: write d0/f2c [641440,74321] 0 2024-04-02T04:42:33.832 INFO:tasks.workunit.client.0.smithi016.stdout:3/210: dwrite d2/fa [0,4194304] 0 2024-04-02T04:42:33.834 INFO:tasks.workunit.client.0.smithi016.stdout:1/182: creat d14/f34 x:0 0 0 2024-04-02T04:42:33.836 INFO:tasks.workunit.client.0.smithi016.stdout:7/167: getdents d0/d18 0 2024-04-02T04:42:33.837 INFO:tasks.workunit.client.0.smithi016.stdout:7/168: fsync d0/d18/d15/d16/f33 0 2024-04-02T04:42:33.838 INFO:tasks.workunit.client.0.smithi016.stdout:0/145: dread d0/d16/d24/f2b [0,4194304] 0 2024-04-02T04:42:33.838 INFO:tasks.workunit.client.0.smithi016.stdout:8/185: rename d1/d6/l12 to d1/d31/l35 0 2024-04-02T04:42:33.847 INFO:tasks.workunit.client.0.smithi016.stdout:9/175: creat d0/d3/f30 x:0 0 0 2024-04-02T04:42:33.850 INFO:tasks.workunit.client.0.smithi016.stdout:4/158: creat d5/d17/d18/f2b x:0 0 0 2024-04-02T04:42:33.851 INFO:tasks.workunit.client.0.smithi016.stdout:3/211: symlink d2/d15/d16/l42 0 2024-04-02T04:42:33.852 INFO:tasks.workunit.client.0.smithi016.stdout:1/183: symlink d14/d19/d20/l35 0 2024-04-02T04:42:33.855 INFO:tasks.workunit.client.0.smithi016.stdout:1/184: chown d14/d19/d1d/c33 2782693 1 2024-04-02T04:42:33.856 INFO:tasks.workunit.client.0.smithi016.stdout:2/167: mknod d12/c3e 0 2024-04-02T04:42:33.859 INFO:tasks.workunit.client.0.smithi016.stdout:6/166: rename d0/cb to d0/c2e 0 2024-04-02T04:42:33.860 INFO:tasks.workunit.client.0.smithi016.stdout:9/176: dwrite d0/d3/f30 [0,4194304] 0 2024-04-02T04:42:33.860 INFO:tasks.workunit.client.0.smithi016.stdout:8/186: dread d1/d6/f1a [0,4194304] 0 2024-04-02T04:42:33.862 INFO:tasks.workunit.client.0.smithi016.stdout:3/212: dread d2/d15/d1a/f29 [0,4194304] 0 2024-04-02T04:42:33.863 INFO:tasks.workunit.client.0.smithi016.stdout:7/169: dread d0/d18/d15/f1a [0,4194304] 0 2024-04-02T04:42:33.863 INFO:tasks.workunit.client.0.smithi016.stdout:1/185: mknod d14/d19/d1d/c36 0 2024-04-02T04:42:33.864 INFO:tasks.workunit.client.0.smithi016.stdout:6/167: mknod d0/d1f/c2f 0 2024-04-02T04:42:33.865 INFO:tasks.workunit.client.0.smithi016.stdout:3/213: truncate d2/d2e/f30 429037 0 2024-04-02T04:42:33.866 INFO:tasks.workunit.client.0.smithi016.stdout:9/177: mkdir d0/d3/d31 0 2024-04-02T04:42:33.866 INFO:tasks.workunit.client.0.smithi016.stdout:4/159: dread d5/d8/fb [0,4194304] 0 2024-04-02T04:42:33.868 INFO:tasks.workunit.client.0.smithi016.stdout:3/214: write d2/d15/f31 [293351,5722] 0 2024-04-02T04:42:33.868 INFO:tasks.workunit.client.0.smithi016.stdout:3/215: dread - d2/f28 zero size 2024-04-02T04:42:33.869 INFO:tasks.workunit.client.0.smithi016.stdout:8/187: mkdir d1/d6/d36 0 2024-04-02T04:42:33.872 INFO:tasks.workunit.client.0.smithi016.stdout:0/146: dread d0/fe [0,4194304] 0 2024-04-02T04:42:33.872 INFO:tasks.workunit.client.0.smithi016.stdout:3/216: dread d2/d15/d16/f27 [0,4194304] 0 2024-04-02T04:42:33.874 INFO:tasks.workunit.client.0.smithi016.stdout:1/186: rename f11 to d14/d19/d1e/f37 0 2024-04-02T04:42:33.874 INFO:tasks.workunit.client.0.smithi016.stdout:3/217: dread - d2/f28 zero size 2024-04-02T04:42:33.874 INFO:tasks.workunit.client.0.smithi016.stdout:6/168: dwrite d0/d9/df/f10 [0,4194304] 0 2024-04-02T04:42:33.875 INFO:tasks.workunit.client.0.smithi016.stdout:0/147: dread d0/fe [0,4194304] 0 2024-04-02T04:42:33.876 INFO:tasks.workunit.client.0.smithi016.stdout:0/148: chown d0/f12 1830473 1 2024-04-02T04:42:33.877 INFO:tasks.workunit.client.0.smithi016.stdout:4/160: mknod d5/d17/c2c 0 2024-04-02T04:42:33.895 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:33 smithi027 ceph-mon[37550]: Reconfiguring daemon prometheus.smithi016 on smithi016 2024-04-02T04:42:33.895 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:33 smithi027 ceph-mon[37550]: pgmap v15: 65 pgs: 65 active+clean; 245 MiB data, 854 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s rd, 7.5 MiB/s wr, 260 op/s 2024-04-02T04:42:33.928 INFO:tasks.workunit.client.0.smithi016.stdout:9/178: rename d0/l6 to d0/d16/d1a/l32 0 2024-04-02T04:42:33.928 INFO:tasks.workunit.client.0.smithi016.stdout:3/218: mkdir d2/d15/d16/d35/d43 0 2024-04-02T04:42:33.928 INFO:tasks.workunit.client.0.smithi016.stdout:0/149: truncate d0/d16/f18 965593 0 2024-04-02T04:42:33.930 INFO:tasks.workunit.client.0.smithi016.stdout:3/219: fdatasync d2/d15/d1a/f22 0 2024-04-02T04:42:33.934 INFO:tasks.workunit.client.0.smithi016.stdout:8/188: link d1/d6/f2e d1/d6/d36/f37 0 2024-04-02T04:42:33.935 INFO:tasks.workunit.client.0.smithi016.stdout:0/150: mkdir d0/d16/d2e/d36 0 2024-04-02T04:42:33.936 INFO:tasks.workunit.client.0.smithi016.stdout:9/179: dwrite d0/d9/db/f25 [0,4194304] 0 2024-04-02T04:42:33.936 INFO:tasks.workunit.client.0.smithi016.stdout:0/151: stat d0/d16/f31 0 2024-04-02T04:42:33.937 INFO:tasks.workunit.client.0.smithi016.stdout:0/152: write d0/f4 [1479565,98416] 0 2024-04-02T04:42:33.938 INFO:tasks.workunit.client.0.smithi016.stdout:6/169: rename d0/d9/c14 to d0/c30 0 2024-04-02T04:42:33.940 INFO:tasks.workunit.client.0.smithi016.stdout:0/153: dread - d0/fc zero size 2024-04-02T04:42:33.944 INFO:tasks.workunit.client.0.smithi016.stdout:0/154: dread d0/fe [0,4194304] 0 2024-04-02T04:42:33.944 INFO:tasks.workunit.client.0.smithi016.stdout:9/180: dwrite d0/f2d [0,4194304] 0 2024-04-02T04:42:33.945 INFO:tasks.workunit.client.0.smithi016.stdout:1/187: dread fb [0,4194304] 0 2024-04-02T04:42:33.945 INFO:tasks.workunit.client.0.smithi016.stdout:4/161: rename d5/d17/d28 to d5/d1d/d2d 0 2024-04-02T04:42:33.950 INFO:tasks.workunit.client.0.smithi016.stdout:6/170: fdatasync d0/d1f/f2d 0 2024-04-02T04:42:33.951 INFO:tasks.workunit.client.0.smithi016.stdout:6/171: read - d0/d9/f2b zero size 2024-04-02T04:42:33.954 INFO:tasks.workunit.client.0.smithi016.stdout:9/181: readlink d0/d16/d1a/l32 0 2024-04-02T04:42:33.954 INFO:tasks.workunit.client.0.smithi016.stdout:9/182: dread d0/f7 [0,4194304] 0 2024-04-02T04:42:33.956 INFO:tasks.workunit.client.0.smithi016.stdout:3/220: rename f1 to d2/d15/d16/f44 0 2024-04-02T04:42:33.957 INFO:tasks.workunit.client.0.smithi016.stdout:0/155: mkdir d0/d16/d2e/d37 0 2024-04-02T04:42:33.959 INFO:tasks.workunit.client.0.smithi016.stdout:0/156: truncate d0/f10 586683 0 2024-04-02T04:42:33.960 INFO:tasks.workunit.client.0.smithi016.stdout:3/221: dread d2/d15/f31 [0,4194304] 0 2024-04-02T04:42:33.960 INFO:tasks.workunit.client.0.smithi016.stdout:4/162: creat d5/d17/d18/f2e x:0 0 0 2024-04-02T04:42:33.962 INFO:tasks.workunit.client.0.smithi016.stdout:4/163: read d5/d1d/f21 [5101547,90716] 0 2024-04-02T04:42:33.962 INFO:tasks.workunit.client.0.smithi016.stdout:1/188: unlink d14/d19/d1e/f37 0 2024-04-02T04:42:33.962 INFO:tasks.workunit.client.0.smithi016.stdout:4/164: write d5/f7 [199711,126395] 0 2024-04-02T04:42:33.964 INFO:tasks.workunit.client.0.smithi016.stdout:4/165: chown d5/d1d/f21 287 1 2024-04-02T04:42:33.969 INFO:tasks.workunit.client.0.smithi016.stdout:0/157: creat d0/f38 x:0 0 0 2024-04-02T04:42:33.970 INFO:tasks.workunit.client.0.smithi016.stdout:0/158: dread - d0/d16/f31 zero size 2024-04-02T04:42:33.974 INFO:tasks.workunit.client.0.smithi016.stdout:8/189: write d1/d6/f10 [4595619,92520] 0 2024-04-02T04:42:33.975 INFO:tasks.workunit.client.0.smithi016.stdout:4/166: symlink d5/d16/l2f 0 2024-04-02T04:42:33.976 INFO:tasks.workunit.client.0.smithi016.stdout:6/172: truncate d0/f6 3554220 0 2024-04-02T04:42:33.978 INFO:tasks.workunit.client.0.smithi016.stdout:9/183: mkdir d0/d33 0 2024-04-02T04:42:33.979 INFO:tasks.workunit.client.0.smithi016.stdout:9/184: write d0/d3/f30 [3518015,95322] 0 2024-04-02T04:42:33.980 INFO:tasks.workunit.client.0.smithi016.stdout:0/159: readlink d0/l5 0 2024-04-02T04:42:33.984 INFO:tasks.workunit.client.0.smithi016.stdout:1/189: link d14/f17 d14/d19/d20/f38 0 2024-04-02T04:42:33.984 INFO:tasks.workunit.client.0.smithi016.stdout:1/190: chown c5 74 1 2024-04-02T04:42:33.985 INFO:tasks.workunit.client.0.smithi016.stdout:8/190: mkdir d1/d24/d38 0 2024-04-02T04:42:33.985 INFO:tasks.workunit.client.0.smithi016.stdout:1/191: write d14/d19/d1d/f30 [570287,107658] 0 2024-04-02T04:42:33.985 INFO:tasks.workunit.client.0.smithi016.stdout:8/191: read - d1/d24/f2b zero size 2024-04-02T04:42:33.987 INFO:tasks.workunit.client.0.smithi016.stdout:4/167: mknod d5/d1d/c30 0 2024-04-02T04:42:33.993 INFO:tasks.workunit.client.0.smithi016.stdout:9/185: rename d0/d16/d1a/l1d to d0/d16/d1a/l34 0 2024-04-02T04:42:33.994 INFO:tasks.workunit.client.0.smithi016.stdout:5/207: sync 2024-04-02T04:42:33.994 INFO:tasks.workunit.client.0.smithi016.stdout:2/168: sync 2024-04-02T04:42:33.994 INFO:tasks.workunit.client.0.smithi016.stdout:7/170: sync 2024-04-02T04:42:33.994 INFO:tasks.workunit.client.0.smithi016.stdout:0/160: symlink d0/d16/d23/l39 0 2024-04-02T04:42:33.995 INFO:tasks.workunit.client.0.smithi016.stdout:7/171: fdatasync d0/d18/d15/f1d 0 2024-04-02T04:42:33.995 INFO:tasks.workunit.client.0.smithi016.stdout:5/208: fdatasync d3/df/d18/d26/d48/f49 0 2024-04-02T04:42:33.995 INFO:tasks.workunit.client.0.smithi016.stdout:8/192: fdatasync d1/d24/f2a 0 2024-04-02T04:42:33.995 INFO:tasks.workunit.client.0.smithi016.stdout:3/222: write d2/d15/f23 [1164982,64712] 0 2024-04-02T04:42:33.996 INFO:tasks.workunit.client.0.smithi016.stdout:7/172: stat d0/d18/d15/f1d 0 2024-04-02T04:42:33.997 INFO:tasks.workunit.client.0.smithi016.stdout:1/192: rename d14/d19/d31 to d14/d19/d1d/d29/d39 0 2024-04-02T04:42:33.997 INFO:tasks.workunit.client.0.smithi016.stdout:3/223: read - d2/d2e/f2f zero size 2024-04-02T04:42:33.998 INFO:tasks.workunit.client.0.smithi016.stdout:7/173: read d0/d18/d15/f1d [2316345,28696] 0 2024-04-02T04:42:33.999 INFO:tasks.workunit.client.0.smithi016.stdout:4/168: mknod d5/de/c31 0 2024-04-02T04:42:33.999 INFO:tasks.workunit.client.0.smithi016.stdout:7/174: fdatasync d0/d21/f36 0 2024-04-02T04:42:34.005 INFO:tasks.workunit.client.0.smithi016.stdout:7/175: dwrite d0/d18/d15/f2c [0,4194304] 0 2024-04-02T04:42:34.010 INFO:tasks.workunit.client.0.smithi016.stdout:5/209: unlink d3/f3e 0 2024-04-02T04:42:34.013 INFO:tasks.workunit.client.0.smithi016.stdout:8/193: creat d1/d6/f39 x:0 0 0 2024-04-02T04:42:34.017 INFO:tasks.workunit.client.0.smithi016.stdout:2/169: rename d12/d14/d21 to d12/d35/d3f 0 2024-04-02T04:42:34.018 INFO:tasks.workunit.client.0.smithi016.stdout:8/194: dwrite d1/f15 [0,4194304] 0 2024-04-02T04:42:34.020 INFO:tasks.workunit.client.0.smithi016.stdout:3/224: mkdir d2/d15/d1a/d45 0 2024-04-02T04:42:34.020 INFO:tasks.workunit.client.0.smithi016.stdout:4/169: mkdir d5/d16/d32 0 2024-04-02T04:42:34.022 INFO:tasks.workunit.client.0.smithi016.stdout:9/186: creat d0/d33/f35 x:0 0 0 2024-04-02T04:42:34.023 INFO:tasks.workunit.client.0.smithi016.stdout:7/176: mkdir d0/d18/d15/d37 0 2024-04-02T04:42:34.024 INFO:tasks.workunit.client.0.smithi016.stdout:5/210: creat d3/df/d18/d26/d2c/f52 x:0 0 0 2024-04-02T04:42:34.024 INFO:tasks.workunit.client.0.smithi016.stdout:7/177: chown d0/d18/d15/l12 1776 1 2024-04-02T04:42:34.026 INFO:tasks.workunit.client.0.smithi016.stdout:2/170: creat d12/d35/f40 x:0 0 0 2024-04-02T04:42:34.027 INFO:tasks.workunit.client.0.smithi016.stdout:8/195: rename d1/d6/l23 to d1/d31/l3a 0 2024-04-02T04:42:34.028 INFO:tasks.workunit.client.0.smithi016.stdout:3/225: dwrite d2/f20 [0,4194304] 0 2024-04-02T04:42:34.028 INFO:tasks.workunit.client.0.smithi016.stdout:4/170: mkdir d5/de/d33 0 2024-04-02T04:42:34.029 INFO:tasks.workunit.client.0.smithi016.stdout:9/187: creat d0/d33/f36 x:0 0 0 2024-04-02T04:42:34.034 INFO:tasks.workunit.client.0.smithi016.stdout:5/211: symlink d3/df/d18/d2f/l53 0 2024-04-02T04:42:34.036 INFO:tasks.workunit.client.0.smithi016.stdout:9/188: creat d0/d16/d1a/f37 x:0 0 0 2024-04-02T04:42:34.037 INFO:tasks.workunit.client.0.smithi016.stdout:2/171: dwrite d12/d35/d3f/f2d [0,4194304] 0 2024-04-02T04:42:34.038 INFO:tasks.workunit.client.0.smithi016.stdout:4/171: rename d5/de/c11 to d5/c34 0 2024-04-02T04:42:34.039 INFO:tasks.workunit.client.0.smithi016.stdout:5/212: creat d3/df/d18/d26/f54 x:0 0 0 2024-04-02T04:42:34.041 INFO:tasks.workunit.client.0.smithi016.stdout:2/172: rename d12/d14/l17 to d12/d35/d3b/l41 0 2024-04-02T04:42:34.042 INFO:tasks.workunit.client.0.smithi016.stdout:3/226: mkdir d2/d15/d16/d3d/d46 0 2024-04-02T04:42:34.042 INFO:tasks.workunit.client.0.smithi016.stdout:3/227: readlink d2/l33 0 2024-04-02T04:42:34.046 INFO:tasks.workunit.client.0.smithi016.stdout:5/213: dwrite d3/f3b [0,4194304] 0 2024-04-02T04:42:34.050 INFO:tasks.workunit.client.0.smithi016.stdout:3/228: creat d2/d15/d1a/f47 x:0 0 0 2024-04-02T04:42:34.061 INFO:tasks.workunit.client.0.smithi016.stdout:3/229: dread d2/f9 [0,4194304] 0 2024-04-02T04:42:34.061 INFO:tasks.workunit.client.0.smithi016.stdout:3/230: stat d2/l10 0 2024-04-02T04:42:34.068 INFO:tasks.workunit.client.0.smithi016.stdout:6/173: write d0/d9/d16/f2a [3666472,115722] 0 2024-04-02T04:42:34.074 INFO:tasks.workunit.client.0.smithi016.stdout:6/174: chown d0/c5 31537 1 2024-04-02T04:42:34.074 INFO:tasks.workunit.client.0.smithi016.stdout:1/193: dwrite d14/d19/d1e/f2b [0,4194304] 0 2024-04-02T04:42:34.077 INFO:tasks.workunit.client.0.smithi016.stdout:0/161: dwrite d0/d16/d24/f2b [0,4194304] 0 2024-04-02T04:42:34.077 INFO:tasks.workunit.client.0.smithi016.stdout:6/175: mknod d0/d1f/c31 0 2024-04-02T04:42:34.083 INFO:tasks.workunit.client.0.smithi016.stdout:1/194: mknod d14/d19/d1d/d29/d39/c3a 0 2024-04-02T04:42:34.086 INFO:tasks.workunit.client.0.smithi016.stdout:0/162: dwrite d0/f10 [0,4194304] 0 2024-04-02T04:42:34.088 INFO:tasks.workunit.client.0.smithi016.stdout:2/173: getdents d12/d35 0 2024-04-02T04:42:34.096 INFO:tasks.workunit.client.0.smithi016.stdout:1/195: creat d14/d19/d20/f3b x:0 0 0 2024-04-02T04:42:34.096 INFO:tasks.workunit.client.0.smithi016.stdout:6/176: getdents d0/d9 0 2024-04-02T04:42:34.097 INFO:tasks.workunit.client.0.smithi016.stdout:6/177: fsync d0/d9/f2b 0 2024-04-02T04:42:34.101 INFO:tasks.workunit.client.0.smithi016.stdout:8/196: write d1/d6/f21 [1671488,84789] 0 2024-04-02T04:42:34.102 INFO:tasks.workunit.client.0.smithi016.stdout:1/196: creat d14/d19/d1d/d29/f3c x:0 0 0 2024-04-02T04:42:34.104 INFO:tasks.workunit.client.0.smithi016.stdout:6/178: creat d0/d9/f32 x:0 0 0 2024-04-02T04:42:34.104 INFO:tasks.workunit.client.0.smithi016.stdout:7/178: truncate d0/d18/d15/d16/f34 8373 0 2024-04-02T04:42:34.105 INFO:tasks.workunit.client.0.smithi016.stdout:8/197: mknod d1/d6/c3b 0 2024-04-02T04:42:34.108 INFO:tasks.workunit.client.0.smithi016.stdout:1/197: creat d14/d19/d1d/d29/f3d x:0 0 0 2024-04-02T04:42:34.112 INFO:tasks.workunit.client.0.smithi016.stdout:7/179: unlink d0/d18/d15/d16/f33 0 2024-04-02T04:42:34.114 INFO:tasks.workunit.client.0.smithi016.stdout:8/198: symlink d1/d31/l3c 0 2024-04-02T04:42:34.117 INFO:tasks.workunit.client.0.smithi016.stdout:9/189: truncate d0/d9/db/f25 3294869 0 2024-04-02T04:42:34.119 INFO:tasks.workunit.client.0.smithi016.stdout:4/172: truncate d5/f7 202040 0 2024-04-02T04:42:34.120 INFO:tasks.workunit.client.0.smithi016.stdout:8/199: rmdir d1/d31 39 2024-04-02T04:42:34.125 INFO:tasks.workunit.client.0.smithi016.stdout:9/190: rename d0/d3/f5 to d0/d33/f38 0 2024-04-02T04:42:34.125 INFO:tasks.workunit.client.0.smithi016.stdout:7/180: symlink d0/l38 0 2024-04-02T04:42:34.181 INFO:tasks.workunit.client.0.smithi016.stdout:4/173: rmdir d5/d1d/d1e 39 2024-04-02T04:42:34.182 INFO:tasks.workunit.client.0.smithi016.stdout:9/191: mkdir d0/d3/d39 0 2024-04-02T04:42:34.183 INFO:tasks.workunit.client.0.smithi016.stdout:7/181: creat d0/d28/f39 x:0 0 0 2024-04-02T04:42:34.183 INFO:tasks.workunit.client.0.smithi016.stdout:0/163: truncate d0/d16/f17 4299016 0 2024-04-02T04:42:34.185 INFO:tasks.workunit.client.0.smithi016.stdout:7/182: write d0/f32 [375604,89347] 0 2024-04-02T04:42:34.186 INFO:tasks.workunit.client.0.smithi016.stdout:3/231: dwrite d2/f1f [0,4194304] 0 2024-04-02T04:42:34.187 INFO:tasks.workunit.client.0.smithi016.stdout:7/183: fsync d0/d21/f36 0 2024-04-02T04:42:34.187 INFO:tasks.workunit.client.0.smithi016.stdout:1/198: truncate d14/d19/d1e/f2e 1592244 0 2024-04-02T04:42:34.189 INFO:tasks.workunit.client.0.smithi016.stdout:7/184: fsync d0/d28/f39 0 2024-04-02T04:42:34.191 INFO:tasks.workunit.client.0.smithi016.stdout:4/174: fdatasync d5/d8/ff 0 2024-04-02T04:42:34.192 INFO:tasks.workunit.client.0.smithi016.stdout:3/232: mkdir d2/d2e/d36/d48 0 2024-04-02T04:42:34.193 INFO:tasks.workunit.client.0.smithi016.stdout:8/200: truncate d1/d24/f2c 42738 0 2024-04-02T04:42:34.194 INFO:tasks.workunit.client.0.smithi016.stdout:0/164: dwrite d0/d16/f34 [0,4194304] 0 2024-04-02T04:42:34.195 INFO:tasks.workunit.client.0.smithi016.stdout:1/199: fsync d14/d19/d1e/f23 0 2024-04-02T04:42:34.196 INFO:tasks.workunit.client.0.smithi016.stdout:1/200: stat d14/d19/d20/l35 0 2024-04-02T04:42:34.196 INFO:tasks.workunit.client.0.smithi016.stdout:0/165: write d0/d16/f31 [544786,87604] 0 2024-04-02T04:42:34.202 INFO:tasks.workunit.client.0.smithi016.stdout:7/185: rmdir d0/d18 39 2024-04-02T04:42:34.207 INFO:tasks.workunit.client.0.smithi016.stdout:3/233: symlink d2/d15/d16/l49 0 2024-04-02T04:42:34.210 INFO:tasks.workunit.client.0.smithi016.stdout:1/201: rmdir d14/d19/d1d/d29 39 2024-04-02T04:42:34.211 INFO:tasks.workunit.client.0.smithi016.stdout:9/192: truncate d0/d16/f20 3519224 0 2024-04-02T04:42:34.212 INFO:tasks.workunit.client.0.smithi016.stdout:3/234: dwrite d2/f26 [0,4194304] 0 2024-04-02T04:42:34.213 INFO:tasks.workunit.client.0.smithi016.stdout:9/193: write d0/d16/f1b [5124400,120450] 0 2024-04-02T04:42:34.214 INFO:tasks.workunit.client.0.smithi016.stdout:0/166: rename d0/c25 to d0/d16/d24/c3a 0 2024-04-02T04:42:34.215 INFO:tasks.workunit.client.0.smithi016.stdout:0/167: fsync d0/d16/d24/f2b 0 2024-04-02T04:42:34.217 INFO:tasks.workunit.client.0.smithi016.stdout:0/168: write d0/d16/d24/f2b [4980773,98285] 0 2024-04-02T04:42:34.218 INFO:tasks.workunit.client.0.smithi016.stdout:0/169: read d0/d16/d24/f2b [3745616,26236] 0 2024-04-02T04:42:34.219 INFO:tasks.workunit.client.0.smithi016.stdout:0/170: write d0/d16/d24/f1e [792525,63855] 0 2024-04-02T04:42:34.224 INFO:tasks.workunit.client.0.smithi016.stdout:1/202: rmdir d14/d19 39 2024-04-02T04:42:34.228 INFO:tasks.workunit.client.0.smithi016.stdout:4/175: truncate d5/d8/fd 3017365 0 2024-04-02T04:42:34.229 INFO:tasks.workunit.client.0.smithi016.stdout:9/194: fsync d0/d16/d1a/f1f 0 2024-04-02T04:42:34.230 INFO:tasks.workunit.client.0.smithi016.stdout:3/235: fdatasync d2/f26 0 2024-04-02T04:42:34.231 INFO:tasks.workunit.client.0.smithi016.stdout:8/201: rename d1/d6/f20 to d1/d6/d36/f3d 0 2024-04-02T04:42:34.232 INFO:tasks.workunit.client.0.smithi016.stdout:6/179: dwrite d0/d9/df/f1b [0,4194304] 0 2024-04-02T04:42:34.238 INFO:tasks.workunit.client.0.smithi016.stdout:8/202: symlink d1/d6/d36/l3e 0 2024-04-02T04:42:34.238 INFO:tasks.workunit.client.0.smithi016.stdout:4/176: dwrite d5/d16/f20 [0,4194304] 0 2024-04-02T04:42:34.241 INFO:tasks.workunit.client.0.smithi016.stdout:7/186: mkdir d0/d18/d15/d37/d3a 0 2024-04-02T04:42:34.242 INFO:tasks.workunit.client.0.smithi016.stdout:7/187: chown d0/d28 45829 1 2024-04-02T04:42:34.244 INFO:tasks.workunit.client.0.smithi016.stdout:7/188: fsync d0/d18/d15/f1d 0 2024-04-02T04:42:34.244 INFO:tasks.workunit.client.0.smithi016.stdout:6/180: dwrite d0/d8/f2c [0,4194304] 0 2024-04-02T04:42:34.245 INFO:tasks.workunit.client.0.smithi016.stdout:7/189: write d0/d18/d15/f1d [4617206,24908] 0 2024-04-02T04:42:34.249 INFO:tasks.workunit.client.0.smithi016.stdout:3/236: rmdir d2/d15/d16/d35/d43 0 2024-04-02T04:42:34.250 INFO:tasks.workunit.client.0.smithi016.stdout:3/237: chown d2/d15/d16 446576326 1 2024-04-02T04:42:34.251 INFO:tasks.workunit.client.0.smithi016.stdout:4/177: link f4 d5/d17/f35 0 2024-04-02T04:42:34.251 INFO:tasks.workunit.client.0.smithi016.stdout:4/178: stat d5/d8/fb 0 2024-04-02T04:42:34.253 INFO:tasks.workunit.client.0.smithi016.stdout:7/190: rename d0/d18/d15/f24 to d0/d18/d15/d37/f3b 0 2024-04-02T04:42:34.254 INFO:tasks.workunit.client.0.smithi016.stdout:1/203: getdents d14/d19 0 2024-04-02T04:42:34.254 INFO:tasks.workunit.client.0.smithi016.stdout:7/191: chown d0/d18/f11 2476 1 2024-04-02T04:42:34.255 INFO:tasks.workunit.client.0.smithi016.stdout:7/192: dread - d0/d28/f39 zero size 2024-04-02T04:42:34.255 INFO:tasks.workunit.client.0.smithi016.stdout:1/204: write d14/d19/d1e/f2b [2373705,68223] 0 2024-04-02T04:42:34.256 INFO:tasks.workunit.client.0.smithi016.stdout:6/181: creat d0/d9/f33 x:0 0 0 2024-04-02T04:42:34.257 INFO:tasks.workunit.client.0.smithi016.stdout:7/193: chown d0/d18/d15/d16/d31 992586 1 2024-04-02T04:42:34.258 INFO:tasks.workunit.client.0.smithi016.stdout:1/205: mknod d14/d19/d1e/c3e 0 2024-04-02T04:42:34.259 INFO:tasks.workunit.client.0.smithi016.stdout:3/238: creat d2/d15/f4a x:0 0 0 2024-04-02T04:42:34.260 INFO:tasks.workunit.client.0.smithi016.stdout:6/182: rename d0 to d0/d9/d16/d34 22 2024-04-02T04:42:34.260 INFO:tasks.workunit.client.0.smithi016.stdout:6/183: write d0/f7 [2069317,93229] 0 2024-04-02T04:42:34.261 INFO:tasks.workunit.client.0.smithi016.stdout:7/194: creat d0/d21/f3c x:0 0 0 2024-04-02T04:42:34.262 INFO:tasks.workunit.client.0.smithi016.stdout:1/206: rmdir d14/d19 39 2024-04-02T04:42:34.263 INFO:tasks.workunit.client.0.smithi016.stdout:1/207: write f2 [2796403,44082] 0 2024-04-02T04:42:34.264 INFO:tasks.workunit.client.0.smithi016.stdout:1/208: write d14/f28 [1114392,80713] 0 2024-04-02T04:42:34.269 INFO:tasks.workunit.client.0.smithi016.stdout:7/195: creat d0/d18/d15/d16/f3d x:0 0 0 2024-04-02T04:42:34.270 INFO:tasks.workunit.client.0.smithi016.stdout:7/196: chown d0/d21 140 1 2024-04-02T04:42:34.273 INFO:tasks.workunit.client.0.smithi016.stdout:1/209: dwrite d14/f34 [0,4194304] 0 2024-04-02T04:42:34.275 INFO:tasks.workunit.client.0.smithi016.stdout:5/214: sync 2024-04-02T04:42:34.275 INFO:tasks.workunit.client.0.smithi016.stdout:6/184: unlink d0/d9/f2b 0 2024-04-02T04:42:34.276 INFO:tasks.workunit.client.0.smithi016.stdout:3/239: creat d2/d15/f4b x:0 0 0 2024-04-02T04:42:34.279 INFO:tasks.workunit.client.0.smithi016.stdout:9/195: truncate d0/d9/f14 2785396 0 2024-04-02T04:42:34.281 INFO:tasks.workunit.client.0.smithi016.stdout:7/197: rename d0/d18/d15/d16/d31/f35 to d0/d18/f3e 0 2024-04-02T04:42:34.282 INFO:tasks.workunit.client.0.smithi016.stdout:1/210: write fb [1103187,56644] 0 2024-04-02T04:42:34.283 INFO:tasks.workunit.client.0.smithi016.stdout:1/211: write f2 [2744036,120955] 0 2024-04-02T04:42:34.283 INFO:tasks.workunit.client.0.smithi016.stdout:1/212: stat d14/f34 0 2024-04-02T04:42:34.286 INFO:tasks.workunit.client.0.smithi016.stdout:5/215: read d3/f5 [2535375,28111] 0 2024-04-02T04:42:34.293 INFO:tasks.workunit.client.0.smithi016.stdout:9/196: creat d0/d16/f3a x:0 0 0 2024-04-02T04:42:34.293 INFO:tasks.workunit.client.0.smithi016.stdout:7/198: symlink d0/d28/l3f 0 2024-04-02T04:42:34.295 INFO:tasks.workunit.client.0.smithi016.stdout:9/197: write d0/f2d [1532097,78229] 0 2024-04-02T04:42:34.295 INFO:tasks.workunit.client.0.smithi016.stdout:4/179: dwrite d5/d1d/d2d/f29 [0,4194304] 0 2024-04-02T04:42:34.297 INFO:tasks.workunit.client.0.smithi016.stdout:9/198: stat d0/d16/l28 0 2024-04-02T04:42:34.297 INFO:tasks.workunit.client.0.smithi016.stdout:5/216: stat d3/d25/l4d 0 2024-04-02T04:42:34.298 INFO:tasks.workunit.client.0.smithi016.stdout:4/180: dread - d5/d17/d18/f2b zero size 2024-04-02T04:42:34.300 INFO:tasks.workunit.client.0.smithi016.stdout:3/240: mknod d2/c4c 0 2024-04-02T04:42:34.300 INFO:tasks.workunit.client.0.smithi016.stdout:3/241: truncate d2/d15/d16/f25 235149 0 2024-04-02T04:42:34.302 INFO:tasks.workunit.client.0.smithi016.stdout:3/242: dread d2/d2e/f30 [0,4194304] 0 2024-04-02T04:42:34.307 INFO:tasks.workunit.client.0.smithi016.stdout:1/213: fsync d14/d19/d20/f24 0 2024-04-02T04:42:34.307 INFO:tasks.workunit.client.0.smithi016.stdout:1/214: stat d14/d19/d1d/f1f 0 2024-04-02T04:42:34.311 INFO:tasks.workunit.client.0.smithi016.stdout:9/199: symlink d0/d33/l3b 0 2024-04-02T04:42:34.312 INFO:tasks.workunit.client.0.smithi016.stdout:5/217: mknod d3/d25/c55 0 2024-04-02T04:42:34.312 INFO:tasks.workunit.client.0.smithi016.stdout:5/218: chown d3/df/d18/d26/f4b 212482462 1 2024-04-02T04:42:34.318 INFO:tasks.workunit.client.0.smithi016.stdout:3/243: write d2/f3 [6714214,35143] 0 2024-04-02T04:42:34.319 INFO:tasks.workunit.client.0.smithi016.stdout:7/199: symlink d0/d18/d15/d37/d3a/l40 0 2024-04-02T04:42:34.320 INFO:tasks.workunit.client.0.smithi016.stdout:1/215: creat d14/d19/d20/f3f x:0 0 0 2024-04-02T04:42:34.320 INFO:tasks.workunit.client.0.smithi016.stdout:7/200: fsync d0/d18/d15/f2c 0 2024-04-02T04:42:34.321 INFO:tasks.workunit.client.0.smithi016.stdout:7/201: fsync d0/d18/d15/d16/f3d 0 2024-04-02T04:42:34.322 INFO:tasks.workunit.client.0.smithi016.stdout:9/200: mknod d0/d16/c3c 0 2024-04-02T04:42:34.322 INFO:tasks.workunit.client.0.smithi016.stdout:1/216: read d14/d19/d1d/f1f [3107893,88841] 0 2024-04-02T04:42:34.323 INFO:tasks.workunit.client.0.smithi016.stdout:4/181: creat d5/d1d/d1e/f36 x:0 0 0 2024-04-02T04:42:34.330 INFO:tasks.workunit.client.0.smithi016.stdout:7/202: creat d0/d18/d15/d16/f41 x:0 0 0 2024-04-02T04:42:34.330 INFO:tasks.workunit.client.0.smithi016.stdout:5/219: getdents d3/d25/d2a 0 2024-04-02T04:42:34.331 INFO:tasks.workunit.client.0.smithi016.stdout:6/185: truncate d0/d9/df/f1b 2325902 0 2024-04-02T04:42:34.332 INFO:tasks.workunit.client.0.smithi016.stdout:9/201: creat d0/d9/f3d x:0 0 0 2024-04-02T04:42:34.334 INFO:tasks.workunit.client.0.smithi016.stdout:1/217: write d14/d19/d1d/d29/f3c [378453,113179] 0 2024-04-02T04:42:34.335 INFO:tasks.workunit.client.0.smithi016.stdout:6/186: dread d0/d9/df/f10 [0,4194304] 0 2024-04-02T04:42:34.335 INFO:tasks.workunit.client.0.smithi016.stdout:1/218: chown d14/d19/d1e 19954037 1 2024-04-02T04:42:34.336 INFO:tasks.workunit.client.0.smithi016.stdout:4/182: dread d5/f7 [0,4194304] 0 2024-04-02T04:42:34.341 INFO:tasks.workunit.client.0.smithi016.stdout:7/203: creat d0/d28/f42 x:0 0 0 2024-04-02T04:42:34.343 INFO:tasks.workunit.client.0.smithi016.stdout:1/219: dwrite d14/f1c [0,4194304] 0 2024-04-02T04:42:34.344 INFO:tasks.workunit.client.0.smithi016.stdout:6/187: dwrite d0/d9/d16/f29 [0,4194304] 0 2024-04-02T04:42:34.345 INFO:tasks.workunit.client.0.smithi016.stdout:3/244: link d2/d15/d16/l49 d2/d15/d16/d3d/l4d 0 2024-04-02T04:42:34.351 INFO:tasks.workunit.client.0.smithi016.stdout:5/220: symlink d3/df/l56 0 2024-04-02T04:42:34.357 INFO:tasks.workunit.client.0.smithi016.stdout:0/171: sync 2024-04-02T04:42:34.357 INFO:tasks.workunit.client.0.smithi016.stdout:2/174: sync 2024-04-02T04:42:34.358 INFO:tasks.workunit.client.0.smithi016.stdout:9/202: unlink d0/d16/c1e 0 2024-04-02T04:42:34.358 INFO:tasks.workunit.client.0.smithi016.stdout:2/175: write d12/d35/d3f/f2d [787892,45092] 0 2024-04-02T04:42:34.359 INFO:tasks.workunit.client.0.smithi016.stdout:6/188: mkdir d0/d1f/d35 0 2024-04-02T04:42:34.361 INFO:tasks.workunit.client.0.smithi016.stdout:1/220: mknod d14/d19/d1e/c40 0 2024-04-02T04:42:34.365 INFO:tasks.workunit.client.0.smithi016.stdout:7/204: rename d0/d18/d15/d16/d31 to d0/d18/d15/d37/d3a/d43 0 2024-04-02T04:42:34.368 INFO:tasks.workunit.client.0.smithi016.stdout:3/245: truncate d2/d15/f31 1410330 0 2024-04-02T04:42:34.369 INFO:tasks.workunit.client.0.smithi016.stdout:3/246: dread - d2/f3e zero size 2024-04-02T04:42:34.374 INFO:tasks.workunit.client.0.smithi016.stdout:9/203: unlink d0/d16/f1b 0 2024-04-02T04:42:34.374 INFO:tasks.workunit.client.0.smithi016.stdout:1/221: dread f2 [0,4194304] 0 2024-04-02T04:42:34.378 INFO:tasks.workunit.client.0.smithi016.stdout:8/203: sync 2024-04-02T04:42:34.379 INFO:tasks.workunit.client.0.smithi016.stdout:4/183: creat d5/d1d/f37 x:0 0 0 2024-04-02T04:42:34.380 INFO:tasks.workunit.client.0.smithi016.stdout:4/184: write d5/d8/ff [416615,123105] 0 2024-04-02T04:42:34.380 INFO:tasks.workunit.client.0.smithi016.stdout:7/205: chown d0/d18/d15/cf 10763 1 2024-04-02T04:42:34.381 INFO:tasks.workunit.client.0.smithi016.stdout:7/206: dread - d0/d18/d15/d16/f3d zero size 2024-04-02T04:42:34.383 INFO:tasks.workunit.client.0.smithi016.stdout:3/247: creat d2/d15/d16/d3d/f4e x:0 0 0 2024-04-02T04:42:34.389 INFO:tasks.workunit.client.0.smithi016.stdout:5/221: dwrite d3/f36 [0,4194304] 0 2024-04-02T04:42:34.390 INFO:tasks.workunit.client.0.smithi016.stdout:7/207: dwrite d0/d18/d15/f2f [0,4194304] 0 2024-04-02T04:42:34.394 INFO:tasks.workunit.client.0.smithi016.stdout:8/204: mkdir d1/d6/d36/d3f 0 2024-04-02T04:42:34.397 INFO:tasks.workunit.client.0.smithi016.stdout:0/172: write d0/d16/f17 [1781257,128935] 0 2024-04-02T04:42:34.397 INFO:tasks.workunit.client.0.smithi016.stdout:5/222: dwrite d3/df/f22 [0,4194304] 0 2024-04-02T04:42:34.398 INFO:tasks.workunit.client.0.smithi016.stdout:0/173: readlink d0/l6 0 2024-04-02T04:42:34.399 INFO:tasks.workunit.client.0.smithi016.stdout:5/223: stat d3/df/d18/d26/d48/f49 0 2024-04-02T04:42:34.400 INFO:tasks.workunit.client.0.smithi016.stdout:0/174: write d0/d16/d24/f1e [457550,18171] 0 2024-04-02T04:42:34.402 INFO:tasks.workunit.client.0.smithi016.stdout:2/176: truncate d12/f13 1778716 0 2024-04-02T04:42:34.403 INFO:tasks.workunit.client.0.smithi016.stdout:0/175: write d0/f38 [511776,7756] 0 2024-04-02T04:42:34.403 INFO:tasks.workunit.client.0.smithi016.stdout:0/176: chown d0/d29 50 1 2024-04-02T04:42:34.404 INFO:tasks.workunit.client.0.smithi016.stdout:6/189: creat d0/d9/f36 x:0 0 0 2024-04-02T04:42:34.406 INFO:tasks.workunit.client.0.smithi016.stdout:1/222: mkdir d14/d41 0 2024-04-02T04:42:34.408 INFO:tasks.workunit.client.0.smithi016.stdout:0/177: dread d0/d16/f34 [0,4194304] 0 2024-04-02T04:42:34.410 INFO:tasks.workunit.client.0.smithi016.stdout:7/208: mknod d0/d18/d15/d37/d3a/c44 0 2024-04-02T04:42:34.411 INFO:tasks.workunit.client.0.smithi016.stdout:4/185: truncate f4 4110307 0 2024-04-02T04:42:34.419 INFO:tasks.workunit.client.0.smithi016.stdout:3/248: truncate d2/d15/d16/f2c 432306 0 2024-04-02T04:42:34.420 INFO:tasks.workunit.client.0.smithi016.stdout:3/249: chown d2/d15/d16/l42 1117282 1 2024-04-02T04:42:34.422 INFO:tasks.workunit.client.0.smithi016.stdout:1/223: creat d14/d19/d1e/f42 x:0 0 0 2024-04-02T04:42:34.423 INFO:tasks.workunit.client.0.smithi016.stdout:9/204: link d0/d16/d1a/l26 d0/d16/d1a/l3e 0 2024-04-02T04:42:34.428 INFO:tasks.workunit.client.0.smithi016.stdout:9/205: dwrite d0/d3/f30 [0,4194304] 0 2024-04-02T04:42:34.429 INFO:tasks.workunit.client.0.smithi016.stdout:9/206: dread - d0/d33/f35 zero size 2024-04-02T04:42:34.436 INFO:tasks.workunit.client.0.smithi016.stdout:6/190: write d0/d9/df/f10 [5085092,6053] 0 2024-04-02T04:42:34.436 INFO:tasks.workunit.client.0.smithi016.stdout:9/207: dwrite d0/d9/f19 [0,4194304] 0 2024-04-02T04:42:34.441 INFO:tasks.workunit.client.0.smithi016.stdout:9/208: dread d0/f2d [0,4194304] 0 2024-04-02T04:42:34.445 INFO:tasks.workunit.client.0.smithi016.stdout:7/209: truncate d0/d18/d15/d16/f34 966298 0 2024-04-02T04:42:34.448 INFO:tasks.workunit.client.0.smithi016.stdout:2/177: truncate d12/d35/d3f/f2d 839712 0 2024-04-02T04:42:34.451 INFO:tasks.workunit.client.0.smithi016.stdout:5/224: creat d3/df/f57 x:0 0 0 2024-04-02T04:42:34.454 INFO:tasks.workunit.client.0.smithi016.stdout:3/250: mknod d2/d15/d16/d3d/d46/c4f 0 2024-04-02T04:42:34.455 INFO:tasks.workunit.client.0.smithi016.stdout:6/191: stat d0/c30 0 2024-04-02T04:42:34.458 INFO:tasks.workunit.client.0.smithi016.stdout:1/224: mkdir d14/d41/d43 0 2024-04-02T04:42:34.459 INFO:tasks.workunit.client.0.smithi016.stdout:3/251: dwrite d2/d15/d1a/f39 [0,4194304] 0 2024-04-02T04:42:34.460 INFO:tasks.workunit.client.0.smithi016.stdout:3/252: rename d2/d15 to d2/d15/d16/d3d/d50 22 2024-04-02T04:42:34.461 INFO:tasks.workunit.client.0.smithi016.stdout:3/253: write d2/f26 [2151168,10847] 0 2024-04-02T04:42:34.465 INFO:tasks.workunit.client.0.smithi016.stdout:8/205: truncate d1/f3 1314835 0 2024-04-02T04:42:34.467 INFO:tasks.workunit.client.0.smithi016.stdout:8/206: fdatasync d1/d24/f2a 0 2024-04-02T04:42:34.468 INFO:tasks.workunit.client.0.smithi016.stdout:3/254: dwrite d2/f26 [0,4194304] 0 2024-04-02T04:42:34.472 INFO:tasks.workunit.client.0.smithi016.stdout:0/178: rmdir d0/d16/d24/d35 0 2024-04-02T04:42:34.479 INFO:tasks.workunit.client.0.smithi016.stdout:2/178: mkdir d12/d35/d3b/d42 0 2024-04-02T04:42:34.480 INFO:tasks.workunit.client.0.smithi016.stdout:2/179: fdatasync d12/d35/d3f/d2b/f3d 0 2024-04-02T04:42:34.481 INFO:tasks.workunit.client.0.smithi016.stdout:4/186: link d5/de/c31 d5/d17/c38 0 2024-04-02T04:42:34.484 INFO:tasks.workunit.client.0.smithi016.stdout:2/180: dread f8 [0,4194304] 0 2024-04-02T04:42:34.485 INFO:tasks.workunit.client.0.smithi016.stdout:4/187: dread d5/d16/f20 [0,4194304] 0 2024-04-02T04:42:34.485 INFO:tasks.workunit.client.0.smithi016.stdout:4/188: chown d5/d17/d18/f2b 169 1 2024-04-02T04:42:34.490 INFO:tasks.workunit.client.0.smithi016.stdout:4/189: dwrite d5/d8/fb [0,4194304] 0 2024-04-02T04:42:34.491 INFO:tasks.workunit.client.0.smithi016.stdout:1/225: truncate d14/d19/d1e/f23 4232283 0 2024-04-02T04:42:34.497 INFO:tasks.workunit.client.0.smithi016.stdout:9/209: fdatasync d0/d9/f14 0 2024-04-02T04:42:34.498 INFO:tasks.workunit.client.0.smithi016.stdout:9/210: write d0/f27 [1599576,79756] 0 2024-04-02T04:42:34.499 INFO:tasks.workunit.client.0.smithi016.stdout:9/211: dread d0/f7 [0,4194304] 0 2024-04-02T04:42:34.500 INFO:tasks.workunit.client.0.smithi016.stdout:9/212: read d0/d3/f30 [1519224,113725] 0 2024-04-02T04:42:34.501 INFO:tasks.workunit.client.0.smithi016.stdout:9/213: write d0/f2c [585640,47136] 0 2024-04-02T04:42:34.503 INFO:tasks.workunit.client.0.smithi016.stdout:7/210: mknod d0/c45 0 2024-04-02T04:42:34.504 INFO:tasks.workunit.client.0.smithi016.stdout:7/211: dread - d0/d18/d15/d16/f3d zero size 2024-04-02T04:42:34.507 INFO:tasks.workunit.client.0.smithi016.stdout:5/225: symlink d3/d4c/l58 0 2024-04-02T04:42:34.527 INFO:tasks.workunit.client.0.smithi016.stdout:6/192: write d0/d1f/f26 [2523738,119209] 0 2024-04-02T04:42:34.533 INFO:tasks.workunit.client.0.smithi016.stdout:6/193: dwrite d0/d8/f25 [0,4194304] 0 2024-04-02T04:42:34.544 INFO:tasks.workunit.client.0.smithi016.stdout:6/194: dread d0/d9/d16/f2a [0,4194304] 0 2024-04-02T04:42:34.548 INFO:tasks.workunit.client.0.smithi016.stdout:4/190: rmdir d5/d16/d32 0 2024-04-02T04:42:34.551 INFO:tasks.workunit.client.0.smithi016.stdout:8/207: getdents d1 0 2024-04-02T04:42:34.554 INFO:tasks.workunit.client.0.smithi016.stdout:7/212: rename d0/d18/d15 to d0/d46 0 2024-04-02T04:42:34.555 INFO:tasks.workunit.client.0.smithi016.stdout:8/208: dwrite d1/f15 [0,4194304] 0 2024-04-02T04:42:34.556 INFO:tasks.workunit.client.0.smithi016.stdout:5/226: link d3/df/d18/f38 d3/f59 0 2024-04-02T04:42:34.557 INFO:tasks.workunit.client.0.smithi016.stdout:5/227: chown d3/df/d18/d2f/f3f 1870 1 2024-04-02T04:42:34.558 INFO:tasks.workunit.client.0.smithi016.stdout:5/228: stat d3/df/d18/d26/f54 0 2024-04-02T04:42:34.562 INFO:tasks.workunit.client.0.smithi016.stdout:1/226: write d14/d19/d1e/f23 [3473014,72228] 0 2024-04-02T04:42:34.563 INFO:tasks.workunit.client.0.smithi016.stdout:4/191: creat d5/d17/d18/d1f/f39 x:0 0 0 2024-04-02T04:42:34.568 INFO:tasks.workunit.client.0.smithi016.stdout:9/214: truncate d0/d9/f2e 410257 0 2024-04-02T04:42:34.570 INFO:tasks.workunit.client.0.smithi016.stdout:7/213: mkdir d0/d46/d37/d47 0 2024-04-02T04:42:34.572 INFO:tasks.workunit.client.0.smithi016.stdout:8/209: unlink d1/d6/l1f 0 2024-04-02T04:42:34.576 INFO:tasks.workunit.client.0.smithi016.stdout:5/229: dread d3/f1a [0,4194304] 0 2024-04-02T04:42:34.577 INFO:tasks.workunit.client.0.smithi016.stdout:5/230: chown d3/df/d18/d26/f54 0 1 2024-04-02T04:42:34.578 INFO:tasks.workunit.client.0.smithi016.stdout:1/227: rename d14/c1a to d14/d41/d43/c44 0 2024-04-02T04:42:34.579 INFO:tasks.workunit.client.0.smithi016.stdout:4/192: symlink d5/d8/l3a 0 2024-04-02T04:42:34.582 INFO:tasks.workunit.client.0.smithi016.stdout:5/231: dwrite d3/fe [0,4194304] 0 2024-04-02T04:42:34.584 INFO:tasks.workunit.client.0.smithi016.stdout:6/195: write d0/d1f/f2d [2230715,42211] 0 2024-04-02T04:42:34.584 INFO:tasks.workunit.client.0.smithi016.stdout:4/193: dread d5/d8/ff [0,4194304] 0 2024-04-02T04:42:34.585 INFO:tasks.workunit.client.0.smithi016.stdout:8/210: unlink d1/d24/f2c 0 2024-04-02T04:42:34.586 INFO:tasks.workunit.client.0.smithi016.stdout:4/194: stat d5/d17/d18/f2b 0 2024-04-02T04:42:34.587 INFO:tasks.workunit.client.0.smithi016.stdout:8/211: dread d1/d24/f33 [0,4194304] 0 2024-04-02T04:42:34.588 INFO:tasks.workunit.client.0.smithi016.stdout:8/212: write d1/ff [7077303,56463] 0 2024-04-02T04:42:34.589 INFO:tasks.workunit.client.0.smithi016.stdout:8/213: write d1/f19 [1396427,17765] 0 2024-04-02T04:42:34.592 INFO:tasks.workunit.client.0.smithi016.stdout:1/228: fdatasync f7 0 2024-04-02T04:42:34.592 INFO:tasks.workunit.client.0.smithi016.stdout:1/229: chown c5 2854855 1 2024-04-02T04:42:34.593 INFO:tasks.workunit.client.0.smithi016.stdout:1/230: chown f2 85844 1 2024-04-02T04:42:34.599 INFO:tasks.workunit.client.0.smithi016.stdout:7/214: mknod d0/c48 0 2024-04-02T04:42:34.606 INFO:tasks.workunit.client.0.smithi016.stdout:6/196: unlink d0/d8/c1a 0 2024-04-02T04:42:34.618 INFO:tasks.workunit.client.0.smithi016.stdout:9/215: truncate d0/f27 1610617 0 2024-04-02T04:42:34.618 INFO:tasks.workunit.client.0.smithi016.stdout:9/216: chown d0/d3/f30 0 1 2024-04-02T04:42:34.624 INFO:tasks.workunit.client.0.smithi016.stdout:7/215: rename d0/d18 to d0/d46/d16/d49 0 2024-04-02T04:42:34.629 INFO:tasks.workunit.client.0.smithi016.stdout:4/195: creat d5/d17/d18/d25/f3b x:0 0 0 2024-04-02T04:42:34.630 INFO:tasks.workunit.client.0.smithi016.stdout:4/196: fsync d5/d1d/f23 0 2024-04-02T04:42:34.633 INFO:tasks.workunit.client.0.smithi016.stdout:8/214: mkdir d1/d6/d36/d3f/d40 0 2024-04-02T04:42:34.635 INFO:tasks.workunit.client.0.smithi016.stdout:2/181: sync 2024-04-02T04:42:34.636 INFO:tasks.workunit.client.0.smithi016.stdout:0/179: sync 2024-04-02T04:42:34.636 INFO:tasks.workunit.client.0.smithi016.stdout:3/255: sync 2024-04-02T04:42:34.636 INFO:tasks.workunit.client.0.smithi016.stdout:1/231: mkdir d14/d19/d1d/d45 0 2024-04-02T04:42:34.639 INFO:tasks.workunit.client.0.smithi016.stdout:1/232: dread d14/d19/d1d/f30 [0,4194304] 0 2024-04-02T04:42:34.641 INFO:tasks.workunit.client.0.smithi016.stdout:5/232: creat d3/df/f5a x:0 0 0 2024-04-02T04:42:34.642 INFO:tasks.workunit.client.0.smithi016.stdout:5/233: write d3/f3b [957728,56058] 0 2024-04-02T04:42:34.642 INFO:tasks.workunit.client.0.smithi016.stdout:5/234: fsync d3/f28 0 2024-04-02T04:42:34.643 INFO:tasks.workunit.client.0.smithi016.stdout:5/235: write d3/df/f22 [4915200,43234] 0 2024-04-02T04:42:34.644 INFO:tasks.workunit.client.0.smithi016.stdout:5/236: write d3/df/d18/d26/f4b [248188,92559] 0 2024-04-02T04:42:34.645 INFO:tasks.workunit.client.0.smithi016.stdout:4/197: creat d5/d1d/d1e/f3c x:0 0 0 2024-04-02T04:42:34.646 INFO:tasks.workunit.client.0.smithi016.stdout:5/237: write d3/df/d18/d26/f54 [5729,74395] 0 2024-04-02T04:42:34.646 INFO:tasks.workunit.client.0.smithi016.stdout:4/198: read - d5/d17/d18/d25/f3b zero size 2024-04-02T04:42:34.648 INFO:tasks.workunit.client.0.smithi016.stdout:0/180: unlink d0/f4 0 2024-04-02T04:42:34.649 INFO:tasks.workunit.client.0.smithi016.stdout:3/256: symlink d2/d15/d1a/l51 0 2024-04-02T04:42:34.650 INFO:tasks.workunit.client.0.smithi016.stdout:3/257: chown d2/d15/f17 459996 1 2024-04-02T04:42:34.652 INFO:tasks.workunit.client.0.smithi016.stdout:9/217: symlink d0/l3f 0 2024-04-02T04:42:34.652 INFO:tasks.workunit.client.0.smithi016.stdout:9/218: dread - d0/d9/f3d zero size 2024-04-02T04:42:34.653 INFO:tasks.workunit.client.0.smithi016.stdout:7/216: symlink d0/l4a 0 2024-04-02T04:42:34.655 INFO:tasks.workunit.client.0.smithi016.stdout:1/233: unlink d14/d19/d20/f26 0 2024-04-02T04:42:34.658 INFO:tasks.workunit.client.0.smithi016.stdout:6/197: creat d0/f37 x:0 0 0 2024-04-02T04:42:34.661 INFO:tasks.workunit.client.0.smithi016.stdout:9/219: dread d0/f11 [0,4194304] 0 2024-04-02T04:42:34.662 INFO:tasks.workunit.client.0.smithi016.stdout:6/198: dread d0/d9/df/f10 [0,4194304] 0 2024-04-02T04:42:34.662 INFO:tasks.workunit.client.0.smithi016.stdout:6/199: fsync d0/d9/f32 0 2024-04-02T04:42:34.663 INFO:tasks.workunit.client.0.smithi016.stdout:0/181: dread d0/d16/f17 [0,4194304] 0 2024-04-02T04:42:34.664 INFO:tasks.workunit.client.0.smithi016.stdout:8/215: link d1/d24/f33 d1/d24/d38/f41 0 2024-04-02T04:42:34.665 INFO:tasks.workunit.client.0.smithi016.stdout:8/216: write d1/f15 [2583240,93772] 0 2024-04-02T04:42:34.667 INFO:tasks.workunit.client.0.smithi016.stdout:4/199: unlink d5/d16/f1b 0 2024-04-02T04:42:34.667 INFO:tasks.workunit.client.0.smithi016.stdout:0/182: dread d0/f10 [0,4194304] 0 2024-04-02T04:42:34.677 INFO:tasks.workunit.client.0.smithi016.stdout:3/258: rmdir d2/d15/d16/d35 39 2024-04-02T04:42:34.688 INFO:tasks.workunit.client.0.smithi016.stdout:7/217: dwrite d0/d46/d16/d49/f3e [0,4194304] 0 2024-04-02T04:42:34.697 INFO:tasks.workunit.client.0.smithi016.stdout:6/200: rename d0/d8 to d0/d9/df/d38 0 2024-04-02T04:42:34.702 INFO:tasks.workunit.client.0.smithi016.stdout:6/201: dwrite d0/f13 [0,4194304] 0 2024-04-02T04:42:34.705 INFO:tasks.workunit.client.0.smithi016.stdout:8/217: symlink d1/d6/d36/l42 0 2024-04-02T04:42:34.707 INFO:tasks.workunit.client.0.smithi016.stdout:4/200: rmdir d5/de 39 2024-04-02T04:42:34.727 INFO:tasks.workunit.client.0.smithi016.stdout:5/238: creat d3/f5b x:0 0 0 2024-04-02T04:42:34.729 INFO:tasks.workunit.client.0.smithi016.stdout:5/239: write d3/d25/d45/f51 [85314,107693] 0 2024-04-02T04:42:34.729 INFO:tasks.workunit.client.0.smithi016.stdout:6/202: symlink d0/d9/d16/l39 0 2024-04-02T04:42:34.730 INFO:tasks.workunit.client.0.smithi016.stdout:5/240: read d3/f59 [5409893,22892] 0 2024-04-02T04:42:34.731 INFO:tasks.workunit.client.0.smithi016.stdout:8/218: creat d1/d24/f43 x:0 0 0 2024-04-02T04:42:34.732 INFO:tasks.workunit.client.0.smithi016.stdout:5/241: truncate d3/df/d18/f31 518618 0 2024-04-02T04:42:34.732 INFO:tasks.workunit.client.0.smithi016.stdout:4/201: mknod d5/d17/c3d 0 2024-04-02T04:42:34.732 INFO:tasks.workunit.client.0.smithi016.stdout:8/219: read d1/d6/f10 [722049,120177] 0 2024-04-02T04:42:34.734 INFO:tasks.workunit.client.0.smithi016.stdout:0/183: creat d0/d16/d2e/d37/f3b x:0 0 0 2024-04-02T04:42:34.735 INFO:tasks.workunit.client.0.smithi016.stdout:2/182: getdents d12/d35/d3f/d2b 0 2024-04-02T04:42:34.737 INFO:tasks.workunit.client.0.smithi016.stdout:3/259: mknod d2/d15/d1a/d45/c52 0 2024-04-02T04:42:34.738 INFO:tasks.workunit.client.0.smithi016.stdout:1/234: write d14/f1c [4577950,81503] 0 2024-04-02T04:42:34.738 INFO:tasks.workunit.client.0.smithi016.stdout:8/220: dwrite d1/f19 [0,4194304] 0 2024-04-02T04:42:34.738 INFO:tasks.workunit.client.0.smithi016.stdout:3/260: chown d2/d15/d1a/d45/c52 3 1 2024-04-02T04:42:34.740 INFO:tasks.workunit.client.0.smithi016.stdout:8/221: write d1/f1d [871704,100519] 0 2024-04-02T04:42:34.748 INFO:tasks.workunit.client.0.smithi016.stdout:6/203: creat d0/d1f/f3a x:0 0 0 2024-04-02T04:42:34.758 INFO:tasks.workunit.client.0.smithi016.stdout:4/202: rename d5/d17/d18/d1f/l22 to d5/d17/d18/d1f/l3e 0 2024-04-02T04:42:34.758 INFO:tasks.workunit.client.0.smithi016.stdout:4/203: chown d5/d1d/f23 14314 1 2024-04-02T04:42:34.759 INFO:tasks.workunit.client.0.smithi016.stdout:0/184: rmdir d0/d16/d2e 39 2024-04-02T04:42:34.767 INFO:tasks.workunit.client.0.smithi016.stdout:3/261: creat d2/d15/d1a/f53 x:0 0 0 2024-04-02T04:42:34.768 INFO:tasks.workunit.client.0.smithi016.stdout:1/235: mkdir d14/d19/d1d/d29/d39/d46 0 2024-04-02T04:42:34.770 INFO:tasks.workunit.client.0.smithi016.stdout:8/222: creat d1/d24/f44 x:0 0 0 2024-04-02T04:42:34.773 INFO:tasks.workunit.client.0.smithi016.stdout:3/262: dwrite d2/d15/d1a/f47 [0,4194304] 0 2024-04-02T04:42:34.774 INFO:tasks.workunit.client.0.smithi016.stdout:3/263: fsync d2/f28 0 2024-04-02T04:42:34.778 INFO:tasks.workunit.client.0.smithi016.stdout:5/242: mknod d3/c5c 0 2024-04-02T04:42:34.780 INFO:tasks.workunit.client.0.smithi016.stdout:4/204: mkdir d5/d16/d3f 0 2024-04-02T04:42:34.786 INFO:tasks.workunit.client.0.smithi016.stdout:7/218: getdents d0/d46/d16/d49 0 2024-04-02T04:42:34.792 INFO:tasks.workunit.client.0.smithi016.stdout:9/220: dread d0/f27 [0,4194304] 0 2024-04-02T04:42:34.797 INFO:tasks.workunit.client.0.smithi016.stdout:3/264: write d2/d15/f1b [4967696,125762] 0 2024-04-02T04:42:34.798 INFO:tasks.workunit.client.0.smithi016.stdout:3/265: write d2/f3 [3246479,120771] 0 2024-04-02T04:42:34.799 INFO:tasks.workunit.client.0.smithi016.stdout:2/183: write d12/d14/f1e [546640,72294] 0 2024-04-02T04:42:34.803 INFO:tasks.workunit.client.0.smithi016.stdout:0/185: creat d0/d16/d2e/d37/f3c x:0 0 0 2024-04-02T04:42:34.804 INFO:tasks.workunit.client.0.smithi016.stdout:0/186: chown d0/d16/d23 0 1 2024-04-02T04:42:34.804 INFO:tasks.workunit.client.0.smithi016.stdout:4/205: dread d5/d8/ff [0,4194304] 0 2024-04-02T04:42:34.805 INFO:tasks.workunit.client.0.smithi016.stdout:0/187: dread - d0/d16/d2e/d37/f3c zero size 2024-04-02T04:42:34.805 INFO:tasks.workunit.client.0.smithi016.stdout:2/184: dread d12/d20/f2f [0,4194304] 0 2024-04-02T04:42:34.806 INFO:tasks.workunit.client.0.smithi016.stdout:0/188: readlink d0/d29/l2f 0 2024-04-02T04:42:34.806 INFO:tasks.workunit.client.0.smithi016.stdout:7/219: rename d0/l4a to d0/d46/d16/l4b 0 2024-04-02T04:42:34.806 INFO:tasks.workunit.client.0.smithi016.stdout:2/185: truncate d12/f2a 252468 0 2024-04-02T04:42:34.809 INFO:tasks.workunit.client.0.smithi016.stdout:3/266: dread d2/f3 [0,4194304] 0 2024-04-02T04:42:34.810 INFO:tasks.workunit.client.0.smithi016.stdout:7/220: dread d0/d46/f1a [0,4194304] 0 2024-04-02T04:42:34.810 INFO:tasks.workunit.client.0.smithi016.stdout:3/267: write d2/f14 [2787835,128361] 0 2024-04-02T04:42:34.810 INFO:tasks.workunit.client.0.smithi016.stdout:8/223: mkdir d1/d6/d36/d3f/d40/d45 0 2024-04-02T04:42:34.811 INFO:tasks.workunit.client.0.smithi016.stdout:7/221: write d0/d46/f2f [230225,125779] 0 2024-04-02T04:42:34.811 INFO:tasks.workunit.client.0.smithi016.stdout:8/224: write d1/ff [3734082,79142] 0 2024-04-02T04:42:34.814 INFO:tasks.workunit.client.0.smithi016.stdout:6/204: write d0/f6 [641074,92468] 0 2024-04-02T04:42:34.816 INFO:tasks.workunit.client.0.smithi016.stdout:5/243: mkdir d3/d5d 0 2024-04-02T04:42:34.817 INFO:tasks.workunit.client.0.smithi016.stdout:5/244: write d3/df/f3c [284624,90223] 0 2024-04-02T04:42:34.825 INFO:tasks.workunit.client.0.smithi016.stdout:1/236: dwrite d14/f17 [0,4194304] 0 2024-04-02T04:42:34.826 INFO:tasks.workunit.client.0.smithi016.stdout:1/237: fsync fb 0 2024-04-02T04:42:34.828 INFO:tasks.workunit.client.0.smithi016.stdout:9/221: fdatasync d0/d9/f2e 0 2024-04-02T04:42:34.836 INFO:tasks.workunit.client.0.smithi016.stdout:8/225: rename d1/f1d to d1/d31/f46 0 2024-04-02T04:42:34.836 INFO:tasks.workunit.client.0.smithi016.stdout:7/222: mkdir d0/d46/d37/d3a/d4c 0 2024-04-02T04:42:34.837 INFO:tasks.workunit.client.0.smithi016.stdout:6/205: creat d0/d1f/f3b x:0 0 0 2024-04-02T04:42:34.838 INFO:tasks.workunit.client.0.smithi016.stdout:5/245: symlink d3/df/d18/d26/d48/l5e 0 2024-04-02T04:42:34.840 INFO:tasks.workunit.client.0.smithi016.stdout:4/206: fsync d5/d8/fd 0 2024-04-02T04:42:34.841 INFO:tasks.workunit.client.0.smithi016.stdout:0/189: creat d0/d16/d2e/d36/f3d x:0 0 0 2024-04-02T04:42:34.842 INFO:tasks.workunit.client.0.smithi016.stdout:9/222: dread d0/d16/d1a/f1f [0,4194304] 0 2024-04-02T04:42:34.912 INFO:tasks.workunit.client.0.smithi016.stdout:0/190: write d0/d16/d24/f1e [540373,33265] 0 2024-04-02T04:42:34.912 INFO:tasks.workunit.client.0.smithi016.stdout:1/238: creat d14/d41/f47 x:0 0 0 2024-04-02T04:42:34.912 INFO:tasks.workunit.client.0.smithi016.stdout:7/223: rename d0/d46/d16/f20 to d0/d46/f4d 0 2024-04-02T04:42:34.913 INFO:tasks.workunit.client.0.smithi016.stdout:4/207: creat d5/d17/d18/d1f/f40 x:0 0 0 2024-04-02T04:42:34.914 INFO:tasks.workunit.client.0.smithi016.stdout:2/186: creat d12/f43 x:0 0 0 2024-04-02T04:42:34.914 INFO:tasks.workunit.client.0.smithi016.stdout:9/223: creat d0/d3/f40 x:0 0 0 2024-04-02T04:42:34.915 INFO:tasks.workunit.client.0.smithi016.stdout:4/208: chown d5/d17/d18 553169862 1 2024-04-02T04:42:34.917 INFO:tasks.workunit.client.0.smithi016.stdout:0/191: mkdir d0/d16/d23/d3e 0 2024-04-02T04:42:34.920 INFO:tasks.workunit.client.0.smithi016.stdout:2/187: creat d12/d20/f44 x:0 0 0 2024-04-02T04:42:34.922 INFO:tasks.workunit.client.0.smithi016.stdout:4/209: creat d5/d17/d18/d25/f41 x:0 0 0 2024-04-02T04:42:34.922 INFO:tasks.workunit.client.0.smithi016.stdout:7/224: dwrite d0/d46/f30 [0,4194304] 0 2024-04-02T04:42:34.924 INFO:tasks.workunit.client.0.smithi016.stdout:9/224: rename d0/d16/d1a/l32 to d0/d3/d39/l41 0 2024-04-02T04:42:34.925 INFO:tasks.workunit.client.0.smithi016.stdout:8/226: dread d1/f15 [0,4194304] 0 2024-04-02T04:42:34.926 INFO:tasks.workunit.client.0.smithi016.stdout:6/206: dread d0/d9/d16/f20 [0,4194304] 0 2024-04-02T04:42:34.928 INFO:tasks.workunit.client.0.smithi016.stdout:0/192: creat d0/d16/d23/f3f x:0 0 0 2024-04-02T04:42:34.934 INFO:tasks.workunit.client.0.smithi016.stdout:7/225: creat d0/d21/f4e x:0 0 0 2024-04-02T04:42:34.943 INFO:tasks.workunit.client.0.smithi016.stdout:6/207: creat d0/d9/d16/f3c x:0 0 0 2024-04-02T04:42:34.944 INFO:tasks.workunit.client.0.smithi016.stdout:6/208: write d0/d9/df/d38/f25 [1118083,36090] 0 2024-04-02T04:42:34.944 INFO:tasks.workunit.client.0.smithi016.stdout:6/209: chown d0/f7 37371 1 2024-04-02T04:42:34.945 INFO:tasks.workunit.client.0.smithi016.stdout:0/193: mknod d0/d16/d2e/d37/c40 0 2024-04-02T04:42:34.946 INFO:tasks.workunit.client.0.smithi016.stdout:1/239: getdents d14/d41/d43 0 2024-04-02T04:42:34.946 INFO:tasks.workunit.client.0.smithi016.stdout:3/268: sync 2024-04-02T04:42:34.946 INFO:tasks.workunit.client.0.smithi016.stdout:5/246: truncate d3/f17 1123810 0 2024-04-02T04:42:34.947 INFO:tasks.workunit.client.0.smithi016.stdout:3/269: readlink d2/l33 0 2024-04-02T04:42:34.947 INFO:tasks.workunit.client.0.smithi016.stdout:1/240: write d14/d19/d1e/f23 [3729512,41024] 0 2024-04-02T04:42:34.951 INFO:tasks.workunit.client.0.smithi016.stdout:7/226: mknod d0/d46/d16/d49/c4f 0 2024-04-02T04:42:34.952 INFO:tasks.workunit.client.0.smithi016.stdout:1/241: dwrite fd [0,4194304] 0 2024-04-02T04:42:34.953 INFO:tasks.workunit.client.0.smithi016.stdout:8/227: write d1/d31/f46 [3238404,65083] 0 2024-04-02T04:42:34.954 INFO:tasks.workunit.client.0.smithi016.stdout:1/242: write d14/f1c [448628,56099] 0 2024-04-02T04:42:34.958 INFO:tasks.workunit.client.0.smithi016.stdout:6/210: mknod d0/d9/df/d38/c3d 0 2024-04-02T04:42:34.964 INFO:tasks.workunit.client.0.smithi016.stdout:6/211: dwrite d0/f37 [0,4194304] 0 2024-04-02T04:42:34.972 INFO:tasks.workunit.client.0.smithi016.stdout:9/225: dwrite d0/d9/f14 [0,4194304] 0 2024-04-02T04:42:34.973 INFO:tasks.workunit.client.0.smithi016.stdout:9/226: read d0/d9/f19 [3001367,7289] 0 2024-04-02T04:42:34.973 INFO:tasks.workunit.client.0.smithi016.stdout:4/210: rename d5/d17/d18/f2e to d5/d1d/f42 0 2024-04-02T04:42:34.974 INFO:tasks.workunit.client.0.smithi016.stdout:4/211: chown d5/d17/d18/d25 27 1 2024-04-02T04:42:34.975 INFO:tasks.workunit.client.0.smithi016.stdout:2/188: write d12/f13 [1353007,37190] 0 2024-04-02T04:42:34.976 INFO:tasks.workunit.client.0.smithi016.stdout:4/212: read - d5/d1d/d1e/f3c zero size 2024-04-02T04:42:34.978 INFO:tasks.workunit.client.0.smithi016.stdout:2/189: stat d12/f2a 0 2024-04-02T04:42:34.981 INFO:tasks.workunit.client.0.smithi016.stdout:5/247: creat d3/d25/d2a/f5f x:0 0 0 2024-04-02T04:42:34.982 INFO:tasks.workunit.client.0.smithi016.stdout:6/212: mkdir d0/d1f/d3e 0 2024-04-02T04:42:34.984 INFO:tasks.workunit.client.0.smithi016.stdout:0/194: dwrite d0/d16/d24/f2b [0,4194304] 0 2024-04-02T04:42:34.986 INFO:tasks.workunit.client.0.smithi016.stdout:7/227: mknod d0/d46/d37/d47/c50 0 2024-04-02T04:42:34.990 INFO:tasks.workunit.client.0.smithi016.stdout:6/213: dread d0/d9/df/d38/f25 [0,4194304] 0 2024-04-02T04:42:34.991 INFO:tasks.workunit.client.0.smithi016.stdout:9/227: creat d0/d9/f42 x:0 0 0 2024-04-02T04:42:34.992 INFO:tasks.workunit.client.0.smithi016.stdout:9/228: stat d0/d16/c3c 0 2024-04-02T04:42:34.993 INFO:tasks.workunit.client.0.smithi016.stdout:6/214: dread - d0/d9/f32 zero size 2024-04-02T04:42:34.994 INFO:tasks.workunit.client.0.smithi016.stdout:8/228: truncate d1/d6/f10 1654935 0 2024-04-02T04:42:34.995 INFO:tasks.workunit.client.0.smithi016.stdout:4/213: dread d5/f10 [0,4194304] 0 2024-04-02T04:42:34.996 INFO:tasks.workunit.client.0.smithi016.stdout:0/195: dwrite d0/f38 [0,4194304] 0 2024-04-02T04:42:34.999 INFO:tasks.workunit.client.0.smithi016.stdout:5/248: creat d3/d40/f60 x:0 0 0 2024-04-02T04:42:34.999 INFO:tasks.workunit.client.0.smithi016.stdout:3/270: truncate d2/d15/d1a/f39 1187918 0 2024-04-02T04:42:35.001 INFO:tasks.workunit.client.0.smithi016.stdout:3/271: chown d2/l10 805944 1 2024-04-02T04:42:35.001 INFO:tasks.workunit.client.0.smithi016.stdout:6/215: dread d0/d9/df/d38/f25 [0,4194304] 0 2024-04-02T04:42:35.003 INFO:tasks.workunit.client.0.smithi016.stdout:7/228: symlink d0/d28/l51 0 2024-04-02T04:42:35.003 INFO:tasks.workunit.client.0.smithi016.stdout:3/272: read - d2/d15/f4b zero size 2024-04-02T04:42:35.004 INFO:tasks.workunit.client.0.smithi016.stdout:4/214: dwrite d5/d1d/f37 [0,4194304] 0 2024-04-02T04:42:35.007 INFO:tasks.workunit.client.0.smithi016.stdout:1/243: creat d14/f48 x:0 0 0 2024-04-02T04:42:35.007 INFO:tasks.workunit.client.0.smithi016.stdout:4/215: dread - d5/d17/d18/d25/f3b zero size 2024-04-02T04:42:35.007 INFO:tasks.workunit.client.0.smithi016.stdout:8/229: creat d1/d6/d36/d3f/f47 x:0 0 0 2024-04-02T04:42:35.008 INFO:tasks.workunit.client.0.smithi016.stdout:9/229: read d0/d3/f30 [2474957,1113] 0 2024-04-02T04:42:35.009 INFO:tasks.workunit.client.0.smithi016.stdout:0/196: creat d0/d16/d2e/d36/f41 x:0 0 0 2024-04-02T04:42:35.010 INFO:tasks.workunit.client.0.smithi016.stdout:0/197: stat d0/d16/d23 0 2024-04-02T04:42:35.011 INFO:tasks.workunit.client.0.smithi016.stdout:5/249: creat d3/df/d18/d26/d48/f61 x:0 0 0 2024-04-02T04:42:35.011 INFO:tasks.workunit.client.0.smithi016.stdout:6/216: dwrite d0/f37 [0,4194304] 0 2024-04-02T04:42:35.012 INFO:tasks.workunit.client.0.smithi016.stdout:0/198: readlink d0/d16/d23/l39 0 2024-04-02T04:42:35.012 INFO:tasks.workunit.client.0.smithi016.stdout:6/217: stat d0/d9/d16/f20 0 2024-04-02T04:42:35.017 INFO:tasks.workunit.client.0.smithi016.stdout:7/229: mkdir d0/d46/d52 0 2024-04-02T04:42:35.017 INFO:tasks.workunit.client.0.smithi016.stdout:4/216: mkdir d5/d8/d43 0 2024-04-02T04:42:35.018 INFO:tasks.workunit.client.0.smithi016.stdout:8/230: symlink d1/d6/l48 0 2024-04-02T04:42:35.022 INFO:tasks.workunit.client.0.smithi016.stdout:1/244: creat d14/d41/d43/f49 x:0 0 0 2024-04-02T04:42:35.022 INFO:tasks.workunit.client.0.smithi016.stdout:1/245: write d14/f1c [4023204,78868] 0 2024-04-02T04:42:35.024 INFO:tasks.workunit.client.0.smithi016.stdout:5/250: creat d3/df/d18/d2f/f62 x:0 0 0 2024-04-02T04:42:35.027 INFO:tasks.workunit.client.0.smithi016.stdout:0/199: unlink d0/c2c 0 2024-04-02T04:42:35.032 INFO:tasks.workunit.client.0.smithi016.stdout:7/230: creat d0/d28/f53 x:0 0 0 2024-04-02T04:42:35.037 INFO:tasks.workunit.client.0.smithi016.stdout:1/246: unlink d14/d19/d1d/d29/d39/c3a 0 2024-04-02T04:42:35.039 INFO:tasks.workunit.client.0.smithi016.stdout:5/251: mkdir d3/d25/d63 0 2024-04-02T04:42:35.041 INFO:tasks.workunit.client.0.smithi016.stdout:9/230: dread d0/d16/f20 [0,4194304] 0 2024-04-02T04:42:35.041 INFO:tasks.workunit.client.0.smithi016.stdout:9/231: chown d0/d3/l2a 1320 1 2024-04-02T04:42:35.042 INFO:tasks.workunit.client.0.smithi016.stdout:4/217: fsync d5/d1d/f21 0 2024-04-02T04:42:35.046 INFO:tasks.workunit.client.0.smithi016.stdout:8/231: rename d1/d24 to d1/d6/d36/d3f/d40/d49 0 2024-04-02T04:42:35.047 INFO:tasks.workunit.client.0.smithi016.stdout:6/218: creat d0/f3f x:0 0 0 2024-04-02T04:42:35.048 INFO:tasks.workunit.client.0.smithi016.stdout:5/252: symlink d3/d4c/l64 0 2024-04-02T04:42:35.049 INFO:tasks.workunit.client.0.smithi016.stdout:9/232: dwrite d0/d3/f40 [0,4194304] 0 2024-04-02T04:42:35.050 INFO:tasks.workunit.client.0.smithi016.stdout:5/253: write d3/df/d18/d26/d48/f49 [495357,51009] 0 2024-04-02T04:42:35.051 INFO:tasks.workunit.client.0.smithi016.stdout:3/273: getdents d2/d15/d16/d3d 0 2024-04-02T04:42:35.052 INFO:tasks.workunit.client.0.smithi016.stdout:4/218: chown d5/de/d33 25 1 2024-04-02T04:42:35.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:34 smithi016 ceph-mon[28969]: pgmap v16: 65 pgs: 65 active+clean; 368 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 31 MiB/s wr, 290 op/s 2024-04-02T04:42:35.060 INFO:tasks.workunit.client.0.smithi016.stdout:7/231: rename d0/c45 to d0/d46/d37/d3a/d43/c54 0 2024-04-02T04:42:35.061 INFO:tasks.workunit.client.0.smithi016.stdout:7/232: read - d0/d28/f39 zero size 2024-04-02T04:42:35.064 INFO:tasks.workunit.client.0.smithi016.stdout:5/254: unlink d3/fe 0 2024-04-02T04:42:35.065 INFO:tasks.workunit.client.0.smithi016.stdout:3/274: truncate d2/f3 6645639 0 2024-04-02T04:42:35.072 INFO:tasks.workunit.client.0.smithi016.stdout:1/247: rename d14/d19/d20/l2a to d14/d19/d1d/d29/d39/d46/l4a 0 2024-04-02T04:42:35.075 INFO:tasks.workunit.client.0.smithi016.stdout:7/233: unlink d0/d46/d16/d49/c1f 0 2024-04-02T04:42:35.076 INFO:tasks.workunit.client.0.smithi016.stdout:0/200: link d0/d16/d23/l39 d0/d16/l42 0 2024-04-02T04:42:35.076 INFO:tasks.workunit.client.0.smithi016.stdout:7/234: fsync d0/d46/f2c 0 2024-04-02T04:42:35.078 INFO:tasks.workunit.client.0.smithi016.stdout:4/219: mknod d5/d8/d43/c44 0 2024-04-02T04:42:35.079 INFO:tasks.workunit.client.0.smithi016.stdout:8/232: chown d1/d31/l3c 1150 1 2024-04-02T04:42:35.080 INFO:tasks.workunit.client.0.smithi016.stdout:6/219: creat d0/f40 x:0 0 0 2024-04-02T04:42:35.082 INFO:tasks.workunit.client.0.smithi016.stdout:7/235: rename d0/d46/d37/d47/c50 to d0/d46/d37/d3a/c55 0 2024-04-02T04:42:35.089 INFO:tasks.workunit.client.0.smithi016.stdout:0/201: fsync d0/d16/f18 0 2024-04-02T04:42:35.089 INFO:tasks.workunit.client.0.smithi016.stdout:4/220: dwrite d5/d16/f20 [0,4194304] 0 2024-04-02T04:42:35.091 INFO:tasks.workunit.client.0.smithi016.stdout:4/221: read d5/d1d/f21 [6459880,130028] 0 2024-04-02T04:42:35.093 INFO:tasks.workunit.client.0.smithi016.stdout:7/236: creat d0/d46/d16/f56 x:0 0 0 2024-04-02T04:42:35.096 INFO:tasks.workunit.client.0.smithi016.stdout:5/255: creat d3/df/f65 x:0 0 0 2024-04-02T04:42:35.096 INFO:tasks.workunit.client.0.smithi016.stdout:3/275: dread d2/f3f [0,4194304] 0 2024-04-02T04:42:35.099 INFO:tasks.workunit.client.0.smithi016.stdout:8/233: mkdir d1/d6/d36/d3f/d40/d49/d4a 0 2024-04-02T04:42:35.100 INFO:tasks.workunit.client.0.smithi016.stdout:8/234: fdatasync d1/d6/d36/d3f/d40/d49/f44 0 2024-04-02T04:42:35.106 INFO:tasks.workunit.client.0.smithi016.stdout:7/237: creat d0/d21/f57 x:0 0 0 2024-04-02T04:42:35.107 INFO:tasks.workunit.client.0.smithi016.stdout:7/238: readlink d0/d21/l23 0 2024-04-02T04:42:35.110 INFO:tasks.workunit.client.0.smithi016.stdout:5/256: rename d3/c5c to d3/d25/d63/c66 0 2024-04-02T04:42:35.111 INFO:tasks.workunit.client.0.smithi016.stdout:5/257: fdatasync d3/f6 0 2024-04-02T04:42:35.112 INFO:tasks.workunit.client.0.smithi016.stdout:6/220: creat d0/f41 x:0 0 0 2024-04-02T04:42:35.113 INFO:tasks.workunit.client.0.smithi016.stdout:0/202: creat d0/d16/d23/d3e/f43 x:0 0 0 2024-04-02T04:42:35.114 INFO:tasks.workunit.client.0.smithi016.stdout:0/203: fdatasync d0/f12 0 2024-04-02T04:42:35.119 INFO:tasks.workunit.client.0.smithi016.stdout:0/204: dwrite d0/d16/d23/d3e/f43 [0,4194304] 0 2024-04-02T04:42:35.121 INFO:tasks.workunit.client.0.smithi016.stdout:8/235: rename d1/d6/d36/f3d to d1/d6/f4b 0 2024-04-02T04:42:35.124 INFO:tasks.workunit.client.0.smithi016.stdout:4/222: write d5/f7 [11820,96562] 0 2024-04-02T04:42:35.125 INFO:tasks.workunit.client.0.smithi016.stdout:4/223: fsync d5/d1d/d2d/f29 0 2024-04-02T04:42:35.126 INFO:tasks.workunit.client.0.smithi016.stdout:4/224: read - d5/d17/d18/d1f/f39 zero size 2024-04-02T04:42:35.133 INFO:tasks.workunit.client.0.smithi016.stdout:5/258: mkdir d3/d40/d4f/d67 0 2024-04-02T04:42:35.134 INFO:tasks.workunit.client.0.smithi016.stdout:8/236: read d1/f19 [831374,93872] 0 2024-04-02T04:42:35.135 INFO:tasks.workunit.client.0.smithi016.stdout:0/205: rename d0/d16/d24/f2b to d0/d16/d2e/d36/f44 0 2024-04-02T04:42:35.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:34 smithi027 ceph-mon[37550]: pgmap v16: 65 pgs: 65 active+clean; 368 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 31 MiB/s wr, 290 op/s 2024-04-02T04:42:35.217 INFO:tasks.workunit.client.0.smithi016.stdout:6/221: link d0/d9/df/d38/f2c d0/d1f/f42 0 2024-04-02T04:42:35.219 INFO:tasks.workunit.client.0.smithi016.stdout:8/237: chown d1/ff 0 1 2024-04-02T04:42:35.220 INFO:tasks.workunit.client.0.smithi016.stdout:7/239: getdents d0/d28 0 2024-04-02T04:42:35.222 INFO:tasks.workunit.client.0.smithi016.stdout:5/259: rename d3/df/f3c to d3/df/f68 0 2024-04-02T04:42:35.225 INFO:tasks.workunit.client.0.smithi016.stdout:7/240: chown d0/d28/f53 31 1 2024-04-02T04:42:35.227 INFO:tasks.workunit.client.0.smithi016.stdout:0/206: dwrite d0/d16/d2e/d37/f3c [0,4194304] 0 2024-04-02T04:42:35.227 INFO:tasks.workunit.client.0.smithi016.stdout:3/276: dwrite d2/d15/d1a/f29 [0,4194304] 0 2024-04-02T04:42:35.230 INFO:tasks.workunit.client.0.smithi016.stdout:6/222: dwrite d0/d1f/f26 [0,4194304] 0 2024-04-02T04:42:35.231 INFO:tasks.workunit.client.0.smithi016.stdout:3/277: dread - d2/d2e/f2f zero size 2024-04-02T04:42:35.231 INFO:tasks.workunit.client.0.smithi016.stdout:7/241: mknod d0/d46/c58 0 2024-04-02T04:42:35.232 INFO:tasks.workunit.client.0.smithi016.stdout:8/238: mkdir d1/d6/d36/d3f/d40/d49/d4a/d4c 0 2024-04-02T04:42:35.234 INFO:tasks.workunit.client.0.smithi016.stdout:6/223: write d0/d9/d16/f2a [2733914,110472] 0 2024-04-02T04:42:35.234 INFO:tasks.workunit.client.0.smithi016.stdout:7/242: chown d0/d21/f3c 0 1 2024-04-02T04:42:35.234 INFO:tasks.workunit.client.0.smithi016.stdout:5/260: creat d3/d5d/f69 x:0 0 0 2024-04-02T04:42:35.238 INFO:tasks.workunit.client.0.smithi016.stdout:0/207: unlink d0/d16/f17 0 2024-04-02T04:42:35.239 INFO:tasks.workunit.client.0.smithi016.stdout:6/224: creat d0/d9/df/d38/f43 x:0 0 0 2024-04-02T04:42:35.242 INFO:tasks.workunit.client.0.smithi016.stdout:7/243: creat d0/d28/f59 x:0 0 0 2024-04-02T04:42:35.242 INFO:tasks.workunit.client.0.smithi016.stdout:0/208: rename d0/d29/c15 to d0/d16/d23/c45 0 2024-04-02T04:42:35.244 INFO:tasks.workunit.client.0.smithi016.stdout:0/209: write d0/f38 [977372,62221] 0 2024-04-02T04:42:35.245 INFO:tasks.workunit.client.0.smithi016.stdout:5/261: creat d3/d40/d4f/d67/f6a x:0 0 0 2024-04-02T04:42:35.247 INFO:tasks.workunit.client.0.smithi016.stdout:0/210: dread - d0/d16/d2e/d36/f3d zero size 2024-04-02T04:42:35.248 INFO:tasks.workunit.client.0.smithi016.stdout:6/225: creat d0/d9/d16/f44 x:0 0 0 2024-04-02T04:42:35.248 INFO:tasks.workunit.client.0.smithi016.stdout:5/262: read - d3/f10 zero size 2024-04-02T04:42:35.249 INFO:tasks.workunit.client.0.smithi016.stdout:8/239: dwrite d1/d6/f21 [0,4194304] 0 2024-04-02T04:42:35.254 INFO:tasks.workunit.client.0.smithi016.stdout:0/211: mknod d0/d16/d24/c46 0 2024-04-02T04:42:35.255 INFO:tasks.workunit.client.0.smithi016.stdout:7/244: dwrite d0/d46/f4d [0,4194304] 0 2024-04-02T04:42:35.256 INFO:tasks.workunit.client.0.smithi016.stdout:5/263: rename d3/d25 to d3/d40/d4f/d6b 0 2024-04-02T04:42:35.256 INFO:tasks.workunit.client.0.smithi016.stdout:7/245: chown d0/d21/f57 858 1 2024-04-02T04:42:35.257 INFO:tasks.workunit.client.0.smithi016.stdout:2/190: sync 2024-04-02T04:42:35.260 INFO:tasks.workunit.client.0.smithi016.stdout:8/240: mkdir d1/d6/d36/d3f/d40/d49/d4d 0 2024-04-02T04:42:35.264 INFO:tasks.workunit.client.0.smithi016.stdout:6/226: dwrite d0/d9/df/d38/f25 [4194304,4194304] 0 2024-04-02T04:42:35.265 INFO:tasks.workunit.client.0.smithi016.stdout:6/227: readlink d0/d9/df/l24 0 2024-04-02T04:42:35.271 INFO:tasks.workunit.client.0.smithi016.stdout:6/228: dread d0/f37 [0,4194304] 0 2024-04-02T04:42:35.271 INFO:tasks.workunit.client.0.smithi016.stdout:4/225: truncate d5/d16/f20 1016522 0 2024-04-02T04:42:35.271 INFO:tasks.workunit.client.0.smithi016.stdout:8/241: mknod d1/d6/d36/c4e 0 2024-04-02T04:42:35.272 INFO:tasks.workunit.client.0.smithi016.stdout:4/226: chown d5/l9 17 1 2024-04-02T04:42:35.273 INFO:tasks.workunit.client.0.smithi016.stdout:6/229: truncate d0/d1f/f3a 508349 0 2024-04-02T04:42:35.273 INFO:tasks.workunit.client.0.smithi016.stdout:4/227: write d5/d1d/d1e/f36 [121008,108859] 0 2024-04-02T04:42:35.276 INFO:tasks.workunit.client.0.smithi016.stdout:2/191: truncate d12/d35/d3f/f2d 1511097 0 2024-04-02T04:42:35.279 INFO:tasks.workunit.client.0.smithi016.stdout:8/242: mkdir d1/d6/d36/d4f 0 2024-04-02T04:42:35.280 INFO:tasks.workunit.client.0.smithi016.stdout:5/264: link d3/c37 d3/d4c/c6c 0 2024-04-02T04:42:35.280 INFO:tasks.workunit.client.0.smithi016.stdout:4/228: dwrite d5/d17/d18/f2b [0,4194304] 0 2024-04-02T04:42:35.281 INFO:tasks.workunit.client.0.smithi016.stdout:4/229: chown d5/d16/d3f 14 1 2024-04-02T04:42:35.282 INFO:tasks.workunit.client.0.smithi016.stdout:6/230: rename d0/f13 to d0/d1f/f45 0 2024-04-02T04:42:35.282 INFO:tasks.workunit.client.0.smithi016.stdout:4/230: fdatasync d5/d1d/d1e/f3c 0 2024-04-02T04:42:35.282 INFO:tasks.workunit.client.0.smithi016.stdout:4/231: stat d5/de/f14 0 2024-04-02T04:42:35.284 INFO:tasks.workunit.client.0.smithi016.stdout:4/232: write d5/d17/d18/d25/f3b [889798,89922] 0 2024-04-02T04:42:35.292 INFO:tasks.workunit.client.0.smithi016.stdout:8/243: creat d1/d6/d36/d3f/d40/d49/d4a/f50 x:0 0 0 2024-04-02T04:42:35.295 INFO:tasks.workunit.client.0.smithi016.stdout:4/233: creat d5/d8/d43/f45 x:0 0 0 2024-04-02T04:42:35.297 INFO:tasks.workunit.client.0.smithi016.stdout:8/244: symlink d1/d31/l51 0 2024-04-02T04:42:35.299 INFO:tasks.workunit.client.0.smithi016.stdout:8/245: stat d1/d6/d36/d3f/d40/d49/f43 0 2024-04-02T04:42:35.299 INFO:tasks.workunit.client.0.smithi016.stdout:6/231: dwrite d0/d9/fd [4194304,4194304] 0 2024-04-02T04:42:35.302 INFO:tasks.workunit.client.0.smithi016.stdout:8/246: write d1/d6/d36/d3f/d40/d49/f44 [404540,12061] 0 2024-04-02T04:42:35.303 INFO:tasks.workunit.client.0.smithi016.stdout:8/247: chown d1/d6/f25 3687582 1 2024-04-02T04:42:35.308 INFO:tasks.workunit.client.0.smithi016.stdout:6/232: dwrite d0/d1f/f2d [4194304,4194304] 0 2024-04-02T04:42:35.309 INFO:tasks.workunit.client.0.smithi016.stdout:4/234: link d5/l27 d5/d17/d18/d1f/l46 0 2024-04-02T04:42:35.310 INFO:tasks.workunit.client.0.smithi016.stdout:4/235: chown d5/f7 504729 1 2024-04-02T04:42:35.311 INFO:tasks.workunit.client.0.smithi016.stdout:6/233: mkdir d0/d9/d16/d46 0 2024-04-02T04:42:35.311 INFO:tasks.workunit.client.0.smithi016.stdout:6/234: dread - d0/f41 zero size 2024-04-02T04:42:35.312 INFO:tasks.workunit.client.0.smithi016.stdout:6/235: readlink d0/d9/df/d38/l15 0 2024-04-02T04:42:35.312 INFO:tasks.workunit.client.0.smithi016.stdout:6/236: readlink d0/d9/df/d38/l15 0 2024-04-02T04:42:35.313 INFO:tasks.workunit.client.0.smithi016.stdout:0/212: truncate d0/d16/d24/f1e 503230 0 2024-04-02T04:42:35.314 INFO:tasks.workunit.client.0.smithi016.stdout:0/213: fsync d0/f32 0 2024-04-02T04:42:35.318 INFO:tasks.workunit.client.0.smithi016.stdout:6/237: unlink d0/d9/df/d38/c1d 0 2024-04-02T04:42:35.319 INFO:tasks.workunit.client.0.smithi016.stdout:6/238: stat d0/f6 0 2024-04-02T04:42:35.319 INFO:tasks.workunit.client.0.smithi016.stdout:4/236: dwrite d5/f10 [0,4194304] 0 2024-04-02T04:42:35.322 INFO:tasks.workunit.client.0.smithi016.stdout:5/265: getdents d3/d4c 0 2024-04-02T04:42:35.324 INFO:tasks.workunit.client.0.smithi016.stdout:6/239: unlink d0/f7 0 2024-04-02T04:42:35.325 INFO:tasks.workunit.client.0.smithi016.stdout:4/237: readlink d5/d17/d18/d1f/l46 0 2024-04-02T04:42:35.325 INFO:tasks.workunit.client.0.smithi016.stdout:4/238: chown d5 369744 1 2024-04-02T04:42:35.326 INFO:tasks.workunit.client.0.smithi016.stdout:4/239: chown d5/d8/d43/f45 39226707 1 2024-04-02T04:42:35.326 INFO:tasks.workunit.client.0.smithi016.stdout:5/266: mkdir d3/d40/d4f/d6b/d2a/d6d 0 2024-04-02T04:42:35.327 INFO:tasks.workunit.client.0.smithi016.stdout:5/267: chown d3/f3b 5077 1 2024-04-02T04:42:35.331 INFO:tasks.workunit.client.0.smithi016.stdout:4/240: rename d5/d17/d18/c19 to d5/d1d/d2d/c47 0 2024-04-02T04:42:35.332 INFO:tasks.workunit.client.0.smithi016.stdout:5/268: mknod d3/d5d/c6e 0 2024-04-02T04:42:35.333 INFO:tasks.workunit.client.0.smithi016.stdout:5/269: write d3/df/d18/d26/d48/f49 [1502034,123862] 0 2024-04-02T04:42:35.335 INFO:tasks.workunit.client.0.smithi016.stdout:6/240: rmdir d0/d1f/d3e 0 2024-04-02T04:42:35.336 INFO:tasks.workunit.client.0.smithi016.stdout:5/270: creat d3/d5d/f6f x:0 0 0 2024-04-02T04:42:35.337 INFO:tasks.workunit.client.0.smithi016.stdout:5/271: write d3/f9 [195554,114455] 0 2024-04-02T04:42:35.338 INFO:tasks.workunit.client.0.smithi016.stdout:5/272: dread - d3/df/d18/d2f/f62 zero size 2024-04-02T04:42:35.341 INFO:tasks.workunit.client.0.smithi016.stdout:6/241: link d0/d9/df/d38/f25 d0/d9/d16/f47 0 2024-04-02T04:42:35.349 INFO:tasks.workunit.client.0.smithi016.stdout:8/248: dwrite d1/d6/d36/f37 [0,4194304] 0 2024-04-02T04:42:35.354 INFO:tasks.workunit.client.0.smithi016.stdout:4/241: dread d5/d8/fd [0,4194304] 0 2024-04-02T04:42:35.357 INFO:tasks.workunit.client.0.smithi016.stdout:4/242: creat d5/d17/d18/f48 x:0 0 0 2024-04-02T04:42:35.358 INFO:tasks.workunit.client.0.smithi016.stdout:8/249: creat d1/f52 x:0 0 0 2024-04-02T04:42:35.359 INFO:tasks.workunit.client.0.smithi016.stdout:5/273: dread d3/df/d18/f35 [0,4194304] 0 2024-04-02T04:42:35.361 INFO:tasks.workunit.client.0.smithi016.stdout:4/243: dread d5/d8/ff [0,4194304] 0 2024-04-02T04:42:35.361 INFO:tasks.workunit.client.0.smithi016.stdout:4/244: chown d5/d1d/f23 0 1 2024-04-02T04:42:35.362 INFO:tasks.workunit.client.0.smithi016.stdout:8/250: creat d1/d6/f53 x:0 0 0 2024-04-02T04:42:35.371 INFO:tasks.workunit.client.0.smithi016.stdout:8/251: truncate d1/d6/d36/d3f/d40/d49/f2a 3054446 0 2024-04-02T04:42:35.407 INFO:tasks.workunit.client.0.smithi016.stdout:9/233: sync 2024-04-02T04:42:35.408 INFO:tasks.workunit.client.0.smithi016.stdout:6/242: dwrite d0/d1f/f42 [0,4194304] 0 2024-04-02T04:42:35.409 INFO:tasks.workunit.client.0.smithi016.stdout:5/274: write d3/f17 [216754,65537] 0 2024-04-02T04:42:35.409 INFO:tasks.workunit.client.0.smithi016.stdout:8/252: mkdir d1/d6/d36/d3f/d40/d49/d4a/d4c/d54 0 2024-04-02T04:42:35.409 INFO:tasks.workunit.client.0.smithi016.stdout:9/234: chown d0/d3 264652 1 2024-04-02T04:42:35.409 INFO:tasks.workunit.client.0.smithi016.stdout:4/245: write d5/d8/fd [1260419,118816] 0 2024-04-02T04:42:35.409 INFO:tasks.workunit.client.0.smithi016.stdout:6/243: stat d0/d9/d16/d46 0 2024-04-02T04:42:35.412 INFO:tasks.workunit.client.0.smithi016.stdout:8/253: read - d1/d6/d36/d3f/d40/d49/f43 zero size 2024-04-02T04:42:35.413 INFO:tasks.workunit.client.0.smithi016.stdout:9/235: fsync d0/d9/f19 0 2024-04-02T04:42:35.413 INFO:tasks.workunit.client.0.smithi016.stdout:5/275: read d3/f59 [1052063,128082] 0 2024-04-02T04:42:35.446 INFO:tasks.workunit.client.0.smithi016.stdout:9/236: readlink d0/d3/l2a 0 2024-04-02T04:42:35.448 INFO:tasks.workunit.client.0.smithi016.stdout:4/246: write d5/d17/f35 [1914985,122092] 0 2024-04-02T04:42:35.448 INFO:tasks.workunit.client.0.smithi016.stdout:9/237: fdatasync d0/d3/f40 0 2024-04-02T04:42:35.453 INFO:tasks.workunit.client.0.smithi016.stdout:6/244: dwrite d0/d1f/f3b [0,4194304] 0 2024-04-02T04:42:35.453 INFO:tasks.workunit.client.0.smithi016.stdout:5/276: dread d3/df/f33 [0,4194304] 0 2024-04-02T04:42:35.456 INFO:tasks.workunit.client.0.smithi016.stdout:9/238: creat d0/d9/db/f43 x:0 0 0 2024-04-02T04:42:35.457 INFO:tasks.workunit.client.0.smithi016.stdout:4/247: creat d5/d16/f49 x:0 0 0 2024-04-02T04:42:35.458 INFO:tasks.workunit.client.0.smithi016.stdout:4/248: chown d5/d16/l2f 4075 1 2024-04-02T04:42:35.462 INFO:tasks.workunit.client.0.smithi016.stdout:5/277: mkdir d3/df/d18/d26/d70 0 2024-04-02T04:42:35.463 INFO:tasks.workunit.client.0.smithi016.stdout:5/278: write d3/d5d/f69 [771309,64104] 0 2024-04-02T04:42:35.463 INFO:tasks.workunit.client.0.smithi016.stdout:9/239: creat d0/d16/f44 x:0 0 0 2024-04-02T04:42:35.464 INFO:tasks.workunit.client.0.smithi016.stdout:5/279: fsync d3/df/f65 0 2024-04-02T04:42:35.464 INFO:tasks.workunit.client.0.smithi016.stdout:9/240: dread - d0/d16/d1a/f37 zero size 2024-04-02T04:42:35.467 INFO:tasks.workunit.client.0.smithi016.stdout:8/254: rmdir d1/d6/d36/d3f/d40/d49/d4a/d4c/d54 0 2024-04-02T04:42:35.470 INFO:tasks.workunit.client.0.smithi016.stdout:4/249: link d5/d1d/f21 d5/d17/d18/d25/f4a 0 2024-04-02T04:42:35.475 INFO:tasks.workunit.client.0.smithi016.stdout:9/241: unlink d0/f11 0 2024-04-02T04:42:35.476 INFO:tasks.workunit.client.0.smithi016.stdout:9/242: truncate d0/d3/f40 4410314 0 2024-04-02T04:42:35.477 INFO:tasks.workunit.client.0.smithi016.stdout:1/248: sync 2024-04-02T04:42:35.477 INFO:tasks.workunit.client.0.smithi016.stdout:3/278: sync 2024-04-02T04:42:35.478 INFO:tasks.workunit.client.0.smithi016.stdout:5/280: mknod d3/df/d18/d26/d48/c71 0 2024-04-02T04:42:35.479 INFO:tasks.workunit.client.0.smithi016.stdout:5/281: write d3/d5d/f69 [1022973,74901] 0 2024-04-02T04:42:35.480 INFO:tasks.workunit.client.0.smithi016.stdout:8/255: fsync d1/d31/f46 0 2024-04-02T04:42:35.484 INFO:tasks.workunit.client.0.smithi016.stdout:1/249: truncate d14/d19/d1d/f30 106555 0 2024-04-02T04:42:35.484 INFO:tasks.workunit.client.0.smithi016.stdout:5/282: dwrite d3/d5d/f69 [0,4194304] 0 2024-04-02T04:42:35.485 INFO:tasks.workunit.client.0.smithi016.stdout:5/283: chown d3/df/d18 140698 1 2024-04-02T04:42:35.485 INFO:tasks.workunit.client.0.smithi016.stdout:1/250: write d14/d19/d20/f24 [1225967,129225] 0 2024-04-02T04:42:35.489 INFO:tasks.workunit.client.0.smithi016.stdout:4/250: rename d5/d16/f20 to d5/d8/d43/f4b 0 2024-04-02T04:42:35.490 INFO:tasks.workunit.client.0.smithi016.stdout:8/256: dread d1/f19 [0,4194304] 0 2024-04-02T04:42:35.491 INFO:tasks.workunit.client.0.smithi016.stdout:9/243: creat d0/d3/d31/f45 x:0 0 0 2024-04-02T04:42:35.491 INFO:tasks.workunit.client.0.smithi016.stdout:8/257: chown d1/d6/f2e 365489189 1 2024-04-02T04:42:35.493 INFO:tasks.workunit.client.0.smithi016.stdout:5/284: mkdir d3/d40/d72 0 2024-04-02T04:42:35.495 INFO:tasks.workunit.client.0.smithi016.stdout:1/251: creat d14/d19/f4b x:0 0 0 2024-04-02T04:42:35.496 INFO:tasks.workunit.client.0.smithi016.stdout:4/251: mknod d5/d17/d18/d1f/c4c 0 2024-04-02T04:42:35.499 INFO:tasks.workunit.client.0.smithi016.stdout:5/285: dwrite d3/d5d/f69 [0,4194304] 0 2024-04-02T04:42:35.500 INFO:tasks.workunit.client.0.smithi016.stdout:3/279: creat d2/d15/d16/f54 x:0 0 0 2024-04-02T04:42:35.500 INFO:tasks.workunit.client.0.smithi016.stdout:5/286: chown d3/df/d18 21737 1 2024-04-02T04:42:35.500 INFO:tasks.workunit.client.0.smithi016.stdout:3/280: chown d2/d15/f23 6 1 2024-04-02T04:42:35.501 INFO:tasks.workunit.client.0.smithi016.stdout:8/258: rmdir d1/d6/d36/d3f/d40 39 2024-04-02T04:42:35.511 INFO:tasks.workunit.client.0.smithi016.stdout:4/252: truncate d5/d8/ff 3995290 0 2024-04-02T04:42:35.514 INFO:tasks.workunit.client.0.smithi016.stdout:3/281: rename d2/f20 to d2/d15/d1a/d45/f55 0 2024-04-02T04:42:35.515 INFO:tasks.workunit.client.0.smithi016.stdout:3/282: write d2/f14 [939487,90465] 0 2024-04-02T04:42:35.517 INFO:tasks.workunit.client.0.smithi016.stdout:5/287: truncate d3/df/d18/f38 7779919 0 2024-04-02T04:42:35.518 INFO:tasks.workunit.client.0.smithi016.stdout:8/259: stat d1/d6/d36/d3f/d40/d49 0 2024-04-02T04:42:35.520 INFO:tasks.workunit.client.0.smithi016.stdout:4/253: creat d5/d1d/d2d/f4d x:0 0 0 2024-04-02T04:42:35.521 INFO:tasks.workunit.client.0.smithi016.stdout:4/254: chown d5/de/f14 1491109 1 2024-04-02T04:42:35.530 INFO:tasks.workunit.client.0.smithi016.stdout:3/283: mkdir d2/d15/d1a/d45/d56 0 2024-04-02T04:42:35.530 INFO:tasks.workunit.client.0.smithi016.stdout:7/246: sync 2024-04-02T04:42:35.531 INFO:tasks.workunit.client.0.smithi016.stdout:3/284: read d2/f9 [2798429,4448] 0 2024-04-02T04:42:35.532 INFO:tasks.workunit.client.0.smithi016.stdout:5/288: rmdir d3/d40/d4f/d67 39 2024-04-02T04:42:35.533 INFO:tasks.workunit.client.0.smithi016.stdout:5/289: dread - d3/df/d18/d26/d48/f61 zero size 2024-04-02T04:42:35.536 INFO:tasks.workunit.client.0.smithi016.stdout:7/247: mknod d0/d46/c5a 0 2024-04-02T04:42:35.539 INFO:tasks.workunit.client.0.smithi016.stdout:5/290: symlink d3/df/d18/d26/d2c/l73 0 2024-04-02T04:42:35.542 INFO:tasks.workunit.client.0.smithi016.stdout:9/244: write d0/f7 [586036,82384] 0 2024-04-02T04:42:35.543 INFO:tasks.workunit.client.0.smithi016.stdout:9/245: fdatasync d0/d3/d31/f45 0 2024-04-02T04:42:35.544 INFO:tasks.workunit.client.0.smithi016.stdout:3/285: mkdir d2/d15/d16/d57 0 2024-04-02T04:42:35.546 INFO:tasks.workunit.client.0.smithi016.stdout:5/291: creat d3/d40/d4f/d6b/f74 x:0 0 0 2024-04-02T04:42:35.548 INFO:tasks.workunit.client.0.smithi016.stdout:5/292: dread d3/d40/d4f/d6b/f47 [0,4194304] 0 2024-04-02T04:42:35.548 INFO:tasks.workunit.client.0.smithi016.stdout:1/252: chown d14/d19/d1e/c3e 145 1 2024-04-02T04:42:35.553 INFO:tasks.workunit.client.0.smithi016.stdout:9/246: mkdir d0/d16/d1a/d46 0 2024-04-02T04:42:35.562 INFO:tasks.workunit.client.0.smithi016.stdout:8/260: getdents d1/d6 0 2024-04-02T04:42:35.567 INFO:tasks.workunit.client.0.smithi016.stdout:3/286: mknod d2/d15/d1a/d45/d56/c58 0 2024-04-02T04:42:35.571 INFO:tasks.workunit.client.0.smithi016.stdout:1/253: rmdir d14/d19/d20 39 2024-04-02T04:42:35.574 INFO:tasks.workunit.client.0.smithi016.stdout:7/248: getdents d0/d46/d37/d3a/d43 0 2024-04-02T04:42:35.578 INFO:tasks.workunit.client.0.smithi016.stdout:9/247: chown d0/d9/f2e 10871 1 2024-04-02T04:42:35.591 INFO:tasks.workunit.client.0.smithi016.stdout:5/293: getdents d3/d40 0 2024-04-02T04:42:35.595 INFO:tasks.workunit.client.0.smithi016.stdout:7/249: mknod d0/d46/d52/c5b 0 2024-04-02T04:42:35.596 INFO:tasks.workunit.client.0.smithi016.stdout:5/294: dwrite d3/f6 [0,4194304] 0 2024-04-02T04:42:35.601 INFO:tasks.workunit.client.0.smithi016.stdout:1/254: rmdir d14/d19/d1d/d29/d39/d46 39 2024-04-02T04:42:35.604 INFO:tasks.workunit.client.0.smithi016.stdout:6/245: sync 2024-04-02T04:42:35.604 INFO:tasks.workunit.client.0.smithi016.stdout:0/214: sync 2024-04-02T04:42:35.604 INFO:tasks.workunit.client.0.smithi016.stdout:2/192: sync 2024-04-02T04:42:35.605 INFO:tasks.workunit.client.0.smithi016.stdout:7/250: creat d0/d46/d37/d47/f5c x:0 0 0 2024-04-02T04:42:35.606 INFO:tasks.workunit.client.0.smithi016.stdout:8/261: dwrite d1/d6/d36/d3f/d40/d49/f2a [0,4194304] 0 2024-04-02T04:42:35.607 INFO:tasks.workunit.client.0.smithi016.stdout:6/246: chown d0/d9/d16/l39 0 1 2024-04-02T04:42:35.607 INFO:tasks.workunit.client.0.smithi016.stdout:2/193: dread d12/f2a [0,4194304] 0 2024-04-02T04:42:35.607 INFO:tasks.workunit.client.0.smithi016.stdout:7/251: readlink d0/d46/l1e 0 2024-04-02T04:42:35.607 INFO:tasks.workunit.client.0.smithi016.stdout:9/248: rename d0/d16/c3c to d0/d9/c47 0 2024-04-02T04:42:35.609 INFO:tasks.workunit.client.0.smithi016.stdout:3/287: getdents d2/d15/d16/d3d/d46 0 2024-04-02T04:42:35.637 INFO:tasks.workunit.client.0.smithi016.stdout:1/255: mkdir d14/d19/d1d/d29/d4c 0 2024-04-02T04:42:35.638 INFO:tasks.workunit.client.0.smithi016.stdout:8/262: unlink d1/d6/c34 0 2024-04-02T04:42:35.638 INFO:tasks.workunit.client.0.smithi016.stdout:6/247: symlink d0/d9/df/l48 0 2024-04-02T04:42:35.638 INFO:tasks.workunit.client.0.smithi016.stdout:7/252: symlink d0/d46/d52/l5d 0 2024-04-02T04:42:35.639 INFO:tasks.workunit.client.0.smithi016.stdout:6/248: read d0/f4 [4748,80257] 0 2024-04-02T04:42:35.640 INFO:tasks.workunit.client.0.smithi016.stdout:6/249: dread - d0/f3f zero size 2024-04-02T04:42:35.645 INFO:tasks.workunit.client.0.smithi016.stdout:3/288: dread d2/d15/d1a/d45/f55 [0,4194304] 0 2024-04-02T04:42:35.647 INFO:tasks.workunit.client.0.smithi016.stdout:8/263: symlink d1/d31/l55 0 2024-04-02T04:42:35.653 INFO:tasks.workunit.client.0.smithi016.stdout:5/295: rename d3/df/f1c to d3/d40/d4f/f75 0 2024-04-02T04:42:35.655 INFO:tasks.workunit.client.0.smithi016.stdout:2/194: getdents d12 0 2024-04-02T04:42:35.656 INFO:tasks.workunit.client.0.smithi016.stdout:1/256: mkdir d14/d19/d1d/d29/d39/d4d 0 2024-04-02T04:42:35.658 INFO:tasks.workunit.client.0.smithi016.stdout:3/289: symlink d2/d15/d16/d3d/d46/l59 0 2024-04-02T04:42:35.659 INFO:tasks.workunit.client.0.smithi016.stdout:3/290: write d2/d15/d1a/f53 [388417,107804] 0 2024-04-02T04:42:35.660 INFO:tasks.workunit.client.0.smithi016.stdout:8/264: creat d1/d6/d36/d3f/d40/d49/d4a/f56 x:0 0 0 2024-04-02T04:42:35.660 INFO:tasks.workunit.client.0.smithi016.stdout:3/291: dread - d2/d2e/f2f zero size 2024-04-02T04:42:35.664 INFO:tasks.workunit.client.0.smithi016.stdout:3/292: dwrite d2/fd [0,4194304] 0 2024-04-02T04:42:35.668 INFO:tasks.workunit.client.0.smithi016.stdout:9/249: rename d0/d16/f20 to d0/d16/d1a/f48 0 2024-04-02T04:42:35.672 INFO:tasks.workunit.client.0.smithi016.stdout:2/195: readlink d12/d35/d3b/l41 0 2024-04-02T04:42:35.675 INFO:tasks.workunit.client.0.smithi016.stdout:4/255: sync 2024-04-02T04:42:35.676 INFO:tasks.workunit.client.0.smithi016.stdout:4/256: fsync d5/f7 0 2024-04-02T04:42:35.678 INFO:tasks.workunit.client.0.smithi016.stdout:8/265: rename d1/d6/f39 to d1/d6/f57 0 2024-04-02T04:42:35.679 INFO:tasks.workunit.client.0.smithi016.stdout:8/266: truncate d1/d31/f46 4978232 0 2024-04-02T04:42:35.680 INFO:tasks.workunit.client.0.smithi016.stdout:3/293: creat d2/d15/d16/d3d/d46/f5a x:0 0 0 2024-04-02T04:42:35.681 INFO:tasks.workunit.client.0.smithi016.stdout:3/294: write d2/f28 [94896,95442] 0 2024-04-02T04:42:35.682 INFO:tasks.workunit.client.0.smithi016.stdout:4/257: dwrite d5/d17/d18/d25/f41 [0,4194304] 0 2024-04-02T04:42:35.684 INFO:tasks.workunit.client.0.smithi016.stdout:2/196: rmdir d12/d35/d3f 39 2024-04-02T04:42:35.696 INFO:tasks.workunit.client.0.smithi016.stdout:6/250: truncate d0/f6 844985 0 2024-04-02T04:42:35.697 INFO:tasks.workunit.client.0.smithi016.stdout:8/267: symlink d1/d6/d36/d3f/d40/d49/l58 0 2024-04-02T04:42:35.700 INFO:tasks.workunit.client.0.smithi016.stdout:5/296: creat d3/df/d18/f76 x:0 0 0 2024-04-02T04:42:35.701 INFO:tasks.workunit.client.0.smithi016.stdout:3/295: mkdir d2/d2e/d36/d5b 0 2024-04-02T04:42:35.702 INFO:tasks.workunit.client.0.smithi016.stdout:3/296: readlink d2/l33 0 2024-04-02T04:42:35.705 INFO:tasks.workunit.client.0.smithi016.stdout:2/197: mknod d12/d26/c45 0 2024-04-02T04:42:35.707 INFO:tasks.workunit.client.0.smithi016.stdout:9/250: link d0/d16/l28 d0/d16/d1a/l49 0 2024-04-02T04:42:35.709 INFO:tasks.workunit.client.0.smithi016.stdout:6/251: mknod d0/d9/d16/c49 0 2024-04-02T04:42:35.717 INFO:tasks.workunit.client.0.smithi016.stdout:1/257: dwrite d14/d19/d1d/f30 [0,4194304] 0 2024-04-02T04:42:35.717 INFO:tasks.workunit.client.0.smithi016.stdout:2/198: creat d12/d35/d3b/f46 x:0 0 0 2024-04-02T04:42:35.719 INFO:tasks.workunit.client.0.smithi016.stdout:3/297: dwrite d2/d15/d1a/d45/f55 [0,4194304] 0 2024-04-02T04:42:35.721 INFO:tasks.workunit.client.0.smithi016.stdout:6/252: write d0/d9/d16/f20 [5119429,17332] 0 2024-04-02T04:42:35.722 INFO:tasks.workunit.client.0.smithi016.stdout:6/253: stat d0/d1f/f26 0 2024-04-02T04:42:35.724 INFO:tasks.workunit.client.0.smithi016.stdout:6/254: write d0/d9/df/d38/f11 [583210,115276] 0 2024-04-02T04:42:35.733 INFO:tasks.workunit.client.0.smithi016.stdout:1/258: write d14/d19/d20/f3f [249683,130855] 0 2024-04-02T04:42:35.735 INFO:tasks.workunit.client.0.smithi016.stdout:9/251: mkdir d0/d4a 0 2024-04-02T04:42:35.736 INFO:tasks.workunit.client.0.smithi016.stdout:3/298: rmdir d2/d2e/d36 39 2024-04-02T04:42:35.737 INFO:tasks.workunit.client.0.smithi016.stdout:4/258: write d5/d8/d43/f4b [220090,92477] 0 2024-04-02T04:42:35.738 INFO:tasks.workunit.client.0.smithi016.stdout:4/259: chown d5/d17/d18/f48 202526705 1 2024-04-02T04:42:35.739 INFO:tasks.workunit.client.0.smithi016.stdout:8/268: link d1/d6/f2e d1/f59 0 2024-04-02T04:42:35.739 INFO:tasks.workunit.client.0.smithi016.stdout:4/260: dread - d5/d17/d18/d1f/f39 zero size 2024-04-02T04:42:35.742 INFO:tasks.workunit.client.0.smithi016.stdout:5/297: creat d3/f77 x:0 0 0 2024-04-02T04:42:35.744 INFO:tasks.workunit.client.0.smithi016.stdout:2/199: symlink d12/d35/d3b/d42/l47 0 2024-04-02T04:42:35.746 INFO:tasks.workunit.client.0.smithi016.stdout:9/252: mkdir d0/d16/d4b 0 2024-04-02T04:42:35.749 INFO:tasks.workunit.client.0.smithi016.stdout:4/261: rmdir d5/d1d/d2d 39 2024-04-02T04:42:35.750 INFO:tasks.workunit.client.0.smithi016.stdout:8/269: creat d1/d6/d36/d3f/d40/d49/d4a/f5a x:0 0 0 2024-04-02T04:42:35.752 INFO:tasks.workunit.client.0.smithi016.stdout:6/255: mknod d0/d9/c4a 0 2024-04-02T04:42:35.752 INFO:tasks.workunit.client.0.smithi016.stdout:6/256: fdatasync d0/d9/d16/f23 0 2024-04-02T04:42:35.755 INFO:tasks.workunit.client.0.smithi016.stdout:7/253: sync 2024-04-02T04:42:35.755 INFO:tasks.workunit.client.0.smithi016.stdout:0/215: sync 2024-04-02T04:42:35.756 INFO:tasks.workunit.client.0.smithi016.stdout:7/254: write d0/d28/f59 [1005622,74705] 0 2024-04-02T04:42:35.757 INFO:tasks.workunit.client.0.smithi016.stdout:1/259: mknod d14/d19/c4e 0 2024-04-02T04:42:35.759 INFO:tasks.workunit.client.0.smithi016.stdout:2/200: creat d12/d26/f48 x:0 0 0 2024-04-02T04:42:35.761 INFO:tasks.workunit.client.0.smithi016.stdout:9/253: write d0/d33/f38 [4950418,74893] 0 2024-04-02T04:42:35.761 INFO:tasks.workunit.client.0.smithi016.stdout:0/216: dwrite d0/f19 [0,4194304] 0 2024-04-02T04:42:35.766 INFO:tasks.workunit.client.0.smithi016.stdout:1/260: dwrite d14/d19/d20/f38 [0,4194304] 0 2024-04-02T04:42:35.774 INFO:tasks.workunit.client.0.smithi016.stdout:7/255: creat d0/d46/d37/d3a/d43/f5e x:0 0 0 2024-04-02T04:42:35.780 INFO:tasks.workunit.client.0.smithi016.stdout:2/201: unlink d12/f13 0 2024-04-02T04:42:35.787 INFO:tasks.workunit.client.0.smithi016.stdout:0/217: read d0/d16/d24/f1e [84914,110956] 0 2024-04-02T04:42:35.788 INFO:tasks.workunit.client.0.smithi016.stdout:0/218: chown d0/d16/d24/c46 7 1 2024-04-02T04:42:35.789 INFO:tasks.workunit.client.0.smithi016.stdout:0/219: chown d0/d16/d24 1 1 2024-04-02T04:42:35.789 INFO:tasks.workunit.client.0.smithi016.stdout:1/261: write d14/d19/d1e/f2b [2781971,99235] 0 2024-04-02T04:42:35.791 INFO:tasks.workunit.client.0.smithi016.stdout:1/262: chown d14/d19/c4e 4370 1 2024-04-02T04:42:35.792 INFO:tasks.workunit.client.0.smithi016.stdout:6/257: link d0/d9/df/d38/f11 d0/d1f/d35/f4b 0 2024-04-02T04:42:35.796 INFO:tasks.workunit.client.0.smithi016.stdout:1/263: dwrite d14/d19/d20/f38 [4194304,4194304] 0 2024-04-02T04:42:35.796 INFO:tasks.workunit.client.0.smithi016.stdout:3/299: rename d2/d15/d16/d3d/l4d to d2/d15/d16/l5c 0 2024-04-02T04:42:35.798 INFO:tasks.workunit.client.0.smithi016.stdout:9/254: mknod d0/d4a/c4c 0 2024-04-02T04:42:35.801 INFO:tasks.workunit.client.0.smithi016.stdout:4/262: creat d5/f4e x:0 0 0 2024-04-02T04:42:35.801 INFO:tasks.workunit.client.0.smithi016.stdout:4/263: fdatasync d5/f10 0 2024-04-02T04:42:35.804 INFO:tasks.workunit.client.0.smithi016.stdout:1/264: dwrite d14/d19/d1e/f42 [0,4194304] 0 2024-04-02T04:42:35.811 INFO:tasks.workunit.client.0.smithi016.stdout:6/258: symlink d0/d1f/l4c 0 2024-04-02T04:42:35.813 INFO:tasks.workunit.client.0.smithi016.stdout:3/300: write d2/f9 [2753620,38744] 0 2024-04-02T04:42:35.815 INFO:tasks.workunit.client.0.smithi016.stdout:5/298: rename d3/df/d18/d26/f54 to d3/d40/d4f/d67/f78 0 2024-04-02T04:42:35.820 INFO:tasks.workunit.client.0.smithi016.stdout:3/301: dwrite d2/d15/d16/d3d/d46/f5a [0,4194304] 0 2024-04-02T04:42:35.823 INFO:tasks.workunit.client.0.smithi016.stdout:4/264: symlink d5/d8/d43/l4f 0 2024-04-02T04:42:35.824 INFO:tasks.workunit.client.0.smithi016.stdout:4/265: fsync d5/d17/d18/d1f/f39 0 2024-04-02T04:42:35.824 INFO:tasks.workunit.client.0.smithi016.stdout:2/202: stat d12/d14 0 2024-04-02T04:42:35.825 INFO:tasks.workunit.client.0.smithi016.stdout:9/255: rmdir d0 39 2024-04-02T04:42:35.829 INFO:tasks.workunit.client.0.smithi016.stdout:8/270: getdents d1/d6 0 2024-04-02T04:42:35.832 INFO:tasks.workunit.client.0.smithi016.stdout:6/259: creat d0/d9/df/f4d x:0 0 0 2024-04-02T04:42:35.833 INFO:tasks.workunit.client.0.smithi016.stdout:0/220: dwrite d0/f10 [0,4194304] 0 2024-04-02T04:42:35.837 INFO:tasks.workunit.client.0.smithi016.stdout:7/256: rename d0/d46/d52/l5d to d0/d46/l5f 0 2024-04-02T04:42:35.842 INFO:tasks.workunit.client.0.smithi016.stdout:5/299: dwrite d3/d40/d4f/d67/f78 [0,4194304] 0 2024-04-02T04:42:35.843 INFO:tasks.workunit.client.0.smithi016.stdout:2/203: chown d12/d35/c36 34 1 2024-04-02T04:42:35.845 INFO:tasks.workunit.client.0.smithi016.stdout:1/265: mkdir d14/d19/d1d/d45/d4f 0 2024-04-02T04:42:35.845 INFO:tasks.workunit.client.0.smithi016.stdout:4/266: dread d5/d8/ff [0,4194304] 0 2024-04-02T04:42:35.846 INFO:tasks.workunit.client.0.smithi016.stdout:4/267: fsync d5/d8/d43/f45 0 2024-04-02T04:42:35.848 INFO:tasks.workunit.client.0.smithi016.stdout:6/260: symlink d0/d9/df/l4e 0 2024-04-02T04:42:35.852 INFO:tasks.workunit.client.0.smithi016.stdout:8/271: rename d1/f59 to d1/d6/d36/d3f/d40/d45/f5b 0 2024-04-02T04:42:35.853 INFO:tasks.workunit.client.0.smithi016.stdout:5/300: creat d3/d40/f79 x:0 0 0 2024-04-02T04:42:35.856 INFO:tasks.workunit.client.0.smithi016.stdout:6/261: dwrite d0/d9/fd [4194304,4194304] 0 2024-04-02T04:42:35.856 INFO:tasks.workunit.client.0.smithi016.stdout:4/268: mkdir d5/d1d/d1e/d50 0 2024-04-02T04:42:35.858 INFO:tasks.workunit.client.0.smithi016.stdout:0/221: mknod d0/d16/c47 0 2024-04-02T04:42:35.858 INFO:tasks.workunit.client.0.smithi016.stdout:3/302: rename d2/d2e/l32 to d2/d2e/l5d 0 2024-04-02T04:42:35.862 INFO:tasks.workunit.client.0.smithi016.stdout:2/204: mknod d12/d26/c49 0 2024-04-02T04:42:35.862 INFO:tasks.workunit.client.0.smithi016.stdout:8/272: dread d1/f19 [0,4194304] 0 2024-04-02T04:42:35.866 INFO:tasks.workunit.client.0.smithi016.stdout:6/262: dwrite d0/d9/df/f4d [0,4194304] 0 2024-04-02T04:42:35.868 INFO:tasks.workunit.client.0.smithi016.stdout:8/273: truncate d1/d6/d36/d3f/d40/d49/d4a/f5a 29074 0 2024-04-02T04:42:35.868 INFO:tasks.workunit.client.0.smithi016.stdout:1/266: mkdir d14/d19/d50 0 2024-04-02T04:42:35.871 INFO:tasks.workunit.client.0.smithi016.stdout:0/222: mkdir d0/d48 0 2024-04-02T04:42:35.874 INFO:tasks.workunit.client.0.smithi016.stdout:7/257: rename d0/d46/d37/d3a/c55 to d0/d46/d52/c60 0 2024-04-02T04:42:35.876 INFO:tasks.workunit.client.0.smithi016.stdout:6/263: dwrite d0/f40 [0,4194304] 0 2024-04-02T04:42:35.877 INFO:tasks.workunit.client.0.smithi016.stdout:3/303: creat d2/d15/d1a/d45/d56/f5e x:0 0 0 2024-04-02T04:42:35.878 INFO:tasks.workunit.client.0.smithi016.stdout:6/264: dread - d0/d9/f32 zero size 2024-04-02T04:42:35.884 INFO:tasks.workunit.client.0.smithi016.stdout:9/256: getdents d0/d4a 0 2024-04-02T04:42:35.888 INFO:tasks.workunit.client.0.smithi016.stdout:7/258: creat d0/d46/d52/f61 x:0 0 0 2024-04-02T04:42:35.889 INFO:tasks.workunit.client.0.smithi016.stdout:5/301: dwrite d3/df/f1e [0,4194304] 0 2024-04-02T04:42:35.889 INFO:tasks.workunit.client.0.smithi016.stdout:4/269: dwrite d5/d17/d18/d25/f4a [0,4194304] 0 2024-04-02T04:42:35.891 INFO:tasks.workunit.client.0.smithi016.stdout:2/205: mknod d12/d35/d3f/c4a 0 2024-04-02T04:42:35.892 INFO:tasks.workunit.client.0.smithi016.stdout:9/257: dread d0/f7 [0,4194304] 0 2024-04-02T04:42:35.893 INFO:tasks.workunit.client.0.smithi016.stdout:6/265: creat d0/d9/df/d38/f4f x:0 0 0 2024-04-02T04:42:35.895 INFO:tasks.workunit.client.0.smithi016.stdout:2/206: chown d12/d35/d3f/f39 1 1 2024-04-02T04:42:35.898 INFO:tasks.workunit.client.0.smithi016.stdout:3/304: mknod d2/d15/c5f 0 2024-04-02T04:42:35.902 INFO:tasks.workunit.client.0.smithi016.stdout:0/223: dwrite d0/d16/f18 [0,4194304] 0 2024-04-02T04:42:35.904 INFO:tasks.workunit.client.0.smithi016.stdout:8/274: creat d1/f5c x:0 0 0 2024-04-02T04:42:35.904 INFO:tasks.workunit.client.0.smithi016.stdout:0/224: dread - d0/f32 zero size 2024-04-02T04:42:35.906 INFO:tasks.workunit.client.0.smithi016.stdout:9/258: dwrite d0/f2c [0,4194304] 0 2024-04-02T04:42:35.907 INFO:tasks.workunit.client.0.smithi016.stdout:1/267: dread d14/d19/d1e/f23 [0,4194304] 0 2024-04-02T04:42:35.909 INFO:tasks.workunit.client.0.smithi016.stdout:1/268: dread - d14/d19/d1d/d29/f3d zero size 2024-04-02T04:42:35.909 INFO:tasks.workunit.client.0.smithi016.stdout:6/266: symlink d0/d9/d16/l50 0 2024-04-02T04:42:35.912 INFO:tasks.workunit.client.0.smithi016.stdout:2/207: creat d12/d35/d3f/f4b x:0 0 0 2024-04-02T04:42:35.912 INFO:tasks.workunit.client.0.smithi016.stdout:4/270: dread d5/d1d/f23 [0,4194304] 0 2024-04-02T04:42:35.918 INFO:tasks.workunit.client.0.smithi016.stdout:1/269: creat d14/d19/d1d/d29/f51 x:0 0 0 2024-04-02T04:42:35.920 INFO:tasks.workunit.client.0.smithi016.stdout:3/305: mkdir d2/d15/d60 0 2024-04-02T04:42:35.920 INFO:tasks.workunit.client.0.smithi016.stdout:4/271: dwrite d5/d17/d18/d25/f41 [0,4194304] 0 2024-04-02T04:42:35.921 INFO:tasks.workunit.client.0.smithi016.stdout:8/275: mkdir d1/d6/d36/d4f/d5d 0 2024-04-02T04:42:35.922 INFO:tasks.workunit.client.0.smithi016.stdout:9/259: mkdir d0/d9/d4d 0 2024-04-02T04:42:35.924 INFO:tasks.workunit.client.0.smithi016.stdout:5/302: getdents d3/d40/d4f/d6b/d2a 0 2024-04-02T04:42:35.925 INFO:tasks.workunit.client.0.smithi016.stdout:4/272: rmdir d5/d17 39 2024-04-02T04:42:35.926 INFO:tasks.workunit.client.0.smithi016.stdout:3/306: dread d2/d15/d16/f27 [0,4194304] 0 2024-04-02T04:42:35.927 INFO:tasks.workunit.client.0.smithi016.stdout:6/267: creat d0/f51 x:0 0 0 2024-04-02T04:42:35.928 INFO:tasks.workunit.client.0.smithi016.stdout:9/260: truncate d0/d9/f3d 232574 0 2024-04-02T04:42:35.930 INFO:tasks.workunit.client.0.smithi016.stdout:8/276: rmdir d1/d6/d36/d3f 39 2024-04-02T04:42:35.931 INFO:tasks.workunit.client.0.smithi016.stdout:1/270: mknod d14/c52 0 2024-04-02T04:42:35.933 INFO:tasks.workunit.client.0.smithi016.stdout:5/303: creat d3/df/d18/d26/f7a x:0 0 0 2024-04-02T04:42:35.934 INFO:tasks.workunit.client.0.smithi016.stdout:0/225: getdents d0/d16 0 2024-04-02T04:42:35.935 INFO:tasks.workunit.client.0.smithi016.stdout:3/307: rename d2/d2e/f2f to d2/d15/d1a/d45/f61 0 2024-04-02T04:42:35.936 INFO:tasks.workunit.client.0.smithi016.stdout:7/259: dread d0/d46/f1d [0,4194304] 0 2024-04-02T04:42:35.938 INFO:tasks.workunit.client.0.smithi016.stdout:2/208: rename d12/f16 to d12/f4c 0 2024-04-02T04:42:35.939 INFO:tasks.workunit.client.0.smithi016.stdout:4/273: unlink d5/d17/d18/d1f/f39 0 2024-04-02T04:42:35.941 INFO:tasks.workunit.client.0.smithi016.stdout:0/226: dwrite d0/f38 [0,4194304] 0 2024-04-02T04:42:35.943 INFO:tasks.workunit.client.0.smithi016.stdout:9/261: mkdir d0/d4e 0 2024-04-02T04:42:35.946 INFO:tasks.workunit.client.0.smithi016.stdout:3/308: unlink d2/f28 0 2024-04-02T04:42:35.948 INFO:tasks.workunit.client.0.smithi016.stdout:7/260: rmdir d0/d46/d37 39 2024-04-02T04:42:35.951 INFO:tasks.workunit.client.0.smithi016.stdout:8/277: creat d1/d6/d36/d4f/d5d/f5e x:0 0 0 2024-04-02T04:42:35.956 INFO:tasks.workunit.client.0.smithi016.stdout:5/304: mknod d3/df/c7b 0 2024-04-02T04:42:35.957 INFO:tasks.workunit.client.0.smithi016.stdout:5/305: fsync d3/df/f65 0 2024-04-02T04:42:35.958 INFO:tasks.workunit.client.0.smithi016.stdout:5/306: readlink d3/d4c/l58 0 2024-04-02T04:42:35.959 INFO:tasks.workunit.client.0.smithi016.stdout:4/274: rename d5/l9 to d5/d17/d18/l51 0 2024-04-02T04:42:35.960 INFO:tasks.workunit.client.0.smithi016.stdout:4/275: write d5/d17/d18/f48 [194036,89902] 0 2024-04-02T04:42:35.961 INFO:tasks.workunit.client.0.smithi016.stdout:0/227: symlink d0/d16/d2e/d36/l49 0 2024-04-02T04:42:35.962 INFO:tasks.workunit.client.0.smithi016.stdout:9/262: symlink d0/d3/l4f 0 2024-04-02T04:42:35.965 INFO:tasks.workunit.client.0.smithi016.stdout:3/309: readlink d2/d15/d16/l5c 0 2024-04-02T04:42:35.966 INFO:tasks.workunit.client.0.smithi016.stdout:7/261: creat d0/d46/d16/f62 x:0 0 0 2024-04-02T04:42:35.970 INFO:tasks.workunit.client.0.smithi016.stdout:6/268: getdents d0/d1f 0 2024-04-02T04:42:35.971 INFO:tasks.workunit.client.0.smithi016.stdout:6/269: write d0/f51 [360895,66132] 0 2024-04-02T04:42:35.972 INFO:tasks.workunit.client.0.smithi016.stdout:5/307: mknod d3/df/d18/d2f/c7c 0 2024-04-02T04:42:35.972 INFO:tasks.workunit.client.0.smithi016.stdout:6/270: stat d0/d1f/f3b 0 2024-04-02T04:42:35.973 INFO:tasks.workunit.client.0.smithi016.stdout:4/276: write d5/de/f14 [1748504,8987] 0 2024-04-02T04:42:35.974 INFO:tasks.workunit.client.0.smithi016.stdout:4/277: write d5/d17/d18/d1f/f2a [26021,40158] 0 2024-04-02T04:42:35.976 INFO:tasks.workunit.client.0.smithi016.stdout:0/228: fsync d0/fe 0 2024-04-02T04:42:35.977 INFO:tasks.workunit.client.0.smithi016.stdout:9/263: dread - d0/d9/db/f43 zero size 2024-04-02T04:42:35.978 INFO:tasks.workunit.client.0.smithi016.stdout:6/271: dwrite d0/d1f/f3a [0,4194304] 0 2024-04-02T04:42:35.979 INFO:tasks.workunit.client.0.smithi016.stdout:6/272: fdatasync d0/d9/d16/f3c 0 2024-04-02T04:42:35.980 INFO:tasks.workunit.client.0.smithi016.stdout:3/310: unlink d2/d15/f17 0 2024-04-02T04:42:35.980 INFO:tasks.workunit.client.0.smithi016.stdout:3/311: stat d2 0 2024-04-02T04:42:35.985 INFO:tasks.workunit.client.0.smithi016.stdout:8/278: mknod d1/d6/d36/c5f 0 2024-04-02T04:42:35.991 INFO:tasks.workunit.client.0.smithi016.stdout:4/278: symlink d5/d17/l52 0 2024-04-02T04:42:35.992 INFO:tasks.workunit.client.0.smithi016.stdout:4/279: write d5/d1d/d1e/f3c [91551,92259] 0 2024-04-02T04:42:35.996 INFO:tasks.workunit.client.0.smithi016.stdout:0/229: mknod d0/d16/d23/c4a 0 2024-04-02T04:42:35.998 INFO:tasks.workunit.client.0.smithi016.stdout:4/280: dwrite d5/d16/f49 [0,4194304] 0 2024-04-02T04:42:35.999 INFO:tasks.workunit.client.0.smithi016.stdout:9/264: truncate d0/f27 304287 0 2024-04-02T04:42:36.000 INFO:tasks.workunit.client.0.smithi016.stdout:6/273: mknod d0/d9/df/c52 0 2024-04-02T04:42:36.001 INFO:tasks.workunit.client.0.smithi016.stdout:6/274: chown d0/d9/d16/f20 184 1 2024-04-02T04:42:36.002 INFO:tasks.workunit.client.0.smithi016.stdout:6/275: chown d0/d9/d16 16590 1 2024-04-02T04:42:36.003 INFO:tasks.workunit.client.0.smithi016.stdout:3/312: creat d2/d15/d16/d3d/d46/f62 x:0 0 0 2024-04-02T04:42:36.010 INFO:tasks.workunit.client.0.smithi016.stdout:0/230: creat d0/d16/d2e/d37/f4b x:0 0 0 2024-04-02T04:42:36.016 INFO:tasks.workunit.client.0.smithi016.stdout:7/262: rename d0/d46/l12 to d0/d46/l63 0 2024-04-02T04:42:36.018 INFO:tasks.workunit.client.0.smithi016.stdout:6/276: symlink d0/d9/df/l53 0 2024-04-02T04:42:36.020 INFO:tasks.workunit.client.0.smithi016.stdout:8/279: symlink d1/l60 0 2024-04-02T04:42:36.022 INFO:tasks.workunit.client.0.smithi016.stdout:5/308: write d3/df/d18/f38 [2515057,8660] 0 2024-04-02T04:42:36.024 INFO:tasks.workunit.client.0.smithi016.stdout:0/231: mknod d0/d29/c4c 0 2024-04-02T04:42:36.025 INFO:tasks.workunit.client.0.smithi016.stdout:5/309: write d3/df/d18/d26/f4b [828018,50337] 0 2024-04-02T04:42:36.026 INFO:tasks.workunit.client.0.smithi016.stdout:2/209: dwrite d12/d14/f37 [4194304,4194304] 0 2024-04-02T04:42:36.027 INFO:tasks.workunit.client.0.smithi016.stdout:7/263: mkdir d0/d28/d64 0 2024-04-02T04:42:36.028 INFO:tasks.workunit.client.0.smithi016.stdout:3/313: symlink d2/d15/d60/l63 0 2024-04-02T04:42:36.030 INFO:tasks.workunit.client.0.smithi016.stdout:6/277: mknod d0/d9/df/c54 0 2024-04-02T04:42:36.031 INFO:tasks.workunit.client.0.smithi016.stdout:3/314: dread d2/f3f [4194304,4194304] 0 2024-04-02T04:42:36.032 INFO:tasks.workunit.client.0.smithi016.stdout:3/315: stat d2/f3e 0 2024-04-02T04:42:36.033 INFO:tasks.workunit.client.0.smithi016.stdout:4/281: rmdir d5/de/d33 0 2024-04-02T04:42:36.033 INFO:tasks.workunit.client.0.smithi016.stdout:8/280: truncate d1/d6/d36/d3f/d40/d49/f43 588678 0 2024-04-02T04:42:36.034 INFO:tasks.workunit.client.0.smithi016.stdout:4/282: write d5/de/f14 [2284646,87555] 0 2024-04-02T04:42:36.037 INFO:tasks.workunit.client.0.smithi016.stdout:0/232: symlink d0/d16/d23/l4d 0 2024-04-02T04:42:36.037 INFO:tasks.workunit.client.0.smithi016.stdout:9/265: chown d0/f27 79069 1 2024-04-02T04:42:36.041 INFO:tasks.workunit.client.0.smithi016.stdout:7/264: fsync d0/d46/d37/d47/f5c 0 2024-04-02T04:42:36.043 INFO:tasks.workunit.client.0.smithi016.stdout:8/281: symlink d1/d6/d36/d3f/d40/d45/l61 0 2024-04-02T04:42:36.045 INFO:tasks.workunit.client.0.smithi016.stdout:8/282: write d1/d6/d36/d3f/d40/d49/f43 [1615572,121743] 0 2024-04-02T04:42:36.047 INFO:tasks.workunit.client.0.smithi016.stdout:0/233: dread d0/d16/d2e/d36/f44 [0,4194304] 0 2024-04-02T04:42:36.050 INFO:tasks.workunit.client.0.smithi016.stdout:2/210: mkdir d12/d35/d3f/d4d 0 2024-04-02T04:42:36.051 INFO:tasks.workunit.client.0.smithi016.stdout:6/278: creat d0/d9/d16/d46/f55 x:0 0 0 2024-04-02T04:42:36.054 INFO:tasks.workunit.client.0.smithi016.stdout:3/316: creat d2/d15/d16/d57/f64 x:0 0 0 2024-04-02T04:42:36.058 INFO:tasks.workunit.client.0.smithi016.stdout:4/283: link d5/f7 d5/d17/d18/d1f/f53 0 2024-04-02T04:42:36.058 INFO:tasks.workunit.client.0.smithi016.stdout:4/284: readlink d5/d16/l2f 0 2024-04-02T04:42:36.060 INFO:tasks.workunit.client.0.smithi016.stdout:9/266: write d0/d3/f30 [2348326,57121] 0 2024-04-02T04:42:36.062 INFO:tasks.workunit.client.0.smithi016.stdout:8/283: unlink d1/d6/d36/c4e 0 2024-04-02T04:42:36.063 INFO:tasks.workunit.client.0.smithi016.stdout:0/234: unlink d0/d16/d24/f1e 0 2024-04-02T04:42:36.065 INFO:tasks.workunit.client.0.smithi016.stdout:2/211: dread d12/d14/f1f [0,4194304] 0 2024-04-02T04:42:36.066 INFO:tasks.workunit.client.0.smithi016.stdout:6/279: rename d0/d9/df/d38 to d0/d9/d16/d56 0 2024-04-02T04:42:36.068 INFO:tasks.workunit.client.0.smithi016.stdout:7/265: link d0/d46/f30 d0/d21/f65 0 2024-04-02T04:42:36.073 INFO:tasks.workunit.client.0.smithi016.stdout:7/266: dwrite d0/d46/f2f [0,4194304] 0 2024-04-02T04:42:36.075 INFO:tasks.workunit.client.0.smithi016.stdout:7/267: write d0/d46/d16/f3d [518607,109622] 0 2024-04-02T04:42:36.075 INFO:tasks.workunit.client.0.smithi016.stdout:0/235: mkdir d0/d16/d24/d4e 0 2024-04-02T04:42:36.079 INFO:tasks.workunit.client.0.smithi016.stdout:2/212: creat d12/d35/d3f/f4e x:0 0 0 2024-04-02T04:42:36.081 INFO:tasks.workunit.client.0.smithi016.stdout:0/236: dread d0/d16/f31 [0,4194304] 0 2024-04-02T04:42:36.083 INFO:tasks.workunit.client.0.smithi016.stdout:4/285: creat d5/d16/d3f/f54 x:0 0 0 2024-04-02T04:42:36.088 INFO:tasks.workunit.client.0.smithi016.stdout:2/213: write d12/d26/f48 [705358,68135] 0 2024-04-02T04:42:36.094 INFO:tasks.workunit.client.0.smithi016.stdout:0/237: dwrite d0/d16/d2e/d36/f44 [0,4194304] 0 2024-04-02T04:42:36.094 INFO:tasks.workunit.client.0.smithi016.stdout:8/284: link d1/d6/d36/d3f/d40/d45/f5b d1/d6/d36/d4f/f62 0 2024-04-02T04:42:36.100 INFO:tasks.workunit.client.0.smithi016.stdout:3/317: link d2/c4c d2/c65 0 2024-04-02T04:42:36.101 INFO:tasks.workunit.client.0.smithi016.stdout:0/238: dwrite d0/f19 [0,4194304] 0 2024-04-02T04:42:36.101 INFO:tasks.workunit.client.0.smithi016.stdout:3/318: dread - d2/d15/f38 zero size 2024-04-02T04:42:36.102 INFO:tasks.workunit.client.0.smithi016.stdout:9/267: write d0/d9/db/f25 [2357135,121045] 0 2024-04-02T04:42:36.103 INFO:tasks.workunit.client.0.smithi016.stdout:4/286: truncate d5/f10 2685810 0 2024-04-02T04:42:36.109 INFO:tasks.workunit.client.0.smithi016.stdout:7/268: link d0/d28/l2d d0/d46/d37/d3a/l66 0 2024-04-02T04:42:36.111 INFO:tasks.workunit.client.0.smithi016.stdout:0/239: mkdir d0/d4f 0 2024-04-02T04:42:36.113 INFO:tasks.workunit.client.0.smithi016.stdout:6/280: dwrite d0/fc [0,4194304] 0 2024-04-02T04:42:36.117 INFO:tasks.workunit.client.0.smithi016.stdout:4/287: creat d5/d16/f55 x:0 0 0 2024-04-02T04:42:36.121 INFO:tasks.workunit.client.0.smithi016.stdout:4/288: dread d5/d17/d18/d25/f4a [0,4194304] 0 2024-04-02T04:42:36.121 INFO:tasks.workunit.client.0.smithi016.stdout:3/319: mkdir d2/d2e/d36/d66 0 2024-04-02T04:42:36.129 INFO:tasks.workunit.client.0.smithi016.stdout:7/269: truncate d0/d21/f65 328688 0 2024-04-02T04:42:36.129 INFO:tasks.workunit.client.0.smithi016.stdout:2/214: dread d12/d35/d3f/f34 [0,4194304] 0 2024-04-02T04:42:36.130 INFO:tasks.workunit.client.0.smithi016.stdout:4/289: creat d5/d1d/d1e/d50/f56 x:0 0 0 2024-04-02T04:42:36.131 INFO:tasks.workunit.client.0.smithi016.stdout:4/290: chown d5/d17/d18/d1f/f53 107764 1 2024-04-02T04:42:36.132 INFO:tasks.workunit.client.0.smithi016.stdout:4/291: readlink d5/d8/lc 0 2024-04-02T04:42:36.132 INFO:tasks.workunit.client.0.smithi016.stdout:3/320: getdents d2/d2e/d36/d5b 0 2024-04-02T04:42:36.132 INFO:tasks.workunit.client.0.smithi016.stdout:4/292: write d5/d1d/d1e/f3c [531178,28030] 0 2024-04-02T04:42:36.133 INFO:tasks.workunit.client.0.smithi016.stdout:3/321: dread - d2/d15/d1a/d45/d56/f5e zero size 2024-04-02T04:42:36.134 INFO:tasks.workunit.client.0.smithi016.stdout:0/240: getdents d0/d16 0 2024-04-02T04:42:36.134 INFO:tasks.workunit.client.0.smithi016.stdout:4/293: stat d5/d17/d18/d1f 0 2024-04-02T04:42:36.135 INFO:tasks.workunit.client.0.smithi016.stdout:7/270: symlink d0/d46/d37/d3a/d4c/l67 0 2024-04-02T04:42:36.135 INFO:tasks.workunit.client.0.smithi016.stdout:7/271: write d0/f22 [3601058,552] 0 2024-04-02T04:42:36.138 INFO:tasks.workunit.client.0.smithi016.stdout:2/215: mkdir d12/d20/d4f 0 2024-04-02T04:42:36.140 INFO:tasks.workunit.client.0.smithi016.stdout:7/272: dread d0/d46/f4d [0,4194304] 0 2024-04-02T04:42:36.141 INFO:tasks.workunit.client.0.smithi016.stdout:7/273: chown d0/d46/d37/d3a/d43/c54 179 1 2024-04-02T04:42:36.145 INFO:tasks.workunit.client.0.smithi016.stdout:8/285: dwrite d1/d6/d36/d4f/f62 [0,4194304] 0 2024-04-02T04:42:36.151 INFO:tasks.workunit.client.0.smithi016.stdout:9/268: dwrite d0/d16/d1a/f48 [0,4194304] 0 2024-04-02T04:42:36.151 INFO:tasks.workunit.client.0.smithi016.stdout:6/281: dread d0/d9/df/f10 [0,4194304] 0 2024-04-02T04:42:36.155 INFO:tasks.workunit.client.0.smithi016.stdout:6/282: dread d0/d9/df/f10 [0,4194304] 0 2024-04-02T04:42:36.156 INFO:tasks.workunit.client.0.smithi016.stdout:4/294: link d5/d1d/d1e/d50/f56 d5/d17/d18/d25/f57 0 2024-04-02T04:42:36.156 INFO:tasks.workunit.client.0.smithi016.stdout:6/283: chown d0/d9/fd 40 1 2024-04-02T04:42:36.157 INFO:tasks.workunit.client.0.smithi016.stdout:8/286: rmdir d1/d6/d36/d3f/d40/d45 39 2024-04-02T04:42:36.158 INFO:tasks.workunit.client.0.smithi016.stdout:0/241: write d0/d16/f34 [3826765,67607] 0 2024-04-02T04:42:36.163 INFO:tasks.workunit.client.0.smithi016.stdout:3/322: creat d2/d2e/f67 x:0 0 0 2024-04-02T04:42:36.165 INFO:tasks.workunit.client.0.smithi016.stdout:7/274: mknod d0/d28/d64/c68 0 2024-04-02T04:42:36.167 INFO:tasks.workunit.client.0.smithi016.stdout:9/269: mkdir d0/d4e/d50 0 2024-04-02T04:42:36.168 INFO:tasks.workunit.client.0.smithi016.stdout:3/323: mknod d2/d15/d1a/d45/c68 0 2024-04-02T04:42:36.171 INFO:tasks.workunit.client.0.smithi016.stdout:9/270: write d0/d3/f40 [2707726,39455] 0 2024-04-02T04:42:36.172 INFO:tasks.workunit.client.0.smithi016.stdout:7/275: rename d0/l38 to d0/d28/d64/l69 0 2024-04-02T04:42:36.175 INFO:tasks.workunit.client.0.smithi016.stdout:9/271: chown d0/d9/d4d 25881419 1 2024-04-02T04:42:36.175 INFO:tasks.workunit.client.0.smithi016.stdout:4/295: dwrite d5/d1d/f42 [0,4194304] 0 2024-04-02T04:42:36.175 INFO:tasks.workunit.client.0.smithi016.stdout:8/287: mkdir d1/d6/d36/d3f/d40/d63 0 2024-04-02T04:42:36.176 INFO:tasks.workunit.client.0.smithi016.stdout:0/242: dread d0/d16/f34 [0,4194304] 0 2024-04-02T04:42:36.177 INFO:tasks.workunit.client.0.smithi016.stdout:9/272: write d0/d16/d1a/f48 [1685594,129235] 0 2024-04-02T04:42:36.177 INFO:tasks.workunit.client.0.smithi016.stdout:2/216: write d12/d35/d3f/f2d [799461,118858] 0 2024-04-02T04:42:36.178 INFO:tasks.workunit.client.0.smithi016.stdout:0/243: read - d0/d16/d2e/d37/f3b zero size 2024-04-02T04:42:36.178 INFO:tasks.workunit.client.0.smithi016.stdout:8/288: truncate d1/d6/f57 499418 0 2024-04-02T04:42:36.179 INFO:tasks.workunit.client.0.smithi016.stdout:2/217: stat d12/d35/d3f/d2b 0 2024-04-02T04:42:36.179 INFO:tasks.workunit.client.0.smithi016.stdout:3/324: creat d2/d15/d1a/f69 x:0 0 0 2024-04-02T04:42:36.179 INFO:tasks.workunit.client.0.smithi016.stdout:3/325: readlink d2/d15/d16/l2b 0 2024-04-02T04:42:36.180 INFO:tasks.workunit.client.0.smithi016.stdout:3/326: chown d2/d2e/d36 1 1 2024-04-02T04:42:36.186 INFO:tasks.workunit.client.0.smithi016.stdout:3/327: dwrite d2/d15/d16/d57/f64 [0,4194304] 0 2024-04-02T04:42:36.188 INFO:tasks.workunit.client.0.smithi016.stdout:4/296: mkdir d5/d17/d18/d1f/d58 0 2024-04-02T04:42:36.196 INFO:tasks.workunit.client.0.smithi016.stdout:3/328: dwrite d2/d15/d1a/f69 [0,4194304] 0 2024-04-02T04:42:36.198 INFO:tasks.workunit.client.0.smithi016.stdout:2/218: symlink d12/d26/l50 0 2024-04-02T04:42:36.202 INFO:tasks.workunit.client.0.smithi016.stdout:2/219: write d12/d20/f44 [65329,9401] 0 2024-04-02T04:42:36.204 INFO:tasks.workunit.client.0.smithi016.stdout:2/220: write d12/d14/f3c [135434,125814] 0 2024-04-02T04:42:36.205 INFO:tasks.workunit.client.0.smithi016.stdout:0/244: symlink d0/d48/l50 0 2024-04-02T04:42:36.205 INFO:tasks.workunit.client.0.smithi016.stdout:6/284: dwrite d0/d9/d16/f2a [0,4194304] 0 2024-04-02T04:42:36.209 INFO:tasks.workunit.client.0.smithi016.stdout:0/245: chown d0/d29/l2f 312431099 1 2024-04-02T04:42:36.209 INFO:tasks.workunit.client.0.smithi016.stdout:3/329: unlink d2/d15/d16/l49 0 2024-04-02T04:42:36.210 INFO:tasks.workunit.client.0.smithi016.stdout:3/330: stat d2/d2e/f67 0 2024-04-02T04:42:36.212 INFO:tasks.workunit.client.0.smithi016.stdout:3/331: chown d2/d15/d1a/d45/d56/f5e 935 1 2024-04-02T04:42:36.214 INFO:tasks.workunit.client.0.smithi016.stdout:2/221: mknod d12/d26/c51 0 2024-04-02T04:42:36.215 INFO:tasks.workunit.client.0.smithi016.stdout:6/285: dwrite d0/d9/f32 [0,4194304] 0 2024-04-02T04:42:36.215 INFO:tasks.workunit.client.0.smithi016.stdout:3/332: read d2/d15/d16/f25 [110243,25166] 0 2024-04-02T04:42:36.215 INFO:tasks.workunit.client.0.smithi016.stdout:0/246: symlink d0/d16/d2e/d37/l51 0 2024-04-02T04:42:36.216 INFO:tasks.workunit.client.0.smithi016.stdout:6/286: truncate d0/f28 811062 0 2024-04-02T04:42:36.221 INFO:tasks.workunit.client.0.smithi016.stdout:0/247: creat d0/d29/f52 x:0 0 0 2024-04-02T04:42:36.221 INFO:tasks.workunit.client.0.smithi016.stdout:9/273: write d0/d9/f2e [356160,38257] 0 2024-04-02T04:42:36.224 INFO:tasks.workunit.client.0.smithi016.stdout:3/333: rename d2/d15/d1a/d45/c52 to d2/d15/d1a/d45/c6a 0 2024-04-02T04:42:36.226 INFO:tasks.workunit.client.0.smithi016.stdout:6/287: mknod d0/d9/d16/d46/c57 0 2024-04-02T04:42:36.230 INFO:tasks.workunit.client.0.smithi016.stdout:6/288: dread d0/d9/d16/f2a [0,4194304] 0 2024-04-02T04:42:36.231 INFO:tasks.workunit.client.0.smithi016.stdout:6/289: dread - d0/f41 zero size 2024-04-02T04:42:36.231 INFO:tasks.workunit.client.0.smithi016.stdout:3/334: dread d2/d15/d16/f25 [0,4194304] 0 2024-04-02T04:42:36.234 INFO:tasks.workunit.client.0.smithi016.stdout:7/276: dwrite d0/d21/f65 [0,4194304] 0 2024-04-02T04:42:36.236 INFO:tasks.workunit.client.0.smithi016.stdout:8/289: truncate d1/f1c 11647431 0 2024-04-02T04:42:36.237 INFO:tasks.workunit.client.0.smithi016.stdout:8/290: read - d1/f5c zero size 2024-04-02T04:42:36.238 INFO:tasks.workunit.client.0.smithi016.stdout:2/222: rmdir d12/d26 39 2024-04-02T04:42:36.239 INFO:tasks.workunit.client.0.smithi016.stdout:6/290: dwrite d0/f3f [0,4194304] 0 2024-04-02T04:42:36.239 INFO:tasks.workunit.client.0.smithi016.stdout:6/291: chown d0 60447214 1 2024-04-02T04:42:36.247 INFO:tasks.workunit.client.0.smithi016.stdout:0/248: rename d0/d29 to d0/d16/d2e/d53 0 2024-04-02T04:42:36.260 INFO:tasks.workunit.client.0.smithi016.stdout:3/335: write d2/f3 [3926124,49336] 0 2024-04-02T04:42:36.261 INFO:tasks.workunit.client.0.smithi016.stdout:6/292: unlink d0/c27 0 2024-04-02T04:42:36.262 INFO:tasks.workunit.client.0.smithi016.stdout:1/271: sync 2024-04-02T04:42:36.262 INFO:tasks.workunit.client.0.smithi016.stdout:7/277: dwrite d0/d46/f30 [4194304,4194304] 0 2024-04-02T04:42:36.263 INFO:tasks.workunit.client.0.smithi016.stdout:6/293: write d0/d9/df/f1b [1670297,69854] 0 2024-04-02T04:42:36.264 INFO:tasks.workunit.client.0.smithi016.stdout:0/249: creat d0/d16/f54 x:0 0 0 2024-04-02T04:42:36.265 INFO:tasks.workunit.client.0.smithi016.stdout:9/274: getdents d0/d16/d1a 0 2024-04-02T04:42:36.268 INFO:tasks.workunit.client.0.smithi016.stdout:9/275: write d0/d33/f36 [20417,27015] 0 2024-04-02T04:42:36.275 INFO:tasks.workunit.client.0.smithi016.stdout:1/272: mknod d14/d19/d1e/c53 0 2024-04-02T04:42:36.275 INFO:tasks.workunit.client.0.smithi016.stdout:0/250: dwrite d0/fc [0,4194304] 0 2024-04-02T04:42:36.276 INFO:tasks.workunit.client.0.smithi016.stdout:6/294: mknod d0/d9/d16/c58 0 2024-04-02T04:42:36.276 INFO:tasks.workunit.client.0.smithi016.stdout:7/278: creat d0/d46/d52/f6a x:0 0 0 2024-04-02T04:42:36.276 INFO:tasks.workunit.client.0.smithi016.stdout:2/223: creat d12/d20/f52 x:0 0 0 2024-04-02T04:42:36.278 INFO:tasks.workunit.client.0.smithi016.stdout:1/273: read d14/f28 [3480954,70306] 0 2024-04-02T04:42:36.279 INFO:tasks.workunit.client.0.smithi016.stdout:3/336: fsync d2/d15/d16/f54 0 2024-04-02T04:42:36.280 INFO:tasks.workunit.client.0.smithi016.stdout:1/274: read fb [2020811,68953] 0 2024-04-02T04:42:36.284 INFO:tasks.workunit.client.0.smithi016.stdout:0/251: rmdir d0/d16 39 2024-04-02T04:42:36.285 INFO:tasks.workunit.client.0.smithi016.stdout:2/224: chown d12/c15 16501919 1 2024-04-02T04:42:36.286 INFO:tasks.workunit.client.0.smithi016.stdout:1/275: dwrite d14/f1c [0,4194304] 0 2024-04-02T04:42:36.288 INFO:tasks.workunit.client.0.smithi016.stdout:8/291: link d1/d6/c13 d1/d6/d36/c64 0 2024-04-02T04:42:36.288 INFO:tasks.workunit.client.0.smithi016.stdout:1/276: read - d14/d41/d43/f49 zero size 2024-04-02T04:42:36.289 INFO:tasks.workunit.client.0.smithi016.stdout:9/276: mknod d0/d16/d1a/d46/c51 0 2024-04-02T04:42:36.289 INFO:tasks.workunit.client.0.smithi016.stdout:7/279: stat d0/d46/c26 0 2024-04-02T04:42:36.290 INFO:tasks.workunit.client.0.smithi016.stdout:7/280: chown d0/d46/c26 3 1 2024-04-02T04:42:36.291 INFO:tasks.workunit.client.0.smithi016.stdout:9/277: dread d0/d9/f3d [0,4194304] 0 2024-04-02T04:42:36.292 INFO:tasks.workunit.client.0.smithi016.stdout:9/278: fdatasync d0/d33/f36 0 2024-04-02T04:42:36.293 INFO:tasks.workunit.client.0.smithi016.stdout:6/295: rename d0/d1f/d35 to d0/d9/d16/d56/d59 0 2024-04-02T04:42:36.293 INFO:tasks.workunit.client.0.smithi016.stdout:9/279: chown d0/d9/db/f1c 2008742 1 2024-04-02T04:42:36.296 INFO:tasks.workunit.client.0.smithi016.stdout:9/280: read d0/d9/db/f25 [1767943,57983] 0 2024-04-02T04:42:36.301 INFO:tasks.workunit.client.0.smithi016.stdout:5/310: sync 2024-04-02T04:42:36.301 INFO:tasks.workunit.client.0.smithi016.stdout:4/297: sync 2024-04-02T04:42:36.303 INFO:tasks.workunit.client.0.smithi016.stdout:7/281: write d0/d46/f1d [7092479,5225] 0 2024-04-02T04:42:36.303 INFO:tasks.workunit.client.0.smithi016.stdout:6/296: mknod d0/d9/d16/c5a 0 2024-04-02T04:42:36.305 INFO:tasks.workunit.client.0.smithi016.stdout:0/252: dwrite d0/d16/d2e/d37/f4b [0,4194304] 0 2024-04-02T04:42:36.306 INFO:tasks.workunit.client.0.smithi016.stdout:6/297: write d0/d9/d16/d56/f2c [3492809,117590] 0 2024-04-02T04:42:36.307 INFO:tasks.workunit.client.0.smithi016.stdout:9/281: symlink d0/d4a/l52 0 2024-04-02T04:42:36.309 INFO:tasks.workunit.client.0.smithi016.stdout:2/225: creat d12/d20/d4f/f53 x:0 0 0 2024-04-02T04:42:36.310 INFO:tasks.workunit.client.0.smithi016.stdout:5/311: dread d3/df/d18/d26/f4b [0,4194304] 0 2024-04-02T04:42:36.310 INFO:tasks.workunit.client.0.smithi016.stdout:8/292: mknod d1/d6/d36/c65 0 2024-04-02T04:42:36.310 INFO:tasks.workunit.client.0.smithi016.stdout:6/298: chown d0/d9/d16/d56/c3d 144 1 2024-04-02T04:42:36.314 INFO:tasks.workunit.client.0.smithi016.stdout:0/253: dwrite d0/f32 [0,4194304] 0 2024-04-02T04:42:36.320 INFO:tasks.workunit.client.0.smithi016.stdout:3/337: truncate d2/f14 3654410 0 2024-04-02T04:42:36.323 INFO:tasks.workunit.client.0.smithi016.stdout:7/282: creat d0/d46/d37/d47/f6b x:0 0 0 2024-04-02T04:42:36.323 INFO:tasks.workunit.client.0.smithi016.stdout:9/282: dread - d0/d9/f42 zero size 2024-04-02T04:42:36.324 INFO:tasks.workunit.client.0.smithi016.stdout:7/283: dread - d0/d21/f36 zero size 2024-04-02T04:42:36.330 INFO:tasks.workunit.client.0.smithi016.stdout:1/277: truncate d14/d19/d1e/f42 135749 0 2024-04-02T04:42:36.330 INFO:tasks.workunit.client.0.smithi016.stdout:1/278: readlink d14/l2d 0 2024-04-02T04:42:36.332 INFO:tasks.workunit.client.0.smithi016.stdout:6/299: rename d0/f28 to d0/d9/d16/f5b 0 2024-04-02T04:42:36.332 INFO:tasks.workunit.client.0.smithi016.stdout:6/300: chown d0/d9/f36 434696 1 2024-04-02T04:42:36.333 INFO:tasks.workunit.client.0.smithi016.stdout:1/279: read d14/d19/d1d/f30 [1450914,91857] 0 2024-04-02T04:42:36.333 INFO:tasks.workunit.client.0.smithi016.stdout:1/280: fdatasync d14/d19/f4b 0 2024-04-02T04:42:36.335 INFO:tasks.workunit.client.0.smithi016.stdout:0/254: symlink d0/d16/l55 0 2024-04-02T04:42:36.336 INFO:tasks.workunit.client.0.smithi016.stdout:1/281: dread d14/d19/d20/f3f [0,4194304] 0 2024-04-02T04:42:36.339 INFO:tasks.workunit.client.0.smithi016.stdout:2/226: creat d12/d35/d3b/f54 x:0 0 0 2024-04-02T04:42:36.340 INFO:tasks.workunit.client.0.smithi016.stdout:3/338: dwrite d2/d15/d16/d57/f64 [0,4194304] 0 2024-04-02T04:42:36.341 INFO:tasks.workunit.client.0.smithi016.stdout:3/339: readlink d2/d15/d1a/l40 0 2024-04-02T04:42:36.342 INFO:tasks.workunit.client.0.smithi016.stdout:8/293: truncate d1/f15 5180496 0 2024-04-02T04:42:36.345 INFO:tasks.workunit.client.0.smithi016.stdout:8/294: write d1/d6/d36/d3f/d40/d49/d4a/f5a [23491,124187] 0 2024-04-02T04:42:36.348 INFO:tasks.workunit.client.0.smithi016.stdout:5/312: dwrite d3/d40/d4f/d6b/f47 [0,4194304] 0 2024-04-02T04:42:36.350 INFO:tasks.workunit.client.0.smithi016.stdout:5/313: write d3/d5d/f69 [2638626,25213] 0 2024-04-02T04:42:36.359 INFO:tasks.workunit.client.0.smithi016.stdout:1/282: chown d14/d19/d1e/f2e 1 1 2024-04-02T04:42:36.368 INFO:tasks.workunit.client.0.smithi016.stdout:6/301: dwrite d0/f4 [0,4194304] 0 2024-04-02T04:42:36.371 INFO:tasks.workunit.client.0.smithi016.stdout:9/283: truncate d0/d16/d1a/f48 1860738 0 2024-04-02T04:42:36.372 INFO:tasks.workunit.client.0.smithi016.stdout:9/284: chown d0/d9/f2e 3 1 2024-04-02T04:42:36.373 INFO:tasks.workunit.client.0.smithi016.stdout:9/285: write d0/d33/f38 [2588563,57560] 0 2024-04-02T04:42:36.376 INFO:tasks.workunit.client.0.smithi016.stdout:1/283: symlink d14/d41/l54 0 2024-04-02T04:42:36.383 INFO:tasks.workunit.client.0.smithi016.stdout:5/314: link d3/df/f22 d3/d40/d72/f7d 0 2024-04-02T04:42:36.385 INFO:tasks.workunit.client.0.smithi016.stdout:5/315: read d3/df/f1f [3185455,17949] 0 2024-04-02T04:42:36.385 INFO:tasks.workunit.client.0.smithi016.stdout:5/316: readlink d3/df/d18/d2f/l53 0 2024-04-02T04:42:36.386 INFO:tasks.workunit.client.0.smithi016.stdout:0/255: truncate d0/fc 1276444 0 2024-04-02T04:42:36.387 INFO:tasks.workunit.client.0.smithi016.stdout:9/286: dread d0/d9/f3d [0,4194304] 0 2024-04-02T04:42:36.389 INFO:tasks.workunit.client.0.smithi016.stdout:7/284: getdents d0/d46/d16/d49 0 2024-04-02T04:42:36.390 INFO:tasks.workunit.client.0.smithi016.stdout:7/285: write d0/d46/f30 [3216005,8097] 0 2024-04-02T04:42:36.391 INFO:tasks.workunit.client.0.smithi016.stdout:7/286: write d0/d21/f3c [647335,45181] 0 2024-04-02T04:42:36.395 INFO:tasks.workunit.client.0.smithi016.stdout:3/340: dwrite d2/d15/d1a/f39 [0,4194304] 0 2024-04-02T04:42:36.396 INFO:tasks.workunit.client.0.smithi016.stdout:5/317: mknod d3/d40/d4f/d6b/d63/c7e 0 2024-04-02T04:42:36.397 INFO:tasks.workunit.client.0.smithi016.stdout:0/256: mknod d0/d16/d2e/d37/c56 0 2024-04-02T04:42:36.397 INFO:tasks.workunit.client.0.smithi016.stdout:5/318: chown d3/df/l56 710 1 2024-04-02T04:42:36.399 INFO:tasks.workunit.client.0.smithi016.stdout:5/319: fdatasync d3/f10 0 2024-04-02T04:42:36.401 INFO:tasks.workunit.client.0.smithi016.stdout:5/320: fdatasync d3/df/f1e 0 2024-04-02T04:42:36.401 INFO:tasks.workunit.client.0.smithi016.stdout:6/302: dwrite d0/d9/d16/d56/d59/f4b [0,4194304] 0 2024-04-02T04:42:36.403 INFO:tasks.workunit.client.0.smithi016.stdout:2/227: getdents d12/d20 0 2024-04-02T04:42:36.404 INFO:tasks.workunit.client.0.smithi016.stdout:6/303: readlink d0/d9/df/l53 0 2024-04-02T04:42:36.407 INFO:tasks.workunit.client.0.smithi016.stdout:2/228: dread - d12/d35/d3f/d2b/f3d zero size 2024-04-02T04:42:36.407 INFO:tasks.workunit.client.0.smithi016.stdout:6/304: readlink d0/d9/d16/l39 0 2024-04-02T04:42:36.407 INFO:tasks.workunit.client.0.smithi016.stdout:1/284: dwrite f7 [0,4194304] 0 2024-04-02T04:42:36.408 INFO:tasks.workunit.client.0.smithi016.stdout:7/287: chown d0/d46/d16/f34 196124 1 2024-04-02T04:42:36.408 INFO:tasks.workunit.client.0.smithi016.stdout:2/229: dread - d12/d35/d3f/f4b zero size 2024-04-02T04:42:36.410 INFO:tasks.workunit.client.0.smithi016.stdout:6/305: fdatasync d0/d9/d16/d56/f43 0 2024-04-02T04:42:36.411 INFO:tasks.workunit.client.0.smithi016.stdout:6/306: chown d0/f40 111556338 1 2024-04-02T04:42:36.412 INFO:tasks.workunit.client.0.smithi016.stdout:3/341: write d2/d15/d16/d3d/f4e [692344,61112] 0 2024-04-02T04:42:36.415 INFO:tasks.workunit.client.0.smithi016.stdout:5/321: truncate d3/df/d18/d26/f4b 1746175 0 2024-04-02T04:42:36.415 INFO:tasks.workunit.client.0.smithi016.stdout:5/322: stat d3/df/d18/d2f/l53 0 2024-04-02T04:42:36.416 INFO:tasks.workunit.client.0.smithi016.stdout:5/323: stat d3/f5 0 2024-04-02T04:42:36.416 INFO:tasks.workunit.client.0.smithi016.stdout:2/230: chown d12/d26/f2c 10 1 2024-04-02T04:42:36.417 INFO:tasks.workunit.client.0.smithi016.stdout:1/285: rename d14/c52 to d14/d41/c55 0 2024-04-02T04:42:36.417 INFO:tasks.workunit.client.0.smithi016.stdout:2/231: read - d12/d20/d4f/f53 zero size 2024-04-02T04:42:36.418 INFO:tasks.workunit.client.0.smithi016.stdout:1/286: readlink d14/d41/l54 0 2024-04-02T04:42:36.418 INFO:tasks.workunit.client.0.smithi016.stdout:7/288: mknod d0/d46/d37/d47/c6c 0 2024-04-02T04:42:36.419 INFO:tasks.workunit.client.0.smithi016.stdout:0/257: getdents d0/d16/d24/d4e 0 2024-04-02T04:42:36.421 INFO:tasks.workunit.client.0.smithi016.stdout:7/289: stat d0/d46/d52/f6a 0 2024-04-02T04:42:36.421 INFO:tasks.workunit.client.0.smithi016.stdout:2/232: stat d12/f43 0 2024-04-02T04:42:36.422 INFO:tasks.workunit.client.0.smithi016.stdout:0/258: dread - d0/d16/d2e/d37/f3b zero size 2024-04-02T04:42:36.423 INFO:tasks.workunit.client.0.smithi016.stdout:6/307: mkdir d0/d5c 0 2024-04-02T04:42:36.424 INFO:tasks.workunit.client.0.smithi016.stdout:5/324: dwrite d3/d40/f60 [0,4194304] 0 2024-04-02T04:42:36.425 INFO:tasks.workunit.client.0.smithi016.stdout:6/308: dread - d0/d9/d16/f3c zero size 2024-04-02T04:42:36.426 INFO:tasks.workunit.client.0.smithi016.stdout:5/325: readlink d3/df/d18/d2f/l53 0 2024-04-02T04:42:36.427 INFO:tasks.workunit.client.0.smithi016.stdout:6/309: chown d0/d1f/f26 959152 1 2024-04-02T04:42:36.427 INFO:tasks.workunit.client.0.smithi016.stdout:9/287: truncate d0/d3/f40 2689103 0 2024-04-02T04:42:36.429 INFO:tasks.workunit.client.0.smithi016.stdout:0/259: mkdir d0/d16/d23/d3e/d57 0 2024-04-02T04:42:36.429 INFO:tasks.workunit.client.0.smithi016.stdout:6/310: fdatasync d0/d1f/f42 0 2024-04-02T04:42:36.431 INFO:tasks.workunit.client.0.smithi016.stdout:5/326: read d3/df/f33 [396579,101110] 0 2024-04-02T04:42:36.431 INFO:tasks.workunit.client.0.smithi016.stdout:9/288: fsync d0/d9/f19 0 2024-04-02T04:42:36.433 INFO:tasks.workunit.client.0.smithi016.stdout:7/290: dwrite d0/d46/d52/f61 [0,4194304] 0 2024-04-02T04:42:36.436 INFO:tasks.workunit.client.0.smithi016.stdout:0/260: creat d0/d48/f58 x:0 0 0 2024-04-02T04:42:36.438 INFO:tasks.workunit.client.0.smithi016.stdout:9/289: rmdir d0/d16/d1a 39 2024-04-02T04:42:36.439 INFO:tasks.workunit.client.0.smithi016.stdout:9/290: dread - d0/d33/f35 zero size 2024-04-02T04:42:36.442 INFO:tasks.workunit.client.0.smithi016.stdout:3/342: write d2/d15/d16/f27 [662314,81928] 0 2024-04-02T04:42:36.443 INFO:tasks.workunit.client.0.smithi016.stdout:3/343: chown d2/d2e/d36/d48 41812159 1 2024-04-02T04:42:36.450 INFO:tasks.workunit.client.0.smithi016.stdout:7/291: mknod d0/d28/c6d 0 2024-04-02T04:42:36.450 INFO:tasks.workunit.client.0.smithi016.stdout:1/287: truncate d14/d19/d1e/f23 543026 0 2024-04-02T04:42:36.452 INFO:tasks.workunit.client.0.smithi016.stdout:9/291: mkdir d0/d3/d31/d53 0 2024-04-02T04:42:36.453 INFO:tasks.workunit.client.0.smithi016.stdout:9/292: write d0/d9/f2e [1103123,115951] 0 2024-04-02T04:42:36.454 INFO:tasks.workunit.client.0.smithi016.stdout:2/233: link d12/d35/c36 d12/d14/c55 0 2024-04-02T04:42:36.454 INFO:tasks.workunit.client.0.smithi016.stdout:2/234: chown d12/d35/d3f/d2b 30 1 2024-04-02T04:42:36.456 INFO:tasks.workunit.client.0.smithi016.stdout:6/311: creat d0/f5d x:0 0 0 2024-04-02T04:42:36.457 INFO:tasks.workunit.client.0.smithi016.stdout:6/312: chown d0/d9/d16/f44 0 1 2024-04-02T04:42:36.458 INFO:tasks.workunit.client.0.smithi016.stdout:1/288: dread d14/f34 [0,4194304] 0 2024-04-02T04:42:36.460 INFO:tasks.workunit.client.0.smithi016.stdout:1/289: fdatasync d14/d19/d20/f38 0 2024-04-02T04:42:36.463 INFO:tasks.workunit.client.0.smithi016.stdout:5/327: link d3/f36 d3/f7f 0 2024-04-02T04:42:36.466 INFO:tasks.workunit.client.0.smithi016.stdout:1/290: dwrite d14/f48 [0,4194304] 0 2024-04-02T04:42:36.470 INFO:tasks.workunit.client.0.smithi016.stdout:3/344: mknod d2/d2e/d36/d48/c6b 0 2024-04-02T04:42:36.471 INFO:tasks.workunit.client.0.smithi016.stdout:7/292: dread d0/d46/d16/d49/f11 [0,4194304] 0 2024-04-02T04:42:36.474 INFO:tasks.workunit.client.0.smithi016.stdout:2/235: write d12/d14/f1e [1264953,44368] 0 2024-04-02T04:42:36.474 INFO:tasks.workunit.client.0.smithi016.stdout:6/313: mkdir d0/d9/d16/d46/d5e 0 2024-04-02T04:42:36.477 INFO:tasks.workunit.client.0.smithi016.stdout:9/293: dwrite d0/d16/d1a/f1f [0,4194304] 0 2024-04-02T04:42:36.479 INFO:tasks.workunit.client.0.smithi016.stdout:9/294: write d0/f2c [1466764,118530] 0 2024-04-02T04:42:36.484 INFO:tasks.workunit.client.0.smithi016.stdout:0/261: link d0/fe d0/d16/d23/d3e/d57/f59 0 2024-04-02T04:42:36.484 INFO:tasks.workunit.client.0.smithi016.stdout:0/262: chown d0/d16/l28 1791973145 1 2024-04-02T04:42:36.494 INFO:tasks.workunit.client.0.smithi016.stdout:3/345: creat d2/d2e/d36/d48/f6c x:0 0 0 2024-04-02T04:42:36.499 INFO:tasks.workunit.client.0.smithi016.stdout:2/236: mkdir d12/d35/d3f/d56 0 2024-04-02T04:42:36.506 INFO:tasks.workunit.client.0.smithi016.stdout:7/293: dwrite d0/d46/f4d [0,4194304] 0 2024-04-02T04:42:36.508 INFO:tasks.workunit.client.0.smithi016.stdout:0/263: rename d0/d16/d2e/d36/f41 to d0/d16/d23/f5a 0 2024-04-02T04:42:36.513 INFO:tasks.workunit.client.0.smithi016.stdout:2/237: chown d12/d14/c55 118 1 2024-04-02T04:42:36.514 INFO:tasks.workunit.client.0.smithi016.stdout:6/314: truncate d0/d9/d16/d56/f25 4447829 0 2024-04-02T04:42:36.517 INFO:tasks.workunit.client.0.smithi016.stdout:7/294: chown d0/d46/d52/c60 136 1 2024-04-02T04:42:36.517 INFO:tasks.workunit.client.0.smithi016.stdout:9/295: dread d0/d9/db/f25 [0,4194304] 0 2024-04-02T04:42:36.518 INFO:tasks.workunit.client.0.smithi016.stdout:3/346: rename d2/d2e/d36/d48/f6c to d2/d15/d1a/d45/f6d 0 2024-04-02T04:42:36.520 INFO:tasks.workunit.client.0.smithi016.stdout:7/295: dread d0/d46/f1d [4194304,4194304] 0 2024-04-02T04:42:36.520 INFO:tasks.workunit.client.0.smithi016.stdout:5/328: link d3/df/d18/d26/d48/f49 d3/df/f80 0 2024-04-02T04:42:36.521 INFO:tasks.workunit.client.0.smithi016.stdout:5/329: chown d3/df/l27 20920 1 2024-04-02T04:42:36.523 INFO:tasks.workunit.client.0.smithi016.stdout:1/291: creat d14/f56 x:0 0 0 2024-04-02T04:42:36.523 INFO:tasks.workunit.client.0.smithi016.stdout:1/292: fsync d14/d19/f4b 0 2024-04-02T04:42:36.523 INFO:tasks.workunit.client.0.smithi016.stdout:2/238: creat d12/d35/d3b/f57 x:0 0 0 2024-04-02T04:42:36.525 INFO:tasks.workunit.client.0.smithi016.stdout:4/298: sync 2024-04-02T04:42:36.525 INFO:tasks.workunit.client.0.smithi016.stdout:4/299: fsync d5/d17/d18/d1f/f40 0 2024-04-02T04:42:36.526 INFO:tasks.workunit.client.0.smithi016.stdout:4/300: dread - d5/d17/d18/d1f/f40 zero size 2024-04-02T04:42:36.527 INFO:tasks.workunit.client.0.smithi016.stdout:9/296: mkdir d0/d9/db/d54 0 2024-04-02T04:42:36.527 INFO:tasks.workunit.client.0.smithi016.stdout:9/297: write d0/f2c [1973022,37399] 0 2024-04-02T04:42:36.528 INFO:tasks.workunit.client.0.smithi016.stdout:4/301: write d5/d17/d18/f2b [706213,1884] 0 2024-04-02T04:42:36.529 INFO:tasks.workunit.client.0.smithi016.stdout:9/298: truncate d0/d33/f36 261614 0 2024-04-02T04:42:36.530 INFO:tasks.workunit.client.0.smithi016.stdout:7/296: creat d0/d46/d16/d49/f6e x:0 0 0 2024-04-02T04:42:36.530 INFO:tasks.workunit.client.0.smithi016.stdout:0/264: creat d0/d16/d23/d3e/d57/f5b x:0 0 0 2024-04-02T04:42:36.531 INFO:tasks.workunit.client.0.smithi016.stdout:0/265: dread - d0/d16/d2e/d36/f3d zero size 2024-04-02T04:42:36.532 INFO:tasks.workunit.client.0.smithi016.stdout:0/266: chown d0/d16/d23/l4d 73579388 1 2024-04-02T04:42:36.536 INFO:tasks.workunit.client.0.smithi016.stdout:2/239: rename d12/d26/c45 to d12/d35/d3b/d42/c58 0 2024-04-02T04:42:36.546 INFO:tasks.workunit.client.0.smithi016.stdout:5/330: dread d3/f3b [0,4194304] 0 2024-04-02T04:42:36.547 INFO:tasks.workunit.client.0.smithi016.stdout:1/293: dread d14/d19/d20/f38 [0,4194304] 0 2024-04-02T04:42:36.550 INFO:tasks.workunit.client.0.smithi016.stdout:7/297: mkdir d0/d46/d37/d3a/d6f 0 2024-04-02T04:42:36.551 INFO:tasks.workunit.client.0.smithi016.stdout:3/347: fsync d2/d15/d1a/d45/f6d 0 2024-04-02T04:42:36.552 INFO:tasks.workunit.client.0.smithi016.stdout:3/348: dread - d2/d15/f4a zero size 2024-04-02T04:42:36.553 INFO:tasks.workunit.client.0.smithi016.stdout:3/349: chown d2/d2e/d36/c3b 17314 1 2024-04-02T04:42:36.553 INFO:tasks.workunit.client.0.smithi016.stdout:0/267: mknod d0/d16/d2e/d36/c5c 0 2024-04-02T04:42:36.554 INFO:tasks.workunit.client.0.smithi016.stdout:3/350: fsync d2/d15/d1a/f29 0 2024-04-02T04:42:36.554 INFO:tasks.workunit.client.0.smithi016.stdout:3/351: chown d2/f9 262 1 2024-04-02T04:42:36.556 INFO:tasks.workunit.client.0.smithi016.stdout:2/240: rmdir d12/d20 39 2024-04-02T04:42:36.557 INFO:tasks.workunit.client.0.smithi016.stdout:9/299: dread d0/f2d [0,4194304] 0 2024-04-02T04:42:36.558 INFO:tasks.workunit.client.0.smithi016.stdout:9/300: read - d0/d16/f44 zero size 2024-04-02T04:42:36.565 INFO:tasks.workunit.client.0.smithi016.stdout:0/268: fsync d0/d16/d23/d3e/d57/f59 0 2024-04-02T04:42:36.567 INFO:tasks.workunit.client.0.smithi016.stdout:3/352: mkdir d2/d15/d16/d3d/d6e 0 2024-04-02T04:42:36.570 INFO:tasks.workunit.client.0.smithi016.stdout:5/331: mknod d3/d40/d72/c81 0 2024-04-02T04:42:36.571 INFO:tasks.workunit.client.0.smithi016.stdout:6/315: truncate d0/d9/d16/f5b 247260 0 2024-04-02T04:42:36.574 INFO:tasks.workunit.client.0.smithi016.stdout:8/295: sync 2024-04-02T04:42:36.574 INFO:tasks.workunit.client.0.smithi016.stdout:1/294: symlink d14/l57 0 2024-04-02T04:42:36.575 INFO:tasks.workunit.client.0.smithi016.stdout:1/295: write d14/f48 [4741095,67621] 0 2024-04-02T04:42:36.576 INFO:tasks.workunit.client.0.smithi016.stdout:7/298: link d0/d46/d37/d47/f5c d0/d46/d16/d49/f70 0 2024-04-02T04:42:36.581 INFO:tasks.workunit.client.0.smithi016.stdout:2/241: rmdir d12/d20 39 2024-04-02T04:42:36.582 INFO:tasks.workunit.client.0.smithi016.stdout:5/332: dread d3/d40/d4f/f75 [0,4194304] 0 2024-04-02T04:42:36.582 INFO:tasks.workunit.client.0.smithi016.stdout:7/299: dwrite d0/d46/d16/f41 [0,4194304] 0 2024-04-02T04:42:36.583 INFO:tasks.workunit.client.0.smithi016.stdout:5/333: truncate d3/df/d18/d26/f7a 907419 0 2024-04-02T04:42:36.586 INFO:tasks.workunit.client.0.smithi016.stdout:9/301: truncate d0/d9/f14 3007366 0 2024-04-02T04:42:36.591 INFO:tasks.workunit.client.0.smithi016.stdout:4/302: truncate d5/d1d/f21 3315408 0 2024-04-02T04:42:36.597 INFO:tasks.workunit.client.0.smithi016.stdout:1/296: read d14/d19/d1e/f42 [30435,72515] 0 2024-04-02T04:42:36.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.598+0000 7fb4b1fbd700 1 -- 172.21.15.16:0/2650491127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4ac0ffc10 msgr2=0x7fb4ac100090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:36.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.598+0000 7fb4b1fbd700 1 --2- 172.21.15.16:0/2650491127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4ac0ffc10 0x7fb4ac100090 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0009910 tx=0x7fb4a0009c20 comp rx=0 tx=0).stop 2024-04-02T04:42:36.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.598+0000 7fb4b1fbd700 1 -- 172.21.15.16:0/2650491127 shutdown_connections 2024-04-02T04:42:36.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.598+0000 7fb4b1fbd700 1 --2- 172.21.15.16:0/2650491127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4ac0ffc10 0x7fb4ac100090 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:36.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.598+0000 7fb4b1fbd700 1 --2- 172.21.15.16:0/2650491127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4ac100fb0 0x7fb4ac1013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:36.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.598+0000 7fb4b1fbd700 1 -- 172.21.15.16:0/2650491127 >> 172.21.15.16:0/2650491127 conn(0x7fb4ac0fa700 msgr2=0x7fb4ac0fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:42:36.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.598+0000 7fb4b1fbd700 1 -- 172.21.15.16:0/2650491127 shutdown_connections 2024-04-02T04:42:36.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.598+0000 7fb4b1fbd700 1 -- 172.21.15.16:0/2650491127 wait complete. 2024-04-02T04:42:36.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.599+0000 7fb4b1fbd700 1 Processor -- start 2024-04-02T04:42:36.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.599+0000 7fb4b1fbd700 1 -- start start 2024-04-02T04:42:36.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.599+0000 7fb4b1fbd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4ac0ffc10 0x7fb4ac0ff610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:36.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.599+0000 7fb4b1fbd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4ac100fb0 0x7fb4ac19b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:36.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.599+0000 7fb4b1fbd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ac19b850 con 0x7fb4ac0ffc10 2024-04-02T04:42:36.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.599+0000 7fb4b1fbd700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ac19b9c0 con 0x7fb4ac100fb0 2024-04-02T04:42:36.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.599+0000 7fb4ab7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4ac0ffc10 0x7fb4ac0ff610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:36.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.599+0000 7fb4aaffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4ac100fb0 0x7fb4ac19b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:36.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.599+0000 7fb4aaffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4ac100fb0 0x7fb4ac19b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.16:54480/0 (socket says 172.21.15.16:54480) 2024-04-02T04:42:36.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.599+0000 7fb4aaffd700 1 -- 172.21.15.16:0/1508981951 learned_addr learned my addr 172.21.15.16:0/1508981951 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:42:36.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.600+0000 7fb4ab7fe700 1 -- 172.21.15.16:0/1508981951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4ac100fb0 msgr2=0x7fb4ac19b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:36.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.600+0000 7fb4ab7fe700 1 --2- 172.21.15.16:0/1508981951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4ac100fb0 0x7fb4ac19b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:36.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.600+0000 7fb4ab7fe700 1 -- 172.21.15.16:0/1508981951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a00095f0 con 0x7fb4ac0ffc10 2024-04-02T04:42:36.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.600+0000 7fb4aaffd700 1 --2- 172.21.15.16:0/1508981951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4ac100fb0 0x7fb4ac19b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T04:42:36.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.600+0000 7fb4ab7fe700 1 --2- 172.21.15.16:0/1508981951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4ac0ffc10 0x7fb4ac0ff610 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb49c00e940 tx=0x7fb49c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:42:36.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.600+0000 7fb4a8ff9700 1 -- 172.21.15.16:0/1508981951 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb49c0041f0 con 0x7fb4ac0ffc10 2024-04-02T04:42:36.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.600+0000 7fb4a8ff9700 1 -- 172.21.15.16:0/1508981951 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb49c004350 con 0x7fb4ac0ffc10 2024-04-02T04:42:36.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.600+0000 7fb4a8ff9700 1 -- 172.21.15.16:0/1508981951 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb49c003c90 con 0x7fb4ac0ffc10 2024-04-02T04:42:36.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.600+0000 7fb4b1fbd700 1 -- 172.21.15.16:0/1508981951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4ac19bca0 con 0x7fb4ac0ffc10 2024-04-02T04:42:36.605 INFO:tasks.workunit.client.0.smithi016.stdout:1/297: dwrite d14/d19/d1d/d29/f3c [0,4194304] 0 2024-04-02T04:42:36.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.600+0000 7fb4b1fbd700 1 -- 172.21.15.16:0/1508981951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4ac19c220 con 0x7fb4ac0ffc10 2024-04-02T04:42:36.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.601+0000 7fb4b1fbd700 1 -- 172.21.15.16:0/1508981951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4ac18f590 con 0x7fb4ac0ffc10 2024-04-02T04:42:36.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.602+0000 7fb4a8ff9700 1 -- 172.21.15.16:0/1508981951 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 91380+0+0 (secure 0 0 0) 0x7fb49c00c7c0 con 0x7fb4ac0ffc10 2024-04-02T04:42:36.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.603+0000 7fb4a8ff9700 1 --2- 172.21.15.16:0/1508981951 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7fb49406d490 0x7fb49406f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:36.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.603+0000 7fb4a8ff9700 1 -- 172.21.15.16:0/1508981951 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fb49c014070 con 0x7fb4ac0ffc10 2024-04-02T04:42:36.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.603+0000 7fb4aaffd700 1 --2- 172.21.15.16:0/1508981951 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7fb49406d490 0x7fb49406f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:36.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.604+0000 7fb4aaffd700 1 --2- 172.21.15.16:0/1508981951 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7fb49406d490 0x7fb49406f950 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb4a000b800 tx=0x7fb4a0009f70 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:42:36.608 INFO:tasks.workunit.client.0.smithi016.stdout:7/300: mknod d0/d28/c71 0 2024-04-02T04:42:36.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.608+0000 7fb4a8ff9700 1 -- 172.21.15.16:0/1508981951 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184299 (secure 0 0 0) 0x7fb49c05a680 con 0x7fb4ac0ffc10 2024-04-02T04:42:36.611 INFO:tasks.workunit.client.0.smithi016.stdout:6/316: symlink d0/d5c/l5f 0 2024-04-02T04:42:36.616 INFO:tasks.workunit.client.0.smithi016.stdout:8/296: truncate d1/f1c 2894755 0 2024-04-02T04:42:36.627 INFO:tasks.workunit.client.0.smithi016.stdout:0/269: truncate d0/f32 3712763 0 2024-04-02T04:42:36.630 INFO:tasks.workunit.client.0.smithi016.stdout:0/270: dread d0/f38 [0,4194304] 0 2024-04-02T04:42:36.633 INFO:tasks.workunit.client.0.smithi016.stdout:9/302: symlink d0/l55 0 2024-04-02T04:42:36.633 INFO:tasks.workunit.client.0.smithi016.stdout:9/303: chown d0/d16/f44 55133 1 2024-04-02T04:42:36.635 INFO:tasks.workunit.client.0.smithi016.stdout:2/242: write f8 [4439613,60694] 0 2024-04-02T04:42:36.639 INFO:tasks.workunit.client.0.smithi016.stdout:3/353: getdents d2/d15/d16/d3d/d46 0 2024-04-02T04:42:36.653 INFO:tasks.workunit.client.0.smithi016.stdout:5/334: creat d3/df/f82 x:0 0 0 2024-04-02T04:42:36.653 INFO:tasks.workunit.client.0.smithi016.stdout:1/298: write d14/d19/d1e/f2b [3663308,72243] 0 2024-04-02T04:42:36.657 INFO:tasks.workunit.client.0.smithi016.stdout:8/297: link d1/f5c d1/d6/d36/d3f/d40/d63/f66 0 2024-04-02T04:42:36.660 INFO:tasks.workunit.client.0.smithi016.stdout:2/243: mkdir d12/d35/d3b/d42/d59 0 2024-04-02T04:42:36.663 INFO:tasks.workunit.client.0.smithi016.stdout:4/303: truncate d5/d17/d18/d25/f3b 74914 0 2024-04-02T04:42:36.667 INFO:tasks.workunit.client.0.smithi016.stdout:7/301: rename d0/d46/d16/d49/c2b to d0/c72 0 2024-04-02T04:42:36.667 INFO:tasks.workunit.client.0.smithi016.stdout:8/298: dread d1/d6/f25 [0,4194304] 0 2024-04-02T04:42:36.669 INFO:tasks.workunit.client.0.smithi016.stdout:8/299: dread d1/d6/f57 [0,4194304] 0 2024-04-02T04:42:36.669 INFO:tasks.workunit.client.0.smithi016.stdout:8/300: readlink d1/d31/l3c 0 2024-04-02T04:42:36.671 INFO:tasks.workunit.client.0.smithi016.stdout:5/335: fsync d3/df/d18/f35 0 2024-04-02T04:42:36.672 INFO:tasks.workunit.client.0.smithi016.stdout:6/317: truncate d0/d9/d16/f23 2890209 0 2024-04-02T04:42:36.674 INFO:tasks.workunit.client.0.smithi016.stdout:0/271: mknod d0/c5d 0 2024-04-02T04:42:36.676 INFO:tasks.workunit.client.0.smithi016.stdout:1/299: mknod d14/d41/d43/c58 0 2024-04-02T04:42:36.677 INFO:tasks.workunit.client.0.smithi016.stdout:1/300: write d14/f56 [628840,78533] 0 2024-04-02T04:42:36.678 INFO:tasks.workunit.client.0.smithi016.stdout:9/304: unlink d0/d9/c47 0 2024-04-02T04:42:36.683 INFO:tasks.workunit.client.0.smithi016.stdout:2/244: dread - d12/d20/f52 zero size 2024-04-02T04:42:36.684 INFO:tasks.workunit.client.0.smithi016.stdout:3/354: mkdir d2/d15/d16/d3d/d6e/d6f 0 2024-04-02T04:42:36.685 INFO:tasks.workunit.client.0.smithi016.stdout:4/304: creat d5/d17/d18/d1f/f59 x:0 0 0 2024-04-02T04:42:36.688 INFO:tasks.workunit.client.0.smithi016.stdout:7/302: rename d0/d46/d37/f3b to d0/d46/d37/d3a/d4c/f73 0 2024-04-02T04:42:36.693 INFO:tasks.workunit.client.0.smithi016.stdout:8/301: mkdir d1/d31/d67 0 2024-04-02T04:42:36.697 INFO:tasks.workunit.client.0.smithi016.stdout:6/318: creat d0/d9/d16/d56/d59/f60 x:0 0 0 2024-04-02T04:42:36.697 INFO:tasks.workunit.client.0.smithi016.stdout:6/319: dread - d0/f5d zero size 2024-04-02T04:42:36.698 INFO:tasks.workunit.client.0.smithi016.stdout:1/301: stat d14/c27 0 2024-04-02T04:42:36.706 INFO:tasks.workunit.client.0.smithi016.stdout:2/245: creat d12/d26/f5a x:0 0 0 2024-04-02T04:42:36.710 INFO:tasks.workunit.client.0.smithi016.stdout:4/305: dread - d5/d1d/d1e/d50/f56 zero size 2024-04-02T04:42:36.712 INFO:tasks.workunit.client.0.smithi016.stdout:0/272: rename d0/d16/d2e/d37/l51 to d0/d48/l5e 0 2024-04-02T04:42:36.716 INFO:tasks.workunit.client.0.smithi016.stdout:6/320: mknod d0/d9/d16/d56/c61 0 2024-04-02T04:42:36.717 INFO:tasks.workunit.client.0.smithi016.stdout:6/321: write d0/d9/d16/d56/f4f [901575,55607] 0 2024-04-02T04:42:36.721 INFO:tasks.workunit.client.0.smithi016.stdout:3/355: fsync d2/d15/d16/f25 0 2024-04-02T04:42:36.726 INFO:tasks.workunit.client.0.smithi016.stdout:0/273: symlink d0/d16/d2e/l5f 0 2024-04-02T04:42:36.727 INFO:tasks.workunit.client.0.smithi016.stdout:0/274: chown d0/d4f 902 1 2024-04-02T04:42:36.729 INFO:tasks.workunit.client.0.smithi016.stdout:5/336: creat d3/df/d18/f83 x:0 0 0 2024-04-02T04:42:36.730 INFO:tasks.workunit.client.0.smithi016.stdout:9/305: dwrite d0/d9/db/f1c [0,4194304] 0 2024-04-02T04:42:36.731 INFO:tasks.workunit.client.0.smithi016.stdout:5/337: write d3/df/f1e [2324369,77828] 0 2024-04-02T04:42:36.735 INFO:tasks.workunit.client.0.smithi016.stdout:2/246: dread d12/f23 [0,4194304] 0 2024-04-02T04:42:36.737 INFO:tasks.workunit.client.0.smithi016.stdout:3/356: readlink d2/d15/d16/l2b 0 2024-04-02T04:42:36.738 INFO:tasks.workunit.client.0.smithi016.stdout:3/357: chown d2/d15/d1a/f69 22 1 2024-04-02T04:42:36.738 INFO:tasks.workunit.client.0.smithi016.stdout:4/306: mknod d5/d1d/c5a 0 2024-04-02T04:42:36.742 INFO:tasks.workunit.client.0.smithi016.stdout:7/303: rename d0/d46/cf to d0/d46/d16/d49/c74 0 2024-04-02T04:42:36.743 INFO:tasks.workunit.client.0.smithi016.stdout:0/275: mkdir d0/d16/d60 0 2024-04-02T04:42:36.745 INFO:tasks.workunit.client.0.smithi016.stdout:5/338: symlink d3/d40/d4f/d6b/d2a/l84 0 2024-04-02T04:42:36.745 INFO:tasks.workunit.client.0.smithi016.stdout:5/339: dread - d3/df/f5a zero size 2024-04-02T04:42:36.748 INFO:tasks.workunit.client.0.smithi016.stdout:2/247: mknod d12/d20/d4f/c5b 0 2024-04-02T04:42:36.750 INFO:tasks.workunit.client.0.smithi016.stdout:2/248: dread - d12/d20/d4f/f53 zero size 2024-04-02T04:42:36.750 INFO:tasks.workunit.client.0.smithi016.stdout:2/249: dread d12/d26/f29 [0,4194304] 0 2024-04-02T04:42:36.750 INFO:tasks.workunit.client.0.smithi016.stdout:4/307: creat d5/d8/d43/f5b x:0 0 0 2024-04-02T04:42:36.750 INFO:tasks.workunit.client.0.smithi016.stdout:2/250: truncate d12/d20/f52 642096 0 2024-04-02T04:42:36.751 INFO:tasks.workunit.client.0.smithi016.stdout:8/302: getdents d1/d6/d36/d3f 0 2024-04-02T04:42:36.751 INFO:tasks.workunit.client.0.smithi016.stdout:4/308: readlink d5/d1d/d1e/l26 0 2024-04-02T04:42:36.751 INFO:tasks.workunit.client.0.smithi016.stdout:2/251: read - d12/d35/d3b/f57 zero size 2024-04-02T04:42:36.752 INFO:tasks.workunit.client.0.smithi016.stdout:1/302: rename d14/d19/d1d/c33 to d14/d41/d43/c59 0 2024-04-02T04:42:36.752 INFO:tasks.workunit.client.0.smithi016.stdout:2/252: write d12/d35/d3b/f46 [845636,34108] 0 2024-04-02T04:42:36.754 INFO:tasks.workunit.client.0.smithi016.stdout:7/304: mknod d0/d46/d37/d3a/c75 0 2024-04-02T04:42:36.755 INFO:tasks.workunit.client.0.smithi016.stdout:7/305: readlink d0/d46/l1e 0 2024-04-02T04:42:36.755 INFO:tasks.workunit.client.0.smithi016.stdout:9/306: mknod d0/d4e/d50/c56 0 2024-04-02T04:42:36.755 INFO:tasks.workunit.client.0.smithi016.stdout:9/307: chown d0/d9/f19 16149689 1 2024-04-02T04:42:36.758 INFO:tasks.workunit.client.0.smithi016.stdout:0/276: creat d0/d16/d2e/f61 x:0 0 0 2024-04-02T04:42:36.759 INFO:tasks.workunit.client.0.smithi016.stdout:0/277: chown d0/d16/d24/l27 2521 1 2024-04-02T04:42:36.759 INFO:tasks.workunit.client.0.smithi016.stdout:0/278: write d0/d16/d2e/d53/f52 [853621,59650] 0 2024-04-02T04:42:36.760 INFO:tasks.workunit.client.0.smithi016.stdout:0/279: write d0/d16/d2e/d36/f3d [380938,38192] 0 2024-04-02T04:42:36.762 INFO:tasks.workunit.client.0.smithi016.stdout:0/280: write d0/d16/d2e/d37/f4b [4839234,121405] 0 2024-04-02T04:42:36.766 INFO:tasks.workunit.client.0.smithi016.stdout:3/358: mkdir d2/d15/d70 0 2024-04-02T04:42:36.768 INFO:tasks.workunit.client.0.smithi016.stdout:8/303: creat d1/d6/d36/d3f/d40/d49/f68 x:0 0 0 2024-04-02T04:42:36.770 INFO:tasks.workunit.client.0.smithi016.stdout:1/303: fdatasync d14/d19/d1e/f2e 0 2024-04-02T04:42:36.773 INFO:tasks.workunit.client.0.smithi016.stdout:5/340: rmdir d3 39 2024-04-02T04:42:36.775 INFO:tasks.workunit.client.0.smithi016.stdout:9/308: creat d0/d3/d31/f57 x:0 0 0 2024-04-02T04:42:36.775 INFO:tasks.workunit.client.0.smithi016.stdout:9/309: chown d0/d33/f38 31497 1 2024-04-02T04:42:36.776 INFO:tasks.workunit.client.0.smithi016.stdout:9/310: fsync d0/f2c 0 2024-04-02T04:42:36.777 INFO:tasks.workunit.client.0.smithi016.stdout:9/311: truncate d0/d33/f36 1254512 0 2024-04-02T04:42:36.778 INFO:tasks.workunit.client.0.smithi016.stdout:0/281: mkdir d0/d16/d24/d62 0 2024-04-02T04:42:36.781 INFO:tasks.workunit.client.0.smithi016.stdout:4/309: symlink d5/l5c 0 2024-04-02T04:42:36.792 INFO:tasks.workunit.client.0.smithi016.stdout:6/322: dwrite d0/d1f/f26 [0,4194304] 0 2024-04-02T04:42:36.793 INFO:tasks.workunit.client.0.smithi016.stdout:4/310: mknod d5/d17/c5d 0 2024-04-02T04:42:36.794 INFO:tasks.workunit.client.0.smithi016.stdout:6/323: read d0/f40 [4016323,51716] 0 2024-04-02T04:42:36.796 INFO:tasks.workunit.client.0.smithi016.stdout:3/359: write d2/d2e/f30 [347386,122579] 0 2024-04-02T04:42:36.799 INFO:tasks.workunit.client.0.smithi016.stdout:2/253: rename f10 to d12/d35/f5c 0 2024-04-02T04:42:36.799 INFO:tasks.workunit.client.0.smithi016.stdout:9/312: dread d0/f2c [0,4194304] 0 2024-04-02T04:42:36.806 INFO:tasks.workunit.client.0.smithi016.stdout:4/311: mknod d5/d1d/d1e/d50/c5e 0 2024-04-02T04:42:36.807 INFO:tasks.workunit.client.0.smithi016.stdout:8/304: write d1/f3 [1353391,66857] 0 2024-04-02T04:42:36.813 INFO:tasks.workunit.client.0.smithi016.stdout:1/304: rename c12 to d14/d19/d1d/d29/d4c/c5a 0 2024-04-02T04:42:36.815 INFO:tasks.workunit.client.0.smithi016.stdout:2/254: rmdir d12/d35 39 2024-04-02T04:42:36.816 INFO:tasks.workunit.client.0.smithi016.stdout:4/312: dwrite d5/d8/d43/f4b [0,4194304] 0 2024-04-02T04:42:36.819 INFO:tasks.workunit.client.0.smithi016.stdout:7/306: getdents d0/d21 0 2024-04-02T04:42:36.819 INFO:tasks.workunit.client.0.smithi016.stdout:2/255: dread - d12/f43 zero size 2024-04-02T04:42:36.820 INFO:tasks.workunit.client.0.smithi016.stdout:5/341: dwrite d3/df/f1f [4194304,4194304] 0 2024-04-02T04:42:36.821 INFO:tasks.workunit.client.0.smithi016.stdout:3/360: dwrite d2/d15/d16/f2c [4194304,4194304] 0 2024-04-02T04:42:36.823 INFO:tasks.workunit.client.0.smithi016.stdout:6/324: mknod d0/d9/c62 0 2024-04-02T04:42:36.823 INFO:tasks.workunit.client.0.smithi016.stdout:6/325: stat d0/c17 0 2024-04-02T04:42:36.827 INFO:tasks.workunit.client.0.smithi016.stdout:9/313: mkdir d0/d16/d4b/d58 0 2024-04-02T04:42:36.828 INFO:tasks.workunit.client.0.smithi016.stdout:0/282: getdents d0/d16/d23/d3e 0 2024-04-02T04:42:36.829 INFO:tasks.workunit.client.0.smithi016.stdout:1/305: read f2 [1911120,130701] 0 2024-04-02T04:42:36.832 INFO:tasks.workunit.client.0.smithi016.stdout:7/307: creat d0/d46/d37/d3a/f76 x:0 0 0 2024-04-02T04:42:36.836 INFO:tasks.workunit.client.0.smithi016.stdout:5/342: mknod d3/d4c/c85 0 2024-04-02T04:42:36.837 INFO:tasks.workunit.client.0.smithi016.stdout:5/343: chown d3/d40/d4f/d67/f78 248565180 1 2024-04-02T04:42:36.838 INFO:tasks.workunit.client.0.smithi016.stdout:8/305: mknod d1/d6/d36/d3f/d40/d49/d38/c69 0 2024-04-02T04:42:36.839 INFO:tasks.workunit.client.0.smithi016.stdout:8/306: fsync d1/d6/d36/d4f/d5d/f5e 0 2024-04-02T04:42:36.839 INFO:tasks.workunit.client.0.smithi016.stdout:6/326: creat d0/d9/d16/f63 x:0 0 0 2024-04-02T04:42:36.840 INFO:tasks.workunit.client.0.smithi016.stdout:9/314: chown d0/d16/d1a/l26 11308289 1 2024-04-02T04:42:36.841 INFO:tasks.workunit.client.0.smithi016.stdout:4/313: mkdir d5/d1d/d5f 0 2024-04-02T04:42:36.842 INFO:tasks.workunit.client.0.smithi016.stdout:4/314: truncate d5/f4e 576785 0 2024-04-02T04:42:36.843 INFO:tasks.workunit.client.0.smithi016.stdout:2/256: rename d12/d35/l38 to d12/d14/l5d 0 2024-04-02T04:42:36.843 INFO:tasks.workunit.client.0.smithi016.stdout:4/315: write f4 [4550068,4156] 0 2024-04-02T04:42:36.844 INFO:tasks.workunit.client.0.smithi016.stdout:2/257: truncate d12/d20/f44 268961 0 2024-04-02T04:42:36.845 INFO:tasks.workunit.client.0.smithi016.stdout:1/306: chown d14/d19/d1d/d29/d39/d46 37066 1 2024-04-02T04:42:36.847 INFO:tasks.workunit.client.0.smithi016.stdout:6/327: unlink d0/d9/d16/c5a 0 2024-04-02T04:42:36.848 INFO:tasks.workunit.client.0.smithi016.stdout:8/307: rmdir d1/d6/d36/d4f 39 2024-04-02T04:42:36.852 INFO:tasks.workunit.client.0.smithi016.stdout:9/315: mknod d0/d16/d4b/c59 0 2024-04-02T04:42:36.852 INFO:tasks.workunit.client.0.smithi016.stdout:0/283: truncate d0/fc 1295293 0 2024-04-02T04:42:36.852 INFO:tasks.workunit.client.0.smithi016.stdout:4/316: dwrite d5/d8/fd [0,4194304] 0 2024-04-02T04:42:36.853 INFO:tasks.workunit.client.0.smithi016.stdout:8/308: write d1/d6/d36/d3f/d40/d49/f44 [438062,14851] 0 2024-04-02T04:42:36.854 INFO:tasks.workunit.client.0.smithi016.stdout:5/344: dwrite d3/f1a [0,4194304] 0 2024-04-02T04:42:36.855 INFO:tasks.workunit.client.0.smithi016.stdout:5/345: fsync d3/d5d/f6f 0 2024-04-02T04:42:36.857 INFO:tasks.workunit.client.0.smithi016.stdout:7/308: mkdir d0/d46/d37/d3a/d6f/d77 0 2024-04-02T04:42:36.857 INFO:tasks.workunit.client.0.smithi016.stdout:7/309: fdatasync d0/d21/f65 0 2024-04-02T04:42:36.859 INFO:tasks.workunit.client.0.smithi016.stdout:7/310: read - d0/d21/f4e zero size 2024-04-02T04:42:36.860 INFO:tasks.workunit.client.0.smithi016.stdout:7/311: read d0/d46/d16/f3d [431937,40468] 0 2024-04-02T04:42:36.862 INFO:tasks.workunit.client.0.smithi016.stdout:1/307: creat d14/d19/d20/f5b x:0 0 0 2024-04-02T04:42:36.863 INFO:tasks.workunit.client.0.smithi016.stdout:5/346: dwrite d3/d5d/f69 [0,4194304] 0 2024-04-02T04:42:36.867 INFO:tasks.workunit.client.0.smithi016.stdout:2/258: dread d12/d35/d3f/f2d [0,4194304] 0 2024-04-02T04:42:36.867 INFO:tasks.workunit.client.0.smithi016.stdout:5/347: write d3/df/d18/d2f/f62 [717691,90549] 0 2024-04-02T04:42:36.870 INFO:tasks.workunit.client.0.smithi016.stdout:9/316: rmdir d0/d9/db 39 2024-04-02T04:42:36.873 INFO:tasks.workunit.client.0.smithi016.stdout:7/312: dwrite d0/d46/d16/d49/f3e [0,4194304] 0 2024-04-02T04:42:36.875 INFO:tasks.workunit.client.0.smithi016.stdout:3/361: getdents d2/d2e/d36 0 2024-04-02T04:42:36.879 INFO:tasks.workunit.client.0.smithi016.stdout:6/328: creat d0/d9/d16/d46/d5e/f64 x:0 0 0 2024-04-02T04:42:36.881 INFO:tasks.workunit.client.0.smithi016.stdout:1/308: creat d14/d41/f5c x:0 0 0 2024-04-02T04:42:36.881 INFO:tasks.workunit.client.0.smithi016.stdout:2/259: creat d12/d35/d3b/d42/f5e x:0 0 0 2024-04-02T04:42:36.884 INFO:tasks.workunit.client.0.smithi016.stdout:8/309: symlink d1/d6/d36/d3f/d40/d63/l6a 0 2024-04-02T04:42:36.884 INFO:tasks.workunit.client.0.smithi016.stdout:3/362: dwrite d2/fa [0,4194304] 0 2024-04-02T04:42:36.887 INFO:tasks.workunit.client.0.smithi016.stdout:7/313: symlink d0/d28/d64/l78 0 2024-04-02T04:42:36.888 INFO:tasks.workunit.client.0.smithi016.stdout:1/309: chown d14/d19/d20/f24 112163588 1 2024-04-02T04:42:36.888 INFO:tasks.workunit.client.0.smithi016.stdout:3/363: readlink d2/d15/d16/d3d/d46/l59 0 2024-04-02T04:42:36.888 INFO:tasks.workunit.client.0.smithi016.stdout:6/329: truncate d0/d9/d16/d46/d5e/f64 267883 0 2024-04-02T04:42:36.888 INFO:tasks.workunit.client.0.smithi016.stdout:2/260: chown d12/d35/d3b/f54 6 1 2024-04-02T04:42:36.888 INFO:tasks.workunit.client.0.smithi016.stdout:9/317: chown d0/d9/db/f1c 10 1 2024-04-02T04:42:36.889 INFO:tasks.workunit.client.0.smithi016.stdout:1/310: dread - d14/d41/f47 zero size 2024-04-02T04:42:36.892 INFO:tasks.workunit.client.0.smithi016.stdout:3/364: chown d2/d15/d1a/d45/f55 10508 1 2024-04-02T04:42:36.892 INFO:tasks.workunit.client.0.smithi016.stdout:2/261: truncate d12/f43 206438 0 2024-04-02T04:42:36.892 INFO:tasks.workunit.client.0.smithi016.stdout:6/330: write d0/d1f/f26 [1295810,100802] 0 2024-04-02T04:42:36.892 INFO:tasks.workunit.client.0.smithi016.stdout:4/317: creat d5/f60 x:0 0 0 2024-04-02T04:42:36.896 INFO:tasks.workunit.client.0.smithi016.stdout:3/365: write d2/d2e/f67 [545195,21790] 0 2024-04-02T04:42:36.897 INFO:tasks.workunit.client.0.smithi016.stdout:1/311: creat d14/d19/d1e/f5d x:0 0 0 2024-04-02T04:42:36.897 INFO:tasks.workunit.client.0.smithi016.stdout:8/310: truncate d1/d6/d36/d3f/d40/d49/f68 656934 0 2024-04-02T04:42:36.899 INFO:tasks.workunit.client.0.smithi016.stdout:1/312: readlink d14/l2d 0 2024-04-02T04:42:36.900 INFO:tasks.workunit.client.0.smithi016.stdout:6/331: creat d0/d5c/f65 x:0 0 0 2024-04-02T04:42:36.904 INFO:tasks.workunit.client.0.smithi016.stdout:7/314: dwrite d0/d28/f42 [0,4194304] 0 2024-04-02T04:42:36.907 INFO:tasks.workunit.client.0.smithi016.stdout:8/311: creat d1/d31/f6b x:0 0 0 2024-04-02T04:42:36.910 INFO:tasks.workunit.client.0.smithi016.stdout:2/262: dwrite d12/d20/f44 [0,4194304] 0 2024-04-02T04:42:36.911 INFO:tasks.workunit.client.0.smithi016.stdout:0/284: write d0/fe [2336788,20868] 0 2024-04-02T04:42:36.912 INFO:tasks.workunit.client.0.smithi016.stdout:1/313: dread d14/f34 [0,4194304] 0 2024-04-02T04:42:36.912 INFO:tasks.workunit.client.0.smithi016.stdout:3/366: getdents d2/d15/d16/d3d/d6e/d6f 0 2024-04-02T04:42:36.914 INFO:tasks.workunit.client.0.smithi016.stdout:7/315: mknod d0/d46/d16/d49/c79 0 2024-04-02T04:42:36.915 INFO:tasks.workunit.client.0.smithi016.stdout:6/332: symlink d0/l66 0 2024-04-02T04:42:36.915 INFO:tasks.workunit.client.0.smithi016.stdout:7/316: chown d0/d46/d37/d3a 15985 1 2024-04-02T04:42:36.916 INFO:tasks.workunit.client.0.smithi016.stdout:7/317: read d0/f22 [3770845,115323] 0 2024-04-02T04:42:36.916 INFO:tasks.workunit.client.0.smithi016.stdout:3/367: read d2/d15/d1a/f1e [3850843,15615] 0 2024-04-02T04:42:36.917 INFO:tasks.workunit.client.0.smithi016.stdout:8/312: creat d1/d6/d36/d3f/f6c x:0 0 0 2024-04-02T04:42:36.919 INFO:tasks.workunit.client.0.smithi016.stdout:4/318: dread d5/d17/f35 [0,4194304] 0 2024-04-02T04:42:36.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.918+0000 7fb4b1fbd700 1 -- 172.21.15.16:0/1508981951 --> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb4ac02ce20 con 0x7fb49406d490 2024-04-02T04:42:36.921 INFO:tasks.workunit.client.0.smithi016.stdout:0/285: read d0/fe [1406231,68322] 0 2024-04-02T04:42:36.922 INFO:tasks.workunit.client.0.smithi016.stdout:2/263: creat d12/d20/d4f/f5f x:0 0 0 2024-04-02T04:42:36.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.920+0000 7fb4a8ff9700 1 -- 172.21.15.16:0/1508981951 <== mgr.24415 v2:172.21.15.27:6828/1554339939 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7fb4ac02ce20 con 0x7fb49406d490 2024-04-02T04:42:36.923 INFO:tasks.workunit.client.0.smithi016.stdout:1/314: stat d14/d19/d1d/d29/d4c/c5a 0 2024-04-02T04:42:36.925 INFO:tasks.workunit.client.0.smithi016.stdout:0/286: write d0/d16/d2e/d37/f3c [4099181,50454] 0 2024-04-02T04:42:36.927 INFO:tasks.workunit.client.0.smithi016.stdout:1/315: fdatasync d14/d19/d20/f38 0 2024-04-02T04:42:36.927 INFO:tasks.workunit.client.0.smithi016.stdout:7/318: readlink d0/d46/l5f 0 2024-04-02T04:42:36.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.923+0000 7fb4b1fbd700 1 -- 172.21.15.16:0/1508981951 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7fb49406d490 msgr2=0x7fb49406f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:36.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.924+0000 7fb4b1fbd700 1 --2- 172.21.15.16:0/1508981951 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7fb49406d490 0x7fb49406f950 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb4a000b800 tx=0x7fb4a0009f70 comp rx=0 tx=0).stop 2024-04-02T04:42:36.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.924+0000 7fb4b1fbd700 1 -- 172.21.15.16:0/1508981951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4ac0ffc10 msgr2=0x7fb4ac0ff610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:36.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.924+0000 7fb4b1fbd700 1 --2- 172.21.15.16:0/1508981951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4ac0ffc10 0x7fb4ac0ff610 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb49c00e940 tx=0x7fb49c00ec50 comp rx=0 tx=0).stop 2024-04-02T04:42:36.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.924+0000 7fb4b1fbd700 1 -- 172.21.15.16:0/1508981951 shutdown_connections 2024-04-02T04:42:36.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.924+0000 7fb4b1fbd700 1 --2- 172.21.15.16:0/1508981951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4ac0ffc10 0x7fb4ac0ff610 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:36.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.924+0000 7fb4b1fbd700 1 --2- 172.21.15.16:0/1508981951 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7fb49406d490 0x7fb49406f950 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:36.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.924+0000 7fb4b1fbd700 1 --2- 172.21.15.16:0/1508981951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4ac100fb0 0x7fb4ac19b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:36.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.924+0000 7fb4b1fbd700 1 -- 172.21.15.16:0/1508981951 >> 172.21.15.16:0/1508981951 conn(0x7fb4ac0fa700 msgr2=0x7fb4ac104270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:42:36.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.924+0000 7fb4b1fbd700 1 -- 172.21.15.16:0/1508981951 shutdown_connections 2024-04-02T04:42:36.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:36.924+0000 7fb4b1fbd700 1 -- 172.21.15.16:0/1508981951 wait complete. 2024-04-02T04:42:36.929 INFO:tasks.workunit.client.0.smithi016.stdout:6/333: dwrite d0/d9/f36 [0,4194304] 0 2024-04-02T04:42:36.932 INFO:tasks.workunit.client.0.smithi016.stdout:4/319: read d5/d17/d18/d25/f3b [72360,97692] 0 2024-04-02T04:42:36.933 INFO:tasks.workunit.client.0.smithi016.stdout:2/264: creat d12/d35/d3b/f60 x:0 0 0 2024-04-02T04:42:36.934 INFO:tasks.workunit.client.0.smithi016.stdout:3/368: symlink d2/d2e/d36/d66/l71 0 2024-04-02T04:42:36.937 INFO:tasks.workunit.client.0.smithi016.stdout:2/265: write d12/d20/f25 [8155939,66948] 0 2024-04-02T04:42:36.939 INFO:tasks.workunit.client.0.smithi016.stdout:3/369: read d2/d15/f23 [715348,64266] 0 2024-04-02T04:42:36.939 INFO:tasks.workunit.client.0.smithi016.stdout:6/334: mkdir d0/d1f/d67 0 2024-04-02T04:42:36.941 INFO:tasks.workunit.client.0.smithi016.stdout:0/287: rename d0/d16/d2e/d37/c56 to d0/d4f/c63 0 2024-04-02T04:42:36.941 INFO:tasks.workunit.client.0.smithi016.stdout:9/318: dwrite d0/f2d [0,4194304] 0 2024-04-02T04:42:36.941 INFO:tasks.workunit.client.0.smithi016.stdout:2/266: dread - d12/d35/d3f/f39 zero size 2024-04-02T04:42:36.944 INFO:tasks.workunit.client.0.smithi016.stdout:8/313: dread d1/f1c [0,4194304] 0 2024-04-02T04:42:36.944 INFO:tasks.workunit.client.0.smithi016.stdout:9/319: write d0/f2c [1786549,21707] 0 2024-04-02T04:42:36.950 INFO:tasks.workunit.client.0.smithi016.stdout:3/370: dread d2/d2e/f67 [0,4194304] 0 2024-04-02T04:42:36.951 INFO:teuthology.orchestra.run.smithi016.stdout:true 2024-04-02T04:42:36.952 INFO:tasks.workunit.client.0.smithi016.stdout:0/288: read d0/d16/d23/d3e/d57/f59 [1851542,119470] 0 2024-04-02T04:42:36.952 INFO:tasks.workunit.client.0.smithi016.stdout:9/320: read - d0/d3/d31/f45 zero size 2024-04-02T04:42:36.952 INFO:tasks.workunit.client.0.smithi016.stdout:3/371: chown d2/fd 7033757 1 2024-04-02T04:42:36.955 INFO:tasks.workunit.client.0.smithi016.stdout:2/267: rename d12/d35/d3f/f4e to d12/d35/d3f/f61 0 2024-04-02T04:42:36.960 INFO:tasks.workunit.client.0.smithi016.stdout:1/316: dwrite d14/d19/d1d/f1f [0,4194304] 0 2024-04-02T04:42:36.961 INFO:tasks.workunit.client.0.smithi016.stdout:1/317: chown d14/c1b 6 1 2024-04-02T04:42:36.966 INFO:tasks.workunit.client.0.smithi016.stdout:0/289: dread d0/f32 [0,4194304] 0 2024-04-02T04:42:36.966 INFO:tasks.workunit.client.0.smithi016.stdout:8/314: dread d1/d6/d36/f37 [0,4194304] 0 2024-04-02T04:42:36.969 INFO:tasks.workunit.client.0.smithi016.stdout:4/320: dwrite d5/d17/d18/d25/f4a [0,4194304] 0 2024-04-02T04:42:36.970 INFO:tasks.workunit.client.0.smithi016.stdout:9/321: mknod d0/d9/c5a 0 2024-04-02T04:42:36.973 INFO:tasks.workunit.client.0.smithi016.stdout:4/321: write d5/d17/d18/d1f/f59 [576535,127624] 0 2024-04-02T04:42:36.974 INFO:tasks.workunit.client.0.smithi016.stdout:3/372: symlink d2/d15/d16/d3d/d6e/d6f/l72 0 2024-04-02T04:42:36.976 INFO:tasks.workunit.client.0.smithi016.stdout:1/318: mkdir d14/d19/d1e/d5e 0 2024-04-02T04:42:36.976 INFO:tasks.workunit.client.0.smithi016.stdout:4/322: chown d5/d1d/d5f 15 1 2024-04-02T04:42:36.977 INFO:tasks.workunit.client.0.smithi016.stdout:0/290: dwrite d0/d16/d2e/d36/f44 [4194304,4194304] 0 2024-04-02T04:42:36.978 INFO:tasks.workunit.client.0.smithi016.stdout:8/315: chown d1/l7 0 1 2024-04-02T04:42:36.980 INFO:tasks.workunit.client.0.smithi016.stdout:4/323: fsync d5/d17/d18/d1f/f59 0 2024-04-02T04:42:36.980 INFO:tasks.workunit.client.0.smithi016.stdout:6/335: truncate d0/d9/f32 3963891 0 2024-04-02T04:42:36.982 INFO:tasks.workunit.client.0.smithi016.stdout:9/322: mknod d0/d3/d31/c5b 0 2024-04-02T04:42:36.982 INFO:tasks.workunit.client.0.smithi016.stdout:6/336: stat d0/d9/df/c52 0 2024-04-02T04:42:36.984 INFO:tasks.workunit.client.0.smithi016.stdout:3/373: rename d2/d15/d1a/d45/c68 to d2/d2e/d36/d48/c73 0 2024-04-02T04:42:36.986 INFO:tasks.workunit.client.0.smithi016.stdout:2/268: dwrite d12/f4c [0,4194304] 0 2024-04-02T04:42:36.988 INFO:tasks.workunit.client.0.smithi016.stdout:4/324: fdatasync d5/d1d/d2d/f4d 0 2024-04-02T04:42:36.993 INFO:tasks.workunit.client.0.smithi016.stdout:1/319: rename d14/d19/d1e/f42 to d14/d41/d43/f5f 0 2024-04-02T04:42:36.994 INFO:tasks.workunit.client.0.smithi016.stdout:0/291: mknod d0/d16/d24/d62/c64 0 2024-04-02T04:42:36.996 INFO:tasks.workunit.client.0.smithi016.stdout:3/374: rmdir d2/d15/d16/d3d/d6e 39 2024-04-02T04:42:36.999 INFO:tasks.workunit.client.0.smithi016.stdout:2/269: symlink d12/d35/d3f/d2b/l62 0 2024-04-02T04:42:37.000 INFO:tasks.workunit.client.0.smithi016.stdout:8/316: fsync d1/d6/d36/d3f/d40/d49/f2a 0 2024-04-02T04:42:37.000 INFO:tasks.workunit.client.0.smithi016.stdout:8/317: chown d1 650 1 2024-04-02T04:42:37.001 INFO:tasks.workunit.client.0.smithi016.stdout:9/323: mknod d0/d16/c5c 0 2024-04-02T04:42:37.014 INFO:tasks.workunit.client.0.smithi016.stdout:8/318: rmdir d1/d6/d36/d3f/d40/d49 39 2024-04-02T04:42:37.017 INFO:tasks.workunit.client.0.smithi016.stdout:4/325: truncate d5/d17/d18/d1f/f59 58763 0 2024-04-02T04:42:37.027 INFO:tasks.workunit.client.0.smithi016.stdout:6/337: getdents d0/d9/df 0 2024-04-02T04:42:37.028 INFO:tasks.workunit.client.0.smithi016.stdout:1/320: creat d14/d19/f60 x:0 0 0 2024-04-02T04:42:37.031 INFO:tasks.workunit.client.0.smithi016.stdout:2/270: dwrite f9 [0,4194304] 0 2024-04-02T04:42:37.032 INFO:tasks.workunit.client.0.smithi016.stdout:9/324: creat d0/f5d x:0 0 0 2024-04-02T04:42:37.033 INFO:tasks.workunit.client.0.smithi016.stdout:9/325: fdatasync d0/d9/f2e 0 2024-04-02T04:42:37.036 INFO:tasks.workunit.client.0.smithi016.stdout:3/375: getdents d2/d15/d1a 0 2024-04-02T04:42:37.037 INFO:tasks.workunit.client.0.smithi016.stdout:6/338: creat d0/d9/d16/d46/f68 x:0 0 0 2024-04-02T04:42:37.039 INFO:tasks.workunit.client.0.smithi016.stdout:1/321: creat d14/d19/d20/f61 x:0 0 0 2024-04-02T04:42:37.041 INFO:tasks.workunit.client.0.smithi016.stdout:2/271: dread d12/f2a [0,4194304] 0 2024-04-02T04:42:37.041 INFO:tasks.workunit.client.0.smithi016.stdout:9/326: rmdir d0/d16 39 2024-04-02T04:42:37.044 INFO:tasks.workunit.client.0.smithi016.stdout:3/376: symlink d2/d2e/d36/d66/l74 0 2024-04-02T04:42:37.046 INFO:tasks.workunit.client.0.smithi016.stdout:8/319: rmdir d1/d6/d36/d3f/d40/d49/d4d 0 2024-04-02T04:42:37.051 INFO:tasks.workunit.client.0.smithi016.stdout:3/377: symlink d2/d15/d1a/d45/l75 0 2024-04-02T04:42:37.052 INFO:tasks.workunit.client.0.smithi016.stdout:8/320: mknod d1/d6/d36/d3f/d40/d49/d4a/c6d 0 2024-04-02T04:42:37.056 INFO:tasks.workunit.client.0.smithi016.stdout:9/327: truncate d0/d9/f19 871367 0 2024-04-02T04:42:37.061 INFO:tasks.workunit.client.0.smithi016.stdout:3/378: unlink d2/f3 0 2024-04-02T04:42:37.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.071+0000 7fe4dd777700 1 -- 172.21.15.16:0/699314999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4d8101590 msgr2=0x7fe4d8103980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:37.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.071+0000 7fe4dd777700 1 --2- 172.21.15.16:0/699314999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4d8101590 0x7fe4d8103980 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe4c800b0a0 tx=0x7fe4c800b3b0 comp rx=0 tx=0).stop 2024-04-02T04:42:37.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.071+0000 7fe4dd777700 1 -- 172.21.15.16:0/699314999 shutdown_connections 2024-04-02T04:42:37.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.071+0000 7fe4dd777700 1 --2- 172.21.15.16:0/699314999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4d8103ec0 0x7fe4d81062b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:37.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.071+0000 7fe4dd777700 1 --2- 172.21.15.16:0/699314999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4d8101590 0x7fe4d8103980 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:37.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.071+0000 7fe4dd777700 1 -- 172.21.15.16:0/699314999 >> 172.21.15.16:0/699314999 conn(0x7fe4d80fafc0 msgr2=0x7fe4d80fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:42:37.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.071+0000 7fe4dd777700 1 -- 172.21.15.16:0/699314999 shutdown_connections 2024-04-02T04:42:37.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.071+0000 7fe4dd777700 1 -- 172.21.15.16:0/699314999 wait complete. 2024-04-02T04:42:37.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.072+0000 7fe4dd777700 1 Processor -- start 2024-04-02T04:42:37.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.072+0000 7fe4dd777700 1 -- start start 2024-04-02T04:42:37.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.072+0000 7fe4dd777700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4d8101590 0x7fe4d8196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:37.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.072+0000 7fe4dd777700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4d8103ec0 0x7fe4d81969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:37.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.072+0000 7fe4dd777700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d8196fc0 con 0x7fe4d8103ec0 2024-04-02T04:42:37.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.072+0000 7fe4dd777700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d8197100 con 0x7fe4d8101590 2024-04-02T04:42:37.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.072+0000 7fe4d67fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4d8103ec0 0x7fe4d81969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:37.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.072+0000 7fe4d67fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4d8103ec0 0x7fe4d81969a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51904/0 (socket says 172.21.15.16:51904) 2024-04-02T04:42:37.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.072+0000 7fe4d67fc700 1 -- 172.21.15.16:0/1720423253 learned_addr learned my addr 172.21.15.16:0/1720423253 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:42:37.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.072+0000 7fe4d6ffd700 1 --2- 172.21.15.16:0/1720423253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4d8101590 0x7fe4d8196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:37.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.073+0000 7fe4d67fc700 1 -- 172.21.15.16:0/1720423253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4d8101590 msgr2=0x7fe4d8196460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:37.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.073+0000 7fe4d67fc700 1 --2- 172.21.15.16:0/1720423253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4d8101590 0x7fe4d8196460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:37.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.073+0000 7fe4d67fc700 1 -- 172.21.15.16:0/1720423253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c8009d00 con 0x7fe4d8103ec0 2024-04-02T04:42:37.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.073+0000 7fe4d67fc700 1 --2- 172.21.15.16:0/1720423253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4d8103ec0 0x7fe4d81969a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00e920 tx=0x7fe4cc00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:42:37.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.073+0000 7fe4bffff700 1 -- 172.21.15.16:0/1720423253 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4cc00c9d0 con 0x7fe4d8103ec0 2024-04-02T04:42:37.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.073+0000 7fe4bffff700 1 -- 172.21.15.16:0/1720423253 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe4cc010430 con 0x7fe4d8103ec0 2024-04-02T04:42:37.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.073+0000 7fe4bffff700 1 -- 172.21.15.16:0/1720423253 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4cc018630 con 0x7fe4d8103ec0 2024-04-02T04:42:37.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.073+0000 7fe4dd777700 1 -- 172.21.15.16:0/1720423253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4d819bb60 con 0x7fe4d8103ec0 2024-04-02T04:42:37.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.073+0000 7fe4dd777700 1 -- 172.21.15.16:0/1720423253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d819c130 con 0x7fe4d8103ec0 2024-04-02T04:42:37.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.074+0000 7fe4dd777700 1 -- 172.21.15.16:0/1720423253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4d804ea80 con 0x7fe4d8103ec0 2024-04-02T04:42:37.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.075+0000 7fe4bffff700 1 -- 172.21.15.16:0/1720423253 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 91380+0+0 (secure 0 0 0) 0x7fe4cc0105a0 con 0x7fe4d8103ec0 2024-04-02T04:42:37.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.075+0000 7fe4bffff700 1 --2- 172.21.15.16:0/1720423253 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7fe4c006d4a0 0x7fe4c006f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:37.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.075+0000 7fe4bffff700 1 -- 172.21.15.16:0/1720423253 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fe4cc014070 con 0x7fe4d8103ec0 2024-04-02T04:42:37.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.075+0000 7fe4d6ffd700 1 --2- 172.21.15.16:0/1720423253 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7fe4c006d4a0 0x7fe4c006f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:37.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.076+0000 7fe4d6ffd700 1 --2- 172.21.15.16:0/1720423253 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7fe4c006d4a0 0x7fe4c006f960 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8007710 tx=0x7fe4c8006000 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:42:37.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.080+0000 7fe4bffff700 1 -- 172.21.15.16:0/1720423253 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184299 (secure 0 0 0) 0x7fe4cc053000 con 0x7fe4d8103ec0 2024-04-02T04:42:37.184 INFO:tasks.workunit.client.0.smithi016.stdout:3/379: mkdir d2/d15/d16/d57/d76 0 2024-04-02T04:42:37.184 INFO:tasks.workunit.client.0.smithi016.stdout:2/272: dwrite d12/d20/f2f [0,4194304] 0 2024-04-02T04:42:37.187 INFO:tasks.workunit.client.0.smithi016.stdout:3/380: read d2/d15/f1b [3039003,117450] 0 2024-04-02T04:42:37.190 INFO:tasks.workunit.client.0.smithi016.stdout:2/273: write d12/d35/d3b/f60 [16946,119674] 0 2024-04-02T04:42:37.190 INFO:tasks.workunit.client.0.smithi016.stdout:6/339: dwrite d0/d9/d16/f5b [0,4194304] 0 2024-04-02T04:42:37.192 INFO:tasks.workunit.client.0.smithi016.stdout:3/381: readlink d2/d15/d1a/l51 0 2024-04-02T04:42:37.194 INFO:tasks.workunit.client.0.smithi016.stdout:3/382: dread d2/d15/d16/f25 [0,4194304] 0 2024-04-02T04:42:37.194 INFO:tasks.workunit.client.0.smithi016.stdout:9/328: dwrite d0/d16/f3a [0,4194304] 0 2024-04-02T04:42:37.197 INFO:tasks.workunit.client.0.smithi016.stdout:2/274: creat d12/d35/d3b/f63 x:0 0 0 2024-04-02T04:42:37.200 INFO:tasks.workunit.client.0.smithi016.stdout:8/321: getdents d1/d6/d36/d3f/d40/d45 0 2024-04-02T04:42:37.208 INFO:tasks.workunit.client.0.smithi016.stdout:8/322: creat d1/d6/d36/d3f/d40/d45/f6e x:0 0 0 2024-04-02T04:42:37.210 INFO:tasks.workunit.client.0.smithi016.stdout:9/329: symlink d0/l5e 0 2024-04-02T04:42:37.211 INFO:tasks.workunit.client.0.smithi016.stdout:9/330: stat d0/d16/d4b 0 2024-04-02T04:42:37.211 INFO:tasks.workunit.client.0.smithi016.stdout:9/331: fsync d0/d9/db/f1c 0 2024-04-02T04:42:37.214 INFO:tasks.workunit.client.0.smithi016.stdout:8/323: creat d1/d6/d36/d3f/d40/d49/d38/f6f x:0 0 0 2024-04-02T04:42:37.215 INFO:tasks.workunit.client.0.smithi016.stdout:8/324: chown d1/d6/d36/d3f/d40/d49/f43 2370 1 2024-04-02T04:42:37.217 INFO:tasks.workunit.client.0.smithi016.stdout:9/332: rmdir d0/d3 39 2024-04-02T04:42:37.218 INFO:tasks.workunit.client.0.smithi016.stdout:9/333: truncate d0/d33/f35 898743 0 2024-04-02T04:42:37.220 INFO:tasks.workunit.client.0.smithi016.stdout:8/325: mkdir d1/d70 0 2024-04-02T04:42:37.222 INFO:tasks.workunit.client.0.smithi016.stdout:9/334: creat d0/d9/d4d/f5f x:0 0 0 2024-04-02T04:42:37.223 INFO:tasks.workunit.client.0.smithi016.stdout:8/326: mkdir d1/d70/d71 0 2024-04-02T04:42:37.228 INFO:tasks.workunit.client.0.smithi016.stdout:8/327: dread d1/d6/f25 [0,4194304] 0 2024-04-02T04:42:37.230 INFO:tasks.workunit.client.0.smithi016.stdout:8/328: mknod d1/d31/d67/c72 0 2024-04-02T04:42:37.235 INFO:tasks.workunit.client.0.smithi016.stdout:8/329: rename d1/le to d1/d6/d36/d4f/d5d/l73 0 2024-04-02T04:42:37.236 INFO:tasks.workunit.client.0.smithi016.stdout:8/330: fdatasync d1/d6/f21 0 2024-04-02T04:42:37.242 INFO:tasks.workunit.client.0.smithi016.stdout:8/331: dwrite d1/d6/d36/d3f/d40/d49/f2b [0,4194304] 0 2024-04-02T04:42:37.248 INFO:tasks.workunit.client.0.smithi016.stdout:6/340: truncate d0/d1f/f45 1079320 0 2024-04-02T04:42:37.248 INFO:tasks.workunit.client.0.smithi016.stdout:6/341: chown d0/c5 2 1 2024-04-02T04:42:37.250 INFO:tasks.workunit.client.0.smithi016.stdout:2/275: truncate d12/d20/f25 375245 0 2024-04-02T04:42:37.253 INFO:tasks.workunit.client.0.smithi016.stdout:2/276: creat d12/d20/f64 x:0 0 0 2024-04-02T04:42:37.255 INFO:tasks.workunit.client.0.smithi016.stdout:2/277: chown d12/d35/d3b/f63 3 1 2024-04-02T04:42:37.255 INFO:tasks.workunit.client.0.smithi016.stdout:9/335: write d0/d16/d1a/f48 [457709,90343] 0 2024-04-02T04:42:37.256 INFO:tasks.workunit.client.0.smithi016.stdout:6/342: dwrite d0/d9/d16/f23 [0,4194304] 0 2024-04-02T04:42:37.257 INFO:tasks.workunit.client.0.smithi016.stdout:2/278: dread - d12/d35/d3f/d2b/f3d zero size 2024-04-02T04:42:37.260 INFO:tasks.workunit.client.0.smithi016.stdout:6/343: mknod d0/d5c/c69 0 2024-04-02T04:42:37.261 INFO:tasks.workunit.client.0.smithi016.stdout:2/279: creat d12/d35/d3b/d42/f65 x:0 0 0 2024-04-02T04:42:37.267 INFO:tasks.workunit.client.0.smithi016.stdout:2/280: mkdir d12/d35/d3f/d56/d66 0 2024-04-02T04:42:37.267 INFO:tasks.workunit.client.0.smithi016.stdout:9/336: dwrite d0/d9/f2e [0,4194304] 0 2024-04-02T04:42:37.268 INFO:tasks.workunit.client.0.smithi016.stdout:2/281: readlink d12/d35/d3f/d2b/l62 0 2024-04-02T04:42:37.269 INFO:tasks.workunit.client.0.smithi016.stdout:6/344: link d0/f37 d0/d1f/d67/f6a 0 2024-04-02T04:42:37.269 INFO:tasks.workunit.client.0.smithi016.stdout:2/282: write d12/d20/f44 [759468,69614] 0 2024-04-02T04:42:37.270 INFO:tasks.workunit.client.0.smithi016.stdout:2/283: chown d12/d35/d3f/d2b/f3d 58211 1 2024-04-02T04:42:37.280 INFO:tasks.workunit.client.0.smithi016.stdout:6/345: rename d0/f4 to d0/d5c/f6b 0 2024-04-02T04:42:37.289 INFO:tasks.workunit.client.0.smithi016.stdout:6/346: write d0/f6 [626964,9140] 0 2024-04-02T04:42:37.289 INFO:tasks.workunit.client.0.smithi016.stdout:6/347: chown d0/d9/d16/d56/f4f 28947828 1 2024-04-02T04:42:37.292 INFO:tasks.workunit.client.0.smithi016.stdout:6/348: creat d0/d5c/f6c x:0 0 0 2024-04-02T04:42:37.293 INFO:tasks.workunit.client.0.smithi016.stdout:6/349: write d0/d1f/f3a [4344342,601] 0 2024-04-02T04:42:37.298 INFO:tasks.workunit.client.0.smithi016.stdout:9/337: write d0/d9/f3d [77902,30477] 0 2024-04-02T04:42:37.301 INFO:tasks.workunit.client.0.smithi016.stdout:9/338: dread - d0/d3/d31/f57 zero size 2024-04-02T04:42:37.303 INFO:tasks.workunit.client.0.smithi016.stdout:9/339: mknod d0/d16/d1a/d46/c60 0 2024-04-02T04:42:37.304 INFO:tasks.workunit.client.0.smithi016.stdout:2/284: dread d12/d14/f1f [0,4194304] 0 2024-04-02T04:42:37.304 INFO:tasks.workunit.client.0.smithi016.stdout:6/350: dread d0/f3f [0,4194304] 0 2024-04-02T04:42:37.305 INFO:tasks.workunit.client.0.smithi016.stdout:2/285: dread d12/f43 [0,4194304] 0 2024-04-02T04:42:37.309 INFO:tasks.workunit.client.0.smithi016.stdout:2/286: mkdir d12/d35/d3b/d42/d67 0 2024-04-02T04:42:37.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:37 smithi016 ceph-mon[28969]: pgmap v17: 65 pgs: 65 active+clean; 656 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 6.1 MiB/s rd, 67 MiB/s wr, 354 op/s 2024-04-02T04:42:37.310 INFO:tasks.workunit.client.0.smithi016.stdout:6/351: mkdir d0/d1f/d67/d6d 0 2024-04-02T04:42:37.310 INFO:tasks.workunit.client.0.smithi016.stdout:9/340: dwrite d0/d9/db/f1c [4194304,4194304] 0 2024-04-02T04:42:37.311 INFO:tasks.workunit.client.0.smithi016.stdout:2/287: write d12/d26/f2c [1626371,102928] 0 2024-04-02T04:42:37.312 INFO:tasks.workunit.client.0.smithi016.stdout:9/341: write d0/d3/d31/f45 [864079,29257] 0 2024-04-02T04:42:37.318 INFO:tasks.workunit.client.0.smithi016.stdout:9/342: rmdir d0/d16/d1a/d46 39 2024-04-02T04:42:37.321 INFO:tasks.workunit.client.0.smithi016.stdout:6/352: truncate d0/d9/d16/f47 1910630 0 2024-04-02T04:42:37.327 INFO:tasks.workunit.client.0.smithi016.stdout:6/353: dwrite d0/d1f/f42 [0,4194304] 0 2024-04-02T04:42:37.331 INFO:tasks.workunit.client.0.smithi016.stdout:6/354: dread d0/f3f [0,4194304] 0 2024-04-02T04:42:37.336 INFO:tasks.workunit.client.0.smithi016.stdout:6/355: mkdir d0/d6e 0 2024-04-02T04:42:37.336 INFO:tasks.workunit.client.0.smithi016.stdout:9/343: write d0/f27 [244264,15703] 0 2024-04-02T04:42:37.339 INFO:tasks.workunit.client.0.smithi016.stdout:9/344: creat d0/d4e/f61 x:0 0 0 2024-04-02T04:42:37.345 INFO:tasks.workunit.client.0.smithi016.stdout:6/356: write d0/f37 [5215365,119508] 0 2024-04-02T04:42:37.346 INFO:tasks.workunit.client.0.smithi016.stdout:5/348: sync 2024-04-02T04:42:37.346 INFO:tasks.workunit.client.0.smithi016.stdout:7/319: sync 2024-04-02T04:42:37.349 INFO:tasks.workunit.client.0.smithi016.stdout:7/320: symlink d0/d46/d37/d3a/l7a 0 2024-04-02T04:42:37.351 INFO:tasks.workunit.client.0.smithi016.stdout:5/349: mkdir d3/df/d86 0 2024-04-02T04:42:37.351 INFO:tasks.workunit.client.0.smithi016.stdout:5/350: chown d3/f77 671284035 1 2024-04-02T04:42:37.352 INFO:tasks.workunit.client.0.smithi016.stdout:7/321: creat d0/d46/d37/d3a/d6f/f7b x:0 0 0 2024-04-02T04:42:37.355 INFO:tasks.workunit.client.0.smithi016.stdout:5/351: symlink d3/d40/d4f/d67/l87 0 2024-04-02T04:42:37.356 INFO:tasks.workunit.client.0.smithi016.stdout:7/322: creat d0/d28/f7c x:0 0 0 2024-04-02T04:42:37.357 INFO:tasks.workunit.client.0.smithi016.stdout:7/323: write d0/d46/d52/f61 [1539563,107421] 0 2024-04-02T04:42:37.360 INFO:tasks.workunit.client.0.smithi016.stdout:5/352: symlink d3/df/d86/l88 0 2024-04-02T04:42:37.362 INFO:tasks.workunit.client.0.smithi016.stdout:6/357: dwrite d0/d9/f32 [0,4194304] 0 2024-04-02T04:42:37.363 INFO:tasks.workunit.client.0.smithi016.stdout:7/324: rename d0/d46/d52/f6a to d0/d46/d37/d3a/f7d 0 2024-04-02T04:42:37.363 INFO:tasks.workunit.client.0.smithi016.stdout:9/345: dread d0/d3/f30 [0,4194304] 0 2024-04-02T04:42:37.364 INFO:tasks.workunit.client.0.smithi016.stdout:9/346: stat d0/d16 0 2024-04-02T04:42:37.365 INFO:tasks.workunit.client.0.smithi016.stdout:9/347: chown d0/d16/d1a/f1f 881102771 1 2024-04-02T04:42:37.365 INFO:tasks.workunit.client.0.smithi016.stdout:6/358: mkdir d0/d1f/d67/d6f 0 2024-04-02T04:42:37.367 INFO:tasks.workunit.client.0.smithi016.stdout:6/359: write d0/f6 [1478980,90256] 0 2024-04-02T04:42:37.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.368+0000 7fe4dd777700 1 -- 172.21.15.16:0/1720423253 --> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe4d8105d60 con 0x7fe4c006d4a0 2024-04-02T04:42:37.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.371+0000 7fe4bffff700 1 -- 172.21.15.16:0/1720423253 <== mgr.24415 v2:172.21.15.27:6828/1554339939 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7fe4d8105d60 con 0x7fe4c006d4a0 2024-04-02T04:42:37.377 INFO:tasks.workunit.client.0.smithi016.stdout:9/348: link d0/d9/db/l15 d0/d16/l62 0 2024-04-02T04:42:37.377 INFO:tasks.workunit.client.0.smithi016.stdout:7/325: dread d0/d46/d52/f61 [0,4194304] 0 2024-04-02T04:42:37.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.374+0000 7fe4dd777700 1 -- 172.21.15.16:0/1720423253 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7fe4c006d4a0 msgr2=0x7fe4c006f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:37.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.374+0000 7fe4dd777700 1 --2- 172.21.15.16:0/1720423253 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7fe4c006d4a0 0x7fe4c006f960 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8007710 tx=0x7fe4c8006000 comp rx=0 tx=0).stop 2024-04-02T04:42:37.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.374+0000 7fe4dd777700 1 -- 172.21.15.16:0/1720423253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4d8103ec0 msgr2=0x7fe4d81969a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:37.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.374+0000 7fe4dd777700 1 --2- 172.21.15.16:0/1720423253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4d8103ec0 0x7fe4d81969a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00e920 tx=0x7fe4cc00ec30 comp rx=0 tx=0).stop 2024-04-02T04:42:37.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.374+0000 7fe4dd777700 1 -- 172.21.15.16:0/1720423253 shutdown_connections 2024-04-02T04:42:37.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.374+0000 7fe4dd777700 1 --2- 172.21.15.16:0/1720423253 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7fe4c006d4a0 0x7fe4c006f960 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:37.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.374+0000 7fe4dd777700 1 --2- 172.21.15.16:0/1720423253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4d8101590 0x7fe4d8196460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:37.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.374+0000 7fe4dd777700 1 --2- 172.21.15.16:0/1720423253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4d8103ec0 0x7fe4d81969a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:37.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.374+0000 7fe4dd777700 1 -- 172.21.15.16:0/1720423253 >> 172.21.15.16:0/1720423253 conn(0x7fe4d80fafc0 msgr2=0x7fe4d80fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:42:37.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.374+0000 7fe4dd777700 1 -- 172.21.15.16:0/1720423253 shutdown_connections 2024-04-02T04:42:37.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.374+0000 7fe4dd777700 1 -- 172.21.15.16:0/1720423253 wait complete. 2024-04-02T04:42:37.379 INFO:tasks.workunit.client.0.smithi016.stdout:9/349: fdatasync d0/d9/f42 0 2024-04-02T04:42:37.379 INFO:tasks.workunit.client.0.smithi016.stdout:7/326: read - d0/d46/d16/d49/f6e zero size 2024-04-02T04:42:37.379 INFO:tasks.workunit.client.0.smithi016.stdout:7/327: chown d0/d46/d37/d3a/d4c/l67 369 1 2024-04-02T04:42:37.380 INFO:tasks.workunit.client.0.smithi016.stdout:7/328: dread - d0/d46/d16/d49/f6e zero size 2024-04-02T04:42:37.385 INFO:tasks.workunit.client.0.smithi016.stdout:9/350: getdents d0/d16/d4b 0 2024-04-02T04:42:37.386 INFO:tasks.workunit.client.0.smithi016.stdout:6/360: truncate d0/d9/d16/d56/d59/f4b 3949549 0 2024-04-02T04:42:37.387 INFO:tasks.workunit.client.0.smithi016.stdout:6/361: dread d0/d9/d16/d46/d5e/f64 [0,4194304] 0 2024-04-02T04:42:37.389 INFO:tasks.workunit.client.0.smithi016.stdout:7/329: getdents d0/d46/d16/d49 0 2024-04-02T04:42:37.395 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:37 smithi027 ceph-mon[37550]: pgmap v17: 65 pgs: 65 active+clean; 656 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 6.1 MiB/s rd, 67 MiB/s wr, 354 op/s 2024-04-02T04:42:37.396 INFO:tasks.workunit.client.0.smithi016.stdout:7/330: unlink d0/d28/d64/l78 0 2024-04-02T04:42:37.401 INFO:tasks.workunit.client.0.smithi016.stdout:7/331: dread d0/d46/d37/d3a/d4c/f73 [0,4194304] 0 2024-04-02T04:42:37.403 INFO:tasks.workunit.client.0.smithi016.stdout:6/362: dread d0/fc [0,4194304] 0 2024-04-02T04:42:37.404 INFO:tasks.workunit.client.0.smithi016.stdout:7/332: rmdir d0/d46/d16 39 2024-04-02T04:42:37.404 INFO:tasks.workunit.client.0.smithi016.stdout:7/333: dread - d0/d21/f36 zero size 2024-04-02T04:42:37.406 INFO:tasks.workunit.client.0.smithi016.stdout:6/363: rename d0/d9/d16/f44 to d0/d6e/f70 0 2024-04-02T04:42:37.407 INFO:tasks.workunit.client.0.smithi016.stdout:6/364: readlink d0/l66 0 2024-04-02T04:42:37.407 INFO:tasks.workunit.client.0.smithi016.stdout:6/365: chown d0/d9/d16/f2a 58501982 1 2024-04-02T04:42:37.408 INFO:tasks.workunit.client.0.smithi016.stdout:7/334: dread d0/d46/d37/d3a/d4c/f73 [0,4194304] 0 2024-04-02T04:42:37.451 INFO:tasks.workunit.client.0.smithi016.stdout:0/292: sync 2024-04-02T04:42:37.453 INFO:tasks.workunit.client.0.smithi016.stdout:7/335: write d0/d46/fc [3303525,51308] 0 2024-04-02T04:42:37.455 INFO:tasks.workunit.client.0.smithi016.stdout:0/293: creat d0/d16/d2e/d53/f65 x:0 0 0 2024-04-02T04:42:37.458 INFO:tasks.workunit.client.0.smithi016.stdout:6/366: dwrite d0/d9/d16/f2a [4194304,4194304] 0 2024-04-02T04:42:37.462 INFO:tasks.workunit.client.0.smithi016.stdout:6/367: chown d0/d1f/f45 0 1 2024-04-02T04:42:37.462 INFO:tasks.workunit.client.0.smithi016.stdout:0/294: dwrite d0/f10 [0,4194304] 0 2024-04-02T04:42:37.464 INFO:tasks.workunit.client.0.smithi016.stdout:0/295: chown d0/d16/d24/c46 18 1 2024-04-02T04:42:37.466 INFO:tasks.workunit.client.0.smithi016.stdout:6/368: symlink d0/d9/d16/d46/d5e/l71 0 2024-04-02T04:42:37.466 INFO:tasks.workunit.client.0.smithi016.stdout:6/369: chown d0/d9/fd 1994243 1 2024-04-02T04:42:37.469 INFO:tasks.workunit.client.0.smithi016.stdout:6/370: read d0/f37 [2189934,62738] 0 2024-04-02T04:42:37.469 INFO:tasks.workunit.client.0.smithi016.stdout:0/296: unlink d0/d16/d2e/d36/f44 0 2024-04-02T04:42:37.481 INFO:tasks.workunit.client.0.smithi016.stdout:6/371: link d0/d9/d16/d56/d59/f4b d0/d9/df/f72 0 2024-04-02T04:42:37.482 INFO:tasks.workunit.client.0.smithi016.stdout:6/372: write d0/d9/d16/f5b [622328,19547] 0 2024-04-02T04:42:37.489 INFO:tasks.workunit.client.0.smithi016.stdout:6/373: rename d0/d6e/f70 to d0/f73 0 2024-04-02T04:42:37.493 INFO:tasks.workunit.client.0.smithi016.stdout:6/374: creat d0/d9/df/f74 x:0 0 0 2024-04-02T04:42:37.497 INFO:tasks.workunit.client.0.smithi016.stdout:6/375: dwrite d0/d9/d16/f2a [4194304,4194304] 0 2024-04-02T04:42:37.499 INFO:tasks.workunit.client.0.smithi016.stdout:6/376: chown d0/d9/d16/d56/d59/f60 1721496 1 2024-04-02T04:42:37.499 INFO:tasks.workunit.client.0.smithi016.stdout:6/377: write d0/d9/fd [4705408,7850] 0 2024-04-02T04:42:37.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.497+0000 7f12a4f38700 1 -- 172.21.15.16:0/421268292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12a00ff9d0 msgr2=0x7f12a00ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:37.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.497+0000 7f12a4f38700 1 --2- 172.21.15.16:0/421268292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12a00ff9d0 0x7f12a00ffdf0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1290009910 tx=0x7f1290009c20 comp rx=0 tx=0).stop 2024-04-02T04:42:37.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.498+0000 7f12a4f38700 1 -- 172.21.15.16:0/421268292 shutdown_connections 2024-04-02T04:42:37.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.498+0000 7f12a4f38700 1 --2- 172.21.15.16:0/421268292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12a0100bd0 0x7f12a0101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:37.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.498+0000 7f12a4f38700 1 --2- 172.21.15.16:0/421268292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12a00ff9d0 0x7f12a00ffdf0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:37.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.498+0000 7f12a4f38700 1 -- 172.21.15.16:0/421268292 >> 172.21.15.16:0/421268292 conn(0x7f12a00faf70 msgr2=0x7f12a00fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:42:37.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.498+0000 7f12a4f38700 1 -- 172.21.15.16:0/421268292 shutdown_connections 2024-04-02T04:42:37.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.498+0000 7f12a4f38700 1 -- 172.21.15.16:0/421268292 wait complete. 2024-04-02T04:42:37.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.498+0000 7f12a4f38700 1 Processor -- start 2024-04-02T04:42:37.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.499+0000 7f12a4f38700 1 -- start start 2024-04-02T04:42:37.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.499+0000 7f12a4f38700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12a00ff9d0 0x7f12a01953f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:37.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.499+0000 7f12a4f38700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12a0100bd0 0x7f12a0195930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:37.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.499+0000 7f12a4f38700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a0195f50 con 0x7f12a0100bd0 2024-04-02T04:42:37.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.499+0000 7f12a4f38700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a0196090 con 0x7f12a00ff9d0 2024-04-02T04:42:37.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.499+0000 7f129dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12a0100bd0 0x7f12a0195930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:37.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.499+0000 7f129dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12a0100bd0 0x7f12a0195930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51914/0 (socket says 172.21.15.16:51914) 2024-04-02T04:42:37.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.499+0000 7f129dd9b700 1 -- 172.21.15.16:0/2138895213 learned_addr learned my addr 172.21.15.16:0/2138895213 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:42:37.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.499+0000 7f129e59c700 1 --2- 172.21.15.16:0/2138895213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12a00ff9d0 0x7f12a01953f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:37.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.499+0000 7f129dd9b700 1 -- 172.21.15.16:0/2138895213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12a00ff9d0 msgr2=0x7f12a01953f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:37.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.499+0000 7f129dd9b700 1 --2- 172.21.15.16:0/2138895213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12a00ff9d0 0x7f12a01953f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:37.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.499+0000 7f129dd9b700 1 -- 172.21.15.16:0/2138895213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12900095f0 con 0x7f12a0100bd0 2024-04-02T04:42:37.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.499+0000 7f129dd9b700 1 --2- 172.21.15.16:0/2138895213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12a0100bd0 0x7f12a0195930 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f129400d8d0 tx=0x7f129400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:42:37.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.500+0000 7f128f7fe700 1 -- 172.21.15.16:0/2138895213 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12940098a0 con 0x7f12a0100bd0 2024-04-02T04:42:37.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.500+0000 7f128f7fe700 1 -- 172.21.15.16:0/2138895213 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1294010460 con 0x7f12a0100bd0 2024-04-02T04:42:37.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.500+0000 7f128f7fe700 1 -- 172.21.15.16:0/2138895213 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f129400f5f0 con 0x7f12a0100bd0 2024-04-02T04:42:37.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.500+0000 7f12a4f38700 1 -- 172.21.15.16:0/2138895213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a0068d70 con 0x7f12a0100bd0 2024-04-02T04:42:37.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.500+0000 7f12a4f38700 1 -- 172.21.15.16:0/2138895213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a0069340 con 0x7f12a0100bd0 2024-04-02T04:42:37.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.501+0000 7f12a4f38700 1 -- 172.21.15.16:0/2138895213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12a004ea80 con 0x7f12a0100bd0 2024-04-02T04:42:37.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.501+0000 7f128f7fe700 1 -- 172.21.15.16:0/2138895213 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 91380+0+0 (secure 0 0 0) 0x7f1294009a00 con 0x7f12a0100bd0 2024-04-02T04:42:37.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.502+0000 7f128f7fe700 1 --2- 172.21.15.16:0/2138895213 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7f128806d4a0 0x7f128806f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:37.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.502+0000 7f128f7fe700 1 -- 172.21.15.16:0/2138895213 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f129408ccb0 con 0x7f12a0100bd0 2024-04-02T04:42:37.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.502+0000 7f129e59c700 1 --2- 172.21.15.16:0/2138895213 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7f128806d4a0 0x7f128806f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:37.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.504+0000 7f129e59c700 1 --2- 172.21.15.16:0/2138895213 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7f128806d4a0 0x7f128806f960 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1290000c00 tx=0x7f1290005960 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:42:37.508 INFO:tasks.workunit.client.0.smithi016.stdout:7/336: truncate d0/d46/d16/d49/fa 5389950 0 2024-04-02T04:42:37.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.509+0000 7f128f7fe700 1 -- 172.21.15.16:0/2138895213 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184299 (secure 0 0 0) 0x7f129405a200 con 0x7f12a0100bd0 2024-04-02T04:42:37.512 INFO:tasks.workunit.client.0.smithi016.stdout:7/337: mknod d0/d21/c7e 0 2024-04-02T04:42:37.515 INFO:tasks.workunit.client.0.smithi016.stdout:7/338: mknod d0/d46/d37/d3a/c7f 0 2024-04-02T04:42:37.515 INFO:tasks.workunit.client.0.smithi016.stdout:6/378: dread d0/d1f/d67/f6a [0,4194304] 0 2024-04-02T04:42:37.516 INFO:tasks.workunit.client.0.smithi016.stdout:6/379: fsync d0/f5d 0 2024-04-02T04:42:37.517 INFO:tasks.workunit.client.0.smithi016.stdout:6/380: readlink d0/d9/d16/d56/l15 0 2024-04-02T04:42:37.517 INFO:tasks.workunit.client.0.smithi016.stdout:6/381: truncate d0/f5d 641016 0 2024-04-02T04:42:37.526 INFO:tasks.workunit.client.0.smithi016.stdout:6/382: creat d0/d1f/d67/d6f/f75 x:0 0 0 2024-04-02T04:42:37.546 INFO:tasks.workunit.client.0.smithi016.stdout:6/383: dwrite d0/d1f/d67/f6a [0,4194304] 0 2024-04-02T04:42:37.549 INFO:tasks.workunit.client.0.smithi016.stdout:6/384: chown d0/d9/df/f72 6 1 2024-04-02T04:42:37.550 INFO:tasks.workunit.client.0.smithi016.stdout:6/385: write d0/d1f/f2d [1109399,49124] 0 2024-04-02T04:42:37.555 INFO:tasks.workunit.client.0.smithi016.stdout:6/386: creat d0/d5c/f76 x:0 0 0 2024-04-02T04:42:37.557 INFO:tasks.workunit.client.0.smithi016.stdout:7/339: write d0/d46/d16/d49/fa [4407844,14670] 0 2024-04-02T04:42:37.558 INFO:tasks.workunit.client.0.smithi016.stdout:7/340: write d0/d28/f42 [956466,880] 0 2024-04-02T04:42:37.559 INFO:tasks.workunit.client.0.smithi016.stdout:7/341: chown d0/d28/l29 15 1 2024-04-02T04:42:37.564 INFO:tasks.workunit.client.0.smithi016.stdout:7/342: write d0/d46/f4d [2185397,2948] 0 2024-04-02T04:42:37.565 INFO:tasks.workunit.client.0.smithi016.stdout:6/387: link d0/d1f/l4c d0/d9/d16/l77 0 2024-04-02T04:42:37.565 INFO:tasks.workunit.client.0.smithi016.stdout:6/388: chown d0/d9/f36 1651017985 1 2024-04-02T04:42:37.569 INFO:tasks.workunit.client.0.smithi016.stdout:4/326: sync 2024-04-02T04:42:37.569 INFO:tasks.workunit.client.0.smithi016.stdout:8/332: sync 2024-04-02T04:42:37.569 INFO:tasks.workunit.client.0.smithi016.stdout:7/343: dread d0/d28/f59 [0,4194304] 0 2024-04-02T04:42:37.571 INFO:tasks.workunit.client.0.smithi016.stdout:8/333: dread - d1/d6/d36/d3f/d40/d49/d38/f6f zero size 2024-04-02T04:42:37.572 INFO:tasks.workunit.client.0.smithi016.stdout:4/327: fdatasync d5/d17/d18/d25/f57 0 2024-04-02T04:42:37.573 INFO:tasks.workunit.client.0.smithi016.stdout:8/334: dread d1/d6/d36/d3f/d40/d49/f33 [0,4194304] 0 2024-04-02T04:42:37.574 INFO:tasks.workunit.client.0.smithi016.stdout:6/389: dwrite d0/d5c/f6b [0,4194304] 0 2024-04-02T04:42:37.580 INFO:tasks.workunit.client.0.smithi016.stdout:6/390: dread d0/f3f [0,4194304] 0 2024-04-02T04:42:37.581 INFO:tasks.workunit.client.0.smithi016.stdout:6/391: write d0/f6 [4695726,30679] 0 2024-04-02T04:42:37.582 INFO:tasks.workunit.client.0.smithi016.stdout:7/344: dwrite d0/d28/f59 [0,4194304] 0 2024-04-02T04:42:37.583 INFO:tasks.workunit.client.0.smithi016.stdout:7/345: write d0/d21/f4e [1032278,36020] 0 2024-04-02T04:42:37.585 INFO:tasks.workunit.client.0.smithi016.stdout:8/335: chown d1/d31/l3a 11042 1 2024-04-02T04:42:37.586 INFO:tasks.workunit.client.0.smithi016.stdout:8/336: read - d1/d6/d36/d3f/f6c zero size 2024-04-02T04:42:37.593 INFO:tasks.workunit.client.0.smithi016.stdout:8/337: chown d1/ff 126644 1 2024-04-02T04:42:37.594 INFO:tasks.workunit.client.0.smithi016.stdout:8/338: write d1/f3 [2338854,130072] 0 2024-04-02T04:42:37.594 INFO:tasks.workunit.client.0.smithi016.stdout:6/392: mkdir d0/d78 0 2024-04-02T04:42:37.598 INFO:tasks.workunit.client.0.smithi016.stdout:4/328: rename d5/d8/d43/f5b to d5/d1d/f61 0 2024-04-02T04:42:37.600 INFO:tasks.workunit.client.0.smithi016.stdout:8/339: fsync d1/d6/d36/d4f/f62 0 2024-04-02T04:42:37.601 INFO:tasks.workunit.client.0.smithi016.stdout:6/393: mknod d0/d9/d16/d56/c79 0 2024-04-02T04:42:37.603 INFO:tasks.workunit.client.0.smithi016.stdout:7/346: rename d0/d46/d16/f56 to d0/d46/d52/f80 0 2024-04-02T04:42:37.604 INFO:tasks.workunit.client.0.smithi016.stdout:6/394: dread d0/d5c/f6b [0,4194304] 0 2024-04-02T04:42:37.604 INFO:tasks.workunit.client.0.smithi016.stdout:1/322: sync 2024-04-02T04:42:37.605 INFO:tasks.workunit.client.0.smithi016.stdout:1/323: chown d14/f48 2630 1 2024-04-02T04:42:37.605 INFO:tasks.workunit.client.0.smithi016.stdout:6/395: write d0/f51 [1428685,52915] 0 2024-04-02T04:42:37.606 INFO:tasks.workunit.client.0.smithi016.stdout:6/396: chown d0/d9/d16 7 1 2024-04-02T04:42:37.607 INFO:tasks.workunit.client.0.smithi016.stdout:6/397: write d0/d1f/d67/f6a [3341451,44730] 0 2024-04-02T04:42:37.608 INFO:tasks.workunit.client.0.smithi016.stdout:6/398: chown d0/f5d 234 1 2024-04-02T04:42:37.611 INFO:tasks.workunit.client.0.smithi016.stdout:8/340: mkdir d1/d70/d71/d74 0 2024-04-02T04:42:37.613 INFO:tasks.workunit.client.0.smithi016.stdout:1/324: creat d14/d19/d20/f62 x:0 0 0 2024-04-02T04:42:37.614 INFO:tasks.workunit.client.0.smithi016.stdout:8/341: fdatasync d1/d6/d36/d3f/d40/d49/f68 0 2024-04-02T04:42:37.614 INFO:tasks.workunit.client.0.smithi016.stdout:6/399: dwrite d0/d9/d16/f20 [4194304,4194304] 0 2024-04-02T04:42:37.616 INFO:tasks.workunit.client.0.smithi016.stdout:6/400: fsync d0/d9/d16/d56/f4f 0 2024-04-02T04:42:37.623 INFO:tasks.workunit.client.0.smithi016.stdout:8/342: dwrite d1/d6/f53 [0,4194304] 0 2024-04-02T04:42:37.625 INFO:tasks.workunit.client.0.smithi016.stdout:6/401: dwrite d0/f41 [0,4194304] 0 2024-04-02T04:42:37.628 INFO:tasks.workunit.client.0.smithi016.stdout:1/325: mknod d14/d19/d1d/d45/d4f/c63 0 2024-04-02T04:42:37.631 INFO:tasks.workunit.client.0.smithi016.stdout:1/326: dread - d14/d19/d20/f3b zero size 2024-04-02T04:42:37.637 INFO:tasks.workunit.client.0.smithi016.stdout:8/343: dwrite d1/d6/f2e [0,4194304] 0 2024-04-02T04:42:37.640 INFO:tasks.workunit.client.0.smithi016.stdout:6/402: dwrite d0/d5c/f6c [0,4194304] 0 2024-04-02T04:42:37.642 INFO:tasks.workunit.client.0.smithi016.stdout:1/327: mknod d14/d19/d1d/d29/d39/d46/c64 0 2024-04-02T04:42:37.643 INFO:tasks.workunit.client.0.smithi016.stdout:4/329: dwrite d5/d8/d43/f4b [4194304,4194304] 0 2024-04-02T04:42:37.646 INFO:tasks.workunit.client.0.smithi016.stdout:1/328: fsync d14/d19/f60 0 2024-04-02T04:42:37.648 INFO:tasks.workunit.client.0.smithi016.stdout:1/329: dread - d14/d19/d20/f62 zero size 2024-04-02T04:42:37.652 INFO:tasks.workunit.client.0.smithi016.stdout:6/403: dwrite d0/d1f/f3b [0,4194304] 0 2024-04-02T04:42:37.655 INFO:tasks.workunit.client.0.smithi016.stdout:6/404: stat d0/d9/d16/f23 0 2024-04-02T04:42:37.658 INFO:tasks.workunit.client.0.smithi016.stdout:6/405: dread d0/f37 [0,4194304] 0 2024-04-02T04:42:37.659 INFO:tasks.workunit.client.0.smithi016.stdout:4/330: symlink d5/d1d/l62 0 2024-04-02T04:42:37.660 INFO:tasks.workunit.client.0.smithi016.stdout:6/406: write d0/d9/d16/f2a [9279770,130891] 0 2024-04-02T04:42:37.666 INFO:tasks.workunit.client.0.smithi016.stdout:8/344: rename d1/d6/c29 to d1/c75 0 2024-04-02T04:42:37.670 INFO:tasks.workunit.client.0.smithi016.stdout:3/383: sync 2024-04-02T04:42:37.670 INFO:tasks.workunit.client.0.smithi016.stdout:2/288: sync 2024-04-02T04:42:37.670 INFO:tasks.workunit.client.0.smithi016.stdout:5/353: sync 2024-04-02T04:42:37.670 INFO:tasks.workunit.client.0.smithi016.stdout:9/351: sync 2024-04-02T04:42:37.670 INFO:tasks.workunit.client.0.smithi016.stdout:0/297: sync 2024-04-02T04:42:37.674 INFO:tasks.workunit.client.0.smithi016.stdout:5/354: read d3/df/d18/f42 [3120318,20306] 0 2024-04-02T04:42:37.678 INFO:tasks.workunit.client.0.smithi016.stdout:1/330: creat d14/d19/d1d/d29/d39/f65 x:0 0 0 2024-04-02T04:42:37.678 INFO:tasks.workunit.client.0.smithi016.stdout:5/355: truncate d3/d5d/f6f 867150 0 2024-04-02T04:42:37.678 INFO:tasks.workunit.client.0.smithi016.stdout:5/356: chown d3/df/d86 0 1 2024-04-02T04:42:37.678 INFO:tasks.workunit.client.0.smithi016.stdout:8/345: creat d1/d6/d36/d3f/d40/d49/d38/f76 x:0 0 0 2024-04-02T04:42:37.680 INFO:tasks.workunit.client.0.smithi016.stdout:2/289: creat d12/d20/d4f/f68 x:0 0 0 2024-04-02T04:42:37.682 INFO:tasks.workunit.client.0.smithi016.stdout:9/352: mkdir d0/d4e/d50/d63 0 2024-04-02T04:42:37.682 INFO:tasks.workunit.client.0.smithi016.stdout:9/353: fdatasync d0/f2c 0 2024-04-02T04:42:37.688 INFO:tasks.workunit.client.0.smithi016.stdout:1/331: symlink d14/d41/l66 0 2024-04-02T04:42:37.689 INFO:tasks.workunit.client.0.smithi016.stdout:1/332: dread - d14/d41/f47 zero size 2024-04-02T04:42:37.691 INFO:tasks.workunit.client.0.smithi016.stdout:8/346: fdatasync d1/f19 0 2024-04-02T04:42:37.696 INFO:tasks.workunit.client.0.smithi016.stdout:6/407: write d0/d5c/f6b [1158755,30203] 0 2024-04-02T04:42:37.697 INFO:tasks.workunit.client.0.smithi016.stdout:8/347: dwrite d1/d31/f6b [0,4194304] 0 2024-04-02T04:42:37.698 INFO:tasks.workunit.client.0.smithi016.stdout:2/290: dread d12/d35/d3f/f34 [0,4194304] 0 2024-04-02T04:42:37.699 INFO:tasks.workunit.client.0.smithi016.stdout:6/408: chown d0/d9/d16/f3c 72417405 1 2024-04-02T04:42:37.700 INFO:tasks.workunit.client.0.smithi016.stdout:8/348: read d1/d6/d36/d3f/d40/d49/f68 [519257,80952] 0 2024-04-02T04:42:37.704 INFO:tasks.workunit.client.0.smithi016.stdout:1/333: symlink d14/d41/l67 0 2024-04-02T04:42:37.709 INFO:tasks.workunit.client.0.smithi016.stdout:2/291: dwrite d12/d35/d3f/f4b [0,4194304] 0 2024-04-02T04:42:37.711 INFO:tasks.workunit.client.0.smithi016.stdout:4/331: dwrite d5/d17/d18/d1f/f59 [0,4194304] 0 2024-04-02T04:42:37.716 INFO:tasks.workunit.client.0.smithi016.stdout:8/349: dread d1/d6/d36/d3f/d40/d49/f2a [0,4194304] 0 2024-04-02T04:42:37.723 INFO:tasks.workunit.client.0.smithi016.stdout:5/357: creat d3/df/d18/f89 x:0 0 0 2024-04-02T04:42:37.725 INFO:tasks.workunit.client.0.smithi016.stdout:1/334: mkdir d14/d41/d43/d68 0 2024-04-02T04:42:37.729 INFO:tasks.workunit.client.0.smithi016.stdout:2/292: rename d12/d20/f52 to d12/d35/d3f/d2b/f69 0 2024-04-02T04:42:37.731 INFO:tasks.workunit.client.0.smithi016.stdout:1/335: dwrite d14/d19/d1d/f2c [0,4194304] 0 2024-04-02T04:42:37.733 INFO:tasks.workunit.client.0.smithi016.stdout:4/332: creat d5/d17/d18/d1f/f63 x:0 0 0 2024-04-02T04:42:37.735 INFO:tasks.workunit.client.0.smithi016.stdout:3/384: getdents d2/d15 0 2024-04-02T04:42:37.737 INFO:tasks.workunit.client.0.smithi016.stdout:2/293: dwrite d12/d26/f5a [0,4194304] 0 2024-04-02T04:42:37.741 INFO:tasks.workunit.client.0.smithi016.stdout:9/354: truncate d0/f2d 1163320 0 2024-04-02T04:42:37.741 INFO:tasks.workunit.client.0.smithi016.stdout:9/355: dread - d0/d4e/f61 zero size 2024-04-02T04:42:37.742 INFO:tasks.workunit.client.0.smithi016.stdout:6/409: creat d0/d9/f7a x:0 0 0 2024-04-02T04:42:37.743 INFO:tasks.workunit.client.0.smithi016.stdout:9/356: dread - d0/d3/d31/f57 zero size 2024-04-02T04:42:37.748 INFO:tasks.workunit.client.0.smithi016.stdout:4/333: creat d5/d16/d3f/f64 x:0 0 0 2024-04-02T04:42:37.750 INFO:tasks.workunit.client.0.smithi016.stdout:3/385: rmdir d2/d2e/d36/d66 39 2024-04-02T04:42:37.750 INFO:tasks.workunit.client.0.smithi016.stdout:3/386: chown d2/f26 7928070 1 2024-04-02T04:42:37.750 INFO:tasks.workunit.client.0.smithi016.stdout:3/387: readlink d2/l33 0 2024-04-02T04:42:37.752 INFO:tasks.workunit.client.0.smithi016.stdout:8/350: mkdir d1/d6/d36/d3f/d77 0 2024-04-02T04:42:37.754 INFO:tasks.workunit.client.0.smithi016.stdout:2/294: write d12/d14/f1f [2474452,73403] 0 2024-04-02T04:42:37.758 INFO:tasks.workunit.client.0.smithi016.stdout:2/295: dread d12/d26/f5a [0,4194304] 0 2024-04-02T04:42:37.758 INFO:tasks.workunit.client.0.smithi016.stdout:6/410: creat d0/d9/d16/d46/f7b x:0 0 0 2024-04-02T04:42:37.764 INFO:tasks.workunit.client.0.smithi016.stdout:6/411: dread d0/d9/f32 [0,4194304] 0 2024-04-02T04:42:37.764 INFO:tasks.workunit.client.0.smithi016.stdout:2/296: dwrite d12/d14/f3c [0,4194304] 0 2024-04-02T04:42:37.765 INFO:tasks.workunit.client.0.smithi016.stdout:8/351: fsync d1/d6/d36/d3f/d40/d49/f43 0 2024-04-02T04:42:37.766 INFO:tasks.workunit.client.0.smithi016.stdout:6/412: write d0/d9/fd [1896144,80564] 0 2024-04-02T04:42:37.769 INFO:tasks.workunit.client.0.smithi016.stdout:8/352: readlink d1/d6/d36/d3f/d40/d49/l58 0 2024-04-02T04:42:37.771 INFO:tasks.workunit.client.0.smithi016.stdout:9/357: symlink d0/d16/d1a/d46/l64 0 2024-04-02T04:42:37.772 INFO:tasks.workunit.client.0.smithi016.stdout:8/353: write d1/d6/d36/f37 [823626,124390] 0 2024-04-02T04:42:37.776 INFO:tasks.workunit.client.0.smithi016.stdout:1/336: creat d14/d19/f69 x:0 0 0 2024-04-02T04:42:37.779 INFO:tasks.workunit.client.0.smithi016.stdout:4/334: mkdir d5/d1d/d5f/d65 0 2024-04-02T04:42:37.781 INFO:tasks.workunit.client.0.smithi016.stdout:3/388: dread d2/f14 [0,4194304] 0 2024-04-02T04:42:37.782 INFO:tasks.workunit.client.0.smithi016.stdout:3/389: write d2/d15/f4b [759318,76347] 0 2024-04-02T04:42:37.792 INFO:tasks.workunit.client.0.smithi016.stdout:1/337: creat d14/d19/d1d/d29/f6a x:0 0 0 2024-04-02T04:42:37.794 INFO:tasks.workunit.client.0.smithi016.stdout:4/335: mknod d5/d17/c66 0 2024-04-02T04:42:37.795 INFO:tasks.workunit.client.0.smithi016.stdout:2/297: mkdir d12/d35/d3b/d42/d67/d6a 0 2024-04-02T04:42:37.797 INFO:tasks.workunit.client.0.smithi016.stdout:6/413: mkdir d0/d78/d7c 0 2024-04-02T04:42:37.799 INFO:tasks.workunit.client.0.smithi016.stdout:1/338: dwrite d14/d19/f60 [0,4194304] 0 2024-04-02T04:42:37.799 INFO:tasks.workunit.client.0.smithi016.stdout:8/354: creat d1/d70/d71/d74/f78 x:0 0 0 2024-04-02T04:42:37.800 INFO:tasks.workunit.client.0.smithi016.stdout:8/355: fsync d1/d6/d36/d3f/f6c 0 2024-04-02T04:42:37.800 INFO:tasks.workunit.client.0.smithi016.stdout:1/339: write d14/d41/f5c [258060,128527] 0 2024-04-02T04:42:37.800 INFO:tasks.workunit.client.0.smithi016.stdout:8/356: chown d1/l7 4830193 1 2024-04-02T04:42:37.802 INFO:tasks.workunit.client.0.smithi016.stdout:2/298: write d12/d35/d3f/f34 [4636558,36965] 0 2024-04-02T04:42:37.803 INFO:tasks.workunit.client.0.smithi016.stdout:9/358: creat d0/d16/f65 x:0 0 0 2024-04-02T04:42:37.803 INFO:tasks.workunit.client.0.smithi016.stdout:6/414: creat d0/d9/df/f7d x:0 0 0 2024-04-02T04:42:37.805 INFO:tasks.workunit.client.0.smithi016.stdout:9/359: dread - d0/d9/d4d/f5f zero size 2024-04-02T04:42:37.805 INFO:tasks.workunit.client.1.smithi027.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-04-02T04:42:37.807 INFO:tasks.workunit.client.0.smithi016.stdout:4/336: creat d5/f67 x:0 0 0 2024-04-02T04:42:37.809 INFO:tasks.workunit.client.0.smithi016.stdout:3/390: dread d2/f3f [4194304,4194304] 0 2024-04-02T04:42:37.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.806+0000 7f12a4f38700 1 -- 172.21.15.16:0/2138895213 --> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f12a018f0e0 con 0x7f128806d4a0 2024-04-02T04:42:37.810 INFO:tasks.workunit.client.0.smithi016.stdout:2/299: mknod d12/d35/d3b/d42/d67/c6b 0 2024-04-02T04:42:37.812 INFO:tasks.workunit.client.1.smithi027.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-02T04:42:37.813 INFO:tasks.workunit.client.1.smithi027.stderr:+ make 2024-04-02T04:42:37.813 INFO:tasks.workunit.client.0.smithi016.stdout:6/415: rename d0/d78/d7c to d0/d1f/d67/d6f/d7e 0 2024-04-02T04:42:37.817 INFO:tasks.workunit.client.0.smithi016.stdout:6/416: dread d0/d9/d16/d56/f2c [0,4194304] 0 2024-04-02T04:42:37.818 INFO:tasks.workunit.client.0.smithi016.stdout:8/357: truncate d1/d6/f21 1976108 0 2024-04-02T04:42:37.820 INFO:tasks.workunit.client.0.smithi016.stdout:6/417: dread d0/fc [0,4194304] 0 2024-04-02T04:42:37.822 INFO:tasks.workunit.client.0.smithi016.stdout:0/298: sync 2024-04-02T04:42:37.822 INFO:tasks.workunit.client.0.smithi016.stdout:7/347: sync 2024-04-02T04:42:37.822 INFO:tasks.workunit.client.0.smithi016.stdout:5/358: sync 2024-04-02T04:42:37.822 INFO:tasks.workunit.client.0.smithi016.stdout:3/391: symlink d2/d15/d16/d3d/d46/l77 0 2024-04-02T04:42:37.823 INFO:tasks.workunit.client.0.smithi016.stdout:7/348: fsync d0/f32 0 2024-04-02T04:42:37.824 INFO:tasks.workunit.client.0.smithi016.stdout:7/349: dread - d0/d21/f36 zero size 2024-04-02T04:42:37.825 INFO:tasks.workunit.client.0.smithi016.stdout:8/358: dwrite d1/d31/f6b [0,4194304] 0 2024-04-02T04:42:37.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.824+0000 7f128f7fe700 1 -- 172.21.15.16:0/2138895213 <== mgr.24415 v2:172.21.15.27:6828/1554339939 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f12a018f0e0 con 0x7f128806d4a0 2024-04-02T04:42:37.827 INFO:teuthology.orchestra.run.smithi016.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:42:37.827 INFO:teuthology.orchestra.run.smithi016.stdout:alertmanager.smithi016 smithi016 *:9093,9094 running (7m) 17s ago 8m 24.2M - 0.25.0 c8568f914cd2 b95d8354b016 2024-04-02T04:42:37.827 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi016 smithi016 running (8m) 17s ago 8m 7654k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 9de6c16a487f 2024-04-02T04:42:37.827 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi027 smithi027 running (7m) 18s ago 7m 7633k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 99956592471a 2024-04-02T04:42:37.827 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi016 smithi016 running (8m) 17s ago 8m 6656k - 18.2.2-1059-g4d05f4ca 909dce4fe53d bd73919b0711 2024-04-02T04:42:37.827 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi027 smithi027 running (7m) 18s ago 7m 6660k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 7a54c37f202e 2024-04-02T04:42:37.827 INFO:teuthology.orchestra.run.smithi016.stdout:grafana.smithi016 smithi016 *:3000 running (6m) 17s ago 8m 83.7M - 9.4.7 954c08fa6188 7c43d133569c 2024-04-02T04:42:37.827 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.fyzrnm smithi016 running (2m) 17s ago 2m 14.9M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 70f28b9572a6 2024-04-02T04:42:37.827 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.uhgkgg smithi016 running (3m) 17s ago 3m 13.2M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 0aa4d16562a6 2024-04-02T04:42:37.828 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.sdublg smithi027 running (2m) 18s ago 2m 17.2M - 18.2.2-1059-g4d05f4ca 909dce4fe53d a538451099db 2024-04-02T04:42:37.828 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.zwrfoo smithi027 running (2m) 18s ago 2m 174M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 4cb3fa428d71 2024-04-02T04:42:37.828 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi016.eqnebx smithi016 *:9283,8765,8443 running (10m) 17s ago 10m 448M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 7d7ad6d79d01 2024-04-02T04:42:37.828 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi027.kjzcsc smithi027 *:8443,9283,8765 running (62s) 18s ago 7m 488M - 19.0.0-1979-g6cec793d 78f830190839 a1e2b7522dcb 2024-04-02T04:42:37.828 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi016 smithi016 running (10m) 17s ago 10m 53.5M 2048M 18.2.2-1059-g4d05f4ca 909dce4fe53d 24d5820dc994 2024-04-02T04:42:37.828 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi027 smithi027 running (7m) 18s ago 7m 41.1M 2048M 18.2.2-1059-g4d05f4ca 909dce4fe53d b77563184a1e 2024-04-02T04:42:37.828 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi016 smithi016 *:9100 running (8m) 17s ago 8m 13.8M - 1.5.0 0da6a335fe13 09f54c65b53c 2024-04-02T04:42:37.828 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi027 smithi027 *:9100 running (7m) 18s ago 7m 15.4M - 1.5.0 0da6a335fe13 e41a657c7443 2024-04-02T04:42:37.828 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 smithi016 running (6m) 17s ago 6m 87.4M 1217M 18.2.2-1059-g4d05f4ca 909dce4fe53d 3f40108debd6 2024-04-02T04:42:37.828 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 smithi016 running (6m) 17s ago 6m 90.3M 1217M 18.2.2-1059-g4d05f4ca 909dce4fe53d 7361abc9e1b6 2024-04-02T04:42:37.828 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 smithi016 running (5m) 17s ago 5m 78.7M 1217M 18.2.2-1059-g4d05f4ca 909dce4fe53d 52e592b4d1f2 2024-04-02T04:42:37.828 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 smithi027 running (5m) 18s ago 5m 89.1M 2240M 18.2.2-1059-g4d05f4ca 909dce4fe53d cf7c7fe022f0 2024-04-02T04:42:37.828 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 smithi027 running (4m) 18s ago 4m 90.0M 2240M 18.2.2-1059-g4d05f4ca 909dce4fe53d 8cbbedf8cac9 2024-04-02T04:42:37.828 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 smithi027 running (4m) 18s ago 4m 84.8M 2240M 18.2.2-1059-g4d05f4ca 909dce4fe53d ac6094415529 2024-04-02T04:42:37.828 INFO:teuthology.orchestra.run.smithi016.stdout:prometheus.smithi016 smithi016 *:9095 running (6m) 17s ago 8m 58.2M - 2.43.0 a07b618ecd1d 232d70bae1ba 2024-04-02T04:42:37.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.827+0000 7f12a4f38700 1 -- 172.21.15.16:0/2138895213 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7f128806d4a0 msgr2=0x7f128806f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:37.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.827+0000 7f12a4f38700 1 --2- 172.21.15.16:0/2138895213 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7f128806d4a0 0x7f128806f960 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1290000c00 tx=0x7f1290005960 comp rx=0 tx=0).stop 2024-04-02T04:42:37.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.827+0000 7f12a4f38700 1 -- 172.21.15.16:0/2138895213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12a0100bd0 msgr2=0x7f12a0195930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:37.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.827+0000 7f12a4f38700 1 --2- 172.21.15.16:0/2138895213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12a0100bd0 0x7f12a0195930 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f129400d8d0 tx=0x7f129400dbe0 comp rx=0 tx=0).stop 2024-04-02T04:42:37.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.827+0000 7f12a4f38700 1 -- 172.21.15.16:0/2138895213 shutdown_connections 2024-04-02T04:42:37.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.827+0000 7f12a4f38700 1 --2- 172.21.15.16:0/2138895213 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7f128806d4a0 0x7f128806f960 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:37.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.827+0000 7f12a4f38700 1 --2- 172.21.15.16:0/2138895213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12a00ff9d0 0x7f12a01953f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:37.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.827+0000 7f12a4f38700 1 --2- 172.21.15.16:0/2138895213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12a0100bd0 0x7f12a0195930 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:37.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.827+0000 7f12a4f38700 1 -- 172.21.15.16:0/2138895213 >> 172.21.15.16:0/2138895213 conn(0x7f12a00faf70 msgr2=0x7f12a0104180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:42:37.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.827+0000 7f12a4f38700 1 -- 172.21.15.16:0/2138895213 shutdown_connections 2024-04-02T04:42:37.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.827+0000 7f12a4f38700 1 -- 172.21.15.16:0/2138895213 wait complete. 2024-04-02T04:42:37.832 INFO:tasks.workunit.client.0.smithi016.stdout:1/340: getdents d14/d19/d20 0 2024-04-02T04:42:37.840 INFO:tasks.workunit.client.0.smithi016.stdout:4/337: write d5/f7 [216290,83586] 0 2024-04-02T04:42:37.841 INFO:tasks.workunit.client.0.smithi016.stdout:2/300: creat d12/d35/d3f/d56/d66/f6c x:0 0 0 2024-04-02T04:42:37.844 INFO:tasks.workunit.client.0.smithi016.stdout:9/360: write d0/d3/f40 [714937,82350] 0 2024-04-02T04:42:37.845 INFO:tasks.workunit.client.0.smithi016.stdout:5/359: mkdir d3/d40/d8a 0 2024-04-02T04:42:37.847 INFO:tasks.workunit.client.0.smithi016.stdout:9/361: fsync d0/d16/d1a/f48 0 2024-04-02T04:42:37.849 INFO:tasks.workunit.client.0.smithi016.stdout:9/362: fdatasync d0/d33/f36 0 2024-04-02T04:42:37.849 INFO:tasks.workunit.client.0.smithi016.stdout:4/338: dwrite d5/d1d/d2d/f4d [0,4194304] 0 2024-04-02T04:42:37.856 INFO:tasks.workunit.client.0.smithi016.stdout:2/301: symlink d12/d35/d3f/l6d 0 2024-04-02T04:42:37.856 INFO:tasks.workunit.client.0.smithi016.stdout:6/418: dwrite d0/d9/d16/d56/d59/f4b [0,4194304] 0 2024-04-02T04:42:37.858 INFO:tasks.workunit.client.0.smithi016.stdout:9/363: dwrite d0/d33/f36 [0,4194304] 0 2024-04-02T04:42:37.859 INFO:tasks.workunit.client.0.smithi016.stdout:8/359: write d1/d6/f25 [412363,32014] 0 2024-04-02T04:42:37.861 INFO:tasks.workunit.client.0.smithi016.stdout:6/419: write d0/d1f/d67/d6f/f75 [846733,93966] 0 2024-04-02T04:42:37.865 INFO:tasks.workunit.client.0.smithi016.stdout:3/392: mkdir d2/d15/d16/d3d/d6e/d6f/d78 0 2024-04-02T04:42:37.865 INFO:tasks.workunit.client.0.smithi016.stdout:6/420: dread d0/d1f/f3a [4194304,4194304] 0 2024-04-02T04:42:37.866 INFO:tasks.workunit.client.0.smithi016.stdout:6/421: dread - d0/d9/f33 zero size 2024-04-02T04:42:37.870 INFO:tasks.workunit.client.0.smithi016.stdout:4/339: unlink d5/d16/l2f 0 2024-04-02T04:42:37.871 INFO:tasks.workunit.client.0.smithi016.stdout:4/340: chown d5/d1d/f21 3431710 1 2024-04-02T04:42:37.871 INFO:tasks.workunit.client.0.smithi016.stdout:3/393: dwrite d2/d15/d1a/d45/f55 [0,4194304] 0 2024-04-02T04:42:37.873 INFO:tasks.workunit.client.0.smithi016.stdout:4/341: write d5/d17/d18/d1f/f53 [718443,75385] 0 2024-04-02T04:42:37.874 INFO:tasks.workunit.client.0.smithi016.stdout:4/342: stat d5/d16 0 2024-04-02T04:42:37.876 INFO:tasks.workunit.client.0.smithi016.stdout:0/299: truncate d0/f19 3480364 0 2024-04-02T04:42:37.877 INFO:tasks.workunit.client.0.smithi016.stdout:0/300: chown d0/d16/l28 0 1 2024-04-02T04:42:37.879 INFO:tasks.workunit.client.0.smithi016.stdout:0/301: fsync d0/d16/d2e/d53/f65 0 2024-04-02T04:42:37.879 INFO:tasks.workunit.client.0.smithi016.stdout:0/302: chown d0/d16/d2e/d37/c40 201 1 2024-04-02T04:42:37.880 INFO:tasks.workunit.client.1.smithi027.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-04-02T04:42:37.886 INFO:tasks.workunit.client.0.smithi016.stdout:1/341: creat d14/d19/f6b x:0 0 0 2024-04-02T04:42:37.886 INFO:tasks.workunit.client.0.smithi016.stdout:4/343: symlink d5/d1d/d1e/d50/l68 0 2024-04-02T04:42:37.886 INFO:tasks.workunit.client.0.smithi016.stdout:4/344: stat d5/d17/d18/f2b 0 2024-04-02T04:42:37.886 INFO:tasks.workunit.client.0.smithi016.stdout:1/342: chown d14/d19/d1e/c32 342 1 2024-04-02T04:42:37.887 INFO:tasks.workunit.client.0.smithi016.stdout:5/360: creat d3/df/d18/f8b x:0 0 0 2024-04-02T04:42:37.888 INFO:tasks.workunit.client.0.smithi016.stdout:8/360: truncate d1/d6/d36/d3f/d40/d49/f43 166434 0 2024-04-02T04:42:37.890 INFO:tasks.workunit.client.0.smithi016.stdout:7/350: getdents d0/d46/d37 0 2024-04-02T04:42:37.892 INFO:tasks.workunit.client.0.smithi016.stdout:0/303: dwrite d0/d16/d2e/f61 [0,4194304] 0 2024-04-02T04:42:37.893 INFO:tasks.workunit.client.0.smithi016.stdout:6/422: fsync d0/d9/d16/f47 0 2024-04-02T04:42:37.897 INFO:tasks.workunit.client.0.smithi016.stdout:9/364: write d0/d9/f19 [1509403,128459] 0 2024-04-02T04:42:37.900 INFO:tasks.workunit.client.0.smithi016.stdout:5/361: creat d3/df/d18/d26/f8c x:0 0 0 2024-04-02T04:42:37.903 INFO:tasks.workunit.client.0.smithi016.stdout:1/343: dwrite d14/d19/d1d/d29/f6a [0,4194304] 0 2024-04-02T04:42:37.904 INFO:tasks.workunit.client.0.smithi016.stdout:7/351: creat d0/d46/d37/f81 x:0 0 0 2024-04-02T04:42:37.904 INFO:tasks.workunit.client.0.smithi016.stdout:3/394: truncate d2/d15/d1a/f53 15971 0 2024-04-02T04:42:37.904 INFO:tasks.workunit.client.0.smithi016.stdout:8/361: dwrite d1/d6/d36/d4f/f62 [0,4194304] 0 2024-04-02T04:42:37.909 INFO:tasks.workunit.client.0.smithi016.stdout:2/302: getdents d12/d35 0 2024-04-02T04:42:37.913 INFO:tasks.workunit.client.0.smithi016.stdout:3/395: dread - d2/d15/f4a zero size 2024-04-02T04:42:37.914 INFO:tasks.workunit.client.0.smithi016.stdout:0/304: mkdir d0/d16/d2e/d37/d66 0 2024-04-02T04:42:37.914 INFO:tasks.workunit.client.0.smithi016.stdout:2/303: dread - d12/d35/d3b/d42/f5e zero size 2024-04-02T04:42:37.914 INFO:tasks.workunit.client.0.smithi016.stdout:4/345: mkdir d5/d1d/d69 0 2024-04-02T04:42:37.916 INFO:tasks.workunit.client.0.smithi016.stdout:6/423: creat d0/d1f/d67/d6f/f7f x:0 0 0 2024-04-02T04:42:37.916 INFO:tasks.workunit.client.0.smithi016.stdout:9/365: rmdir d0/d16/d1a 39 2024-04-02T04:42:37.918 INFO:tasks.workunit.client.0.smithi016.stdout:0/305: readlink d0/d16/l55 0 2024-04-02T04:42:37.918 INFO:tasks.workunit.client.0.smithi016.stdout:8/362: unlink d1/d31/f6b 0 2024-04-02T04:42:37.921 INFO:tasks.workunit.client.0.smithi016.stdout:3/396: readlink d2/d15/d16/l5c 0 2024-04-02T04:42:37.924 INFO:tasks.workunit.client.0.smithi016.stdout:6/424: symlink d0/d1f/d67/l80 0 2024-04-02T04:42:37.925 INFO:tasks.workunit.client.0.smithi016.stdout:7/352: dwrite d0/d46/d37/f81 [0,4194304] 0 2024-04-02T04:42:37.925 INFO:tasks.workunit.client.0.smithi016.stdout:9/366: symlink d0/d3/d39/l66 0 2024-04-02T04:42:37.927 INFO:tasks.workunit.client.0.smithi016.stdout:4/346: rename d5/d1d/l62 to d5/de/l6a 0 2024-04-02T04:42:37.933 INFO:tasks.workunit.client.0.smithi016.stdout:7/353: mkdir d0/d46/d52/d82 0 2024-04-02T04:42:37.934 INFO:tasks.workunit.client.0.smithi016.stdout:9/367: mkdir d0/d4e/d50/d67 0 2024-04-02T04:42:37.936 INFO:tasks.workunit.client.0.smithi016.stdout:4/347: dwrite d5/d16/f49 [4194304,4194304] 0 2024-04-02T04:42:37.938 INFO:tasks.workunit.client.0.smithi016.stdout:2/304: rename d12/d35/d3b/d42/f5e to d12/d35/f6e 0 2024-04-02T04:42:37.942 INFO:tasks.workunit.client.0.smithi016.stdout:8/363: link d1/f19 d1/d31/d67/f79 0 2024-04-02T04:42:37.942 INFO:tasks.workunit.client.0.smithi016.stdout:5/362: dread d3/d40/d4f/d6b/f47 [0,4194304] 0 2024-04-02T04:42:37.944 INFO:tasks.workunit.client.0.smithi016.stdout:5/363: chown d3/df/d18/f43 1389 1 2024-04-02T04:42:37.945 INFO:tasks.workunit.client.0.smithi016.stdout:3/397: creat d2/d15/d70/f79 x:0 0 0 2024-04-02T04:42:37.945 INFO:tasks.workunit.client.0.smithi016.stdout:5/364: readlink d3/df/l56 0 2024-04-02T04:42:37.946 INFO:tasks.workunit.client.0.smithi016.stdout:7/354: write d0/d46/d52/f61 [3539809,110791] 0 2024-04-02T04:42:37.948 INFO:tasks.workunit.client.0.smithi016.stdout:0/306: rename d0/f38 to d0/f67 0 2024-04-02T04:42:37.950 INFO:tasks.workunit.client.0.smithi016.stdout:8/364: dread d1/d6/d36/d3f/d40/d49/f44 [0,4194304] 0 2024-04-02T04:42:37.950 INFO:tasks.workunit.client.0.smithi016.stdout:2/305: unlink d12/d20/c32 0 2024-04-02T04:42:37.959 INFO:tasks.workunit.client.0.smithi016.stdout:5/365: creat d3/df/d18/d26/d2c/f8d x:0 0 0 2024-04-02T04:42:37.959 INFO:tasks.workunit.client.0.smithi016.stdout:0/307: mknod d0/d16/d24/d62/c68 0 2024-04-02T04:42:37.962 INFO:tasks.workunit.client.0.smithi016.stdout:1/344: dwrite d14/d19/d1e/f2e [0,4194304] 0 2024-04-02T04:42:37.963 INFO:tasks.workunit.client.0.smithi016.stdout:3/398: mkdir d2/d2e/d36/d66/d7a 0 2024-04-02T04:42:37.966 INFO:tasks.workunit.client.0.smithi016.stdout:0/308: stat d0/d16/l42 0 2024-04-02T04:42:37.969 INFO:tasks.workunit.client.0.smithi016.stdout:6/425: dwrite d0/d1f/f45 [0,4194304] 0 2024-04-02T04:42:37.970 INFO:tasks.workunit.client.0.smithi016.stdout:6/426: chown d0/d9/d16/d46/f68 357 1 2024-04-02T04:42:37.978 INFO:tasks.workunit.client.0.smithi016.stdout:0/309: creat d0/d16/d24/f69 x:0 0 0 2024-04-02T04:42:37.981 INFO:tasks.workunit.client.0.smithi016.stdout:8/365: creat d1/d6/d36/d3f/d40/f7a x:0 0 0 2024-04-02T04:42:37.982 INFO:tasks.workunit.client.0.smithi016.stdout:9/368: truncate d0/d33/f38 347639 0 2024-04-02T04:42:37.983 INFO:tasks.workunit.client.0.smithi016.stdout:1/345: mknod d14/d19/d1d/c6c 0 2024-04-02T04:42:37.983 INFO:tasks.workunit.client.0.smithi016.stdout:4/348: write d5/f10 [1253134,33658] 0 2024-04-02T04:42:37.987 INFO:tasks.workunit.client.0.smithi016.stdout:1/346: dread - d14/d19/d1d/d29/f51 zero size 2024-04-02T04:42:37.987 INFO:tasks.workunit.client.0.smithi016.stdout:5/366: rmdir d3/d40/d8a 0 2024-04-02T04:42:37.988 INFO:tasks.workunit.client.0.smithi016.stdout:4/349: read d5/d1d/d1e/f3c [542550,52263] 0 2024-04-02T04:42:37.988 INFO:tasks.workunit.client.0.smithi016.stdout:2/306: dread d12/d35/d3b/f46 [0,4194304] 0 2024-04-02T04:42:37.988 INFO:tasks.workunit.client.0.smithi016.stdout:1/347: stat cc 0 2024-04-02T04:42:37.989 INFO:tasks.workunit.client.0.smithi016.stdout:3/399: link d2/d15/d16/d3d/f4e d2/d15/d1a/d45/d56/f7b 0 2024-04-02T04:42:37.990 INFO:tasks.workunit.client.0.smithi016.stdout:0/310: mknod d0/d16/d24/c6a 0 2024-04-02T04:42:37.991 INFO:tasks.workunit.client.0.smithi016.stdout:8/366: dread d1/d6/f1a [0,4194304] 0 2024-04-02T04:42:37.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.988+0000 7fe24fcd0700 1 -- 172.21.15.16:0/4062642331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2480ff9d0 msgr2=0x7fe2480ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:37.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.988+0000 7fe24fcd0700 1 --2- 172.21.15.16:0/4062642331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2480ff9d0 0x7fe2480ffdf0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe244009910 tx=0x7fe244009c20 comp rx=0 tx=0).stop 2024-04-02T04:42:37.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.988+0000 7fe24fcd0700 1 -- 172.21.15.16:0/4062642331 shutdown_connections 2024-04-02T04:42:37.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.988+0000 7fe24fcd0700 1 --2- 172.21.15.16:0/4062642331 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe248100bd0 0x7fe248101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:37.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.988+0000 7fe24fcd0700 1 --2- 172.21.15.16:0/4062642331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2480ff9d0 0x7fe2480ffdf0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:37.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.988+0000 7fe24fcd0700 1 -- 172.21.15.16:0/4062642331 >> 172.21.15.16:0/4062642331 conn(0x7fe2480faf70 msgr2=0x7fe2480fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:42:37.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.989+0000 7fe24fcd0700 1 -- 172.21.15.16:0/4062642331 shutdown_connections 2024-04-02T04:42:37.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.989+0000 7fe24fcd0700 1 -- 172.21.15.16:0/4062642331 wait complete. 2024-04-02T04:42:37.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.990+0000 7fe24fcd0700 1 Processor -- start 2024-04-02T04:42:37.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.990+0000 7fe24fcd0700 1 -- start start 2024-04-02T04:42:37.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.990+0000 7fe24fcd0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2480ff9d0 0x7fe248069880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:37.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.990+0000 7fe24fcd0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe248100bd0 0x7fe24819b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:37.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.990+0000 7fe24fcd0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24819b7c0 con 0x7fe2480ff9d0 2024-04-02T04:42:37.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.990+0000 7fe24fcd0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24819b900 con 0x7fe248100bd0 2024-04-02T04:42:37.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.990+0000 7fe24da6c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2480ff9d0 0x7fe248069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:37.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.990+0000 7fe24d26b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe248100bd0 0x7fe24819b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:37.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.990+0000 7fe24da6c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2480ff9d0 0x7fe248069880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51936/0 (socket says 172.21.15.16:51936) 2024-04-02T04:42:37.994 INFO:tasks.workunit.client.0.smithi016.stdout:6/427: link d0/d1f/f26 d0/d6e/f81 0 2024-04-02T04:42:37.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.990+0000 7fe24d26b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe248100bd0 0x7fe24819b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.16:54530/0 (socket says 172.21.15.16:54530) 2024-04-02T04:42:37.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.990+0000 7fe24da6c700 1 -- 172.21.15.16:0/2982319367 learned_addr learned my addr 172.21.15.16:0/2982319367 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:42:37.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.991+0000 7fe24da6c700 1 -- 172.21.15.16:0/2982319367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe248100bd0 msgr2=0x7fe24819b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:37.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.991+0000 7fe24da6c700 1 --2- 172.21.15.16:0/2982319367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe248100bd0 0x7fe24819b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:37.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.991+0000 7fe24da6c700 1 -- 172.21.15.16:0/2982319367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2440095f0 con 0x7fe2480ff9d0 2024-04-02T04:42:37.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.991+0000 7fe24da6c700 1 --2- 172.21.15.16:0/2982319367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2480ff9d0 0x7fe248069880 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe24400b800 tx=0x7fe24400b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:42:37.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.991+0000 7fe23effd700 1 -- 172.21.15.16:0/2982319367 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe24401d070 con 0x7fe2480ff9d0 2024-04-02T04:42:37.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.991+0000 7fe23effd700 1 -- 172.21.15.16:0/2982319367 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe2440052a0 con 0x7fe2480ff9d0 2024-04-02T04:42:37.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.991+0000 7fe23effd700 1 -- 172.21.15.16:0/2982319367 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe244017400 con 0x7fe2480ff9d0 2024-04-02T04:42:37.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.991+0000 7fe24fcd0700 1 -- 172.21.15.16:0/2982319367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe24819bb80 con 0x7fe2480ff9d0 2024-04-02T04:42:37.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.991+0000 7fe24fcd0700 1 -- 172.21.15.16:0/2982319367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe24819c0a0 con 0x7fe2480ff9d0 2024-04-02T04:42:37.997 INFO:tasks.workunit.client.0.smithi016.stdout:9/369: dwrite d0/d9/db/f43 [0,4194304] 0 2024-04-02T04:42:37.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.993+0000 7fe24fcd0700 1 -- 172.21.15.16:0/2982319367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe24818f550 con 0x7fe2480ff9d0 2024-04-02T04:42:37.997 INFO:tasks.workunit.client.0.smithi016.stdout:4/350: rmdir d5/d17/d18/d25 39 2024-04-02T04:42:37.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.993+0000 7fe23effd700 1 -- 172.21.15.16:0/2982319367 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 91380+0+0 (secure 0 0 0) 0x7fe24400f7d0 con 0x7fe2480ff9d0 2024-04-02T04:42:37.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.994+0000 7fe23effd700 1 --2- 172.21.15.16:0/2982319367 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7fe23406d4a0 0x7fe23406f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:37.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.994+0000 7fe23effd700 1 -- 172.21.15.16:0/2982319367 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fe24408e1a0 con 0x7fe2480ff9d0 2024-04-02T04:42:37.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.994+0000 7fe24d26b700 1 --2- 172.21.15.16:0/2982319367 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7fe23406d4a0 0x7fe23406f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:37.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:37.996+0000 7fe24d26b700 1 --2- 172.21.15.16:0/2982319367 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7fe23406d4a0 0x7fe23406f960 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe238009c60 tx=0x7fe238009260 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:42:38.002 INFO:tasks.workunit.client.0.smithi016.stdout:1/348: mkdir d14/d19/d1e/d6d 0 2024-04-02T04:42:38.003 INFO:tasks.workunit.client.0.smithi016.stdout:9/370: write d0/d9/f2e [3364574,88178] 0 2024-04-02T04:42:38.003 INFO:tasks.workunit.client.0.smithi016.stdout:2/307: symlink d12/d35/d3f/l6f 0 2024-04-02T04:42:38.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.002+0000 7fe23effd700 1 -- 172.21.15.16:0/2982319367 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184299 (secure 0 0 0) 0x7fe24405b670 con 0x7fe2480ff9d0 2024-04-02T04:42:38.006 INFO:tasks.workunit.client.0.smithi016.stdout:4/351: write d5/d8/fd [3462796,120998] 0 2024-04-02T04:42:38.006 INFO:tasks.workunit.client.0.smithi016.stdout:0/311: mknod d0/d16/d24/d62/c6b 0 2024-04-02T04:42:38.009 INFO:tasks.workunit.client.0.smithi016.stdout:5/367: dwrite d3/df/d18/f38 [4194304,4194304] 0 2024-04-02T04:42:38.009 INFO:tasks.workunit.client.0.smithi016.stdout:6/428: creat d0/d1f/d67/d6f/f82 x:0 0 0 2024-04-02T04:42:38.011 INFO:tasks.workunit.client.0.smithi016.stdout:0/312: write d0/f12 [1024700,10237] 0 2024-04-02T04:42:38.013 INFO:tasks.workunit.client.0.smithi016.stdout:3/400: dwrite d2/d15/f4a [0,4194304] 0 2024-04-02T04:42:38.017 INFO:tasks.workunit.client.0.smithi016.stdout:5/368: write d3/df/f5a [445172,112643] 0 2024-04-02T04:42:38.018 INFO:tasks.workunit.client.0.smithi016.stdout:1/349: mknod d14/d19/d50/c6e 0 2024-04-02T04:42:38.020 INFO:tasks.workunit.client.0.smithi016.stdout:3/401: fsync d2/d2e/f30 0 2024-04-02T04:42:38.020 INFO:tasks.workunit.client.0.smithi016.stdout:6/429: write d0/d9/d16/f29 [435450,48414] 0 2024-04-02T04:42:38.025 INFO:tasks.workunit.client.0.smithi016.stdout:2/308: dwrite d12/d20/f64 [0,4194304] 0 2024-04-02T04:42:38.025 INFO:tasks.workunit.client.0.smithi016.stdout:1/350: write d14/f48 [1658920,6657] 0 2024-04-02T04:42:38.025 INFO:tasks.workunit.client.0.smithi016.stdout:4/352: creat d5/d17/d18/d25/f6b x:0 0 0 2024-04-02T04:42:38.029 INFO:tasks.workunit.client.0.smithi016.stdout:5/369: creat d3/df/d18/d26/f8e x:0 0 0 2024-04-02T04:42:38.031 INFO:tasks.workunit.client.0.smithi016.stdout:4/353: unlink d5/d1d/f42 0 2024-04-02T04:42:38.032 INFO:tasks.workunit.client.0.smithi016.stdout:6/430: creat d0/d1f/d67/d6d/f83 x:0 0 0 2024-04-02T04:42:38.034 INFO:tasks.workunit.client.0.smithi016.stdout:9/371: link d0/d9/db/l15 d0/d9/db/l68 0 2024-04-02T04:42:38.035 INFO:tasks.workunit.client.0.smithi016.stdout:0/313: link d0/d16/d24/c30 d0/d16/d2e/c6c 0 2024-04-02T04:42:38.038 INFO:tasks.workunit.client.0.smithi016.stdout:5/370: mkdir d3/d40/d4f/d6b/d2a/d6d/d8f 0 2024-04-02T04:42:38.041 INFO:tasks.workunit.client.0.smithi016.stdout:3/402: fsync d2/d15/f4a 0 2024-04-02T04:42:38.042 INFO:tasks.workunit.client.0.smithi016.stdout:9/372: write d0/d16/d1a/f48 [1871936,25803] 0 2024-04-02T04:42:38.043 INFO:tasks.workunit.client.0.smithi016.stdout:5/371: readlink d3/d40/d4f/d6b/d2a/l84 0 2024-04-02T04:42:38.044 INFO:tasks.workunit.client.0.smithi016.stdout:4/354: rename d5/d1d/d1e/d50 to d5/d1d/d6c 0 2024-04-02T04:42:38.045 INFO:tasks.workunit.client.0.smithi016.stdout:1/351: dwrite d14/d19/d1d/f30 [0,4194304] 0 2024-04-02T04:42:38.045 INFO:tasks.workunit.client.0.smithi016.stdout:3/403: fsync d2/d15/d16/f2c 0 2024-04-02T04:42:38.047 INFO:tasks.workunit.client.0.smithi016.stdout:6/431: dwrite d0/d9/df/f7d [0,4194304] 0 2024-04-02T04:42:38.049 INFO:tasks.workunit.client.0.smithi016.stdout:5/372: truncate d3/f17 1999942 0 2024-04-02T04:42:38.098 INFO:tasks.workunit.client.0.smithi016.stdout:7/355: sync 2024-04-02T04:42:38.108 INFO:tasks.workunit.client.0.smithi016.stdout:0/314: dwrite d0/f10 [4194304,4194304] 0 2024-04-02T04:42:38.109 INFO:tasks.workunit.client.0.smithi016.stdout:2/309: truncate d12/f4c 2669868 0 2024-04-02T04:42:38.110 INFO:tasks.workunit.client.0.smithi016.stdout:4/355: mkdir d5/d17/d18/d6d 0 2024-04-02T04:42:38.111 INFO:tasks.workunit.client.0.smithi016.stdout:5/373: creat d3/d40/d4f/d6b/d2a/f90 x:0 0 0 2024-04-02T04:42:38.117 INFO:tasks.workunit.client.0.smithi016.stdout:4/356: creat d5/d8/d43/f6e x:0 0 0 2024-04-02T04:42:38.117 INFO:tasks.workunit.client.0.smithi016.stdout:0/315: mknod d0/c6d 0 2024-04-02T04:42:38.117 INFO:tasks.workunit.client.0.smithi016.stdout:7/356: dread d0/d21/f3c [0,4194304] 0 2024-04-02T04:42:38.117 INFO:tasks.workunit.client.0.smithi016.stdout:9/373: dread d0/d9/f2e [0,4194304] 0 2024-04-02T04:42:38.118 INFO:tasks.workunit.client.0.smithi016.stdout:5/374: creat d3/df/d18/d26/f91 x:0 0 0 2024-04-02T04:42:38.120 INFO:tasks.workunit.client.0.smithi016.stdout:6/432: rename d0/d1f/d67/d6f/f75 to d0/f84 0 2024-04-02T04:42:38.122 INFO:tasks.workunit.client.0.smithi016.stdout:5/375: write d3/df/f82 [365138,5280] 0 2024-04-02T04:42:38.122 INFO:tasks.workunit.client.0.smithi016.stdout:2/310: dwrite f8 [0,4194304] 0 2024-04-02T04:42:38.122 INFO:tasks.workunit.client.0.smithi016.stdout:7/357: dread - d0/d46/d37/d3a/f76 zero size 2024-04-02T04:42:38.124 INFO:tasks.workunit.client.0.smithi016.stdout:5/376: write d3/f5b [479410,39648] 0 2024-04-02T04:42:38.124 INFO:tasks.workunit.client.0.smithi016.stdout:6/433: fdatasync d0/d1f/d67/d6d/f83 0 2024-04-02T04:42:38.126 INFO:tasks.workunit.client.0.smithi016.stdout:5/377: write d3/df/d18/f89 [673748,56635] 0 2024-04-02T04:42:38.126 INFO:tasks.workunit.client.0.smithi016.stdout:6/434: chown d0/d1f/d67/d6f/f82 114146272 1 2024-04-02T04:42:38.129 INFO:tasks.workunit.client.0.smithi016.stdout:3/404: truncate d2/fa 3264688 0 2024-04-02T04:42:38.131 INFO:tasks.workunit.client.0.smithi016.stdout:3/405: read - d2/d15/d16/f54 zero size 2024-04-02T04:42:38.132 INFO:tasks.workunit.client.0.smithi016.stdout:4/357: mkdir d5/d1d/d5f/d65/d6f 0 2024-04-02T04:42:38.132 INFO:tasks.workunit.client.0.smithi016.stdout:7/358: dwrite d0/d46/d16/d49/fa [0,4194304] 0 2024-04-02T04:42:38.134 INFO:tasks.workunit.client.0.smithi016.stdout:4/358: write d5/d8/fd [2543154,45622] 0 2024-04-02T04:42:38.135 INFO:tasks.workunit.client.0.smithi016.stdout:7/359: chown d0/d46/d52/c5b 11480 1 2024-04-02T04:42:38.135 INFO:tasks.workunit.client.0.smithi016.stdout:6/435: creat d0/d1f/f85 x:0 0 0 2024-04-02T04:42:38.136 INFO:tasks.workunit.client.0.smithi016.stdout:5/378: creat d3/d40/d4f/d6b/d2a/f92 x:0 0 0 2024-04-02T04:42:38.137 INFO:tasks.workunit.client.0.smithi016.stdout:5/379: chown d3/df/d18/f89 55 1 2024-04-02T04:42:38.138 INFO:tasks.workunit.client.0.smithi016.stdout:5/380: dread - d3/df/d18/f76 zero size 2024-04-02T04:42:38.143 INFO:tasks.workunit.client.0.smithi016.stdout:2/311: dread d12/d14/f37 [0,4194304] 0 2024-04-02T04:42:38.144 INFO:tasks.workunit.client.0.smithi016.stdout:2/312: chown d12/d35/d3f/c4a 5630 1 2024-04-02T04:42:38.144 INFO:tasks.workunit.client.0.smithi016.stdout:6/436: fsync d0/d1f/f42 0 2024-04-02T04:42:38.145 INFO:tasks.workunit.client.0.smithi016.stdout:6/437: stat d0/d9/f32 0 2024-04-02T04:42:38.145 INFO:tasks.workunit.client.0.smithi016.stdout:2/313: write f9 [2880824,53319] 0 2024-04-02T04:42:38.147 INFO:tasks.workunit.client.0.smithi016.stdout:0/316: dread d0/d16/d24/f2a [0,4194304] 0 2024-04-02T04:42:38.148 INFO:tasks.workunit.client.0.smithi016.stdout:0/317: readlink d0/d16/d2e/d53/l2f 0 2024-04-02T04:42:38.149 INFO:tasks.workunit.client.0.smithi016.stdout:0/318: rename d0/d16 to d0/d16/d24/d4e/d6e 22 2024-04-02T04:42:38.150 INFO:tasks.workunit.client.0.smithi016.stdout:6/438: rmdir d0/d1f/d67/d6d 39 2024-04-02T04:42:38.151 INFO:tasks.workunit.client.0.smithi016.stdout:3/406: link d2/cc d2/d15/c7c 0 2024-04-02T04:42:38.151 INFO:tasks.workunit.client.0.smithi016.stdout:6/439: fsync d0/d9/d16/d56/f11 0 2024-04-02T04:42:38.151 INFO:tasks.workunit.client.0.smithi016.stdout:6/440: chown d0/d9/c4a 1 1 2024-04-02T04:42:38.152 INFO:tasks.workunit.client.0.smithi016.stdout:5/381: mkdir d3/df/d93 0 2024-04-02T04:42:38.154 INFO:tasks.workunit.client.0.smithi016.stdout:2/314: mkdir d12/d70 0 2024-04-02T04:42:38.156 INFO:tasks.workunit.client.0.smithi016.stdout:0/319: mkdir d0/d16/d2e/d53/d6f 0 2024-04-02T04:42:38.159 INFO:tasks.workunit.client.0.smithi016.stdout:5/382: creat d3/df/d18/d26/f94 x:0 0 0 2024-04-02T04:42:38.160 INFO:tasks.workunit.client.0.smithi016.stdout:9/374: truncate d0/d3/f40 2248493 0 2024-04-02T04:42:38.160 INFO:tasks.workunit.client.0.smithi016.stdout:2/315: creat d12/d35/d3f/d56/d66/f71 x:0 0 0 2024-04-02T04:42:38.161 INFO:tasks.workunit.client.0.smithi016.stdout:2/316: write d12/d35/d3b/f60 [242604,51465] 0 2024-04-02T04:42:38.163 INFO:tasks.workunit.client.0.smithi016.stdout:5/383: creat d3/d40/d72/f95 x:0 0 0 2024-04-02T04:42:38.165 INFO:tasks.workunit.client.0.smithi016.stdout:9/375: creat d0/d4a/f69 x:0 0 0 2024-04-02T04:42:38.166 INFO:tasks.workunit.client.0.smithi016.stdout:4/359: write d5/d1d/d6c/f56 [699640,53446] 0 2024-04-02T04:42:38.166 INFO:tasks.workunit.client.0.smithi016.stdout:4/360: write d5/f7 [1482478,107466] 0 2024-04-02T04:42:38.168 INFO:tasks.workunit.client.0.smithi016.stdout:4/361: write d5/d1d/d2d/f4d [2110515,17154] 0 2024-04-02T04:42:38.172 INFO:tasks.workunit.client.0.smithi016.stdout:5/384: creat d3/df/d18/d26/d70/f96 x:0 0 0 2024-04-02T04:42:38.175 INFO:tasks.workunit.client.0.smithi016.stdout:4/362: creat d5/d17/d18/d1f/f70 x:0 0 0 2024-04-02T04:42:38.176 INFO:tasks.workunit.client.0.smithi016.stdout:9/376: mkdir d0/d16/d4b/d58/d6a 0 2024-04-02T04:42:38.177 INFO:tasks.workunit.client.0.smithi016.stdout:5/385: rename d3/df/d18/c29 to d3/df/d18/d26/d70/c97 0 2024-04-02T04:42:38.178 INFO:tasks.workunit.client.0.smithi016.stdout:3/407: rename d2/d15/d1a/d45 to d2/d15/d1a/d45/d7d 22 2024-04-02T04:42:38.179 INFO:tasks.workunit.client.0.smithi016.stdout:5/386: fdatasync d3/d5d/f6f 0 2024-04-02T04:42:38.180 INFO:tasks.workunit.client.0.smithi016.stdout:4/363: mknod d5/de/c71 0 2024-04-02T04:42:38.181 INFO:tasks.workunit.client.0.smithi016.stdout:5/387: stat d3/d40/d4f/d6b/d2a/f5f 0 2024-04-02T04:42:38.182 INFO:tasks.workunit.client.0.smithi016.stdout:4/364: write d5/d8/fd [543922,35773] 0 2024-04-02T04:42:38.183 INFO:tasks.workunit.client.0.smithi016.stdout:9/377: dwrite d0/d16/f65 [0,4194304] 0 2024-04-02T04:42:38.185 INFO:tasks.workunit.client.0.smithi016.stdout:9/378: write d0/d9/db/f43 [1644394,55324] 0 2024-04-02T04:42:38.185 INFO:tasks.workunit.client.0.smithi016.stdout:3/408: mkdir d2/d15/d7e 0 2024-04-02T04:42:38.188 INFO:tasks.workunit.client.0.smithi016.stdout:9/379: dread - d0/d9/f42 zero size 2024-04-02T04:42:38.188 INFO:tasks.workunit.client.0.smithi016.stdout:0/320: truncate d0/f10 5515281 0 2024-04-02T04:42:38.188 INFO:tasks.workunit.client.0.smithi016.stdout:4/365: creat d5/d1d/d6c/f72 x:0 0 0 2024-04-02T04:42:38.189 INFO:tasks.workunit.client.0.smithi016.stdout:0/321: chown d0/d16/d24/f20 847 1 2024-04-02T04:42:38.198 INFO:tasks.workunit.client.0.smithi016.stdout:4/366: symlink d5/d17/d18/d1f/l73 0 2024-04-02T04:42:38.201 INFO:tasks.workunit.client.0.smithi016.stdout:4/367: write d5/d17/d18/d25/f41 [4968948,67713] 0 2024-04-02T04:42:38.201 INFO:tasks.workunit.client.0.smithi016.stdout:5/388: creat d3/df/d18/f98 x:0 0 0 2024-04-02T04:42:38.201 INFO:tasks.workunit.client.0.smithi016.stdout:3/409: dwrite d2/d15/d16/f54 [0,4194304] 0 2024-04-02T04:42:38.209 INFO:tasks.workunit.client.0.smithi016.stdout:9/380: getdents d0/d3/d39 0 2024-04-02T04:42:38.216 INFO:tasks.workunit.client.0.smithi016.stdout:0/322: dwrite d0/fe [0,4194304] 0 2024-04-02T04:42:38.218 INFO:tasks.workunit.client.0.smithi016.stdout:0/323: stat d0/d16/l1b 0 2024-04-02T04:42:38.219 INFO:tasks.workunit.client.0.smithi016.stdout:9/381: symlink d0/d3/d31/d53/l6b 0 2024-04-02T04:42:38.220 INFO:tasks.workunit.client.0.smithi016.stdout:5/389: dwrite d3/d40/f79 [0,4194304] 0 2024-04-02T04:42:38.222 INFO:tasks.workunit.client.0.smithi016.stdout:4/368: dread d5/d17/d18/d25/f4a [0,4194304] 0 2024-04-02T04:42:38.225 INFO:tasks.workunit.client.0.smithi016.stdout:0/324: creat d0/d16/d2e/d37/d66/f70 x:0 0 0 2024-04-02T04:42:38.229 INFO:tasks.workunit.client.0.smithi016.stdout:3/410: dread d2/d15/d1a/f47 [0,4194304] 0 2024-04-02T04:42:38.230 INFO:tasks.workunit.client.0.smithi016.stdout:4/369: truncate d5/d8/fb 3299029 0 2024-04-02T04:42:38.232 INFO:tasks.workunit.client.0.smithi016.stdout:3/411: unlink d2/c37 0 2024-04-02T04:42:38.234 INFO:tasks.workunit.client.0.smithi016.stdout:4/370: write d5/d17/d18/d1f/f2a [867218,51890] 0 2024-04-02T04:42:38.235 INFO:tasks.workunit.client.0.smithi016.stdout:9/382: dwrite d0/d3/f30 [0,4194304] 0 2024-04-02T04:42:38.237 INFO:tasks.workunit.client.0.smithi016.stdout:0/325: dwrite d0/d16/d2e/d37/d66/f70 [0,4194304] 0 2024-04-02T04:42:38.237 INFO:tasks.workunit.client.0.smithi016.stdout:9/383: read d0/d33/f36 [2888942,29883] 0 2024-04-02T04:42:38.238 INFO:tasks.workunit.client.0.smithi016.stdout:0/326: chown d0/d16/f31 158 1 2024-04-02T04:42:38.239 INFO:tasks.workunit.client.0.smithi016.stdout:3/412: unlink d2/d15/d1a/d45/f61 0 2024-04-02T04:42:38.239 INFO:tasks.workunit.client.0.smithi016.stdout:4/371: readlink d5/d17/d18/d1f/l3e 0 2024-04-02T04:42:38.241 INFO:tasks.workunit.client.0.smithi016.stdout:4/372: stat d5/d17/d18/d25/f4a 0 2024-04-02T04:42:38.243 INFO:tasks.workunit.client.0.smithi016.stdout:9/384: symlink d0/d33/l6c 0 2024-04-02T04:42:38.243 INFO:tasks.workunit.client.0.smithi016.stdout:3/413: dread d2/f14 [0,4194304] 0 2024-04-02T04:42:38.248 INFO:tasks.workunit.client.0.smithi016.stdout:0/327: mkdir d0/d16/d60/d71 0 2024-04-02T04:42:38.249 INFO:tasks.workunit.client.0.smithi016.stdout:0/328: dread - d0/d16/d24/f69 zero size 2024-04-02T04:42:38.249 INFO:tasks.workunit.client.0.smithi016.stdout:0/329: read - d0/d16/d24/f69 zero size 2024-04-02T04:42:38.250 INFO:tasks.workunit.client.0.smithi016.stdout:3/414: mknod d2/d15/d16/d57/c7f 0 2024-04-02T04:42:38.253 INFO:tasks.workunit.client.0.smithi016.stdout:9/385: rename d0/d9/f42 to d0/d33/f6d 0 2024-04-02T04:42:38.254 INFO:tasks.workunit.client.0.smithi016.stdout:4/373: creat d5/d17/d18/d6d/f74 x:0 0 0 2024-04-02T04:42:38.258 INFO:tasks.workunit.client.0.smithi016.stdout:4/374: dread d5/d17/d18/d25/f57 [0,4194304] 0 2024-04-02T04:42:38.260 INFO:tasks.workunit.client.0.smithi016.stdout:4/375: rmdir d5/d1d/d2d 39 2024-04-02T04:42:38.261 INFO:tasks.workunit.client.0.smithi016.stdout:4/376: write d5/d1d/f37 [2655938,122236] 0 2024-04-02T04:42:38.261 INFO:tasks.workunit.client.0.smithi016.stdout:4/377: chown d5/d8/d43/f6e 3515 1 2024-04-02T04:42:38.263 INFO:tasks.workunit.client.0.smithi016.stdout:3/415: rename d2/d15/d1a/d45/f55 to d2/d15/f80 0 2024-04-02T04:42:38.264 INFO:tasks.workunit.client.0.smithi016.stdout:3/416: write d2/f3e [534926,122218] 0 2024-04-02T04:42:38.269 INFO:tasks.workunit.client.0.smithi016.stdout:4/378: dread d5/f10 [0,4194304] 0 2024-04-02T04:42:38.269 INFO:tasks.workunit.client.0.smithi016.stdout:9/386: rename d0/d16 to d0/d4e/d50/d63/d6e 0 2024-04-02T04:42:38.270 INFO:tasks.workunit.client.0.smithi016.stdout:4/379: chown d5/d17/c2c 2730 1 2024-04-02T04:42:38.270 INFO:tasks.workunit.client.0.smithi016.stdout:3/417: creat d2/d15/d16/d57/f81 x:0 0 0 2024-04-02T04:42:38.271 INFO:tasks.workunit.client.0.smithi016.stdout:4/380: dread - d5/d1d/d6c/f72 zero size 2024-04-02T04:42:38.274 INFO:tasks.workunit.client.0.smithi016.stdout:9/387: mknod d0/d4e/d50/d63/d6e/d1a/c6f 0 2024-04-02T04:42:38.274 INFO:tasks.workunit.client.0.smithi016.stdout:9/388: readlink d0/l55 0 2024-04-02T04:42:38.274 INFO:tasks.workunit.client.0.smithi016.stdout:4/381: mkdir d5/d17/d18/d25/d75 0 2024-04-02T04:42:38.275 INFO:tasks.workunit.client.0.smithi016.stdout:3/418: dread d2/f14 [0,4194304] 0 2024-04-02T04:42:38.276 INFO:tasks.workunit.client.0.smithi016.stdout:4/382: fsync d5/f4e 0 2024-04-02T04:42:38.278 INFO:tasks.workunit.client.0.smithi016.stdout:0/330: dwrite d0/f19 [0,4194304] 0 2024-04-02T04:42:38.286 INFO:tasks.workunit.client.0.smithi016.stdout:3/419: truncate d2/fa 1058563 0 2024-04-02T04:42:38.290 INFO:tasks.workunit.client.0.smithi016.stdout:0/331: creat d0/d16/d60/d71/f72 x:0 0 0 2024-04-02T04:42:38.291 INFO:tasks.workunit.client.0.smithi016.stdout:4/383: dwrite d5/f4e [0,4194304] 0 2024-04-02T04:42:38.293 INFO:tasks.workunit.client.0.smithi016.stdout:9/389: dwrite d0/d33/f36 [0,4194304] 0 2024-04-02T04:42:38.294 INFO:tasks.workunit.client.0.smithi016.stdout:0/332: mknod d0/d16/d23/c73 0 2024-04-02T04:42:38.298 INFO:tasks.workunit.client.0.smithi016.stdout:0/333: write d0/f19 [4783554,63924] 0 2024-04-02T04:42:38.298 INFO:tasks.workunit.client.0.smithi016.stdout:3/420: dwrite d2/d15/d70/f79 [0,4194304] 0 2024-04-02T04:42:38.307 INFO:tasks.workunit.client.0.smithi016.stdout:0/334: rename d0/d16/f34 to d0/d16/d2e/d53/d6f/f74 0 2024-04-02T04:42:38.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:38 smithi016 ceph-mon[28969]: from='client.14716 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:42:38.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:38 smithi016 ceph-mon[28969]: from='client.14720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:42:38.310 INFO:tasks.workunit.client.0.smithi016.stdout:0/335: chown d0/d16/d24/c30 809172 1 2024-04-02T04:42:38.310 INFO:tasks.workunit.client.0.smithi016.stdout:0/336: stat d0/d4f 0 2024-04-02T04:42:38.311 INFO:tasks.workunit.client.0.smithi016.stdout:3/421: rename d2/d2e/l5d to d2/d15/d16/d57/d76/l82 0 2024-04-02T04:42:38.312 INFO:tasks.workunit.client.0.smithi016.stdout:3/422: fsync d2/d15/d1a/f69 0 2024-04-02T04:42:38.313 INFO:tasks.workunit.client.0.smithi016.stdout:0/337: creat d0/d16/d60/f75 x:0 0 0 2024-04-02T04:42:38.313 INFO:tasks.workunit.client.0.smithi016.stdout:4/384: truncate d5/d17/d18/d25/f57 296311 0 2024-04-02T04:42:38.333 INFO:tasks.workunit.client.0.smithi016.stdout:8/367: sync 2024-04-02T04:42:38.372 INFO:tasks.workunit.client.0.smithi016.stdout:3/423: creat d2/d15/d16/d3d/d46/f83 x:0 0 0 2024-04-02T04:42:38.373 INFO:tasks.workunit.client.0.smithi016.stdout:3/424: write d2/d15/d70/f79 [956106,72532] 0 2024-04-02T04:42:38.374 INFO:tasks.workunit.client.0.smithi016.stdout:3/425: chown d2/d15/d16/f27 188 1 2024-04-02T04:42:38.376 INFO:tasks.workunit.client.0.smithi016.stdout:0/338: mkdir d0/d4f/d76 0 2024-04-02T04:42:38.380 INFO:tasks.workunit.client.0.smithi016.stdout:1/352: sync 2024-04-02T04:42:38.381 INFO:tasks.workunit.client.0.smithi016.stdout:8/368: rename d1/d6/d36/l3e to d1/d70/d71/l7b 0 2024-04-02T04:42:38.385 INFO:tasks.workunit.client.0.smithi016.stdout:3/426: chown d2/d15/d16/l41 1261 1 2024-04-02T04:42:38.385 INFO:tasks.workunit.client.0.smithi016.stdout:1/353: dwrite d14/d19/d1d/f2c [0,4194304] 0 2024-04-02T04:42:38.387 INFO:tasks.workunit.client.0.smithi016.stdout:1/354: read d14/d19/d1d/f30 [2587205,97142] 0 2024-04-02T04:42:38.388 INFO:tasks.workunit.client.0.smithi016.stdout:0/339: mknod d0/d16/d23/d3e/d57/c77 0 2024-04-02T04:42:38.390 INFO:tasks.workunit.client.0.smithi016.stdout:0/340: chown d0/d16/d23/f3f 23858682 1 2024-04-02T04:42:38.390 INFO:tasks.workunit.client.0.smithi016.stdout:1/355: write d14/d19/d1d/f2c [3941363,22854] 0 2024-04-02T04:42:38.395 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:38 smithi027 ceph-mon[37550]: from='client.14716 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:42:38.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:38 smithi027 ceph-mon[37550]: from='client.14720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:42:38.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.392+0000 7fe24fcd0700 1 -- 172.21.15.16:0/2982319367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe248108560 con 0x7fe2480ff9d0 2024-04-02T04:42:38.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.393+0000 7fe23effd700 1 -- 172.21.15.16:0/2982319367 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+841 (secure 0 0 0) 0x7fe24405b200 con 0x7fe2480ff9d0 2024-04-02T04:42:38.396 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:42:38.396 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": { 2024-04-02T04:42:38.397 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 2 2024-04-02T04:42:38.397 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:42:38.397 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": { 2024-04-02T04:42:38.397 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 1, 2024-04-02T04:42:38.397 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 1 2024-04-02T04:42:38.397 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:42:38.397 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": { 2024-04-02T04:42:38.397 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 6 2024-04-02T04:42:38.397 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:42:38.397 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-04-02T04:42:38.397 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 4 2024-04-02T04:42:38.397 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:42:38.397 INFO:teuthology.orchestra.run.smithi016.stdout: "overall": { 2024-04-02T04:42:38.397 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 13, 2024-04-02T04:42:38.398 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 1 2024-04-02T04:42:38.398 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-02T04:42:38.398 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:42:38.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.396+0000 7fe24fcd0700 1 -- 172.21.15.16:0/2982319367 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7fe23406d4a0 msgr2=0x7fe23406f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:38.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.396+0000 7fe24fcd0700 1 --2- 172.21.15.16:0/2982319367 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7fe23406d4a0 0x7fe23406f960 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe238009c60 tx=0x7fe238009260 comp rx=0 tx=0).stop 2024-04-02T04:42:38.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.396+0000 7fe24fcd0700 1 -- 172.21.15.16:0/2982319367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2480ff9d0 msgr2=0x7fe248069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:38.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.396+0000 7fe24fcd0700 1 --2- 172.21.15.16:0/2982319367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2480ff9d0 0x7fe248069880 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe24400b800 tx=0x7fe24400b600 comp rx=0 tx=0).stop 2024-04-02T04:42:38.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.396+0000 7fe24fcd0700 1 -- 172.21.15.16:0/2982319367 shutdown_connections 2024-04-02T04:42:38.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.396+0000 7fe24fcd0700 1 --2- 172.21.15.16:0/2982319367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2480ff9d0 0x7fe248069880 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:38.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.396+0000 7fe24fcd0700 1 --2- 172.21.15.16:0/2982319367 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7fe23406d4a0 0x7fe23406f960 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:38.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.396+0000 7fe24fcd0700 1 --2- 172.21.15.16:0/2982319367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe248100bd0 0x7fe24819b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:38.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.396+0000 7fe24fcd0700 1 -- 172.21.15.16:0/2982319367 >> 172.21.15.16:0/2982319367 conn(0x7fe2480faf70 msgr2=0x7fe248104180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:42:38.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.396+0000 7fe24fcd0700 1 -- 172.21.15.16:0/2982319367 shutdown_connections 2024-04-02T04:42:38.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.396+0000 7fe24fcd0700 1 -- 172.21.15.16:0/2982319367 wait complete. 2024-04-02T04:42:38.400 INFO:tasks.workunit.client.0.smithi016.stdout:3/427: rename d2/d15/d16/d3d/d46/l59 to d2/d15/d1a/d45/l84 0 2024-04-02T04:42:38.401 INFO:tasks.workunit.client.0.smithi016.stdout:3/428: write d2/d15/d16/d3d/d46/f62 [557902,51443] 0 2024-04-02T04:42:38.403 INFO:tasks.workunit.client.0.smithi016.stdout:0/341: mknod d0/d16/d2e/d53/c78 0 2024-04-02T04:42:38.404 INFO:tasks.workunit.client.0.smithi016.stdout:0/342: fdatasync d0/d16/d2e/d36/f3d 0 2024-04-02T04:42:38.405 INFO:tasks.workunit.client.0.smithi016.stdout:1/356: creat d14/d41/d43/f6f x:0 0 0 2024-04-02T04:42:38.407 INFO:tasks.workunit.client.0.smithi016.stdout:4/385: rmdir d5/d17/d18/d1f/d58 0 2024-04-02T04:42:38.408 INFO:tasks.workunit.client.0.smithi016.stdout:8/369: dread d1/f15 [0,4194304] 0 2024-04-02T04:42:38.410 INFO:tasks.workunit.client.0.smithi016.stdout:3/429: truncate d2/f3f 4557596 0 2024-04-02T04:42:38.412 INFO:tasks.workunit.client.0.smithi016.stdout:3/430: chown d2/d2e/d36/d66/l74 16 1 2024-04-02T04:42:38.413 INFO:tasks.workunit.client.0.smithi016.stdout:4/386: dwrite d5/d16/f49 [8388608,4194304] 0 2024-04-02T04:42:38.414 INFO:tasks.workunit.client.0.smithi016.stdout:4/387: write d5/d16/f55 [146078,121296] 0 2024-04-02T04:42:38.415 INFO:tasks.workunit.client.0.smithi016.stdout:0/343: creat d0/d16/d2e/d37/f79 x:0 0 0 2024-04-02T04:42:38.416 INFO:tasks.workunit.client.0.smithi016.stdout:1/357: mknod d14/d19/d1d/d45/c70 0 2024-04-02T04:42:38.416 INFO:tasks.workunit.client.0.smithi016.stdout:4/388: write d5/d1d/f23 [4490398,5978] 0 2024-04-02T04:42:38.417 INFO:tasks.workunit.client.0.smithi016.stdout:1/358: read - d14/d19/d20/f5b zero size 2024-04-02T04:42:38.418 INFO:tasks.workunit.client.0.smithi016.stdout:1/359: chown d14/d19/d1d/d29/d39/d46/c64 1811 1 2024-04-02T04:42:38.422 INFO:tasks.workunit.client.0.smithi016.stdout:1/360: dread d14/f56 [0,4194304] 0 2024-04-02T04:42:38.427 INFO:tasks.workunit.client.0.smithi016.stdout:3/431: dwrite d2/d15/f4b [0,4194304] 0 2024-04-02T04:42:38.436 INFO:tasks.workunit.client.1.smithi027.stderr:++ readlink -f fsstress 2024-04-02T04:42:38.439 INFO:tasks.workunit.client.1.smithi027.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-04-02T04:42:38.439 INFO:tasks.workunit.client.1.smithi027.stderr:+ popd 2024-04-02T04:42:38.439 INFO:tasks.workunit.client.0.smithi016.stdout:3/432: mknod d2/d15/d1a/d45/c85 0 2024-04-02T04:42:38.440 INFO:tasks.workunit.client.0.smithi016.stdout:3/433: chown d2/d15/d1a/f69 7 1 2024-04-02T04:42:38.441 INFO:tasks.workunit.client.1.smithi027.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-02T04:42:38.441 INFO:tasks.workunit.client.1.smithi027.stderr:+ popd 2024-04-02T04:42:38.442 INFO:tasks.workunit.client.1.smithi027.stdout:~/cephtest/mnt.1/client.1/tmp 2024-04-02T04:42:38.442 INFO:tasks.workunit.client.1.smithi027.stderr:++ mktemp -d -p . 2024-04-02T04:42:38.443 INFO:tasks.workunit.client.0.smithi016.stdout:4/389: rmdir d5/d17/d18/d25/d75 0 2024-04-02T04:42:38.448 INFO:tasks.workunit.client.1.smithi027.stderr:+ T=./tmp.awxrg0SToX 2024-04-02T04:42:38.448 INFO:tasks.workunit.client.1.smithi027.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.awxrg0SToX -l 1 -n 1000 -p 10 -v 2024-04-02T04:42:38.457 INFO:tasks.workunit.client.1.smithi027.stdout:seed = 1712160311 2024-04-02T04:42:38.463 INFO:tasks.workunit.client.1.smithi027.stdout:0/0: rmdir - no directory 2024-04-02T04:42:38.465 INFO:tasks.workunit.client.1.smithi027.stdout:1/0: read - no filename 2024-04-02T04:42:38.467 INFO:tasks.workunit.client.1.smithi027.stdout:2/0: truncate - no filename 2024-04-02T04:42:38.467 INFO:tasks.workunit.client.1.smithi027.stdout:2/1: dwrite - no filename 2024-04-02T04:42:38.467 INFO:tasks.workunit.client.1.smithi027.stdout:2/2: unlink - no file 2024-04-02T04:42:38.467 INFO:tasks.workunit.client.1.smithi027.stdout:2/3: dwrite - no filename 2024-04-02T04:42:38.467 INFO:tasks.workunit.client.1.smithi027.stdout:2/4: dread - no filename 2024-04-02T04:42:38.467 INFO:tasks.workunit.client.1.smithi027.stdout:2/5: chown . 535601661 1 2024-04-02T04:42:38.467 INFO:tasks.workunit.client.1.smithi027.stdout:2/6: chown . 0 1 2024-04-02T04:42:38.467 INFO:tasks.workunit.client.1.smithi027.stdout:0/1: symlink l0 0 2024-04-02T04:42:38.468 INFO:tasks.workunit.client.1.smithi027.stdout:0/2: readlink l0 0 2024-04-02T04:42:38.468 INFO:tasks.workunit.client.1.smithi027.stdout:0/3: write - no filename 2024-04-02T04:42:38.469 INFO:tasks.workunit.client.1.smithi027.stdout:3/0: rename - no filename 2024-04-02T04:42:38.469 INFO:tasks.workunit.client.1.smithi027.stdout:3/1: write - no filename 2024-04-02T04:42:38.469 INFO:tasks.workunit.client.1.smithi027.stdout:3/2: read - no filename 2024-04-02T04:42:38.469 INFO:tasks.workunit.client.1.smithi027.stdout:3/3: dread - no filename 2024-04-02T04:42:38.469 INFO:tasks.workunit.client.1.smithi027.stdout:3/4: truncate - no filename 2024-04-02T04:42:38.470 INFO:tasks.workunit.client.1.smithi027.stdout:1/1: mkdir d0 0 2024-04-02T04:42:38.473 INFO:tasks.workunit.client.1.smithi027.stdout:0/4: creat f1 x:0 0 0 2024-04-02T04:42:38.473 INFO:tasks.workunit.client.1.smithi027.stdout:0/5: chown l0 762341 1 2024-04-02T04:42:38.474 INFO:tasks.workunit.client.1.smithi027.stdout:2/7: symlink l0 0 2024-04-02T04:42:38.474 INFO:tasks.workunit.client.1.smithi027.stdout:2/8: dread - no filename 2024-04-02T04:42:38.474 INFO:tasks.workunit.client.1.smithi027.stdout:2/9: fdatasync - no filename 2024-04-02T04:42:38.474 INFO:tasks.workunit.client.1.smithi027.stdout:2/10: chown l0 4 1 2024-04-02T04:42:38.474 INFO:tasks.workunit.client.1.smithi027.stdout:2/11: write - no filename 2024-04-02T04:42:38.475 INFO:tasks.workunit.client.1.smithi027.stdout:2/12: chown l0 34976 1 2024-04-02T04:42:38.475 INFO:tasks.workunit.client.1.smithi027.stdout:2/13: stat l0 0 2024-04-02T04:42:38.475 INFO:tasks.workunit.client.1.smithi027.stdout:2/14: write - no filename 2024-04-02T04:42:38.475 INFO:tasks.workunit.client.1.smithi027.stdout:2/15: chown l0 24 1 2024-04-02T04:42:38.475 INFO:tasks.workunit.client.1.smithi027.stdout:2/16: dread - no filename 2024-04-02T04:42:38.475 INFO:tasks.workunit.client.1.smithi027.stdout:4/0: rename - no filename 2024-04-02T04:42:38.475 INFO:tasks.workunit.client.1.smithi027.stdout:4/1: truncate - no filename 2024-04-02T04:42:38.478 INFO:tasks.workunit.client.1.smithi027.stdout:0/6: creat f2 x:0 0 0 2024-04-02T04:42:38.480 INFO:tasks.workunit.client.1.smithi027.stdout:2/17: mkdir d1 0 2024-04-02T04:42:38.480 INFO:tasks.workunit.client.1.smithi027.stdout:2/18: truncate - no filename 2024-04-02T04:42:38.485 INFO:tasks.workunit.client.1.smithi027.stdout:5/0: truncate - no filename 2024-04-02T04:42:38.485 INFO:tasks.workunit.client.1.smithi027.stdout:5/1: dwrite - no filename 2024-04-02T04:42:38.485 INFO:tasks.workunit.client.1.smithi027.stdout:3/5: mknod c0 0 2024-04-02T04:42:38.485 INFO:tasks.workunit.client.1.smithi027.stdout:3/6: fdatasync - no filename 2024-04-02T04:42:38.486 INFO:tasks.workunit.client.0.smithi016.stdout:7/360: sync 2024-04-02T04:42:38.487 INFO:tasks.workunit.client.0.smithi016.stdout:6/441: sync 2024-04-02T04:42:38.487 INFO:tasks.workunit.client.1.smithi027.stdout:1/2: mknod d0/c1 0 2024-04-02T04:42:38.489 INFO:tasks.workunit.client.1.smithi027.stdout:0/7: dwrite f2 [0,4194304] 0 2024-04-02T04:42:38.490 INFO:tasks.workunit.client.1.smithi027.stdout:2/19: rename l0 to d1/l2 0 2024-04-02T04:42:38.491 INFO:tasks.workunit.client.0.smithi016.stdout:6/442: dwrite d0/d9/df/f4d [0,4194304] 0 2024-04-02T04:42:38.494 INFO:tasks.workunit.client.0.smithi016.stdout:7/361: mkdir d0/d46/d16/d83 0 2024-04-02T04:42:38.495 INFO:tasks.workunit.client.1.smithi027.stdout:4/2: getdents . 0 2024-04-02T04:42:38.495 INFO:tasks.workunit.client.1.smithi027.stdout:4/3: dread - no filename 2024-04-02T04:42:38.495 INFO:tasks.workunit.client.1.smithi027.stdout:4/4: rmdir - no directory 2024-04-02T04:42:38.495 INFO:tasks.workunit.client.1.smithi027.stdout:4/5: readlink - no filename 2024-04-02T04:42:38.495 INFO:tasks.workunit.client.1.smithi027.stdout:4/6: dread - no filename 2024-04-02T04:42:38.495 INFO:tasks.workunit.client.1.smithi027.stdout:4/7: truncate - no filename 2024-04-02T04:42:38.496 INFO:tasks.workunit.client.1.smithi027.stdout:4/8: read - no filename 2024-04-02T04:42:38.496 INFO:tasks.workunit.client.1.smithi027.stdout:4/9: fdatasync - no filename 2024-04-02T04:42:38.496 INFO:tasks.workunit.client.1.smithi027.stdout:4/10: dwrite - no filename 2024-04-02T04:42:38.496 INFO:tasks.workunit.client.1.smithi027.stdout:4/11: read - no filename 2024-04-02T04:42:38.496 INFO:tasks.workunit.client.1.smithi027.stdout:4/12: readlink - no filename 2024-04-02T04:42:38.496 INFO:tasks.workunit.client.1.smithi027.stdout:4/13: chown . 619254 1 2024-04-02T04:42:38.496 INFO:tasks.workunit.client.1.smithi027.stdout:4/14: dread - no filename 2024-04-02T04:42:38.496 INFO:tasks.workunit.client.1.smithi027.stdout:4/15: dread - no filename 2024-04-02T04:42:38.506 INFO:tasks.workunit.client.0.smithi016.stdout:7/362: unlink d0/d21/c7e 0 2024-04-02T04:42:38.509 INFO:tasks.workunit.client.1.smithi027.stdout:3/7: mkdir d1 0 2024-04-02T04:42:38.509 INFO:tasks.workunit.client.1.smithi027.stdout:1/3: unlink d0/c1 0 2024-04-02T04:42:38.511 INFO:tasks.workunit.client.1.smithi027.stdout:0/8: symlink l3 0 2024-04-02T04:42:38.512 INFO:tasks.workunit.client.0.smithi016.stdout:7/363: mkdir d0/d46/d37/d3a/d84 0 2024-04-02T04:42:38.514 INFO:tasks.workunit.client.1.smithi027.stdout:7/0: dwrite - no filename 2024-04-02T04:42:38.514 INFO:tasks.workunit.client.1.smithi027.stdout:7/1: dread - no filename 2024-04-02T04:42:38.514 INFO:tasks.workunit.client.1.smithi027.stdout:7/2: truncate - no filename 2024-04-02T04:42:38.515 INFO:tasks.workunit.client.0.smithi016.stdout:7/364: write d0/d46/d37/d3a/f7d [162643,72307] 0 2024-04-02T04:42:38.518 INFO:tasks.workunit.client.0.smithi016.stdout:7/365: write d0/d46/d16/d49/f11 [526725,4160] 0 2024-04-02T04:42:38.521 INFO:tasks.workunit.client.0.smithi016.stdout:7/366: creat d0/d46/d37/d3a/d43/f85 x:0 0 0 2024-04-02T04:42:38.522 INFO:tasks.workunit.client.0.smithi016.stdout:1/361: getdents d14/d19/d1d/d45 0 2024-04-02T04:42:38.523 INFO:tasks.workunit.client.0.smithi016.stdout:1/362: chown d14/d19/d20/f5b 2401687 1 2024-04-02T04:42:38.524 INFO:tasks.workunit.client.0.smithi016.stdout:7/367: creat d0/d21/f86 x:0 0 0 2024-04-02T04:42:38.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.522+0000 7fea71243700 1 -- 172.21.15.16:0/4088074323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea6c101d10 msgr2=0x7fea6c102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:38.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.522+0000 7fea71243700 1 --2- 172.21.15.16:0/4088074323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea6c101d10 0x7fea6c102170 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fea60009910 tx=0x7fea60009c20 comp rx=0 tx=0).stop 2024-04-02T04:42:38.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.522+0000 7fea71243700 1 -- 172.21.15.16:0/4088074323 shutdown_connections 2024-04-02T04:42:38.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.522+0000 7fea71243700 1 --2- 172.21.15.16:0/4088074323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea6c101d10 0x7fea6c102170 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:38.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.522+0000 7fea71243700 1 --2- 172.21.15.16:0/4088074323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea6c100b10 0x7fea6c100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:38.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.522+0000 7fea71243700 1 -- 172.21.15.16:0/4088074323 >> 172.21.15.16:0/4088074323 conn(0x7fea6c0fc0d0 msgr2=0x7fea6c0fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:42:38.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.522+0000 7fea71243700 1 -- 172.21.15.16:0/4088074323 shutdown_connections 2024-04-02T04:42:38.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.522+0000 7fea71243700 1 -- 172.21.15.16:0/4088074323 wait complete. 2024-04-02T04:42:38.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.523+0000 7fea71243700 1 Processor -- start 2024-04-02T04:42:38.526 INFO:tasks.workunit.client.0.smithi016.stdout:1/363: mkdir d14/d19/d1e/d71 0 2024-04-02T04:42:38.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.523+0000 7fea71243700 1 -- start start 2024-04-02T04:42:38.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.524+0000 7fea71243700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea6c100b10 0x7fea6c074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:38.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.524+0000 7fea71243700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea6c101d10 0x7fea6c0731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:38.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.524+0000 7fea71243700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c0736e0 con 0x7fea6c101d10 2024-04-02T04:42:38.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.524+0000 7fea71243700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c073820 con 0x7fea6c100b10 2024-04-02T04:42:38.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.524+0000 7fea6ad9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea6c100b10 0x7fea6c074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:38.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.524+0000 7fea6a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea6c101d10 0x7fea6c0731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:38.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.524+0000 7fea6ad9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea6c100b10 0x7fea6c074b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.16:54548/0 (socket says 172.21.15.16:54548) 2024-04-02T04:42:38.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.524+0000 7fea6a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea6c101d10 0x7fea6c0731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51968/0 (socket says 172.21.15.16:51968) 2024-04-02T04:42:38.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.524+0000 7fea6ad9d700 1 -- 172.21.15.16:0/1928380577 learned_addr learned my addr 172.21.15.16:0/1928380577 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:42:38.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.524+0000 7fea6ad9d700 1 -- 172.21.15.16:0/1928380577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea6c101d10 msgr2=0x7fea6c0731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:38.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.524+0000 7fea6ad9d700 1 --2- 172.21.15.16:0/1928380577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea6c101d10 0x7fea6c0731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:38.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.524+0000 7fea6ad9d700 1 -- 172.21.15.16:0/1928380577 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea600095f0 con 0x7fea6c100b10 2024-04-02T04:42:38.529 INFO:tasks.workunit.client.0.smithi016.stdout:4/390: mkdir d5/d17/d76 0 2024-04-02T04:42:38.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.525+0000 7fea6a59c700 1 --2- 172.21.15.16:0/1928380577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea6c101d10 0x7fea6c0731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T04:42:38.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.525+0000 7fea6ad9d700 1 --2- 172.21.15.16:0/1928380577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea6c100b10 0x7fea6c074b00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fea5c00e8f0 tx=0x7fea5c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:42:38.530 INFO:tasks.workunit.client.0.smithi016.stdout:1/364: chown d14/d19/d1d/d29/f6a 81818506 1 2024-04-02T04:42:38.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.525+0000 7fea53fff700 1 -- 172.21.15.16:0/1928380577 <== mon.1 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea5c009790 con 0x7fea6c100b10 2024-04-02T04:42:38.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.525+0000 7fea71243700 1 -- 172.21.15.16:0/1928380577 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea6c073b00 con 0x7fea6c100b10 2024-04-02T04:42:38.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.525+0000 7fea53fff700 1 -- 172.21.15.16:0/1928380577 <== mon.1 v2:172.21.15.27:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fea5c0044e0 con 0x7fea6c100b10 2024-04-02T04:42:38.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.525+0000 7fea71243700 1 -- 172.21.15.16:0/1928380577 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea6c19e530 con 0x7fea6c100b10 2024-04-02T04:42:38.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.525+0000 7fea53fff700 1 -- 172.21.15.16:0/1928380577 <== mon.1 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea5c003e00 con 0x7fea6c100b10 2024-04-02T04:42:38.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.527+0000 7fea71243700 1 -- 172.21.15.16:0/1928380577 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea58005320 con 0x7fea6c100b10 2024-04-02T04:42:38.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.527+0000 7fea53fff700 1 -- 172.21.15.16:0/1928380577 <== mon.1 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) v1 ==== 91380+0+0 (secure 0 0 0) 0x7fea5c0104c0 con 0x7fea6c100b10 2024-04-02T04:42:38.531 INFO:tasks.workunit.client.0.smithi016.stdout:7/368: dread d0/d46/d37/f81 [0,4194304] 0 2024-04-02T04:42:38.531 INFO:tasks.workunit.client.0.smithi016.stdout:4/391: write d5/de/f14 [1334913,115073] 0 2024-04-02T04:42:38.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.527+0000 7fea53fff700 1 --2- 172.21.15.16:0/1928380577 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7fea5406d4a0 0x7fea5406f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:38.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.528+0000 7fea53fff700 1 -- 172.21.15.16:0/1928380577 <== mon.1 v2:172.21.15.27:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fea5c014070 con 0x7fea6c100b10 2024-04-02T04:42:38.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.528+0000 7fea6a59c700 1 --2- 172.21.15.16:0/1928380577 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7fea5406d4a0 0x7fea5406f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:38.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.528+0000 7fea6a59c700 1 --2- 172.21.15.16:0/1928380577 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7fea5406d4a0 0x7fea5406f960 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fea600059d0 tx=0x7fea60005940 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:42:38.532 INFO:tasks.workunit.client.0.smithi016.stdout:1/365: mknod d14/d19/d20/c72 0 2024-04-02T04:42:38.534 INFO:tasks.workunit.client.0.smithi016.stdout:0/344: truncate d0/d16/d2e/d53/d6f/f74 1431549 0 2024-04-02T04:42:38.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.534+0000 7fea53fff700 1 -- 172.21.15.16:0/1928380577 <== mon.1 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184299 (secure 0 0 0) 0x7fea5c05a6c0 con 0x7fea6c100b10 2024-04-02T04:42:38.537 INFO:tasks.workunit.client.0.smithi016.stdout:4/392: mknod d5/d1d/d5f/d65/c77 0 2024-04-02T04:42:38.540 INFO:tasks.workunit.client.0.smithi016.stdout:8/370: rename d1/l2d to d1/d70/l7c 0 2024-04-02T04:42:38.541 INFO:tasks.workunit.client.0.smithi016.stdout:0/345: creat d0/d16/f7a x:0 0 0 2024-04-02T04:42:38.543 INFO:tasks.workunit.client.0.smithi016.stdout:7/369: getdents d0/d46/d16/d83 0 2024-04-02T04:42:38.545 INFO:tasks.workunit.client.0.smithi016.stdout:3/434: dwrite d2/f14 [0,4194304] 0 2024-04-02T04:42:38.547 INFO:tasks.workunit.client.0.smithi016.stdout:8/371: rmdir d1/d70/d71 39 2024-04-02T04:42:38.553 INFO:tasks.workunit.client.0.smithi016.stdout:3/435: dwrite d2/d15/d16/d3d/d46/f83 [0,4194304] 0 2024-04-02T04:42:38.565 INFO:tasks.workunit.client.0.smithi016.stdout:7/370: mkdir d0/d87 0 2024-04-02T04:42:38.568 INFO:tasks.workunit.client.0.smithi016.stdout:0/346: link d0/d16/d60/d71/f72 d0/d16/d24/f7b 0 2024-04-02T04:42:38.569 INFO:tasks.workunit.client.0.smithi016.stdout:2/317: sync 2024-04-02T04:42:38.569 INFO:tasks.workunit.client.0.smithi016.stdout:9/390: sync 2024-04-02T04:42:38.569 INFO:tasks.workunit.client.0.smithi016.stdout:5/390: sync 2024-04-02T04:42:38.569 INFO:tasks.workunit.client.0.smithi016.stdout:7/371: dwrite d0/d46/d16/d49/f11 [0,4194304] 0 2024-04-02T04:42:38.573 INFO:tasks.workunit.client.0.smithi016.stdout:7/372: dread - d0/d46/d37/d3a/d43/f85 zero size 2024-04-02T04:42:38.581 INFO:tasks.workunit.client.0.smithi016.stdout:0/347: dread d0/d16/d2e/d37/f3c [0,4194304] 0 2024-04-02T04:42:38.581 INFO:tasks.workunit.client.0.smithi016.stdout:4/393: creat d5/d1d/f78 x:0 0 0 2024-04-02T04:42:38.584 INFO:tasks.workunit.client.0.smithi016.stdout:9/391: mkdir d0/d4e/d50/d63/d70 0 2024-04-02T04:42:38.590 INFO:tasks.workunit.client.0.smithi016.stdout:5/391: unlink d3/df/d18/c39 0 2024-04-02T04:42:38.591 INFO:tasks.workunit.client.0.smithi016.stdout:6/443: truncate d0/d9/d16/f20 343589 0 2024-04-02T04:42:38.592 INFO:tasks.workunit.client.0.smithi016.stdout:6/444: write d0/d9/d16/d46/f7b [973938,118667] 0 2024-04-02T04:42:38.593 INFO:tasks.workunit.client.0.smithi016.stdout:6/445: fsync d0/d9/d16/d46/f68 0 2024-04-02T04:42:38.593 INFO:tasks.workunit.client.0.smithi016.stdout:0/348: unlink d0/d16/d2e/d37/f79 0 2024-04-02T04:42:38.598 INFO:tasks.workunit.client.0.smithi016.stdout:7/373: symlink d0/l88 0 2024-04-02T04:42:38.598 INFO:tasks.workunit.client.0.smithi016.stdout:7/374: write d0/d21/f36 [1022195,61787] 0 2024-04-02T04:42:38.600 INFO:tasks.workunit.client.0.smithi016.stdout:9/392: rename d0/d33/f35 to d0/d4e/d50/d63/d6e/d4b/d58/d6a/f71 0 2024-04-02T04:42:38.602 INFO:tasks.workunit.client.0.smithi016.stdout:5/392: mknod d3/df/d18/d26/d48/c99 0 2024-04-02T04:42:38.616 INFO:tasks.workunit.client.0.smithi016.stdout:9/393: rmdir d0/d9 39 2024-04-02T04:42:38.616 INFO:tasks.workunit.client.0.smithi016.stdout:9/394: fdatasync d0/f2c 0 2024-04-02T04:42:38.622 INFO:tasks.workunit.client.0.smithi016.stdout:1/366: dwrite d14/d19/d20/f3f [0,4194304] 0 2024-04-02T04:42:38.627 INFO:tasks.workunit.client.0.smithi016.stdout:8/372: write d1/f19 [2025244,39306] 0 2024-04-02T04:42:38.628 INFO:tasks.workunit.client.0.smithi016.stdout:9/395: dread d0/d4e/d50/d63/d6e/d1a/f48 [0,4194304] 0 2024-04-02T04:42:38.628 INFO:tasks.workunit.client.0.smithi016.stdout:2/318: getdents d12/d35/d3f 0 2024-04-02T04:42:38.628 INFO:tasks.workunit.client.0.smithi016.stdout:8/373: fdatasync d1/d6/f2e 0 2024-04-02T04:42:38.631 INFO:tasks.workunit.client.0.smithi016.stdout:2/319: read d12/d14/f1f [2394325,121027] 0 2024-04-02T04:42:38.631 INFO:tasks.workunit.client.0.smithi016.stdout:2/320: readlink d12/d35/d3f/l2e 0 2024-04-02T04:42:38.639 INFO:tasks.workunit.client.0.smithi016.stdout:4/394: getdents d5/d17 0 2024-04-02T04:42:38.641 INFO:tasks.workunit.client.0.smithi016.stdout:2/321: dread d12/d26/f5a [0,4194304] 0 2024-04-02T04:42:38.642 INFO:tasks.workunit.client.0.smithi016.stdout:2/322: readlink d12/d26/l50 0 2024-04-02T04:42:38.645 INFO:tasks.workunit.client.0.smithi016.stdout:8/374: write d1/d6/f4b [802244,21332] 0 2024-04-02T04:42:38.648 INFO:tasks.workunit.client.0.smithi016.stdout:3/436: dwrite d2/fa [0,4194304] 0 2024-04-02T04:42:38.649 INFO:tasks.workunit.client.0.smithi016.stdout:3/437: fdatasync d2/d15/d16/f54 0 2024-04-02T04:42:38.651 INFO:tasks.workunit.client.0.smithi016.stdout:8/375: dread d1/d6/d36/d4f/f62 [0,4194304] 0 2024-04-02T04:42:38.661 INFO:tasks.workunit.client.0.smithi016.stdout:4/395: creat d5/de/f79 x:0 0 0 2024-04-02T04:42:38.662 INFO:tasks.workunit.client.0.smithi016.stdout:4/396: chown d5/d8/fd 154 1 2024-04-02T04:42:38.664 INFO:tasks.workunit.client.0.smithi016.stdout:2/323: creat d12/d26/f72 x:0 0 0 2024-04-02T04:42:38.667 INFO:tasks.workunit.client.0.smithi016.stdout:7/375: rename d0/d46/c2a to d0/c89 0 2024-04-02T04:42:38.674 INFO:tasks.workunit.client.0.smithi016.stdout:5/393: getdents d3/d40/d4f/d6b 0 2024-04-02T04:42:38.675 INFO:tasks.workunit.client.0.smithi016.stdout:8/376: chown d1/d6/d36/d3f/f47 4743 1 2024-04-02T04:42:38.679 INFO:tasks.workunit.client.0.smithi016.stdout:6/446: truncate d0/d1f/f2d 4778910 0 2024-04-02T04:42:38.681 INFO:tasks.workunit.client.0.smithi016.stdout:0/349: truncate d0/f12 1363998 0 2024-04-02T04:42:38.682 INFO:tasks.workunit.client.0.smithi016.stdout:0/350: readlink d0/d16/l1b 0 2024-04-02T04:42:38.689 INFO:tasks.workunit.client.0.smithi016.stdout:8/377: creat d1/d6/d36/d3f/d40/d49/d4a/d4c/f7d x:0 0 0 2024-04-02T04:42:38.691 INFO:tasks.workunit.client.0.smithi016.stdout:6/447: creat d0/d9/d16/d56/d59/f86 x:0 0 0 2024-04-02T04:42:38.694 INFO:tasks.workunit.client.0.smithi016.stdout:7/376: mkdir d0/d46/d37/d3a/d84/d8a 0 2024-04-02T04:42:38.696 INFO:tasks.workunit.client.0.smithi016.stdout:7/377: write d0/d46/d37/d47/f6b [1044425,12807] 0 2024-04-02T04:42:38.696 INFO:tasks.workunit.client.0.smithi016.stdout:0/351: fdatasync d0/d16/d24/f2a 0 2024-04-02T04:42:38.697 INFO:tasks.workunit.client.0.smithi016.stdout:9/396: link d0/d4e/d50/d63/d6e/d1a/l49 d0/d4e/d50/d63/d6e/d4b/d58/l72 0 2024-04-02T04:42:38.699 INFO:tasks.workunit.client.0.smithi016.stdout:1/367: truncate d14/f48 4189061 0 2024-04-02T04:42:38.699 INFO:tasks.workunit.client.0.smithi016.stdout:1/368: write d14/f17 [3447510,77004] 0 2024-04-02T04:42:38.699 INFO:tasks.workunit.client.0.smithi016.stdout:8/378: creat d1/d31/f7e x:0 0 0 2024-04-02T04:42:38.707 INFO:tasks.workunit.client.0.smithi016.stdout:7/378: creat d0/d28/d64/f8b x:0 0 0 2024-04-02T04:42:38.707 INFO:tasks.workunit.client.0.smithi016.stdout:7/379: write d0/d28/f59 [1102817,29619] 0 2024-04-02T04:42:38.708 INFO:tasks.workunit.client.0.smithi016.stdout:9/397: creat d0/d3/d31/d53/f73 x:0 0 0 2024-04-02T04:42:38.709 INFO:tasks.workunit.client.0.smithi016.stdout:9/398: write d0/d3/f30 [4734121,39759] 0 2024-04-02T04:42:38.714 INFO:tasks.workunit.client.0.smithi016.stdout:1/369: rename d14/d19/d20/c72 to d14/d19/d20/c73 0 2024-04-02T04:42:38.714 INFO:tasks.workunit.client.0.smithi016.stdout:3/438: write d2/d15/d1a/d45/d56/f7b [1110742,72451] 0 2024-04-02T04:42:38.715 INFO:tasks.workunit.client.0.smithi016.stdout:4/397: getdents d5/d17/d18/d1f 0 2024-04-02T04:42:38.716 INFO:tasks.workunit.client.0.smithi016.stdout:3/439: truncate d2/d15/d1a/f22 934122 0 2024-04-02T04:42:38.717 INFO:tasks.workunit.client.0.smithi016.stdout:8/379: mknod d1/d6/d36/d3f/d40/d63/c7f 0 2024-04-02T04:42:38.723 INFO:tasks.workunit.client.0.smithi016.stdout:0/352: fsync d0/f10 0 2024-04-02T04:42:38.728 INFO:tasks.workunit.client.0.smithi016.stdout:7/380: rename d0/d28/c71 to d0/d46/d52/c8c 0 2024-04-02T04:42:38.728 INFO:tasks.workunit.client.0.smithi016.stdout:7/381: fsync d0/d46/d16/d49/f3e 0 2024-04-02T04:42:38.730 INFO:tasks.workunit.client.1.smithi027.stdout:5/2: mkdir d0 0 2024-04-02T04:42:38.733 INFO:tasks.workunit.client.1.smithi027.stdout:1/4: symlink d0/l2 0 2024-04-02T04:42:38.734 INFO:tasks.workunit.client.0.smithi016.stdout:4/398: rmdir d5/d1d/d5f/d65 39 2024-04-02T04:42:38.734 INFO:tasks.workunit.client.0.smithi016.stdout:7/382: dwrite d0/d21/f4e [0,4194304] 0 2024-04-02T04:42:38.734 INFO:tasks.workunit.client.0.smithi016.stdout:7/383: readlink d0/l88 0 2024-04-02T04:42:38.735 INFO:tasks.workunit.client.0.smithi016.stdout:7/384: readlink d0/d21/l23 0 2024-04-02T04:42:38.735 INFO:tasks.workunit.client.1.smithi027.stdout:6/0: getdents . 0 2024-04-02T04:42:38.736 INFO:tasks.workunit.client.1.smithi027.stdout:6/1: chown . 25 1 2024-04-02T04:42:38.736 INFO:tasks.workunit.client.1.smithi027.stdout:6/2: chown . 142 1 2024-04-02T04:42:38.741 INFO:tasks.workunit.client.0.smithi016.stdout:2/324: dwrite d12/f4c [0,4194304] 0 2024-04-02T04:42:38.741 INFO:tasks.workunit.client.1.smithi027.stdout:4/16: creat f0 x:0 0 0 2024-04-02T04:42:38.742 INFO:tasks.workunit.client.0.smithi016.stdout:2/325: fdatasync d12/d35/d3f/f39 0 2024-04-02T04:42:38.742 INFO:tasks.workunit.client.1.smithi027.stdout:4/17: truncate f0 737858 0 2024-04-02T04:42:38.742 INFO:tasks.workunit.client.1.smithi027.stdout:4/18: stat f0 0 2024-04-02T04:42:38.743 INFO:tasks.workunit.client.1.smithi027.stdout:4/19: read f0 [508353,48747] 0 2024-04-02T04:42:38.744 INFO:tasks.workunit.client.0.smithi016.stdout:0/353: creat d0/d16/d2e/d53/f7c x:0 0 0 2024-04-02T04:42:38.745 INFO:tasks.workunit.client.1.smithi027.stdout:4/20: dread f0 [0,4194304] 0 2024-04-02T04:42:38.746 INFO:tasks.workunit.client.1.smithi027.stdout:4/21: chown f0 1 1 2024-04-02T04:42:38.747 INFO:tasks.workunit.client.1.smithi027.stdout:2/20: rename d1/l2 to d1/l3 0 2024-04-02T04:42:38.748 INFO:tasks.workunit.client.1.smithi027.stdout:4/22: dread f0 [0,4194304] 0 2024-04-02T04:42:38.751 INFO:tasks.workunit.client.1.smithi027.stdout:3/8: creat d1/f2 x:0 0 0 2024-04-02T04:42:38.751 INFO:tasks.workunit.client.1.smithi027.stdout:3/9: readlink - no filename 2024-04-02T04:42:38.752 INFO:tasks.workunit.client.1.smithi027.stdout:3/10: truncate d1/f2 929649 0 2024-04-02T04:42:38.753 INFO:tasks.workunit.client.0.smithi016.stdout:9/399: mkdir d0/d4e/d50/d63/d6e/d1a/d74 0 2024-04-02T04:42:38.754 INFO:tasks.workunit.client.1.smithi027.stdout:1/5: symlink d0/l3 0 2024-04-02T04:42:38.754 INFO:tasks.workunit.client.1.smithi027.stdout:1/6: rename d0 to d0/d4 22 2024-04-02T04:42:38.756 INFO:tasks.workunit.client.0.smithi016.stdout:1/370: creat d14/d41/d43/d68/f74 x:0 0 0 2024-04-02T04:42:38.760 INFO:tasks.workunit.client.1.smithi027.stdout:9/0: write - no filename 2024-04-02T04:42:38.762 INFO:tasks.workunit.client.1.smithi027.stdout:7/3: creat f0 x:0 0 0 2024-04-02T04:42:38.765 INFO:tasks.workunit.client.0.smithi016.stdout:6/448: getdents d0/d9/d16/d56 0 2024-04-02T04:42:38.765 INFO:tasks.workunit.client.1.smithi027.stdout:2/21: rmdir d1 39 2024-04-02T04:42:38.767 INFO:tasks.workunit.client.1.smithi027.stdout:4/23: mknod c1 0 2024-04-02T04:42:38.769 INFO:tasks.workunit.client.0.smithi016.stdout:0/354: rename d0/d16/d2e/d53/f52 to d0/d16/d60/d71/f7d 0 2024-04-02T04:42:38.770 INFO:tasks.workunit.client.1.smithi027.stdout:5/3: symlink d0/l1 0 2024-04-02T04:42:38.772 INFO:tasks.workunit.client.0.smithi016.stdout:9/400: unlink d0/d4a/c4c 0 2024-04-02T04:42:38.773 INFO:tasks.workunit.client.1.smithi027.stdout:1/7: creat d0/f5 x:0 0 0 2024-04-02T04:42:38.773 INFO:tasks.workunit.client.1.smithi027.stdout:1/8: fdatasync d0/f5 0 2024-04-02T04:42:38.773 INFO:tasks.workunit.client.0.smithi016.stdout:2/326: dwrite d12/d35/d3f/f61 [0,4194304] 0 2024-04-02T04:42:38.774 INFO:tasks.workunit.client.0.smithi016.stdout:9/401: read - d0/d4e/d50/d63/d6e/f44 zero size 2024-04-02T04:42:38.776 INFO:tasks.workunit.client.0.smithi016.stdout:4/399: symlink d5/l7a 0 2024-04-02T04:42:38.776 INFO:tasks.workunit.client.1.smithi027.stdout:6/3: getdents . 0 2024-04-02T04:42:38.777 INFO:tasks.workunit.client.0.smithi016.stdout:9/402: read d0/d33/f36 [1572503,53169] 0 2024-04-02T04:42:38.777 INFO:tasks.workunit.client.0.smithi016.stdout:4/400: write d5/d1d/f78 [177071,95722] 0 2024-04-02T04:42:38.778 INFO:tasks.workunit.client.0.smithi016.stdout:7/385: symlink d0/l8d 0 2024-04-02T04:42:38.778 INFO:tasks.workunit.client.0.smithi016.stdout:9/403: dread - d0/d4a/f69 zero size 2024-04-02T04:42:38.779 INFO:tasks.workunit.client.0.smithi016.stdout:4/401: dread - d5/d8/d43/f6e zero size 2024-04-02T04:42:38.779 INFO:tasks.workunit.client.1.smithi027.stdout:8/0: creat f0 x:0 0 0 2024-04-02T04:42:38.780 INFO:tasks.workunit.client.1.smithi027.stdout:8/1: truncate f0 937357 0 2024-04-02T04:42:38.780 INFO:tasks.workunit.client.0.smithi016.stdout:9/404: chown d0/l3f 4139642 1 2024-04-02T04:42:38.780 INFO:tasks.workunit.client.1.smithi027.stdout:8/2: fdatasync f0 0 2024-04-02T04:42:38.781 INFO:tasks.workunit.client.1.smithi027.stdout:8/3: fdatasync f0 0 2024-04-02T04:42:38.781 INFO:tasks.workunit.client.1.smithi027.stdout:8/4: stat f0 0 2024-04-02T04:42:38.781 INFO:tasks.workunit.client.0.smithi016.stdout:8/380: creat d1/d6/d36/d3f/f80 x:0 0 0 2024-04-02T04:42:38.783 INFO:tasks.workunit.client.1.smithi027.stdout:8/5: dread f0 [0,4194304] 0 2024-04-02T04:42:38.783 INFO:tasks.workunit.client.1.smithi027.stdout:8/6: rmdir - no directory 2024-04-02T04:42:38.784 INFO:tasks.workunit.client.1.smithi027.stdout:8/7: dread f0 [0,4194304] 0 2024-04-02T04:42:38.786 INFO:tasks.workunit.client.1.smithi027.stdout:8/8: dread f0 [0,4194304] 0 2024-04-02T04:42:38.786 INFO:tasks.workunit.client.1.smithi027.stdout:8/9: rmdir - no directory 2024-04-02T04:42:38.789 INFO:tasks.workunit.client.0.smithi016.stdout:3/440: write d2/d15/d1a/f53 [989554,53040] 0 2024-04-02T04:42:38.790 INFO:tasks.workunit.client.1.smithi027.stdout:5/4: mknod d0/c2 0 2024-04-02T04:42:38.791 INFO:tasks.workunit.client.1.smithi027.stdout:1/9: creat d0/f6 x:0 0 0 2024-04-02T04:42:38.792 INFO:tasks.workunit.client.1.smithi027.stdout:1/10: fdatasync d0/f5 0 2024-04-02T04:42:38.792 INFO:tasks.workunit.client.0.smithi016.stdout:7/386: dread d0/d46/f2f [0,4194304] 0 2024-04-02T04:42:38.793 INFO:tasks.workunit.client.1.smithi027.stdout:1/11: chown d0/l3 2697411 1 2024-04-02T04:42:38.793 INFO:tasks.workunit.client.1.smithi027.stdout:0/9: getdents . 0 2024-04-02T04:42:38.795 INFO:tasks.workunit.client.1.smithi027.stdout:1/12: write d0/f6 [662816,14498] 0 2024-04-02T04:42:38.795 INFO:tasks.workunit.client.0.smithi016.stdout:1/371: truncate d14/d19/d1e/f23 982144 0 2024-04-02T04:42:38.795 INFO:tasks.workunit.client.0.smithi016.stdout:0/355: dread d0/d16/f31 [0,4194304] 0 2024-04-02T04:42:38.797 INFO:tasks.workunit.client.1.smithi027.stdout:9/1: mknod c0 0 2024-04-02T04:42:38.797 INFO:tasks.workunit.client.1.smithi027.stdout:9/2: write - no filename 2024-04-02T04:42:38.798 INFO:tasks.workunit.client.1.smithi027.stdout:9/3: fdatasync - no filename 2024-04-02T04:42:38.801 INFO:tasks.workunit.client.0.smithi016.stdout:5/394: write d3/df/d18/d26/f4b [1756834,4244] 0 2024-04-02T04:42:38.801 INFO:tasks.workunit.client.1.smithi027.stdout:2/22: symlink d1/l4 0 2024-04-02T04:42:38.803 INFO:tasks.workunit.client.0.smithi016.stdout:9/405: mkdir d0/d4a/d75 0 2024-04-02T04:42:38.805 INFO:tasks.workunit.client.0.smithi016.stdout:4/402: rename d5/d1d/f23 to d5/d8/f7b 0 2024-04-02T04:42:38.806 INFO:tasks.workunit.client.1.smithi027.stdout:4/24: rename f0 to f2 0 2024-04-02T04:42:38.806 INFO:tasks.workunit.client.1.smithi027.stdout:4/25: chown c1 1849073922 1 2024-04-02T04:42:38.807 INFO:tasks.workunit.client.1.smithi027.stdout:4/26: dread f2 [0,4194304] 0 2024-04-02T04:42:38.808 INFO:tasks.workunit.client.0.smithi016.stdout:7/387: creat d0/d46/d16/f8e x:0 0 0 2024-04-02T04:42:38.809 INFO:tasks.workunit.client.1.smithi027.stdout:0/10: mkdir d4 0 2024-04-02T04:42:38.809 INFO:tasks.workunit.client.0.smithi016.stdout:7/388: write d0/d46/fc [2627776,37985] 0 2024-04-02T04:42:38.810 INFO:tasks.workunit.client.1.smithi027.stdout:0/11: write f1 [202512,102304] 0 2024-04-02T04:42:38.810 INFO:tasks.workunit.client.1.smithi027.stdout:1/13: creat d0/f7 x:0 0 0 2024-04-02T04:42:38.810 INFO:tasks.workunit.client.1.smithi027.stdout:0/12: stat f2 0 2024-04-02T04:42:38.811 INFO:tasks.workunit.client.1.smithi027.stdout:6/4: creat f0 x:0 0 0 2024-04-02T04:42:38.811 INFO:tasks.workunit.client.1.smithi027.stdout:1/14: chown d0/f5 0 1 2024-04-02T04:42:38.812 INFO:tasks.workunit.client.1.smithi027.stdout:9/4: rename c0 to c1 0 2024-04-02T04:42:38.814 INFO:tasks.workunit.client.0.smithi016.stdout:5/395: mknod d3/d40/d4f/d6b/d2a/d6d/c9a 0 2024-04-02T04:42:38.817 INFO:tasks.workunit.client.1.smithi027.stdout:8/10: unlink f0 0 2024-04-02T04:42:38.817 INFO:tasks.workunit.client.1.smithi027.stdout:8/11: truncate - no filename 2024-04-02T04:42:38.817 INFO:tasks.workunit.client.1.smithi027.stdout:8/12: chown . 3271623 1 2024-04-02T04:42:38.817 INFO:tasks.workunit.client.1.smithi027.stdout:8/13: dread - no filename 2024-04-02T04:42:38.818 INFO:tasks.workunit.client.1.smithi027.stdout:8/14: rename - no filename 2024-04-02T04:42:38.820 INFO:tasks.workunit.client.0.smithi016.stdout:0/356: rename d0/d16/d2e/d36/l49 to d0/d16/d23/d3e/d57/l7e 0 2024-04-02T04:42:38.821 INFO:tasks.workunit.client.1.smithi027.stdout:4/27: rename c1 to c3 0 2024-04-02T04:42:38.825 INFO:tasks.workunit.client.1.smithi027.stdout:6/5: creat f1 x:0 0 0 2024-04-02T04:42:38.825 INFO:tasks.workunit.client.1.smithi027.stdout:6/6: dread - f0 zero size 2024-04-02T04:42:38.826 INFO:tasks.workunit.client.1.smithi027.stdout:6/7: readlink - no filename 2024-04-02T04:42:38.826 INFO:tasks.workunit.client.1.smithi027.stdout:9/5: mknod c2 0 2024-04-02T04:42:38.826 INFO:tasks.workunit.client.1.smithi027.stdout:6/8: write f1 [426175,83782] 0 2024-04-02T04:42:38.828 INFO:tasks.workunit.client.1.smithi027.stdout:7/4: getdents . 0 2024-04-02T04:42:38.830 INFO:tasks.workunit.client.0.smithi016.stdout:1/372: mkdir d14/d19/d1e/d5e/d75 0 2024-04-02T04:42:38.831 INFO:tasks.workunit.client.1.smithi027.stdout:6/9: dread f1 [0,4194304] 0 2024-04-02T04:42:38.833 INFO:tasks.workunit.client.0.smithi016.stdout:5/396: unlink d3/d4c/l58 0 2024-04-02T04:42:38.834 INFO:tasks.workunit.client.0.smithi016.stdout:5/397: readlink d3/df/d18/l2b 0 2024-04-02T04:42:38.837 INFO:tasks.workunit.client.0.smithi016.stdout:8/381: truncate d1/f19 951298 0 2024-04-02T04:42:38.837 INFO:tasks.workunit.client.1.smithi027.stdout:0/13: mknod d4/c5 0 2024-04-02T04:42:38.840 INFO:tasks.workunit.client.0.smithi016.stdout:7/389: rename d0/d28/l51 to d0/d46/d37/d3a/d4c/l8f 0 2024-04-02T04:42:38.845 INFO:tasks.workunit.client.1.smithi027.stdout:7/5: rename f0 to f1 0 2024-04-02T04:42:38.845 INFO:tasks.workunit.client.0.smithi016.stdout:9/406: write d0/f2d [1386827,112023] 0 2024-04-02T04:42:38.845 INFO:tasks.workunit.client.1.smithi027.stdout:0/14: dwrite f2 [4194304,4194304] 0 2024-04-02T04:42:38.846 INFO:tasks.workunit.client.0.smithi016.stdout:2/327: getdents d12/d35/d3f/d56 0 2024-04-02T04:42:38.850 INFO:tasks.workunit.client.1.smithi027.stdout:5/5: getdents d0 0 2024-04-02T04:42:38.850 INFO:tasks.workunit.client.1.smithi027.stdout:5/6: dwrite - no filename 2024-04-02T04:42:38.852 INFO:tasks.workunit.client.0.smithi016.stdout:8/382: chown d1/d6/d36/c64 7563 1 2024-04-02T04:42:38.855 INFO:tasks.workunit.client.1.smithi027.stdout:0/15: dwrite f1 [0,4194304] 0 2024-04-02T04:42:38.858 INFO:tasks.workunit.client.1.smithi027.stdout:9/6: rename c1 to c3 0 2024-04-02T04:42:38.860 INFO:tasks.workunit.client.0.smithi016.stdout:1/373: creat d14/d19/d1d/d29/d39/d4d/f76 x:0 0 0 2024-04-02T04:42:38.864 INFO:tasks.workunit.client.0.smithi016.stdout:5/398: mknod d3/c9b 0 2024-04-02T04:42:38.864 INFO:tasks.workunit.client.1.smithi027.stdout:4/28: truncate f2 214157 0 2024-04-02T04:42:38.865 INFO:tasks.workunit.client.1.smithi027.stdout:7/6: mkdir d2 0 2024-04-02T04:42:38.865 INFO:tasks.workunit.client.1.smithi027.stdout:7/7: read - f1 zero size 2024-04-02T04:42:38.867 INFO:tasks.workunit.client.1.smithi027.stdout:2/23: getdents d1 0 2024-04-02T04:42:38.867 INFO:tasks.workunit.client.1.smithi027.stdout:2/24: write - no filename 2024-04-02T04:42:38.867 INFO:tasks.workunit.client.1.smithi027.stdout:2/25: truncate - no filename 2024-04-02T04:42:38.868 INFO:tasks.workunit.client.0.smithi016.stdout:4/403: getdents d5/d17/d18/d6d 0 2024-04-02T04:42:38.869 INFO:tasks.workunit.client.1.smithi027.stdout:6/10: rename f1 to f2 0 2024-04-02T04:42:38.869 INFO:tasks.workunit.client.1.smithi027.stdout:6/11: stat f2 0 2024-04-02T04:42:38.870 INFO:tasks.workunit.client.0.smithi016.stdout:8/383: dread d1/d6/d36/d3f/d40/d49/f43 [0,4194304] 0 2024-04-02T04:42:38.870 INFO:tasks.workunit.client.0.smithi016.stdout:5/399: write d3/df/d18/d26/d2c/f8d [562106,40356] 0 2024-04-02T04:42:38.871 INFO:tasks.workunit.client.1.smithi027.stdout:5/7: creat d0/f3 x:0 0 0 2024-04-02T04:42:38.872 INFO:tasks.workunit.client.0.smithi016.stdout:1/374: dwrite d14/d19/d20/f3b [0,4194304] 0 2024-04-02T04:42:38.872 INFO:tasks.workunit.client.1.smithi027.stdout:5/8: dread - d0/f3 zero size 2024-04-02T04:42:38.875 INFO:tasks.workunit.client.1.smithi027.stdout:4/29: symlink l4 0 2024-04-02T04:42:38.875 INFO:tasks.workunit.client.1.smithi027.stdout:4/30: readlink l4 0 2024-04-02T04:42:38.877 INFO:tasks.workunit.client.0.smithi016.stdout:8/384: chown d1/d6/d36/d3f/d40/d49/d4a/f56 14907 1 2024-04-02T04:42:38.877 INFO:tasks.workunit.client.1.smithi027.stdout:2/26: symlink d1/l5 0 2024-04-02T04:42:38.877 INFO:tasks.workunit.client.1.smithi027.stdout:2/27: dwrite - no filename 2024-04-02T04:42:38.877 INFO:tasks.workunit.client.1.smithi027.stdout:2/28: truncate - no filename 2024-04-02T04:42:38.877 INFO:tasks.workunit.client.0.smithi016.stdout:0/357: dwrite d0/f67 [0,4194304] 0 2024-04-02T04:42:38.879 INFO:tasks.workunit.client.1.smithi027.stdout:8/15: getdents . 0 2024-04-02T04:42:38.883 INFO:tasks.workunit.client.1.smithi027.stdout:6/12: rename f0 to f3 0 2024-04-02T04:42:38.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.881+0000 7fea71243700 1 -- 172.21.15.16:0/1928380577 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fea580058a0 con 0x7fea6c100b10 2024-04-02T04:42:38.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.882+0000 7fea53fff700 1 -- 172.21.15.16:0/1928380577 <== mon.1 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1843 (secure 0 0 0) 0x7fea5c05a270 con 0x7fea6c100b10 2024-04-02T04:42:38.885 INFO:teuthology.orchestra.run.smithi016.stdout:e15 2024-04-02T04:42:38.885 INFO:teuthology.orchestra.run.smithi016.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:42:38.885 INFO:teuthology.orchestra.run.smithi016.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:42:38.885 INFO:teuthology.orchestra.run.smithi016.stdout:legacy client fscid: 1 2024-04-02T04:42:38.885 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:42:38.885 INFO:teuthology.orchestra.run.smithi016.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:42:38.885 INFO:teuthology.orchestra.run.smithi016.stdout:fs_name cephfs 2024-04-02T04:42:38.886 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 15 2024-04-02T04:42:38.886 INFO:teuthology.orchestra.run.smithi016.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:42:38.886 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-04-02T04:39:30.740168+0000 2024-04-02T04:42:38.886 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-04-02T04:39:49.875319+0000 2024-04-02T04:42:38.886 INFO:teuthology.orchestra.run.smithi016.stdout:tableserver 0 2024-04-02T04:42:38.886 INFO:teuthology.orchestra.run.smithi016.stdout:root 0 2024-04-02T04:42:38.886 INFO:teuthology.orchestra.run.smithi016.stdout:session_timeout 60 2024-04-02T04:42:38.886 INFO:teuthology.orchestra.run.smithi016.stdout:session_autoclose 300 2024-04-02T04:42:38.886 INFO:teuthology.orchestra.run.smithi016.stdout:max_file_size 1099511627776 2024-04-02T04:42:38.886 INFO:teuthology.orchestra.run.smithi016.stdout:max_xattr_size 65536 2024-04-02T04:42:38.886 INFO:teuthology.orchestra.run.smithi016.stdout:required_client_features {} 2024-04-02T04:42:38.886 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure 0 2024-04-02T04:42:38.886 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure_osd_epoch 46 2024-04-02T04:42:38.887 INFO:teuthology.orchestra.run.smithi016.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:42:38.887 INFO:teuthology.orchestra.run.smithi016.stdout:max_mds 1 2024-04-02T04:42:38.887 INFO:teuthology.orchestra.run.smithi016.stdout:in 0 2024-04-02T04:42:38.887 INFO:teuthology.orchestra.run.smithi016.stdout:up {0=24285} 2024-04-02T04:42:38.887 INFO:teuthology.orchestra.run.smithi016.stdout:failed 2024-04-02T04:42:38.887 INFO:teuthology.orchestra.run.smithi016.stdout:damaged 2024-04-02T04:42:38.887 INFO:teuthology.orchestra.run.smithi016.stdout:stopped 2024-04-02T04:42:38.887 INFO:teuthology.orchestra.run.smithi016.stdout:data_pools [3] 2024-04-02T04:42:38.887 INFO:teuthology.orchestra.run.smithi016.stdout:metadata_pool 2 2024-04-02T04:42:38.887 INFO:teuthology.orchestra.run.smithi016.stdout:inline_data enabled 2024-04-02T04:42:38.887 INFO:teuthology.orchestra.run.smithi016.stdout:balancer 2024-04-02T04:42:38.887 INFO:teuthology.orchestra.run.smithi016.stdout:bal_rank_mask -1 2024-04-02T04:42:38.888 INFO:teuthology.orchestra.run.smithi016.stdout:standby_count_wanted 1 2024-04-02T04:42:38.888 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.zwrfoo{0:24285} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.27:6824/1461455976,v1:172.21.15.27:6825/1461455976] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:42:38.888 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:42:38.888 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:42:38.888 INFO:teuthology.orchestra.run.smithi016.stdout:Standby daemons: 2024-04-02T04:42:38.888 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:42:38.888 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.fyzrnm{-1:14488} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.16:6828/2578719859,v1:172.21.15.16:6829/2578719859] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:42:38.888 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.uhgkgg{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.16:6826/601263579,v1:172.21.15.16:6827/601263579] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:42:38.888 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.sdublg{-1:24295} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.27:6826/4177192740,v1:172.21.15.27:6827/4177192740] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:42:38.889 INFO:tasks.workunit.client.0.smithi016.stdout:9/407: creat d0/d9/f76 x:0 0 0 2024-04-02T04:42:38.889 INFO:tasks.workunit.client.1.smithi027.stdout:5/9: rmdir d0 39 2024-04-02T04:42:38.889 INFO:tasks.workunit.client.1.smithi027.stdout:7/8: getdents d2 0 2024-04-02T04:42:38.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.884+0000 7fea71243700 1 -- 172.21.15.16:0/1928380577 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7fea5406d4a0 msgr2=0x7fea5406f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:38.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.884+0000 7fea71243700 1 --2- 172.21.15.16:0/1928380577 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7fea5406d4a0 0x7fea5406f960 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fea600059d0 tx=0x7fea60005940 comp rx=0 tx=0).stop 2024-04-02T04:42:38.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.884+0000 7fea71243700 1 -- 172.21.15.16:0/1928380577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea6c100b10 msgr2=0x7fea6c074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:38.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.884+0000 7fea71243700 1 --2- 172.21.15.16:0/1928380577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea6c100b10 0x7fea6c074b00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fea5c00e8f0 tx=0x7fea5c00ecb0 comp rx=0 tx=0).stop 2024-04-02T04:42:38.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.885+0000 7fea71243700 1 -- 172.21.15.16:0/1928380577 shutdown_connections 2024-04-02T04:42:38.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.885+0000 7fea71243700 1 --2- 172.21.15.16:0/1928380577 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7fea5406d4a0 0x7fea5406f960 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:38.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.885+0000 7fea71243700 1 --2- 172.21.15.16:0/1928380577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea6c100b10 0x7fea6c074b00 secure :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fea5c00e8f0 tx=0x7fea5c00ecb0 comp rx=0 tx=0).stop 2024-04-02T04:42:38.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.885+0000 7fea71243700 1 --2- 172.21.15.16:0/1928380577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea6c101d10 0x7fea6c0731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:38.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.885+0000 7fea71243700 1 -- 172.21.15.16:0/1928380577 >> 172.21.15.16:0/1928380577 conn(0x7fea6c0fc0d0 msgr2=0x7fea6c1052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:42:38.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.885+0000 7fea71243700 1 -- 172.21.15.16:0/1928380577 shutdown_connections 2024-04-02T04:42:38.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:38.885+0000 7fea71243700 1 -- 172.21.15.16:0/1928380577 wait complete. 2024-04-02T04:42:38.891 INFO:tasks.workunit.client.0.smithi016.stdout:8/385: stat d1/l28 0 2024-04-02T04:42:38.892 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 15 2024-04-02T04:42:38.892 INFO:tasks.workunit.client.0.smithi016.stdout:0/358: unlink d0/d16/d2e/d53/f65 0 2024-04-02T04:42:38.893 INFO:tasks.workunit.client.1.smithi027.stdout:2/29: creat d1/f6 x:0 0 0 2024-04-02T04:42:38.893 INFO:tasks.workunit.client.1.smithi027.stdout:2/30: dread - d1/f6 zero size 2024-04-02T04:42:38.894 INFO:tasks.workunit.client.1.smithi027.stdout:2/31: chown d1/f6 28 1 2024-04-02T04:42:38.894 INFO:tasks.workunit.client.1.smithi027.stdout:8/16: symlink l1 0 2024-04-02T04:42:38.896 INFO:tasks.workunit.client.1.smithi027.stdout:6/13: mknod c4 0 2024-04-02T04:42:38.896 INFO:tasks.workunit.client.1.smithi027.stdout:6/14: dread - f3 zero size 2024-04-02T04:42:38.898 INFO:tasks.workunit.client.0.smithi016.stdout:8/386: rmdir d1/d6/d36/d3f/d40/d49 39 2024-04-02T04:42:38.901 INFO:tasks.workunit.client.0.smithi016.stdout:0/359: rename d0/d16/d2e/c6c to d0/d16/d2e/c7f 0 2024-04-02T04:42:38.902 INFO:tasks.workunit.client.0.smithi016.stdout:0/360: truncate d0/d16/d2e/d53/f7c 921475 0 2024-04-02T04:42:38.903 INFO:tasks.workunit.client.1.smithi027.stdout:4/31: fsync f2 0 2024-04-02T04:42:38.904 INFO:tasks.workunit.client.1.smithi027.stdout:2/32: mkdir d1/d7 0 2024-04-02T04:42:38.905 INFO:tasks.workunit.client.1.smithi027.stdout:5/10: dwrite d0/f3 [0,4194304] 0 2024-04-02T04:42:38.906 INFO:tasks.workunit.client.0.smithi016.stdout:0/361: dread d0/d16/d2e/d37/d66/f70 [0,4194304] 0 2024-04-02T04:42:38.907 INFO:tasks.workunit.client.0.smithi016.stdout:4/404: link d5/c34 d5/d17/d18/c7c 0 2024-04-02T04:42:38.910 INFO:tasks.workunit.client.1.smithi027.stdout:6/15: creat f5 x:0 0 0 2024-04-02T04:42:38.912 INFO:tasks.workunit.client.1.smithi027.stdout:9/7: link c3 c4 0 2024-04-02T04:42:38.912 INFO:tasks.workunit.client.1.smithi027.stdout:9/8: write - no filename 2024-04-02T04:42:38.914 INFO:tasks.workunit.client.1.smithi027.stdout:7/9: getdents d2 0 2024-04-02T04:42:38.915 INFO:tasks.workunit.client.0.smithi016.stdout:0/362: creat d0/d16/f80 x:0 0 0 2024-04-02T04:42:38.917 INFO:tasks.workunit.client.0.smithi016.stdout:9/408: truncate d0/d4e/d50/d63/d6e/f3a 4044459 0 2024-04-02T04:42:38.920 INFO:tasks.workunit.client.1.smithi027.stdout:5/11: symlink d0/l4 0 2024-04-02T04:42:38.921 INFO:tasks.workunit.client.0.smithi016.stdout:0/363: symlink d0/d16/d2e/d37/l81 0 2024-04-02T04:42:38.923 INFO:tasks.workunit.client.1.smithi027.stdout:4/32: write f2 [875643,50837] 0 2024-04-02T04:42:38.924 INFO:tasks.workunit.client.0.smithi016.stdout:4/405: creat d5/d17/d76/f7d x:0 0 0 2024-04-02T04:42:38.925 INFO:tasks.workunit.client.1.smithi027.stdout:5/12: dread d0/f3 [0,4194304] 0 2024-04-02T04:42:38.926 INFO:tasks.workunit.client.1.smithi027.stdout:6/16: creat f6 x:0 0 0 2024-04-02T04:42:38.926 INFO:tasks.workunit.client.1.smithi027.stdout:6/17: stat f3 0 2024-04-02T04:42:38.928 INFO:tasks.workunit.client.1.smithi027.stdout:9/9: chown c4 30280 1 2024-04-02T04:42:38.928 INFO:tasks.workunit.client.1.smithi027.stdout:9/10: dread - no filename 2024-04-02T04:42:38.928 INFO:tasks.workunit.client.1.smithi027.stdout:9/11: write - no filename 2024-04-02T04:42:38.928 INFO:tasks.workunit.client.0.smithi016.stdout:8/387: rename d1/d6/d36/d3f/d40/d49/c30 to d1/d6/d36/d3f/c81 0 2024-04-02T04:42:38.928 INFO:tasks.workunit.client.0.smithi016.stdout:8/388: chown d1/f5c 330853 1 2024-04-02T04:42:38.930 INFO:tasks.workunit.client.0.smithi016.stdout:8/389: dread - d1/d6/d36/d3f/f80 zero size 2024-04-02T04:42:38.930 INFO:tasks.workunit.client.0.smithi016.stdout:4/406: chown d5/d17/d18/d25/f57 41353 1 2024-04-02T04:42:38.931 INFO:tasks.workunit.client.1.smithi027.stdout:2/33: symlink d1/d7/l8 0 2024-04-02T04:42:38.934 INFO:tasks.workunit.client.0.smithi016.stdout:9/409: creat d0/d4e/d50/d63/d6e/f77 x:0 0 0 2024-04-02T04:42:38.934 INFO:tasks.workunit.client.1.smithi027.stdout:8/17: link l1 l2 0 2024-04-02T04:42:38.934 INFO:tasks.workunit.client.1.smithi027.stdout:8/18: write - no filename 2024-04-02T04:42:38.936 INFO:tasks.workunit.client.1.smithi027.stdout:4/33: dwrite f2 [0,4194304] 0 2024-04-02T04:42:38.937 INFO:tasks.workunit.client.1.smithi027.stdout:5/13: dread d0/f3 [0,4194304] 0 2024-04-02T04:42:38.945 INFO:tasks.workunit.client.1.smithi027.stdout:7/10: dwrite f1 [0,4194304] 0 2024-04-02T04:42:38.947 INFO:tasks.workunit.client.1.smithi027.stdout:9/12: creat f5 x:0 0 0 2024-04-02T04:42:38.947 INFO:tasks.workunit.client.0.smithi016.stdout:0/364: dread d0/d16/d24/f20 [0,4194304] 0 2024-04-02T04:42:38.949 INFO:tasks.workunit.client.1.smithi027.stdout:9/13: truncate f5 693762 0 2024-04-02T04:42:38.950 INFO:tasks.workunit.client.1.smithi027.stdout:4/34: dwrite f2 [0,4194304] 0 2024-04-02T04:42:38.951 INFO:tasks.workunit.client.1.smithi027.stdout:5/14: dread d0/f3 [0,4194304] 0 2024-04-02T04:42:38.952 INFO:tasks.workunit.client.1.smithi027.stdout:9/14: dread f5 [0,4194304] 0 2024-04-02T04:42:38.953 INFO:tasks.workunit.client.1.smithi027.stdout:8/19: creat f3 x:0 0 0 2024-04-02T04:42:38.953 INFO:tasks.workunit.client.1.smithi027.stdout:8/20: read - f3 zero size 2024-04-02T04:42:38.954 INFO:tasks.workunit.client.0.smithi016.stdout:8/390: write d1/d6/d36/d3f/d40/d49/f43 [306552,112775] 0 2024-04-02T04:42:38.955 INFO:tasks.workunit.client.1.smithi027.stdout:8/21: write f3 [710584,117615] 0 2024-04-02T04:42:38.955 INFO:tasks.workunit.client.0.smithi016.stdout:8/391: dread - d1/f5c zero size 2024-04-02T04:42:38.956 INFO:tasks.workunit.client.1.smithi027.stdout:6/18: write f2 [612162,75791] 0 2024-04-02T04:42:38.960 INFO:tasks.workunit.client.0.smithi016.stdout:4/407: getdents d5/d17 0 2024-04-02T04:42:38.964 INFO:tasks.workunit.client.0.smithi016.stdout:8/392: fsync d1/f15 0 2024-04-02T04:42:38.964 INFO:tasks.workunit.client.1.smithi027.stdout:4/35: chown c3 347 1 2024-04-02T04:42:38.965 INFO:tasks.workunit.client.1.smithi027.stdout:4/36: read f2 [3800296,24343] 0 2024-04-02T04:42:38.965 INFO:tasks.workunit.client.1.smithi027.stdout:5/15: mkdir d0/d5 0 2024-04-02T04:42:38.966 INFO:tasks.workunit.client.0.smithi016.stdout:9/410: dwrite d0/f7 [0,4194304] 0 2024-04-02T04:42:38.966 INFO:tasks.workunit.client.1.smithi027.stdout:5/16: stat d0/f3 0 2024-04-02T04:42:38.966 INFO:tasks.workunit.client.0.smithi016.stdout:4/408: mkdir d5/d17/d76/d7e 0 2024-04-02T04:42:38.966 INFO:tasks.workunit.client.0.smithi016.stdout:9/411: readlink d0/l5e 0 2024-04-02T04:42:38.967 INFO:tasks.workunit.client.1.smithi027.stdout:9/15: symlink l6 0 2024-04-02T04:42:38.969 INFO:tasks.workunit.client.1.smithi027.stdout:8/22: chown l2 9951 1 2024-04-02T04:42:38.969 INFO:tasks.workunit.client.1.smithi027.stdout:8/23: readlink l1 0 2024-04-02T04:42:38.970 INFO:tasks.workunit.client.1.smithi027.stdout:5/17: dread d0/f3 [0,4194304] 0 2024-04-02T04:42:38.973 INFO:tasks.workunit.client.1.smithi027.stdout:6/19: rename f5 to f7 0 2024-04-02T04:42:38.974 INFO:tasks.workunit.client.0.smithi016.stdout:0/365: write d0/d16/d60/d71/f72 [835610,124661] 0 2024-04-02T04:42:38.976 INFO:tasks.workunit.client.0.smithi016.stdout:9/412: chown d0/d33/f38 12214819 1 2024-04-02T04:42:38.978 INFO:tasks.workunit.client.1.smithi027.stdout:2/34: link d1/l4 d1/l9 0 2024-04-02T04:42:38.979 INFO:tasks.workunit.client.1.smithi027.stdout:2/35: chown d1/f6 475913861 1 2024-04-02T04:42:38.980 INFO:tasks.workunit.client.0.smithi016.stdout:8/393: mkdir d1/d70/d71/d74/d82 0 2024-04-02T04:42:38.980 INFO:tasks.workunit.client.1.smithi027.stdout:4/37: dwrite f2 [4194304,4194304] 0 2024-04-02T04:42:38.982 INFO:tasks.workunit.client.0.smithi016.stdout:0/366: mknod d0/d16/d2e/d37/c82 0 2024-04-02T04:42:38.982 INFO:tasks.workunit.client.0.smithi016.stdout:8/394: write d1/d6/d36/d3f/d40/d49/d4a/f56 [18837,118387] 0 2024-04-02T04:42:38.986 INFO:tasks.workunit.client.1.smithi027.stdout:9/16: symlink l7 0 2024-04-02T04:42:38.987 INFO:tasks.workunit.client.0.smithi016.stdout:9/413: dread d0/d33/f36 [0,4194304] 0 2024-04-02T04:42:38.993 INFO:tasks.workunit.client.0.smithi016.stdout:0/367: rename d0/d16/d23/d3e/d57 to d0/d48/d83 0 2024-04-02T04:42:38.994 INFO:tasks.workunit.client.0.smithi016.stdout:5/400: read d3/d5d/f69 [92824,99305] 0 2024-04-02T04:42:38.994 INFO:tasks.workunit.client.0.smithi016.stdout:0/368: chown d0/d16/d24/c46 13404721 1 2024-04-02T04:42:38.994 INFO:tasks.workunit.client.0.smithi016.stdout:1/375: read d14/f1c [3070155,115733] 0 2024-04-02T04:42:38.996 INFO:tasks.workunit.client.1.smithi027.stdout:7/11: truncate f1 1995338 0 2024-04-02T04:42:38.996 INFO:tasks.workunit.client.1.smithi027.stdout:7/12: chown f1 3 1 2024-04-02T04:42:38.999 INFO:tasks.workunit.client.0.smithi016.stdout:8/395: mknod d1/d6/d36/d3f/d40/d63/c83 0 2024-04-02T04:42:39.000 INFO:tasks.workunit.client.1.smithi027.stdout:0/16: fdatasync f2 0 2024-04-02T04:42:39.000 INFO:tasks.workunit.client.1.smithi027.stdout:5/18: creat d0/f6 x:0 0 0 2024-04-02T04:42:39.004 INFO:tasks.workunit.client.1.smithi027.stdout:6/20: unlink f2 0 2024-04-02T04:42:39.004 INFO:tasks.workunit.client.1.smithi027.stdout:6/21: dread - f7 zero size 2024-04-02T04:42:39.005 INFO:tasks.workunit.client.0.smithi016.stdout:0/369: mknod d0/d48/d83/c84 0 2024-04-02T04:42:39.007 INFO:tasks.workunit.client.0.smithi016.stdout:1/376: creat d14/d41/d43/f77 x:0 0 0 2024-04-02T04:42:39.008 INFO:tasks.workunit.client.1.smithi027.stdout:4/38: creat f5 x:0 0 0 2024-04-02T04:42:39.009 INFO:tasks.workunit.client.1.smithi027.stdout:2/36: rename d1/l5 to d1/la 0 2024-04-02T04:42:39.010 INFO:tasks.workunit.client.1.smithi027.stdout:2/37: dread - d1/f6 zero size 2024-04-02T04:42:39.014 INFO:tasks.workunit.client.1.smithi027.stdout:5/19: write d0/f3 [3031433,24656] 0 2024-04-02T04:42:39.015 INFO:tasks.workunit.client.1.smithi027.stdout:0/17: creat d4/f6 x:0 0 0 2024-04-02T04:42:39.015 INFO:tasks.workunit.client.1.smithi027.stdout:5/20: dread - d0/f6 zero size 2024-04-02T04:42:39.018 INFO:tasks.workunit.client.1.smithi027.stdout:6/22: rename c4 to c8 0 2024-04-02T04:42:39.018 INFO:tasks.workunit.client.1.smithi027.stdout:6/23: rmdir - no directory 2024-04-02T04:42:39.018 INFO:tasks.workunit.client.1.smithi027.stdout:4/39: symlink l6 0 2024-04-02T04:42:39.019 INFO:tasks.workunit.client.1.smithi027.stdout:2/38: stat d1/l4 0 2024-04-02T04:42:39.019 INFO:tasks.workunit.client.1.smithi027.stdout:4/40: chown f5 38 1 2024-04-02T04:42:39.021 INFO:tasks.workunit.client.1.smithi027.stdout:2/39: write d1/f6 [92825,81183] 0 2024-04-02T04:42:39.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.018+0000 7f55f8236700 1 -- 172.21.15.16:0/3442921937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f00ff170 msgr2=0x7f55f00ff590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:39.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.018+0000 7f55f8236700 1 --2- 172.21.15.16:0/3442921937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f00ff170 0x7f55f00ff590 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f55ec009910 tx=0x7f55ec009c20 comp rx=0 tx=0).stop 2024-04-02T04:42:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.018+0000 7f55f8236700 1 -- 172.21.15.16:0/3442921937 shutdown_connections 2024-04-02T04:42:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.018+0000 7f55f8236700 1 --2- 172.21.15.16:0/3442921937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55f0100370 0x7f55f01007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.018+0000 7f55f8236700 1 --2- 172.21.15.16:0/3442921937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f00ff170 0x7f55f00ff590 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.018+0000 7f55f8236700 1 -- 172.21.15.16:0/3442921937 >> 172.21.15.16:0/3442921937 conn(0x7f55f00fa730 msgr2=0x7f55f00fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:42:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.019+0000 7f55f8236700 1 -- 172.21.15.16:0/3442921937 shutdown_connections 2024-04-02T04:42:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.019+0000 7f55f8236700 1 -- 172.21.15.16:0/3442921937 wait complete. 2024-04-02T04:42:39.023 INFO:tasks.workunit.client.1.smithi027.stdout:6/24: creat f9 x:0 0 0 2024-04-02T04:42:39.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.019+0000 7f55f8236700 1 Processor -- start 2024-04-02T04:42:39.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.019+0000 7f55f8236700 1 -- start start 2024-04-02T04:42:39.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.019+0000 7f55f8236700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f00ff170 0x7f55f0196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:39.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.019+0000 7f55f8236700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55f0100370 0x7f55f0196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:39.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.019+0000 7f55f8236700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f0197060 con 0x7f55f00ff170 2024-04-02T04:42:39.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.019+0000 7f55f8236700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f01971a0 con 0x7f55f0100370 2024-04-02T04:42:39.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.020+0000 7f55f5fd2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f00ff170 0x7f55f0196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:39.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.020+0000 7f55f5fd2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f00ff170 0x7f55f0196500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51984/0 (socket says 172.21.15.16:51984) 2024-04-02T04:42:39.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.020+0000 7f55f5fd2700 1 -- 172.21.15.16:0/499111552 learned_addr learned my addr 172.21.15.16:0/499111552 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:42:39.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.020+0000 7f55f57d1700 1 --2- 172.21.15.16:0/499111552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55f0100370 0x7f55f0196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:39.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.020+0000 7f55f5fd2700 1 -- 172.21.15.16:0/499111552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55f0100370 msgr2=0x7f55f0196a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:39.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.020+0000 7f55f5fd2700 1 --2- 172.21.15.16:0/499111552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55f0100370 0x7f55f0196a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:39.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.020+0000 7f55f5fd2700 1 -- 172.21.15.16:0/499111552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55ec0095f0 con 0x7f55f00ff170 2024-04-02T04:42:39.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.020+0000 7f55f57d1700 1 --2- 172.21.15.16:0/499111552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55f0100370 0x7f55f0196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T04:42:39.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.020+0000 7f55f5fd2700 1 --2- 172.21.15.16:0/499111552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f00ff170 0x7f55f0196500 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f55ec0059d0 tx=0x7f55ec00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:42:39.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.020+0000 7f55e6ffd700 1 -- 172.21.15.16:0/499111552 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55ec01d070 con 0x7f55f00ff170 2024-04-02T04:42:39.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.020+0000 7f55f8236700 1 -- 172.21.15.16:0/499111552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f019bba0 con 0x7f55f00ff170 2024-04-02T04:42:39.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.020+0000 7f55e6ffd700 1 -- 172.21.15.16:0/499111552 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f55ec00f460 con 0x7f55f00ff170 2024-04-02T04:42:39.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.020+0000 7f55e6ffd700 1 -- 172.21.15.16:0/499111552 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55ec021640 con 0x7f55f00ff170 2024-04-02T04:42:39.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.021+0000 7f55f8236700 1 -- 172.21.15.16:0/499111552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f019c090 con 0x7f55f00ff170 2024-04-02T04:42:39.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.021+0000 7f55f8236700 1 -- 172.21.15.16:0/499111552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f004ea80 con 0x7f55f00ff170 2024-04-02T04:42:39.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.022+0000 7f55e6ffd700 1 -- 172.21.15.16:0/499111552 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 91380+0+0 (secure 0 0 0) 0x7f55ec00f5d0 con 0x7f55f00ff170 2024-04-02T04:42:39.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.022+0000 7f55e6ffd700 1 --2- 172.21.15.16:0/499111552 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7f55dc06d4a0 0x7f55dc06f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:39.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.022+0000 7f55e6ffd700 1 -- 172.21.15.16:0/499111552 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f55ec08e1a0 con 0x7f55f00ff170 2024-04-02T04:42:39.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.023+0000 7f55f57d1700 1 --2- 172.21.15.16:0/499111552 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7f55dc06d4a0 0x7f55dc06f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:39.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.023+0000 7f55f57d1700 1 --2- 172.21.15.16:0/499111552 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7f55dc06d4a0 0x7f55dc06f960 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f55e0005d90 tx=0x7f55e0005d20 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:42:39.029 INFO:tasks.workunit.client.1.smithi027.stdout:0/18: dwrite f1 [0,4194304] 0 2024-04-02T04:42:39.029 INFO:tasks.workunit.client.1.smithi027.stdout:5/21: link d0/f6 d0/d5/f7 0 2024-04-02T04:42:39.029 INFO:tasks.workunit.client.1.smithi027.stdout:6/25: creat fa x:0 0 0 2024-04-02T04:42:39.029 INFO:tasks.workunit.client.1.smithi027.stdout:6/26: readlink - no filename 2024-04-02T04:42:39.029 INFO:tasks.workunit.client.1.smithi027.stdout:8/24: dread f3 [0,4194304] 0 2024-04-02T04:42:39.030 INFO:tasks.workunit.client.1.smithi027.stdout:6/27: chown f3 8 1 2024-04-02T04:42:39.030 INFO:tasks.workunit.client.1.smithi027.stdout:6/28: dread - f3 zero size 2024-04-02T04:42:39.030 INFO:tasks.workunit.client.1.smithi027.stdout:8/25: unlink l2 0 2024-04-02T04:42:39.030 INFO:tasks.workunit.client.1.smithi027.stdout:8/26: truncate f3 898154 0 2024-04-02T04:42:39.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.028+0000 7f55e6ffd700 1 -- 172.21.15.16:0/499111552 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184299 (secure 0 0 0) 0x7f55ec05b610 con 0x7f55f00ff170 2024-04-02T04:42:39.031 INFO:tasks.workunit.client.1.smithi027.stdout:6/29: mknod cb 0 2024-04-02T04:42:39.032 INFO:tasks.workunit.client.1.smithi027.stdout:2/40: dwrite d1/f6 [0,4194304] 0 2024-04-02T04:42:39.033 INFO:tasks.workunit.client.1.smithi027.stdout:6/30: truncate fa 562128 0 2024-04-02T04:42:39.033 INFO:tasks.workunit.client.1.smithi027.stdout:8/27: truncate f3 1826299 0 2024-04-02T04:42:39.033 INFO:tasks.workunit.client.1.smithi027.stdout:6/31: stat f3 0 2024-04-02T04:42:39.035 INFO:tasks.workunit.client.1.smithi027.stdout:2/41: mkdir d1/db 0 2024-04-02T04:42:39.037 INFO:tasks.workunit.client.0.smithi016.stdout:3/441: sync 2024-04-02T04:42:39.037 INFO:tasks.workunit.client.0.smithi016.stdout:6/449: sync 2024-04-02T04:42:39.038 INFO:tasks.workunit.client.1.smithi027.stdout:2/42: read d1/f6 [20391,99201] 0 2024-04-02T04:42:39.039 INFO:tasks.workunit.client.0.smithi016.stdout:3/442: stat d2/d15/d16/d3d/d6e/d6f/d78 0 2024-04-02T04:42:39.041 INFO:tasks.workunit.client.0.smithi016.stdout:3/443: write d2/f9 [142297,44784] 0 2024-04-02T04:42:39.043 INFO:tasks.workunit.client.0.smithi016.stdout:4/409: getdents d5/de 0 2024-04-02T04:42:39.044 INFO:tasks.workunit.client.1.smithi027.stdout:2/43: link d1/f6 d1/d7/fc 0 2024-04-02T04:42:39.044 INFO:tasks.workunit.client.1.smithi027.stdout:6/32: dwrite f6 [0,4194304] 0 2024-04-02T04:42:39.044 INFO:tasks.workunit.client.1.smithi027.stdout:6/33: dread - f3 zero size 2024-04-02T04:42:39.045 INFO:tasks.workunit.client.0.smithi016.stdout:5/401: mkdir d3/d40/d4f/d9c 0 2024-04-02T04:42:39.046 INFO:tasks.workunit.client.1.smithi027.stdout:2/44: mkdir d1/dd 0 2024-04-02T04:42:39.046 INFO:tasks.workunit.client.0.smithi016.stdout:0/370: creat d0/d16/d2e/f85 x:0 0 0 2024-04-02T04:42:39.047 INFO:tasks.workunit.client.1.smithi027.stdout:6/34: creat fc x:0 0 0 2024-04-02T04:42:39.048 INFO:tasks.workunit.client.0.smithi016.stdout:3/444: symlink d2/d15/d1a/d45/l86 0 2024-04-02T04:42:39.049 INFO:tasks.workunit.client.1.smithi027.stdout:2/45: write d1/f6 [124339,98778] 0 2024-04-02T04:42:39.051 INFO:tasks.workunit.client.1.smithi027.stdout:6/35: rename f6 to fd 0 2024-04-02T04:42:39.054 INFO:tasks.workunit.client.1.smithi027.stdout:7/13: dread f1 [0,4194304] 0 2024-04-02T04:42:39.055 INFO:tasks.workunit.client.1.smithi027.stdout:2/46: dread d1/d7/fc [0,4194304] 0 2024-04-02T04:42:39.057 INFO:tasks.workunit.client.1.smithi027.stdout:7/14: dread f1 [0,4194304] 0 2024-04-02T04:42:39.060 INFO:tasks.workunit.client.1.smithi027.stdout:6/36: symlink le 0 2024-04-02T04:42:39.062 INFO:tasks.workunit.client.1.smithi027.stdout:2/47: rename d1/l3 to d1/d7/le 0 2024-04-02T04:42:39.063 INFO:tasks.workunit.client.1.smithi027.stdout:9/17: truncate f5 346061 0 2024-04-02T04:42:39.065 INFO:tasks.workunit.client.1.smithi027.stdout:6/37: creat ff x:0 0 0 2024-04-02T04:42:39.066 INFO:tasks.workunit.client.1.smithi027.stdout:6/38: truncate f3 402598 0 2024-04-02T04:42:39.066 INFO:tasks.workunit.client.1.smithi027.stdout:2/48: mkdir d1/df 0 2024-04-02T04:42:39.068 INFO:tasks.workunit.client.1.smithi027.stdout:7/15: symlink d2/l3 0 2024-04-02T04:42:39.072 INFO:tasks.workunit.client.1.smithi027.stdout:7/16: creat d2/f4 x:0 0 0 2024-04-02T04:42:39.074 INFO:tasks.workunit.client.1.smithi027.stdout:6/39: dwrite f9 [0,4194304] 0 2024-04-02T04:42:39.079 INFO:tasks.workunit.client.1.smithi027.stdout:5/22: getdents d0/d5 0 2024-04-02T04:42:39.080 INFO:tasks.workunit.client.0.smithi016.stdout:2/328: sync 2024-04-02T04:42:39.080 INFO:tasks.workunit.client.0.smithi016.stdout:7/390: sync 2024-04-02T04:42:39.082 INFO:tasks.workunit.client.1.smithi027.stdout:8/28: getdents . 0 2024-04-02T04:42:39.082 INFO:tasks.workunit.client.1.smithi027.stdout:5/23: rename d0 to d0/d5/d8 22 2024-04-02T04:42:39.087 INFO:tasks.workunit.client.1.smithi027.stdout:0/19: dwrite f1 [4194304,4194304] 0 2024-04-02T04:42:39.087 INFO:tasks.workunit.client.1.smithi027.stdout:8/29: rename f3 to f4 0 2024-04-02T04:42:39.089 INFO:tasks.workunit.client.1.smithi027.stdout:5/24: mknod d0/c9 0 2024-04-02T04:42:39.090 INFO:tasks.workunit.client.1.smithi027.stdout:8/30: read f4 [1144661,60091] 0 2024-04-02T04:42:39.090 INFO:tasks.workunit.client.1.smithi027.stdout:5/25: readlink d0/l1 0 2024-04-02T04:42:39.090 INFO:tasks.workunit.client.1.smithi027.stdout:6/40: dwrite f3 [0,4194304] 0 2024-04-02T04:42:39.092 INFO:tasks.workunit.client.1.smithi027.stdout:8/31: read f4 [336229,60082] 0 2024-04-02T04:42:39.096 INFO:tasks.workunit.client.1.smithi027.stdout:6/41: mknod c10 0 2024-04-02T04:42:39.096 INFO:tasks.workunit.client.0.smithi016.stdout:1/377: dwrite d14/d19/d1d/d29/f3c [0,4194304] 0 2024-04-02T04:42:39.099 INFO:tasks.workunit.client.0.smithi016.stdout:3/445: readlink d2/d2e/d36/d66/l74 0 2024-04-02T04:42:39.099 INFO:tasks.workunit.client.0.smithi016.stdout:8/396: link d1/d6/d36/d3f/d40/d49/d38/f76 d1/d70/f84 0 2024-04-02T04:42:39.100 INFO:tasks.workunit.client.1.smithi027.stdout:6/42: creat f11 x:0 0 0 2024-04-02T04:42:39.101 INFO:tasks.workunit.client.1.smithi027.stdout:6/43: readlink le 0 2024-04-02T04:42:39.103 INFO:tasks.workunit.client.1.smithi027.stdout:8/32: dread f4 [0,4194304] 0 2024-04-02T04:42:39.103 INFO:tasks.workunit.client.0.smithi016.stdout:3/446: dread d2/d15/d16/f25 [0,4194304] 0 2024-04-02T04:42:39.103 INFO:tasks.workunit.client.1.smithi027.stdout:0/20: dwrite d4/f6 [0,4194304] 0 2024-04-02T04:42:39.104 INFO:tasks.workunit.client.1.smithi027.stdout:0/21: chown d4 342 1 2024-04-02T04:42:39.106 INFO:tasks.workunit.client.0.smithi016.stdout:6/450: dwrite d0/d9/df/f4d [0,4194304] 0 2024-04-02T04:42:39.107 INFO:tasks.workunit.client.1.smithi027.stdout:8/33: dread f4 [0,4194304] 0 2024-04-02T04:42:39.108 INFO:tasks.workunit.client.0.smithi016.stdout:2/329: readlink d12/l30 0 2024-04-02T04:42:39.110 INFO:tasks.workunit.client.0.smithi016.stdout:0/371: dwrite d0/d16/d23/d3e/f43 [0,4194304] 0 2024-04-02T04:42:39.113 INFO:tasks.workunit.client.0.smithi016.stdout:1/378: dwrite d14/d19/d1d/d29/f51 [0,4194304] 0 2024-04-02T04:42:39.114 INFO:tasks.workunit.client.1.smithi027.stdout:7/17: rmdir d2 39 2024-04-02T04:42:39.119 INFO:tasks.workunit.client.0.smithi016.stdout:7/391: mknod d0/d46/c90 0 2024-04-02T04:42:39.121 INFO:tasks.workunit.client.1.smithi027.stdout:0/22: mknod d4/c7 0 2024-04-02T04:42:39.123 INFO:tasks.workunit.client.1.smithi027.stdout:8/34: mkdir d5 0 2024-04-02T04:42:39.125 INFO:tasks.workunit.client.1.smithi027.stdout:7/18: fsync f1 0 2024-04-02T04:42:39.126 INFO:tasks.workunit.client.0.smithi016.stdout:3/447: rmdir d2/d15/d16/d57/d76 39 2024-04-02T04:42:39.127 INFO:tasks.workunit.client.1.smithi027.stdout:5/26: truncate d0/f3 2821181 0 2024-04-02T04:42:39.128 INFO:tasks.workunit.client.0.smithi016.stdout:7/392: dwrite d0/d28/d64/f8b [0,4194304] 0 2024-04-02T04:42:39.131 INFO:tasks.workunit.client.1.smithi027.stdout:0/23: mknod d4/c8 0 2024-04-02T04:42:39.132 INFO:tasks.workunit.client.0.smithi016.stdout:7/393: dread - d0/d46/d37/d3a/d43/f5e zero size 2024-04-02T04:42:39.133 INFO:tasks.workunit.client.0.smithi016.stdout:6/451: dwrite d0/f84 [0,4194304] 0 2024-04-02T04:42:39.134 INFO:tasks.workunit.client.1.smithi027.stdout:9/18: dwrite f5 [0,4194304] 0 2024-04-02T04:42:39.138 INFO:tasks.workunit.client.1.smithi027.stdout:7/19: chown d2/f4 494 1 2024-04-02T04:42:39.139 INFO:tasks.workunit.client.0.smithi016.stdout:4/410: creat d5/d17/d18/f7f x:0 0 0 2024-04-02T04:42:39.139 INFO:tasks.workunit.client.1.smithi027.stdout:0/24: dread f1 [4194304,4194304] 0 2024-04-02T04:42:39.140 INFO:tasks.workunit.client.0.smithi016.stdout:2/330: dwrite d12/d26/f5a [0,4194304] 0 2024-04-02T04:42:39.144 INFO:tasks.workunit.client.0.smithi016.stdout:3/448: symlink d2/d15/d16/d3d/d46/l87 0 2024-04-02T04:42:39.145 INFO:tasks.workunit.client.0.smithi016.stdout:3/449: chown d2/d15/d1a/f1e 5 1 2024-04-02T04:42:39.152 INFO:tasks.workunit.client.1.smithi027.stdout:9/19: symlink l8 0 2024-04-02T04:42:39.153 INFO:tasks.workunit.client.1.smithi027.stdout:7/20: symlink d2/l5 0 2024-04-02T04:42:39.153 INFO:tasks.workunit.client.1.smithi027.stdout:7/21: fdatasync d2/f4 0 2024-04-02T04:42:39.153 INFO:tasks.workunit.client.1.smithi027.stdout:7/22: stat f1 0 2024-04-02T04:42:39.154 INFO:tasks.workunit.client.1.smithi027.stdout:7/23: stat f1 0 2024-04-02T04:42:39.156 INFO:tasks.workunit.client.0.smithi016.stdout:7/394: unlink d0/d46/d16/f41 0 2024-04-02T04:42:39.157 INFO:tasks.workunit.client.0.smithi016.stdout:7/395: chown d0/d28/c6d 2977 1 2024-04-02T04:42:39.160 INFO:tasks.workunit.client.1.smithi027.stdout:6/44: getdents . 0 2024-04-02T04:42:39.163 INFO:tasks.workunit.client.1.smithi027.stdout:0/25: unlink d4/c7 0 2024-04-02T04:42:39.164 INFO:tasks.workunit.client.1.smithi027.stdout:0/26: readlink l3 0 2024-04-02T04:42:39.166 INFO:tasks.workunit.client.1.smithi027.stdout:8/35: truncate f4 1186707 0 2024-04-02T04:42:39.167 INFO:tasks.workunit.client.0.smithi016.stdout:4/411: mknod d5/d16/c80 0 2024-04-02T04:42:39.169 INFO:tasks.workunit.client.0.smithi016.stdout:2/331: mkdir d12/d35/d3b/d73 0 2024-04-02T04:42:39.170 INFO:tasks.workunit.client.1.smithi027.stdout:7/24: mknod d2/c6 0 2024-04-02T04:42:39.172 INFO:tasks.workunit.client.0.smithi016.stdout:3/450: mkdir d2/d15/d1a/d45/d56/d88 0 2024-04-02T04:42:39.173 INFO:tasks.workunit.client.1.smithi027.stdout:5/27: write d0/f3 [2077717,44242] 0 2024-04-02T04:42:39.173 INFO:tasks.workunit.client.0.smithi016.stdout:0/372: write d0/d16/d2e/d53/d6f/f74 [1027092,32009] 0 2024-04-02T04:42:39.173 INFO:tasks.workunit.client.0.smithi016.stdout:5/402: getdents d3/df/d86 0 2024-04-02T04:42:39.175 INFO:tasks.workunit.client.0.smithi016.stdout:3/451: write d2/d15/d1a/f39 [3296533,120319] 0 2024-04-02T04:42:39.176 INFO:tasks.workunit.client.0.smithi016.stdout:1/379: truncate d14/f28 1100752 0 2024-04-02T04:42:39.178 INFO:tasks.workunit.client.1.smithi027.stdout:0/27: truncate f1 6147255 0 2024-04-02T04:42:39.179 INFO:tasks.workunit.client.0.smithi016.stdout:6/452: mknod d0/c87 0 2024-04-02T04:42:39.179 INFO:tasks.workunit.client.0.smithi016.stdout:2/332: dwrite d12/d35/d3f/d56/d66/f6c [0,4194304] 0 2024-04-02T04:42:39.182 INFO:tasks.workunit.client.0.smithi016.stdout:2/333: chown d12/d35/d3b/d42/d67 2515487 1 2024-04-02T04:42:39.183 INFO:tasks.workunit.client.1.smithi027.stdout:0/28: dread d4/f6 [0,4194304] 0 2024-04-02T04:42:39.184 INFO:tasks.workunit.client.0.smithi016.stdout:7/396: stat d0/d28/d64/l69 0 2024-04-02T04:42:39.185 INFO:tasks.workunit.client.0.smithi016.stdout:2/334: chown d12/l30 986897 1 2024-04-02T04:42:39.185 INFO:tasks.workunit.client.1.smithi027.stdout:7/25: unlink d2/f4 0 2024-04-02T04:42:39.186 INFO:tasks.workunit.client.1.smithi027.stdout:5/28: mknod d0/d5/ca 0 2024-04-02T04:42:39.189 INFO:tasks.workunit.client.1.smithi027.stdout:8/36: symlink d5/l6 0 2024-04-02T04:42:39.189 INFO:tasks.workunit.client.0.smithi016.stdout:0/373: dwrite d0/d16/d60/f75 [0,4194304] 0 2024-04-02T04:42:39.191 INFO:tasks.workunit.client.1.smithi027.stdout:9/20: link c2 c9 0 2024-04-02T04:42:39.195 INFO:tasks.workunit.client.0.smithi016.stdout:7/397: dwrite d0/d46/d16/d49/f3e [0,4194304] 0 2024-04-02T04:42:39.199 INFO:tasks.workunit.client.1.smithi027.stdout:9/21: dwrite f5 [0,4194304] 0 2024-04-02T04:42:39.202 INFO:tasks.workunit.client.1.smithi027.stdout:7/26: mkdir d2/d7 0 2024-04-02T04:42:39.203 INFO:tasks.workunit.client.0.smithi016.stdout:3/452: dread d2/d15/d16/d57/f64 [0,4194304] 0 2024-04-02T04:42:39.204 INFO:tasks.workunit.client.1.smithi027.stdout:7/27: write f1 [816638,15042] 0 2024-04-02T04:42:39.206 INFO:tasks.workunit.client.1.smithi027.stdout:5/29: unlink d0/c2 0 2024-04-02T04:42:39.209 INFO:tasks.workunit.client.0.smithi016.stdout:5/403: mknod d3/df/c9d 0 2024-04-02T04:42:39.209 INFO:tasks.workunit.client.0.smithi016.stdout:5/404: dread - d3/d40/d4f/d6b/f74 zero size 2024-04-02T04:42:39.211 INFO:tasks.workunit.client.1.smithi027.stdout:8/37: mkdir d5/d7 0 2024-04-02T04:42:39.215 INFO:tasks.workunit.client.1.smithi027.stdout:9/22: dwrite f5 [0,4194304] 0 2024-04-02T04:42:39.218 INFO:tasks.workunit.client.1.smithi027.stdout:7/28: dwrite f1 [0,4194304] 0 2024-04-02T04:42:39.220 INFO:tasks.workunit.client.1.smithi027.stdout:7/29: write f1 [2390448,86637] 0 2024-04-02T04:42:39.221 INFO:tasks.workunit.client.1.smithi027.stdout:7/30: chown d2/c6 286438 1 2024-04-02T04:42:39.223 INFO:tasks.workunit.client.1.smithi027.stdout:9/23: dread f5 [0,4194304] 0 2024-04-02T04:42:39.228 INFO:tasks.workunit.client.1.smithi027.stdout:5/30: mkdir d0/d5/db 0 2024-04-02T04:42:39.231 INFO:tasks.workunit.client.1.smithi027.stdout:5/31: dread d0/f3 [0,4194304] 0 2024-04-02T04:42:39.232 INFO:tasks.workunit.client.1.smithi027.stdout:7/31: symlink d2/l8 0 2024-04-02T04:42:39.234 INFO:tasks.workunit.client.1.smithi027.stdout:9/24: creat fa x:0 0 0 2024-04-02T04:42:39.240 INFO:tasks.workunit.client.1.smithi027.stdout:7/32: dwrite f1 [0,4194304] 0 2024-04-02T04:42:39.240 INFO:tasks.workunit.client.1.smithi027.stdout:5/32: rename d0/d5/db to d0/dc 0 2024-04-02T04:42:39.241 INFO:tasks.workunit.client.1.smithi027.stdout:0/29: dread f1 [0,4194304] 0 2024-04-02T04:42:39.244 INFO:tasks.workunit.client.1.smithi027.stdout:9/25: chown c2 2035439786 1 2024-04-02T04:42:39.248 INFO:tasks.workunit.client.1.smithi027.stdout:7/33: unlink d2/l8 0 2024-04-02T04:42:39.250 INFO:tasks.workunit.client.1.smithi027.stdout:7/34: write f1 [1680289,66450] 0 2024-04-02T04:42:39.250 INFO:tasks.workunit.client.1.smithi027.stdout:5/33: mknod d0/dc/cd 0 2024-04-02T04:42:39.250 INFO:tasks.workunit.client.1.smithi027.stdout:7/35: stat d2 0 2024-04-02T04:42:39.251 INFO:tasks.workunit.client.1.smithi027.stdout:0/30: creat d4/f9 x:0 0 0 2024-04-02T04:42:39.254 INFO:tasks.workunit.client.1.smithi027.stdout:7/36: dread f1 [0,4194304] 0 2024-04-02T04:42:39.256 INFO:tasks.workunit.client.1.smithi027.stdout:5/34: fsync d0/f6 0 2024-04-02T04:42:39.257 INFO:tasks.workunit.client.1.smithi027.stdout:0/31: creat d4/fa x:0 0 0 2024-04-02T04:42:39.262 INFO:tasks.workunit.client.1.smithi027.stdout:5/35: unlink d0/f3 0 2024-04-02T04:42:39.264 INFO:tasks.workunit.client.1.smithi027.stdout:0/32: unlink l3 0 2024-04-02T04:42:39.265 INFO:tasks.workunit.client.1.smithi027.stdout:9/26: link c3 cb 0 2024-04-02T04:42:39.266 INFO:tasks.workunit.client.0.smithi016.stdout:0/374: symlink d0/d16/d2e/l86 0 2024-04-02T04:42:39.267 INFO:tasks.workunit.client.0.smithi016.stdout:2/335: symlink d12/d35/d3b/d42/d59/l74 0 2024-04-02T04:42:39.268 INFO:tasks.workunit.client.0.smithi016.stdout:1/380: truncate fa 949993 0 2024-04-02T04:42:39.269 INFO:tasks.workunit.client.0.smithi016.stdout:2/336: chown d12/d35/d3f/f4b 2554 1 2024-04-02T04:42:39.269 INFO:tasks.workunit.client.1.smithi027.stdout:5/36: rename d0/d5/f7 to d0/dc/fe 0 2024-04-02T04:42:39.270 INFO:tasks.workunit.client.1.smithi027.stdout:0/33: dwrite f2 [4194304,4194304] 0 2024-04-02T04:42:39.271 INFO:tasks.workunit.client.1.smithi027.stdout:9/27: mknod cc 0 2024-04-02T04:42:39.272 INFO:tasks.workunit.client.0.smithi016.stdout:7/398: symlink d0/d46/d37/d3a/d84/d8a/l91 0 2024-04-02T04:42:39.274 INFO:tasks.workunit.client.1.smithi027.stdout:5/37: mkdir d0/df 0 2024-04-02T04:42:39.274 INFO:tasks.workunit.client.0.smithi016.stdout:0/375: dwrite d0/d16/d23/d3e/f43 [0,4194304] 0 2024-04-02T04:42:39.277 INFO:tasks.workunit.client.0.smithi016.stdout:0/376: truncate d0/d48/d83/f5b 885185 0 2024-04-02T04:42:39.279 INFO:tasks.workunit.client.0.smithi016.stdout:0/377: write d0/d16/d24/f2a [3920593,127001] 0 2024-04-02T04:42:39.281 INFO:tasks.workunit.client.0.smithi016.stdout:3/453: dread d2/f26 [0,4194304] 0 2024-04-02T04:42:39.284 INFO:tasks.workunit.client.0.smithi016.stdout:2/337: fsync d12/f43 0 2024-04-02T04:42:39.285 INFO:tasks.workunit.client.1.smithi027.stdout:8/38: dwrite f4 [0,4194304] 0 2024-04-02T04:42:39.285 INFO:tasks.workunit.client.0.smithi016.stdout:2/338: chown d12/d35/d3f/f61 42755 1 2024-04-02T04:42:39.290 INFO:tasks.workunit.client.0.smithi016.stdout:0/378: mkdir d0/d16/d2e/d37/d87 0 2024-04-02T04:42:39.291 INFO:tasks.workunit.client.1.smithi027.stdout:9/28: dwrite f5 [4194304,4194304] 0 2024-04-02T04:42:39.293 INFO:tasks.workunit.client.1.smithi027.stdout:7/37: truncate f1 1931546 0 2024-04-02T04:42:39.297 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:39 smithi027 ceph-mon[37550]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:42:39.297 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:39 smithi027 ceph-mon[37550]: pgmap v18: 65 pgs: 65 active+clean; 656 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 6.1 MiB/s rd, 66 MiB/s wr, 267 op/s 2024-04-02T04:42:39.297 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:39 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:42:39.297 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:39 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/2982319367' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:42:39.297 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:39 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/1928380577' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:42:39.298 INFO:tasks.workunit.client.1.smithi027.stdout:8/39: mkdir d5/d8 0 2024-04-02T04:42:39.298 INFO:tasks.workunit.client.1.smithi027.stdout:5/38: fsync d0/dc/fe 0 2024-04-02T04:42:39.302 INFO:tasks.workunit.client.1.smithi027.stdout:9/29: symlink ld 0 2024-04-02T04:42:39.305 INFO:tasks.workunit.client.1.smithi027.stdout:7/38: rmdir d2 39 2024-04-02T04:42:39.306 INFO:tasks.workunit.client.0.smithi016.stdout:7/399: dread d0/d46/f1a [0,4194304] 0 2024-04-02T04:42:39.307 INFO:tasks.workunit.client.1.smithi027.stdout:0/34: dwrite d4/f6 [0,4194304] 0 2024-04-02T04:42:39.308 INFO:tasks.workunit.client.0.smithi016.stdout:7/400: write d0/d28/f53 [962018,72486] 0 2024-04-02T04:42:39.308 INFO:tasks.workunit.client.1.smithi027.stdout:0/35: chown d4/c5 771232 1 2024-04-02T04:42:39.309 INFO:tasks.workunit.client.1.smithi027.stdout:0/36: chown d4/c5 1880029 1 2024-04-02T04:42:39.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:39 smithi016 ceph-mon[28969]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:42:39.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:39 smithi016 ceph-mon[28969]: pgmap v18: 65 pgs: 65 active+clean; 656 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 6.1 MiB/s rd, 66 MiB/s wr, 267 op/s 2024-04-02T04:42:39.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:39 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:42:39.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:39 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/2982319367' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:42:39.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:39 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/1928380577' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:42:39.310 INFO:tasks.workunit.client.1.smithi027.stdout:8/40: mknod d5/c9 0 2024-04-02T04:42:39.312 INFO:tasks.workunit.client.0.smithi016.stdout:9/414: sync 2024-04-02T04:42:39.313 INFO:tasks.workunit.client.1.smithi027.stdout:5/39: write d0/dc/fe [600716,5112] 0 2024-04-02T04:42:39.316 INFO:tasks.workunit.client.1.smithi027.stdout:9/30: chown c3 123194315 1 2024-04-02T04:42:39.316 INFO:tasks.workunit.client.1.smithi027.stdout:9/31: stat ld 0 2024-04-02T04:42:39.317 INFO:tasks.workunit.client.0.smithi016.stdout:1/381: creat d14/d19/d1e/d71/f78 x:0 0 0 2024-04-02T04:42:39.321 INFO:tasks.workunit.client.0.smithi016.stdout:3/454: dwrite d2/d15/d1a/f1e [4194304,4194304] 0 2024-04-02T04:42:39.321 INFO:tasks.workunit.client.0.smithi016.stdout:0/379: creat d0/d16/d2e/d53/d6f/f88 x:0 0 0 2024-04-02T04:42:39.321 INFO:tasks.workunit.client.1.smithi027.stdout:5/40: fdatasync d0/dc/fe 0 2024-04-02T04:42:39.326 INFO:tasks.workunit.client.0.smithi016.stdout:7/401: creat d0/d28/d64/f92 x:0 0 0 2024-04-02T04:42:39.326 INFO:tasks.workunit.client.1.smithi027.stdout:9/32: dwrite fa [0,4194304] 0 2024-04-02T04:42:39.326 INFO:tasks.workunit.client.0.smithi016.stdout:7/402: readlink d0/d46/l5f 0 2024-04-02T04:42:39.329 INFO:tasks.workunit.client.0.smithi016.stdout:7/403: fsync d0/d46/d16/d49/f3e 0 2024-04-02T04:42:39.330 INFO:tasks.workunit.client.1.smithi027.stdout:9/33: chown l6 25 1 2024-04-02T04:42:39.330 INFO:tasks.workunit.client.1.smithi027.stdout:5/41: mknod d0/c10 0 2024-04-02T04:42:39.330 INFO:tasks.workunit.client.1.smithi027.stdout:0/37: dwrite f1 [0,4194304] 0 2024-04-02T04:42:39.331 INFO:tasks.workunit.client.0.smithi016.stdout:9/415: mkdir d0/d9/d4d/d78 0 2024-04-02T04:42:39.331 INFO:tasks.workunit.client.0.smithi016.stdout:2/339: creat d12/d70/f75 x:0 0 0 2024-04-02T04:42:39.336 INFO:tasks.workunit.client.1.smithi027.stdout:9/34: symlink le 0 2024-04-02T04:42:39.337 INFO:tasks.workunit.client.0.smithi016.stdout:3/455: symlink d2/d15/d1a/d45/d56/l89 0 2024-04-02T04:42:39.337 INFO:tasks.workunit.client.1.smithi027.stdout:9/35: write fa [2166914,103036] 0 2024-04-02T04:42:39.337 INFO:tasks.workunit.client.0.smithi016.stdout:2/340: dread d12/d35/d3f/d56/d66/f6c [0,4194304] 0 2024-04-02T04:42:39.338 INFO:tasks.workunit.client.0.smithi016.stdout:3/456: chown d2/d15/d1a 11626 1 2024-04-02T04:42:39.338 INFO:tasks.workunit.client.1.smithi027.stdout:0/38: creat d4/fb x:0 0 0 2024-04-02T04:42:39.339 INFO:tasks.workunit.client.1.smithi027.stdout:0/39: fsync d4/f9 0 2024-04-02T04:42:39.339 INFO:tasks.workunit.client.1.smithi027.stdout:5/42: mkdir d0/df/d11 0 2024-04-02T04:42:39.341 INFO:tasks.workunit.client.1.smithi027.stdout:5/43: write d0/f6 [1488011,52412] 0 2024-04-02T04:42:39.341 INFO:tasks.workunit.client.0.smithi016.stdout:1/382: symlink d14/d19/d1d/l79 0 2024-04-02T04:42:39.342 INFO:tasks.workunit.client.1.smithi027.stdout:5/44: write d0/dc/fe [2323189,109308] 0 2024-04-02T04:42:39.343 INFO:tasks.workunit.client.1.smithi027.stdout:5/45: write d0/f6 [525033,35916] 0 2024-04-02T04:42:39.345 INFO:tasks.workunit.client.1.smithi027.stdout:0/40: mkdir d4/dc 0 2024-04-02T04:42:39.346 INFO:tasks.workunit.client.1.smithi027.stdout:0/41: chown d4/fa 3746 1 2024-04-02T04:42:39.346 INFO:tasks.workunit.client.0.smithi016.stdout:3/457: dwrite d2/fd [0,4194304] 0 2024-04-02T04:42:39.349 INFO:tasks.workunit.client.0.smithi016.stdout:7/404: creat d0/d46/d16/d83/f93 x:0 0 0 2024-04-02T04:42:39.349 INFO:tasks.workunit.client.0.smithi016.stdout:3/458: write d2/d2e/f30 [736710,128342] 0 2024-04-02T04:42:39.350 INFO:tasks.workunit.client.1.smithi027.stdout:9/36: link l8 lf 0 2024-04-02T04:42:39.351 INFO:tasks.workunit.client.1.smithi027.stdout:0/42: creat d4/dc/fd x:0 0 0 2024-04-02T04:42:39.352 INFO:tasks.workunit.client.1.smithi027.stdout:0/43: dread - d4/fa zero size 2024-04-02T04:42:39.352 INFO:tasks.workunit.client.1.smithi027.stdout:0/44: chown f1 15354705 1 2024-04-02T04:42:39.354 INFO:tasks.workunit.client.0.smithi016.stdout:7/405: dread - d0/d46/d37/d47/f5c zero size 2024-04-02T04:42:39.354 INFO:tasks.workunit.client.0.smithi016.stdout:3/459: chown d2/d2e/d36/d48 2765 1 2024-04-02T04:42:39.354 INFO:tasks.workunit.client.0.smithi016.stdout:1/383: dwrite d14/d19/f6b [0,4194304] 0 2024-04-02T04:42:39.355 INFO:tasks.workunit.client.1.smithi027.stdout:0/45: chown d4/dc/fd 2746392 1 2024-04-02T04:42:39.356 INFO:tasks.workunit.client.1.smithi027.stdout:0/46: stat d4/c5 0 2024-04-02T04:42:39.356 INFO:tasks.workunit.client.0.smithi016.stdout:7/406: dread - d0/d46/d52/f80 zero size 2024-04-02T04:42:39.358 INFO:tasks.workunit.client.0.smithi016.stdout:2/341: dread d12/d35/d3b/f46 [0,4194304] 0 2024-04-02T04:42:39.359 INFO:tasks.workunit.client.1.smithi027.stdout:7/39: truncate f1 2545496 0 2024-04-02T04:42:39.360 INFO:tasks.workunit.client.0.smithi016.stdout:2/342: fdatasync d12/d20/d4f/f53 0 2024-04-02T04:42:39.360 INFO:tasks.workunit.client.0.smithi016.stdout:3/460: rmdir d2/d15/d16 39 2024-04-02T04:42:39.362 INFO:tasks.workunit.client.1.smithi027.stdout:0/47: dwrite f1 [0,4194304] 0 2024-04-02T04:42:39.363 INFO:tasks.workunit.client.0.smithi016.stdout:7/407: rmdir d0/d46/d37/d3a/d6f 39 2024-04-02T04:42:39.364 INFO:tasks.workunit.client.1.smithi027.stdout:5/46: dread d0/dc/fe [0,4194304] 0 2024-04-02T04:42:39.368 INFO:tasks.workunit.client.1.smithi027.stdout:0/48: dread d4/f6 [0,4194304] 0 2024-04-02T04:42:39.369 INFO:tasks.workunit.client.1.smithi027.stdout:0/49: readlink l0 0 2024-04-02T04:42:39.369 INFO:tasks.workunit.client.0.smithi016.stdout:1/384: mkdir d14/d19/d7a 0 2024-04-02T04:42:39.370 INFO:tasks.workunit.client.1.smithi027.stdout:5/47: creat d0/d5/f12 x:0 0 0 2024-04-02T04:42:39.372 INFO:tasks.workunit.client.0.smithi016.stdout:2/343: mkdir d12/d20/d76 0 2024-04-02T04:42:39.373 INFO:tasks.workunit.client.1.smithi027.stdout:7/40: mknod d2/c9 0 2024-04-02T04:42:39.374 INFO:tasks.workunit.client.0.smithi016.stdout:0/380: write d0/f12 [1715975,58838] 0 2024-04-02T04:42:39.376 INFO:tasks.workunit.client.0.smithi016.stdout:7/408: mknod d0/d46/d37/d3a/d43/c94 0 2024-04-02T04:42:39.376 INFO:tasks.workunit.client.1.smithi027.stdout:0/50: creat d4/dc/fe x:0 0 0 2024-04-02T04:42:39.378 INFO:tasks.workunit.client.0.smithi016.stdout:0/381: write d0/f19 [576553,58828] 0 2024-04-02T04:42:39.379 INFO:tasks.workunit.client.0.smithi016.stdout:7/409: chown d0/d21/f57 0 1 2024-04-02T04:42:39.379 INFO:tasks.workunit.client.1.smithi027.stdout:9/37: getdents . 0 2024-04-02T04:42:39.382 INFO:tasks.workunit.client.0.smithi016.stdout:7/410: write d0/d46/d37/d3a/f7d [118017,19170] 0 2024-04-02T04:42:39.383 INFO:tasks.workunit.client.1.smithi027.stdout:0/51: symlink d4/dc/lf 0 2024-04-02T04:42:39.383 INFO:tasks.workunit.client.0.smithi016.stdout:2/344: dwrite d12/d70/f75 [0,4194304] 0 2024-04-02T04:42:39.384 INFO:tasks.workunit.client.1.smithi027.stdout:0/52: read - d4/dc/fd zero size 2024-04-02T04:42:39.384 INFO:tasks.workunit.client.1.smithi027.stdout:0/53: read - d4/fa zero size 2024-04-02T04:42:39.384 INFO:tasks.workunit.client.1.smithi027.stdout:5/48: mkdir d0/df/d11/d13 0 2024-04-02T04:42:39.385 INFO:tasks.workunit.client.1.smithi027.stdout:5/49: read - d0/d5/f12 zero size 2024-04-02T04:42:39.388 INFO:tasks.workunit.client.0.smithi016.stdout:3/461: mkdir d2/d15/d16/d8a 0 2024-04-02T04:42:39.390 INFO:tasks.workunit.client.0.smithi016.stdout:3/462: chown d2/d15/d16/d57/f64 61650 1 2024-04-02T04:42:39.391 INFO:tasks.workunit.client.0.smithi016.stdout:2/345: dwrite f8 [4194304,4194304] 0 2024-04-02T04:42:39.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.388+0000 7f55f8236700 1 -- 172.21.15.16:0/499111552 --> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f55f01047b0 con 0x7f55dc06d4a0 2024-04-02T04:42:39.392 INFO:tasks.workunit.client.0.smithi016.stdout:2/346: write d12/d35/d3b/f63 [44523,91804] 0 2024-04-02T04:42:39.395 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:42:39.395 INFO:teuthology.orchestra.run.smithi016.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:42:39.395 INFO:teuthology.orchestra.run.smithi016.stdout: "in_progress": true, 2024-04-02T04:42:39.395 INFO:teuthology.orchestra.run.smithi016.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-02T04:42:39.395 INFO:teuthology.orchestra.run.smithi016.stdout: "services_complete": [], 2024-04-02T04:42:39.395 INFO:teuthology.orchestra.run.smithi016.stdout: "progress": "1/23 daemons upgraded", 2024-04-02T04:42:39.395 INFO:teuthology.orchestra.run.smithi016.stdout: "message": "", 2024-04-02T04:42:39.395 INFO:teuthology.orchestra.run.smithi016.stdout: "is_paused": false 2024-04-02T04:42:39.395 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:42:39.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.391+0000 7f55e6ffd700 1 -- 172.21.15.16:0/499111552 <== mgr.24415 v2:172.21.15.27:6828/1554339939 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f55f01047b0 con 0x7f55dc06d4a0 2024-04-02T04:42:39.396 INFO:tasks.workunit.client.1.smithi027.stdout:0/54: dwrite d4/f6 [0,4194304] 0 2024-04-02T04:42:39.396 INFO:tasks.workunit.client.1.smithi027.stdout:0/55: dread - d4/dc/fe zero size 2024-04-02T04:42:39.397 INFO:tasks.workunit.client.0.smithi016.stdout:1/385: dread fd [0,4194304] 0 2024-04-02T04:42:39.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.395+0000 7f55f8236700 1 -- 172.21.15.16:0/499111552 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7f55dc06d4a0 msgr2=0x7f55dc06f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:39.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.395+0000 7f55f8236700 1 --2- 172.21.15.16:0/499111552 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7f55dc06d4a0 0x7f55dc06f960 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f55e0005d90 tx=0x7f55e0005d20 comp rx=0 tx=0).stop 2024-04-02T04:42:39.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.395+0000 7f55f8236700 1 -- 172.21.15.16:0/499111552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f00ff170 msgr2=0x7f55f0196500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:39.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.395+0000 7f55f8236700 1 --2- 172.21.15.16:0/499111552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f00ff170 0x7f55f0196500 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f55ec0059d0 tx=0x7f55ec00be20 comp rx=0 tx=0).stop 2024-04-02T04:42:39.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.395+0000 7f55f8236700 1 -- 172.21.15.16:0/499111552 shutdown_connections 2024-04-02T04:42:39.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.395+0000 7f55f8236700 1 --2- 172.21.15.16:0/499111552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f00ff170 0x7f55f0196500 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:39.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.395+0000 7f55f8236700 1 --2- 172.21.15.16:0/499111552 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7f55dc06d4a0 0x7f55dc06f960 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:39.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.395+0000 7f55f8236700 1 --2- 172.21.15.16:0/499111552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55f0100370 0x7f55f0196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:39.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.395+0000 7f55f8236700 1 -- 172.21.15.16:0/499111552 >> 172.21.15.16:0/499111552 conn(0x7f55f00fa730 msgr2=0x7f55f00fc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:42:39.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.395+0000 7f55f8236700 1 -- 172.21.15.16:0/499111552 shutdown_connections 2024-04-02T04:42:39.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.395+0000 7f55f8236700 1 -- 172.21.15.16:0/499111552 wait complete. 2024-04-02T04:42:39.399 INFO:tasks.workunit.client.0.smithi016.stdout:0/382: symlink d0/d4f/d76/l89 0 2024-04-02T04:42:39.402 INFO:tasks.workunit.client.0.smithi016.stdout:2/347: fdatasync d12/d20/f25 0 2024-04-02T04:42:39.406 INFO:tasks.workunit.client.1.smithi027.stdout:0/56: dwrite f2 [8388608,4194304] 0 2024-04-02T04:42:39.408 INFO:tasks.workunit.client.0.smithi016.stdout:1/386: creat d14/d19/d20/f7b x:0 0 0 2024-04-02T04:42:39.409 INFO:tasks.workunit.client.0.smithi016.stdout:1/387: dread - d14/d19/d20/f62 zero size 2024-04-02T04:42:39.410 INFO:tasks.workunit.client.1.smithi027.stdout:0/57: stat d4/dc/fe 0 2024-04-02T04:42:39.410 INFO:tasks.workunit.client.1.smithi027.stdout:5/50: link d0/c10 d0/df/d11/d13/c14 0 2024-04-02T04:42:39.410 INFO:tasks.workunit.client.1.smithi027.stdout:5/51: write d0/d5/f12 [149012,101701] 0 2024-04-02T04:42:39.411 INFO:tasks.workunit.client.0.smithi016.stdout:0/383: creat d0/d16/f8a x:0 0 0 2024-04-02T04:42:39.411 INFO:tasks.workunit.client.1.smithi027.stdout:7/41: write f1 [3231870,88904] 0 2024-04-02T04:42:39.412 INFO:tasks.workunit.client.1.smithi027.stdout:0/58: rename d4/dc to d4/d10 0 2024-04-02T04:42:39.414 INFO:tasks.workunit.client.0.smithi016.stdout:2/348: creat d12/d20/d4f/f77 x:0 0 0 2024-04-02T04:42:39.415 INFO:tasks.workunit.client.0.smithi016.stdout:0/384: dwrite d0/d16/f80 [0,4194304] 0 2024-04-02T04:42:39.416 INFO:tasks.workunit.client.0.smithi016.stdout:1/388: creat d14/d41/d43/d68/f7c x:0 0 0 2024-04-02T04:42:39.417 INFO:tasks.workunit.client.1.smithi027.stdout:0/59: creat d4/f11 x:0 0 0 2024-04-02T04:42:39.420 INFO:tasks.workunit.client.1.smithi027.stdout:5/52: mknod d0/df/d11/d13/c15 0 2024-04-02T04:42:39.422 INFO:tasks.workunit.client.1.smithi027.stdout:0/60: mknod d4/d10/c12 0 2024-04-02T04:42:39.424 INFO:tasks.workunit.client.0.smithi016.stdout:2/349: dread d12/d35/d3b/f60 [0,4194304] 0 2024-04-02T04:42:39.425 INFO:tasks.workunit.client.1.smithi027.stdout:5/53: mkdir d0/df/d11/d13/d16 0 2024-04-02T04:42:39.426 INFO:tasks.workunit.client.0.smithi016.stdout:0/385: creat d0/d16/d2e/d36/f8b x:0 0 0 2024-04-02T04:42:39.429 INFO:tasks.workunit.client.1.smithi027.stdout:0/61: rename d4/d10 to d4/d13 0 2024-04-02T04:42:39.432 INFO:tasks.workunit.client.1.smithi027.stdout:0/62: creat d4/f14 x:0 0 0 2024-04-02T04:42:39.433 INFO:tasks.workunit.client.1.smithi027.stdout:0/63: write d4/d13/fd [721183,70374] 0 2024-04-02T04:42:39.434 INFO:tasks.workunit.client.0.smithi016.stdout:2/350: mknod d12/d20/c78 0 2024-04-02T04:42:39.435 INFO:tasks.workunit.client.0.smithi016.stdout:0/386: mkdir d0/d48/d83/d8c 0 2024-04-02T04:42:39.437 INFO:tasks.workunit.client.0.smithi016.stdout:1/389: creat d14/d19/d1d/f7d x:0 0 0 2024-04-02T04:42:39.438 INFO:tasks.workunit.client.1.smithi027.stdout:0/64: creat d4/d13/f15 x:0 0 0 2024-04-02T04:42:39.438 INFO:tasks.workunit.client.1.smithi027.stdout:0/65: chown l0 7654 1 2024-04-02T04:42:39.439 INFO:tasks.workunit.client.1.smithi027.stdout:0/66: fsync d4/f11 0 2024-04-02T04:42:39.439 INFO:tasks.workunit.client.0.smithi016.stdout:2/351: rename d12/d14/c19 to d12/d14/c79 0 2024-04-02T04:42:39.440 INFO:tasks.workunit.client.1.smithi027.stdout:0/67: write d4/f11 [841541,115349] 0 2024-04-02T04:42:39.440 INFO:tasks.workunit.client.0.smithi016.stdout:8/397: sync 2024-04-02T04:42:39.440 INFO:tasks.workunit.client.0.smithi016.stdout:4/412: sync 2024-04-02T04:42:39.440 INFO:tasks.workunit.client.1.smithi027.stdout:5/54: rmdir d0/df/d11/d13/d16 0 2024-04-02T04:42:39.441 INFO:tasks.workunit.client.1.smithi027.stdout:5/55: chown d0/df 11818 1 2024-04-02T04:42:39.444 INFO:tasks.workunit.client.0.smithi016.stdout:0/387: rename d0/d16/d60 to d0/d16/d24/d4e/d8d 0 2024-04-02T04:42:39.448 INFO:tasks.workunit.client.0.smithi016.stdout:2/352: creat d12/d35/d3f/f7a x:0 0 0 2024-04-02T04:42:39.449 INFO:tasks.workunit.client.1.smithi027.stdout:5/56: rename d0/d5/ca to d0/df/d11/d13/c17 0 2024-04-02T04:42:39.450 INFO:tasks.workunit.client.1.smithi027.stdout:0/68: rmdir d4 39 2024-04-02T04:42:39.454 INFO:tasks.workunit.client.0.smithi016.stdout:0/388: creat d0/d16/d24/f8e x:0 0 0 2024-04-02T04:42:39.454 INFO:tasks.workunit.client.0.smithi016.stdout:1/390: truncate d14/d41/d43/f5f 435612 0 2024-04-02T04:42:39.457 INFO:tasks.workunit.client.0.smithi016.stdout:1/391: symlink d14/d41/l7e 0 2024-04-02T04:42:39.458 INFO:tasks.workunit.client.1.smithi027.stdout:5/57: link d0/df/d11/d13/c14 d0/df/d11/d13/c18 0 2024-04-02T04:42:39.458 INFO:tasks.workunit.client.0.smithi016.stdout:1/392: write d14/d19/d1d/d29/d39/f65 [316431,45088] 0 2024-04-02T04:42:39.459 INFO:tasks.workunit.client.0.smithi016.stdout:8/398: link d1/d6/c3b d1/d6/d36/d3f/d40/d49/c85 0 2024-04-02T04:42:39.460 INFO:tasks.workunit.client.1.smithi027.stdout:5/58: creat d0/dc/f19 x:0 0 0 2024-04-02T04:42:39.461 INFO:tasks.workunit.client.0.smithi016.stdout:2/353: creat d12/f7b x:0 0 0 2024-04-02T04:42:39.463 INFO:tasks.workunit.client.0.smithi016.stdout:4/413: getdents d5/d1d/d1e 0 2024-04-02T04:42:39.463 INFO:tasks.workunit.client.0.smithi016.stdout:4/414: chown d5/d8/ff 1 1 2024-04-02T04:42:39.468 INFO:tasks.workunit.client.1.smithi027.stdout:5/59: dwrite d0/d5/f12 [0,4194304] 0 2024-04-02T04:42:39.470 INFO:tasks.workunit.client.1.smithi027.stdout:5/60: mkdir d0/d1a 0 2024-04-02T04:42:39.473 INFO:tasks.workunit.client.1.smithi027.stdout:5/61: write d0/dc/fe [2165333,128290] 0 2024-04-02T04:42:39.474 INFO:tasks.workunit.client.1.smithi027.stdout:5/62: write d0/dc/fe [1489142,13026] 0 2024-04-02T04:42:39.481 INFO:tasks.workunit.client.1.smithi027.stdout:5/63: dwrite d0/f6 [0,4194304] 0 2024-04-02T04:42:39.484 INFO:tasks.workunit.client.1.smithi027.stdout:5/64: symlink d0/df/d11/l1b 0 2024-04-02T04:42:39.486 INFO:tasks.workunit.client.1.smithi027.stdout:5/65: mknod d0/c1c 0 2024-04-02T04:42:39.487 INFO:tasks.workunit.client.1.smithi027.stdout:5/66: mkdir d0/d1d 0 2024-04-02T04:42:39.490 INFO:tasks.workunit.client.1.smithi027.stdout:5/67: creat d0/df/d11/d13/f1e x:0 0 0 2024-04-02T04:42:39.490 INFO:tasks.workunit.client.1.smithi027.stdout:5/68: stat d0/d1d 0 2024-04-02T04:42:39.498 INFO:tasks.workunit.client.1.smithi027.stdout:5/69: truncate d0/d5/f12 2641352 0 2024-04-02T04:42:39.500 INFO:tasks.workunit.client.1.smithi027.stdout:5/70: creat d0/df/d11/f1f x:0 0 0 2024-04-02T04:42:39.502 INFO:tasks.workunit.client.1.smithi027.stdout:5/71: mkdir d0/d1a/d20 0 2024-04-02T04:42:39.504 INFO:tasks.workunit.client.1.smithi027.stdout:5/72: mknod d0/df/c21 0 2024-04-02T04:42:39.505 INFO:tasks.workunit.client.1.smithi027.stdout:5/73: write d0/dc/fe [532131,69975] 0 2024-04-02T04:42:39.507 INFO:tasks.workunit.client.0.smithi016.stdout:1/393: mknod d14/d19/d1d/d45/c7f 0 2024-04-02T04:42:39.507 INFO:tasks.workunit.client.0.smithi016.stdout:2/354: mknod d12/d35/d3b/d42/d67/c7c 0 2024-04-02T04:42:39.515 INFO:tasks.workunit.client.0.smithi016.stdout:2/355: dwrite f8 [4194304,4194304] 0 2024-04-02T04:42:39.516 INFO:tasks.workunit.client.0.smithi016.stdout:8/399: dread d1/d6/f21 [0,4194304] 0 2024-04-02T04:42:39.520 INFO:tasks.workunit.client.0.smithi016.stdout:2/356: symlink d12/d35/d3f/d2b/l7d 0 2024-04-02T04:42:39.521 INFO:tasks.workunit.client.1.smithi027.stdout:5/74: dwrite d0/dc/fe [0,4194304] 0 2024-04-02T04:42:39.521 INFO:tasks.workunit.client.0.smithi016.stdout:2/357: chown d12/d20/c78 512955899 1 2024-04-02T04:42:39.522 INFO:tasks.workunit.client.1.smithi027.stdout:5/75: read - d0/df/d11/d13/f1e zero size 2024-04-02T04:42:39.526 INFO:tasks.workunit.client.0.smithi016.stdout:1/394: dread d14/f17 [0,4194304] 0 2024-04-02T04:42:39.528 INFO:tasks.workunit.client.0.smithi016.stdout:2/358: dwrite f8 [4194304,4194304] 0 2024-04-02T04:42:39.530 INFO:tasks.workunit.client.0.smithi016.stdout:1/395: write d14/f1c [693301,25548] 0 2024-04-02T04:42:39.531 INFO:tasks.workunit.client.0.smithi016.stdout:4/415: dread d5/d8/fb [0,4194304] 0 2024-04-02T04:42:39.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.529+0000 7f1579e53700 1 -- 172.21.15.16:0/3574212577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1574072ac0 msgr2=0x7f157410aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:39.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.529+0000 7f1579e53700 1 --2- 172.21.15.16:0/3574212577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1574072ac0 0x7f157410aed0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1564009910 tx=0x7f1564009c20 comp rx=0 tx=0).stop 2024-04-02T04:42:39.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.529+0000 7f1579e53700 1 -- 172.21.15.16:0/3574212577 shutdown_connections 2024-04-02T04:42:39.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.529+0000 7f1579e53700 1 --2- 172.21.15.16:0/3574212577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f157410b410 0x7f157410d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:39.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.529+0000 7f1579e53700 1 --2- 172.21.15.16:0/3574212577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1574072ac0 0x7f157410aed0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:39.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.529+0000 7f1579e53700 1 -- 172.21.15.16:0/3574212577 >> 172.21.15.16:0/3574212577 conn(0x7f157406da40 msgr2=0x7f157406fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:42:39.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.529+0000 7f1579e53700 1 -- 172.21.15.16:0/3574212577 shutdown_connections 2024-04-02T04:42:39.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.529+0000 7f1579e53700 1 -- 172.21.15.16:0/3574212577 wait complete. 2024-04-02T04:42:39.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.531+0000 7f1579e53700 1 Processor -- start 2024-04-02T04:42:39.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.532+0000 7f1579e53700 1 -- start start 2024-04-02T04:42:39.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.532+0000 7f1579e53700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1574072ac0 0x7f157419ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:39.535 INFO:tasks.workunit.client.0.smithi016.stdout:4/416: write d5/d17/d18/d1f/f53 [1802687,108503] 0 2024-04-02T04:42:39.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.532+0000 7f1579e53700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f157410b410 0x7f157419f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:39.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.532+0000 7f1579e53700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157419f960 con 0x7f1574072ac0 2024-04-02T04:42:39.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.532+0000 7f15737fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1574072ac0 0x7f157419ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:39.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.533+0000 7f15737fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1574072ac0 0x7f157419ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52006/0 (socket says 172.21.15.16:52006) 2024-04-02T04:42:39.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.533+0000 7f15737fe700 1 -- 172.21.15.16:0/1606188615 learned_addr learned my addr 172.21.15.16:0/1606188615 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:42:39.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.533+0000 7f1579e53700 1 -- 172.21.15.16:0/1606188615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15741a4320 con 0x7f157410b410 2024-04-02T04:42:39.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.533+0000 7f1572ffd700 1 --2- 172.21.15.16:0/1606188615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f157410b410 0x7f157419f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:39.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.534+0000 7f15737fe700 1 -- 172.21.15.16:0/1606188615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f157410b410 msgr2=0x7f157419f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:39.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.534+0000 7f15737fe700 1 --2- 172.21.15.16:0/1606188615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f157410b410 0x7f157419f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:39.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.534+0000 7f15737fe700 1 -- 172.21.15.16:0/1606188615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15640095f0 con 0x7f1574072ac0 2024-04-02T04:42:39.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.534+0000 7f1572ffd700 1 --2- 172.21.15.16:0/1606188615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f157410b410 0x7f157419f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T04:42:39.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.535+0000 7f15737fe700 1 --2- 172.21.15.16:0/1606188615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1574072ac0 0x7f157419ee00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1564005090 tx=0x7f156400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:42:39.538 INFO:tasks.workunit.client.0.smithi016.stdout:2/359: link d12/d14/f1f d12/d35/d3f/f7e 0 2024-04-02T04:42:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.535+0000 7f1570ff9700 1 -- 172.21.15.16:0/1606188615 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f156401d070 con 0x7f1574072ac0 2024-04-02T04:42:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.536+0000 7f1570ff9700 1 -- 172.21.15.16:0/1606188615 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f15640048e0 con 0x7f1574072ac0 2024-04-02T04:42:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.536+0000 7f1570ff9700 1 -- 172.21.15.16:0/1606188615 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1564003ba0 con 0x7f1574072ac0 2024-04-02T04:42:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.536+0000 7f1579e53700 1 -- 172.21.15.16:0/1606188615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15741a44c0 con 0x7f1574072ac0 2024-04-02T04:42:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.536+0000 7f1579e53700 1 -- 172.21.15.16:0/1606188615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15741a49a0 con 0x7f1574072ac0 2024-04-02T04:42:39.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.538+0000 7f1579e53700 1 -- 172.21.15.16:0/1606188615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1574198e30 con 0x7f1574072ac0 2024-04-02T04:42:39.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.539+0000 7f1570ff9700 1 -- 172.21.15.16:0/1606188615 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 91380+0+0 (secure 0 0 0) 0x7f1564004a50 con 0x7f1574072ac0 2024-04-02T04:42:39.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.539+0000 7f1570ff9700 1 --2- 172.21.15.16:0/1606188615 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7f155c06d4e0 0x7f155c06f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:42:39.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.540+0000 7f1572ffd700 1 --2- 172.21.15.16:0/1606188615 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7f155c06d4e0 0x7f155c06f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:42:39.543 INFO:tasks.workunit.client.0.smithi016.stdout:1/396: getdents d14/d41 0 2024-04-02T04:42:39.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.540+0000 7f1570ff9700 1 -- 172.21.15.16:0/1606188615 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f156408e400 con 0x7f1574072ac0 2024-04-02T04:42:39.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.541+0000 7f1572ffd700 1 --2- 172.21.15.16:0/1606188615 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7f155c06d4e0 0x7f155c06f9a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1574072fb0 tx=0x7f1568009190 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:42:39.544 INFO:tasks.workunit.client.0.smithi016.stdout:2/360: symlink d12/d35/d3b/d73/l7f 0 2024-04-02T04:42:39.544 INFO:tasks.workunit.client.0.smithi016.stdout:1/397: write d14/d19/d20/f3b [1980986,38245] 0 2024-04-02T04:42:39.546 INFO:tasks.workunit.client.0.smithi016.stdout:1/398: write d14/d19/d1d/d29/f51 [1501579,14337] 0 2024-04-02T04:42:39.547 INFO:tasks.workunit.client.0.smithi016.stdout:2/361: fsync d12/d35/d3f/d2b/f69 0 2024-04-02T04:42:39.548 INFO:tasks.workunit.client.0.smithi016.stdout:2/362: chown d12/d26/f48 1857876 1 2024-04-02T04:42:39.548 INFO:tasks.workunit.client.0.smithi016.stdout:2/363: readlink d12/d35/d3f/d2b/l7d 0 2024-04-02T04:42:39.549 INFO:tasks.workunit.client.0.smithi016.stdout:2/364: chown d12/d14/f37 2889714 1 2024-04-02T04:42:39.549 INFO:tasks.workunit.client.1.smithi027.stdout:5/76: fsync d0/f6 0 2024-04-02T04:42:39.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.548+0000 7f1570ff9700 1 -- 172.21.15.16:0/1606188615 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184299 (secure 0 0 0) 0x7f1564057d80 con 0x7f1574072ac0 2024-04-02T04:42:39.552 INFO:tasks.workunit.client.0.smithi016.stdout:6/453: sync 2024-04-02T04:42:39.552 INFO:tasks.workunit.client.0.smithi016.stdout:5/405: sync 2024-04-02T04:42:39.553 INFO:tasks.workunit.client.0.smithi016.stdout:6/454: fdatasync d0/d9/f36 0 2024-04-02T04:42:39.561 INFO:tasks.workunit.client.0.smithi016.stdout:6/455: mknod d0/d5c/c88 0 2024-04-02T04:42:39.565 INFO:tasks.workunit.client.0.smithi016.stdout:1/399: link d14/d19/d1e/c3e d14/d19/d1e/d6d/c80 0 2024-04-02T04:42:39.567 INFO:tasks.workunit.client.0.smithi016.stdout:5/406: read d3/df/d18/f43 [973237,67615] 0 2024-04-02T04:42:39.569 INFO:tasks.workunit.client.0.smithi016.stdout:6/456: rename d0/d1f to d0/d9/d16/d56/d89 0 2024-04-02T04:42:39.574 INFO:tasks.workunit.client.0.smithi016.stdout:1/400: read d14/f28 [1033389,22697] 0 2024-04-02T04:42:39.576 INFO:tasks.workunit.client.0.smithi016.stdout:5/407: dread - d3/df/f14 zero size 2024-04-02T04:42:39.576 INFO:tasks.workunit.client.0.smithi016.stdout:5/408: dread - d3/df/d18/f98 zero size 2024-04-02T04:42:39.578 INFO:tasks.workunit.client.0.smithi016.stdout:5/409: write d3/d40/f79 [2613887,85665] 0 2024-04-02T04:42:39.580 INFO:tasks.workunit.client.0.smithi016.stdout:2/365: truncate d12/d35/d3f/f7e 2209476 0 2024-04-02T04:42:39.581 INFO:tasks.workunit.client.0.smithi016.stdout:1/401: mkdir d14/d19/d1d/d29/d39/d46/d81 0 2024-04-02T04:42:39.585 INFO:tasks.workunit.client.0.smithi016.stdout:4/417: dwrite d5/d1d/d1e/f3c [0,4194304] 0 2024-04-02T04:42:39.585 INFO:tasks.workunit.client.0.smithi016.stdout:5/410: creat d3/d40/d4f/d6b/f9e x:0 0 0 2024-04-02T04:42:39.586 INFO:tasks.workunit.client.0.smithi016.stdout:6/457: dread d0/d9/d16/d56/d89/f45 [0,4194304] 0 2024-04-02T04:42:39.586 INFO:tasks.workunit.client.0.smithi016.stdout:4/418: readlink d5/l7a 0 2024-04-02T04:42:39.590 INFO:tasks.workunit.client.0.smithi016.stdout:6/458: write d0/d9/d16/d56/d89/d67/d6f/f82 [224260,55828] 0 2024-04-02T04:42:39.592 INFO:tasks.workunit.client.0.smithi016.stdout:6/459: write d0/d9/d16/d46/f55 [237220,105829] 0 2024-04-02T04:42:39.593 INFO:tasks.workunit.client.0.smithi016.stdout:6/460: chown d0/d9/df/f7d 4277109 1 2024-04-02T04:42:39.595 INFO:tasks.workunit.client.0.smithi016.stdout:6/461: read d0/d9/d16/d56/f11 [472826,21094] 0 2024-04-02T04:42:39.599 INFO:tasks.workunit.client.0.smithi016.stdout:5/411: symlink d3/d40/d4f/d6b/d45/l9f 0 2024-04-02T04:42:39.601 INFO:tasks.workunit.client.0.smithi016.stdout:6/462: mkdir d0/d9/d16/d56/d89/d67/d6f/d8a 0 2024-04-02T04:42:39.605 INFO:tasks.workunit.client.0.smithi016.stdout:5/412: creat d3/d40/d4f/d67/fa0 x:0 0 0 2024-04-02T04:42:39.609 INFO:tasks.workunit.client.0.smithi016.stdout:5/413: dread - d3/df/d18/d2f/f3f zero size 2024-04-02T04:42:39.610 INFO:tasks.workunit.client.0.smithi016.stdout:5/414: chown d3/df/d18/c50 102 1 2024-04-02T04:42:39.611 INFO:tasks.workunit.client.0.smithi016.stdout:5/415: dread - d3/d40/d4f/d6b/f9e zero size 2024-04-02T04:42:39.611 INFO:tasks.workunit.client.0.smithi016.stdout:5/416: fdatasync d3/f5b 0 2024-04-02T04:42:39.616 INFO:tasks.workunit.client.0.smithi016.stdout:1/402: rmdir d14/d19/d1d/d29/d39 39 2024-04-02T04:42:39.617 INFO:tasks.workunit.client.0.smithi016.stdout:1/403: dread - d14/d19/d20/f5b zero size 2024-04-02T04:42:39.623 INFO:tasks.workunit.client.0.smithi016.stdout:6/463: dread d0/d9/d16/f2a [4194304,4194304] 0 2024-04-02T04:42:39.624 INFO:tasks.workunit.client.0.smithi016.stdout:6/464: write d0/d9/d16/d56/d59/f60 [653153,122] 0 2024-04-02T04:42:39.626 INFO:tasks.workunit.client.0.smithi016.stdout:2/366: truncate d12/d26/f5a 854944 0 2024-04-02T04:42:39.628 INFO:tasks.workunit.client.0.smithi016.stdout:1/404: getdents d14/d19/d50 0 2024-04-02T04:42:39.629 INFO:tasks.workunit.client.0.smithi016.stdout:1/405: chown d14/d19/d1d/f2c 18330 1 2024-04-02T04:42:39.631 INFO:tasks.workunit.client.0.smithi016.stdout:2/367: dwrite d12/f7b [0,4194304] 0 2024-04-02T04:42:39.633 INFO:tasks.workunit.client.0.smithi016.stdout:4/419: write d5/d1d/d2d/f29 [2926824,37795] 0 2024-04-02T04:42:39.634 INFO:tasks.workunit.client.0.smithi016.stdout:6/465: link d0/d9/f36 d0/d6e/f8b 0 2024-04-02T04:42:39.634 INFO:tasks.workunit.client.0.smithi016.stdout:4/420: write d5/d16/d3f/f54 [287953,124361] 0 2024-04-02T04:42:39.637 INFO:tasks.workunit.client.0.smithi016.stdout:6/466: dread d0/d9/d16/d46/f55 [0,4194304] 0 2024-04-02T04:42:39.637 INFO:tasks.workunit.client.0.smithi016.stdout:4/421: dread d5/d1d/f78 [0,4194304] 0 2024-04-02T04:42:39.648 INFO:tasks.workunit.client.0.smithi016.stdout:2/368: symlink d12/l80 0 2024-04-02T04:42:39.653 INFO:tasks.workunit.client.0.smithi016.stdout:1/406: rename d14/f1c to d14/d19/f82 0 2024-04-02T04:42:39.657 INFO:tasks.workunit.client.0.smithi016.stdout:6/467: link d0/fc d0/d78/f8c 0 2024-04-02T04:42:39.658 INFO:tasks.workunit.client.0.smithi016.stdout:4/422: creat d5/d17/d76/d7e/f81 x:0 0 0 2024-04-02T04:42:39.664 INFO:tasks.workunit.client.0.smithi016.stdout:6/468: unlink d0/d9/d16/d56/d89/f3a 0 2024-04-02T04:42:39.666 INFO:tasks.workunit.client.0.smithi016.stdout:1/407: getdents d14/d19/d7a 0 2024-04-02T04:42:39.668 INFO:tasks.workunit.client.0.smithi016.stdout:6/469: creat d0/d9/d16/f8d x:0 0 0 2024-04-02T04:42:39.668 INFO:tasks.workunit.client.0.smithi016.stdout:4/423: dread d5/d8/ff [0,4194304] 0 2024-04-02T04:42:39.669 INFO:tasks.workunit.client.0.smithi016.stdout:6/470: dread - d0/d9/f33 zero size 2024-04-02T04:42:39.672 INFO:tasks.workunit.client.0.smithi016.stdout:4/424: rmdir d5/d1d/d2d 39 2024-04-02T04:42:39.675 INFO:tasks.workunit.client.0.smithi016.stdout:1/408: creat d14/f83 x:0 0 0 2024-04-02T04:42:39.675 INFO:tasks.workunit.client.0.smithi016.stdout:6/471: dwrite d0/d9/f33 [0,4194304] 0 2024-04-02T04:42:39.682 INFO:tasks.workunit.client.0.smithi016.stdout:7/411: sync 2024-04-02T04:42:39.682 INFO:tasks.workunit.client.0.smithi016.stdout:9/416: sync 2024-04-02T04:42:39.684 INFO:tasks.workunit.client.0.smithi016.stdout:1/409: creat d14/d19/d1e/d71/f84 x:0 0 0 2024-04-02T04:42:39.685 INFO:tasks.workunit.client.0.smithi016.stdout:4/425: mknod d5/d1d/d5f/d65/c82 0 2024-04-02T04:42:39.690 INFO:tasks.workunit.client.0.smithi016.stdout:9/417: rmdir d0/d4a 39 2024-04-02T04:42:39.691 INFO:tasks.workunit.client.0.smithi016.stdout:9/418: write d0/f2d [842890,18800] 0 2024-04-02T04:42:39.693 INFO:tasks.workunit.client.0.smithi016.stdout:4/426: mknod d5/d1d/d1e/c83 0 2024-04-02T04:42:39.693 INFO:tasks.workunit.client.0.smithi016.stdout:6/472: dread d0/d9/f32 [0,4194304] 0 2024-04-02T04:42:39.695 INFO:tasks.workunit.client.0.smithi016.stdout:9/419: read d0/d9/db/f43 [1622153,74582] 0 2024-04-02T04:42:39.695 INFO:tasks.workunit.client.0.smithi016.stdout:4/427: read d5/d8/f7b [2025760,73036] 0 2024-04-02T04:42:39.697 INFO:tasks.workunit.client.0.smithi016.stdout:1/410: mknod d14/c85 0 2024-04-02T04:42:39.701 INFO:tasks.workunit.client.0.smithi016.stdout:6/473: unlink d0/d9/df/f7d 0 2024-04-02T04:42:39.703 INFO:tasks.workunit.client.0.smithi016.stdout:2/369: dread d12/d26/f5a [0,4194304] 0 2024-04-02T04:42:39.703 INFO:tasks.workunit.client.0.smithi016.stdout:9/420: dwrite d0/d4e/d50/d63/d6e/f77 [0,4194304] 0 2024-04-02T04:42:39.704 INFO:tasks.workunit.client.0.smithi016.stdout:2/370: chown d12/d14/l3a 360120 1 2024-04-02T04:42:39.712 INFO:tasks.workunit.client.0.smithi016.stdout:2/371: dwrite d12/d35/d3f/f7a [0,4194304] 0 2024-04-02T04:42:39.712 INFO:tasks.workunit.client.0.smithi016.stdout:4/428: creat d5/d1d/d5f/d65/d6f/f84 x:0 0 0 2024-04-02T04:42:39.717 INFO:tasks.workunit.client.0.smithi016.stdout:4/429: mknod d5/de/c85 0 2024-04-02T04:42:39.723 INFO:tasks.workunit.client.0.smithi016.stdout:4/430: mknod d5/d17/d76/c86 0 2024-04-02T04:42:39.723 INFO:tasks.workunit.client.0.smithi016.stdout:9/421: link d0/d4e/d50/d63/d6e/d1a/f48 d0/d4e/d50/d63/d6e/d1a/d74/f79 0 2024-04-02T04:42:39.724 INFO:tasks.workunit.client.1.smithi027.stdout:5/77: truncate d0/dc/fe 1520316 0 2024-04-02T04:42:39.725 INFO:tasks.workunit.client.1.smithi027.stdout:5/78: stat d0/dc/fe 0 2024-04-02T04:42:39.726 INFO:tasks.workunit.client.0.smithi016.stdout:1/411: getdents d14/d19/d1d/d29/d39 0 2024-04-02T04:42:39.727 INFO:tasks.workunit.client.0.smithi016.stdout:4/431: unlink d5/d17/d18/f48 0 2024-04-02T04:42:39.730 INFO:tasks.workunit.client.0.smithi016.stdout:1/412: mkdir d14/d19/d1d/d45/d4f/d86 0 2024-04-02T04:42:39.731 INFO:tasks.workunit.client.0.smithi016.stdout:9/422: getdents d0/d3/d39 0 2024-04-02T04:42:39.731 INFO:tasks.workunit.client.0.smithi016.stdout:4/432: write d5/f10 [3120077,86068] 0 2024-04-02T04:42:39.733 INFO:tasks.workunit.client.0.smithi016.stdout:9/423: readlink d0/d3/l17 0 2024-04-02T04:42:39.735 INFO:tasks.workunit.client.1.smithi027.stdout:5/79: dread d0/f6 [0,4194304] 0 2024-04-02T04:42:39.736 INFO:tasks.workunit.client.0.smithi016.stdout:6/474: dwrite d0/d9/d16/d56/f25 [0,4194304] 0 2024-04-02T04:42:39.737 INFO:tasks.workunit.client.1.smithi027.stdout:5/80: dread d0/dc/fe [0,4194304] 0 2024-04-02T04:42:39.738 INFO:tasks.workunit.client.0.smithi016.stdout:9/424: creat d0/d3/d31/d53/f7a x:0 0 0 2024-04-02T04:42:39.738 INFO:tasks.workunit.client.0.smithi016.stdout:6/475: dread - d0/f73 zero size 2024-04-02T04:42:39.742 INFO:tasks.workunit.client.0.smithi016.stdout:1/413: rename d14/d19/d1e/d6d/c80 to d14/d19/d1d/d29/d4c/c87 0 2024-04-02T04:42:39.742 INFO:tasks.workunit.client.1.smithi027.stdout:5/81: creat d0/d5/f22 x:0 0 0 2024-04-02T04:42:39.743 INFO:tasks.workunit.client.0.smithi016.stdout:1/414: dread - d14/d41/d43/d68/f7c zero size 2024-04-02T04:42:39.743 INFO:tasks.workunit.client.0.smithi016.stdout:4/433: dwrite d5/f4e [4194304,4194304] 0 2024-04-02T04:42:39.744 INFO:tasks.workunit.client.0.smithi016.stdout:4/434: dread - d5/f60 zero size 2024-04-02T04:42:39.748 INFO:tasks.workunit.client.1.smithi027.stdout:5/82: dwrite d0/dc/f19 [0,4194304] 0 2024-04-02T04:42:39.748 INFO:tasks.workunit.client.0.smithi016.stdout:4/435: dread d5/d16/f55 [0,4194304] 0 2024-04-02T04:42:39.750 INFO:tasks.workunit.client.0.smithi016.stdout:4/436: mknod d5/d16/c87 0 2024-04-02T04:42:39.752 INFO:tasks.workunit.client.1.smithi027.stdout:5/83: unlink d0/df/d11/l1b 0 2024-04-02T04:42:39.754 INFO:tasks.workunit.client.1.smithi027.stdout:5/84: write d0/df/d11/d13/f1e [274470,104856] 0 2024-04-02T04:42:39.754 INFO:tasks.workunit.client.0.smithi016.stdout:4/437: mkdir d5/d16/d88 0 2024-04-02T04:42:39.756 INFO:tasks.workunit.client.0.smithi016.stdout:4/438: mknod d5/d17/d76/c89 0 2024-04-02T04:42:39.757 INFO:tasks.workunit.client.1.smithi027.stdout:5/85: unlink d0/dc/f19 0 2024-04-02T04:42:39.758 INFO:tasks.workunit.client.0.smithi016.stdout:5/417: sync 2024-04-02T04:42:39.758 INFO:tasks.workunit.client.0.smithi016.stdout:0/389: sync 2024-04-02T04:42:39.759 INFO:tasks.workunit.client.0.smithi016.stdout:3/463: sync 2024-04-02T04:42:39.759 INFO:tasks.workunit.client.0.smithi016.stdout:8/400: sync 2024-04-02T04:42:39.760 INFO:tasks.workunit.client.1.smithi027.stdout:5/86: unlink d0/c1c 0 2024-04-02T04:42:39.762 INFO:tasks.workunit.client.1.smithi027.stdout:5/87: chown d0/df/d11/d13/c18 4374 1 2024-04-02T04:42:39.764 INFO:tasks.workunit.client.1.smithi027.stdout:5/88: mknod d0/dc/c23 0 2024-04-02T04:42:39.768 INFO:tasks.workunit.client.0.smithi016.stdout:5/418: dwrite d3/df/d18/f89 [0,4194304] 0 2024-04-02T04:42:39.770 INFO:tasks.workunit.client.0.smithi016.stdout:4/439: link d5/d1d/d5f/d65/c77 d5/d17/d76/d7e/c8a 0 2024-04-02T04:42:39.772 INFO:tasks.workunit.client.0.smithi016.stdout:3/464: truncate d2/f3f 5307505 0 2024-04-02T04:42:39.772 INFO:tasks.workunit.client.0.smithi016.stdout:9/425: dread d0/f2c [0,4194304] 0 2024-04-02T04:42:39.772 INFO:tasks.workunit.client.0.smithi016.stdout:5/419: dread - d3/df/d18/d26/f8e zero size 2024-04-02T04:42:39.773 INFO:tasks.workunit.client.1.smithi027.stdout:5/89: dwrite d0/df/d11/d13/f1e [0,4194304] 0 2024-04-02T04:42:39.774 INFO:tasks.workunit.client.0.smithi016.stdout:8/401: getdents d1 0 2024-04-02T04:42:39.774 INFO:tasks.workunit.client.0.smithi016.stdout:9/426: fsync d0/d9/f3d 0 2024-04-02T04:42:39.774 INFO:tasks.workunit.client.0.smithi016.stdout:0/390: dread d0/d16/d24/f20 [0,4194304] 0 2024-04-02T04:42:39.778 INFO:tasks.workunit.client.1.smithi027.stdout:5/90: creat d0/f24 x:0 0 0 2024-04-02T04:42:39.778 INFO:tasks.workunit.client.0.smithi016.stdout:0/391: truncate d0/d16/d23/f3f 997085 0 2024-04-02T04:42:39.781 INFO:tasks.workunit.client.0.smithi016.stdout:9/427: symlink d0/d4e/l7b 0 2024-04-02T04:42:39.783 INFO:tasks.workunit.client.0.smithi016.stdout:8/402: mkdir d1/d6/d86 0 2024-04-02T04:42:39.784 INFO:tasks.workunit.client.0.smithi016.stdout:1/415: truncate d14/d19/d1d/f2c 2115710 0 2024-04-02T04:42:39.784 INFO:tasks.workunit.client.1.smithi027.stdout:5/91: mknod d0/d1d/c25 0 2024-04-02T04:42:39.784 INFO:tasks.workunit.client.1.smithi027.stdout:5/92: chown d0/df 1699609210 1 2024-04-02T04:42:39.785 INFO:tasks.workunit.client.0.smithi016.stdout:3/465: dwrite d2/d15/d1a/f39 [0,4194304] 0 2024-04-02T04:42:39.785 INFO:tasks.workunit.client.0.smithi016.stdout:0/392: write d0/d16/d2e/d37/f3b [726643,75586] 0 2024-04-02T04:42:39.786 INFO:tasks.workunit.client.1.smithi027.stdout:5/93: creat d0/df/d11/d13/f26 x:0 0 0 2024-04-02T04:42:39.787 INFO:tasks.workunit.client.1.smithi027.stdout:5/94: readlink d0/l1 0 2024-04-02T04:42:39.788 INFO:tasks.workunit.client.1.smithi027.stdout:5/95: symlink d0/d1a/l27 0 2024-04-02T04:42:39.791 INFO:tasks.workunit.client.0.smithi016.stdout:0/393: fsync d0/d16/f18 0 2024-04-02T04:42:39.791 INFO:tasks.workunit.client.0.smithi016.stdout:9/428: dread d0/d33/f38 [0,4194304] 0 2024-04-02T04:42:39.792 INFO:tasks.workunit.client.0.smithi016.stdout:6/476: dwrite d0/d9/d16/d56/d89/f26 [0,4194304] 0 2024-04-02T04:42:39.793 INFO:tasks.workunit.client.0.smithi016.stdout:6/477: stat d0/c5 0 2024-04-02T04:42:39.798 INFO:tasks.workunit.client.0.smithi016.stdout:1/416: creat d14/d19/d1d/d29/d4c/f88 x:0 0 0 2024-04-02T04:42:39.798 INFO:tasks.workunit.client.0.smithi016.stdout:4/440: dread d5/d17/f35 [0,4194304] 0 2024-04-02T04:42:39.798 INFO:tasks.workunit.client.0.smithi016.stdout:9/429: dread - d0/d9/d4d/f5f zero size 2024-04-02T04:42:39.799 INFO:tasks.workunit.client.0.smithi016.stdout:8/403: rename d1/d6/d36/d3f/d40/d49/d38/f76 to d1/d70/d71/d74/d82/f87 0 2024-04-02T04:42:39.800 INFO:tasks.workunit.client.0.smithi016.stdout:0/394: mknod d0/d16/d24/d4e/c8f 0 2024-04-02T04:42:39.801 INFO:tasks.workunit.client.1.smithi027.stdout:5/96: rmdir d0/dc 39 2024-04-02T04:42:39.802 INFO:tasks.workunit.client.0.smithi016.stdout:1/417: dread - d14/d41/d43/d68/f7c zero size 2024-04-02T04:42:39.804 INFO:tasks.workunit.client.0.smithi016.stdout:4/441: write d5/d1d/f21 [935715,44272] 0 2024-04-02T04:42:39.804 INFO:tasks.workunit.client.1.smithi027.stdout:5/97: creat d0/df/d11/d13/f28 x:0 0 0 2024-04-02T04:42:39.806 INFO:tasks.workunit.client.0.smithi016.stdout:1/418: fdatasync d14/d19/d20/f5b 0 2024-04-02T04:42:39.808 INFO:tasks.workunit.client.0.smithi016.stdout:4/442: stat d5/d1d/d6c/f72 0 2024-04-02T04:42:39.808 INFO:tasks.workunit.client.0.smithi016.stdout:3/466: dwrite d2/d15/d16/d3d/d46/f5a [0,4194304] 0 2024-04-02T04:42:39.811 INFO:tasks.workunit.client.0.smithi016.stdout:6/478: dwrite d0/d9/d16/d56/d59/f86 [0,4194304] 0 2024-04-02T04:42:39.811 INFO:tasks.workunit.client.0.smithi016.stdout:0/395: unlink d0/d16/d2e/d36/c5c 0 2024-04-02T04:42:39.811 INFO:tasks.workunit.client.0.smithi016.stdout:4/443: stat d5/d17/l52 0 2024-04-02T04:42:39.812 INFO:tasks.workunit.client.0.smithi016.stdout:0/396: stat d0/d16/f8a 0 2024-04-02T04:42:39.812 INFO:tasks.workunit.client.1.smithi027.stdout:5/98: rename d0/dc to d0/df/d29 0 2024-04-02T04:42:39.814 INFO:tasks.workunit.client.0.smithi016.stdout:0/397: readlink d0/d16/d24/l27 0 2024-04-02T04:42:39.814 INFO:tasks.workunit.client.0.smithi016.stdout:6/479: write d0/d5c/f6b [881552,63861] 0 2024-04-02T04:42:39.817 INFO:tasks.workunit.client.0.smithi016.stdout:9/430: rename d0/d9/d4d/f5f to d0/d4e/d50/d63/d6e/d4b/d58/d6a/f7c 0 2024-04-02T04:42:39.818 INFO:tasks.workunit.client.0.smithi016.stdout:0/398: write d0/d16/d24/f7b [1968003,94292] 0 2024-04-02T04:42:39.820 INFO:tasks.workunit.client.0.smithi016.stdout:8/404: creat d1/d70/f88 x:0 0 0 2024-04-02T04:42:39.822 INFO:tasks.workunit.client.0.smithi016.stdout:9/431: fsync d0/d9/f3d 0 2024-04-02T04:42:39.823 INFO:tasks.workunit.client.0.smithi016.stdout:4/444: creat d5/d17/d76/f8b x:0 0 0 2024-04-02T04:42:39.826 INFO:tasks.workunit.client.0.smithi016.stdout:1/419: dwrite d14/d19/f4b [0,4194304] 0 2024-04-02T04:42:39.827 INFO:tasks.workunit.client.0.smithi016.stdout:3/467: dwrite d2/d15/d16/d3d/d46/f5a [0,4194304] 0 2024-04-02T04:42:39.829 INFO:tasks.workunit.client.1.smithi027.stdout:5/99: link d0/df/d29/cd d0/d1a/d20/c2a 0 2024-04-02T04:42:39.832 INFO:tasks.workunit.client.0.smithi016.stdout:6/480: mknod d0/d9/df/c8e 0 2024-04-02T04:42:39.836 INFO:tasks.workunit.client.0.smithi016.stdout:0/399: mknod d0/d48/c90 0 2024-04-02T04:42:39.837 INFO:tasks.workunit.client.0.smithi016.stdout:0/400: fdatasync d0/d16/d24/d4e/d8d/d71/f72 0 2024-04-02T04:42:39.838 INFO:tasks.workunit.client.0.smithi016.stdout:0/401: dread - d0/d16/d2e/d53/d6f/f88 zero size 2024-04-02T04:42:39.839 INFO:tasks.workunit.client.1.smithi027.stdout:5/100: dwrite d0/df/d11/f1f [0,4194304] 0 2024-04-02T04:42:39.842 INFO:tasks.workunit.client.1.smithi027.stdout:5/101: dread d0/df/d29/fe [0,4194304] 0 2024-04-02T04:42:39.843 INFO:tasks.workunit.client.0.smithi016.stdout:9/432: chown d0/d4e/d50/d63/d6e/l62 6352953 1 2024-04-02T04:42:39.845 INFO:tasks.workunit.client.0.smithi016.stdout:9/433: write d0/f5d [791314,35881] 0 2024-04-02T04:42:39.851 INFO:tasks.workunit.client.0.smithi016.stdout:6/481: unlink d0/d9/d16/d56/l15 0 2024-04-02T04:42:39.852 INFO:tasks.workunit.client.1.smithi027.stdout:5/102: dwrite d0/df/d11/f1f [0,4194304] 0 2024-04-02T04:42:39.859 INFO:tasks.workunit.client.0.smithi016.stdout:1/420: mkdir d14/d89 0 2024-04-02T04:42:39.862 INFO:tasks.workunit.client.1.smithi027.stdout:5/103: rename d0/df/d29/cd to d0/d1a/c2b 0 2024-04-02T04:42:39.863 INFO:tasks.workunit.client.0.smithi016.stdout:9/434: symlink d0/d4e/d50/d63/d6e/d4b/d58/l7d 0 2024-04-02T04:42:39.865 INFO:tasks.workunit.client.1.smithi027.stdout:5/104: creat d0/d5/f2c x:0 0 0 2024-04-02T04:42:39.866 INFO:tasks.workunit.client.1.smithi027.stdout:5/105: write d0/f24 [223247,45679] 0 2024-04-02T04:42:39.866 INFO:tasks.workunit.client.0.smithi016.stdout:1/421: dwrite d14/d19/d1d/d29/d39/d4d/f76 [0,4194304] 0 2024-04-02T04:42:39.866 INFO:tasks.workunit.client.0.smithi016.stdout:6/482: creat d0/d9/d16/d46/d5e/f8f x:0 0 0 2024-04-02T04:42:39.867 INFO:tasks.workunit.client.0.smithi016.stdout:0/402: mknod d0/d16/d2e/d37/d87/c91 0 2024-04-02T04:42:39.871 INFO:tasks.workunit.client.0.smithi016.stdout:7/412: sync 2024-04-02T04:42:39.871 INFO:tasks.workunit.client.0.smithi016.stdout:8/405: getdents d1/d6/d36/d4f/d5d 0 2024-04-02T04:42:39.872 INFO:tasks.workunit.client.0.smithi016.stdout:9/435: mkdir d0/d3/d31/d53/d7e 0 2024-04-02T04:42:39.874 INFO:tasks.workunit.client.1.smithi027.stdout:5/106: stat d0/d1a/d20/c2a 0 2024-04-02T04:42:39.874 INFO:tasks.workunit.client.0.smithi016.stdout:9/436: read d0/d33/f36 [1719418,69032] 0 2024-04-02T04:42:39.877 INFO:tasks.workunit.client.0.smithi016.stdout:4/445: getdents d5/d1d/d5f 0 2024-04-02T04:42:39.879 INFO:tasks.workunit.client.0.smithi016.stdout:6/483: creat d0/d9/d16/d56/d89/d67/d6f/f90 x:0 0 0 2024-04-02T04:42:39.879 INFO:tasks.workunit.client.0.smithi016.stdout:7/413: symlink d0/d46/d37/d47/l95 0 2024-04-02T04:42:39.880 INFO:tasks.workunit.client.0.smithi016.stdout:3/468: dread d2/d15/f4b [0,4194304] 0 2024-04-02T04:42:39.880 INFO:tasks.workunit.client.0.smithi016.stdout:1/422: dwrite d14/d19/d1d/f1f [0,4194304] 0 2024-04-02T04:42:39.882 INFO:tasks.workunit.client.1.smithi027.stdout:5/107: link d0/df/d11/d13/c15 d0/df/c2d 0 2024-04-02T04:42:39.886 INFO:tasks.workunit.client.0.smithi016.stdout:4/446: rmdir d5/d17 39 2024-04-02T04:42:39.888 INFO:tasks.workunit.client.0.smithi016.stdout:9/437: dwrite d0/d4e/d50/d63/d6e/f65 [0,4194304] 0 2024-04-02T04:42:39.890 INFO:tasks.workunit.client.1.smithi027.stdout:5/108: dwrite d0/df/d11/f1f [0,4194304] 0 2024-04-02T04:42:39.890 INFO:tasks.workunit.client.0.smithi016.stdout:0/403: dwrite d0/f32 [0,4194304] 0 2024-04-02T04:42:39.898 INFO:tasks.workunit.client.0.smithi016.stdout:6/484: mkdir d0/d9/d16/d56/d89/d67/d6f/d8a/d91 0 2024-04-02T04:42:39.898 INFO:tasks.workunit.client.1.smithi027.stdout:5/109: getdents d0/df 0 2024-04-02T04:42:39.901 INFO:tasks.workunit.client.0.smithi016.stdout:6/485: fsync d0/d9/d16/d56/d59/f86 0 2024-04-02T04:42:39.902 INFO:tasks.workunit.client.0.smithi016.stdout:8/406: dwrite d1/d6/d36/d3f/d40/d49/f33 [0,4194304] 0 2024-04-02T04:42:39.903 INFO:tasks.workunit.client.1.smithi027.stdout:5/110: creat d0/d5/f2e x:0 0 0 2024-04-02T04:42:39.905 INFO:tasks.workunit.client.0.smithi016.stdout:1/423: rmdir d14/d19/d1d/d29/d39/d46/d81 0 2024-04-02T04:42:39.907 INFO:tasks.workunit.client.0.smithi016.stdout:0/404: creat d0/d48/d83/d8c/f92 x:0 0 0 2024-04-02T04:42:39.908 INFO:tasks.workunit.client.0.smithi016.stdout:4/447: dwrite d5/d8/f7b [0,4194304] 0 2024-04-02T04:42:39.911 INFO:tasks.workunit.client.1.smithi027.stdout:5/111: dread d0/d5/f12 [0,4194304] 0 2024-04-02T04:42:39.914 INFO:tasks.workunit.client.0.smithi016.stdout:6/486: creat d0/d9/d16/d46/f92 x:0 0 0 2024-04-02T04:42:39.914 INFO:tasks.workunit.client.1.smithi027.stdout:5/112: dread d0/df/d29/fe [0,4194304] 0 2024-04-02T04:42:39.917 INFO:tasks.workunit.client.0.smithi016.stdout:0/405: unlink d0/d16/f18 0 2024-04-02T04:42:39.918 INFO:tasks.workunit.client.0.smithi016.stdout:8/407: dwrite d1/d6/d36/d3f/d40/d49/f68 [0,4194304] 0 2024-04-02T04:42:39.920 INFO:tasks.workunit.client.0.smithi016.stdout:4/448: unlink d5/f10 0 2024-04-02T04:42:39.921 INFO:tasks.workunit.client.0.smithi016.stdout:7/414: dread d0/d21/f65 [4194304,4194304] 0 2024-04-02T04:42:39.921 INFO:tasks.workunit.client.0.smithi016.stdout:4/449: dread - d5/d16/d3f/f64 zero size 2024-04-02T04:42:39.921 INFO:tasks.workunit.client.1.smithi027.stdout:5/113: mkdir d0/df/d2f 0 2024-04-02T04:42:39.923 INFO:tasks.workunit.client.0.smithi016.stdout:9/438: rename d0/d4e/d50/d63/d6e/d1a/l29 to d0/d4e/d50/d63/d6e/d1a/l7f 0 2024-04-02T04:42:39.924 INFO:tasks.workunit.client.0.smithi016.stdout:9/439: stat d0/d4e/d50/d63/d6e/d1a 0 2024-04-02T04:42:39.924 INFO:tasks.workunit.client.0.smithi016.stdout:9/440: read d0/f2c [1636771,42929] 0 2024-04-02T04:42:39.925 INFO:tasks.workunit.client.0.smithi016.stdout:9/441: chown d0/l55 1198166 1 2024-04-02T04:42:39.927 INFO:tasks.workunit.client.1.smithi027.stdout:5/114: unlink d0/df/d29/fe 0 2024-04-02T04:42:39.930 INFO:tasks.workunit.client.0.smithi016.stdout:1/424: truncate d14/f48 2628902 0 2024-04-02T04:42:39.931 INFO:tasks.workunit.client.1.smithi027.stdout:5/115: rename d0/df/c21 to d0/df/c30 0 2024-04-02T04:42:39.934 INFO:tasks.workunit.client.0.smithi016.stdout:0/406: mknod d0/d16/d24/d4e/d8d/d71/c93 0 2024-04-02T04:42:39.936 INFO:tasks.workunit.client.1.smithi027.stdout:5/116: creat d0/d1d/f31 x:0 0 0 2024-04-02T04:42:39.937 INFO:tasks.workunit.client.0.smithi016.stdout:4/450: dread - d5/d17/d18/d6d/f74 zero size 2024-04-02T04:42:39.938 INFO:tasks.workunit.client.0.smithi016.stdout:4/451: chown d5/d1d/f61 199 1 2024-04-02T04:42:39.938 INFO:tasks.workunit.client.0.smithi016.stdout:8/408: mknod d1/d6/d36/d3f/d40/d49/c89 0 2024-04-02T04:42:39.940 INFO:tasks.workunit.client.0.smithi016.stdout:8/409: truncate d1/d6/d36/d3f/f80 310537 0 2024-04-02T04:42:39.941 INFO:tasks.workunit.client.0.smithi016.stdout:7/415: mkdir d0/d46/d37/d3a/d43/d96 0 2024-04-02T04:42:39.941 INFO:tasks.workunit.client.0.smithi016.stdout:7/416: stat d0/d46/d37/d47/c6c 0 2024-04-02T04:42:39.942 INFO:tasks.workunit.client.0.smithi016.stdout:7/417: write d0/d28/d64/f8b [2122096,29397] 0 2024-04-02T04:42:39.942 INFO:tasks.workunit.client.1.smithi027.stdout:5/117: dwrite d0/d5/f2c [0,4194304] 0 2024-04-02T04:42:39.944 INFO:tasks.workunit.client.1.smithi027.stdout:5/118: chown d0/d1a/c2b 0 1 2024-04-02T04:42:39.945 INFO:tasks.workunit.client.0.smithi016.stdout:1/425: mkdir d14/d19/d1d/d45/d8a 0 2024-04-02T04:42:39.947 INFO:tasks.workunit.client.0.smithi016.stdout:0/407: truncate d0/f10 1883918 0 2024-04-02T04:42:39.949 INFO:tasks.workunit.client.0.smithi016.stdout:8/410: fsync d1/d6/d36/d3f/d40/d49/f44 0 2024-04-02T04:42:39.951 INFO:tasks.workunit.client.0.smithi016.stdout:9/442: symlink d0/d9/db/d54/l80 0 2024-04-02T04:42:39.953 INFO:tasks.workunit.client.0.smithi016.stdout:2/372: sync 2024-04-02T04:42:39.954 INFO:tasks.workunit.client.0.smithi016.stdout:7/418: readlink d0/d28/l2d 0 2024-04-02T04:42:39.954 INFO:tasks.workunit.client.0.smithi016.stdout:6/487: dwrite d0/d9/d16/d56/f2c [4194304,4194304] 0 2024-04-02T04:42:39.956 INFO:tasks.workunit.client.0.smithi016.stdout:4/452: creat d5/d16/d88/f8c x:0 0 0 2024-04-02T04:42:39.957 INFO:tasks.workunit.client.0.smithi016.stdout:6/488: fsync d0/d9/d16/d56/d59/f60 0 2024-04-02T04:42:39.958 INFO:tasks.workunit.client.0.smithi016.stdout:7/419: fsync d0/d46/d16/d49/f11 0 2024-04-02T04:42:39.958 INFO:tasks.workunit.client.0.smithi016.stdout:1/426: rename d14/d41/d43/d68/f74 to d14/d19/d1e/d6d/f8b 0 2024-04-02T04:42:39.963 INFO:tasks.workunit.client.0.smithi016.stdout:2/373: creat d12/d35/d3f/d56/f81 x:0 0 0 2024-04-02T04:42:39.965 INFO:tasks.workunit.client.0.smithi016.stdout:0/408: link d0/d16/f54 d0/d16/f94 0 2024-04-02T04:42:39.969 INFO:tasks.workunit.client.0.smithi016.stdout:4/453: chown d5/d17/d18/d1f/l46 9691598 1 2024-04-02T04:42:39.970 INFO:tasks.workunit.client.0.smithi016.stdout:4/454: stat d5/d1d/c5a 0 2024-04-02T04:42:39.972 INFO:tasks.workunit.client.0.smithi016.stdout:8/411: dread d1/d31/d67/f79 [0,4194304] 0 2024-04-02T04:42:39.972 INFO:tasks.workunit.client.0.smithi016.stdout:7/420: rmdir d0/d28 39 2024-04-02T04:42:39.974 INFO:tasks.workunit.client.0.smithi016.stdout:9/443: rename d0/d4e/d50/d63/d6e/d4b/d58/d6a/f71 to d0/d3/f81 0 2024-04-02T04:42:39.976 INFO:tasks.workunit.client.0.smithi016.stdout:2/374: mkdir d12/d35/d3b/d42/d67/d82 0 2024-04-02T04:42:39.977 INFO:tasks.workunit.client.0.smithi016.stdout:0/409: creat d0/d4f/f95 x:0 0 0 2024-04-02T04:42:39.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.974+0000 7f1579e53700 1 -- 172.21.15.16:0/1606188615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f157404ea80 con 0x7f1574072ac0 2024-04-02T04:42:39.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.974+0000 7f1570ff9700 1 -- 172.21.15.16:0/1606188615 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f156405b440 con 0x7f1574072ac0 2024-04-02T04:42:39.978 INFO:teuthology.orchestra.run.smithi016.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-02T04:42:39.978 INFO:teuthology.orchestra.run.smithi016.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-02T04:42:39.978 INFO:teuthology.orchestra.run.smithi016.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-02T04:42:39.978 INFO:tasks.workunit.client.0.smithi016.stdout:4/455: unlink d5/d8/d43/l4f 0 2024-04-02T04:42:39.979 INFO:tasks.workunit.client.0.smithi016.stdout:2/375: read d12/f23 [2287103,30223] 0 2024-04-02T04:42:39.979 INFO:tasks.workunit.client.0.smithi016.stdout:6/489: dwrite d0/f3f [4194304,4194304] 0 2024-04-02T04:42:39.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.978+0000 7f1579e53700 1 -- 172.21.15.16:0/1606188615 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7f155c06d4e0 msgr2=0x7f155c06f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:39.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.978+0000 7f1579e53700 1 --2- 172.21.15.16:0/1606188615 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7f155c06d4e0 0x7f155c06f9a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1574072fb0 tx=0x7f1568009190 comp rx=0 tx=0).stop 2024-04-02T04:42:39.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.978+0000 7f1579e53700 1 -- 172.21.15.16:0/1606188615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1574072ac0 msgr2=0x7f157419ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:42:39.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.978+0000 7f1579e53700 1 --2- 172.21.15.16:0/1606188615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1574072ac0 0x7f157419ee00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1564005090 tx=0x7f156400bd70 comp rx=0 tx=0).stop 2024-04-02T04:42:39.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.978+0000 7f1579e53700 1 -- 172.21.15.16:0/1606188615 shutdown_connections 2024-04-02T04:42:39.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.978+0000 7f1579e53700 1 --2- 172.21.15.16:0/1606188615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1574072ac0 0x7f157419ee00 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:39.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.978+0000 7f1579e53700 1 --2- 172.21.15.16:0/1606188615 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7f155c06d4e0 0x7f155c06f9a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:39.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.978+0000 7f1579e53700 1 --2- 172.21.15.16:0/1606188615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f157410b410 0x7f157419f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:42:39.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.978+0000 7f1579e53700 1 -- 172.21.15.16:0/1606188615 >> 172.21.15.16:0/1606188615 conn(0x7f157406da40 msgr2=0x7f157406e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:42:39.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.978+0000 7f1579e53700 1 -- 172.21.15.16:0/1606188615 shutdown_connections 2024-04-02T04:42:39.982 INFO:tasks.workunit.client.0.smithi016.stdout:1/427: mknod d14/d19/d1d/d29/c8c 0 2024-04-02T04:42:39.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:42:39.978+0000 7f1579e53700 1 -- 172.21.15.16:0/1606188615 wait complete. 2024-04-02T04:42:39.984 INFO:tasks.workunit.client.0.smithi016.stdout:0/410: creat d0/d16/d2e/f96 x:0 0 0 2024-04-02T04:42:39.985 INFO:tasks.workunit.client.0.smithi016.stdout:1/428: unlink d14/d19/c4e 0 2024-04-02T04:42:39.986 INFO:tasks.workunit.client.0.smithi016.stdout:7/421: write d0/d46/d37/d3a/d6f/f7b [516274,111747] 0 2024-04-02T04:42:39.987 INFO:tasks.workunit.client.0.smithi016.stdout:4/456: rmdir d5/d17/d18/d25 39 2024-04-02T04:42:39.989 INFO:tasks.workunit.client.0.smithi016.stdout:4/457: fdatasync d5/d16/f49 0 2024-04-02T04:42:39.991 INFO:tasks.workunit.client.0.smithi016.stdout:8/412: dread d1/d6/d36/d3f/d40/d49/f33 [0,4194304] 0 2024-04-02T04:42:39.991 INFO:tasks.workunit.client.0.smithi016.stdout:6/490: creat d0/d9/d16/d46/d5e/f93 x:0 0 0 2024-04-02T04:42:39.992 INFO:tasks.workunit.client.0.smithi016.stdout:1/429: write d14/d41/d43/d68/f7c [353494,14019] 0 2024-04-02T04:42:39.995 INFO:tasks.workunit.client.0.smithi016.stdout:7/422: symlink d0/d21/l97 0 2024-04-02T04:42:39.995 INFO:tasks.workunit.client.0.smithi016.stdout:9/444: dwrite d0/d4e/d50/d63/d6e/d1a/f37 [0,4194304] 0 2024-04-02T04:42:39.996 INFO:tasks.workunit.client.0.smithi016.stdout:1/430: stat d14/d19/d1e/d71/f78 0 2024-04-02T04:42:39.997 INFO:tasks.workunit.client.0.smithi016.stdout:6/491: fsync d0/d9/d16/d56/d59/f86 0 2024-04-02T04:42:39.998 INFO:tasks.workunit.client.0.smithi016.stdout:1/431: chown d14/d19/d1d/c36 91 1 2024-04-02T04:42:39.998 INFO:tasks.workunit.client.0.smithi016.stdout:7/423: mknod d0/c98 0 2024-04-02T04:42:40.001 INFO:tasks.workunit.client.0.smithi016.stdout:1/432: chown d14/d19/d1e/f5d 404 1 2024-04-02T04:42:40.002 INFO:tasks.workunit.client.0.smithi016.stdout:4/458: dread d5/d8/f7b [0,4194304] 0 2024-04-02T04:42:40.003 INFO:tasks.workunit.client.0.smithi016.stdout:6/492: mknod d0/d9/d16/d56/d59/c94 0 2024-04-02T04:42:40.005 INFO:tasks.workunit.client.0.smithi016.stdout:4/459: fdatasync d5/d8/f7b 0 2024-04-02T04:42:40.006 INFO:tasks.workunit.client.0.smithi016.stdout:2/376: dwrite d12/d35/d3f/f7e [0,4194304] 0 2024-04-02T04:42:40.006 INFO:tasks.workunit.client.0.smithi016.stdout:8/413: dread d1/d6/d36/d3f/d40/d49/f33 [0,4194304] 0 2024-04-02T04:42:40.008 INFO:tasks.workunit.client.0.smithi016.stdout:7/424: rename d0/d28/d64/f92 to d0/d46/d52/d82/f99 0 2024-04-02T04:42:40.008 INFO:tasks.workunit.client.0.smithi016.stdout:8/414: write d1/d6/d36/d3f/d40/d49/d4a/f50 [192959,90481] 0 2024-04-02T04:42:40.009 INFO:tasks.workunit.client.0.smithi016.stdout:6/493: mkdir d0/d9/d16/d56/d95 0 2024-04-02T04:42:40.009 INFO:tasks.workunit.client.0.smithi016.stdout:7/425: write d0/d28/f53 [1409810,71574] 0 2024-04-02T04:42:40.012 INFO:tasks.workunit.client.0.smithi016.stdout:4/460: creat d5/de/f8d x:0 0 0 2024-04-02T04:42:40.012 INFO:tasks.workunit.client.0.smithi016.stdout:2/377: creat d12/d35/d3b/f83 x:0 0 0 2024-04-02T04:42:40.014 INFO:tasks.workunit.client.0.smithi016.stdout:4/461: fsync d5/d8/d43/f45 0 2024-04-02T04:42:40.015 INFO:tasks.workunit.client.0.smithi016.stdout:1/433: rename d14/d19/d1d/d29/d39/d46/c64 to d14/d19/d1d/d45/c8d 0 2024-04-02T04:42:40.017 INFO:tasks.workunit.client.0.smithi016.stdout:7/426: unlink d0/d46/d37/d3a/d4c/f73 0 2024-04-02T04:42:40.020 INFO:tasks.workunit.client.0.smithi016.stdout:4/462: readlink d5/d17/d18/d1f/l46 0 2024-04-02T04:42:40.021 INFO:tasks.workunit.client.0.smithi016.stdout:8/415: dwrite d1/d6/d36/d4f/f62 [0,4194304] 0 2024-04-02T04:42:40.023 INFO:tasks.workunit.client.0.smithi016.stdout:4/463: dread - d5/d8/d43/f45 zero size 2024-04-02T04:42:40.026 INFO:tasks.workunit.client.0.smithi016.stdout:6/494: dwrite d0/d9/d16/d46/f92 [0,4194304] 0 2024-04-02T04:42:40.028 INFO:tasks.workunit.client.0.smithi016.stdout:1/434: dread d14/d41/d43/f5f [0,4194304] 0 2024-04-02T04:42:40.028 INFO:tasks.workunit.client.0.smithi016.stdout:7/427: creat d0/d46/f9a x:0 0 0 2024-04-02T04:42:40.029 INFO:tasks.workunit.client.0.smithi016.stdout:8/416: rmdir d1/d6/d36/d3f/d40/d49 39 2024-04-02T04:42:40.031 INFO:tasks.workunit.client.0.smithi016.stdout:2/378: dwrite d12/d20/d4f/f5f [0,4194304] 0 2024-04-02T04:42:40.032 INFO:tasks.workunit.client.0.smithi016.stdout:1/435: chown cc 2444667 1 2024-04-02T04:42:40.033 INFO:tasks.workunit.client.0.smithi016.stdout:7/428: chown d0/d46/f2c 1283176020 1 2024-04-02T04:42:40.034 INFO:tasks.workunit.client.0.smithi016.stdout:6/495: fsync d0/d9/d16/d56/d89/f42 0 2024-04-02T04:42:40.035 INFO:tasks.workunit.client.0.smithi016.stdout:4/464: write d5/d17/d76/d7e/f81 [985116,103401] 0 2024-04-02T04:42:40.035 INFO:tasks.workunit.client.0.smithi016.stdout:7/429: stat d0/d46/f30 0 2024-04-02T04:42:40.037 INFO:tasks.workunit.client.0.smithi016.stdout:4/465: readlink d5/d8/lc 0 2024-04-02T04:42:40.039 INFO:tasks.workunit.client.0.smithi016.stdout:2/379: symlink d12/d20/d76/l84 0 2024-04-02T04:42:40.041 INFO:tasks.workunit.client.0.smithi016.stdout:2/380: chown lb 0 1 2024-04-02T04:42:40.043 INFO:tasks.workunit.client.0.smithi016.stdout:8/417: rename d1/d70/d71/d74/f78 to d1/d6/d36/d3f/d77/f8a 0 2024-04-02T04:42:40.044 INFO:tasks.workunit.client.0.smithi016.stdout:7/430: mkdir d0/d46/d37/d3a/d43/d96/d9b 0 2024-04-02T04:42:40.047 INFO:tasks.workunit.client.0.smithi016.stdout:2/381: symlink d12/d26/l85 0 2024-04-02T04:42:40.047 INFO:tasks.workunit.client.0.smithi016.stdout:1/436: dwrite d14/d19/d1d/d29/d4c/f88 [0,4194304] 0 2024-04-02T04:42:40.048 INFO:tasks.workunit.client.0.smithi016.stdout:6/496: rename d0/d5c/f65 to d0/d9/d16/d56/d59/f96 0 2024-04-02T04:42:40.049 INFO:tasks.workunit.client.0.smithi016.stdout:4/466: dread d5/d8/ff [0,4194304] 0 2024-04-02T04:42:40.050 INFO:tasks.workunit.client.0.smithi016.stdout:2/382: read - d12/d35/d3f/f39 zero size 2024-04-02T04:42:40.050 INFO:tasks.workunit.client.0.smithi016.stdout:4/467: chown c3 120279089 1 2024-04-02T04:42:40.054 INFO:tasks.workunit.client.0.smithi016.stdout:2/383: chown d12/d14/c79 6798876 1 2024-04-02T04:42:40.056 INFO:tasks.workunit.client.0.smithi016.stdout:7/431: link d0/d21/f65 d0/f9c 0 2024-04-02T04:42:40.056 INFO:tasks.workunit.client.0.smithi016.stdout:8/418: dwrite d1/d31/f46 [0,4194304] 0 2024-04-02T04:42:40.057 INFO:tasks.workunit.client.0.smithi016.stdout:8/419: chown d1/d6/d36/c5f 27502732 1 2024-04-02T04:42:40.057 INFO:tasks.workunit.client.0.smithi016.stdout:4/468: mkdir d5/d17/d18/d6d/d8e 0 2024-04-02T04:42:40.058 INFO:tasks.workunit.client.0.smithi016.stdout:5/420: sync 2024-04-02T04:42:40.058 INFO:tasks.workunit.client.0.smithi016.stdout:8/420: write d1/f3 [400485,30464] 0 2024-04-02T04:42:40.059 INFO:tasks.workunit.client.0.smithi016.stdout:2/384: creat d12/d35/d3f/d56/f86 x:0 0 0 2024-04-02T04:42:40.060 INFO:tasks.workunit.client.0.smithi016.stdout:4/469: write d5/f67 [179366,44542] 0 2024-04-02T04:42:40.062 INFO:tasks.workunit.client.0.smithi016.stdout:2/385: fdatasync d12/d35/d3f/f61 0 2024-04-02T04:42:40.062 INFO:tasks.workunit.client.0.smithi016.stdout:4/470: fdatasync d5/d17/d18/d1f/f59 0 2024-04-02T04:42:40.063 INFO:tasks.workunit.client.0.smithi016.stdout:8/421: dread d1/d6/f57 [0,4194304] 0 2024-04-02T04:42:40.065 INFO:tasks.workunit.client.0.smithi016.stdout:1/437: dread d14/d19/d1e/f23 [0,4194304] 0 2024-04-02T04:42:40.065 INFO:tasks.workunit.client.0.smithi016.stdout:8/422: chown d1/d6/l48 22850916 1 2024-04-02T04:42:40.066 INFO:tasks.workunit.client.0.smithi016.stdout:4/471: creat d5/d8/f8f x:0 0 0 2024-04-02T04:42:40.067 INFO:tasks.workunit.client.0.smithi016.stdout:7/432: getdents d0/d21 0 2024-04-02T04:42:40.071 INFO:tasks.workunit.client.0.smithi016.stdout:8/423: readlink d1/d6/d36/d3f/d40/d45/l61 0 2024-04-02T04:42:40.075 INFO:tasks.workunit.client.0.smithi016.stdout:1/438: rename c0 to d14/d19/d50/c8e 0 2024-04-02T04:42:40.075 INFO:tasks.workunit.client.0.smithi016.stdout:5/421: dwrite d3/f5b [0,4194304] 0 2024-04-02T04:42:40.077 INFO:tasks.workunit.client.0.smithi016.stdout:8/424: symlink d1/d6/d36/d4f/l8b 0 2024-04-02T04:42:40.079 INFO:tasks.workunit.client.0.smithi016.stdout:2/386: dwrite d12/d35/d3b/f60 [0,4194304] 0 2024-04-02T04:42:40.079 INFO:tasks.workunit.client.0.smithi016.stdout:6/497: truncate d0/d9/d16/f47 3092895 0 2024-04-02T04:42:40.080 INFO:tasks.workunit.client.0.smithi016.stdout:5/422: chown d3/d40/d4f/d6b/l4d 56 1 2024-04-02T04:42:40.081 INFO:tasks.workunit.client.0.smithi016.stdout:4/472: dwrite d5/d17/d18/f7f [0,4194304] 0 2024-04-02T04:42:40.083 INFO:tasks.workunit.client.0.smithi016.stdout:7/433: mkdir d0/d46/d37/d3a/d6f/d77/d9d 0 2024-04-02T04:42:40.084 INFO:tasks.workunit.client.0.smithi016.stdout:1/439: fsync f2 0 2024-04-02T04:42:40.090 INFO:tasks.workunit.client.0.smithi016.stdout:4/473: dread d5/d1d/d1e/f3c [0,4194304] 0 2024-04-02T04:42:40.093 INFO:tasks.workunit.client.0.smithi016.stdout:2/387: rmdir d12 39 2024-04-02T04:42:40.099 INFO:tasks.workunit.client.0.smithi016.stdout:7/434: unlink d0/d28/l2d 0 2024-04-02T04:42:40.104 INFO:tasks.workunit.client.0.smithi016.stdout:4/474: creat d5/d17/d18/d1f/f90 x:0 0 0 2024-04-02T04:42:40.108 INFO:tasks.workunit.client.0.smithi016.stdout:7/435: mkdir d0/d46/d37/d3a/d4c/d9e 0 2024-04-02T04:42:40.111 INFO:tasks.workunit.client.0.smithi016.stdout:2/388: fsync d12/d35/d3f/f2d 0 2024-04-02T04:42:40.112 INFO:tasks.workunit.client.0.smithi016.stdout:5/423: getdents d3/df/d86 0 2024-04-02T04:42:40.113 INFO:tasks.workunit.client.0.smithi016.stdout:1/440: truncate d14/d19/d20/f3f 1477548 0 2024-04-02T04:42:40.113 INFO:tasks.workunit.client.0.smithi016.stdout:5/424: write d3/f10 [141074,97583] 0 2024-04-02T04:42:40.116 INFO:tasks.workunit.client.0.smithi016.stdout:4/475: dread d5/d8/d43/f4b [4194304,4194304] 0 2024-04-02T04:42:40.117 INFO:tasks.workunit.client.0.smithi016.stdout:2/389: symlink d12/d20/l87 0 2024-04-02T04:42:40.118 INFO:tasks.workunit.client.0.smithi016.stdout:1/441: symlink d14/d19/d50/l8f 0 2024-04-02T04:42:40.119 INFO:tasks.workunit.client.0.smithi016.stdout:1/442: write d14/d19/d20/f24 [2143288,91438] 0 2024-04-02T04:42:40.119 INFO:tasks.workunit.client.0.smithi016.stdout:5/425: creat d3/d40/d4f/d6b/d2a/d6d/fa1 x:0 0 0 2024-04-02T04:42:40.121 INFO:tasks.workunit.client.0.smithi016.stdout:6/498: getdents d0/d9/d16/d46/d5e 0 2024-04-02T04:42:40.122 INFO:tasks.workunit.client.0.smithi016.stdout:8/425: getdents d1/d6/d36/d3f/d40/d49/d4a/d4c 0 2024-04-02T04:42:40.123 INFO:tasks.workunit.client.0.smithi016.stdout:4/476: mkdir d5/d17/d18/d6d/d91 0 2024-04-02T04:42:40.124 INFO:tasks.workunit.client.0.smithi016.stdout:2/390: creat d12/d35/d3b/d42/d59/f88 x:0 0 0 2024-04-02T04:42:40.126 INFO:tasks.workunit.client.0.smithi016.stdout:5/426: creat d3/d40/d4f/d6b/fa2 x:0 0 0 2024-04-02T04:42:40.127 INFO:tasks.workunit.client.0.smithi016.stdout:6/499: rmdir d0/d9/d16/d56/d59 39 2024-04-02T04:42:40.127 INFO:tasks.workunit.client.0.smithi016.stdout:1/443: dwrite d14/d19/d1e/f5d [0,4194304] 0 2024-04-02T04:42:40.128 INFO:tasks.workunit.client.0.smithi016.stdout:8/426: rmdir d1/d6/d36/d3f 39 2024-04-02T04:42:40.131 INFO:tasks.workunit.client.0.smithi016.stdout:4/477: rmdir d5/d17/d76 39 2024-04-02T04:42:40.133 INFO:tasks.workunit.client.0.smithi016.stdout:2/391: mknod d12/d35/d3b/d42/c89 0 2024-04-02T04:42:40.136 INFO:tasks.workunit.client.0.smithi016.stdout:5/427: truncate d3/df/f14 304233 0 2024-04-02T04:42:40.137 INFO:tasks.workunit.client.0.smithi016.stdout:2/392: read d12/d70/f75 [2703722,129917] 0 2024-04-02T04:42:40.137 INFO:tasks.workunit.client.0.smithi016.stdout:1/444: mknod d14/d41/c90 0 2024-04-02T04:42:40.138 INFO:tasks.workunit.client.0.smithi016.stdout:4/478: dwrite d5/d16/d3f/f64 [0,4194304] 0 2024-04-02T04:42:40.139 INFO:tasks.workunit.client.0.smithi016.stdout:6/500: mknod d0/d9/d16/d56/d89/d67/d6f/c97 0 2024-04-02T04:42:40.142 INFO:tasks.workunit.client.0.smithi016.stdout:5/428: unlink d3/f7f 0 2024-04-02T04:42:40.145 INFO:tasks.workunit.client.0.smithi016.stdout:1/445: mknod d14/d19/d1e/d6d/c91 0 2024-04-02T04:42:40.145 INFO:tasks.workunit.client.0.smithi016.stdout:8/427: symlink d1/d6/d36/d3f/d40/d49/l8c 0 2024-04-02T04:42:40.146 INFO:tasks.workunit.client.0.smithi016.stdout:8/428: stat d1/d70/d71 0 2024-04-02T04:42:40.147 INFO:tasks.workunit.client.0.smithi016.stdout:8/429: write d1/d6/d36/d3f/d40/d49/f2b [2111583,55978] 0 2024-04-02T04:42:40.148 INFO:tasks.workunit.client.0.smithi016.stdout:2/393: rename d12/l30 to d12/d35/d3b/d42/d59/l8a 0 2024-04-02T04:42:40.152 INFO:tasks.workunit.client.0.smithi016.stdout:5/429: creat d3/df/d18/d26/d2c/fa3 x:0 0 0 2024-04-02T04:42:40.154 INFO:tasks.workunit.client.0.smithi016.stdout:1/446: creat d14/d19/d1d/d45/f92 x:0 0 0 2024-04-02T04:42:40.154 INFO:tasks.workunit.client.0.smithi016.stdout:1/447: stat d14/d19/f6b 0 2024-04-02T04:42:40.156 INFO:tasks.workunit.client.0.smithi016.stdout:5/430: dread d3/df/d18/d2f/f62 [0,4194304] 0 2024-04-02T04:42:40.158 INFO:tasks.workunit.client.0.smithi016.stdout:5/431: dread - d3/d40/d4f/d6b/d2a/d6d/fa1 zero size 2024-04-02T04:42:40.158 INFO:tasks.workunit.client.0.smithi016.stdout:2/394: rename d12/d35/d3b/d42/d59/l74 to d12/d20/l8b 0 2024-04-02T04:42:40.160 INFO:tasks.workunit.client.0.smithi016.stdout:6/501: mknod d0/d9/d16/d56/d95/c98 0 2024-04-02T04:42:40.163 INFO:tasks.workunit.client.0.smithi016.stdout:4/479: truncate d5/d8/f7b 161910 0 2024-04-02T04:42:40.167 INFO:tasks.workunit.client.0.smithi016.stdout:1/448: creat d14/d41/f93 x:0 0 0 2024-04-02T04:42:40.168 INFO:tasks.workunit.client.0.smithi016.stdout:1/449: dread - d14/d19/d1d/f7d zero size 2024-04-02T04:42:40.169 INFO:tasks.workunit.client.0.smithi016.stdout:5/432: rmdir d3/d40/d4f/d67 39 2024-04-02T04:42:40.170 INFO:tasks.workunit.client.0.smithi016.stdout:2/395: symlink d12/d26/l8c 0 2024-04-02T04:42:40.171 INFO:tasks.workunit.client.0.smithi016.stdout:8/430: write d1/d6/d36/d3f/d40/d49/f44 [389965,71314] 0 2024-04-02T04:42:40.171 INFO:tasks.workunit.client.0.smithi016.stdout:2/396: write d12/d20/d4f/f68 [261946,1920] 0 2024-04-02T04:42:40.172 INFO:tasks.workunit.client.0.smithi016.stdout:6/502: symlink d0/d9/d16/d56/d89/l99 0 2024-04-02T04:42:40.173 INFO:tasks.workunit.client.0.smithi016.stdout:6/503: write d0/f51 [1793567,118821] 0 2024-04-02T04:42:40.177 INFO:tasks.workunit.client.0.smithi016.stdout:5/433: readlink d3/d40/d4f/d67/l87 0 2024-04-02T04:42:40.181 INFO:tasks.workunit.client.0.smithi016.stdout:8/431: symlink d1/d6/d36/d3f/d40/d49/d4a/d4c/l8d 0 2024-04-02T04:42:40.181 INFO:tasks.workunit.client.0.smithi016.stdout:6/504: dwrite d0/f41 [4194304,4194304] 0 2024-04-02T04:42:40.184 INFO:tasks.workunit.client.0.smithi016.stdout:4/480: rename d5/d8/l3a to d5/d17/d76/l92 0 2024-04-02T04:42:40.185 INFO:tasks.workunit.client.0.smithi016.stdout:1/450: mkdir d14/d19/d1d/d29/d94 0 2024-04-02T04:42:40.187 INFO:tasks.workunit.client.0.smithi016.stdout:5/434: creat d3/df/d18/d2f/fa4 x:0 0 0 2024-04-02T04:42:40.194 INFO:tasks.workunit.client.0.smithi016.stdout:6/505: dwrite d0/d9/d16/f8d [0,4194304] 0 2024-04-02T04:42:40.195 INFO:tasks.workunit.client.0.smithi016.stdout:5/435: write d3/df/d18/d26/d2c/f8d [1212399,91157] 0 2024-04-02T04:42:40.195 INFO:tasks.workunit.client.0.smithi016.stdout:4/481: creat d5/d1d/d5f/d65/d6f/f93 x:0 0 0 2024-04-02T04:42:40.197 INFO:tasks.workunit.client.0.smithi016.stdout:8/432: mkdir d1/d6/d36/d8e 0 2024-04-02T04:42:40.198 INFO:tasks.workunit.client.0.smithi016.stdout:2/397: dwrite d12/f43 [0,4194304] 0 2024-04-02T04:42:40.200 INFO:tasks.workunit.client.0.smithi016.stdout:5/436: chown d3/d40/d4f/d6b/d2a/f5f 3 1 2024-04-02T04:42:40.201 INFO:tasks.workunit.client.0.smithi016.stdout:8/433: mkdir d1/d31/d67/d8f 0 2024-04-02T04:42:40.202 INFO:tasks.workunit.client.0.smithi016.stdout:5/437: chown d3/d40/d4f/d6b/l4d 51737856 1 2024-04-02T04:42:40.202 INFO:tasks.workunit.client.0.smithi016.stdout:2/398: mkdir d12/d35/d3b/d42/d8d 0 2024-04-02T04:42:40.204 INFO:tasks.workunit.client.0.smithi016.stdout:3/469: sync 2024-04-02T04:42:40.205 INFO:tasks.workunit.client.0.smithi016.stdout:4/482: dread d5/f4e [4194304,4194304] 0 2024-04-02T04:42:40.208 INFO:tasks.workunit.client.0.smithi016.stdout:5/438: chown d3/df/c21 0 1 2024-04-02T04:42:40.210 INFO:tasks.workunit.client.0.smithi016.stdout:3/470: mkdir d2/d15/d16/d3d/d8b 0 2024-04-02T04:42:40.213 INFO:tasks.workunit.client.0.smithi016.stdout:2/399: truncate d12/d35/d3f/f61 877022 0 2024-04-02T04:42:40.214 INFO:tasks.workunit.client.0.smithi016.stdout:5/439: creat d3/df/d18/d26/d2c/fa5 x:0 0 0 2024-04-02T04:42:40.215 INFO:tasks.workunit.client.0.smithi016.stdout:8/434: dwrite d1/d6/f1a [0,4194304] 0 2024-04-02T04:42:40.216 INFO:tasks.workunit.client.0.smithi016.stdout:6/506: dread d0/d9/d16/d56/f11 [0,4194304] 0 2024-04-02T04:42:40.221 INFO:tasks.workunit.client.0.smithi016.stdout:4/483: truncate d5/d17/d18/d1f/f2a 143402 0 2024-04-02T04:42:40.223 INFO:tasks.workunit.client.0.smithi016.stdout:8/435: creat d1/d6/d36/d3f/d40/d49/d38/f90 x:0 0 0 2024-04-02T04:42:40.224 INFO:tasks.workunit.client.0.smithi016.stdout:6/507: fdatasync d0/d9/d16/d56/d59/f96 0 2024-04-02T04:42:40.226 INFO:tasks.workunit.client.0.smithi016.stdout:4/484: symlink d5/d16/l94 0 2024-04-02T04:42:40.226 INFO:tasks.workunit.client.0.smithi016.stdout:6/508: chown d0/d9/d16/d56/d89/f45 30 1 2024-04-02T04:42:40.228 INFO:tasks.workunit.client.0.smithi016.stdout:5/440: getdents d3/df 0 2024-04-02T04:42:40.228 INFO:tasks.workunit.client.0.smithi016.stdout:8/436: fdatasync d1/d6/d36/d3f/d40/d49/d4a/f56 0 2024-04-02T04:42:40.228 INFO:tasks.workunit.client.0.smithi016.stdout:6/509: chown d0/d9/f7a 3 1 2024-04-02T04:42:40.230 INFO:tasks.workunit.client.0.smithi016.stdout:5/441: stat d3/d40/d4f/d6b/l2e 0 2024-04-02T04:42:40.231 INFO:tasks.workunit.client.0.smithi016.stdout:8/437: mknod d1/d6/d36/d3f/d40/d63/c91 0 2024-04-02T04:42:40.232 INFO:tasks.workunit.client.0.smithi016.stdout:5/442: dread d3/f10 [0,4194304] 0 2024-04-02T04:42:40.233 INFO:tasks.workunit.client.0.smithi016.stdout:8/438: write d1/d6/f4b [22201,48063] 0 2024-04-02T04:42:40.234 INFO:tasks.workunit.client.0.smithi016.stdout:5/443: rmdir d3/df 39 2024-04-02T04:42:40.234 INFO:tasks.workunit.client.0.smithi016.stdout:2/400: dread f8 [4194304,4194304] 0 2024-04-02T04:42:40.236 INFO:tasks.workunit.client.0.smithi016.stdout:8/439: unlink d1/d6/d36/d3f/d40/d49/c2f 0 2024-04-02T04:42:40.240 INFO:tasks.workunit.client.0.smithi016.stdout:8/440: rmdir d1/d6/d36/d3f/d40/d49/d4a 39 2024-04-02T04:42:40.241 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:40 smithi016 ceph-mon[28969]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:42:40.241 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:40 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/1606188615' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:42:40.243 INFO:tasks.workunit.client.0.smithi016.stdout:8/441: mknod d1/d6/d36/d3f/d77/c92 0 2024-04-02T04:42:40.246 INFO:tasks.workunit.client.0.smithi016.stdout:8/442: symlink d1/d6/d36/d3f/d40/d49/d38/l93 0 2024-04-02T04:42:40.246 INFO:tasks.workunit.client.0.smithi016.stdout:4/485: dread d5/d17/d18/f2b [0,4194304] 0 2024-04-02T04:42:40.247 INFO:tasks.workunit.client.0.smithi016.stdout:8/443: chown d1/d70/l7c 3641512 1 2024-04-02T04:42:40.248 INFO:tasks.workunit.client.0.smithi016.stdout:4/486: mkdir d5/d1d/d5f/d95 0 2024-04-02T04:42:40.251 INFO:tasks.workunit.client.0.smithi016.stdout:4/487: read d5/d1d/f78 [228337,68470] 0 2024-04-02T04:42:40.251 INFO:tasks.workunit.client.0.smithi016.stdout:2/401: truncate d12/d14/f1e 227196 0 2024-04-02T04:42:40.255 INFO:tasks.workunit.client.0.smithi016.stdout:8/444: dwrite d1/d6/f53 [0,4194304] 0 2024-04-02T04:42:40.255 INFO:tasks.workunit.client.0.smithi016.stdout:6/510: dread d0/d9/d16/f5b [0,4194304] 0 2024-04-02T04:42:40.259 INFO:tasks.workunit.client.0.smithi016.stdout:5/444: dwrite d3/f36 [0,4194304] 0 2024-04-02T04:42:40.261 INFO:tasks.workunit.client.0.smithi016.stdout:4/488: chown d5/d1d/d5f/d65/d6f/f93 105048 1 2024-04-02T04:42:40.268 INFO:tasks.workunit.client.0.smithi016.stdout:5/445: fsync d3/df/d18/d2f/f62 0 2024-04-02T04:42:40.269 INFO:tasks.workunit.client.0.smithi016.stdout:4/489: write d5/d17/d18/d1f/f70 [575520,69526] 0 2024-04-02T04:42:40.270 INFO:tasks.workunit.client.0.smithi016.stdout:8/445: dread d1/d6/f21 [0,4194304] 0 2024-04-02T04:42:40.271 INFO:tasks.workunit.client.0.smithi016.stdout:2/402: dwrite d12/d35/d3b/d42/d59/f88 [0,4194304] 0 2024-04-02T04:42:40.274 INFO:tasks.workunit.client.0.smithi016.stdout:8/446: dread - d1/d31/f7e zero size 2024-04-02T04:42:40.275 INFO:tasks.workunit.client.0.smithi016.stdout:2/403: write d12/f4c [4491336,30104] 0 2024-04-02T04:42:40.276 INFO:tasks.workunit.client.0.smithi016.stdout:5/446: mkdir d3/d40/d4f/d6b/d45/da6 0 2024-04-02T04:42:40.277 INFO:tasks.workunit.client.0.smithi016.stdout:6/511: dwrite d0/d9/df/f4d [0,4194304] 0 2024-04-02T04:42:40.279 INFO:tasks.workunit.client.0.smithi016.stdout:2/404: symlink d12/d20/d76/l8e 0 2024-04-02T04:42:40.280 INFO:tasks.workunit.client.0.smithi016.stdout:5/447: write d3/df/d18/d26/d48/f61 [912865,68612] 0 2024-04-02T04:42:40.281 INFO:tasks.workunit.client.0.smithi016.stdout:8/447: readlink d1/d6/d36/d4f/d5d/l73 0 2024-04-02T04:42:40.282 INFO:tasks.workunit.client.0.smithi016.stdout:5/448: write d3/df/d18/f38 [6071726,86198] 0 2024-04-02T04:42:40.282 INFO:tasks.workunit.client.0.smithi016.stdout:5/449: readlink d3/df/l27 0 2024-04-02T04:42:40.286 INFO:tasks.workunit.client.0.smithi016.stdout:5/450: truncate d3/d5d/f6f 1251961 0 2024-04-02T04:42:40.286 INFO:tasks.workunit.client.0.smithi016.stdout:2/405: symlink d12/d35/d3b/d42/d8d/l8f 0 2024-04-02T04:42:40.288 INFO:tasks.workunit.client.0.smithi016.stdout:5/451: dread - d3/d40/d4f/d6b/f9e zero size 2024-04-02T04:42:40.300 INFO:tasks.workunit.client.0.smithi016.stdout:5/452: dread d3/df/d18/f31 [0,4194304] 0 2024-04-02T04:42:40.301 INFO:tasks.workunit.client.0.smithi016.stdout:8/448: dread d1/f1c [0,4194304] 0 2024-04-02T04:42:40.301 INFO:tasks.workunit.client.0.smithi016.stdout:4/490: dwrite d5/d1d/f78 [0,4194304] 0 2024-04-02T04:42:40.303 INFO:tasks.workunit.client.0.smithi016.stdout:6/512: dwrite d0/d9/d16/d56/d89/d67/f6a [4194304,4194304] 0 2024-04-02T04:42:40.304 INFO:tasks.workunit.client.0.smithi016.stdout:2/406: dwrite f9 [0,4194304] 0 2024-04-02T04:42:40.304 INFO:tasks.workunit.client.0.smithi016.stdout:8/449: fdatasync d1/d70/f88 0 2024-04-02T04:42:40.305 INFO:tasks.workunit.client.0.smithi016.stdout:5/453: write d3/df/d18/f83 [288527,60913] 0 2024-04-02T04:42:40.313 INFO:tasks.workunit.client.0.smithi016.stdout:2/407: truncate d12/f2a 433473 0 2024-04-02T04:42:40.316 INFO:tasks.workunit.client.0.smithi016.stdout:4/491: dwrite d5/d16/d3f/f64 [0,4194304] 0 2024-04-02T04:42:40.318 INFO:tasks.workunit.client.0.smithi016.stdout:8/450: write d1/d70/f84 [366076,106637] 0 2024-04-02T04:42:40.318 INFO:tasks.workunit.client.0.smithi016.stdout:5/454: dwrite d3/df/d18/d26/f7a [0,4194304] 0 2024-04-02T04:42:40.325 INFO:tasks.workunit.client.0.smithi016.stdout:2/408: rename d12/d35/d3b/l41 to d12/d35/d3b/d42/d8d/l90 0 2024-04-02T04:42:40.327 INFO:tasks.workunit.client.0.smithi016.stdout:4/492: creat d5/d16/d88/f96 x:0 0 0 2024-04-02T04:42:40.330 INFO:tasks.workunit.client.0.smithi016.stdout:8/451: creat d1/d70/d71/f94 x:0 0 0 2024-04-02T04:42:40.330 INFO:tasks.workunit.client.0.smithi016.stdout:6/513: write d0/d9/d16/d56/d89/f2d [5826796,6412] 0 2024-04-02T04:42:40.332 INFO:tasks.workunit.client.0.smithi016.stdout:5/455: rename d3/df/d18/f38 to d3/d40/d4f/d6b/d2a/d6d/fa7 0 2024-04-02T04:42:40.334 INFO:tasks.workunit.client.0.smithi016.stdout:4/493: symlink d5/d17/d76/l97 0 2024-04-02T04:42:40.335 INFO:tasks.workunit.client.0.smithi016.stdout:4/494: dread - d5/f60 zero size 2024-04-02T04:42:40.336 INFO:tasks.workunit.client.0.smithi016.stdout:2/409: dwrite d12/d35/d3f/d56/f86 [0,4194304] 0 2024-04-02T04:42:40.339 INFO:tasks.workunit.client.0.smithi016.stdout:2/410: write d12/f23 [4168873,124852] 0 2024-04-02T04:42:40.342 INFO:tasks.workunit.client.0.smithi016.stdout:4/495: write d5/d17/d18/d1f/f90 [910060,110797] 0 2024-04-02T04:42:40.343 INFO:tasks.workunit.client.0.smithi016.stdout:2/411: dread - d12/d35/d3f/d56/f81 zero size 2024-04-02T04:42:40.345 INFO:tasks.workunit.client.0.smithi016.stdout:4/496: readlink d5/d17/d18/d1f/l3e 0 2024-04-02T04:42:40.345 INFO:tasks.workunit.client.0.smithi016.stdout:6/514: dwrite d0/d9/d16/f23 [0,4194304] 0 2024-04-02T04:42:40.350 INFO:tasks.workunit.client.0.smithi016.stdout:8/452: dwrite d1/d6/d36/d3f/d40/d49/d4a/f50 [0,4194304] 0 2024-04-02T04:42:40.350 INFO:tasks.workunit.client.0.smithi016.stdout:0/411: sync 2024-04-02T04:42:40.350 INFO:tasks.workunit.client.0.smithi016.stdout:7/436: sync 2024-04-02T04:42:40.350 INFO:tasks.workunit.client.0.smithi016.stdout:9/445: sync 2024-04-02T04:42:40.353 INFO:tasks.workunit.client.0.smithi016.stdout:4/497: chown d5/d17/d18/d6d/f74 245 1 2024-04-02T04:42:40.354 INFO:tasks.workunit.client.0.smithi016.stdout:0/412: readlink d0/d4f/d76/l89 0 2024-04-02T04:42:40.354 INFO:tasks.workunit.client.0.smithi016.stdout:4/498: chown d5/d17/d18 11677 1 2024-04-02T04:42:40.355 INFO:tasks.workunit.client.0.smithi016.stdout:2/412: truncate d12/d35/d3f/d2b/f3d 379047 0 2024-04-02T04:42:40.356 INFO:tasks.workunit.client.0.smithi016.stdout:0/413: fsync d0/d16/f80 0 2024-04-02T04:42:40.357 INFO:tasks.workunit.client.0.smithi016.stdout:5/456: dwrite d3/df/f57 [0,4194304] 0 2024-04-02T04:42:40.359 INFO:tasks.workunit.client.0.smithi016.stdout:5/457: stat d3/d40/d72/f7d 0 2024-04-02T04:42:40.360 INFO:tasks.workunit.client.0.smithi016.stdout:4/499: creat d5/d17/d18/d1f/f98 x:0 0 0 2024-04-02T04:42:40.364 INFO:tasks.workunit.client.0.smithi016.stdout:4/500: dread - d5/de/f79 zero size 2024-04-02T04:42:40.365 INFO:tasks.workunit.client.0.smithi016.stdout:2/413: mkdir d12/d35/d3b/d42/d59/d91 0 2024-04-02T04:42:40.366 INFO:tasks.workunit.client.0.smithi016.stdout:4/501: chown d5/d17/d18/d1f/l73 785 1 2024-04-02T04:42:40.367 INFO:tasks.workunit.client.0.smithi016.stdout:2/414: write d12/f7b [4387982,107904] 0 2024-04-02T04:42:40.368 INFO:tasks.workunit.client.0.smithi016.stdout:0/414: dwrite d0/d16/d24/d4e/d8d/f75 [0,4194304] 0 2024-04-02T04:42:40.369 INFO:tasks.workunit.client.0.smithi016.stdout:6/515: dread d0/d9/d16/f20 [0,4194304] 0 2024-04-02T04:42:40.371 INFO:tasks.workunit.client.0.smithi016.stdout:9/446: link d0/d3/d31/f45 d0/d9/d4d/f82 0 2024-04-02T04:42:40.372 INFO:tasks.workunit.client.0.smithi016.stdout:6/516: write d0/d9/d16/d46/f68 [799800,82166] 0 2024-04-02T04:42:40.375 INFO:tasks.workunit.client.0.smithi016.stdout:2/415: creat d12/d20/d76/f92 x:0 0 0 2024-04-02T04:42:40.377 INFO:tasks.workunit.client.0.smithi016.stdout:7/437: getdents d0/d46/d16 0 2024-04-02T04:42:40.380 INFO:tasks.workunit.client.0.smithi016.stdout:2/416: creat d12/d35/d3f/f93 x:0 0 0 2024-04-02T04:42:40.380 INFO:tasks.workunit.client.0.smithi016.stdout:6/517: mknod d0/d9/d16/d46/d5e/c9a 0 2024-04-02T04:42:40.381 INFO:tasks.workunit.client.0.smithi016.stdout:7/438: rmdir d0 39 2024-04-02T04:42:40.382 INFO:tasks.workunit.client.0.smithi016.stdout:4/502: link d5/d17/d18/d1f/l46 d5/d1d/d69/l99 0 2024-04-02T04:42:40.386 INFO:tasks.workunit.client.0.smithi016.stdout:7/439: stat d0/d28/l29 0 2024-04-02T04:42:40.387 INFO:tasks.workunit.client.0.smithi016.stdout:9/447: write d0/d9/db/f43 [4890242,4707] 0 2024-04-02T04:42:40.388 INFO:tasks.workunit.client.0.smithi016.stdout:9/448: stat d0/f2d 0 2024-04-02T04:42:40.388 INFO:tasks.workunit.client.0.smithi016.stdout:4/503: dread d5/d16/d3f/f54 [0,4194304] 0 2024-04-02T04:42:40.390 INFO:tasks.workunit.client.0.smithi016.stdout:9/449: rmdir d0/d9 39 2024-04-02T04:42:40.391 INFO:tasks.workunit.client.0.smithi016.stdout:4/504: creat d5/d1d/d1e/f9a x:0 0 0 2024-04-02T04:42:40.393 INFO:tasks.workunit.client.0.smithi016.stdout:6/518: getdents d0/d9 0 2024-04-02T04:42:40.394 INFO:tasks.workunit.client.0.smithi016.stdout:9/450: symlink d0/d4e/l83 0 2024-04-02T04:42:40.395 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:40 smithi027 ceph-mon[37550]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:42:40.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:40 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/1606188615' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:42:40.396 INFO:tasks.workunit.client.0.smithi016.stdout:4/505: symlink d5/d1d/d6c/l9b 0 2024-04-02T04:42:40.396 INFO:tasks.workunit.client.0.smithi016.stdout:5/458: dread d3/d40/d4f/d67/f78 [0,4194304] 0 2024-04-02T04:42:40.397 INFO:tasks.workunit.client.0.smithi016.stdout:2/417: truncate d12/d35/d3b/d42/d59/f88 157319 0 2024-04-02T04:42:40.398 INFO:tasks.workunit.client.0.smithi016.stdout:9/451: fsync d0/d33/f6d 0 2024-04-02T04:42:40.399 INFO:tasks.workunit.client.0.smithi016.stdout:4/506: fdatasync d5/f4e 0 2024-04-02T04:42:40.399 INFO:tasks.workunit.client.0.smithi016.stdout:4/507: readlink d5/d17/d76/l97 0 2024-04-02T04:42:40.401 INFO:tasks.workunit.client.0.smithi016.stdout:4/508: truncate d5/de/f79 191011 0 2024-04-02T04:42:40.402 INFO:tasks.workunit.client.0.smithi016.stdout:4/509: dread d5/de/f79 [0,4194304] 0 2024-04-02T04:42:40.403 INFO:tasks.workunit.client.0.smithi016.stdout:7/440: dread d0/f9c [0,4194304] 0 2024-04-02T04:42:40.403 INFO:tasks.workunit.client.0.smithi016.stdout:6/519: creat d0/f9b x:0 0 0 2024-04-02T04:42:40.403 INFO:tasks.workunit.client.0.smithi016.stdout:2/418: truncate d12/d35/f40 943957 0 2024-04-02T04:42:40.408 INFO:tasks.workunit.client.0.smithi016.stdout:6/520: rename d0/d9/d16/d46/d5e/f64 to d0/d9/f9c 0 2024-04-02T04:42:40.410 INFO:tasks.workunit.client.0.smithi016.stdout:7/441: mknod d0/d46/d37/d3a/d4c/c9f 0 2024-04-02T04:42:40.412 INFO:tasks.workunit.client.0.smithi016.stdout:9/452: dread d0/d9/f14 [0,4194304] 0 2024-04-02T04:42:40.414 INFO:tasks.workunit.client.0.smithi016.stdout:9/453: write d0/f27 [47732,73617] 0 2024-04-02T04:42:40.414 INFO:tasks.workunit.client.0.smithi016.stdout:7/442: dread d0/d46/d16/f3d [0,4194304] 0 2024-04-02T04:42:40.419 INFO:tasks.workunit.client.0.smithi016.stdout:2/419: dread d12/d35/d3f/f7a [0,4194304] 0 2024-04-02T04:42:40.420 INFO:tasks.workunit.client.0.smithi016.stdout:7/443: write d0/d46/d52/d82/f99 [1018927,111185] 0 2024-04-02T04:42:40.420 INFO:tasks.workunit.client.0.smithi016.stdout:2/420: chown d12/d35/d3b/f60 590 1 2024-04-02T04:42:40.423 INFO:tasks.workunit.client.0.smithi016.stdout:6/521: symlink d0/d9/d16/d56/l9d 0 2024-04-02T04:42:40.425 INFO:tasks.workunit.client.0.smithi016.stdout:7/444: dread d0/d46/d16/f34 [0,4194304] 0 2024-04-02T04:42:40.430 INFO:tasks.workunit.client.0.smithi016.stdout:2/421: dwrite d12/d35/d3b/f57 [0,4194304] 0 2024-04-02T04:42:40.430 INFO:tasks.workunit.client.0.smithi016.stdout:7/445: creat d0/d46/d37/d3a/d4c/fa0 x:0 0 0 2024-04-02T04:42:40.430 INFO:tasks.workunit.client.0.smithi016.stdout:6/522: creat d0/d9/d16/d56/d89/d67/d6f/d8a/d91/f9e x:0 0 0 2024-04-02T04:42:40.431 INFO:tasks.workunit.client.0.smithi016.stdout:2/422: dread - d12/d35/d3f/f93 zero size 2024-04-02T04:42:40.431 INFO:tasks.workunit.client.0.smithi016.stdout:9/454: dwrite d0/d33/f36 [0,4194304] 0 2024-04-02T04:42:40.432 INFO:tasks.workunit.client.0.smithi016.stdout:7/446: symlink d0/d46/d16/d83/la1 0 2024-04-02T04:42:40.434 INFO:tasks.workunit.client.0.smithi016.stdout:6/523: symlink d0/d9/d16/l9f 0 2024-04-02T04:42:40.435 INFO:tasks.workunit.client.0.smithi016.stdout:2/423: unlink d12/d26/f72 0 2024-04-02T04:42:40.436 INFO:tasks.workunit.client.0.smithi016.stdout:6/524: write d0/d9/d16/d56/d89/d67/d6f/f7f [167247,66369] 0 2024-04-02T04:42:40.437 INFO:tasks.workunit.client.0.smithi016.stdout:2/424: truncate d12/d20/f44 4440246 0 2024-04-02T04:42:40.438 INFO:tasks.workunit.client.0.smithi016.stdout:6/525: truncate d0/f51 2703868 0 2024-04-02T04:42:40.439 INFO:tasks.workunit.client.0.smithi016.stdout:2/425: fsync d12/d20/d4f/f68 0 2024-04-02T04:42:40.442 INFO:tasks.workunit.client.0.smithi016.stdout:6/526: creat d0/d78/fa0 x:0 0 0 2024-04-02T04:42:40.444 INFO:tasks.workunit.client.0.smithi016.stdout:6/527: chown d0/d9/d16/d56/d59/f4b 12688 1 2024-04-02T04:42:40.444 INFO:tasks.workunit.client.0.smithi016.stdout:2/426: creat d12/d35/d3f/d56/f94 x:0 0 0 2024-04-02T04:42:40.444 INFO:tasks.workunit.client.0.smithi016.stdout:6/528: fsync d0/d6e/f81 0 2024-04-02T04:42:40.446 INFO:tasks.workunit.client.0.smithi016.stdout:2/427: write d12/d35/d3f/d56/d66/f71 [708844,83415] 0 2024-04-02T04:42:40.447 INFO:tasks.workunit.client.0.smithi016.stdout:6/529: write d0/d9/d16/d56/d59/f96 [351375,48220] 0 2024-04-02T04:42:40.447 INFO:tasks.workunit.client.0.smithi016.stdout:9/455: link d0/d4e/d50/d63/d6e/d4b/d58/l72 d0/d4e/d50/d67/l84 0 2024-04-02T04:42:40.449 INFO:tasks.workunit.client.0.smithi016.stdout:6/530: stat d0/d9/f9c 0 2024-04-02T04:42:40.449 INFO:tasks.workunit.client.0.smithi016.stdout:2/428: chown d12/d26/f29 2039425 1 2024-04-02T04:42:40.450 INFO:tasks.workunit.client.0.smithi016.stdout:9/456: creat d0/d4e/d50/f85 x:0 0 0 2024-04-02T04:42:40.450 INFO:tasks.workunit.client.0.smithi016.stdout:7/447: dread d0/d46/f4d [0,4194304] 0 2024-04-02T04:42:40.451 INFO:tasks.workunit.client.0.smithi016.stdout:6/531: chown d0/d9/c62 100526 1 2024-04-02T04:42:40.451 INFO:tasks.workunit.client.0.smithi016.stdout:2/429: write d12/d35/d3f/d56/f94 [863562,64651] 0 2024-04-02T04:42:40.452 INFO:tasks.workunit.client.0.smithi016.stdout:7/448: fsync d0/d21/f65 0 2024-04-02T04:42:40.454 INFO:tasks.workunit.client.0.smithi016.stdout:9/457: rename d0/d33/l6c to d0/d4e/d50/d67/l86 0 2024-04-02T04:42:40.456 INFO:tasks.workunit.client.0.smithi016.stdout:2/430: creat d12/d35/d3b/d42/f95 x:0 0 0 2024-04-02T04:42:40.457 INFO:tasks.workunit.client.0.smithi016.stdout:7/449: rmdir d0/d46/d16/d49 39 2024-04-02T04:42:40.459 INFO:tasks.workunit.client.0.smithi016.stdout:7/450: mkdir d0/d46/da2 0 2024-04-02T04:42:40.460 INFO:tasks.workunit.client.0.smithi016.stdout:2/431: creat d12/f96 x:0 0 0 2024-04-02T04:42:40.460 INFO:tasks.workunit.client.0.smithi016.stdout:9/458: getdents d0/d3/d31 0 2024-04-02T04:42:40.462 INFO:tasks.workunit.client.0.smithi016.stdout:2/432: write d12/d20/d4f/f77 [129390,115451] 0 2024-04-02T04:42:40.464 INFO:tasks.workunit.client.0.smithi016.stdout:9/459: unlink d0/d33/l3b 0 2024-04-02T04:42:40.467 INFO:tasks.workunit.client.0.smithi016.stdout:2/433: dread d12/d35/d3b/f57 [0,4194304] 0 2024-04-02T04:42:40.468 INFO:tasks.workunit.client.0.smithi016.stdout:9/460: truncate d0/d9/f14 494848 0 2024-04-02T04:42:40.472 INFO:tasks.workunit.client.0.smithi016.stdout:2/434: rename d12/d35/d3f/d56/f86 to d12/d35/d3f/d4d/f97 0 2024-04-02T04:42:40.474 INFO:tasks.workunit.client.0.smithi016.stdout:7/451: dread d0/d28/d64/f8b [0,4194304] 0 2024-04-02T04:42:40.475 INFO:tasks.workunit.client.0.smithi016.stdout:9/461: rename d0/d9/db/f1c to d0/d4e/d50/d63/d6e/d1a/d46/f87 0 2024-04-02T04:42:40.481 INFO:tasks.workunit.client.0.smithi016.stdout:9/462: chown d0/d4e/d50/d63/d6e/d4b/d58/l72 820 1 2024-04-02T04:42:40.481 INFO:tasks.workunit.client.0.smithi016.stdout:2/435: dwrite d12/d26/f48 [0,4194304] 0 2024-04-02T04:42:40.485 INFO:tasks.workunit.client.0.smithi016.stdout:7/452: dwrite d0/d46/d16/d83/f93 [0,4194304] 0 2024-04-02T04:42:40.485 INFO:tasks.workunit.client.0.smithi016.stdout:2/436: creat d12/d70/f98 x:0 0 0 2024-04-02T04:42:40.486 INFO:tasks.workunit.client.0.smithi016.stdout:9/463: write d0/d4e/d50/d63/d6e/f77 [1167111,6111] 0 2024-04-02T04:42:40.488 INFO:tasks.workunit.client.0.smithi016.stdout:2/437: fsync d12/d35/d3f/f4b 0 2024-04-02T04:42:40.489 INFO:tasks.workunit.client.0.smithi016.stdout:2/438: dread - d12/d35/d3b/f54 zero size 2024-04-02T04:42:40.491 INFO:tasks.workunit.client.0.smithi016.stdout:9/464: fdatasync d0/d33/f38 0 2024-04-02T04:42:40.493 INFO:tasks.workunit.client.0.smithi016.stdout:2/439: chown d12/d35/d3b/d42/d59/f88 29836574 1 2024-04-02T04:42:40.496 INFO:tasks.workunit.client.0.smithi016.stdout:2/440: dread f9 [0,4194304] 0 2024-04-02T04:42:40.496 INFO:tasks.workunit.client.0.smithi016.stdout:7/453: creat d0/d46/d37/d3a/d6f/d77/d9d/fa3 x:0 0 0 2024-04-02T04:42:40.497 INFO:tasks.workunit.client.0.smithi016.stdout:2/441: chown d12/f4c 920 1 2024-04-02T04:42:40.498 INFO:tasks.workunit.client.0.smithi016.stdout:9/465: mknod d0/d33/c88 0 2024-04-02T04:42:40.501 INFO:tasks.workunit.client.0.smithi016.stdout:7/454: symlink d0/d87/la4 0 2024-04-02T04:42:40.502 INFO:tasks.workunit.client.0.smithi016.stdout:2/442: mknod d12/d35/c99 0 2024-04-02T04:42:40.507 INFO:tasks.workunit.client.0.smithi016.stdout:9/466: creat d0/d9/f89 x:0 0 0 2024-04-02T04:42:40.507 INFO:tasks.workunit.client.0.smithi016.stdout:1/451: sync 2024-04-02T04:42:40.508 INFO:tasks.workunit.client.0.smithi016.stdout:2/443: mknod d12/c9a 0 2024-04-02T04:42:40.508 INFO:tasks.workunit.client.0.smithi016.stdout:1/452: chown c13 449902 1 2024-04-02T04:42:40.508 INFO:tasks.workunit.client.0.smithi016.stdout:7/455: truncate d0/d46/d37/d3a/d43/f5e 776199 0 2024-04-02T04:42:40.510 INFO:tasks.workunit.client.0.smithi016.stdout:1/453: symlink d14/d19/d1d/d45/d4f/l95 0 2024-04-02T04:42:40.511 INFO:tasks.workunit.client.0.smithi016.stdout:7/456: creat d0/d46/d37/d3a/d43/d96/fa5 x:0 0 0 2024-04-02T04:42:40.513 INFO:tasks.workunit.client.0.smithi016.stdout:9/467: rename d0/d4e/d50/d63/d6e/d4b/d58/l72 to d0/d4e/d50/d63/d6e/d4b/d58/l8a 0 2024-04-02T04:42:40.515 INFO:tasks.workunit.client.0.smithi016.stdout:7/457: symlink d0/d46/d37/d3a/la6 0 2024-04-02T04:42:40.516 INFO:tasks.workunit.client.0.smithi016.stdout:9/468: symlink d0/l8b 0 2024-04-02T04:42:40.518 INFO:tasks.workunit.client.0.smithi016.stdout:9/469: readlink d0/d4e/d50/d67/l84 0 2024-04-02T04:42:40.523 INFO:tasks.workunit.client.0.smithi016.stdout:9/470: mknod d0/d4e/d50/d67/c8c 0 2024-04-02T04:42:40.525 INFO:tasks.workunit.client.0.smithi016.stdout:7/458: dread d0/d46/fc [0,4194304] 0 2024-04-02T04:42:40.526 INFO:tasks.workunit.client.0.smithi016.stdout:9/471: fdatasync d0/d3/d31/f45 0 2024-04-02T04:42:40.528 INFO:tasks.workunit.client.0.smithi016.stdout:2/444: write d12/d35/d3f/f61 [1206451,45551] 0 2024-04-02T04:42:40.529 INFO:tasks.workunit.client.0.smithi016.stdout:7/459: dread d0/d21/f3c [0,4194304] 0 2024-04-02T04:42:40.533 INFO:tasks.workunit.client.0.smithi016.stdout:9/472: dwrite d0/d33/f36 [4194304,4194304] 0 2024-04-02T04:42:40.534 INFO:tasks.workunit.client.0.smithi016.stdout:9/473: write d0/f2c [4888866,60426] 0 2024-04-02T04:42:40.536 INFO:tasks.workunit.client.0.smithi016.stdout:7/460: creat d0/d46/d16/d49/fa7 x:0 0 0 2024-04-02T04:42:40.537 INFO:tasks.workunit.client.0.smithi016.stdout:7/461: dread - d0/d46/d37/d3a/d6f/d77/d9d/fa3 zero size 2024-04-02T04:42:40.537 INFO:tasks.workunit.client.0.smithi016.stdout:7/462: fdatasync d0/d21/f65 0 2024-04-02T04:42:40.542 INFO:tasks.workunit.client.0.smithi016.stdout:9/474: dwrite d0/d3/f81 [0,4194304] 0 2024-04-02T04:42:40.543 INFO:tasks.workunit.client.0.smithi016.stdout:7/463: creat d0/d46/d16/d83/fa8 x:0 0 0 2024-04-02T04:42:40.547 INFO:tasks.workunit.client.0.smithi016.stdout:9/475: dread - d0/d4e/d50/f85 zero size 2024-04-02T04:42:40.547 INFO:tasks.workunit.client.0.smithi016.stdout:9/476: fsync d0/d9/db/f43 0 2024-04-02T04:42:40.547 INFO:tasks.workunit.client.0.smithi016.stdout:9/477: write d0/f2c [5975903,76403] 0 2024-04-02T04:42:40.547 INFO:tasks.workunit.client.0.smithi016.stdout:9/478: chown d0/d3/d31/d53 94699 1 2024-04-02T04:42:40.548 INFO:tasks.workunit.client.0.smithi016.stdout:2/445: getdents d12/d35/d3f/d56 0 2024-04-02T04:42:40.558 INFO:tasks.workunit.client.0.smithi016.stdout:9/479: dread d0/f7 [0,4194304] 0 2024-04-02T04:42:40.561 INFO:tasks.workunit.client.0.smithi016.stdout:2/446: dread d12/d14/f3c [0,4194304] 0 2024-04-02T04:42:40.561 INFO:tasks.workunit.client.0.smithi016.stdout:7/464: mkdir d0/d46/d37/d3a/da9 0 2024-04-02T04:42:40.564 INFO:tasks.workunit.client.0.smithi016.stdout:2/447: mknod d12/d26/c9b 0 2024-04-02T04:42:40.566 INFO:tasks.workunit.client.0.smithi016.stdout:7/465: creat d0/d46/d37/faa x:0 0 0 2024-04-02T04:42:40.567 INFO:tasks.workunit.client.0.smithi016.stdout:2/448: fdatasync d12/d14/f3c 0 2024-04-02T04:42:40.570 INFO:tasks.workunit.client.0.smithi016.stdout:2/449: creat d12/d35/d3f/d56/d66/f9c x:0 0 0 2024-04-02T04:42:40.576 INFO:tasks.workunit.client.0.smithi016.stdout:7/466: dread d0/d21/f4e [0,4194304] 0 2024-04-02T04:42:40.576 INFO:tasks.workunit.client.0.smithi016.stdout:7/467: chown d0/d46/d16 0 1 2024-04-02T04:42:40.579 INFO:tasks.workunit.client.0.smithi016.stdout:7/468: symlink d0/d46/d37/d3a/d6f/d77/lab 0 2024-04-02T04:42:40.580 INFO:tasks.workunit.client.0.smithi016.stdout:2/450: write f9 [441492,7453] 0 2024-04-02T04:42:40.585 INFO:tasks.workunit.client.0.smithi016.stdout:2/451: rename d12/d35/f6e to d12/d20/f9d 0 2024-04-02T04:42:40.586 INFO:tasks.workunit.client.0.smithi016.stdout:2/452: readlink d12/d35/d3f/l2e 0 2024-04-02T04:42:40.587 INFO:tasks.workunit.client.0.smithi016.stdout:7/469: link d0/d46/d37/d3a/l66 d0/d46/d37/d3a/d43/d96/lac 0 2024-04-02T04:42:40.589 INFO:tasks.workunit.client.0.smithi016.stdout:2/453: dread d12/d35/d3f/f7a [0,4194304] 0 2024-04-02T04:42:40.593 INFO:tasks.workunit.client.0.smithi016.stdout:2/454: creat d12/d35/d3b/d42/d59/f9e x:0 0 0 2024-04-02T04:42:40.597 INFO:tasks.workunit.client.0.smithi016.stdout:2/455: mkdir d12/d9f 0 2024-04-02T04:42:40.602 INFO:tasks.workunit.client.0.smithi016.stdout:7/470: dread d0/d28/f59 [0,4194304] 0 2024-04-02T04:42:40.603 INFO:tasks.workunit.client.0.smithi016.stdout:2/456: dread d12/d35/d3b/f46 [0,4194304] 0 2024-04-02T04:42:40.603 INFO:tasks.workunit.client.0.smithi016.stdout:7/471: truncate d0/d46/d16/d49/fa7 225243 0 2024-04-02T04:42:40.604 INFO:tasks.workunit.client.0.smithi016.stdout:7/472: chown d0/d46/f1d 369203 1 2024-04-02T04:42:40.604 INFO:tasks.workunit.client.0.smithi016.stdout:2/457: dread d12/d35/d3f/d2b/f69 [0,4194304] 0 2024-04-02T04:42:40.608 INFO:tasks.workunit.client.0.smithi016.stdout:2/458: mkdir d12/d20/d76/da0 0 2024-04-02T04:42:40.611 INFO:tasks.workunit.client.0.smithi016.stdout:2/459: mkdir d12/d70/da1 0 2024-04-02T04:42:40.611 INFO:tasks.workunit.client.0.smithi016.stdout:2/460: dread - d12/d35/d3b/d42/f95 zero size 2024-04-02T04:42:40.614 INFO:tasks.workunit.client.0.smithi016.stdout:2/461: creat d12/d35/d3b/d42/d8d/fa2 x:0 0 0 2024-04-02T04:42:40.620 INFO:tasks.workunit.client.0.smithi016.stdout:2/462: dwrite d12/d35/d3f/d56/f94 [0,4194304] 0 2024-04-02T04:42:40.663 INFO:tasks.workunit.client.0.smithi016.stdout:7/473: write d0/d46/d16/d49/f70 [703735,109386] 0 2024-04-02T04:42:40.680 INFO:tasks.workunit.client.0.smithi016.stdout:2/463: write d12/d35/d3b/f46 [208330,83999] 0 2024-04-02T04:42:40.681 INFO:tasks.workunit.client.0.smithi016.stdout:3/471: sync 2024-04-02T04:42:40.683 INFO:tasks.workunit.client.0.smithi016.stdout:2/464: mkdir d12/d35/d3f/d2b/da3 0 2024-04-02T04:42:40.686 INFO:tasks.workunit.client.0.smithi016.stdout:3/472: dwrite d2/d15/d1a/f69 [0,4194304] 0 2024-04-02T04:42:40.688 INFO:tasks.workunit.client.0.smithi016.stdout:3/473: fdatasync d2/d15/d16/d57/f81 0 2024-04-02T04:42:40.690 INFO:tasks.workunit.client.0.smithi016.stdout:2/465: dwrite d12/d35/d3f/f39 [0,4194304] 0 2024-04-02T04:42:40.692 INFO:tasks.workunit.client.0.smithi016.stdout:2/466: write d12/d20/d4f/f77 [269367,88065] 0 2024-04-02T04:42:40.693 INFO:tasks.workunit.client.0.smithi016.stdout:2/467: readlink lb 0 2024-04-02T04:42:40.696 INFO:tasks.workunit.client.0.smithi016.stdout:3/474: rename d2/d15/d16/d3d/d46/l77 to d2/d15/d1a/d45/d56/d88/l8c 0 2024-04-02T04:42:40.699 INFO:tasks.workunit.client.0.smithi016.stdout:3/475: chown d2/c65 912626 1 2024-04-02T04:42:40.702 INFO:tasks.workunit.client.0.smithi016.stdout:2/468: link f8 d12/d35/d3b/d73/fa4 0 2024-04-02T04:42:40.705 INFO:tasks.workunit.client.0.smithi016.stdout:3/476: creat d2/d2e/f8d x:0 0 0 2024-04-02T04:42:40.706 INFO:tasks.workunit.client.0.smithi016.stdout:3/477: read - d2/d15/f38 zero size 2024-04-02T04:42:40.707 INFO:tasks.workunit.client.0.smithi016.stdout:2/469: getdents d12/d20/d4f 0 2024-04-02T04:42:40.709 INFO:tasks.workunit.client.0.smithi016.stdout:3/478: unlink d2/d15/d1a/f29 0 2024-04-02T04:42:40.714 INFO:tasks.workunit.client.0.smithi016.stdout:3/479: mknod d2/d15/d16/d3d/d8b/c8e 0 2024-04-02T04:42:40.716 INFO:tasks.workunit.client.0.smithi016.stdout:3/480: creat d2/d15/d1a/d45/f8f x:0 0 0 2024-04-02T04:42:40.734 INFO:tasks.workunit.client.0.smithi016.stdout:2/470: write d12/d14/f1f [83518,109533] 0 2024-04-02T04:42:40.737 INFO:tasks.workunit.client.0.smithi016.stdout:3/481: write d2/d15/f31 [2015606,33964] 0 2024-04-02T04:42:40.737 INFO:tasks.workunit.client.0.smithi016.stdout:2/471: mknod d12/d35/ca5 0 2024-04-02T04:42:40.739 INFO:tasks.workunit.client.0.smithi016.stdout:3/482: stat d2/d15/d16/d3d/f4e 0 2024-04-02T04:42:40.740 INFO:tasks.workunit.client.0.smithi016.stdout:2/472: link d12/d20/f2f d12/d35/d3b/d42/d8d/fa6 0 2024-04-02T04:42:40.741 INFO:tasks.workunit.client.0.smithi016.stdout:3/483: fsync d2/f7 0 2024-04-02T04:42:40.743 INFO:tasks.workunit.client.0.smithi016.stdout:2/473: creat d12/d9f/fa7 x:0 0 0 2024-04-02T04:42:40.743 INFO:tasks.workunit.client.0.smithi016.stdout:3/484: mkdir d2/d2e/d36/d66/d90 0 2024-04-02T04:42:40.744 INFO:tasks.workunit.client.0.smithi016.stdout:2/474: symlink d12/d35/d3f/d56/la8 0 2024-04-02T04:42:40.746 INFO:tasks.workunit.client.0.smithi016.stdout:3/485: creat d2/d2e/d36/d48/f91 x:0 0 0 2024-04-02T04:42:40.748 INFO:tasks.workunit.client.0.smithi016.stdout:2/475: dread d12/d35/d3f/f39 [0,4194304] 0 2024-04-02T04:42:40.748 INFO:tasks.workunit.client.0.smithi016.stdout:3/486: creat d2/d15/d1a/d45/d56/f92 x:0 0 0 2024-04-02T04:42:40.753 INFO:tasks.workunit.client.0.smithi016.stdout:2/476: creat d12/d35/d3b/d42/d67/d6a/fa9 x:0 0 0 2024-04-02T04:42:40.754 INFO:tasks.workunit.client.0.smithi016.stdout:3/487: creat d2/d15/d16/d8a/f93 x:0 0 0 2024-04-02T04:42:40.755 INFO:tasks.workunit.client.0.smithi016.stdout:2/477: rmdir d12/d20/d4f 39 2024-04-02T04:42:40.755 INFO:tasks.workunit.client.0.smithi016.stdout:2/478: chown d12/d35/c99 30045287 1 2024-04-02T04:42:40.755 INFO:tasks.workunit.client.0.smithi016.stdout:2/479: chown d12/d20/l87 975790 1 2024-04-02T04:42:40.756 INFO:tasks.workunit.client.0.smithi016.stdout:3/488: symlink d2/d2e/l94 0 2024-04-02T04:42:40.756 INFO:tasks.workunit.client.0.smithi016.stdout:2/480: fsync d12/d35/d3b/d42/f65 0 2024-04-02T04:42:40.757 INFO:tasks.workunit.client.0.smithi016.stdout:2/481: symlink d12/d35/d3f/d56/laa 0 2024-04-02T04:42:40.759 INFO:tasks.workunit.client.0.smithi016.stdout:3/489: getdents d2/d2e/d36 0 2024-04-02T04:42:40.766 INFO:tasks.workunit.client.0.smithi016.stdout:3/490: rmdir d2/d2e/d36/d5b 0 2024-04-02T04:42:40.766 INFO:tasks.workunit.client.0.smithi016.stdout:3/491: creat d2/d15/d16/d8a/f95 x:0 0 0 2024-04-02T04:42:40.766 INFO:tasks.workunit.client.0.smithi016.stdout:2/482: write d12/d26/f5a [906466,50245] 0 2024-04-02T04:42:40.767 INFO:tasks.workunit.client.0.smithi016.stdout:3/492: fsync d2/d15/f1b 0 2024-04-02T04:42:40.769 INFO:tasks.workunit.client.0.smithi016.stdout:3/493: read - d2/d15/d1a/d45/d56/f92 zero size 2024-04-02T04:42:40.769 INFO:tasks.workunit.client.0.smithi016.stdout:2/483: rename d12/d14 to d12/d35/d3f/d56/dab 0 2024-04-02T04:42:40.774 INFO:tasks.workunit.client.0.smithi016.stdout:2/484: readlink d12/d35/d3f/d56/dab/l5d 0 2024-04-02T04:42:40.774 INFO:tasks.workunit.client.0.smithi016.stdout:2/485: rename d12/d20 to d12/d9f/dac 0 2024-04-02T04:42:40.779 INFO:tasks.workunit.client.0.smithi016.stdout:3/494: rename d2/d15/d16/d57/d76 to d2/d15/d16/d57/d96 0 2024-04-02T04:42:40.786 INFO:tasks.workunit.client.0.smithi016.stdout:3/495: dwrite d2/d15/d16/d3d/f4e [0,4194304] 0 2024-04-02T04:42:40.787 INFO:tasks.workunit.client.0.smithi016.stdout:2/486: dwrite d12/d9f/dac/f25 [0,4194304] 0 2024-04-02T04:42:40.788 INFO:tasks.workunit.client.0.smithi016.stdout:3/496: dread - d2/d15/d1a/d45/d56/f92 zero size 2024-04-02T04:42:40.793 INFO:tasks.workunit.client.0.smithi016.stdout:2/487: rmdir d12/d70/da1 0 2024-04-02T04:42:40.799 INFO:tasks.workunit.client.0.smithi016.stdout:3/497: dwrite d2/d15/f1b [0,4194304] 0 2024-04-02T04:42:40.799 INFO:tasks.workunit.client.0.smithi016.stdout:2/488: link d12/d9f/dac/d4f/c5b d12/d35/d3b/d42/cad 0 2024-04-02T04:42:40.800 INFO:tasks.workunit.client.0.smithi016.stdout:3/498: rename d2/d15 to d2/d15/d1a/d45/d56/d88/d97 22 2024-04-02T04:42:40.802 INFO:tasks.workunit.client.0.smithi016.stdout:2/489: symlink d12/d35/d3b/d42/d67/d6a/lae 0 2024-04-02T04:42:40.803 INFO:tasks.workunit.client.0.smithi016.stdout:3/499: creat d2/d15/d16/d8a/f98 x:0 0 0 2024-04-02T04:42:40.808 INFO:tasks.workunit.client.0.smithi016.stdout:2/490: dread d12/d35/d3f/f61 [0,4194304] 0 2024-04-02T04:42:40.811 INFO:tasks.workunit.client.0.smithi016.stdout:2/491: unlink d12/d35/d3f/l6d 0 2024-04-02T04:42:40.811 INFO:tasks.workunit.client.0.smithi016.stdout:3/500: dread d2/d15/d16/d3d/d46/f62 [0,4194304] 0 2024-04-02T04:42:40.812 INFO:tasks.workunit.client.0.smithi016.stdout:2/492: mkdir d12/daf 0 2024-04-02T04:42:40.813 INFO:tasks.workunit.client.0.smithi016.stdout:3/501: symlink d2/d15/d16/d3d/d8b/l99 0 2024-04-02T04:42:40.814 INFO:tasks.workunit.client.0.smithi016.stdout:2/493: rmdir d12/d35/d3f/d2b 39 2024-04-02T04:42:40.815 INFO:tasks.workunit.client.0.smithi016.stdout:3/502: creat d2/d15/d16/d3d/d8b/f9a x:0 0 0 2024-04-02T04:42:40.818 INFO:tasks.workunit.client.0.smithi016.stdout:3/503: symlink d2/d15/d16/d3d/l9b 0 2024-04-02T04:42:40.820 INFO:tasks.workunit.client.0.smithi016.stdout:2/494: dwrite d12/d35/d3b/d42/f65 [0,4194304] 0 2024-04-02T04:42:40.821 INFO:tasks.workunit.client.0.smithi016.stdout:8/453: sync 2024-04-02T04:42:40.822 INFO:tasks.workunit.client.0.smithi016.stdout:3/504: stat d2/d2e/d36/d48/c73 0 2024-04-02T04:42:40.823 INFO:tasks.workunit.client.0.smithi016.stdout:2/495: rename d12/d26/c9b to d12/d35/d3f/d56/dab/cb0 0 2024-04-02T04:42:40.824 INFO:tasks.workunit.client.0.smithi016.stdout:2/496: stat d12/d26/l50 0 2024-04-02T04:42:40.824 INFO:tasks.workunit.client.0.smithi016.stdout:2/497: truncate d12/d70/f98 830361 0 2024-04-02T04:42:40.825 INFO:tasks.workunit.client.0.smithi016.stdout:3/505: creat d2/d15/d16/d3d/d8b/f9c x:0 0 0 2024-04-02T04:42:40.828 INFO:tasks.workunit.client.0.smithi016.stdout:2/498: dread d12/d35/d3f/d56/f94 [0,4194304] 0 2024-04-02T04:42:40.830 INFO:tasks.workunit.client.0.smithi016.stdout:3/506: mknod d2/d15/d16/d8a/c9d 0 2024-04-02T04:42:40.831 INFO:tasks.workunit.client.0.smithi016.stdout:8/454: link d1/ff d1/d6/d36/d4f/d5d/f95 0 2024-04-02T04:42:40.834 INFO:tasks.workunit.client.0.smithi016.stdout:3/507: rmdir d2/d15/d16/d3d/d6e 39 2024-04-02T04:42:40.838 INFO:tasks.workunit.client.0.smithi016.stdout:8/455: dwrite d1/d6/d36/d3f/d40/d49/f2a [0,4194304] 0 2024-04-02T04:42:40.839 INFO:tasks.workunit.client.0.smithi016.stdout:8/456: stat d1/d70 0 2024-04-02T04:42:40.841 INFO:tasks.workunit.client.0.smithi016.stdout:2/499: dwrite d12/d35/d3f/f61 [0,4194304] 0 2024-04-02T04:42:40.843 INFO:tasks.workunit.client.0.smithi016.stdout:8/457: rmdir d1/d70 39 2024-04-02T04:42:40.846 INFO:tasks.workunit.client.0.smithi016.stdout:8/458: readlink d1/d31/l3a 0 2024-04-02T04:42:40.847 INFO:tasks.workunit.client.0.smithi016.stdout:2/500: dread d12/d35/d3b/f57 [0,4194304] 0 2024-04-02T04:42:40.849 INFO:tasks.workunit.client.0.smithi016.stdout:2/501: write d12/f23 [621672,127020] 0 2024-04-02T04:42:40.852 INFO:tasks.workunit.client.0.smithi016.stdout:3/508: dread d2/d15/f4b [0,4194304] 0 2024-04-02T04:42:40.854 INFO:tasks.workunit.client.0.smithi016.stdout:3/509: stat d2/d2e/d36/d66/d90 0 2024-04-02T04:42:40.855 INFO:tasks.workunit.client.0.smithi016.stdout:2/502: mkdir d12/d35/d3f/d2b/da3/db1 0 2024-04-02T04:42:40.857 INFO:tasks.workunit.client.0.smithi016.stdout:8/459: dwrite d1/d6/d36/d3f/d40/d49/f2b [0,4194304] 0 2024-04-02T04:42:40.857 INFO:tasks.workunit.client.0.smithi016.stdout:3/510: creat d2/d15/d16/d3d/d46/f9e x:0 0 0 2024-04-02T04:42:40.858 INFO:tasks.workunit.client.0.smithi016.stdout:2/503: rmdir d12/d35/d3f/d56 39 2024-04-02T04:42:40.860 INFO:tasks.workunit.client.0.smithi016.stdout:9/480: sync 2024-04-02T04:42:40.860 INFO:tasks.workunit.client.0.smithi016.stdout:1/454: sync 2024-04-02T04:42:40.860 INFO:tasks.workunit.client.0.smithi016.stdout:5/459: sync 2024-04-02T04:42:40.860 INFO:tasks.workunit.client.0.smithi016.stdout:0/415: sync 2024-04-02T04:42:40.860 INFO:tasks.workunit.client.0.smithi016.stdout:6/532: sync 2024-04-02T04:42:40.860 INFO:tasks.workunit.client.0.smithi016.stdout:7/474: sync 2024-04-02T04:42:40.860 INFO:tasks.workunit.client.0.smithi016.stdout:4/510: sync 2024-04-02T04:42:40.861 INFO:tasks.workunit.client.0.smithi016.stdout:0/416: read - d0/d16/f54 zero size 2024-04-02T04:42:40.863 INFO:tasks.workunit.client.0.smithi016.stdout:5/460: chown d3/d40/d4f/d67/f6a 1133 1 2024-04-02T04:42:40.864 INFO:tasks.workunit.client.0.smithi016.stdout:2/504: mkdir d12/d35/d3b/d42/d59/d91/db2 0 2024-04-02T04:42:40.865 INFO:tasks.workunit.client.0.smithi016.stdout:7/475: mknod d0/d28/cad 0 2024-04-02T04:42:40.866 INFO:tasks.workunit.client.0.smithi016.stdout:7/476: stat d0/l8d 0 2024-04-02T04:42:40.867 INFO:tasks.workunit.client.0.smithi016.stdout:6/533: mkdir d0/d9/d16/d56/d59/da1 0 2024-04-02T04:42:40.871 INFO:tasks.workunit.client.0.smithi016.stdout:3/511: rename d2/d15/d1a/l51 to d2/d15/d16/d3d/d6e/d6f/d78/l9f 0 2024-04-02T04:42:40.873 INFO:tasks.workunit.client.0.smithi016.stdout:8/460: write d1/d6/d36/d4f/d5d/f95 [3969367,82118] 0 2024-04-02T04:42:40.874 INFO:tasks.workunit.client.0.smithi016.stdout:4/511: mkdir d5/d17/d18/d6d/d8e/d9c 0 2024-04-02T04:42:40.875 INFO:tasks.workunit.client.0.smithi016.stdout:8/461: chown d1/d6/d36/c65 4 1 2024-04-02T04:42:40.876 INFO:tasks.workunit.client.0.smithi016.stdout:1/455: dread d14/f34 [0,4194304] 0 2024-04-02T04:42:40.876 INFO:tasks.workunit.client.0.smithi016.stdout:4/512: write d5/d17/d18/d6d/f74 [892250,59401] 0 2024-04-02T04:42:40.878 INFO:tasks.workunit.client.0.smithi016.stdout:4/513: fdatasync d5/d8/fd 0 2024-04-02T04:42:40.878 INFO:tasks.workunit.client.0.smithi016.stdout:7/477: rename d0/d46/d37/d3a/d6f/d77 to d0/d46/d37/d3a/d6f/dae 0 2024-04-02T04:42:40.880 INFO:tasks.workunit.client.0.smithi016.stdout:5/461: rename d3/d40/d4f/d6b/d45/l9f to d3/df/d18/d2f/la8 0 2024-04-02T04:42:40.882 INFO:tasks.workunit.client.0.smithi016.stdout:2/505: write d12/d35/d3b/d42/d59/f88 [1088057,64214] 0 2024-04-02T04:42:40.884 INFO:tasks.workunit.client.0.smithi016.stdout:1/456: stat d14/d19/d20/f3f 0 2024-04-02T04:42:40.885 INFO:tasks.workunit.client.0.smithi016.stdout:7/478: mkdir d0/d46/d52/daf 0 2024-04-02T04:42:40.886 INFO:tasks.workunit.client.0.smithi016.stdout:3/512: mknod d2/d15/d16/d3d/ca0 0 2024-04-02T04:42:40.888 INFO:tasks.workunit.client.0.smithi016.stdout:5/462: rename d3/d5d/f6f to d3/d4c/fa9 0 2024-04-02T04:42:40.890 INFO:tasks.workunit.client.0.smithi016.stdout:2/506: creat d12/d35/d3b/d42/d67/fb3 x:0 0 0 2024-04-02T04:42:40.891 INFO:tasks.workunit.client.0.smithi016.stdout:6/534: truncate d0/d9/d16/d56/d89/f3b 2370981 0 2024-04-02T04:42:40.893 INFO:tasks.workunit.client.0.smithi016.stdout:4/514: truncate d5/d17/d18/d1f/f59 267511 0 2024-04-02T04:42:40.896 INFO:tasks.workunit.client.0.smithi016.stdout:7/479: rename d0/d46/d37/d3a/d43/f85 to d0/d46/d37/d3a/d4c/fb0 0 2024-04-02T04:42:40.897 INFO:tasks.workunit.client.0.smithi016.stdout:8/462: link d1/d70/f88 d1/d70/d71/d74/f96 0 2024-04-02T04:42:40.898 INFO:tasks.workunit.client.0.smithi016.stdout:8/463: write d1/d6/d36/d3f/f6c [449799,102997] 0 2024-04-02T04:42:40.898 INFO:tasks.workunit.client.0.smithi016.stdout:6/535: creat d0/d6e/fa2 x:0 0 0 2024-04-02T04:42:40.900 INFO:tasks.workunit.client.0.smithi016.stdout:6/536: write d0/d9/d16/d46/d5e/f93 [322776,126396] 0 2024-04-02T04:42:40.903 INFO:tasks.workunit.client.0.smithi016.stdout:3/513: dread d2/fa [0,4194304] 0 2024-04-02T04:42:40.905 INFO:tasks.workunit.client.0.smithi016.stdout:7/480: creat d0/d87/fb1 x:0 0 0 2024-04-02T04:42:40.906 INFO:tasks.workunit.client.0.smithi016.stdout:8/464: mknod d1/d6/d36/d4f/c97 0 2024-04-02T04:42:40.907 INFO:tasks.workunit.client.0.smithi016.stdout:7/481: truncate d0/d46/d16/f62 114314 0 2024-04-02T04:42:40.907 INFO:tasks.workunit.client.0.smithi016.stdout:8/465: fsync d1/d70/d71/f94 0 2024-04-02T04:42:40.907 INFO:tasks.workunit.client.0.smithi016.stdout:3/514: dread d2/d15/d1a/f53 [0,4194304] 0 2024-04-02T04:42:40.910 INFO:tasks.workunit.client.0.smithi016.stdout:3/515: write d2/d15/d16/d3d/f4e [2097284,32652] 0 2024-04-02T04:42:40.910 INFO:tasks.workunit.client.0.smithi016.stdout:6/537: rmdir d0/d9/d16/d56/d89/d67/d6f/d8a 39 2024-04-02T04:42:40.911 INFO:tasks.workunit.client.0.smithi016.stdout:5/463: creat d3/faa x:0 0 0 2024-04-02T04:42:40.912 INFO:tasks.workunit.client.0.smithi016.stdout:7/482: truncate d0/d46/d37/d3a/d43/d96/fa5 234377 0 2024-04-02T04:42:40.912 INFO:tasks.workunit.client.0.smithi016.stdout:1/457: dread d14/d19/d1d/f30 [0,4194304] 0 2024-04-02T04:42:40.912 INFO:tasks.workunit.client.0.smithi016.stdout:6/538: write d0/d9/d16/f23 [2892690,114857] 0 2024-04-02T04:42:40.913 INFO:tasks.workunit.client.0.smithi016.stdout:1/458: chown d14/d41/f93 5189 1 2024-04-02T04:42:40.915 INFO:tasks.workunit.client.0.smithi016.stdout:2/507: link d12/d35/d3b/d42/d67/c6b d12/d35/d3b/d42/d67/d6a/cb4 0 2024-04-02T04:42:40.915 INFO:tasks.workunit.client.0.smithi016.stdout:8/466: creat d1/d6/d36/d3f/d40/d49/d4a/f98 x:0 0 0 2024-04-02T04:42:40.915 INFO:tasks.workunit.client.0.smithi016.stdout:7/483: chown d0/d46/d37/d47/l95 10 1 2024-04-02T04:42:40.916 INFO:tasks.workunit.client.0.smithi016.stdout:1/459: stat d14/d19/d1d/d29/d39 0 2024-04-02T04:42:40.917 INFO:tasks.workunit.client.0.smithi016.stdout:8/467: write d1/d6/d36/d4f/f62 [3094181,3221] 0 2024-04-02T04:42:40.918 INFO:tasks.workunit.client.0.smithi016.stdout:3/516: mknod d2/ca1 0 2024-04-02T04:42:40.920 INFO:tasks.workunit.client.0.smithi016.stdout:3/517: dread - d2/d15/d1a/d45/d56/f92 zero size 2024-04-02T04:42:40.921 INFO:tasks.workunit.client.0.smithi016.stdout:3/518: stat d2/d15/d70/f79 0 2024-04-02T04:42:40.922 INFO:tasks.workunit.client.0.smithi016.stdout:5/464: link d3/d40/d4f/f75 d3/d40/d4f/d6b/d45/fab 0 2024-04-02T04:42:40.923 INFO:tasks.workunit.client.0.smithi016.stdout:7/484: dwrite d0/d46/d37/d47/f5c [0,4194304] 0 2024-04-02T04:42:40.923 INFO:tasks.workunit.client.0.smithi016.stdout:2/508: symlink d12/lb5 0 2024-04-02T04:42:40.924 INFO:tasks.workunit.client.0.smithi016.stdout:6/539: getdents d0/d9/d16/d56/d89/d67/d6f/d7e 0 2024-04-02T04:42:40.926 INFO:tasks.workunit.client.0.smithi016.stdout:8/468: creat d1/d6/d36/f99 x:0 0 0 2024-04-02T04:42:40.927 INFO:tasks.workunit.client.0.smithi016.stdout:2/509: write f9 [4649747,53587] 0 2024-04-02T04:42:40.929 INFO:tasks.workunit.client.0.smithi016.stdout:3/519: rename d2/d15/d1a/d45/d56 to d2/d2e/d36/d48/da2 0 2024-04-02T04:42:40.930 INFO:tasks.workunit.client.0.smithi016.stdout:7/485: truncate d0/d46/f2f 326091 0 2024-04-02T04:42:40.931 INFO:tasks.workunit.client.0.smithi016.stdout:6/540: chown d0/d9/df/l53 102172503 1 2024-04-02T04:42:40.932 INFO:tasks.workunit.client.0.smithi016.stdout:5/465: creat d3/df/d18/fac x:0 0 0 2024-04-02T04:42:40.934 INFO:tasks.workunit.client.0.smithi016.stdout:1/460: dwrite d14/d19/f82 [4194304,4194304] 0 2024-04-02T04:42:40.935 INFO:tasks.workunit.client.0.smithi016.stdout:8/469: unlink d1/d6/d36/d3f/c81 0 2024-04-02T04:42:40.936 INFO:tasks.workunit.client.0.smithi016.stdout:1/461: dread - d14/d19/d1d/f7d zero size 2024-04-02T04:42:40.938 INFO:tasks.workunit.client.0.smithi016.stdout:8/470: write d1/d70/f84 [99333,22444] 0 2024-04-02T04:42:40.939 INFO:tasks.workunit.client.0.smithi016.stdout:7/486: dread d0/d46/d37/d3a/f7d [0,4194304] 0 2024-04-02T04:42:40.939 INFO:tasks.workunit.client.0.smithi016.stdout:3/520: symlink d2/d15/d16/d3d/d6e/d6f/la3 0 2024-04-02T04:42:40.941 INFO:tasks.workunit.client.0.smithi016.stdout:3/521: stat d2/d15/d16/d3d/d6e/d6f 0 2024-04-02T04:42:40.945 INFO:tasks.workunit.client.0.smithi016.stdout:1/462: creat d14/d41/d43/f96 x:0 0 0 2024-04-02T04:42:40.945 INFO:tasks.workunit.client.0.smithi016.stdout:8/471: rmdir d1/d6/d36/d3f/d40 39 2024-04-02T04:42:40.946 INFO:tasks.workunit.client.0.smithi016.stdout:1/463: fdatasync d14/d19/d20/f61 0 2024-04-02T04:42:40.947 INFO:tasks.workunit.client.0.smithi016.stdout:5/466: creat d3/fad x:0 0 0 2024-04-02T04:42:40.947 INFO:tasks.workunit.client.0.smithi016.stdout:6/541: dread d0/d9/d16/d56/d59/f4b [0,4194304] 0 2024-04-02T04:42:40.948 INFO:tasks.workunit.client.0.smithi016.stdout:5/467: rename d3/df/d18 to d3/df/d18/d26/d2c/dae 22 2024-04-02T04:42:40.948 INFO:tasks.workunit.client.0.smithi016.stdout:7/487: write d0/d46/d37/d3a/d4c/fb0 [296226,61781] 0 2024-04-02T04:42:40.951 INFO:tasks.workunit.client.0.smithi016.stdout:3/522: creat d2/d15/d1a/d45/fa4 x:0 0 0 2024-04-02T04:42:40.956 INFO:tasks.workunit.client.0.smithi016.stdout:2/510: truncate d12/d26/f5a 639824 0 2024-04-02T04:42:40.956 INFO:tasks.workunit.client.0.smithi016.stdout:5/468: dwrite d3/d40/d4f/d6b/fa2 [0,4194304] 0 2024-04-02T04:42:40.958 INFO:tasks.workunit.client.0.smithi016.stdout:3/523: write d2/d15/d16/f54 [1216423,2885] 0 2024-04-02T04:42:40.959 INFO:tasks.workunit.client.0.smithi016.stdout:8/472: creat d1/d6/d36/d3f/d40/d49/d4a/d4c/f9a x:0 0 0 2024-04-02T04:42:40.960 INFO:tasks.workunit.client.0.smithi016.stdout:5/469: readlink d3/df/d18/d26/d48/l5e 0 2024-04-02T04:42:40.962 INFO:tasks.workunit.client.0.smithi016.stdout:8/473: stat d1/d6/d36/d3f/d40/d49/l58 0 2024-04-02T04:42:40.963 INFO:tasks.workunit.client.0.smithi016.stdout:6/542: dwrite d0/d9/fd [4194304,4194304] 0 2024-04-02T04:42:40.965 INFO:tasks.workunit.client.0.smithi016.stdout:7/488: rename d0/c89 to d0/d46/d16/d83/cb2 0 2024-04-02T04:42:40.970 INFO:tasks.workunit.client.0.smithi016.stdout:3/524: mkdir d2/d15/d1a/d45/da5 0 2024-04-02T04:42:40.971 INFO:tasks.workunit.client.0.smithi016.stdout:5/470: creat d3/df/d18/d26/d70/faf x:0 0 0 2024-04-02T04:42:40.974 INFO:tasks.workunit.client.0.smithi016.stdout:8/474: mknod d1/d6/d36/d3f/d40/d49/d4a/d4c/c9b 0 2024-04-02T04:42:40.975 INFO:tasks.workunit.client.0.smithi016.stdout:1/464: getdents d14/d19/d1e 0 2024-04-02T04:42:40.976 INFO:tasks.workunit.client.0.smithi016.stdout:2/511: unlink d12/d35/d3f/d56/d66/f71 0 2024-04-02T04:42:40.982 INFO:tasks.workunit.client.0.smithi016.stdout:1/465: symlink d14/d19/d50/l97 0 2024-04-02T04:42:40.984 INFO:tasks.workunit.client.0.smithi016.stdout:2/512: fdatasync d12/d70/f75 0 2024-04-02T04:42:40.986 INFO:tasks.workunit.client.0.smithi016.stdout:6/543: creat d0/fa3 x:0 0 0 2024-04-02T04:42:40.987 INFO:tasks.workunit.client.0.smithi016.stdout:5/471: write d3/df/d18/d26/d48/f49 [2094106,61019] 0 2024-04-02T04:42:40.988 INFO:tasks.workunit.client.0.smithi016.stdout:6/544: chown d0/d9/d16/d56/f4f 24 1 2024-04-02T04:42:40.990 INFO:tasks.workunit.client.0.smithi016.stdout:6/545: write d0/d9/d16/d56/f4f [302976,53600] 0 2024-04-02T04:42:40.991 INFO:tasks.workunit.client.0.smithi016.stdout:7/489: truncate d0/d46/fc 2849562 0 2024-04-02T04:42:40.992 INFO:tasks.workunit.client.0.smithi016.stdout:1/466: mknod d14/d19/d1d/d45/d8a/c98 0 2024-04-02T04:42:40.992 INFO:tasks.workunit.client.0.smithi016.stdout:7/490: write d0/d28/f53 [1096232,81353] 0 2024-04-02T04:42:40.992 INFO:tasks.workunit.client.0.smithi016.stdout:5/472: symlink d3/d40/d4f/d6b/d45/lb0 0 2024-04-02T04:42:40.994 INFO:tasks.workunit.client.0.smithi016.stdout:8/475: getdents d1/d70 0 2024-04-02T04:42:40.995 INFO:tasks.workunit.client.0.smithi016.stdout:8/476: write d1/d6/f1a [299388,11662] 0 2024-04-02T04:42:40.996 INFO:tasks.workunit.client.0.smithi016.stdout:1/467: rmdir d14/d19/d1d/d29/d4c 39 2024-04-02T04:42:40.998 INFO:tasks.workunit.client.0.smithi016.stdout:2/513: rename d12/f2a to d12/d35/d3f/fb6 0 2024-04-02T04:42:41.000 INFO:tasks.workunit.client.0.smithi016.stdout:6/546: mkdir d0/d9/d16/d56/da4 0 2024-04-02T04:42:41.000 INFO:tasks.workunit.client.0.smithi016.stdout:7/491: chown d0/d46/d37/d3a/l66 739661493 1 2024-04-02T04:42:41.004 INFO:tasks.workunit.client.0.smithi016.stdout:7/492: read d0/d28/f59 [459902,30451] 0 2024-04-02T04:42:41.004 INFO:tasks.workunit.client.0.smithi016.stdout:3/525: dread d2/f1f [0,4194304] 0 2024-04-02T04:42:41.005 INFO:tasks.workunit.client.0.smithi016.stdout:1/468: symlink d14/d19/d1d/d29/d39/l99 0 2024-04-02T04:42:41.005 INFO:tasks.workunit.client.0.smithi016.stdout:5/473: rename d3/d40/d4f/d6b/d45/lb0 to d3/df/d18/d26/d70/lb1 0 2024-04-02T04:42:41.008 INFO:tasks.workunit.client.0.smithi016.stdout:2/514: unlink d12/d35/d3b/d73/fa4 0 2024-04-02T04:42:41.008 INFO:tasks.workunit.client.0.smithi016.stdout:8/477: dwrite d1/d6/d36/d3f/d40/d49/f68 [4194304,4194304] 0 2024-04-02T04:42:41.009 INFO:tasks.workunit.client.0.smithi016.stdout:1/469: truncate d14/d19/d1e/d71/f78 890946 0 2024-04-02T04:42:41.009 INFO:tasks.workunit.client.0.smithi016.stdout:3/526: stat d2/d15/d16/d35 0 2024-04-02T04:42:41.010 INFO:tasks.workunit.client.0.smithi016.stdout:2/515: fdatasync d12/d9f/dac/d4f/f5f 0 2024-04-02T04:42:41.011 INFO:tasks.workunit.client.0.smithi016.stdout:5/474: symlink d3/d5d/lb2 0 2024-04-02T04:42:41.014 INFO:tasks.workunit.client.0.smithi016.stdout:3/527: fdatasync d2/f3f 0 2024-04-02T04:42:41.015 INFO:tasks.workunit.client.0.smithi016.stdout:5/475: dread - d3/df/d18/fac zero size 2024-04-02T04:42:41.016 INFO:tasks.workunit.client.0.smithi016.stdout:1/470: mknod d14/d19/d1d/d45/d4f/d86/c9a 0 2024-04-02T04:42:41.018 INFO:tasks.workunit.client.0.smithi016.stdout:2/516: unlink d12/d35/d3b/d42/l47 0 2024-04-02T04:42:41.022 INFO:tasks.workunit.client.0.smithi016.stdout:1/471: fdatasync d14/d19/d1d/f30 0 2024-04-02T04:42:41.024 INFO:tasks.workunit.client.0.smithi016.stdout:1/472: write d14/d19/d20/f5b [508569,22123] 0 2024-04-02T04:42:41.025 INFO:tasks.workunit.client.0.smithi016.stdout:1/473: readlink d14/d41/l54 0 2024-04-02T04:42:41.027 INFO:tasks.workunit.client.0.smithi016.stdout:1/474: write d14/d41/d43/f96 [99750,55001] 0 2024-04-02T04:42:41.029 INFO:tasks.workunit.client.0.smithi016.stdout:2/517: link d12/d9f/dac/f25 d12/d35/d3f/d56/fb7 0 2024-04-02T04:42:41.032 INFO:tasks.workunit.client.0.smithi016.stdout:6/547: dwrite d0/d9/d16/d56/f25 [0,4194304] 0 2024-04-02T04:42:41.033 INFO:tasks.workunit.client.0.smithi016.stdout:8/478: link d1/f15 d1/d6/d36/d8e/f9c 0 2024-04-02T04:42:41.033 INFO:tasks.workunit.client.0.smithi016.stdout:1/475: fdatasync d14/f17 0 2024-04-02T04:42:41.034 INFO:tasks.workunit.client.0.smithi016.stdout:1/476: stat d14/d19/d1e 0 2024-04-02T04:42:41.037 INFO:tasks.workunit.client.0.smithi016.stdout:2/518: write d12/d35/d3f/d56/dab/f37 [883376,994] 0 2024-04-02T04:42:41.038 INFO:tasks.workunit.client.0.smithi016.stdout:6/548: creat d0/d9/d16/d56/d89/d67/d6f/d8a/fa5 x:0 0 0 2024-04-02T04:42:41.039 INFO:tasks.workunit.client.0.smithi016.stdout:2/519: chown d12/d26/f29 114853796 1 2024-04-02T04:42:41.040 INFO:tasks.workunit.client.0.smithi016.stdout:6/549: creat d0/d9/df/fa6 x:0 0 0 2024-04-02T04:42:41.041 INFO:tasks.workunit.client.0.smithi016.stdout:3/528: truncate d2/d15/d16/f27 632602 0 2024-04-02T04:42:41.041 INFO:tasks.workunit.client.0.smithi016.stdout:8/479: dread d1/d6/f21 [0,4194304] 0 2024-04-02T04:42:41.042 INFO:tasks.workunit.client.0.smithi016.stdout:2/520: unlink d12/d35/d3f/f2d 0 2024-04-02T04:42:41.043 INFO:tasks.workunit.client.0.smithi016.stdout:2/521: dread - d12/d35/d3b/d42/d8d/fa2 zero size 2024-04-02T04:42:41.044 INFO:tasks.workunit.client.0.smithi016.stdout:6/550: rmdir d0/d9/d16/d56/d89/d67/d6d 39 2024-04-02T04:42:41.045 INFO:tasks.workunit.client.0.smithi016.stdout:2/522: read d12/d35/d3b/f63 [67466,118319] 0 2024-04-02T04:42:41.046 INFO:tasks.workunit.client.0.smithi016.stdout:8/480: mknod d1/c9d 0 2024-04-02T04:42:41.047 INFO:tasks.workunit.client.0.smithi016.stdout:1/477: dread d14/d19/d1e/f2b [0,4194304] 0 2024-04-02T04:42:41.048 INFO:tasks.workunit.client.0.smithi016.stdout:3/529: creat d2/d15/d16/d3d/fa6 x:0 0 0 2024-04-02T04:42:41.049 INFO:tasks.workunit.client.0.smithi016.stdout:1/478: chown c13 2 1 2024-04-02T04:42:41.049 INFO:tasks.workunit.client.0.smithi016.stdout:8/481: mkdir d1/d70/d71/d9e 0 2024-04-02T04:42:41.049 INFO:tasks.workunit.client.0.smithi016.stdout:2/523: truncate d12/d35/d3b/d42/f95 772187 0 2024-04-02T04:42:41.050 INFO:tasks.workunit.client.0.smithi016.stdout:6/551: mknod d0/d9/d16/d56/d89/d67/d6f/d7e/ca7 0 2024-04-02T04:42:41.050 INFO:tasks.workunit.client.0.smithi016.stdout:6/552: chown d0 7489 1 2024-04-02T04:42:41.051 INFO:tasks.workunit.client.0.smithi016.stdout:2/524: creat d12/d9f/fb8 x:0 0 0 2024-04-02T04:42:41.052 INFO:tasks.workunit.client.0.smithi016.stdout:6/553: fdatasync d0/fc 0 2024-04-02T04:42:41.052 INFO:tasks.workunit.client.0.smithi016.stdout:6/554: symlink d0/d9/d16/d46/la8 2 2024-04-02T04:42:41.053 INFO:tasks.workunit.client.0.smithi016.stdout:8/482: creat d1/d31/d67/d8f/f9f x:0 0 0 2024-04-02T04:42:41.063 INFO:tasks.workunit.client.0.smithi016.stdout:3/530: getdents d2/d15/d16/d3d 0 2024-04-02T04:42:41.064 INFO:tasks.workunit.client.0.smithi016.stdout:8/483: mknod d1/d6/d36/d8e/ca0 0 2024-04-02T04:42:41.065 INFO:tasks.workunit.client.0.smithi016.stdout:2/525: getdents d12/d35/d3f/d2b/da3 0 2024-04-02T04:42:41.069 INFO:tasks.workunit.client.0.smithi016.stdout:2/526: creat d12/d35/d3f/d2b/da3/fb9 x:0 0 0 2024-04-02T04:42:41.070 INFO:tasks.workunit.client.0.smithi016.stdout:8/484: dwrite d1/d6/d36/d3f/d40/d49/f2b [0,4194304] 0 2024-04-02T04:42:41.072 INFO:tasks.workunit.client.0.smithi016.stdout:6/555: truncate d0/d6e/f81 132643 0 2024-04-02T04:42:41.073 INFO:tasks.workunit.client.0.smithi016.stdout:6/556: chown d0/d9/d16/f29 209639298 1 2024-04-02T04:42:41.075 INFO:tasks.workunit.client.0.smithi016.stdout:8/485: mkdir d1/d6/d36/d3f/d40/d49/d38/da1 0 2024-04-02T04:42:41.077 INFO:tasks.workunit.client.0.smithi016.stdout:6/557: write d0/d9/f33 [1638198,16774] 0 2024-04-02T04:42:41.079 INFO:tasks.workunit.client.0.smithi016.stdout:2/527: link d12/d35/d3f/d56/fb7 d12/d35/d3b/d42/d59/fba 0 2024-04-02T04:42:41.082 INFO:tasks.workunit.client.0.smithi016.stdout:8/486: mknod d1/d31/d67/ca2 0 2024-04-02T04:42:41.086 INFO:tasks.workunit.client.0.smithi016.stdout:2/528: creat d12/d9f/dac/d76/fbb x:0 0 0 2024-04-02T04:42:41.087 INFO:tasks.workunit.client.0.smithi016.stdout:2/529: chown d12/d70/f75 464 1 2024-04-02T04:42:41.089 INFO:tasks.workunit.client.0.smithi016.stdout:3/531: write d2/d15/d16/f27 [42321,63226] 0 2024-04-02T04:42:41.090 INFO:tasks.workunit.client.0.smithi016.stdout:3/532: chown d2/d15/d16/d57/f81 326607 1 2024-04-02T04:42:41.092 INFO:tasks.workunit.client.0.smithi016.stdout:2/530: rmdir d12/d35/d3b/d42/d8d 39 2024-04-02T04:42:41.093 INFO:tasks.workunit.client.0.smithi016.stdout:2/531: dread - d12/d35/d3f/d2b/da3/fb9 zero size 2024-04-02T04:42:41.094 INFO:tasks.workunit.client.0.smithi016.stdout:8/487: rename d1/d6/d36/d3f/d40/d49/d4a/d4c/f9a to d1/d6/d36/fa3 0 2024-04-02T04:42:41.095 INFO:tasks.workunit.client.0.smithi016.stdout:6/558: link d0/d9/d16/l50 d0/d9/la9 0 2024-04-02T04:42:41.095 INFO:tasks.workunit.client.0.smithi016.stdout:6/559: read - d0/f9b zero size 2024-04-02T04:42:41.096 INFO:tasks.workunit.client.0.smithi016.stdout:3/533: mknod d2/d2e/d36/d48/ca7 0 2024-04-02T04:42:41.105 INFO:tasks.workunit.client.0.smithi016.stdout:3/534: symlink d2/d2e/d36/d66/d7a/la8 0 2024-04-02T04:42:41.106 INFO:tasks.workunit.client.0.smithi016.stdout:3/535: write d2/d15/d16/f54 [573286,68734] 0 2024-04-02T04:42:41.109 INFO:tasks.workunit.client.0.smithi016.stdout:6/560: rename d0/d9/d16/d56/d89/d67/d6f/d8a/d91/f9e to d0/faa 0 2024-04-02T04:42:41.110 INFO:tasks.workunit.client.0.smithi016.stdout:8/488: truncate d1/d6/f53 893150 0 2024-04-02T04:42:41.111 INFO:tasks.workunit.client.0.smithi016.stdout:8/489: write d1/d6/f1a [4781817,118753] 0 2024-04-02T04:42:41.112 INFO:tasks.workunit.client.0.smithi016.stdout:2/532: dread d12/d35/d3b/f60 [0,4194304] 0 2024-04-02T04:42:41.113 INFO:tasks.workunit.client.0.smithi016.stdout:3/536: creat d2/d15/d7e/fa9 x:0 0 0 2024-04-02T04:42:41.114 INFO:tasks.workunit.client.0.smithi016.stdout:3/537: fdatasync d2/fd 0 2024-04-02T04:42:41.114 INFO:tasks.workunit.client.0.smithi016.stdout:8/490: unlink d1/d70/d71/d74/d82/f87 0 2024-04-02T04:42:41.117 INFO:tasks.workunit.client.0.smithi016.stdout:6/561: dwrite d0/d9/d16/d56/d59/f60 [0,4194304] 0 2024-04-02T04:42:41.117 INFO:tasks.workunit.client.0.smithi016.stdout:2/533: dread d12/d35/d3f/f7e [0,4194304] 0 2024-04-02T04:42:41.119 INFO:tasks.workunit.client.0.smithi016.stdout:6/562: readlink d0/d9/df/l53 0 2024-04-02T04:42:41.120 INFO:tasks.workunit.client.0.smithi016.stdout:3/538: rename d2/d2e/l94 to d2/d2e/laa 0 2024-04-02T04:42:41.122 INFO:tasks.workunit.client.0.smithi016.stdout:3/539: write d2/d2e/f30 [118242,60301] 0 2024-04-02T04:42:41.123 INFO:tasks.workunit.client.0.smithi016.stdout:2/534: mknod d12/d35/d3f/cbc 0 2024-04-02T04:42:41.131 INFO:tasks.workunit.client.0.smithi016.stdout:2/535: creat d12/d35/d3b/d42/d67/d82/fbd x:0 0 0 2024-04-02T04:42:41.133 INFO:tasks.workunit.client.0.smithi016.stdout:3/540: creat d2/fab x:0 0 0 2024-04-02T04:42:41.136 INFO:tasks.workunit.client.0.smithi016.stdout:6/563: getdents d0/d9/df 0 2024-04-02T04:42:41.136 INFO:tasks.workunit.client.0.smithi016.stdout:8/491: dwrite d1/d6/d36/d3f/f47 [0,4194304] 0 2024-04-02T04:42:41.137 INFO:tasks.workunit.client.0.smithi016.stdout:2/536: mkdir d12/d9f/dac/d76/da0/dbe 0 2024-04-02T04:42:41.138 INFO:tasks.workunit.client.0.smithi016.stdout:8/492: stat d1/d6/d36/d3f/f80 0 2024-04-02T04:42:41.141 INFO:tasks.workunit.client.0.smithi016.stdout:9/481: sync 2024-04-02T04:42:41.149 INFO:tasks.workunit.client.0.smithi016.stdout:8/493: rmdir d1/d6/d36/d3f 39 2024-04-02T04:42:41.150 INFO:tasks.workunit.client.0.smithi016.stdout:2/537: rename d12/d35/d3f/d56/laa to d12/d35/d3f/d4d/lbf 0 2024-04-02T04:42:41.157 INFO:tasks.workunit.client.0.smithi016.stdout:8/494: rename d1/d6/d36/d3f/d40/d49/f33 to d1/d31/d67/d8f/fa4 0 2024-04-02T04:42:41.157 INFO:tasks.workunit.client.0.smithi016.stdout:6/564: dwrite d0/d9/f7a [0,4194304] 0 2024-04-02T04:42:41.158 INFO:tasks.workunit.client.0.smithi016.stdout:9/482: dwrite d0/d3/d31/d53/f7a [0,4194304] 0 2024-04-02T04:42:41.161 INFO:tasks.workunit.client.0.smithi016.stdout:2/538: dread d12/d35/d3f/d56/dab/f3c [0,4194304] 0 2024-04-02T04:42:41.161 INFO:tasks.workunit.client.0.smithi016.stdout:6/565: creat d0/d9/d16/d56/d95/fab x:0 0 0 2024-04-02T04:42:41.163 INFO:tasks.workunit.client.0.smithi016.stdout:8/495: symlink d1/d6/d36/d4f/d5d/la5 0 2024-04-02T04:42:41.166 INFO:tasks.workunit.client.0.smithi016.stdout:2/539: symlink d12/d26/lc0 0 2024-04-02T04:42:41.167 INFO:tasks.workunit.client.0.smithi016.stdout:9/483: mkdir d0/d4a/d75/d8d 0 2024-04-02T04:42:41.168 INFO:tasks.workunit.client.0.smithi016.stdout:6/566: creat d0/d9/d16/d56/da4/fac x:0 0 0 2024-04-02T04:42:41.171 INFO:tasks.workunit.client.0.smithi016.stdout:9/484: creat d0/d9/db/d54/f8e x:0 0 0 2024-04-02T04:42:41.172 INFO:tasks.workunit.client.0.smithi016.stdout:2/540: link d12/f43 d12/d35/d3f/d2b/fc1 0 2024-04-02T04:42:41.172 INFO:tasks.workunit.client.0.smithi016.stdout:8/496: dwrite d1/d6/d36/d3f/d40/d49/f43 [0,4194304] 0 2024-04-02T04:42:41.172 INFO:tasks.workunit.client.0.smithi016.stdout:6/567: mknod d0/d9/d16/cad 0 2024-04-02T04:42:41.176 INFO:tasks.workunit.client.0.smithi016.stdout:2/541: symlink d12/d35/d3f/d56/dab/lc2 0 2024-04-02T04:42:41.176 INFO:tasks.workunit.client.0.smithi016.stdout:9/485: dread d0/f5d [0,4194304] 0 2024-04-02T04:42:41.177 INFO:tasks.workunit.client.0.smithi016.stdout:2/542: read - d12/f96 zero size 2024-04-02T04:42:41.182 INFO:tasks.workunit.client.0.smithi016.stdout:8/497: fsync d1/d70/f88 0 2024-04-02T04:42:41.183 INFO:tasks.workunit.client.0.smithi016.stdout:9/486: mknod d0/d9/db/c8f 0 2024-04-02T04:42:41.183 INFO:tasks.workunit.client.0.smithi016.stdout:8/498: chown d1/d6/d36/d3f/d40/d49/d4a/f5a 1025512906 1 2024-04-02T04:42:41.183 INFO:tasks.workunit.client.0.smithi016.stdout:6/568: dwrite d0/d9/fd [4194304,4194304] 0 2024-04-02T04:42:41.187 INFO:tasks.workunit.client.0.smithi016.stdout:6/569: mknod d0/d9/d16/cae 0 2024-04-02T04:42:41.188 INFO:tasks.workunit.client.0.smithi016.stdout:2/543: link d12/d35/d3f/fb6 d12/d9f/dac/d4f/fc3 0 2024-04-02T04:42:41.190 INFO:tasks.workunit.client.0.smithi016.stdout:8/499: creat d1/d70/d71/d74/fa6 x:0 0 0 2024-04-02T04:42:41.190 INFO:tasks.workunit.client.0.smithi016.stdout:8/500: chown d1/d31/d67/d8f/f9f 0 1 2024-04-02T04:42:41.194 INFO:tasks.workunit.client.0.smithi016.stdout:8/501: creat d1/d6/d36/d3f/d77/fa7 x:0 0 0 2024-04-02T04:42:41.196 INFO:tasks.workunit.client.0.smithi016.stdout:9/487: dread d0/f27 [0,4194304] 0 2024-04-02T04:42:41.196 INFO:tasks.workunit.client.0.smithi016.stdout:6/570: creat d0/d9/d16/d56/faf x:0 0 0 2024-04-02T04:42:41.199 INFO:tasks.workunit.client.0.smithi016.stdout:8/502: chown d1/d6/d36/d3f/d40/d45/f5b 4 1 2024-04-02T04:42:41.200 INFO:tasks.workunit.client.0.smithi016.stdout:9/488: symlink d0/d3/d31/l90 0 2024-04-02T04:42:41.201 INFO:tasks.workunit.client.0.smithi016.stdout:9/489: stat d0/d3/d31/d53/l6b 0 2024-04-02T04:42:41.202 INFO:tasks.workunit.client.0.smithi016.stdout:2/544: getdents d12/d35/d3f/d2b/da3 0 2024-04-02T04:42:41.207 INFO:tasks.workunit.client.0.smithi016.stdout:6/571: dread d0/d9/d16/d56/d89/f2d [0,4194304] 0 2024-04-02T04:42:41.213 INFO:tasks.workunit.client.0.smithi016.stdout:2/545: symlink d12/lc4 0 2024-04-02T04:42:41.215 INFO:tasks.workunit.client.0.smithi016.stdout:9/490: rename d0/d9/c18 to d0/d3/d31/d53/c91 0 2024-04-02T04:42:41.215 INFO:tasks.workunit.client.0.smithi016.stdout:2/546: write d12/d35/d3b/d42/f95 [145458,2387] 0 2024-04-02T04:42:41.216 INFO:tasks.workunit.client.0.smithi016.stdout:6/572: dwrite d0/d9/d16/d56/f43 [0,4194304] 0 2024-04-02T04:42:41.217 INFO:tasks.workunit.client.0.smithi016.stdout:8/503: rmdir d1/d6/d36/d3f/d40/d49/d38/da1 0 2024-04-02T04:42:41.217 INFO:tasks.workunit.client.0.smithi016.stdout:6/573: chown d0/d9/df/f74 17466591 1 2024-04-02T04:42:41.220 INFO:tasks.workunit.client.0.smithi016.stdout:8/504: chown d1/d6/d36/d3f/d40/d49/d4a 962865 1 2024-04-02T04:42:41.222 INFO:tasks.workunit.client.0.smithi016.stdout:9/491: creat d0/d4e/d50/d63/d6e/d1a/d46/f92 x:0 0 0 2024-04-02T04:42:41.223 INFO:tasks.workunit.client.0.smithi016.stdout:6/574: mkdir d0/d9/d16/d56/db0 0 2024-04-02T04:42:41.224 INFO:tasks.workunit.client.0.smithi016.stdout:9/492: write d0/d9/f3d [348056,104474] 0 2024-04-02T04:42:41.224 INFO:tasks.workunit.client.0.smithi016.stdout:8/505: fsync d1/d70/f88 0 2024-04-02T04:42:41.227 INFO:tasks.workunit.client.0.smithi016.stdout:2/547: dwrite d12/d35/d3f/f7a [0,4194304] 0 2024-04-02T04:42:41.230 INFO:tasks.workunit.client.0.smithi016.stdout:9/493: creat d0/f93 x:0 0 0 2024-04-02T04:42:41.230 INFO:tasks.workunit.client.0.smithi016.stdout:9/494: chown d0/f7 24 1 2024-04-02T04:42:41.231 INFO:tasks.workunit.client.0.smithi016.stdout:9/495: dread - d0/d4e/f61 zero size 2024-04-02T04:42:41.232 INFO:tasks.workunit.client.0.smithi016.stdout:9/496: truncate d0/d4e/d50/d63/d6e/d1a/d46/f92 913531 0 2024-04-02T04:42:41.236 INFO:tasks.workunit.client.0.smithi016.stdout:6/575: write d0/d9/d16/d56/d89/f26 [125142,99982] 0 2024-04-02T04:42:41.237 INFO:tasks.workunit.client.0.smithi016.stdout:9/497: rename d0/d4e/d50/d63/d6e/d1a/d74/f79 to d0/d4a/d75/d8d/f94 0 2024-04-02T04:42:41.238 INFO:tasks.workunit.client.0.smithi016.stdout:6/576: chown d0/d9/d16/d56/d89/d67/d6f/d7e/ca7 13124 1 2024-04-02T04:42:41.240 INFO:tasks.workunit.client.0.smithi016.stdout:6/577: truncate d0/d9/d16/d56/d59/f96 482299 0 2024-04-02T04:42:41.240 INFO:tasks.workunit.client.0.smithi016.stdout:9/498: mkdir d0/d4a/d75/d95 0 2024-04-02T04:42:41.242 INFO:tasks.workunit.client.0.smithi016.stdout:9/499: creat d0/d4a/d75/f96 x:0 0 0 2024-04-02T04:42:41.244 INFO:tasks.workunit.client.0.smithi016.stdout:9/500: mkdir d0/d3/d97 0 2024-04-02T04:42:41.246 INFO:tasks.workunit.client.0.smithi016.stdout:6/578: dwrite d0/d9/d16/d56/f11 [0,4194304] 0 2024-04-02T04:42:41.247 INFO:tasks.workunit.client.0.smithi016.stdout:6/579: creat d0/d5c/fb1 x:0 0 0 2024-04-02T04:42:41.249 INFO:tasks.workunit.client.0.smithi016.stdout:6/580: write d0/d9/d16/d56/f43 [4014283,28399] 0 2024-04-02T04:42:41.252 INFO:tasks.workunit.client.0.smithi016.stdout:6/581: symlink d0/d9/d16/d56/d89/lb2 0 2024-04-02T04:42:41.253 INFO:tasks.workunit.client.0.smithi016.stdout:9/501: link d0/d4e/d50/d63/d6e/d1a/l7f d0/l98 0 2024-04-02T04:42:41.254 INFO:tasks.workunit.client.0.smithi016.stdout:6/582: read d0/d9/d16/d46/f7b [843012,29040] 0 2024-04-02T04:42:41.260 INFO:tasks.workunit.client.0.smithi016.stdout:9/502: rename d0/d4e/d50/d63/d6e/d1a to d0/d4a/d99 0 2024-04-02T04:42:41.262 INFO:tasks.workunit.client.0.smithi016.stdout:9/503: mkdir d0/d4a/d75/d95/d9a 0 2024-04-02T04:42:41.264 INFO:tasks.workunit.client.0.smithi016.stdout:9/504: mkdir d0/d4a/d75/d95/d9b 0 2024-04-02T04:42:41.266 INFO:tasks.workunit.client.0.smithi016.stdout:9/505: rename d0/d9/f89 to d0/d4a/f9c 0 2024-04-02T04:42:41.268 INFO:tasks.workunit.client.0.smithi016.stdout:9/506: write d0/f5d [876009,100061] 0 2024-04-02T04:42:41.269 INFO:tasks.workunit.client.0.smithi016.stdout:9/507: write d0/d9/db/d54/f8e [28950,110935] 0 2024-04-02T04:42:41.272 INFO:tasks.workunit.client.0.smithi016.stdout:9/508: read d0/d4a/d99/f37 [2864901,72124] 0 2024-04-02T04:42:41.272 INFO:tasks.workunit.client.0.smithi016.stdout:9/509: chown d0/d9/d4d/d78 1819692 1 2024-04-02T04:42:41.276 INFO:tasks.workunit.client.0.smithi016.stdout:9/510: truncate d0/d9/f14 1192636 0 2024-04-02T04:42:41.277 INFO:tasks.workunit.client.0.smithi016.stdout:9/511: chown d0/l3f 6593362 1 2024-04-02T04:42:41.277 INFO:tasks.workunit.client.0.smithi016.stdout:6/583: dread d0/d9/d16/f2a [4194304,4194304] 0 2024-04-02T04:42:41.279 INFO:tasks.workunit.client.0.smithi016.stdout:9/512: unlink d0/d3/l2a 0 2024-04-02T04:42:41.279 INFO:tasks.workunit.client.0.smithi016.stdout:6/584: creat d0/d9/d16/d56/d89/d67/d6f/d8a/d91/fb3 x:0 0 0 2024-04-02T04:42:41.286 INFO:tasks.workunit.client.0.smithi016.stdout:9/513: dwrite d0/d4a/f9c [0,4194304] 0 2024-04-02T04:42:41.287 INFO:tasks.workunit.client.0.smithi016.stdout:6/585: dwrite d0/d9/d16/f63 [0,4194304] 0 2024-04-02T04:42:41.289 INFO:tasks.workunit.client.0.smithi016.stdout:6/586: chown d0/d9/d16/d56/d89/d67/d6f/d8a/fa5 7074 1 2024-04-02T04:42:41.297 INFO:tasks.workunit.client.0.smithi016.stdout:6/587: mkdir d0/d9/d16/d56/d89/db4 0 2024-04-02T04:42:41.298 INFO:tasks.workunit.client.0.smithi016.stdout:6/588: read - d0/d9/d16/d56/faf zero size 2024-04-02T04:42:41.301 INFO:tasks.workunit.client.0.smithi016.stdout:6/589: mknod d0/d9/cb5 0 2024-04-02T04:42:41.302 INFO:tasks.workunit.client.0.smithi016.stdout:9/514: truncate d0/d9/f2e 3862330 0 2024-04-02T04:42:41.305 INFO:tasks.workunit.client.0.smithi016.stdout:6/590: symlink d0/d9/d16/d56/d89/d67/d6f/lb6 0 2024-04-02T04:42:41.306 INFO:tasks.workunit.client.0.smithi016.stdout:6/591: symlink d0/d6e/lb7 0 2024-04-02T04:42:41.307 INFO:tasks.workunit.client.0.smithi016.stdout:9/515: getdents d0/d33 0 2024-04-02T04:42:41.308 INFO:tasks.workunit.client.0.smithi016.stdout:6/592: write d0/d6e/f81 [415692,2339] 0 2024-04-02T04:42:41.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:41 smithi016 ceph-mon[28969]: pgmap v19: 65 pgs: 65 active+clean; 1009 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 27 MiB/s rd, 138 MiB/s wr, 445 op/s 2024-04-02T04:42:41.309 INFO:tasks.workunit.client.0.smithi016.stdout:9/516: creat d0/d3/d31/d53/f9d x:0 0 0 2024-04-02T04:42:41.313 INFO:tasks.workunit.client.0.smithi016.stdout:0/417: sync 2024-04-02T04:42:41.314 INFO:tasks.workunit.client.0.smithi016.stdout:6/593: dwrite d0/d9/d16/d56/f2c [0,4194304] 0 2024-04-02T04:42:41.317 INFO:tasks.workunit.client.0.smithi016.stdout:9/517: dwrite d0/f2c [4194304,4194304] 0 2024-04-02T04:42:41.318 INFO:tasks.workunit.client.0.smithi016.stdout:0/418: truncate d0/d48/d83/d8c/f92 725914 0 2024-04-02T04:42:41.318 INFO:tasks.workunit.client.0.smithi016.stdout:6/594: mknod d0/d9/df/cb8 0 2024-04-02T04:42:41.320 INFO:tasks.workunit.client.0.smithi016.stdout:6/595: fdatasync d0/d9/d16/d56/faf 0 2024-04-02T04:42:41.323 INFO:tasks.workunit.client.0.smithi016.stdout:9/518: write d0/d4a/d99/d46/f92 [1730674,75164] 0 2024-04-02T04:42:41.324 INFO:tasks.workunit.client.0.smithi016.stdout:6/596: truncate d0/d9/d16/f23 1117743 0 2024-04-02T04:42:41.326 INFO:tasks.workunit.client.0.smithi016.stdout:0/419: dwrite d0/d16/d2e/d53/d6f/f88 [0,4194304] 0 2024-04-02T04:42:41.328 INFO:tasks.workunit.client.0.smithi016.stdout:0/420: chown d0/d16/d24/d62/c6b 10027 1 2024-04-02T04:42:41.329 INFO:tasks.workunit.client.0.smithi016.stdout:9/519: symlink d0/d4e/d50/d63/d70/l9e 0 2024-04-02T04:42:41.330 INFO:tasks.workunit.client.0.smithi016.stdout:6/597: truncate d0/d9/f9c 827561 0 2024-04-02T04:42:41.331 INFO:tasks.workunit.client.0.smithi016.stdout:6/598: fsync d0/d9/d16/d56/d89/f85 0 2024-04-02T04:42:41.331 INFO:tasks.workunit.client.0.smithi016.stdout:0/421: unlink d0/d16/l55 0 2024-04-02T04:42:41.332 INFO:tasks.workunit.client.0.smithi016.stdout:6/599: chown d0/d9/d16/d56/d59/f60 125 1 2024-04-02T04:42:41.332 INFO:tasks.workunit.client.0.smithi016.stdout:6/600: readlink d0/l66 0 2024-04-02T04:42:41.333 INFO:tasks.workunit.client.0.smithi016.stdout:9/520: creat d0/d4e/d50/d63/d6e/d4b/d58/d6a/f9f x:0 0 0 2024-04-02T04:42:41.334 INFO:tasks.workunit.client.0.smithi016.stdout:0/422: mknod d0/d16/d24/d4e/c97 0 2024-04-02T04:42:41.335 INFO:tasks.workunit.client.0.smithi016.stdout:6/601: truncate d0/d9/d16/d56/d89/d67/d6f/d8a/d91/fb3 617036 0 2024-04-02T04:42:41.336 INFO:tasks.workunit.client.0.smithi016.stdout:0/423: truncate d0/d16/d2e/d53/f7c 1684361 0 2024-04-02T04:42:41.339 INFO:tasks.workunit.client.0.smithi016.stdout:6/602: link d0/d9/d16/d56/d59/f4b d0/d9/df/fb9 0 2024-04-02T04:42:41.340 INFO:tasks.workunit.client.0.smithi016.stdout:6/603: chown d0/d9/d16/d56/c79 8407 1 2024-04-02T04:42:41.345 INFO:tasks.workunit.client.0.smithi016.stdout:6/604: dwrite d0/d9/d16/f47 [0,4194304] 0 2024-04-02T04:42:41.350 INFO:tasks.workunit.client.0.smithi016.stdout:0/424: fdatasync d0/d16/d2e/d53/d6f/f88 0 2024-04-02T04:42:41.351 INFO:tasks.workunit.client.0.smithi016.stdout:0/425: write d0/d16/d2e/f61 [4715351,28599] 0 2024-04-02T04:42:41.355 INFO:tasks.workunit.client.0.smithi016.stdout:0/426: creat d0/d16/d23/d3e/f98 x:0 0 0 2024-04-02T04:42:41.355 INFO:tasks.workunit.client.0.smithi016.stdout:9/521: rmdir d0/d4e/d50/d63/d6e 39 2024-04-02T04:42:41.355 INFO:tasks.workunit.client.0.smithi016.stdout:0/427: chown d0/l5 7771469 1 2024-04-02T04:42:41.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:41 smithi027 ceph-mon[37550]: pgmap v19: 65 pgs: 65 active+clean; 1009 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 27 MiB/s rd, 138 MiB/s wr, 445 op/s 2024-04-02T04:42:41.422 INFO:tasks.workunit.client.0.smithi016.stdout:0/428: dwrite d0/d16/d24/f69 [0,4194304] 0 2024-04-02T04:42:41.424 INFO:tasks.workunit.client.0.smithi016.stdout:0/429: mknod d0/d16/d24/d4e/d8d/c99 0 2024-04-02T04:42:41.429 INFO:tasks.workunit.client.0.smithi016.stdout:0/430: dwrite d0/f67 [0,4194304] 0 2024-04-02T04:42:41.452 INFO:tasks.workunit.client.0.smithi016.stdout:0/431: dread d0/f12 [0,4194304] 0 2024-04-02T04:42:41.454 INFO:tasks.workunit.client.0.smithi016.stdout:0/432: mkdir d0/d16/d23/d9a 0 2024-04-02T04:42:41.479 INFO:tasks.workunit.client.0.smithi016.stdout:4/515: sync 2024-04-02T04:42:41.481 INFO:tasks.workunit.client.0.smithi016.stdout:0/433: creat d0/d16/d24/d62/f9b x:0 0 0 2024-04-02T04:42:41.481 INFO:tasks.workunit.client.0.smithi016.stdout:0/434: chown d0/d16/d24/d4e/d8d/c99 32248335 1 2024-04-02T04:42:41.482 INFO:tasks.workunit.client.0.smithi016.stdout:4/516: creat d5/d1d/d1e/f9d x:0 0 0 2024-04-02T04:42:41.483 INFO:tasks.workunit.client.0.smithi016.stdout:0/435: write d0/d16/d2e/d36/f8b [388727,101986] 0 2024-04-02T04:42:41.486 INFO:tasks.workunit.client.0.smithi016.stdout:0/436: rename d0/d48/d83 to d0/d16/d24/d62/d9c 0 2024-04-02T04:42:41.489 INFO:tasks.workunit.client.0.smithi016.stdout:0/437: rename d0/d16/d2e/d37/d87 to d0/d16/d2e/d53/d9d 0 2024-04-02T04:42:41.492 INFO:tasks.workunit.client.0.smithi016.stdout:4/517: write d5/de/f79 [300539,5910] 0 2024-04-02T04:42:41.495 INFO:tasks.workunit.client.0.smithi016.stdout:4/518: mknod d5/d1d/d5f/d95/c9e 0 2024-04-02T04:42:41.496 INFO:tasks.workunit.client.0.smithi016.stdout:0/438: write d0/d16/f94 [23570,36073] 0 2024-04-02T04:42:41.496 INFO:tasks.workunit.client.0.smithi016.stdout:4/519: fdatasync d5/f67 0 2024-04-02T04:42:41.500 INFO:tasks.workunit.client.0.smithi016.stdout:0/439: getdents d0/d16/d23/d3e 0 2024-04-02T04:42:41.501 INFO:tasks.workunit.client.0.smithi016.stdout:4/520: write d5/d8/f7b [174161,10884] 0 2024-04-02T04:42:41.502 INFO:tasks.workunit.client.0.smithi016.stdout:0/440: rename d0/f12 to d0/d4f/f9e 0 2024-04-02T04:42:41.503 INFO:tasks.workunit.client.0.smithi016.stdout:0/441: chown d0/d16/d24/d4e/c8f 18748 1 2024-04-02T04:42:41.506 INFO:tasks.workunit.client.0.smithi016.stdout:0/442: mkdir d0/d16/d2e/d36/d9f 0 2024-04-02T04:42:41.506 INFO:tasks.workunit.client.0.smithi016.stdout:4/521: dread d5/d17/d18/d1f/f90 [0,4194304] 0 2024-04-02T04:42:41.506 INFO:tasks.workunit.client.0.smithi016.stdout:0/443: truncate d0/d16/d23/f3f 1370133 0 2024-04-02T04:42:41.507 INFO:tasks.workunit.client.0.smithi016.stdout:0/444: chown d0/d16/d24/c46 30756 1 2024-04-02T04:42:41.508 INFO:tasks.workunit.client.0.smithi016.stdout:4/522: mknod d5/d17/d18/d6d/d8e/c9f 0 2024-04-02T04:42:41.508 INFO:tasks.workunit.client.0.smithi016.stdout:4/523: chown d5/d16/f55 3 1 2024-04-02T04:42:41.509 INFO:tasks.workunit.client.0.smithi016.stdout:4/524: readlink d5/d17/d76/l97 0 2024-04-02T04:42:41.512 INFO:tasks.workunit.client.0.smithi016.stdout:4/525: unlink d5/de/c31 0 2024-04-02T04:42:41.513 INFO:tasks.workunit.client.0.smithi016.stdout:0/445: getdents d0/d16/d2e/d37/d66 0 2024-04-02T04:42:41.517 INFO:tasks.workunit.client.0.smithi016.stdout:4/526: fsync d5/d17/d18/d25/f57 0 2024-04-02T04:42:41.520 INFO:tasks.workunit.client.0.smithi016.stdout:0/446: getdents d0/d4f/d76 0 2024-04-02T04:42:41.522 INFO:tasks.workunit.client.0.smithi016.stdout:4/527: mkdir d5/d17/d18/da0 0 2024-04-02T04:42:41.525 INFO:tasks.workunit.client.0.smithi016.stdout:0/447: dread d0/d16/d24/f7b [0,4194304] 0 2024-04-02T04:42:41.526 INFO:tasks.workunit.client.0.smithi016.stdout:4/528: symlink d5/d1d/d1e/la1 0 2024-04-02T04:42:41.532 INFO:tasks.workunit.client.0.smithi016.stdout:0/448: dwrite d0/d16/d24/d4e/d8d/f75 [4194304,4194304] 0 2024-04-02T04:42:41.533 INFO:tasks.workunit.client.0.smithi016.stdout:4/529: dwrite d5/d1d/f21 [4194304,4194304] 0 2024-04-02T04:42:41.535 INFO:tasks.workunit.client.0.smithi016.stdout:5/476: sync 2024-04-02T04:42:41.536 INFO:tasks.workunit.client.0.smithi016.stdout:3/541: sync 2024-04-02T04:42:41.536 INFO:tasks.workunit.client.0.smithi016.stdout:1/479: sync 2024-04-02T04:42:41.536 INFO:tasks.workunit.client.0.smithi016.stdout:4/530: write d5/d16/d88/f96 [612657,266] 0 2024-04-02T04:42:41.536 INFO:tasks.workunit.client.0.smithi016.stdout:7/493: sync 2024-04-02T04:42:41.536 INFO:tasks.workunit.client.0.smithi016.stdout:2/548: sync 2024-04-02T04:42:41.536 INFO:tasks.workunit.client.0.smithi016.stdout:8/506: sync 2024-04-02T04:42:41.537 INFO:tasks.workunit.client.0.smithi016.stdout:7/494: readlink d0/l88 0 2024-04-02T04:42:41.541 INFO:tasks.workunit.client.0.smithi016.stdout:4/531: readlink d5/d1d/d6c/l9b 0 2024-04-02T04:42:41.541 INFO:tasks.workunit.client.0.smithi016.stdout:8/507: fsync d1/d6/d36/d3f/f6c 0 2024-04-02T04:42:41.541 INFO:tasks.workunit.client.0.smithi016.stdout:1/480: fsync d14/d19/f6b 0 2024-04-02T04:42:41.542 INFO:tasks.workunit.client.0.smithi016.stdout:3/542: write d2/d15/d16/d8a/f98 [152244,121072] 0 2024-04-02T04:42:41.543 INFO:tasks.workunit.client.0.smithi016.stdout:4/532: write d5/d1d/f37 [1219653,65737] 0 2024-04-02T04:42:41.544 INFO:tasks.workunit.client.0.smithi016.stdout:3/543: chown d2/d2e/f30 0 1 2024-04-02T04:42:41.548 INFO:tasks.workunit.client.0.smithi016.stdout:5/477: rmdir d3/d40/d4f/d6b/d2a 39 2024-04-02T04:42:41.550 INFO:tasks.workunit.client.0.smithi016.stdout:5/478: chown d3/df/d18/f35 3 1 2024-04-02T04:42:41.551 INFO:tasks.workunit.client.0.smithi016.stdout:3/544: dwrite d2/d15/d16/d8a/f98 [0,4194304] 0 2024-04-02T04:42:41.552 INFO:tasks.workunit.client.0.smithi016.stdout:3/545: write d2/d15/d7e/fa9 [847940,76027] 0 2024-04-02T04:42:41.553 INFO:tasks.workunit.client.0.smithi016.stdout:3/546: chown d2/d15/d16/d3d/d46/l87 54833746 1 2024-04-02T04:42:41.560 INFO:tasks.workunit.client.0.smithi016.stdout:1/481: creat d14/d19/d1d/d45/d8a/f9b x:0 0 0 2024-04-02T04:42:41.562 INFO:tasks.workunit.client.0.smithi016.stdout:8/508: creat d1/d6/d36/d3f/d40/d49/d4a/fa8 x:0 0 0 2024-04-02T04:42:41.563 INFO:tasks.workunit.client.0.smithi016.stdout:1/482: dread d14/d19/d1e/d71/f78 [0,4194304] 0 2024-04-02T04:42:41.563 INFO:tasks.workunit.client.0.smithi016.stdout:4/533: symlink d5/de/la2 0 2024-04-02T04:42:41.565 INFO:tasks.workunit.client.0.smithi016.stdout:0/449: dwrite d0/d16/f31 [0,4194304] 0 2024-04-02T04:42:41.565 INFO:tasks.workunit.client.0.smithi016.stdout:1/483: write d14/d19/d1d/f1f [3563331,89388] 0 2024-04-02T04:42:41.569 INFO:tasks.workunit.client.0.smithi016.stdout:5/479: mknod d3/d40/d72/cb3 0 2024-04-02T04:42:41.571 INFO:tasks.workunit.client.0.smithi016.stdout:3/547: mknod d2/d15/d1a/cac 0 2024-04-02T04:42:41.577 INFO:tasks.workunit.client.0.smithi016.stdout:6/605: sync 2024-04-02T04:42:41.577 INFO:tasks.workunit.client.0.smithi016.stdout:9/522: sync 2024-04-02T04:42:41.579 INFO:tasks.workunit.client.0.smithi016.stdout:7/495: write d0/d46/f2f [179454,85843] 0 2024-04-02T04:42:41.580 INFO:tasks.workunit.client.0.smithi016.stdout:7/496: write d0/d46/f2f [709805,127074] 0 2024-04-02T04:42:41.582 INFO:tasks.workunit.client.0.smithi016.stdout:6/606: dread d0/d9/d16/d56/d89/d67/d6f/f82 [0,4194304] 0 2024-04-02T04:42:41.582 INFO:tasks.workunit.client.0.smithi016.stdout:1/484: mkdir d14/d19/d1e/d71/d9c 0 2024-04-02T04:42:41.587 INFO:tasks.workunit.client.0.smithi016.stdout:2/549: write d12/d35/f40 [1427504,122268] 0 2024-04-02T04:42:41.588 INFO:tasks.workunit.client.0.smithi016.stdout:3/548: symlink d2/d15/d16/d3d/d46/lad 0 2024-04-02T04:42:41.593 INFO:tasks.workunit.client.0.smithi016.stdout:9/523: creat d0/d3/d31/fa0 x:0 0 0 2024-04-02T04:42:41.602 INFO:tasks.workunit.client.0.smithi016.stdout:6/607: dread d0/d9/f9c [0,4194304] 0 2024-04-02T04:42:41.603 INFO:tasks.workunit.client.0.smithi016.stdout:5/480: write d3/df/d18/d2f/f3f [846547,18036] 0 2024-04-02T04:42:41.603 INFO:tasks.workunit.client.0.smithi016.stdout:5/481: chown d3/d40 34 1 2024-04-02T04:42:41.604 INFO:tasks.workunit.client.0.smithi016.stdout:1/485: rename d14/d19/d1d/c36 to d14/d19/d7a/c9d 0 2024-04-02T04:42:41.606 INFO:tasks.workunit.client.0.smithi016.stdout:0/450: creat d0/d16/d2e/d36/d9f/fa0 x:0 0 0 2024-04-02T04:42:41.609 INFO:tasks.workunit.client.0.smithi016.stdout:5/482: dwrite d3/df/d18/f35 [0,4194304] 0 2024-04-02T04:42:41.609 INFO:tasks.workunit.client.0.smithi016.stdout:3/549: symlink d2/d15/d1a/d45/lae 0 2024-04-02T04:42:41.610 INFO:tasks.workunit.client.0.smithi016.stdout:5/483: chown d3/f3b 63045729 1 2024-04-02T04:42:41.613 INFO:tasks.workunit.client.0.smithi016.stdout:8/509: dwrite d1/d70/f84 [0,4194304] 0 2024-04-02T04:42:41.615 INFO:tasks.workunit.client.0.smithi016.stdout:8/510: truncate d1/d6/d36/fa3 963086 0 2024-04-02T04:42:41.619 INFO:tasks.workunit.client.0.smithi016.stdout:0/451: rmdir d0/d16/d2e/d37/d66 39 2024-04-02T04:42:41.621 INFO:tasks.workunit.client.0.smithi016.stdout:3/550: rmdir d2/d15 39 2024-04-02T04:42:41.643 INFO:tasks.workunit.client.0.smithi016.stdout:5/484: link d3/d4c/fa9 d3/df/d18/d2f/fb4 0 2024-04-02T04:42:41.645 INFO:tasks.workunit.client.0.smithi016.stdout:7/497: getdents d0/d46/d37/d47 0 2024-04-02T04:42:41.648 INFO:tasks.workunit.client.0.smithi016.stdout:7/498: dread d0/d21/f3c [0,4194304] 0 2024-04-02T04:42:41.651 INFO:tasks.workunit.client.0.smithi016.stdout:1/486: write d14/d19/d20/f3f [499834,87243] 0 2024-04-02T04:42:41.653 INFO:tasks.workunit.client.0.smithi016.stdout:3/551: creat d2/d2e/d36/d66/d90/faf x:0 0 0 2024-04-02T04:42:41.653 INFO:tasks.workunit.client.0.smithi016.stdout:4/534: dwrite d5/d17/d18/d25/f3b [0,4194304] 0 2024-04-02T04:42:41.654 INFO:tasks.workunit.client.0.smithi016.stdout:3/552: chown d2/c3a 48 1 2024-04-02T04:42:41.654 INFO:tasks.workunit.client.0.smithi016.stdout:1/487: chown d14/d41/d43/f49 4136 1 2024-04-02T04:42:41.655 INFO:tasks.workunit.client.0.smithi016.stdout:5/485: write d3/df/f57 [1263603,4918] 0 2024-04-02T04:42:41.656 INFO:tasks.workunit.client.0.smithi016.stdout:6/608: rename d0/d9/d16/l50 to d0/d9/d16/d56/lba 0 2024-04-02T04:42:41.657 INFO:tasks.workunit.client.0.smithi016.stdout:1/488: write d14/d41/f47 [528983,9311] 0 2024-04-02T04:42:41.658 INFO:tasks.workunit.client.0.smithi016.stdout:7/499: creat d0/d46/d52/d82/fb3 x:0 0 0 2024-04-02T04:42:41.661 INFO:tasks.workunit.client.0.smithi016.stdout:4/535: mknod d5/d17/d18/d6d/d8e/ca3 0 2024-04-02T04:42:41.662 INFO:tasks.workunit.client.0.smithi016.stdout:9/524: dwrite d0/d9/f14 [0,4194304] 0 2024-04-02T04:42:41.664 INFO:tasks.workunit.client.0.smithi016.stdout:0/452: getdents d0/d16/d24/d4e/d8d 0 2024-04-02T04:42:41.670 INFO:tasks.workunit.client.0.smithi016.stdout:3/553: rename d2/d15/d16/d3d/d46 to d2/d15/d16/d3d/d6e/d6f/db0 0 2024-04-02T04:42:41.670 INFO:tasks.workunit.client.0.smithi016.stdout:6/609: dread d0/d6e/f81 [0,4194304] 0 2024-04-02T04:42:41.671 INFO:tasks.workunit.client.0.smithi016.stdout:1/489: rename d14/d19/d1d/d29/d39 to d14/d19/d1d/d29/d39/d4d/d9e 22 2024-04-02T04:42:41.672 INFO:tasks.workunit.client.0.smithi016.stdout:8/511: truncate d1/d6/d36/f37 3098991 0 2024-04-02T04:42:41.672 INFO:tasks.workunit.client.0.smithi016.stdout:9/525: mknod d0/d3/d31/ca1 0 2024-04-02T04:42:41.673 INFO:tasks.workunit.client.0.smithi016.stdout:1/490: dread - d14/d19/d1d/d45/d8a/f9b zero size 2024-04-02T04:42:41.674 INFO:tasks.workunit.client.0.smithi016.stdout:5/486: getdents d3/df/d18 0 2024-04-02T04:42:41.675 INFO:tasks.workunit.client.0.smithi016.stdout:9/526: mknod d0/d4e/d50/ca2 0 2024-04-02T04:42:41.711 INFO:tasks.workunit.client.0.smithi016.stdout:1/491: dread - d14/d19/d1e/d71/f84 zero size 2024-04-02T04:42:41.711 INFO:tasks.workunit.client.0.smithi016.stdout:8/512: rename d1/d6/d36/d3f/d40/d49/d38/f41 to d1/d6/d86/fa9 0 2024-04-02T04:42:41.712 INFO:tasks.workunit.client.0.smithi016.stdout:1/492: readlink d14/l2f 0 2024-04-02T04:42:41.714 INFO:tasks.workunit.client.0.smithi016.stdout:3/554: dwrite d2/d15/d70/f79 [0,4194304] 0 2024-04-02T04:42:41.715 INFO:tasks.workunit.client.0.smithi016.stdout:9/527: creat d0/d3/d31/d53/fa3 x:0 0 0 2024-04-02T04:42:41.719 INFO:tasks.workunit.client.0.smithi016.stdout:1/493: rename d14/d19/d20/c73 to d14/d19/d1e/c9f 0 2024-04-02T04:42:41.721 INFO:tasks.workunit.client.0.smithi016.stdout:4/536: dread d5/d17/d18/d25/f4a [0,4194304] 0 2024-04-02T04:42:41.722 INFO:tasks.workunit.client.0.smithi016.stdout:9/528: rmdir d0/d9/db/d54 39 2024-04-02T04:42:41.723 INFO:tasks.workunit.client.0.smithi016.stdout:1/494: unlink d14/d19/d50/c6e 0 2024-04-02T04:42:41.727 INFO:tasks.workunit.client.0.smithi016.stdout:3/555: rename d2/d15/d16/d3d/d6e/d6f/la3 to d2/d15/d16/d8a/lb1 0 2024-04-02T04:42:41.727 INFO:tasks.workunit.client.0.smithi016.stdout:8/513: dwrite d1/d6/d36/d3f/d40/d49/d38/f90 [0,4194304] 0 2024-04-02T04:42:41.730 INFO:tasks.workunit.client.0.smithi016.stdout:1/495: fdatasync d14/d19/d1e/f2b 0 2024-04-02T04:42:41.732 INFO:tasks.workunit.client.0.smithi016.stdout:5/487: dwrite d3/d40/d4f/d6b/d2a/d6d/fa7 [8388608,4194304] 0 2024-04-02T04:42:41.735 INFO:tasks.workunit.client.0.smithi016.stdout:1/496: mknod d14/d19/d50/ca0 0 2024-04-02T04:42:41.737 INFO:tasks.workunit.client.0.smithi016.stdout:2/550: sync 2024-04-02T04:42:41.738 INFO:tasks.workunit.client.0.smithi016.stdout:5/488: symlink d3/df/d93/lb5 0 2024-04-02T04:42:41.738 INFO:tasks.workunit.client.0.smithi016.stdout:1/497: readlink d14/d19/d50/l97 0 2024-04-02T04:42:41.744 INFO:tasks.workunit.client.0.smithi016.stdout:5/489: mkdir d3/d40/db6 0 2024-04-02T04:42:41.747 INFO:tasks.workunit.client.0.smithi016.stdout:3/556: dwrite d2/d15/f4b [4194304,4194304] 0 2024-04-02T04:42:41.747 INFO:tasks.workunit.client.0.smithi016.stdout:8/514: dwrite d1/d6/d36/d3f/f6c [0,4194304] 0 2024-04-02T04:42:41.749 INFO:tasks.workunit.client.0.smithi016.stdout:9/529: dwrite d0/d4e/d50/d63/d6e/d4b/d58/d6a/f9f [0,4194304] 0 2024-04-02T04:42:41.751 INFO:tasks.workunit.client.0.smithi016.stdout:1/498: fdatasync d14/d19/d1d/d45/d8a/f9b 0 2024-04-02T04:42:41.752 INFO:tasks.workunit.client.0.smithi016.stdout:2/551: creat d12/d9f/dac/d76/da0/dbe/fc5 x:0 0 0 2024-04-02T04:42:41.755 INFO:tasks.workunit.client.0.smithi016.stdout:5/490: symlink d3/df/d18/d26/d48/lb7 0 2024-04-02T04:42:41.756 INFO:tasks.workunit.client.0.smithi016.stdout:9/530: creat d0/d3/d31/fa4 x:0 0 0 2024-04-02T04:42:41.757 INFO:tasks.workunit.client.0.smithi016.stdout:4/537: dwrite d5/d1d/d6c/f56 [0,4194304] 0 2024-04-02T04:42:41.759 INFO:tasks.workunit.client.0.smithi016.stdout:1/499: mknod d14/d19/d7a/ca1 0 2024-04-02T04:42:41.760 INFO:tasks.workunit.client.0.smithi016.stdout:1/500: chown d14/d19/d1d/d45/d4f 212581 1 2024-04-02T04:42:41.763 INFO:tasks.workunit.client.0.smithi016.stdout:5/491: read d3/f6 [3388936,69754] 0 2024-04-02T04:42:41.763 INFO:tasks.workunit.client.0.smithi016.stdout:5/492: truncate d3/fad 579225 0 2024-04-02T04:42:41.766 INFO:tasks.workunit.client.0.smithi016.stdout:4/538: mkdir d5/d17/d18/d6d/da4 0 2024-04-02T04:42:41.767 INFO:tasks.workunit.client.0.smithi016.stdout:1/501: mkdir d14/d19/d7a/da2 0 2024-04-02T04:42:41.771 INFO:tasks.workunit.client.0.smithi016.stdout:8/515: creat d1/d6/d36/d3f/d40/d49/faa x:0 0 0 2024-04-02T04:42:41.771 INFO:tasks.workunit.client.0.smithi016.stdout:4/539: mkdir d5/d1d/d1e/da5 0 2024-04-02T04:42:41.772 INFO:tasks.workunit.client.0.smithi016.stdout:9/531: mknod d0/d4e/d50/d63/d6e/ca5 0 2024-04-02T04:42:41.772 INFO:tasks.workunit.client.0.smithi016.stdout:4/540: readlink d5/d17/d18/d1f/l73 0 2024-04-02T04:42:41.773 INFO:tasks.workunit.client.0.smithi016.stdout:3/557: link d2/d15/d1a/cac d2/d2e/d36/d66/d7a/cb2 0 2024-04-02T04:42:41.774 INFO:tasks.workunit.client.0.smithi016.stdout:2/552: link d12/d35/d3f/d56/dab/l5d d12/d9f/dac/d76/lc6 0 2024-04-02T04:42:41.775 INFO:tasks.workunit.client.0.smithi016.stdout:5/493: symlink d3/d40/d4f/d9c/lb8 0 2024-04-02T04:42:41.776 INFO:tasks.workunit.client.0.smithi016.stdout:9/532: mkdir d0/d9/d4d/d78/da6 0 2024-04-02T04:42:41.778 INFO:tasks.workunit.client.0.smithi016.stdout:5/494: creat d3/df/fb9 x:0 0 0 2024-04-02T04:42:41.780 INFO:tasks.workunit.client.0.smithi016.stdout:9/533: creat d0/d3/fa7 x:0 0 0 2024-04-02T04:42:41.782 INFO:tasks.workunit.client.0.smithi016.stdout:9/534: fsync d0/f2d 0 2024-04-02T04:42:41.782 INFO:tasks.workunit.client.0.smithi016.stdout:4/541: dwrite d5/d8/fd [0,4194304] 0 2024-04-02T04:42:41.783 INFO:tasks.workunit.client.0.smithi016.stdout:5/495: creat d3/df/d18/fba x:0 0 0 2024-04-02T04:42:41.785 INFO:tasks.workunit.client.0.smithi016.stdout:5/496: creat d3/df/fbb x:0 0 0 2024-04-02T04:42:41.786 INFO:tasks.workunit.client.0.smithi016.stdout:9/535: rmdir d0/d9/d4d/d78 39 2024-04-02T04:42:41.788 INFO:tasks.workunit.client.0.smithi016.stdout:3/558: dread d2/d15/d1a/f69 [0,4194304] 0 2024-04-02T04:42:41.793 INFO:tasks.workunit.client.0.smithi016.stdout:3/559: chown d2/d15/f4a 13742 1 2024-04-02T04:42:41.795 INFO:tasks.workunit.client.0.smithi016.stdout:3/560: truncate d2/d2e/d36/d48/f91 1037249 0 2024-04-02T04:42:41.795 INFO:tasks.workunit.client.0.smithi016.stdout:3/561: chown d2/d2e 16 1 2024-04-02T04:42:41.799 INFO:tasks.workunit.client.0.smithi016.stdout:3/562: symlink d2/d2e/d36/lb3 0 2024-04-02T04:42:41.800 INFO:tasks.workunit.client.0.smithi016.stdout:2/553: dwrite d12/d35/d3b/d42/d59/fba [0,4194304] 0 2024-04-02T04:42:41.800 INFO:tasks.workunit.client.0.smithi016.stdout:3/563: fsync d2/d15/d16/d8a/f98 0 2024-04-02T04:42:41.805 INFO:tasks.workunit.client.0.smithi016.stdout:5/497: link d3/df/d18/d26/f8c d3/d40/d4f/d6b/fbc 0 2024-04-02T04:42:41.805 INFO:tasks.workunit.client.0.smithi016.stdout:1/502: dread d14/d19/d1e/f2e [0,4194304] 0 2024-04-02T04:42:41.806 INFO:tasks.workunit.client.0.smithi016.stdout:2/554: truncate d12/d26/f5a 999902 0 2024-04-02T04:42:41.807 INFO:tasks.workunit.client.0.smithi016.stdout:2/555: chown d12/d9f/dac 128 1 2024-04-02T04:42:41.809 INFO:tasks.workunit.client.0.smithi016.stdout:1/503: symlink d14/d19/d50/la3 0 2024-04-02T04:42:41.810 INFO:tasks.workunit.client.0.smithi016.stdout:1/504: fdatasync d14/d19/d1e/d71/f84 0 2024-04-02T04:42:41.810 INFO:tasks.workunit.client.0.smithi016.stdout:2/556: creat d12/d35/d3b/fc7 x:0 0 0 2024-04-02T04:42:41.811 INFO:tasks.workunit.client.0.smithi016.stdout:2/557: dread - d12/d35/d3f/d2b/da3/fb9 zero size 2024-04-02T04:42:41.811 INFO:tasks.workunit.client.0.smithi016.stdout:5/498: mkdir d3/dbd 0 2024-04-02T04:42:41.813 INFO:tasks.workunit.client.0.smithi016.stdout:2/558: truncate d12/d35/d3b/f54 97329 0 2024-04-02T04:42:41.813 INFO:tasks.workunit.client.0.smithi016.stdout:5/499: fsync d3/df/d18/d26/d48/f49 0 2024-04-02T04:42:41.813 INFO:tasks.workunit.client.0.smithi016.stdout:3/564: creat d2/d15/fb4 x:0 0 0 2024-04-02T04:42:41.814 INFO:tasks.workunit.client.0.smithi016.stdout:1/505: symlink d14/d19/d1e/d5e/la4 0 2024-04-02T04:42:41.816 INFO:tasks.workunit.client.0.smithi016.stdout:3/565: write d2/f3e [731851,95674] 0 2024-04-02T04:42:41.816 INFO:tasks.workunit.client.0.smithi016.stdout:1/506: rename d14 to d14/d19/da5 22 2024-04-02T04:42:41.818 INFO:tasks.workunit.client.0.smithi016.stdout:5/500: unlink d3/df/d18/d26/d2c/c46 0 2024-04-02T04:42:41.818 INFO:tasks.workunit.client.0.smithi016.stdout:9/536: dread d0/d3/f30 [0,4194304] 0 2024-04-02T04:42:41.818 INFO:tasks.workunit.client.0.smithi016.stdout:1/507: write d14/f17 [9015737,38380] 0 2024-04-02T04:42:41.818 INFO:tasks.workunit.client.0.smithi016.stdout:4/542: dread d5/d16/d3f/f64 [0,4194304] 0 2024-04-02T04:42:41.819 INFO:tasks.workunit.client.0.smithi016.stdout:2/559: truncate d12/d35/f5c 861279 0 2024-04-02T04:42:41.820 INFO:tasks.workunit.client.0.smithi016.stdout:3/566: symlink d2/d15/d1a/d45/da5/lb5 0 2024-04-02T04:42:41.821 INFO:tasks.workunit.client.0.smithi016.stdout:9/537: chown d0/d3/f40 5841936 1 2024-04-02T04:42:41.822 INFO:tasks.workunit.client.0.smithi016.stdout:3/567: stat d2/d15/d1a/d45/da5/lb5 0 2024-04-02T04:42:41.825 INFO:tasks.workunit.client.0.smithi016.stdout:0/453: sync 2024-04-02T04:42:41.825 INFO:tasks.workunit.client.0.smithi016.stdout:7/500: sync 2024-04-02T04:42:41.825 INFO:tasks.workunit.client.0.smithi016.stdout:1/508: creat d14/d19/d1d/d45/d4f/d86/fa6 x:0 0 0 2024-04-02T04:42:41.826 INFO:tasks.workunit.client.0.smithi016.stdout:2/560: mknod d12/d35/d3f/d2b/da3/db1/cc8 0 2024-04-02T04:42:41.827 INFO:tasks.workunit.client.0.smithi016.stdout:3/568: rmdir d2/d2e/d36/d48/da2 39 2024-04-02T04:42:41.828 INFO:tasks.workunit.client.0.smithi016.stdout:0/454: truncate d0/d16/f54 145435 0 2024-04-02T04:42:41.828 INFO:tasks.workunit.client.0.smithi016.stdout:4/543: dwrite d5/d1d/d5f/d65/d6f/f84 [0,4194304] 0 2024-04-02T04:42:41.829 INFO:tasks.workunit.client.0.smithi016.stdout:3/569: chown d2/d2e/d36 516117852 1 2024-04-02T04:42:41.831 INFO:tasks.workunit.client.0.smithi016.stdout:9/538: mkdir d0/d9/d4d/da8 0 2024-04-02T04:42:41.832 INFO:tasks.workunit.client.0.smithi016.stdout:2/561: creat d12/d9f/fc9 x:0 0 0 2024-04-02T04:42:41.832 INFO:tasks.workunit.client.0.smithi016.stdout:0/455: dread d0/d16/d24/d62/d9c/f5b [0,4194304] 0 2024-04-02T04:42:41.832 INFO:tasks.workunit.client.0.smithi016.stdout:5/501: write d3/d40/d4f/d6b/d45/f51 [571027,14981] 0 2024-04-02T04:42:41.833 INFO:tasks.workunit.client.0.smithi016.stdout:9/539: stat d0/d3/d31/fa4 0 2024-04-02T04:42:41.834 INFO:tasks.workunit.client.0.smithi016.stdout:7/501: rename d0/d46/d37/f81 to d0/d46/d37/d47/fb4 0 2024-04-02T04:42:41.837 INFO:tasks.workunit.client.0.smithi016.stdout:9/540: write d0/d3/d31/f57 [970978,70481] 0 2024-04-02T04:42:41.837 INFO:tasks.workunit.client.0.smithi016.stdout:7/502: write d0/d21/f65 [2362725,4630] 0 2024-04-02T04:42:41.838 INFO:tasks.workunit.client.0.smithi016.stdout:4/544: dwrite d5/d16/f49 [0,4194304] 0 2024-04-02T04:42:41.839 INFO:tasks.workunit.client.0.smithi016.stdout:2/562: rmdir d12/d35/d3b/d73 39 2024-04-02T04:42:41.840 INFO:tasks.workunit.client.0.smithi016.stdout:7/503: chown d0/d46/c14 2079 1 2024-04-02T04:42:41.840 INFO:tasks.workunit.client.0.smithi016.stdout:3/570: write d2/d15/d16/d3d/d6e/d6f/db0/f62 [749167,55775] 0 2024-04-02T04:42:41.840 INFO:tasks.workunit.client.0.smithi016.stdout:5/502: mknod d3/d5d/cbe 0 2024-04-02T04:42:41.842 INFO:tasks.workunit.client.0.smithi016.stdout:1/509: creat d14/d19/fa7 x:0 0 0 2024-04-02T04:42:41.843 INFO:tasks.workunit.client.0.smithi016.stdout:9/541: mkdir d0/d4a/da9 0 2024-04-02T04:42:41.844 INFO:tasks.workunit.client.0.smithi016.stdout:2/563: chown d12/d35/d3b/d42/c89 172304 1 2024-04-02T04:42:41.845 INFO:tasks.workunit.client.0.smithi016.stdout:3/571: dread - d2/d15/d16/d3d/d6e/d6f/db0/f9e zero size 2024-04-02T04:42:41.846 INFO:tasks.workunit.client.0.smithi016.stdout:2/564: stat d12/d9f/dac/d4f/f5f 0 2024-04-02T04:42:41.846 INFO:tasks.workunit.client.0.smithi016.stdout:5/503: mkdir d3/d40/d4f/d67/dbf 0 2024-04-02T04:42:41.847 INFO:tasks.workunit.client.0.smithi016.stdout:0/456: rmdir d0/d16/d23/d9a 0 2024-04-02T04:42:41.848 INFO:tasks.workunit.client.0.smithi016.stdout:2/565: chown d12/d35/d3f/f4b 130668169 1 2024-04-02T04:42:41.848 INFO:tasks.workunit.client.0.smithi016.stdout:0/457: readlink d0/d16/d23/l26 0 2024-04-02T04:42:41.851 INFO:tasks.workunit.client.0.smithi016.stdout:0/458: chown d0/d16/d2e/d37/l81 4400 1 2024-04-02T04:42:41.851 INFO:tasks.workunit.client.0.smithi016.stdout:3/572: dread d2/d15/d16/d3d/d6e/d6f/db0/f62 [0,4194304] 0 2024-04-02T04:42:41.852 INFO:tasks.workunit.client.0.smithi016.stdout:0/459: fsync d0/f32 0 2024-04-02T04:42:41.852 INFO:tasks.workunit.client.0.smithi016.stdout:9/542: mknod d0/d3/d31/d53/d7e/caa 0 2024-04-02T04:42:41.854 INFO:tasks.workunit.client.0.smithi016.stdout:0/460: fsync d0/f32 0 2024-04-02T04:42:41.854 INFO:tasks.workunit.client.0.smithi016.stdout:5/504: creat d3/d40/d4f/d6b/d2a/d6d/fc0 x:0 0 0 2024-04-02T04:42:41.854 INFO:tasks.workunit.client.0.smithi016.stdout:9/543: chown d0/d4e/d50 25169587 1 2024-04-02T04:42:41.856 INFO:tasks.workunit.client.0.smithi016.stdout:3/573: mkdir d2/d15/d1a/d45/da5/db6 0 2024-04-02T04:42:41.860 INFO:tasks.workunit.client.0.smithi016.stdout:0/461: creat d0/d16/d24/fa1 x:0 0 0 2024-04-02T04:42:41.861 INFO:tasks.workunit.client.0.smithi016.stdout:2/566: dwrite d12/d9f/dac/d76/f92 [0,4194304] 0 2024-04-02T04:42:41.864 INFO:tasks.workunit.client.0.smithi016.stdout:3/574: write d2/d15/d1a/f69 [2035306,34377] 0 2024-04-02T04:42:41.864 INFO:tasks.workunit.client.0.smithi016.stdout:5/505: write d3/d4c/fa9 [522648,25043] 0 2024-04-02T04:42:41.867 INFO:tasks.workunit.client.0.smithi016.stdout:0/462: write d0/f32 [3998828,77183] 0 2024-04-02T04:42:41.870 INFO:tasks.workunit.client.0.smithi016.stdout:7/504: dwrite d0/d46/f1a [0,4194304] 0 2024-04-02T04:42:41.871 INFO:tasks.workunit.client.0.smithi016.stdout:0/463: creat d0/d4f/fa2 x:0 0 0 2024-04-02T04:42:41.871 INFO:tasks.workunit.client.0.smithi016.stdout:2/567: creat d12/d35/fca x:0 0 0 2024-04-02T04:42:41.873 INFO:tasks.workunit.client.0.smithi016.stdout:3/575: unlink d2/d15/d16/d8a/c9d 0 2024-04-02T04:42:41.873 INFO:tasks.workunit.client.0.smithi016.stdout:9/544: dwrite d0/d4e/f61 [0,4194304] 0 2024-04-02T04:42:41.875 INFO:tasks.workunit.client.0.smithi016.stdout:0/464: stat d0/d16/d2e/l86 0 2024-04-02T04:42:41.875 INFO:tasks.workunit.client.0.smithi016.stdout:1/510: dwrite d14/f48 [0,4194304] 0 2024-04-02T04:42:41.878 INFO:tasks.workunit.client.0.smithi016.stdout:1/511: write d14/d19/d1e/d71/f84 [964549,62676] 0 2024-04-02T04:42:41.880 INFO:tasks.workunit.client.0.smithi016.stdout:2/568: mknod d12/d35/d3f/d56/dab/ccb 0 2024-04-02T04:42:41.881 INFO:tasks.workunit.client.0.smithi016.stdout:1/512: write d14/f17 [3746517,27249] 0 2024-04-02T04:42:41.882 INFO:tasks.workunit.client.0.smithi016.stdout:2/569: chown d12/d9f/dac/d76/da0 5678721 1 2024-04-02T04:42:41.884 INFO:tasks.workunit.client.0.smithi016.stdout:9/545: dwrite d0/d4a/d99/f1f [0,4194304] 0 2024-04-02T04:42:41.884 INFO:tasks.workunit.client.0.smithi016.stdout:2/570: dread - d12/d35/d3b/f83 zero size 2024-04-02T04:42:41.885 INFO:tasks.workunit.client.0.smithi016.stdout:7/505: rename d0/d46/d16/d83/cb2 to d0/d46/d37/d47/cb5 0 2024-04-02T04:42:41.886 INFO:tasks.workunit.client.0.smithi016.stdout:3/576: creat d2/d15/fb7 x:0 0 0 2024-04-02T04:42:41.886 INFO:tasks.workunit.client.0.smithi016.stdout:3/577: chown d2 4344 1 2024-04-02T04:42:41.888 INFO:tasks.workunit.client.0.smithi016.stdout:1/513: mknod d14/d19/d1d/d29/d39/d46/ca8 0 2024-04-02T04:42:41.891 INFO:tasks.workunit.client.0.smithi016.stdout:3/578: mknod d2/d15/d7e/cb8 0 2024-04-02T04:42:41.892 INFO:tasks.workunit.client.0.smithi016.stdout:2/571: dwrite d12/d9f/dac/d4f/f53 [0,4194304] 0 2024-04-02T04:42:41.894 INFO:tasks.workunit.client.0.smithi016.stdout:9/546: link d0/d3/d31/d53/d7e/caa d0/d33/cab 0 2024-04-02T04:42:41.896 INFO:tasks.workunit.client.0.smithi016.stdout:3/579: write d2/d2e/d36/d48/da2/f92 [620644,83069] 0 2024-04-02T04:42:41.897 INFO:tasks.workunit.client.0.smithi016.stdout:7/506: creat d0/d46/d37/d3a/fb6 x:0 0 0 2024-04-02T04:42:41.899 INFO:tasks.workunit.client.0.smithi016.stdout:9/547: mkdir d0/d3/d97/dac 0 2024-04-02T04:42:41.900 INFO:tasks.workunit.client.0.smithi016.stdout:1/514: getdents d14/d19/d1e/d6d 0 2024-04-02T04:42:41.900 INFO:tasks.workunit.client.0.smithi016.stdout:9/548: chown d0/d9/db/l68 5052 1 2024-04-02T04:42:41.902 INFO:tasks.workunit.client.0.smithi016.stdout:7/507: mknod d0/d46/d16/d49/cb7 0 2024-04-02T04:42:41.903 INFO:tasks.workunit.client.0.smithi016.stdout:3/580: rename d2/c4c to d2/d15/d16/cb9 0 2024-04-02T04:42:41.904 INFO:tasks.workunit.client.0.smithi016.stdout:3/581: stat d2/f7 0 2024-04-02T04:42:41.905 INFO:tasks.workunit.client.0.smithi016.stdout:7/508: unlink d0/d46/f4d 0 2024-04-02T04:42:41.906 INFO:tasks.workunit.client.0.smithi016.stdout:1/515: getdents d14/d19/d1e/d5e/d75 0 2024-04-02T04:42:41.907 INFO:tasks.workunit.client.0.smithi016.stdout:3/582: rename d2/d2e/d36/d66/l74 to d2/d2e/d36/d66/d7a/lba 0 2024-04-02T04:42:41.908 INFO:tasks.workunit.client.0.smithi016.stdout:7/509: write d0/d28/d64/f8b [539094,50340] 0 2024-04-02T04:42:41.910 INFO:tasks.workunit.client.0.smithi016.stdout:3/583: symlink d2/d15/d16/d8a/lbb 0 2024-04-02T04:42:41.913 INFO:tasks.workunit.client.0.smithi016.stdout:9/549: dread d0/d4e/d50/d63/d6e/f65 [0,4194304] 0 2024-04-02T04:42:41.913 INFO:tasks.workunit.client.0.smithi016.stdout:1/516: rename d14/d19/d1d/d45/d8a to d14/da9 0 2024-04-02T04:42:41.913 INFO:tasks.workunit.client.0.smithi016.stdout:1/517: readlink l9 0 2024-04-02T04:42:41.915 INFO:tasks.workunit.client.0.smithi016.stdout:0/465: dread d0/d16/d24/f20 [0,4194304] 0 2024-04-02T04:42:41.916 INFO:tasks.workunit.client.0.smithi016.stdout:1/518: dread d14/d41/f5c [0,4194304] 0 2024-04-02T04:42:41.917 INFO:tasks.workunit.client.0.smithi016.stdout:7/510: link d0/d46/d37/d3a/d43/c94 d0/d46/d37/d3a/d43/d96/cb8 0 2024-04-02T04:42:41.918 INFO:tasks.workunit.client.0.smithi016.stdout:7/511: fdatasync d0/d21/f86 0 2024-04-02T04:42:41.918 INFO:tasks.workunit.client.0.smithi016.stdout:0/466: rmdir d0/d16/d2e/d53/d9d 39 2024-04-02T04:42:41.918 INFO:tasks.workunit.client.0.smithi016.stdout:1/519: chown d14/d19/d1e/d71/f78 1 1 2024-04-02T04:42:41.919 INFO:tasks.workunit.client.0.smithi016.stdout:1/520: readlink d14/d19/d50/l97 0 2024-04-02T04:42:41.919 INFO:tasks.workunit.client.0.smithi016.stdout:7/512: dread - d0/d46/d16/d49/f6e zero size 2024-04-02T04:42:41.920 INFO:tasks.workunit.client.0.smithi016.stdout:9/550: read d0/d4e/d50/d63/d6e/f3a [1485475,49326] 0 2024-04-02T04:42:41.920 INFO:tasks.workunit.client.0.smithi016.stdout:7/513: write d0/d46/d37/d3a/d43/f5e [335843,35715] 0 2024-04-02T04:42:41.920 INFO:tasks.workunit.client.0.smithi016.stdout:6/610: sync 2024-04-02T04:42:41.921 INFO:tasks.workunit.client.0.smithi016.stdout:8/516: sync 2024-04-02T04:42:41.921 INFO:tasks.workunit.client.0.smithi016.stdout:0/467: creat d0/d16/d2e/d37/d66/fa3 x:0 0 0 2024-04-02T04:42:41.925 INFO:tasks.workunit.client.0.smithi016.stdout:9/551: truncate d0/d33/f6d 629202 0 2024-04-02T04:42:41.925 INFO:tasks.workunit.client.0.smithi016.stdout:6/611: read - d0/d9/d16/d56/d89/d67/d6f/f90 zero size 2024-04-02T04:42:41.927 INFO:tasks.workunit.client.0.smithi016.stdout:9/552: write d0/d9/f14 [292457,34272] 0 2024-04-02T04:42:41.927 INFO:tasks.workunit.client.0.smithi016.stdout:6/612: write d0/fa3 [189951,120556] 0 2024-04-02T04:42:41.927 INFO:tasks.workunit.client.0.smithi016.stdout:8/517: truncate d1/d6/d36/d3f/d40/d49/d4a/fa8 121839 0 2024-04-02T04:42:41.927 INFO:tasks.workunit.client.0.smithi016.stdout:7/514: rename d0/d46/d37/d3a/d4c/c9f to d0/d46/d16/d49/cb9 0 2024-04-02T04:42:41.930 INFO:tasks.workunit.client.0.smithi016.stdout:9/553: chown d0/d4e/d50/ca2 699 1 2024-04-02T04:42:41.931 INFO:tasks.workunit.client.0.smithi016.stdout:8/518: truncate d1/d31/f7e 974804 0 2024-04-02T04:42:41.932 INFO:tasks.workunit.client.0.smithi016.stdout:9/554: dread - d0/d3/fa7 zero size 2024-04-02T04:42:41.935 INFO:tasks.workunit.client.0.smithi016.stdout:9/555: chown d0/d9/f14 30 1 2024-04-02T04:42:41.945 INFO:tasks.workunit.client.0.smithi016.stdout:6/613: dwrite d0/d6e/fa2 [0,4194304] 0 2024-04-02T04:42:41.946 INFO:tasks.workunit.client.0.smithi016.stdout:0/468: dread d0/d16/d23/d3e/f43 [0,4194304] 0 2024-04-02T04:42:41.949 INFO:tasks.workunit.client.0.smithi016.stdout:7/515: dwrite d0/d46/d37/d3a/fb6 [0,4194304] 0 2024-04-02T04:42:41.951 INFO:tasks.workunit.client.0.smithi016.stdout:3/584: dwrite d2/d15/d16/f2c [4194304,4194304] 0 2024-04-02T04:42:41.952 INFO:tasks.workunit.client.0.smithi016.stdout:8/519: link d1/d6/d36/d3f/d40/d49/d4a/f5a d1/d6/d36/d3f/d40/d49/fab 0 2024-04-02T04:42:41.953 INFO:tasks.workunit.client.0.smithi016.stdout:8/520: chown d1/d6/d86 40 1 2024-04-02T04:42:41.954 INFO:tasks.workunit.client.0.smithi016.stdout:6/614: readlink d0/d9/d16/d56/d89/d67/d6f/lb6 0 2024-04-02T04:42:41.954 INFO:tasks.workunit.client.0.smithi016.stdout:1/521: dwrite d14/d19/d1d/d29/f3d [0,4194304] 0 2024-04-02T04:42:41.957 INFO:tasks.workunit.client.0.smithi016.stdout:0/469: truncate d0/d16/d23/d3e/f98 703661 0 2024-04-02T04:42:41.958 INFO:tasks.workunit.client.0.smithi016.stdout:9/556: dwrite d0/d9/db/d54/f8e [0,4194304] 0 2024-04-02T04:42:41.959 INFO:tasks.workunit.client.0.smithi016.stdout:8/521: rmdir d1/d6/d36/d4f 39 2024-04-02T04:42:41.959 INFO:tasks.workunit.client.0.smithi016.stdout:3/585: truncate d2/d2e/d36/d48/f91 1656891 0 2024-04-02T04:42:41.962 INFO:tasks.workunit.client.0.smithi016.stdout:7/516: creat d0/d46/d37/d3a/d43/d96/fba x:0 0 0 2024-04-02T04:42:41.963 INFO:tasks.workunit.client.0.smithi016.stdout:0/470: chown d0/d16/d2e/d53/d9d 795790 1 2024-04-02T04:42:41.965 INFO:tasks.workunit.client.0.smithi016.stdout:3/586: readlink d2/d15/d16/d3d/d6e/d6f/db0/lad 0 2024-04-02T04:42:41.965 INFO:tasks.workunit.client.0.smithi016.stdout:1/522: dread fa [0,4194304] 0 2024-04-02T04:42:41.966 INFO:tasks.workunit.client.0.smithi016.stdout:0/471: write d0/d16/f7a [735449,118962] 0 2024-04-02T04:42:41.968 INFO:tasks.workunit.client.0.smithi016.stdout:9/557: dread d0/d9/d4d/f82 [0,4194304] 0 2024-04-02T04:42:41.968 INFO:tasks.workunit.client.0.smithi016.stdout:8/522: rmdir d1/d70/d71/d74 39 2024-04-02T04:42:41.975 INFO:tasks.workunit.client.0.smithi016.stdout:7/517: creat d0/d46/d52/d82/fbb x:0 0 0 2024-04-02T04:42:41.978 INFO:tasks.workunit.client.0.smithi016.stdout:0/472: mkdir d0/d16/d2e/d53/da4 0 2024-04-02T04:42:41.978 INFO:tasks.workunit.client.0.smithi016.stdout:7/518: chown d0/d46/d52/d82/fbb 40 1 2024-04-02T04:42:41.980 INFO:tasks.workunit.client.0.smithi016.stdout:0/473: stat d0/d16/d23/f5a 0 2024-04-02T04:42:41.980 INFO:tasks.workunit.client.0.smithi016.stdout:1/523: chown d14/d19/d1d/d29/f6a 1711083665 1 2024-04-02T04:42:41.981 INFO:tasks.workunit.client.0.smithi016.stdout:3/587: rename d2/d15/d16/d8a/f95 to d2/d15/d16/d3d/fbc 0 2024-04-02T04:42:41.982 INFO:tasks.workunit.client.0.smithi016.stdout:1/524: chown d14/d19/d1e/d71 435950 1 2024-04-02T04:42:41.982 INFO:tasks.workunit.client.0.smithi016.stdout:3/588: truncate d2/d2e/f67 887793 0 2024-04-02T04:42:41.983 INFO:tasks.workunit.client.0.smithi016.stdout:9/558: link d0/d4e/d50/d63/d6e/f77 d0/d4e/fad 0 2024-04-02T04:42:41.984 INFO:tasks.workunit.client.0.smithi016.stdout:9/559: chown d0/d4e/d50/d63 517866 1 2024-04-02T04:42:41.984 INFO:tasks.workunit.client.0.smithi016.stdout:7/519: getdents d0/d46/d37/d3a/da9 0 2024-04-02T04:42:41.985 INFO:tasks.workunit.client.0.smithi016.stdout:3/589: dread d2/d2e/d36/d48/f91 [0,4194304] 0 2024-04-02T04:42:41.986 INFO:tasks.workunit.client.0.smithi016.stdout:1/525: creat d14/d19/d1d/d45/d4f/faa x:0 0 0 2024-04-02T04:42:41.988 INFO:tasks.workunit.client.0.smithi016.stdout:8/523: link d1/d6/d36/c64 d1/d6/d36/d4f/d5d/cac 0 2024-04-02T04:42:41.988 INFO:tasks.workunit.client.0.smithi016.stdout:6/615: dread d0/d9/d16/d56/d89/f42 [4194304,4194304] 0 2024-04-02T04:42:41.992 INFO:tasks.workunit.client.0.smithi016.stdout:1/526: unlink d14/d41/c55 0 2024-04-02T04:42:41.993 INFO:tasks.workunit.client.0.smithi016.stdout:9/560: dwrite d0/d33/f36 [0,4194304] 0 2024-04-02T04:42:41.994 INFO:tasks.workunit.client.0.smithi016.stdout:3/590: creat d2/d15/d16/d3d/d6e/fbd x:0 0 0 2024-04-02T04:42:41.998 INFO:tasks.workunit.client.0.smithi016.stdout:9/561: fsync d0/d3/d31/f57 0 2024-04-02T04:42:41.998 INFO:tasks.workunit.client.0.smithi016.stdout:3/591: write d2/fab [793512,84051] 0 2024-04-02T04:42:41.998 INFO:tasks.workunit.client.0.smithi016.stdout:8/524: mkdir d1/d6/d36/d3f/d40/d49/d4a/d4c/dad 0 2024-04-02T04:42:41.998 INFO:tasks.workunit.client.0.smithi016.stdout:3/592: stat d2/d15/d1a/f39 0 2024-04-02T04:42:41.998 INFO:tasks.workunit.client.0.smithi016.stdout:6/616: dwrite d0/d9/d16/d46/f68 [0,4194304] 0 2024-04-02T04:42:41.999 INFO:tasks.workunit.client.0.smithi016.stdout:1/527: unlink d14/f17 0 2024-04-02T04:42:42.004 INFO:tasks.workunit.client.0.smithi016.stdout:9/562: chown d0/d4a/d99/l7f 334 1 2024-04-02T04:42:42.008 INFO:tasks.workunit.client.0.smithi016.stdout:8/525: rename d1/f52 to d1/d6/d36/d3f/d77/fae 0 2024-04-02T04:42:42.009 INFO:tasks.workunit.client.0.smithi016.stdout:6/617: symlink d0/d9/d16/d56/d89/db4/lbb 0 2024-04-02T04:42:42.012 INFO:tasks.workunit.client.0.smithi016.stdout:3/593: dwrite d2/d2e/f8d [0,4194304] 0 2024-04-02T04:42:42.015 INFO:tasks.workunit.client.0.smithi016.stdout:1/528: dwrite d14/d19/d1d/d29/f3c [0,4194304] 0 2024-04-02T04:42:42.018 INFO:tasks.workunit.client.0.smithi016.stdout:7/520: dwrite d0/d46/f2c [0,4194304] 0 2024-04-02T04:42:42.024 INFO:tasks.workunit.client.0.smithi016.stdout:6/618: dread d0/d6e/fa2 [0,4194304] 0 2024-04-02T04:42:42.024 INFO:tasks.workunit.client.0.smithi016.stdout:3/594: truncate d2/d15/d16/d3d/fa6 307864 0 2024-04-02T04:42:42.029 INFO:tasks.workunit.client.0.smithi016.stdout:1/529: getdents d14/d19/d1e/d5e 0 2024-04-02T04:42:42.029 INFO:tasks.workunit.client.0.smithi016.stdout:9/563: dwrite d0/d4e/d50/d63/d6e/d4b/d58/d6a/f7c [0,4194304] 0 2024-04-02T04:42:42.030 INFO:tasks.workunit.client.0.smithi016.stdout:6/619: write d0/d9/d16/d56/f25 [1960514,112370] 0 2024-04-02T04:42:42.034 INFO:tasks.workunit.client.0.smithi016.stdout:9/564: dread - d0/d4a/d75/f96 zero size 2024-04-02T04:42:42.036 INFO:tasks.workunit.client.0.smithi016.stdout:7/521: dwrite d0/d46/d37/d47/fb4 [0,4194304] 0 2024-04-02T04:42:42.038 INFO:tasks.workunit.client.0.smithi016.stdout:3/595: symlink d2/d15/d16/d3d/d8b/lbe 0 2024-04-02T04:42:42.045 INFO:tasks.workunit.client.0.smithi016.stdout:7/522: creat d0/d46/d37/d3a/d6f/fbc x:0 0 0 2024-04-02T04:42:42.046 INFO:tasks.workunit.client.0.smithi016.stdout:6/620: dwrite d0/d9/d16/d56/d59/f60 [0,4194304] 0 2024-04-02T04:42:42.046 INFO:tasks.workunit.client.0.smithi016.stdout:3/596: creat d2/d2e/fbf x:0 0 0 2024-04-02T04:42:42.047 INFO:tasks.workunit.client.0.smithi016.stdout:6/621: fsync d0/d9/d16/d56/d89/f85 0 2024-04-02T04:42:42.049 INFO:tasks.workunit.client.0.smithi016.stdout:3/597: mknod d2/d15/d16/d35/cc0 0 2024-04-02T04:42:42.051 INFO:tasks.workunit.client.0.smithi016.stdout:6/622: mknod d0/d9/d16/d56/da4/cbc 0 2024-04-02T04:42:42.054 INFO:tasks.workunit.client.0.smithi016.stdout:1/530: dread f7 [0,4194304] 0 2024-04-02T04:42:42.054 INFO:tasks.workunit.client.0.smithi016.stdout:6/623: mknod d0/d9/d16/d56/d95/cbd 0 2024-04-02T04:42:42.057 INFO:tasks.workunit.client.0.smithi016.stdout:6/624: mkdir d0/d9/d16/d46/dbe 0 2024-04-02T04:42:42.059 INFO:tasks.workunit.client.0.smithi016.stdout:4/545: sync 2024-04-02T04:42:42.059 INFO:tasks.workunit.client.0.smithi016.stdout:6/625: chown d0/d9/d16/d56/d89/f45 0 1 2024-04-02T04:42:42.060 INFO:tasks.workunit.client.0.smithi016.stdout:9/565: dwrite d0/d4a/d99/f48 [0,4194304] 0 2024-04-02T04:42:42.062 INFO:tasks.workunit.client.0.smithi016.stdout:9/566: fdatasync d0/d4a/f69 0 2024-04-02T04:42:42.065 INFO:tasks.workunit.client.0.smithi016.stdout:6/626: creat d0/d9/d16/d56/fbf x:0 0 0 2024-04-02T04:42:42.068 INFO:tasks.workunit.client.0.smithi016.stdout:3/598: dread d2/fd [0,4194304] 0 2024-04-02T04:42:42.071 INFO:tasks.workunit.client.0.smithi016.stdout:4/546: dwrite d5/d17/d18/f2b [4194304,4194304] 0 2024-04-02T04:42:42.074 INFO:tasks.workunit.client.0.smithi016.stdout:4/547: write d5/d1d/d5f/d65/d6f/f93 [25106,44558] 0 2024-04-02T04:42:42.075 INFO:tasks.workunit.client.0.smithi016.stdout:9/567: dwrite d0/d4e/fad [0,4194304] 0 2024-04-02T04:42:42.085 INFO:tasks.workunit.client.0.smithi016.stdout:6/627: dwrite d0/d9/d16/d56/d89/d67/d6f/d8a/d91/fb3 [0,4194304] 0 2024-04-02T04:42:42.087 INFO:tasks.workunit.client.0.smithi016.stdout:4/548: link d5/d1d/c30 d5/d17/d76/ca6 0 2024-04-02T04:42:42.087 INFO:tasks.workunit.client.0.smithi016.stdout:3/599: dwrite d2/d15/d16/f44 [0,4194304] 0 2024-04-02T04:42:42.087 INFO:tasks.workunit.client.0.smithi016.stdout:9/568: dwrite d0/d33/f36 [4194304,4194304] 0 2024-04-02T04:42:42.089 INFO:tasks.workunit.client.0.smithi016.stdout:4/549: dread - d5/de/f8d zero size 2024-04-02T04:42:42.091 INFO:tasks.workunit.client.0.smithi016.stdout:4/550: fsync d5/d17/d18/f7f 0 2024-04-02T04:42:42.094 INFO:tasks.workunit.client.0.smithi016.stdout:9/569: unlink d0/l98 0 2024-04-02T04:42:42.095 INFO:tasks.workunit.client.0.smithi016.stdout:6/628: creat d0/d9/d16/d56/d89/d67/d6f/d8a/d91/fc0 x:0 0 0 2024-04-02T04:42:42.098 INFO:tasks.workunit.client.0.smithi016.stdout:4/551: unlink d5/d17/d18/d25/f4a 0 2024-04-02T04:42:42.100 INFO:tasks.workunit.client.0.smithi016.stdout:9/570: fsync d0/d3/f30 0 2024-04-02T04:42:42.100 INFO:tasks.workunit.client.0.smithi016.stdout:9/571: stat d0 0 2024-04-02T04:42:42.101 INFO:tasks.workunit.client.0.smithi016.stdout:9/572: readlink d0/d9/db/lf 0 2024-04-02T04:42:42.102 INFO:tasks.workunit.client.0.smithi016.stdout:3/600: dwrite d2/d15/d1a/f47 [0,4194304] 0 2024-04-02T04:42:42.105 INFO:tasks.workunit.client.0.smithi016.stdout:4/552: unlink d5/f4e 0 2024-04-02T04:42:42.106 INFO:tasks.workunit.client.0.smithi016.stdout:4/553: chown d5/d8/lc 68 1 2024-04-02T04:42:42.106 INFO:tasks.workunit.client.0.smithi016.stdout:6/629: mknod d0/d9/d16/d56/db0/cc1 0 2024-04-02T04:42:42.107 INFO:tasks.workunit.client.0.smithi016.stdout:9/573: creat d0/d4e/fae x:0 0 0 2024-04-02T04:42:42.108 INFO:tasks.workunit.client.0.smithi016.stdout:4/554: write d5/d17/d18/f2b [1315612,105101] 0 2024-04-02T04:42:42.109 INFO:tasks.workunit.client.0.smithi016.stdout:3/601: symlink d2/d15/d16/d3d/d8b/lc1 0 2024-04-02T04:42:42.111 INFO:tasks.workunit.client.0.smithi016.stdout:3/602: dread - d2/d15/d16/d3d/d6e/fbd zero size 2024-04-02T04:42:42.112 INFO:tasks.workunit.client.0.smithi016.stdout:9/574: write d0/d4a/d99/d46/f87 [5872241,31289] 0 2024-04-02T04:42:42.112 INFO:tasks.workunit.client.0.smithi016.stdout:9/575: chown d0/d4a/d99/d46 360027841 1 2024-04-02T04:42:42.112 INFO:tasks.workunit.client.0.smithi016.stdout:4/555: creat d5/d17/d18/da0/fa7 x:0 0 0 2024-04-02T04:42:42.114 INFO:tasks.workunit.client.0.smithi016.stdout:6/630: creat d0/d9/fc2 x:0 0 0 2024-04-02T04:42:42.116 INFO:tasks.workunit.client.0.smithi016.stdout:9/576: fdatasync d0/f27 0 2024-04-02T04:42:42.117 INFO:tasks.workunit.client.0.smithi016.stdout:6/631: dread - d0/d9/d16/d46/d5e/f8f zero size 2024-04-02T04:42:42.117 INFO:tasks.workunit.client.0.smithi016.stdout:4/556: unlink d5/d17/d18/d25/f57 0 2024-04-02T04:42:42.118 INFO:tasks.workunit.client.0.smithi016.stdout:3/603: dwrite d2/d15/d7e/fa9 [0,4194304] 0 2024-04-02T04:42:42.120 INFO:tasks.workunit.client.0.smithi016.stdout:3/604: fdatasync d2/d15/d70/f79 0 2024-04-02T04:42:42.120 INFO:tasks.workunit.client.0.smithi016.stdout:3/605: stat d2/d15/d16/f54 0 2024-04-02T04:42:42.120 INFO:tasks.workunit.client.0.smithi016.stdout:4/557: unlink d5/d1d/d1e/la1 0 2024-04-02T04:42:42.125 INFO:tasks.workunit.client.0.smithi016.stdout:4/558: creat d5/d1d/d5f/d65/fa8 x:0 0 0 2024-04-02T04:42:42.126 INFO:tasks.workunit.client.0.smithi016.stdout:4/559: write d5/d17/d18/d1f/f70 [792574,46112] 0 2024-04-02T04:42:42.126 INFO:tasks.workunit.client.0.smithi016.stdout:3/606: mknod d2/d2e/d36/cc2 0 2024-04-02T04:42:42.130 INFO:tasks.workunit.client.0.smithi016.stdout:4/560: mknod d5/d1d/d69/ca9 0 2024-04-02T04:42:42.131 INFO:tasks.workunit.client.0.smithi016.stdout:4/561: truncate d5/d17/d18/d1f/f70 1424073 0 2024-04-02T04:42:42.135 INFO:tasks.workunit.client.0.smithi016.stdout:4/562: unlink d5/d1d/d5f/d65/c77 0 2024-04-02T04:42:42.135 INFO:tasks.workunit.client.0.smithi016.stdout:4/563: dread - d5/d1d/d1e/f9d zero size 2024-04-02T04:42:42.136 INFO:tasks.workunit.client.0.smithi016.stdout:4/564: dread - d5/d17/d76/f8b zero size 2024-04-02T04:42:42.144 INFO:tasks.workunit.client.0.smithi016.stdout:6/632: getdents d0/d9 0 2024-04-02T04:42:42.146 INFO:tasks.workunit.client.0.smithi016.stdout:9/577: truncate d0/d4a/d75/d8d/f94 1435581 0 2024-04-02T04:42:42.147 INFO:tasks.workunit.client.0.smithi016.stdout:3/607: write d2/d2e/d36/d48/f91 [932066,61020] 0 2024-04-02T04:42:42.151 INFO:tasks.workunit.client.0.smithi016.stdout:9/578: rmdir d0/d4a/d99/d74 0 2024-04-02T04:42:42.152 INFO:tasks.workunit.client.0.smithi016.stdout:3/608: link d2/d15/d16/d57/f81 d2/d15/d1a/fc3 0 2024-04-02T04:42:42.154 INFO:tasks.workunit.client.0.smithi016.stdout:4/565: dwrite d5/d1d/d1e/f36 [0,4194304] 0 2024-04-02T04:42:42.156 INFO:tasks.workunit.client.0.smithi016.stdout:9/579: getdents d0/d3/d39 0 2024-04-02T04:42:42.156 INFO:tasks.workunit.client.0.smithi016.stdout:9/580: chown d0/d3 4732 1 2024-04-02T04:42:42.156 INFO:tasks.workunit.client.0.smithi016.stdout:9/581: chown d0/d3 1 1 2024-04-02T04:42:42.158 INFO:tasks.workunit.client.0.smithi016.stdout:9/582: write d0/d9/f3d [538900,81300] 0 2024-04-02T04:42:42.162 INFO:tasks.workunit.client.0.smithi016.stdout:3/609: creat d2/d2e/d36/fc4 x:0 0 0 2024-04-02T04:42:42.167 INFO:tasks.workunit.client.0.smithi016.stdout:4/566: dwrite d5/d17/d18/d1f/f90 [0,4194304] 0 2024-04-02T04:42:42.167 INFO:tasks.workunit.client.0.smithi016.stdout:3/610: chown d2/d15/d16/d3d/d6e/d6f/db0 1155658 1 2024-04-02T04:42:42.168 INFO:tasks.workunit.client.0.smithi016.stdout:9/583: dwrite d0/d4a/f9c [4194304,4194304] 0 2024-04-02T04:42:42.169 INFO:tasks.workunit.client.0.smithi016.stdout:3/611: chown d2/d15/d16/d3d/d6e/d6f/d78 356 1 2024-04-02T04:42:42.169 INFO:tasks.workunit.client.0.smithi016.stdout:4/567: write d5/d1d/d1e/f36 [2368389,45407] 0 2024-04-02T04:42:42.170 INFO:tasks.workunit.client.0.smithi016.stdout:5/506: sync 2024-04-02T04:42:42.170 INFO:tasks.workunit.client.0.smithi016.stdout:9/584: write d0/f5d [1591582,110266] 0 2024-04-02T04:42:42.170 INFO:tasks.workunit.client.0.smithi016.stdout:3/612: stat d2/c1c 0 2024-04-02T04:42:42.171 INFO:tasks.workunit.client.0.smithi016.stdout:5/507: stat d3/d40/d4f/d6b/d2a/d6d 0 2024-04-02T04:42:42.171 INFO:tasks.workunit.client.0.smithi016.stdout:3/613: write d2/fab [1539878,23517] 0 2024-04-02T04:42:42.173 INFO:tasks.workunit.client.0.smithi016.stdout:5/508: fsync d3/d40/f79 0 2024-04-02T04:42:42.174 INFO:tasks.workunit.client.0.smithi016.stdout:9/585: write d0/d9/db/d54/f8e [1057294,58327] 0 2024-04-02T04:42:42.176 INFO:tasks.workunit.client.0.smithi016.stdout:9/586: chown d0/d33/c88 12356 1 2024-04-02T04:42:42.176 INFO:tasks.workunit.client.0.smithi016.stdout:5/509: fsync d3/d40/d4f/d6b/d2a/d6d/fa7 0 2024-04-02T04:42:42.186 INFO:tasks.workunit.client.0.smithi016.stdout:9/587: mknod d0/d3/d39/caf 0 2024-04-02T04:42:42.188 INFO:tasks.workunit.client.0.smithi016.stdout:9/588: truncate d0/d4a/f69 678967 0 2024-04-02T04:42:42.190 INFO:tasks.workunit.client.0.smithi016.stdout:9/589: unlink d0/d4e/d50/d63/d6e/l62 0 2024-04-02T04:42:42.192 INFO:tasks.workunit.client.0.smithi016.stdout:9/590: write d0/d3/d31/d53/fa3 [285984,39186] 0 2024-04-02T04:42:42.202 INFO:tasks.workunit.client.0.smithi016.stdout:3/614: dread d2/d15/d1a/f1e [0,4194304] 0 2024-04-02T04:42:42.202 INFO:tasks.workunit.client.0.smithi016.stdout:9/591: write d0/d3/f40 [837244,45294] 0 2024-04-02T04:42:42.204 INFO:tasks.workunit.client.0.smithi016.stdout:5/510: dread d3/f3b [0,4194304] 0 2024-04-02T04:42:42.205 INFO:tasks.workunit.client.0.smithi016.stdout:9/592: mknod d0/d4a/da9/cb0 0 2024-04-02T04:42:42.207 INFO:tasks.workunit.client.0.smithi016.stdout:9/593: write d0/d4e/d50/d63/d6e/d4b/d58/d6a/f7c [939333,122268] 0 2024-04-02T04:42:42.208 INFO:tasks.workunit.client.0.smithi016.stdout:9/594: stat d0/d4e/d50/d63/d6e/d4b 0 2024-04-02T04:42:42.209 INFO:tasks.workunit.client.0.smithi016.stdout:3/615: rename d2/d15/d1a/d45/c85 to d2/d15/cc5 0 2024-04-02T04:42:42.211 INFO:tasks.workunit.client.0.smithi016.stdout:9/595: symlink d0/d4a/d99/d46/lb1 0 2024-04-02T04:42:42.213 INFO:tasks.workunit.client.0.smithi016.stdout:3/616: symlink d2/d2e/d36/d66/d7a/lc6 0 2024-04-02T04:42:42.214 INFO:tasks.workunit.client.0.smithi016.stdout:9/596: mkdir d0/d4e/d50/d63/d6e/d4b/db2 0 2024-04-02T04:42:42.217 INFO:tasks.workunit.client.0.smithi016.stdout:9/597: creat d0/d9/db/d54/fb3 x:0 0 0 2024-04-02T04:42:42.217 INFO:tasks.workunit.client.0.smithi016.stdout:5/511: dwrite d3/df/d18/d26/f8c [0,4194304] 0 2024-04-02T04:42:42.220 INFO:tasks.workunit.client.0.smithi016.stdout:9/598: mkdir d0/d4a/d75/d95/d9b/db4 0 2024-04-02T04:42:42.222 INFO:tasks.workunit.client.0.smithi016.stdout:9/599: stat d0 0 2024-04-02T04:42:42.222 INFO:tasks.workunit.client.0.smithi016.stdout:3/617: dread d2/d15/f31 [0,4194304] 0 2024-04-02T04:42:42.226 INFO:tasks.workunit.client.0.smithi016.stdout:3/618: symlink d2/d15/d16/d57/lc7 0 2024-04-02T04:42:42.226 INFO:tasks.workunit.client.0.smithi016.stdout:5/512: dwrite d3/faa [0,4194304] 0 2024-04-02T04:42:42.227 INFO:tasks.workunit.client.0.smithi016.stdout:9/600: creat d0/d9/d4d/fb5 x:0 0 0 2024-04-02T04:42:42.228 INFO:tasks.workunit.client.0.smithi016.stdout:5/513: dread - d3/d40/d72/f95 zero size 2024-04-02T04:42:42.229 INFO:tasks.workunit.client.0.smithi016.stdout:9/601: fsync d0/d4e/d50/d63/d6e/d4b/d58/d6a/f7c 0 2024-04-02T04:42:42.230 INFO:tasks.workunit.client.0.smithi016.stdout:9/602: write d0/d9/f14 [2163340,33272] 0 2024-04-02T04:42:42.231 INFO:tasks.workunit.client.0.smithi016.stdout:3/619: mknod d2/d15/d16/cc8 0 2024-04-02T04:42:42.235 INFO:tasks.workunit.client.0.smithi016.stdout:5/514: creat d3/fc1 x:0 0 0 2024-04-02T04:42:42.236 INFO:tasks.workunit.client.0.smithi016.stdout:3/620: creat d2/d15/d60/fc9 x:0 0 0 2024-04-02T04:42:42.237 INFO:tasks.workunit.client.0.smithi016.stdout:3/621: write d2/d2e/d36/fc4 [81297,36989] 0 2024-04-02T04:42:42.238 INFO:tasks.workunit.client.0.smithi016.stdout:5/515: rename d3/d40/d72/c81 to d3/d4c/cc2 0 2024-04-02T04:42:42.241 INFO:tasks.workunit.client.0.smithi016.stdout:5/516: dread - d3/df/d18/fac zero size 2024-04-02T04:42:42.241 INFO:tasks.workunit.client.0.smithi016.stdout:3/622: mknod d2/d15/d16/d3d/d6e/cca 0 2024-04-02T04:42:42.243 INFO:tasks.workunit.client.0.smithi016.stdout:9/603: dread d0/d9/f14 [0,4194304] 0 2024-04-02T04:42:42.244 INFO:tasks.workunit.client.0.smithi016.stdout:9/604: readlink d0/d9/db/lf 0 2024-04-02T04:42:42.244 INFO:tasks.workunit.client.0.smithi016.stdout:5/517: stat d3/df/d18/d2f/l3a 0 2024-04-02T04:42:42.251 INFO:tasks.workunit.client.0.smithi016.stdout:9/605: dwrite d0/f27 [0,4194304] 0 2024-04-02T04:42:42.253 INFO:tasks.workunit.client.0.smithi016.stdout:5/518: dwrite d3/df/d18/f89 [0,4194304] 0 2024-04-02T04:42:42.254 INFO:tasks.workunit.client.0.smithi016.stdout:3/623: truncate d2/d15/d16/f2c 6079939 0 2024-04-02T04:42:42.255 INFO:tasks.workunit.client.0.smithi016.stdout:8/526: sync 2024-04-02T04:42:42.255 INFO:tasks.workunit.client.0.smithi016.stdout:2/572: sync 2024-04-02T04:42:42.255 INFO:tasks.workunit.client.0.smithi016.stdout:0/474: sync 2024-04-02T04:42:42.255 INFO:tasks.workunit.client.0.smithi016.stdout:9/606: truncate d0/d33/f38 704623 0 2024-04-02T04:42:42.257 INFO:tasks.workunit.client.0.smithi016.stdout:8/527: fsync d1/d6/d36/d4f/d5d/f95 0 2024-04-02T04:42:42.258 INFO:tasks.workunit.client.0.smithi016.stdout:3/624: mkdir d2/d2e/d36/d48/da2/dcb 0 2024-04-02T04:42:42.259 INFO:tasks.workunit.client.0.smithi016.stdout:9/607: rmdir d0/d3/d39 39 2024-04-02T04:42:42.259 INFO:tasks.workunit.client.0.smithi016.stdout:3/625: read d2/d15/d1a/f22 [204731,56312] 0 2024-04-02T04:42:42.262 INFO:tasks.workunit.client.0.smithi016.stdout:2/573: creat d12/d35/d3f/d56/fcc x:0 0 0 2024-04-02T04:42:42.268 INFO:tasks.workunit.client.0.smithi016.stdout:5/519: getdents d3/df/d18/d2f 0 2024-04-02T04:42:42.268 INFO:tasks.workunit.client.0.smithi016.stdout:9/608: dwrite d0/d3/d31/d53/f9d [0,4194304] 0 2024-04-02T04:42:42.268 INFO:tasks.workunit.client.0.smithi016.stdout:3/626: symlink d2/d2e/d36/lcc 0 2024-04-02T04:42:42.271 INFO:tasks.workunit.client.0.smithi016.stdout:0/475: getdents d0/d16 0 2024-04-02T04:42:42.273 INFO:tasks.workunit.client.0.smithi016.stdout:9/609: unlink d0/d33/f6d 0 2024-04-02T04:42:42.275 INFO:tasks.workunit.client.0.smithi016.stdout:3/627: mkdir d2/d15/d16/d8a/dcd 0 2024-04-02T04:42:42.277 INFO:tasks.workunit.client.0.smithi016.stdout:2/574: dread d12/d35/d3b/d42/d59/fba [0,4194304] 0 2024-04-02T04:42:42.278 INFO:tasks.workunit.client.0.smithi016.stdout:3/628: fdatasync d2/d15/d16/d57/f64 0 2024-04-02T04:42:42.281 INFO:tasks.workunit.client.0.smithi016.stdout:9/610: creat d0/d4e/d50/d67/fb6 x:0 0 0 2024-04-02T04:42:42.282 INFO:tasks.workunit.client.0.smithi016.stdout:3/629: fsync d2/d15/d16/d8a/f98 0 2024-04-02T04:42:42.283 INFO:tasks.workunit.client.0.smithi016.stdout:5/520: dwrite d3/f28 [0,4194304] 0 2024-04-02T04:42:42.285 INFO:tasks.workunit.client.0.smithi016.stdout:9/611: chown d0/d9/db/d54/f8e 1975 1 2024-04-02T04:42:42.286 INFO:tasks.workunit.client.0.smithi016.stdout:8/528: dread d1/d6/d36/d4f/f62 [0,4194304] 0 2024-04-02T04:42:42.286 INFO:tasks.workunit.client.0.smithi016.stdout:0/476: dwrite d0/d16/d2e/d37/f4b [0,4194304] 0 2024-04-02T04:42:42.286 INFO:tasks.workunit.client.0.smithi016.stdout:2/575: creat d12/d35/d3b/d42/d8d/fcd x:0 0 0 2024-04-02T04:42:42.288 INFO:tasks.workunit.client.0.smithi016.stdout:8/529: chown d1/c9d 1 1 2024-04-02T04:42:42.289 INFO:tasks.workunit.client.0.smithi016.stdout:9/612: creat d0/d4a/d99/d46/fb7 x:0 0 0 2024-04-02T04:42:42.293 INFO:tasks.workunit.client.0.smithi016.stdout:2/576: symlink d12/d35/d3b/d42/d8d/lce 0 2024-04-02T04:42:42.294 INFO:tasks.workunit.client.0.smithi016.stdout:3/630: truncate d2/d15/f23 2217693 0 2024-04-02T04:42:42.294 INFO:tasks.workunit.client.0.smithi016.stdout:2/577: write d12/d26/f48 [592292,28175] 0 2024-04-02T04:42:42.295 INFO:tasks.workunit.client.0.smithi016.stdout:8/530: mknod d1/d31/caf 0 2024-04-02T04:42:42.303 INFO:tasks.workunit.client.0.smithi016.stdout:3/631: mkdir d2/d2e/d36/d66/d90/dce 0 2024-04-02T04:42:42.304 INFO:tasks.workunit.client.0.smithi016.stdout:2/578: rmdir d12/d35/d3f/d56 39 2024-04-02T04:42:42.304 INFO:tasks.workunit.client.0.smithi016.stdout:5/521: rename d3/df/d18/f8b to d3/df/d18/fc3 0 2024-04-02T04:42:42.306 INFO:tasks.workunit.client.0.smithi016.stdout:5/522: write d3/faa [3473892,32309] 0 2024-04-02T04:42:42.306 INFO:tasks.workunit.client.0.smithi016.stdout:2/579: dread - d12/d35/d3b/d42/d67/d6a/fa9 zero size 2024-04-02T04:42:42.309 INFO:tasks.workunit.client.0.smithi016.stdout:0/477: getdents d0/d16/d2e/d53/d6f 0 2024-04-02T04:42:42.310 INFO:tasks.workunit.client.0.smithi016.stdout:0/478: chown d0/d16/d2e/f85 12 1 2024-04-02T04:42:42.311 INFO:tasks.workunit.client.0.smithi016.stdout:8/531: mknod d1/d70/d71/d74/d82/cb0 0 2024-04-02T04:42:42.316 INFO:tasks.workunit.client.0.smithi016.stdout:8/532: symlink d1/d31/lb1 0 2024-04-02T04:42:42.317 INFO:tasks.workunit.client.0.smithi016.stdout:0/479: dwrite d0/d16/d2e/d37/f3b [0,4194304] 0 2024-04-02T04:42:42.318 INFO:tasks.workunit.client.0.smithi016.stdout:3/632: fsync d2/d15/d16/f25 0 2024-04-02T04:42:42.320 INFO:tasks.workunit.client.0.smithi016.stdout:0/480: read d0/d16/d23/f3f [335432,111281] 0 2024-04-02T04:42:42.321 INFO:tasks.workunit.client.0.smithi016.stdout:9/613: write d0/d4a/d75/d8d/f94 [1775492,70976] 0 2024-04-02T04:42:42.321 INFO:tasks.workunit.client.0.smithi016.stdout:8/533: mknod d1/d70/d71/d74/cb2 0 2024-04-02T04:42:42.322 INFO:tasks.workunit.client.0.smithi016.stdout:0/481: truncate d0/d16/f7a 1023332 0 2024-04-02T04:42:42.325 INFO:tasks.workunit.client.0.smithi016.stdout:3/633: mkdir d2/d15/d70/dcf 0 2024-04-02T04:42:42.326 INFO:tasks.workunit.client.0.smithi016.stdout:2/580: dread d12/d35/d3f/f34 [4194304,4194304] 0 2024-04-02T04:42:42.327 INFO:tasks.workunit.client.0.smithi016.stdout:3/634: read d2/d15/f4a [2599937,94853] 0 2024-04-02T04:42:42.331 INFO:tasks.workunit.client.0.smithi016.stdout:2/581: fdatasync d12/d35/d3f/d56/dab/f3c 0 2024-04-02T04:42:42.332 INFO:tasks.workunit.client.0.smithi016.stdout:9/614: creat d0/d9/d4d/da8/fb8 x:0 0 0 2024-04-02T04:42:42.332 INFO:tasks.workunit.client.0.smithi016.stdout:2/582: fsync d12/d9f/dac/d76/fbb 0 2024-04-02T04:42:42.333 INFO:tasks.workunit.client.0.smithi016.stdout:9/615: dread - d0/d3/d31/fa0 zero size 2024-04-02T04:42:42.335 INFO:tasks.workunit.client.0.smithi016.stdout:8/534: creat d1/d70/d71/fb3 x:0 0 0 2024-04-02T04:42:42.337 INFO:tasks.workunit.client.0.smithi016.stdout:2/583: mkdir d12/d9f/dac/d76/dcf 0 2024-04-02T04:42:42.338 INFO:tasks.workunit.client.0.smithi016.stdout:9/616: mknod d0/d4a/cb9 0 2024-04-02T04:42:42.340 INFO:tasks.workunit.client.0.smithi016.stdout:0/482: getdents d0/d16/d2e/d36 0 2024-04-02T04:42:42.341 INFO:tasks.workunit.client.0.smithi016.stdout:9/617: dread d0/d4a/d75/d8d/f94 [0,4194304] 0 2024-04-02T04:42:42.342 INFO:tasks.workunit.client.0.smithi016.stdout:0/483: write d0/d16/d2e/d53/d6f/f88 [2150847,124820] 0 2024-04-02T04:42:42.342 INFO:tasks.workunit.client.0.smithi016.stdout:9/618: write d0/d33/f36 [7588689,83801] 0 2024-04-02T04:42:42.342 INFO:tasks.workunit.client.0.smithi016.stdout:3/635: link d2/d15/d16/d3d/d6e/d6f/db0/c4f d2/d15/d16/d3d/d8b/cd0 0 2024-04-02T04:42:42.343 INFO:tasks.workunit.client.0.smithi016.stdout:0/484: dread - d0/d4f/fa2 zero size 2024-04-02T04:42:42.346 INFO:tasks.workunit.client.0.smithi016.stdout:8/535: creat d1/d6/d36/d3f/fb4 x:0 0 0 2024-04-02T04:42:42.346 INFO:tasks.workunit.client.0.smithi016.stdout:0/485: symlink d0/d4f/d76/la5 0 2024-04-02T04:42:42.347 INFO:tasks.workunit.client.0.smithi016.stdout:0/486: fdatasync d0/d16/f94 0 2024-04-02T04:42:42.348 INFO:tasks.workunit.client.0.smithi016.stdout:2/584: dwrite d12/d35/d3f/d4d/f97 [0,4194304] 0 2024-04-02T04:42:42.348 INFO:tasks.workunit.client.0.smithi016.stdout:8/536: dread - d1/d6/d36/d3f/d40/d63/f66 zero size 2024-04-02T04:42:42.348 INFO:tasks.workunit.client.0.smithi016.stdout:9/619: mkdir d0/d4e/d50/d63/d6e/d4b/db2/dba 0 2024-04-02T04:42:42.350 INFO:tasks.workunit.client.0.smithi016.stdout:2/585: chown d12/d9f/dac/d76/l8e 2816 1 2024-04-02T04:42:42.351 INFO:tasks.workunit.client.0.smithi016.stdout:8/537: dread d1/d6/d36/fa3 [0,4194304] 0 2024-04-02T04:42:42.351 INFO:tasks.workunit.client.0.smithi016.stdout:9/620: write d0/d4e/d50/d63/d6e/d4b/d58/d6a/f7c [1784771,98472] 0 2024-04-02T04:42:42.353 INFO:tasks.workunit.client.0.smithi016.stdout:0/487: fsync d0/d16/d2e/d37/d66/f70 0 2024-04-02T04:42:42.354 INFO:tasks.workunit.client.0.smithi016.stdout:2/586: mknod d12/d35/d3f/d56/cd0 0 2024-04-02T04:42:42.357 INFO:tasks.workunit.client.0.smithi016.stdout:8/538: creat d1/d31/d67/fb5 x:0 0 0 2024-04-02T04:42:42.360 INFO:tasks.workunit.client.0.smithi016.stdout:2/587: mkdir d12/d35/d3b/d42/d67/d82/dd1 0 2024-04-02T04:42:42.363 INFO:tasks.workunit.client.0.smithi016.stdout:3/636: truncate d2/fa 1436825 0 2024-04-02T04:42:42.363 INFO:tasks.workunit.client.0.smithi016.stdout:2/588: readlink d12/d35/d3f/d56/dab/l5d 0 2024-04-02T04:42:42.367 INFO:tasks.workunit.client.0.smithi016.stdout:8/539: creat d1/d6/d36/d3f/d40/fb6 x:0 0 0 2024-04-02T04:42:42.367 INFO:tasks.workunit.client.0.smithi016.stdout:3/637: stat d2/fa 0 2024-04-02T04:42:42.368 INFO:tasks.workunit.client.0.smithi016.stdout:9/621: getdents d0/d3/d39 0 2024-04-02T04:42:42.369 INFO:tasks.workunit.client.0.smithi016.stdout:9/622: fsync d0/d4e/fae 0 2024-04-02T04:42:42.370 INFO:tasks.workunit.client.0.smithi016.stdout:9/623: fdatasync d0/d9/db/f43 0 2024-04-02T04:42:42.370 INFO:tasks.workunit.client.0.smithi016.stdout:2/589: link d12/d9f/dac/d76/l84 d12/ld2 0 2024-04-02T04:42:42.371 INFO:tasks.workunit.client.0.smithi016.stdout:2/590: stat d12/d35/d3f/d56/dab/c79 0 2024-04-02T04:42:42.372 INFO:tasks.workunit.client.0.smithi016.stdout:8/540: rename d1/d6/f53 to d1/d6/d36/d3f/d40/d45/fb7 0 2024-04-02T04:42:42.372 INFO:tasks.workunit.client.0.smithi016.stdout:8/541: stat d1/d31 0 2024-04-02T04:42:42.373 INFO:tasks.workunit.client.0.smithi016.stdout:2/591: symlink d12/d35/d3f/d2b/da3/ld3 0 2024-04-02T04:42:42.373 INFO:tasks.workunit.client.0.smithi016.stdout:8/542: chown d1/d70/d71 435328067 1 2024-04-02T04:42:42.374 INFO:tasks.workunit.client.0.smithi016.stdout:2/592: chown d12/d35/d3b/d42/d8d/fa6 788577 1 2024-04-02T04:42:42.374 INFO:tasks.workunit.client.0.smithi016.stdout:8/543: dread - d1/d6/d36/d3f/d40/d49/faa zero size 2024-04-02T04:42:42.375 INFO:tasks.workunit.client.0.smithi016.stdout:2/593: chown d12/d35/d3f 17925270 1 2024-04-02T04:42:42.375 INFO:tasks.workunit.client.0.smithi016.stdout:3/638: rename d2/d15/d16/d3d to d2/d15/d16/d8a/dd1 0 2024-04-02T04:42:42.376 INFO:tasks.workunit.client.0.smithi016.stdout:8/544: creat d1/d6/d86/fb8 x:0 0 0 2024-04-02T04:42:42.377 INFO:tasks.workunit.client.0.smithi016.stdout:3/639: chown d2/d15/c5f 760140211 1 2024-04-02T04:42:42.377 INFO:tasks.workunit.client.0.smithi016.stdout:9/624: rename d0/d4a/d99/d46/c51 to d0/d4e/d50/d63/d6e/d4b/db2/dba/cbb 0 2024-04-02T04:42:42.378 INFO:tasks.workunit.client.0.smithi016.stdout:9/625: chown d0/d9/f3d 35 1 2024-04-02T04:42:42.378 INFO:tasks.workunit.client.0.smithi016.stdout:2/594: symlink d12/d35/d3b/d42/d67/d82/ld4 0 2024-04-02T04:42:42.379 INFO:tasks.workunit.client.0.smithi016.stdout:7/523: sync 2024-04-02T04:42:42.379 INFO:tasks.workunit.client.0.smithi016.stdout:2/595: stat d12/l80 0 2024-04-02T04:42:42.379 INFO:tasks.workunit.client.0.smithi016.stdout:9/626: read d0/f7 [3687583,91937] 0 2024-04-02T04:42:42.380 INFO:tasks.workunit.client.0.smithi016.stdout:3/640: mkdir d2/d15/d16/d8a/dd1/d8b/dd2 0 2024-04-02T04:42:42.382 INFO:tasks.workunit.client.0.smithi016.stdout:8/545: creat d1/d6/d36/d3f/d40/d45/fb9 x:0 0 0 2024-04-02T04:42:42.385 INFO:tasks.workunit.client.0.smithi016.stdout:8/546: read - d1/d6/d36/d3f/d77/fa7 zero size 2024-04-02T04:42:42.385 INFO:tasks.workunit.client.0.smithi016.stdout:3/641: unlink d2/d15/d16/d8a/dd1/d6e/d6f/db0/l87 0 2024-04-02T04:42:42.387 INFO:tasks.workunit.client.0.smithi016.stdout:7/524: dwrite d0/d28/d64/f8b [0,4194304] 0 2024-04-02T04:42:42.387 INFO:tasks.workunit.client.0.smithi016.stdout:9/627: rename d0/d3/f30 to d0/d4e/d50/d63/d6e/d4b/d58/d6a/fbc 0 2024-04-02T04:42:42.388 INFO:tasks.workunit.client.0.smithi016.stdout:7/525: dread - d0/d46/d37/faa zero size 2024-04-02T04:42:42.388 INFO:tasks.workunit.client.0.smithi016.stdout:9/628: write d0/d3/f81 [3098068,106166] 0 2024-04-02T04:42:42.388 INFO:tasks.workunit.client.0.smithi016.stdout:3/642: rmdir d2/d15/d60 39 2024-04-02T04:42:42.392 INFO:tasks.workunit.client.0.smithi016.stdout:8/547: creat d1/d6/d36/d3f/d40/d49/d4a/d4c/fba x:0 0 0 2024-04-02T04:42:42.392 INFO:tasks.workunit.client.0.smithi016.stdout:7/526: read d0/d46/d37/d3a/d6f/f7b [316256,30986] 0 2024-04-02T04:42:42.394 INFO:tasks.workunit.client.0.smithi016.stdout:7/527: truncate d0/d46/d16/d83/fa8 772009 0 2024-04-02T04:42:42.396 INFO:tasks.workunit.client.0.smithi016.stdout:7/528: write d0/d46/d37/d47/fb4 [1033494,55875] 0 2024-04-02T04:42:42.397 INFO:tasks.workunit.client.0.smithi016.stdout:9/629: symlink d0/d4e/d50/d63/d70/lbd 0 2024-04-02T04:42:42.400 INFO:tasks.workunit.client.0.smithi016.stdout:8/548: mknod d1/d70/d71/d74/d82/cbb 0 2024-04-02T04:42:42.407 INFO:tasks.workunit.client.0.smithi016.stdout:9/630: dwrite d0/d9/db/d54/f8e [0,4194304] 0 2024-04-02T04:42:42.409 INFO:tasks.workunit.client.0.smithi016.stdout:7/529: unlink d0/d46/d16/f3d 0 2024-04-02T04:42:42.409 INFO:tasks.workunit.client.0.smithi016.stdout:9/631: write d0/d4a/f69 [1713095,45324] 0 2024-04-02T04:42:42.409 INFO:tasks.workunit.client.0.smithi016.stdout:3/643: rename d2/d15/cc5 to d2/d15/d16/d8a/dcd/cd3 0 2024-04-02T04:42:42.410 INFO:tasks.workunit.client.0.smithi016.stdout:7/530: symlink d0/d46/d52/d82/lbd 0 2024-04-02T04:42:42.412 INFO:tasks.workunit.client.0.smithi016.stdout:7/531: write d0/d87/fb1 [406143,32031] 0 2024-04-02T04:42:42.413 INFO:tasks.workunit.client.0.smithi016.stdout:2/596: dwrite d12/d35/f5c [0,4194304] 0 2024-04-02T04:42:42.415 INFO:tasks.workunit.client.0.smithi016.stdout:3/644: dwrite d2/d15/d16/d8a/f98 [0,4194304] 0 2024-04-02T04:42:42.418 INFO:tasks.workunit.client.0.smithi016.stdout:8/549: creat d1/d6/d36/d3f/d40/d45/fbc x:0 0 0 2024-04-02T04:42:42.418 INFO:tasks.workunit.client.0.smithi016.stdout:2/597: chown d12/d35/d3b/d42/d67/d82/fbd 103835 1 2024-04-02T04:42:42.421 INFO:tasks.workunit.client.0.smithi016.stdout:7/532: rename d0/d46/d37/d3a/d6f to d0/d46/d37/dbe 0 2024-04-02T04:42:42.422 INFO:tasks.workunit.client.0.smithi016.stdout:7/533: dread - d0/d46/d16/f8e zero size 2024-04-02T04:42:42.423 INFO:tasks.workunit.client.0.smithi016.stdout:9/632: getdents d0/d4e/d50/d63 0 2024-04-02T04:42:42.431 INFO:tasks.workunit.client.0.smithi016.stdout:8/550: unlink d1/d6/d36/d3f/d40/d49/f68 0 2024-04-02T04:42:42.433 INFO:tasks.workunit.client.0.smithi016.stdout:3/645: creat d2/d2e/d36/d48/fd4 x:0 0 0 2024-04-02T04:42:42.434 INFO:tasks.workunit.client.0.smithi016.stdout:7/534: mkdir d0/d46/dbf 0 2024-04-02T04:42:42.435 INFO:tasks.workunit.client.0.smithi016.stdout:9/633: truncate d0/d33/f38 1145541 0 2024-04-02T04:42:42.436 INFO:tasks.workunit.client.0.smithi016.stdout:7/535: read - d0/d46/d37/dbe/dae/d9d/fa3 zero size 2024-04-02T04:42:42.439 INFO:tasks.workunit.client.0.smithi016.stdout:7/536: symlink d0/d46/d52/d82/lc0 0 2024-04-02T04:42:42.440 INFO:tasks.workunit.client.0.smithi016.stdout:7/537: read - d0/d46/f9a zero size 2024-04-02T04:42:42.440 INFO:tasks.workunit.client.0.smithi016.stdout:3/646: dwrite d2/d15/d16/d8a/dd1/d6e/d6f/db0/f83 [0,4194304] 0 2024-04-02T04:42:42.442 INFO:tasks.workunit.client.0.smithi016.stdout:9/634: link d0/d4a/d75/d8d/f94 d0/d3/fbe 0 2024-04-02T04:42:42.444 INFO:tasks.workunit.client.0.smithi016.stdout:3/647: mkdir d2/d15/d1a/dd5 0 2024-04-02T04:42:42.447 INFO:tasks.workunit.client.0.smithi016.stdout:2/598: write d12/d70/f75 [2138473,45440] 0 2024-04-02T04:42:42.448 INFO:tasks.workunit.client.0.smithi016.stdout:3/648: dread d2/d2e/f8d [0,4194304] 0 2024-04-02T04:42:42.452 INFO:tasks.workunit.client.0.smithi016.stdout:2/599: rename d12/d35/d3f/d56/dab/c55 to d12/d9f/dac/d76/dcf/cd5 0 2024-04-02T04:42:42.454 INFO:tasks.workunit.client.0.smithi016.stdout:3/649: dwrite d2/d15/d1a/d45/f8f [0,4194304] 0 2024-04-02T04:42:42.454 INFO:tasks.workunit.client.0.smithi016.stdout:9/635: rmdir d0/d33 39 2024-04-02T04:42:42.456 INFO:tasks.workunit.client.0.smithi016.stdout:2/600: symlink d12/d35/d3f/d56/ld6 0 2024-04-02T04:42:42.456 INFO:tasks.workunit.client.0.smithi016.stdout:9/636: chown d0/d4a/d75 152613566 1 2024-04-02T04:42:42.459 INFO:tasks.workunit.client.0.smithi016.stdout:2/601: dread - d12/d9f/fb8 zero size 2024-04-02T04:42:42.459 INFO:tasks.workunit.client.0.smithi016.stdout:6/633: sync 2024-04-02T04:42:42.459 INFO:tasks.workunit.client.0.smithi016.stdout:4/568: sync 2024-04-02T04:42:42.459 INFO:tasks.workunit.client.0.smithi016.stdout:1/531: sync 2024-04-02T04:42:42.463 INFO:tasks.workunit.client.0.smithi016.stdout:9/637: symlink d0/d4e/d50/d63/d6e/d4b/lbf 0 2024-04-02T04:42:42.465 INFO:tasks.workunit.client.0.smithi016.stdout:8/551: dwrite d1/d6/d36/f37 [0,4194304] 0 2024-04-02T04:42:42.467 INFO:tasks.workunit.client.0.smithi016.stdout:1/532: creat d14/d19/d7a/fab x:0 0 0 2024-04-02T04:42:42.472 INFO:tasks.workunit.client.0.smithi016.stdout:7/538: dwrite d0/d21/f57 [0,4194304] 0 2024-04-02T04:42:42.472 INFO:tasks.workunit.client.0.smithi016.stdout:6/634: creat d0/d9/d16/d56/d89/d67/d6d/fc3 x:0 0 0 2024-04-02T04:42:42.474 INFO:tasks.workunit.client.0.smithi016.stdout:4/569: creat d5/d1d/d1e/da5/faa x:0 0 0 2024-04-02T04:42:42.475 INFO:tasks.workunit.client.0.smithi016.stdout:1/533: write fd [2595518,66606] 0 2024-04-02T04:42:42.476 INFO:tasks.workunit.client.0.smithi016.stdout:1/534: chown d14 283 1 2024-04-02T04:42:42.479 INFO:tasks.workunit.client.0.smithi016.stdout:7/539: write d0/d21/f3c [99561,129760] 0 2024-04-02T04:42:42.479 INFO:tasks.workunit.client.0.smithi016.stdout:3/650: dwrite d2/d2e/f8d [0,4194304] 0 2024-04-02T04:42:42.480 INFO:tasks.workunit.client.0.smithi016.stdout:1/535: fsync d14/d41/d43/f96 0 2024-04-02T04:42:42.484 INFO:tasks.workunit.client.0.smithi016.stdout:4/570: chown d5/d17/d76/d7e/f81 162 1 2024-04-02T04:42:42.485 INFO:tasks.workunit.client.0.smithi016.stdout:8/552: dwrite d1/d6/d36/d3f/d40/d49/d38/f90 [4194304,4194304] 0 2024-04-02T04:42:42.489 INFO:tasks.workunit.client.0.smithi016.stdout:8/553: write d1/d6/d36/d3f/d77/fa7 [573817,72083] 0 2024-04-02T04:42:42.489 INFO:tasks.workunit.client.0.smithi016.stdout:6/635: rename d0/d9/d16/d56/d89/d67/d6f/d8a/d91/fb3 to d0/d9/df/fc4 0 2024-04-02T04:42:42.490 INFO:tasks.workunit.client.0.smithi016.stdout:7/540: mkdir d0/d46/d37/dbe/dae/dc1 0 2024-04-02T04:42:42.490 INFO:tasks.workunit.client.0.smithi016.stdout:3/651: dread d2/d2e/f67 [0,4194304] 0 2024-04-02T04:42:42.492 INFO:tasks.workunit.client.0.smithi016.stdout:6/636: fdatasync d0/d9/df/f4d 0 2024-04-02T04:42:42.493 INFO:tasks.workunit.client.0.smithi016.stdout:7/541: write d0/f9c [4710989,3434] 0 2024-04-02T04:42:42.494 INFO:tasks.workunit.client.0.smithi016.stdout:2/602: dread f8 [0,4194304] 0 2024-04-02T04:42:42.494 INFO:tasks.workunit.client.0.smithi016.stdout:3/652: chown d2/d2e/f8d 5680 1 2024-04-02T04:42:42.494 INFO:tasks.workunit.client.0.smithi016.stdout:4/571: dread d5/d17/d18/f2b [4194304,4194304] 0 2024-04-02T04:42:42.497 INFO:tasks.workunit.client.0.smithi016.stdout:1/536: rename fb to d14/d19/d1e/d5e/fac 0 2024-04-02T04:42:42.497 INFO:tasks.workunit.client.0.smithi016.stdout:7/542: read - d0/d46/d16/f8e zero size 2024-04-02T04:42:42.497 INFO:tasks.workunit.client.0.smithi016.stdout:9/638: dread d0/d9/f19 [0,4194304] 0 2024-04-02T04:42:42.498 INFO:tasks.workunit.client.0.smithi016.stdout:6/637: fsync d0/d9/d16/d56/d89/d67/d6f/f90 0 2024-04-02T04:42:42.500 INFO:tasks.workunit.client.0.smithi016.stdout:2/603: dread - d12/d9f/dac/d76/da0/dbe/fc5 zero size 2024-04-02T04:42:42.501 INFO:tasks.workunit.client.0.smithi016.stdout:1/537: mknod d14/d19/d1e/d5e/cad 0 2024-04-02T04:42:42.503 INFO:tasks.workunit.client.0.smithi016.stdout:3/653: unlink d2/d15/d16/d8a/dd1/d6e/d6f/d78/l9f 0 2024-04-02T04:42:42.504 INFO:tasks.workunit.client.0.smithi016.stdout:4/572: rename d5/d1d/d69/l99 to d5/d1d/d1e/da5/lab 0 2024-04-02T04:42:42.507 INFO:tasks.workunit.client.0.smithi016.stdout:9/639: creat d0/d4e/d50/d63/d6e/d4b/fc0 x:0 0 0 2024-04-02T04:42:42.508 INFO:tasks.workunit.client.0.smithi016.stdout:8/554: dwrite d1/d6/d36/d3f/d40/d49/d4a/f98 [0,4194304] 0 2024-04-02T04:42:42.508 INFO:tasks.workunit.client.0.smithi016.stdout:2/604: read d12/d26/f29 [106854,107007] 0 2024-04-02T04:42:42.508 INFO:tasks.workunit.client.0.smithi016.stdout:1/538: read d14/d19/d1e/d71/f84 [357494,73300] 0 2024-04-02T04:42:42.512 INFO:tasks.workunit.client.0.smithi016.stdout:4/573: symlink d5/d1d/d1e/da5/lac 0 2024-04-02T04:42:42.512 INFO:tasks.workunit.client.0.smithi016.stdout:2/605: stat d12/d35/d3b/d42/d67/fb3 0 2024-04-02T04:42:42.513 INFO:tasks.workunit.client.0.smithi016.stdout:7/543: dwrite d0/d28/f42 [0,4194304] 0 2024-04-02T04:42:42.513 INFO:tasks.workunit.client.0.smithi016.stdout:5/523: sync 2024-04-02T04:42:42.513 INFO:tasks.workunit.client.0.smithi016.stdout:0/488: sync 2024-04-02T04:42:42.513 INFO:tasks.workunit.client.0.smithi016.stdout:1/539: stat d14/d19/d1d/d45/d4f/d86/c9a 0 2024-04-02T04:42:42.515 INFO:tasks.workunit.client.0.smithi016.stdout:9/640: dread d0/d3/d31/f45 [0,4194304] 0 2024-04-02T04:42:42.517 INFO:tasks.workunit.client.0.smithi016.stdout:6/638: dwrite d0/d9/d16/d46/d5e/f93 [0,4194304] 0 2024-04-02T04:42:42.518 INFO:tasks.workunit.client.0.smithi016.stdout:3/654: truncate d2/d15/d70/f79 2291977 0 2024-04-02T04:42:42.523 INFO:tasks.workunit.client.0.smithi016.stdout:5/524: fsync d3/f9 0 2024-04-02T04:42:42.524 INFO:tasks.workunit.client.0.smithi016.stdout:0/489: creat d0/d4f/d76/fa6 x:0 0 0 2024-04-02T04:42:42.525 INFO:tasks.workunit.client.0.smithi016.stdout:1/540: mkdir d14/da9/dae 0 2024-04-02T04:42:42.528 INFO:tasks.workunit.client.0.smithi016.stdout:3/655: mknod d2/d15/d1a/cd6 0 2024-04-02T04:42:42.531 INFO:tasks.workunit.client.0.smithi016.stdout:4/574: dwrite d5/d16/f55 [0,4194304] 0 2024-04-02T04:42:42.535 INFO:tasks.workunit.client.0.smithi016.stdout:2/606: creat d12/d35/d3b/d42/fd7 x:0 0 0 2024-04-02T04:42:42.536 INFO:tasks.workunit.client.0.smithi016.stdout:6/639: dwrite d0/d6e/fa2 [0,4194304] 0 2024-04-02T04:42:42.539 INFO:tasks.workunit.client.0.smithi016.stdout:2/607: write d12/d9f/dac/d4f/f53 [3791180,127713] 0 2024-04-02T04:42:42.541 INFO:tasks.workunit.client.0.smithi016.stdout:1/541: mkdir d14/d19/daf 0 2024-04-02T04:42:42.543 INFO:tasks.workunit.client.0.smithi016.stdout:9/641: mkdir d0/d9/d4d/d78/da6/dc1 0 2024-04-02T04:42:42.544 INFO:tasks.workunit.client.0.smithi016.stdout:4/575: dwrite d5/d1d/d1e/f36 [0,4194304] 0 2024-04-02T04:42:42.550 INFO:tasks.workunit.client.0.smithi016.stdout:6/640: dwrite d0/d9/d16/d46/d5e/f93 [0,4194304] 0 2024-04-02T04:42:42.553 INFO:tasks.workunit.client.0.smithi016.stdout:4/576: dread d5/de/f79 [0,4194304] 0 2024-04-02T04:42:42.556 INFO:tasks.workunit.client.0.smithi016.stdout:8/555: dwrite d1/d6/d36/d3f/d40/d45/f6e [0,4194304] 0 2024-04-02T04:42:42.564 INFO:tasks.workunit.client.0.smithi016.stdout:1/542: mkdir d14/d19/d1e/d5e/db0 0 2024-04-02T04:42:42.565 INFO:tasks.workunit.client.0.smithi016.stdout:7/544: dread d0/d46/f30 [0,4194304] 0 2024-04-02T04:42:42.565 INFO:tasks.workunit.client.0.smithi016.stdout:3/656: write d2/d15/d1a/fc3 [492895,102514] 0 2024-04-02T04:42:42.566 INFO:tasks.workunit.client.0.smithi016.stdout:7/545: dread - d0/d46/d16/d49/f6e zero size 2024-04-02T04:42:42.567 INFO:tasks.workunit.client.0.smithi016.stdout:9/642: rename d0/d9/f14 to d0/d4a/fc2 0 2024-04-02T04:42:42.567 INFO:tasks.workunit.client.0.smithi016.stdout:3/657: write d2/d15/d16/f27 [1586053,106432] 0 2024-04-02T04:42:42.570 INFO:tasks.workunit.client.0.smithi016.stdout:0/490: link d0/d16/d23/c73 d0/d16/d24/ca7 0 2024-04-02T04:42:42.571 INFO:tasks.workunit.client.0.smithi016.stdout:6/641: mkdir d0/d9/d16/d56/d89/dc5 0 2024-04-02T04:42:42.574 INFO:tasks.workunit.client.0.smithi016.stdout:6/642: chown d0/d9/d16/c58 116969 1 2024-04-02T04:42:42.574 INFO:tasks.workunit.client.0.smithi016.stdout:5/525: dread d3/df/d18/f35 [0,4194304] 0 2024-04-02T04:42:42.575 INFO:tasks.workunit.client.0.smithi016.stdout:8/556: creat d1/d70/d71/d74/fbd x:0 0 0 2024-04-02T04:42:42.576 INFO:tasks.workunit.client.0.smithi016.stdout:9/643: dwrite d0/d4a/d99/d46/fb7 [0,4194304] 0 2024-04-02T04:42:42.585 INFO:tasks.workunit.client.0.smithi016.stdout:2/608: dread d12/d35/d3f/f39 [0,4194304] 0 2024-04-02T04:42:42.590 INFO:tasks.workunit.client.0.smithi016.stdout:4/577: creat d5/d17/d18/d6d/da4/fad x:0 0 0 2024-04-02T04:42:42.592 INFO:tasks.workunit.client.0.smithi016.stdout:6/643: symlink d0/d9/lc6 0 2024-04-02T04:42:42.592 INFO:tasks.workunit.client.0.smithi016.stdout:6/644: fsync d0/f3f 0 2024-04-02T04:42:42.594 INFO:tasks.workunit.client.0.smithi016.stdout:5/526: mkdir d3/d4c/dc4 0 2024-04-02T04:42:42.594 INFO:tasks.workunit.client.0.smithi016.stdout:5/527: stat d3/df/l27 0 2024-04-02T04:42:42.596 INFO:tasks.workunit.client.0.smithi016.stdout:8/557: creat d1/d31/d67/d8f/fbe x:0 0 0 2024-04-02T04:42:42.596 INFO:tasks.workunit.client.0.smithi016.stdout:8/558: write d1/d6/f2e [2362173,23693] 0 2024-04-02T04:42:42.599 INFO:tasks.workunit.client.0.smithi016.stdout:9/644: symlink d0/d4e/d50/d63/d6e/d4b/db2/lc3 0 2024-04-02T04:42:42.602 INFO:tasks.workunit.client.0.smithi016.stdout:0/491: truncate d0/fc 1534093 0 2024-04-02T04:42:42.604 INFO:tasks.workunit.client.0.smithi016.stdout:2/609: rename d12/d35/d3f/d2b/l62 to d12/d9f/ld8 0 2024-04-02T04:42:42.605 INFO:tasks.workunit.client.0.smithi016.stdout:2/610: write d12/d35/d3b/d42/d8d/fcd [941355,104074] 0 2024-04-02T04:42:42.606 INFO:tasks.workunit.client.0.smithi016.stdout:2/611: readlink d12/d35/d3f/l2e 0 2024-04-02T04:42:42.606 INFO:tasks.workunit.client.0.smithi016.stdout:4/578: symlink d5/d16/d88/lae 0 2024-04-02T04:42:42.614 INFO:tasks.workunit.client.0.smithi016.stdout:1/543: link d14/d19/d1e/f2b d14/d19/d1e/d5e/fb1 0 2024-04-02T04:42:42.616 INFO:tasks.workunit.client.0.smithi016.stdout:9/645: rmdir d0/d4a/d75/d8d 39 2024-04-02T04:42:42.617 INFO:tasks.workunit.client.0.smithi016.stdout:7/546: write d0/d46/d37/d47/f5c [5003917,41702] 0 2024-04-02T04:42:42.618 INFO:tasks.workunit.client.0.smithi016.stdout:3/658: creat d2/d2e/fd7 x:0 0 0 2024-04-02T04:42:42.618 INFO:tasks.workunit.client.0.smithi016.stdout:9/646: truncate d0/d3/d31/fa0 763266 0 2024-04-02T04:42:42.618 INFO:tasks.workunit.client.0.smithi016.stdout:7/547: chown d0/d46/d37/d47/f6b 469579948 1 2024-04-02T04:42:42.620 INFO:tasks.workunit.client.0.smithi016.stdout:9/647: fsync d0/d9/d4d/da8/fb8 0 2024-04-02T04:42:42.620 INFO:tasks.workunit.client.0.smithi016.stdout:3/659: write d2/d15/d16/f54 [1363429,62686] 0 2024-04-02T04:42:42.620 INFO:tasks.workunit.client.0.smithi016.stdout:7/548: dread - d0/d46/d37/dbe/dae/d9d/fa3 zero size 2024-04-02T04:42:42.622 INFO:tasks.workunit.client.0.smithi016.stdout:9/648: write d0/d3/d31/fa0 [714516,47073] 0 2024-04-02T04:42:42.623 INFO:tasks.workunit.client.0.smithi016.stdout:7/549: truncate d0/d46/d52/f61 4887999 0 2024-04-02T04:42:42.623 INFO:tasks.workunit.client.0.smithi016.stdout:9/649: chown d0/d3/d31/d53/f73 55 1 2024-04-02T04:42:42.624 INFO:tasks.workunit.client.0.smithi016.stdout:9/650: truncate d0/d3/d31/d53/fa3 1118700 0 2024-04-02T04:42:42.627 INFO:tasks.workunit.client.0.smithi016.stdout:5/528: rename d3/df/f5a to d3/df/d18/d26/d2c/fc5 0 2024-04-02T04:42:42.631 INFO:tasks.workunit.client.0.smithi016.stdout:8/559: symlink d1/d6/d36/d3f/d40/d49/lbf 0 2024-04-02T04:42:42.633 INFO:tasks.workunit.client.0.smithi016.stdout:1/544: dwrite d14/d19/d1e/d6d/f8b [0,4194304] 0 2024-04-02T04:42:42.635 INFO:tasks.workunit.client.0.smithi016.stdout:3/660: dread - d2/d15/f24 zero size 2024-04-02T04:42:42.646 INFO:tasks.workunit.client.0.smithi016.stdout:9/651: chown d0/d4e/d50/d63/d6e/l28 11189371 1 2024-04-02T04:42:42.655 INFO:tasks.workunit.client.0.smithi016.stdout:0/492: creat d0/d16/d2e/d53/fa8 x:0 0 0 2024-04-02T04:42:42.655 INFO:tasks.workunit.client.0.smithi016.stdout:6/645: dwrite d0/d9/d16/d46/f7b [0,4194304] 0 2024-04-02T04:42:42.655 INFO:tasks.workunit.client.0.smithi016.stdout:0/493: readlink d0/l6 0 2024-04-02T04:42:42.657 INFO:tasks.workunit.client.0.smithi016.stdout:6/646: write d0/d9/d16/d56/da4/fac [668021,47361] 0 2024-04-02T04:42:42.660 INFO:tasks.workunit.client.0.smithi016.stdout:3/661: chown d2/d15/d1a/cac 452 1 2024-04-02T04:42:42.662 INFO:tasks.workunit.client.0.smithi016.stdout:2/612: creat d12/d35/fd9 x:0 0 0 2024-04-02T04:42:42.663 INFO:tasks.workunit.client.0.smithi016.stdout:4/579: write d5/d17/d18/d1f/f59 [1060815,66302] 0 2024-04-02T04:42:42.664 INFO:tasks.workunit.client.0.smithi016.stdout:7/550: mkdir d0/d46/d16/dc2 0 2024-04-02T04:42:42.666 INFO:tasks.workunit.client.0.smithi016.stdout:3/662: dwrite d2/d15/d1a/f39 [0,4194304] 0 2024-04-02T04:42:42.671 INFO:tasks.workunit.client.0.smithi016.stdout:8/560: mkdir d1/dc0 0 2024-04-02T04:42:42.672 INFO:tasks.workunit.client.0.smithi016.stdout:8/561: rename d1/d70/d71 to d1/d70/d71/d9e/dc1 22 2024-04-02T04:42:42.675 INFO:tasks.workunit.client.0.smithi016.stdout:0/494: read d0/fc [64400,62989] 0 2024-04-02T04:42:42.676 INFO:tasks.workunit.client.0.smithi016.stdout:6/647: creat d0/d9/d16/d56/d89/fc7 x:0 0 0 2024-04-02T04:42:42.676 INFO:tasks.workunit.client.0.smithi016.stdout:0/495: write d0/d16/d2e/d53/fa8 [269927,37922] 0 2024-04-02T04:42:42.677 INFO:tasks.workunit.client.0.smithi016.stdout:0/496: dread - d0/d16/f8a zero size 2024-04-02T04:42:42.679 INFO:tasks.workunit.client.0.smithi016.stdout:2/613: symlink d12/d9f/dac/d76/dcf/lda 0 2024-04-02T04:42:42.681 INFO:tasks.workunit.client.0.smithi016.stdout:4/580: symlink d5/d17/d18/d6d/d8e/laf 0 2024-04-02T04:42:42.684 INFO:tasks.workunit.client.0.smithi016.stdout:7/551: symlink d0/d46/d37/lc3 0 2024-04-02T04:42:42.689 INFO:tasks.workunit.client.0.smithi016.stdout:5/529: creat d3/d40/d4f/fc6 x:0 0 0 2024-04-02T04:42:42.689 INFO:tasks.workunit.client.0.smithi016.stdout:7/552: dwrite d0/d46/d37/d3a/d4c/fa0 [0,4194304] 0 2024-04-02T04:42:42.690 INFO:tasks.workunit.client.0.smithi016.stdout:8/562: creat d1/d70/d71/d74/d82/fc2 x:0 0 0 2024-04-02T04:42:42.693 INFO:tasks.workunit.client.0.smithi016.stdout:1/545: rmdir d14/d19/d1d/d29/d94 0 2024-04-02T04:42:42.694 INFO:tasks.workunit.client.0.smithi016.stdout:7/553: dread d0/f9c [4194304,4194304] 0 2024-04-02T04:42:42.695 INFO:tasks.workunit.client.0.smithi016.stdout:1/546: truncate d14/d19/fa7 569735 0 2024-04-02T04:42:42.695 INFO:tasks.workunit.client.0.smithi016.stdout:7/554: chown d0/d28/d64/l69 770702 1 2024-04-02T04:42:42.696 INFO:tasks.workunit.client.0.smithi016.stdout:2/614: mknod d12/d35/d3f/d2b/da3/cdb 0 2024-04-02T04:42:42.698 INFO:tasks.workunit.client.0.smithi016.stdout:4/581: mkdir d5/d16/d88/db0 0 2024-04-02T04:42:42.703 INFO:tasks.workunit.client.0.smithi016.stdout:5/530: dread - d3/df/d18/d26/d2c/f52 zero size 2024-04-02T04:42:42.704 INFO:tasks.workunit.client.0.smithi016.stdout:8/563: fdatasync d1/d6/d36/fa3 0 2024-04-02T04:42:42.705 INFO:tasks.workunit.client.0.smithi016.stdout:8/564: write d1/d6/d36/d3f/d40/f7a [936273,106613] 0 2024-04-02T04:42:42.707 INFO:tasks.workunit.client.0.smithi016.stdout:8/565: fsync d1/d6/d36/d3f/d40/f7a 0 2024-04-02T04:42:42.708 INFO:tasks.workunit.client.0.smithi016.stdout:0/497: symlink d0/d16/d2e/d37/la9 0 2024-04-02T04:42:42.708 INFO:tasks.workunit.client.0.smithi016.stdout:8/566: write d1/d6/f1a [2627122,117358] 0 2024-04-02T04:42:42.710 INFO:tasks.workunit.client.0.smithi016.stdout:8/567: write d1/d6/d36/d3f/d40/d49/f44 [289243,19209] 0 2024-04-02T04:42:42.712 INFO:tasks.workunit.client.0.smithi016.stdout:7/555: creat d0/d46/d52/fc4 x:0 0 0 2024-04-02T04:42:42.715 INFO:tasks.workunit.client.0.smithi016.stdout:2/615: mknod d12/d35/d3b/d42/d67/d6a/cdc 0 2024-04-02T04:42:42.715 INFO:tasks.workunit.client.0.smithi016.stdout:6/648: dread d0/d9/f36 [0,4194304] 0 2024-04-02T04:42:42.717 INFO:tasks.workunit.client.0.smithi016.stdout:4/582: rmdir d5/d1d/d5f/d65 39 2024-04-02T04:42:42.720 INFO:tasks.workunit.client.0.smithi016.stdout:3/663: creat d2/d2e/fd8 x:0 0 0 2024-04-02T04:42:42.722 INFO:tasks.workunit.client.0.smithi016.stdout:5/531: creat d3/d40/d4f/d9c/fc7 x:0 0 0 2024-04-02T04:42:42.726 INFO:tasks.workunit.client.0.smithi016.stdout:1/547: mknod d14/d19/daf/cb2 0 2024-04-02T04:42:42.728 INFO:tasks.workunit.client.0.smithi016.stdout:0/498: creat d0/d16/d2e/d37/d66/faa x:0 0 0 2024-04-02T04:42:42.731 INFO:tasks.workunit.client.0.smithi016.stdout:8/568: creat d1/d70/d71/d74/fc3 x:0 0 0 2024-04-02T04:42:42.734 INFO:tasks.workunit.client.0.smithi016.stdout:9/652: dwrite d0/d4a/d75/d8d/f94 [0,4194304] 0 2024-04-02T04:42:42.735 INFO:tasks.workunit.client.0.smithi016.stdout:2/616: creat d12/d35/d3f/d56/fdd x:0 0 0 2024-04-02T04:42:42.736 INFO:tasks.workunit.client.0.smithi016.stdout:0/499: dwrite d0/d16/d2e/d36/d9f/fa0 [0,4194304] 0 2024-04-02T04:42:42.739 INFO:tasks.workunit.client.0.smithi016.stdout:0/500: write d0/d16/d2e/d37/f4b [5240894,11124] 0 2024-04-02T04:42:42.741 INFO:tasks.workunit.client.0.smithi016.stdout:4/583: write d5/d1d/d2d/f4d [4329106,85188] 0 2024-04-02T04:42:42.742 INFO:tasks.workunit.client.0.smithi016.stdout:3/664: rmdir d2/d15/d16/d35 39 2024-04-02T04:42:42.742 INFO:tasks.workunit.client.0.smithi016.stdout:4/584: read f4 [2048585,36559] 0 2024-04-02T04:42:42.760 INFO:tasks.workunit.client.0.smithi016.stdout:9/653: symlink d0/d4e/d50/d67/lc4 0 2024-04-02T04:42:42.762 INFO:tasks.workunit.client.0.smithi016.stdout:2/617: readlink d12/d35/d3f/d4d/lbf 0 2024-04-02T04:42:42.763 INFO:tasks.workunit.client.0.smithi016.stdout:2/618: chown d12/d35/d3b/d42/fd7 247867286 1 2024-04-02T04:42:42.765 INFO:tasks.workunit.client.0.smithi016.stdout:9/654: dwrite d0/d4e/d50/d63/d6e/f44 [0,4194304] 0 2024-04-02T04:42:42.766 INFO:tasks.workunit.client.0.smithi016.stdout:0/501: rmdir d0/d16/d24/d62 39 2024-04-02T04:42:42.768 INFO:tasks.workunit.client.0.smithi016.stdout:0/502: chown d0/d16/f54 911 1 2024-04-02T04:42:42.769 INFO:tasks.workunit.client.0.smithi016.stdout:6/649: write d0/d9/d16/f20 [535028,105514] 0 2024-04-02T04:42:42.772 INFO:tasks.workunit.client.0.smithi016.stdout:3/665: creat d2/d2e/d36/d66/d7a/fd9 x:0 0 0 2024-04-02T04:42:42.772 INFO:tasks.workunit.client.0.smithi016.stdout:0/503: write d0/d16/d2e/d53/f7c [2178644,20686] 0 2024-04-02T04:42:42.773 INFO:tasks.workunit.client.0.smithi016.stdout:6/650: truncate d0/d78/fa0 821048 0 2024-04-02T04:42:42.775 INFO:tasks.workunit.client.0.smithi016.stdout:2/619: dwrite d12/d35/d3b/f83 [0,4194304] 0 2024-04-02T04:42:42.777 INFO:tasks.workunit.client.0.smithi016.stdout:1/548: symlink d14/d19/d1e/d71/d9c/lb3 0 2024-04-02T04:42:42.779 INFO:tasks.workunit.client.0.smithi016.stdout:7/556: creat d0/d46/d16/fc5 x:0 0 0 2024-04-02T04:42:42.780 INFO:tasks.workunit.client.0.smithi016.stdout:7/557: write d0/d46/f2c [1121239,82981] 0 2024-04-02T04:42:42.781 INFO:tasks.workunit.client.0.smithi016.stdout:5/532: dwrite d3/d40/d4f/d6b/d2a/f5f [0,4194304] 0 2024-04-02T04:42:42.782 INFO:tasks.workunit.client.0.smithi016.stdout:7/558: fsync d0/d46/d16/d49/f3e 0 2024-04-02T04:42:42.788 INFO:tasks.workunit.client.0.smithi016.stdout:7/559: dwrite d0/d46/d52/f61 [0,4194304] 0 2024-04-02T04:42:42.795 INFO:tasks.workunit.client.0.smithi016.stdout:6/651: unlink d0/d9/d16/d56/d59/f60 0 2024-04-02T04:42:42.797 INFO:tasks.workunit.client.0.smithi016.stdout:3/666: symlink d2/d15/d16/d57/d96/lda 0 2024-04-02T04:42:42.804 INFO:tasks.workunit.client.0.smithi016.stdout:1/549: unlink d14/d41/d43/f49 0 2024-04-02T04:42:42.807 INFO:tasks.workunit.client.0.smithi016.stdout:2/620: dread d12/d35/d3f/d56/d66/f6c [0,4194304] 0 2024-04-02T04:42:42.808 INFO:tasks.workunit.client.0.smithi016.stdout:2/621: dread - d12/d35/d3f/f93 zero size 2024-04-02T04:42:42.809 INFO:tasks.workunit.client.0.smithi016.stdout:2/622: truncate d12/d35/d3b/d42/f95 1125477 0 2024-04-02T04:42:42.812 INFO:tasks.workunit.client.0.smithi016.stdout:9/655: link d0/d9/db/f25 d0/d3/d31/d53/fc5 0 2024-04-02T04:42:42.813 INFO:tasks.workunit.client.0.smithi016.stdout:7/560: mknod d0/d46/d52/cc6 0 2024-04-02T04:42:42.816 INFO:tasks.workunit.client.0.smithi016.stdout:4/585: truncate d5/d17/d18/d25/f3b 966199 0 2024-04-02T04:42:42.817 INFO:tasks.workunit.client.0.smithi016.stdout:0/504: symlink d0/d16/d2e/d53/lab 0 2024-04-02T04:42:42.818 INFO:tasks.workunit.client.0.smithi016.stdout:0/505: write d0/d16/d24/f2a [5150767,43650] 0 2024-04-02T04:42:42.820 INFO:tasks.workunit.client.0.smithi016.stdout:6/652: rename d0/d9/d16/d56/da4/fac to d0/d9/d16/d56/d89/d67/d6f/d8a/fc8 0 2024-04-02T04:42:42.825 INFO:tasks.workunit.client.0.smithi016.stdout:8/569: getdents d1/d70 0 2024-04-02T04:42:42.825 INFO:tasks.workunit.client.0.smithi016.stdout:8/570: stat d1/d70/d71/d74/d82 0 2024-04-02T04:42:42.827 INFO:tasks.workunit.client.0.smithi016.stdout:9/656: dread d0/d3/d31/d53/fc5 [4194304,4194304] 0 2024-04-02T04:42:42.828 INFO:tasks.workunit.client.0.smithi016.stdout:5/533: truncate d3/df/f80 1927926 0 2024-04-02T04:42:42.834 INFO:tasks.workunit.client.0.smithi016.stdout:0/506: symlink d0/d16/d2e/d36/lac 0 2024-04-02T04:42:42.834 INFO:tasks.workunit.client.0.smithi016.stdout:5/534: readlink d3/d40/d4f/d6b/d2a/l84 0 2024-04-02T04:42:42.835 INFO:tasks.workunit.client.0.smithi016.stdout:3/667: mknod d2/d15/d1a/d45/da5/db6/cdb 0 2024-04-02T04:42:42.837 INFO:tasks.workunit.client.0.smithi016.stdout:1/550: read - d14/d19/d20/f62 zero size 2024-04-02T04:42:42.838 INFO:tasks.workunit.client.0.smithi016.stdout:8/571: chown d1/l7 0 1 2024-04-02T04:42:42.839 INFO:tasks.workunit.client.0.smithi016.stdout:8/572: dread - d1/d6/d36/d3f/d40/fb6 zero size 2024-04-02T04:42:42.842 INFO:tasks.workunit.client.0.smithi016.stdout:9/657: symlink d0/d9/d4d/d78/da6/lc6 0 2024-04-02T04:42:42.842 INFO:tasks.workunit.client.0.smithi016.stdout:9/658: fsync d0/d9/f76 0 2024-04-02T04:42:42.846 INFO:tasks.workunit.client.0.smithi016.stdout:7/561: rename d0/d46/d37/d3a/f76 to d0/d46/d52/daf/fc7 0 2024-04-02T04:42:42.849 INFO:tasks.workunit.client.0.smithi016.stdout:9/659: dwrite d0/d4e/d50/d63/d6e/d4b/d58/d6a/f7c [4194304,4194304] 0 2024-04-02T04:42:42.849 INFO:tasks.workunit.client.0.smithi016.stdout:5/535: mknod d3/df/d18/d26/d48/cc8 0 2024-04-02T04:42:42.852 INFO:tasks.workunit.client.0.smithi016.stdout:1/551: mkdir d14/d19/d1d/d45/d4f/d86/db4 0 2024-04-02T04:42:42.855 INFO:tasks.workunit.client.0.smithi016.stdout:6/653: rename d0/f6 to d0/d6e/fc9 0 2024-04-02T04:42:42.856 INFO:tasks.workunit.client.0.smithi016.stdout:0/507: symlink d0/d16/d2e/d37/lad 0 2024-04-02T04:42:42.857 INFO:tasks.workunit.client.0.smithi016.stdout:5/536: mknod d3/df/d18/d26/cc9 0 2024-04-02T04:42:42.858 INFO:tasks.workunit.client.0.smithi016.stdout:5/537: chown d3/d40/d4f/d6b/d45 0 1 2024-04-02T04:42:42.858 INFO:tasks.workunit.client.0.smithi016.stdout:3/668: creat d2/d15/d70/dcf/fdc x:0 0 0 2024-04-02T04:42:42.860 INFO:tasks.workunit.client.0.smithi016.stdout:3/669: write d2/d15/fb4 [150253,72654] 0 2024-04-02T04:42:42.860 INFO:tasks.workunit.client.0.smithi016.stdout:7/562: mkdir d0/d46/da2/dc8 0 2024-04-02T04:42:42.862 INFO:tasks.workunit.client.0.smithi016.stdout:7/563: chown d0/d46/d37/d47/l95 22 1 2024-04-02T04:42:42.862 INFO:tasks.workunit.client.0.smithi016.stdout:1/552: rename d14/d41/l66 to d14/d19/d1d/d29/d39/d4d/lb5 0 2024-04-02T04:42:42.863 INFO:tasks.workunit.client.0.smithi016.stdout:6/654: creat d0/d9/d16/d56/d89/d67/d6f/d7e/fca x:0 0 0 2024-04-02T04:42:42.866 INFO:tasks.workunit.client.0.smithi016.stdout:6/655: truncate d0/d9/d16/d56/d89/d67/d6f/f7f 325631 0 2024-04-02T04:42:42.867 INFO:tasks.workunit.client.0.smithi016.stdout:2/623: truncate d12/d35/d3b/f60 454359 0 2024-04-02T04:42:42.870 INFO:tasks.workunit.client.0.smithi016.stdout:1/553: stat d14/d19/d1d/d45/c8d 0 2024-04-02T04:42:42.870 INFO:tasks.workunit.client.0.smithi016.stdout:5/538: dread d3/df/f22 [0,4194304] 0 2024-04-02T04:42:42.871 INFO:tasks.workunit.client.0.smithi016.stdout:1/554: chown d14/d41 92 1 2024-04-02T04:42:42.871 INFO:tasks.workunit.client.0.smithi016.stdout:9/660: link d0/d3/c23 d0/d9/d4d/d78/da6/cc7 0 2024-04-02T04:42:42.873 INFO:tasks.workunit.client.0.smithi016.stdout:7/564: mknod d0/cc9 0 2024-04-02T04:42:42.873 INFO:tasks.workunit.client.0.smithi016.stdout:9/661: read - d0/f93 zero size 2024-04-02T04:42:42.874 INFO:tasks.workunit.client.0.smithi016.stdout:3/670: mknod d2/d15/d16/d8a/dd1/d8b/dd2/cdd 0 2024-04-02T04:42:42.875 INFO:tasks.workunit.client.0.smithi016.stdout:6/656: dwrite d0/d9/d16/d56/d89/d67/d6f/d7e/fca [0,4194304] 0 2024-04-02T04:42:42.875 INFO:tasks.workunit.client.0.smithi016.stdout:1/555: dread d14/d41/d43/f5f [0,4194304] 0 2024-04-02T04:42:42.876 INFO:tasks.workunit.client.0.smithi016.stdout:2/624: mkdir d12/d35/d3f/d4d/dde 0 2024-04-02T04:42:42.877 INFO:tasks.workunit.client.0.smithi016.stdout:7/565: write d0/d46/d37/d3a/d43/f5e [1550749,56300] 0 2024-04-02T04:42:42.878 INFO:tasks.workunit.client.0.smithi016.stdout:6/657: chown d0/d9/d16/f8d 1 1 2024-04-02T04:42:42.878 INFO:tasks.workunit.client.0.smithi016.stdout:0/508: creat d0/d16/d2e/d37/fae x:0 0 0 2024-04-02T04:42:42.881 INFO:tasks.workunit.client.0.smithi016.stdout:7/566: fsync d0/d46/d37/d47/f6b 0 2024-04-02T04:42:42.882 INFO:tasks.workunit.client.0.smithi016.stdout:3/671: mknod d2/d15/d16/d8a/dd1/d8b/cde 0 2024-04-02T04:42:42.884 INFO:tasks.workunit.client.0.smithi016.stdout:0/509: mkdir d0/d4f/daf 0 2024-04-02T04:42:42.886 INFO:tasks.workunit.client.0.smithi016.stdout:0/510: write d0/d16/f7a [1194096,100397] 0 2024-04-02T04:42:42.887 INFO:tasks.workunit.client.0.smithi016.stdout:9/662: dread d0/f2d [0,4194304] 0 2024-04-02T04:42:42.890 INFO:tasks.workunit.client.0.smithi016.stdout:7/567: creat d0/d46/d16/d49/fca x:0 0 0 2024-04-02T04:42:42.891 INFO:tasks.workunit.client.0.smithi016.stdout:3/672: creat d2/d2e/d36/d66/d90/fdf x:0 0 0 2024-04-02T04:42:42.891 INFO:tasks.workunit.client.0.smithi016.stdout:0/511: dwrite d0/fe [0,4194304] 0 2024-04-02T04:42:42.891 INFO:tasks.workunit.client.0.smithi016.stdout:7/568: dread - d0/d46/d52/f80 zero size 2024-04-02T04:42:42.893 INFO:tasks.workunit.client.0.smithi016.stdout:2/625: creat d12/d35/d3b/d42/fdf x:0 0 0 2024-04-02T04:42:42.895 INFO:tasks.workunit.client.0.smithi016.stdout:3/673: unlink d2/d15/d1a/f69 0 2024-04-02T04:42:42.895 INFO:tasks.workunit.client.0.smithi016.stdout:0/512: creat d0/d48/fb0 x:0 0 0 2024-04-02T04:42:42.897 INFO:tasks.workunit.client.0.smithi016.stdout:0/513: read d0/d16/d2e/d36/f8b [343536,25573] 0 2024-04-02T04:42:42.899 INFO:tasks.workunit.client.0.smithi016.stdout:3/674: creat d2/d15/d70/dcf/fe0 x:0 0 0 2024-04-02T04:42:42.900 INFO:tasks.workunit.client.0.smithi016.stdout:0/514: truncate d0/d4f/f95 510304 0 2024-04-02T04:42:42.903 INFO:tasks.workunit.client.0.smithi016.stdout:5/539: dwrite d3/df/d18/d26/d2c/f52 [0,4194304] 0 2024-04-02T04:42:42.904 INFO:tasks.workunit.client.0.smithi016.stdout:3/675: dread - d2/d15/d1a/d45/f6d zero size 2024-04-02T04:42:42.907 INFO:tasks.workunit.client.0.smithi016.stdout:5/540: creat d3/df/d18/d26/d70/fca x:0 0 0 2024-04-02T04:42:42.908 INFO:tasks.workunit.client.0.smithi016.stdout:2/626: getdents d12/d26 0 2024-04-02T04:42:42.910 INFO:tasks.workunit.client.0.smithi016.stdout:0/515: creat d0/d4f/daf/fb1 x:0 0 0 2024-04-02T04:42:42.913 INFO:tasks.workunit.client.0.smithi016.stdout:3/676: creat d2/d2e/d36/d48/fe1 x:0 0 0 2024-04-02T04:42:42.914 INFO:tasks.workunit.client.0.smithi016.stdout:0/516: mknod d0/d16/cb2 0 2024-04-02T04:42:42.915 INFO:tasks.workunit.client.0.smithi016.stdout:0/517: dread - d0/d16/d2e/d37/d66/faa zero size 2024-04-02T04:42:42.916 INFO:tasks.workunit.client.0.smithi016.stdout:3/677: mkdir d2/d15/d16/d8a/dd1/d6e/d6f/db0/de2 0 2024-04-02T04:42:42.918 INFO:tasks.workunit.client.0.smithi016.stdout:0/518: fdatasync d0/d16/d24/d62/d9c/d8c/f92 0 2024-04-02T04:42:42.919 INFO:tasks.workunit.client.0.smithi016.stdout:0/519: readlink d0/d16/d24/l27 0 2024-04-02T04:42:42.920 INFO:tasks.workunit.client.0.smithi016.stdout:3/678: creat d2/d15/d16/d8a/dd1/fe3 x:0 0 0 2024-04-02T04:42:42.923 INFO:tasks.workunit.client.0.smithi016.stdout:5/541: dread d3/d40/d4f/f75 [0,4194304] 0 2024-04-02T04:42:42.924 INFO:tasks.workunit.client.0.smithi016.stdout:3/679: creat d2/d15/d16/d8a/dd1/d6e/d6f/db0/de2/fe4 x:0 0 0 2024-04-02T04:42:42.929 INFO:tasks.workunit.client.0.smithi016.stdout:5/542: rename d3/d40/d72/cb3 to d3/d40/d4f/d6b/ccb 0 2024-04-02T04:42:42.932 INFO:tasks.workunit.client.0.smithi016.stdout:2/627: dread d12/f4c [0,4194304] 0 2024-04-02T04:42:42.932 INFO:tasks.workunit.client.0.smithi016.stdout:3/680: dwrite d2/d15/d16/d8a/f98 [0,4194304] 0 2024-04-02T04:42:42.933 INFO:tasks.workunit.client.0.smithi016.stdout:0/520: dread d0/d16/d24/d4e/d8d/f75 [4194304,4194304] 0 2024-04-02T04:42:42.936 INFO:tasks.workunit.client.0.smithi016.stdout:2/628: write d12/d35/d3f/d56/fcc [976096,35764] 0 2024-04-02T04:42:42.936 INFO:tasks.workunit.client.0.smithi016.stdout:3/681: write d2/d2e/d36/d48/da2/f92 [1449227,24539] 0 2024-04-02T04:42:42.938 INFO:tasks.workunit.client.0.smithi016.stdout:5/543: truncate d3/d40/d4f/f75 301243 0 2024-04-02T04:42:42.940 INFO:tasks.workunit.client.0.smithi016.stdout:2/629: symlink d12/d9f/le0 0 2024-04-02T04:42:42.941 INFO:tasks.workunit.client.0.smithi016.stdout:0/521: mkdir d0/d16/d23/d3e/db3 0 2024-04-02T04:42:42.942 INFO:tasks.workunit.client.0.smithi016.stdout:0/522: truncate d0/d4f/f95 526514 0 2024-04-02T04:42:42.946 INFO:tasks.workunit.client.0.smithi016.stdout:0/523: unlink d0/d16/cb2 0 2024-04-02T04:42:42.950 INFO:tasks.workunit.client.0.smithi016.stdout:0/524: dwrite d0/f32 [0,4194304] 0 2024-04-02T04:42:42.953 INFO:tasks.workunit.client.0.smithi016.stdout:3/682: write d2/d15/d16/f2c [4198141,44238] 0 2024-04-02T04:42:42.954 INFO:tasks.workunit.client.0.smithi016.stdout:5/544: truncate d3/df/d18/d26/d48/f61 353364 0 2024-04-02T04:42:42.955 INFO:tasks.workunit.client.0.smithi016.stdout:0/525: mknod d0/d48/cb4 0 2024-04-02T04:42:42.955 INFO:tasks.workunit.client.0.smithi016.stdout:3/683: write d2/d15/d16/d8a/f98 [533277,81226] 0 2024-04-02T04:42:42.955 INFO:tasks.workunit.client.0.smithi016.stdout:0/526: chown d0/c5d 14 1 2024-04-02T04:42:42.960 INFO:tasks.workunit.client.0.smithi016.stdout:0/527: rmdir d0/d16/d2e/d53/d9d 39 2024-04-02T04:42:42.962 INFO:tasks.workunit.client.0.smithi016.stdout:0/528: rmdir d0/d16/d24/d4e 39 2024-04-02T04:42:42.963 INFO:tasks.workunit.client.0.smithi016.stdout:3/684: dwrite d2/f26 [0,4194304] 0 2024-04-02T04:42:42.964 INFO:tasks.workunit.client.0.smithi016.stdout:5/545: read d3/df/d18/d2f/f3f [597096,127405] 0 2024-04-02T04:42:42.965 INFO:tasks.workunit.client.0.smithi016.stdout:0/529: write d0/d4f/f9e [2494670,74785] 0 2024-04-02T04:42:42.965 INFO:tasks.workunit.client.0.smithi016.stdout:5/546: dread - d3/d40/d4f/d9c/fc7 zero size 2024-04-02T04:42:42.972 INFO:tasks.workunit.client.0.smithi016.stdout:0/530: dwrite d0/d16/f94 [0,4194304] 0 2024-04-02T04:42:42.973 INFO:tasks.workunit.client.0.smithi016.stdout:2/630: dread d12/d9f/dac/d4f/f5f [0,4194304] 0 2024-04-02T04:42:42.973 INFO:tasks.workunit.client.0.smithi016.stdout:5/547: rename d3/df/d18/d2f/fa4 to d3/d40/d4f/fcc 0 2024-04-02T04:42:42.975 INFO:tasks.workunit.client.0.smithi016.stdout:0/531: symlink d0/d16/d23/lb5 0 2024-04-02T04:42:42.975 INFO:tasks.workunit.client.0.smithi016.stdout:5/548: stat d3/df/d18/d2f/fb4 0 2024-04-02T04:42:42.982 INFO:tasks.workunit.client.0.smithi016.stdout:5/549: rename d3/c37 to d3/d40/db6/ccd 0 2024-04-02T04:42:42.982 INFO:tasks.workunit.client.0.smithi016.stdout:0/532: getdents d0/d16/d2e/d53/da4 0 2024-04-02T04:42:42.983 INFO:tasks.workunit.client.0.smithi016.stdout:0/533: chown d0/f19 63719 1 2024-04-02T04:42:42.983 INFO:tasks.workunit.client.0.smithi016.stdout:0/534: stat d0/d16/d2e/d36/d9f 0 2024-04-02T04:42:42.984 INFO:tasks.workunit.client.0.smithi016.stdout:5/550: truncate d3/df/d18/fc3 687396 0 2024-04-02T04:42:42.985 INFO:tasks.workunit.client.0.smithi016.stdout:0/535: mknod d0/d4f/cb6 0 2024-04-02T04:42:42.985 INFO:tasks.workunit.client.0.smithi016.stdout:5/551: readlink d3/df/d18/d2f/la8 0 2024-04-02T04:42:43.001 INFO:tasks.workunit.client.0.smithi016.stdout:2/631: dread d12/f23 [0,4194304] 0 2024-04-02T04:42:43.003 INFO:tasks.workunit.client.0.smithi016.stdout:2/632: dread d12/d35/d3f/d2b/f69 [0,4194304] 0 2024-04-02T04:42:43.008 INFO:tasks.workunit.client.0.smithi016.stdout:5/552: rmdir d3/df/d18/d26 39 2024-04-02T04:42:43.010 INFO:tasks.workunit.client.0.smithi016.stdout:2/633: rmdir d12/d35/d3f/d56 39 2024-04-02T04:42:43.015 INFO:tasks.workunit.client.0.smithi016.stdout:0/536: link d0/d16/d2e/l86 d0/d4f/lb7 0 2024-04-02T04:42:43.138 INFO:tasks.workunit.client.0.smithi016.stdout:5/553: dwrite d3/f4 [0,4194304] 0 2024-04-02T04:42:43.147 INFO:tasks.workunit.client.0.smithi016.stdout:2/634: link d12/d26/f29 d12/d35/d3f/fe1 0 2024-04-02T04:42:43.148 INFO:tasks.workunit.client.0.smithi016.stdout:5/554: rmdir d3/df/d18/d26/d48 39 2024-04-02T04:42:43.149 INFO:tasks.workunit.client.0.smithi016.stdout:2/635: creat d12/d70/fe2 x:0 0 0 2024-04-02T04:42:43.150 INFO:tasks.workunit.client.0.smithi016.stdout:2/636: dread - d12/d9f/fb8 zero size 2024-04-02T04:42:43.151 INFO:tasks.workunit.client.0.smithi016.stdout:2/637: chown d12/d35/d3f/d2b/f69 27 1 2024-04-02T04:42:43.151 INFO:tasks.workunit.client.0.smithi016.stdout:5/555: dread d3/df/f22 [0,4194304] 0 2024-04-02T04:42:43.171 INFO:tasks.workunit.client.1.smithi027.stdout:3/11: sync 2024-04-02T04:42:43.172 INFO:tasks.workunit.client.1.smithi027.stdout:3/12: write d1/f2 [1772844,61886] 0 2024-04-02T04:42:43.172 INFO:tasks.workunit.client.0.smithi016.stdout:0/537: link d0/d16/d24/d4e/d8d/d71/f72 d0/d16/fb8 0 2024-04-02T04:42:43.173 INFO:tasks.workunit.client.0.smithi016.stdout:2/638: mknod d12/d9f/dac/d76/ce3 0 2024-04-02T04:42:43.178 INFO:tasks.workunit.client.0.smithi016.stdout:2/639: unlink d12/d35/d3f/d56/dab/ccb 0 2024-04-02T04:42:43.178 INFO:tasks.workunit.client.1.smithi027.stdout:3/13: link c0 d1/c3 0 2024-04-02T04:42:43.180 INFO:tasks.workunit.client.0.smithi016.stdout:2/640: mknod d12/d9f/ce4 0 2024-04-02T04:42:43.181 INFO:tasks.workunit.client.1.smithi027.stdout:3/14: symlink d1/l4 0 2024-04-02T04:42:43.182 INFO:tasks.workunit.client.0.smithi016.stdout:0/538: dread d0/d16/d2e/d37/f3c [0,4194304] 0 2024-04-02T04:42:43.183 INFO:tasks.workunit.client.0.smithi016.stdout:0/539: write d0/d16/f80 [5123337,104623] 0 2024-04-02T04:42:43.186 INFO:tasks.workunit.client.0.smithi016.stdout:2/641: rename d12/d35/d3b/d42/d8d/l90 to d12/d9f/dac/d76/da0/le5 0 2024-04-02T04:42:43.187 INFO:tasks.workunit.client.0.smithi016.stdout:0/540: dread - d0/d48/f58 zero size 2024-04-02T04:42:43.188 INFO:tasks.workunit.client.0.smithi016.stdout:5/556: write d3/f10 [508104,110433] 0 2024-04-02T04:42:43.189 INFO:tasks.workunit.client.1.smithi027.stdout:3/15: rename d1/c3 to d1/c5 0 2024-04-02T04:42:43.194 INFO:tasks.workunit.client.0.smithi016.stdout:2/642: dwrite d12/d35/d3b/d42/d59/f9e [0,4194304] 0 2024-04-02T04:42:43.195 INFO:tasks.workunit.client.0.smithi016.stdout:2/643: chown d12/d9f/dac/f44 279022 1 2024-04-02T04:42:43.196 INFO:tasks.workunit.client.0.smithi016.stdout:5/557: rename d3/df/d18/d26/f8e to d3/d4c/dc4/fce 0 2024-04-02T04:42:43.196 INFO:tasks.workunit.client.1.smithi027.stdout:3/16: dread d1/f2 [0,4194304] 0 2024-04-02T04:42:43.198 INFO:tasks.workunit.client.1.smithi027.stdout:3/17: fdatasync d1/f2 0 2024-04-02T04:42:43.199 INFO:tasks.workunit.client.0.smithi016.stdout:2/644: write d12/d35/d3f/d56/dab/f1f [113262,131042] 0 2024-04-02T04:42:43.200 INFO:tasks.workunit.client.0.smithi016.stdout:0/541: dread d0/fc [0,4194304] 0 2024-04-02T04:42:43.201 INFO:tasks.workunit.client.1.smithi027.stdout:3/18: symlink d1/l6 0 2024-04-02T04:42:43.204 INFO:tasks.workunit.client.0.smithi016.stdout:5/558: dwrite d3/d40/d4f/d67/f78 [0,4194304] 0 2024-04-02T04:42:43.206 INFO:tasks.workunit.client.0.smithi016.stdout:0/542: link d0/d16/d24/fa1 d0/d48/fb9 0 2024-04-02T04:42:43.206 INFO:tasks.workunit.client.0.smithi016.stdout:5/559: dread - d3/df/d18/d26/d70/fca zero size 2024-04-02T04:42:43.208 INFO:tasks.workunit.client.1.smithi027.stdout:3/19: dwrite d1/f2 [0,4194304] 0 2024-04-02T04:42:43.211 INFO:tasks.workunit.client.0.smithi016.stdout:5/560: truncate d3/f59 8912284 0 2024-04-02T04:42:43.211 INFO:tasks.workunit.client.0.smithi016.stdout:5/561: stat d3/d40/d4f/d6b/d45/da6 0 2024-04-02T04:42:43.213 INFO:tasks.workunit.client.0.smithi016.stdout:0/543: link d0/d16/d2e/d37/c82 d0/d16/d2e/d36/d9f/cba 0 2024-04-02T04:42:43.218 INFO:tasks.workunit.client.0.smithi016.stdout:0/544: symlink d0/d16/d2e/d37/lbb 0 2024-04-02T04:42:43.220 INFO:tasks.workunit.client.0.smithi016.stdout:0/545: getdents d0/d4f/d76 0 2024-04-02T04:42:43.228 INFO:tasks.workunit.client.0.smithi016.stdout:0/546: dwrite d0/d16/f7a [0,4194304] 0 2024-04-02T04:42:43.229 INFO:tasks.workunit.client.1.smithi027.stdout:3/20: fdatasync d1/f2 0 2024-04-02T04:42:43.234 INFO:tasks.workunit.client.0.smithi016.stdout:0/547: rename d0/d16/d2e/d53/d9d/c91 to d0/d16/d24/d4e/cbc 0 2024-04-02T04:42:43.237 INFO:tasks.workunit.client.0.smithi016.stdout:0/548: dread d0/d16/f7a [0,4194304] 0 2024-04-02T04:42:43.240 INFO:tasks.workunit.client.0.smithi016.stdout:0/549: symlink d0/d16/d2e/d53/da4/lbd 0 2024-04-02T04:42:43.240 INFO:tasks.workunit.client.0.smithi016.stdout:5/562: dread d3/df/d18/f89 [0,4194304] 0 2024-04-02T04:42:43.241 INFO:tasks.workunit.client.0.smithi016.stdout:0/550: symlink d0/d4f/daf/lbe 0 2024-04-02T04:42:43.242 INFO:tasks.workunit.client.0.smithi016.stdout:0/551: chown d0/l5 9181956 1 2024-04-02T04:42:43.243 INFO:tasks.workunit.client.1.smithi027.stdout:1/15: sync 2024-04-02T04:42:43.243 INFO:tasks.workunit.client.1.smithi027.stdout:4/41: sync 2024-04-02T04:42:43.244 INFO:tasks.workunit.client.0.smithi016.stdout:0/552: mknod d0/d16/d24/d4e/d8d/d71/cbf 0 2024-04-02T04:42:43.249 INFO:tasks.workunit.client.1.smithi027.stdout:4/42: dread f2 [4194304,4194304] 0 2024-04-02T04:42:43.251 INFO:tasks.workunit.client.1.smithi027.stdout:1/16: dwrite d0/f5 [0,4194304] 0 2024-04-02T04:42:43.251 INFO:tasks.workunit.client.0.smithi016.stdout:0/553: dwrite d0/d16/d2e/d53/d6f/f74 [0,4194304] 0 2024-04-02T04:42:43.251 INFO:tasks.workunit.client.1.smithi027.stdout:4/43: write f2 [882863,43742] 0 2024-04-02T04:42:43.252 INFO:tasks.workunit.client.1.smithi027.stdout:4/44: write f5 [916448,3576] 0 2024-04-02T04:42:43.254 INFO:tasks.workunit.client.0.smithi016.stdout:0/554: mknod d0/d16/d2e/d53/d9d/cc0 0 2024-04-02T04:42:43.260 INFO:tasks.workunit.client.0.smithi016.stdout:0/555: dwrite d0/d16/d2e/f61 [0,4194304] 0 2024-04-02T04:42:43.261 INFO:tasks.workunit.client.0.smithi016.stdout:0/556: fsync d0/d4f/f9e 0 2024-04-02T04:42:43.265 INFO:tasks.workunit.client.0.smithi016.stdout:0/557: truncate d0/d16/d24/d4e/d8d/d71/f72 914064 0 2024-04-02T04:42:43.271 INFO:tasks.workunit.client.0.smithi016.stdout:0/558: dwrite d0/d16/d2e/d37/d66/fa3 [0,4194304] 0 2024-04-02T04:42:43.274 INFO:tasks.workunit.client.0.smithi016.stdout:0/559: creat d0/d16/d2e/d36/d9f/fc1 x:0 0 0 2024-04-02T04:42:43.276 INFO:tasks.workunit.client.0.smithi016.stdout:0/560: creat d0/d16/d24/d4e/d8d/fc2 x:0 0 0 2024-04-02T04:42:43.278 INFO:tasks.workunit.client.0.smithi016.stdout:0/561: symlink d0/d16/d2e/d53/lc3 0 2024-04-02T04:42:43.278 INFO:tasks.workunit.client.0.smithi016.stdout:4/586: sync 2024-04-02T04:42:43.281 INFO:tasks.workunit.client.0.smithi016.stdout:0/562: symlink d0/d16/d24/d62/d9c/d8c/lc4 0 2024-04-02T04:42:43.281 INFO:tasks.workunit.client.0.smithi016.stdout:4/587: write d5/d17/d18/d6d/f74 [1246676,17972] 0 2024-04-02T04:42:43.284 INFO:tasks.workunit.client.1.smithi027.stdout:4/45: fsync f2 0 2024-04-02T04:42:43.284 INFO:tasks.workunit.client.1.smithi027.stdout:4/46: fsync f5 0 2024-04-02T04:42:43.287 INFO:tasks.workunit.client.0.smithi016.stdout:0/563: creat d0/d16/d24/fc5 x:0 0 0 2024-04-02T04:42:43.288 INFO:tasks.workunit.client.1.smithi027.stdout:3/21: chown d1/c5 648 1 2024-04-02T04:42:43.293 INFO:tasks.workunit.client.0.smithi016.stdout:4/588: dwrite d5/f67 [0,4194304] 0 2024-04-02T04:42:43.294 INFO:tasks.workunit.client.0.smithi016.stdout:4/589: fdatasync d5/d16/d88/f96 0 2024-04-02T04:42:43.297 INFO:tasks.workunit.client.0.smithi016.stdout:4/590: link d5/d17/d76/c86 d5/d1d/d6c/cb1 0 2024-04-02T04:42:43.298 INFO:tasks.workunit.client.0.smithi016.stdout:4/591: fsync d5/d17/d76/d7e/f81 0 2024-04-02T04:42:43.299 INFO:tasks.workunit.client.0.smithi016.stdout:4/592: write d5/d17/d18/d1f/f98 [991876,121572] 0 2024-04-02T04:42:43.301 INFO:tasks.workunit.client.0.smithi016.stdout:4/593: mknod d5/d17/d18/d6d/d8e/cb2 0 2024-04-02T04:42:43.303 INFO:tasks.workunit.client.0.smithi016.stdout:4/594: creat d5/d17/d18/fb3 x:0 0 0 2024-04-02T04:42:43.307 INFO:tasks.workunit.client.0.smithi016.stdout:4/595: chown d5/d16/d3f/f64 432 1 2024-04-02T04:42:43.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:43 smithi016 ceph-mon[28969]: pgmap v20: 65 pgs: 65 active+clean; 1009 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 27 MiB/s rd, 132 MiB/s wr, 306 op/s 2024-04-02T04:42:43.357 INFO:tasks.workunit.client.1.smithi027.stdout:9/38: sync 2024-04-02T04:42:43.357 INFO:tasks.workunit.client.1.smithi027.stdout:2/49: sync 2024-04-02T04:42:43.357 INFO:tasks.workunit.client.1.smithi027.stdout:6/45: sync 2024-04-02T04:42:43.357 INFO:tasks.workunit.client.1.smithi027.stdout:0/69: sync 2024-04-02T04:42:43.357 INFO:tasks.workunit.client.1.smithi027.stdout:5/119: sync 2024-04-02T04:42:43.357 INFO:tasks.workunit.client.1.smithi027.stdout:8/41: sync 2024-04-02T04:42:43.357 INFO:tasks.workunit.client.1.smithi027.stdout:7/42: sync 2024-04-02T04:42:43.357 INFO:tasks.workunit.client.1.smithi027.stdout:9/39: fdatasync f5 0 2024-04-02T04:42:43.358 INFO:tasks.workunit.client.1.smithi027.stdout:7/43: stat f1 0 2024-04-02T04:42:43.359 INFO:tasks.workunit.client.1.smithi027.stdout:7/44: fdatasync f1 0 2024-04-02T04:42:43.359 INFO:tasks.workunit.client.1.smithi027.stdout:7/45: readlink d2/l5 0 2024-04-02T04:42:43.362 INFO:tasks.workunit.client.1.smithi027.stdout:8/42: dread f4 [0,4194304] 0 2024-04-02T04:42:43.365 INFO:tasks.workunit.client.1.smithi027.stdout:3/22: rmdir d1 39 2024-04-02T04:42:43.372 INFO:tasks.workunit.client.1.smithi027.stdout:6/46: unlink f11 0 2024-04-02T04:42:43.373 INFO:tasks.workunit.client.0.smithi016.stdout:2/645: dwrite d12/d9f/dac/f2f [0,4194304] 0 2024-04-02T04:42:43.373 INFO:tasks.workunit.client.1.smithi027.stdout:6/47: read - fc zero size 2024-04-02T04:42:43.374 INFO:tasks.workunit.client.0.smithi016.stdout:2/646: dread - d12/d70/fe2 zero size 2024-04-02T04:42:43.378 INFO:tasks.workunit.client.0.smithi016.stdout:2/647: read d12/d9f/dac/d4f/f68 [218517,70312] 0 2024-04-02T04:42:43.381 INFO:tasks.workunit.client.1.smithi027.stdout:9/40: mknod c10 0 2024-04-02T04:42:43.387 INFO:tasks.workunit.client.1.smithi027.stdout:2/50: fsync d1/d7/fc 0 2024-04-02T04:42:43.388 INFO:tasks.workunit.client.1.smithi027.stdout:9/41: dwrite f5 [4194304,4194304] 0 2024-04-02T04:42:43.389 INFO:tasks.workunit.client.0.smithi016.stdout:2/648: creat d12/d35/d3b/d42/d67/d6a/fe6 x:0 0 0 2024-04-02T04:42:43.392 INFO:tasks.workunit.client.0.smithi016.stdout:4/596: dwrite d5/d1d/d1e/f3c [4194304,4194304] 0 2024-04-02T04:42:43.392 INFO:tasks.workunit.client.0.smithi016.stdout:2/649: stat d12/d35/d3b/d42/d8d/lce 0 2024-04-02T04:42:43.395 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:43 smithi027 ceph-mon[37550]: pgmap v20: 65 pgs: 65 active+clean; 1009 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 27 MiB/s rd, 132 MiB/s wr, 306 op/s 2024-04-02T04:42:43.399 INFO:tasks.workunit.client.0.smithi016.stdout:4/597: dwrite d5/d1d/d1e/f9a [0,4194304] 0 2024-04-02T04:42:43.400 INFO:tasks.workunit.client.1.smithi027.stdout:9/42: dwrite f5 [4194304,4194304] 0 2024-04-02T04:42:43.402 INFO:tasks.workunit.client.1.smithi027.stdout:0/70: mkdir d4/d16 0 2024-04-02T04:42:43.404 INFO:tasks.workunit.client.1.smithi027.stdout:0/71: write d4/f9 [527068,81439] 0 2024-04-02T04:42:43.405 INFO:tasks.workunit.client.1.smithi027.stdout:3/23: dread d1/f2 [0,4194304] 0 2024-04-02T04:42:43.407 INFO:tasks.workunit.client.0.smithi016.stdout:2/650: dwrite d12/d35/d3f/d56/f94 [0,4194304] 0 2024-04-02T04:42:43.408 INFO:tasks.workunit.client.1.smithi027.stdout:9/43: dread f5 [0,4194304] 0 2024-04-02T04:42:43.408 INFO:tasks.workunit.client.0.smithi016.stdout:6/658: sync 2024-04-02T04:42:43.409 INFO:tasks.workunit.client.0.smithi016.stdout:8/573: sync 2024-04-02T04:42:43.409 INFO:tasks.workunit.client.0.smithi016.stdout:1/556: sync 2024-04-02T04:42:43.409 INFO:tasks.workunit.client.1.smithi027.stdout:9/44: write f5 [5962959,6824] 0 2024-04-02T04:42:43.409 INFO:tasks.workunit.client.0.smithi016.stdout:2/651: fsync d12/d9f/dac/d76/f92 0 2024-04-02T04:42:43.410 INFO:tasks.workunit.client.1.smithi027.stdout:6/48: unlink fd 0 2024-04-02T04:42:43.411 INFO:tasks.workunit.client.0.smithi016.stdout:1/557: write d14/d19/d1d/f1f [2356244,107956] 0 2024-04-02T04:42:43.413 INFO:tasks.workunit.client.0.smithi016.stdout:4/598: symlink d5/d1d/d1e/da5/lb4 0 2024-04-02T04:42:43.415 INFO:tasks.workunit.client.0.smithi016.stdout:4/599: dread - d5/d17/d76/f8b zero size 2024-04-02T04:42:43.420 INFO:tasks.workunit.client.1.smithi027.stdout:1/17: sync 2024-04-02T04:42:43.422 INFO:tasks.workunit.client.1.smithi027.stdout:9/45: dwrite f5 [4194304,4194304] 0 2024-04-02T04:42:43.424 INFO:tasks.workunit.client.1.smithi027.stdout:6/49: dwrite ff [0,4194304] 0 2024-04-02T04:42:43.425 INFO:tasks.workunit.client.1.smithi027.stdout:9/46: chown l7 2 1 2024-04-02T04:42:43.433 INFO:tasks.workunit.client.1.smithi027.stdout:9/47: dread fa [0,4194304] 0 2024-04-02T04:42:43.436 INFO:tasks.workunit.client.1.smithi027.stdout:7/46: mkdir d2/d7/da 0 2024-04-02T04:42:43.437 INFO:tasks.workunit.client.1.smithi027.stdout:9/48: dread fa [0,4194304] 0 2024-04-02T04:42:43.438 INFO:tasks.workunit.client.0.smithi016.stdout:8/574: creat d1/d6/d36/d3f/d40/d49/d38/fc4 x:0 0 0 2024-04-02T04:42:43.440 INFO:tasks.workunit.client.0.smithi016.stdout:4/600: read - d5/d1d/d5f/d65/fa8 zero size 2024-04-02T04:42:43.442 INFO:tasks.workunit.client.1.smithi027.stdout:4/47: sync 2024-04-02T04:42:43.445 INFO:tasks.workunit.client.1.smithi027.stdout:3/24: creat d1/f7 x:0 0 0 2024-04-02T04:42:43.446 INFO:tasks.workunit.client.1.smithi027.stdout:3/25: truncate d1/f7 387473 0 2024-04-02T04:42:43.447 INFO:tasks.workunit.client.1.smithi027.stdout:0/72: rename d4/d13/f15 to d4/d16/f17 0 2024-04-02T04:42:43.448 INFO:tasks.workunit.client.1.smithi027.stdout:0/73: fsync d4/d13/fe 0 2024-04-02T04:42:43.450 INFO:tasks.workunit.client.0.smithi016.stdout:8/575: mkdir d1/d6/d36/d3f/d40/d63/dc5 0 2024-04-02T04:42:43.453 INFO:tasks.workunit.client.1.smithi027.stdout:3/26: dwrite d1/f2 [0,4194304] 0 2024-04-02T04:42:43.455 INFO:tasks.workunit.client.1.smithi027.stdout:5/120: rename d0/c10 to d0/c32 0 2024-04-02T04:42:43.455 INFO:tasks.workunit.client.1.smithi027.stdout:3/27: dread d1/f7 [0,4194304] 0 2024-04-02T04:42:43.458 INFO:tasks.workunit.client.1.smithi027.stdout:0/74: dwrite d4/fa [0,4194304] 0 2024-04-02T04:42:43.459 INFO:tasks.workunit.client.1.smithi027.stdout:1/18: symlink d0/l8 0 2024-04-02T04:42:43.461 INFO:tasks.workunit.client.1.smithi027.stdout:0/75: write d4/fa [1673681,102024] 0 2024-04-02T04:42:43.462 INFO:tasks.workunit.client.1.smithi027.stdout:1/19: write d0/f6 [1362108,84379] 0 2024-04-02T04:42:43.464 INFO:tasks.workunit.client.1.smithi027.stdout:6/50: rename f9 to f12 0 2024-04-02T04:42:43.466 INFO:tasks.workunit.client.0.smithi016.stdout:2/652: getdents d12/d9f/dac/d76/dcf 0 2024-04-02T04:42:43.473 INFO:tasks.workunit.client.1.smithi027.stdout:4/48: symlink l7 0 2024-04-02T04:42:43.474 INFO:tasks.workunit.client.1.smithi027.stdout:0/76: dwrite d4/d13/fd [0,4194304] 0 2024-04-02T04:42:43.475 INFO:tasks.workunit.client.1.smithi027.stdout:6/51: dwrite f3 [0,4194304] 0 2024-04-02T04:42:43.475 INFO:tasks.workunit.client.1.smithi027.stdout:4/49: chown f5 2315 1 2024-04-02T04:42:43.478 INFO:tasks.workunit.client.1.smithi027.stdout:4/50: chown l6 0 1 2024-04-02T04:42:43.483 INFO:tasks.workunit.client.1.smithi027.stdout:3/28: creat d1/f8 x:0 0 0 2024-04-02T04:42:43.485 INFO:tasks.workunit.client.1.smithi027.stdout:3/29: write d1/f8 [905942,91664] 0 2024-04-02T04:42:43.486 INFO:tasks.workunit.client.1.smithi027.stdout:0/77: dwrite d4/f11 [0,4194304] 0 2024-04-02T04:42:43.487 INFO:tasks.workunit.client.1.smithi027.stdout:3/30: read d1/f2 [2384282,7663] 0 2024-04-02T04:42:43.488 INFO:tasks.workunit.client.1.smithi027.stdout:3/31: write d1/f8 [1304849,13040] 0 2024-04-02T04:42:43.491 INFO:tasks.workunit.client.1.smithi027.stdout:1/20: creat d0/f9 x:0 0 0 2024-04-02T04:42:43.499 INFO:tasks.workunit.client.1.smithi027.stdout:6/52: mkdir d13 0 2024-04-02T04:42:43.503 INFO:tasks.workunit.client.1.smithi027.stdout:6/53: dread ff [0,4194304] 0 2024-04-02T04:42:43.507 INFO:tasks.workunit.client.1.smithi027.stdout:1/21: symlink d0/la 0 2024-04-02T04:42:43.508 INFO:tasks.workunit.client.1.smithi027.stdout:1/22: fdatasync d0/f7 0 2024-04-02T04:42:43.508 INFO:tasks.workunit.client.1.smithi027.stdout:1/23: readlink d0/la 0 2024-04-02T04:42:43.509 INFO:tasks.workunit.client.1.smithi027.stdout:9/49: link cb c11 0 2024-04-02T04:42:43.514 INFO:tasks.workunit.client.1.smithi027.stdout:5/121: rename d0/d5/f12 to d0/f33 0 2024-04-02T04:42:43.515 INFO:tasks.workunit.client.1.smithi027.stdout:5/122: rename d0/d1a to d0/d1a/d34 22 2024-04-02T04:42:43.518 INFO:tasks.workunit.client.1.smithi027.stdout:0/78: link d4/d13/fd d4/d13/f18 0 2024-04-02T04:42:43.521 INFO:tasks.workunit.client.0.smithi016.stdout:1/558: rename d14/d41/f5c to d14/fb6 0 2024-04-02T04:42:43.522 INFO:tasks.workunit.client.0.smithi016.stdout:1/559: stat d14/da9/f9b 0 2024-04-02T04:42:43.527 INFO:tasks.workunit.client.0.smithi016.stdout:4/601: rename d5/d1d/d1e/da5/lb4 to d5/d17/d18/d6d/da4/lb5 0 2024-04-02T04:42:43.527 INFO:tasks.workunit.client.0.smithi016.stdout:1/560: stat d14/d19/d7a/c9d 0 2024-04-02T04:42:43.530 INFO:tasks.workunit.client.0.smithi016.stdout:2/653: rename d12/d35/d3b/d42/d67/d82 to d12/d9f/dac/d76/da0/de7 0 2024-04-02T04:42:43.531 INFO:tasks.workunit.client.0.smithi016.stdout:2/654: truncate d12/d9f/fc9 303402 0 2024-04-02T04:42:43.531 INFO:tasks.workunit.client.0.smithi016.stdout:2/655: stat f9 0 2024-04-02T04:42:43.532 INFO:tasks.workunit.client.1.smithi027.stdout:6/54: creat d13/f14 x:0 0 0 2024-04-02T04:42:43.532 INFO:tasks.workunit.client.0.smithi016.stdout:4/602: write d5/d17/d18/f2b [5040126,73178] 0 2024-04-02T04:42:43.534 INFO:tasks.workunit.client.1.smithi027.stdout:0/79: write d4/d13/fd [857833,51126] 0 2024-04-02T04:42:43.535 INFO:tasks.workunit.client.0.smithi016.stdout:1/561: fdatasync d14/d19/f69 0 2024-04-02T04:42:43.537 INFO:tasks.workunit.client.1.smithi027.stdout:1/24: link d0/f9 d0/fb 0 2024-04-02T04:42:43.538 INFO:tasks.workunit.client.0.smithi016.stdout:4/603: rmdir d5/d17/d18/d25 39 2024-04-02T04:42:43.539 INFO:tasks.workunit.client.0.smithi016.stdout:4/604: read d5/d1d/d1e/f36 [1954660,4618] 0 2024-04-02T04:42:43.544 INFO:tasks.workunit.client.1.smithi027.stdout:5/123: link d0/df/d11/d13/f1e d0/d1d/f35 0 2024-04-02T04:42:43.544 INFO:tasks.workunit.client.1.smithi027.stdout:1/25: dread d0/f6 [0,4194304] 0 2024-04-02T04:42:43.545 INFO:tasks.workunit.client.1.smithi027.stdout:1/26: chown d0/f9 291990 1 2024-04-02T04:42:43.545 INFO:tasks.workunit.client.1.smithi027.stdout:6/55: fsync f7 0 2024-04-02T04:42:43.546 INFO:tasks.workunit.client.1.smithi027.stdout:5/124: truncate d0/d5/f2e 826968 0 2024-04-02T04:42:43.547 INFO:tasks.workunit.client.1.smithi027.stdout:5/125: fsync d0/df/d11/d13/f28 0 2024-04-02T04:42:43.548 INFO:tasks.workunit.client.1.smithi027.stdout:0/80: creat d4/f19 x:0 0 0 2024-04-02T04:42:43.549 INFO:tasks.workunit.client.0.smithi016.stdout:4/605: readlink d5/d1d/d1e/da5/lab 0 2024-04-02T04:42:43.551 INFO:tasks.workunit.client.0.smithi016.stdout:1/562: mknod d14/d19/d1e/d5e/d75/cb7 0 2024-04-02T04:42:43.555 INFO:tasks.workunit.client.0.smithi016.stdout:1/563: fsync f2 0 2024-04-02T04:42:43.557 INFO:tasks.workunit.client.0.smithi016.stdout:4/606: unlink d5/d17/d18/l51 0 2024-04-02T04:42:43.557 INFO:tasks.workunit.client.0.smithi016.stdout:4/607: readlink d5/de/la2 0 2024-04-02T04:42:43.560 INFO:tasks.workunit.client.0.smithi016.stdout:1/564: symlink d14/d19/d1e/d71/d9c/lb8 0 2024-04-02T04:42:43.561 INFO:tasks.workunit.client.1.smithi027.stdout:1/27: creat d0/fc x:0 0 0 2024-04-02T04:42:43.561 INFO:tasks.workunit.client.0.smithi016.stdout:9/663: sync 2024-04-02T04:42:43.563 INFO:tasks.workunit.client.0.smithi016.stdout:1/565: rmdir d14/d19/d20 39 2024-04-02T04:42:43.564 INFO:tasks.workunit.client.0.smithi016.stdout:5/563: dwrite d3/d40/d4f/d6b/d2a/d6d/fa7 [8388608,4194304] 0 2024-04-02T04:42:43.565 INFO:tasks.workunit.client.0.smithi016.stdout:9/664: write d0/d4e/d50/d63/d6e/d4b/d58/d6a/f9f [3879321,112263] 0 2024-04-02T04:42:43.565 INFO:tasks.workunit.client.0.smithi016.stdout:5/564: stat d3/d40/d4f/fc6 0 2024-04-02T04:42:43.567 INFO:tasks.workunit.client.0.smithi016.stdout:4/608: symlink d5/d16/d88/db0/lb6 0 2024-04-02T04:42:43.568 INFO:tasks.workunit.client.1.smithi027.stdout:0/81: symlink d4/l1a 0 2024-04-02T04:42:43.572 INFO:tasks.workunit.client.0.smithi016.stdout:9/665: dwrite d0/d4e/d50/f85 [0,4194304] 0 2024-04-02T04:42:43.573 INFO:tasks.workunit.client.0.smithi016.stdout:5/565: mknod d3/df/d18/d26/d2c/ccf 0 2024-04-02T04:42:43.575 INFO:tasks.workunit.client.0.smithi016.stdout:5/566: stat d3/df/d18/d2f/la8 0 2024-04-02T04:42:43.576 INFO:tasks.workunit.client.1.smithi027.stdout:6/56: dwrite ff [0,4194304] 0 2024-04-02T04:42:43.576 INFO:tasks.workunit.client.1.smithi027.stdout:2/51: dwrite d1/f6 [4194304,4194304] 0 2024-04-02T04:42:43.576 INFO:tasks.workunit.client.1.smithi027.stdout:6/57: chown c8 76 1 2024-04-02T04:42:43.577 INFO:tasks.workunit.client.0.smithi016.stdout:9/666: write d0/d4e/d50/d63/d6e/d4b/fc0 [795371,118220] 0 2024-04-02T04:42:43.577 INFO:tasks.workunit.client.0.smithi016.stdout:1/566: dread f7 [0,4194304] 0 2024-04-02T04:42:43.578 INFO:tasks.workunit.client.0.smithi016.stdout:4/609: mkdir d5/d17/d18/d6d/da4/db7 0 2024-04-02T04:42:43.579 INFO:tasks.workunit.client.0.smithi016.stdout:1/567: readlink d14/d19/d1e/d71/d9c/lb8 0 2024-04-02T04:42:43.580 INFO:tasks.workunit.client.0.smithi016.stdout:4/610: write d5/d17/d18/da0/fa7 [420294,86646] 0 2024-04-02T04:42:43.580 INFO:tasks.workunit.client.0.smithi016.stdout:9/667: truncate d0/d4e/d50/d63/d6e/f65 4919965 0 2024-04-02T04:42:43.580 INFO:tasks.workunit.client.1.smithi027.stdout:1/28: creat d0/fd x:0 0 0 2024-04-02T04:42:43.581 INFO:tasks.workunit.client.1.smithi027.stdout:1/29: write d0/fc [339999,58613] 0 2024-04-02T04:42:43.593 INFO:tasks.workunit.client.0.smithi016.stdout:1/568: creat d14/d19/d50/fb9 x:0 0 0 2024-04-02T04:42:43.594 INFO:tasks.workunit.client.1.smithi027.stdout:2/52: symlink d1/d7/l10 0 2024-04-02T04:42:43.596 INFO:tasks.workunit.client.0.smithi016.stdout:4/611: unlink d5/d17/d18/f2b 0 2024-04-02T04:42:43.601 INFO:tasks.workunit.client.1.smithi027.stdout:1/30: write d0/fb [51265,50035] 0 2024-04-02T04:42:43.602 INFO:tasks.workunit.client.0.smithi016.stdout:1/569: write f7 [1577738,36131] 0 2024-04-02T04:42:43.602 INFO:tasks.workunit.client.1.smithi027.stdout:7/47: getdents d2/d7 0 2024-04-02T04:42:43.605 INFO:tasks.workunit.client.1.smithi027.stdout:8/43: truncate f4 1995739 0 2024-04-02T04:42:43.609 INFO:tasks.workunit.client.0.smithi016.stdout:1/570: mknod d14/d19/d1e/d71/d9c/cba 0 2024-04-02T04:42:43.611 INFO:tasks.workunit.client.1.smithi027.stdout:1/31: mkdir d0/de 0 2024-04-02T04:42:43.614 INFO:tasks.workunit.client.1.smithi027.stdout:7/48: mknod d2/d7/cb 0 2024-04-02T04:42:43.618 INFO:tasks.workunit.client.0.smithi016.stdout:9/668: rename d0/d9/db/f25 to d0/d4e/d50/d63/d6e/fc8 0 2024-04-02T04:42:43.619 INFO:tasks.workunit.client.0.smithi016.stdout:9/669: chown d0/d4e/fad 441153 1 2024-04-02T04:42:43.621 INFO:tasks.workunit.client.0.smithi016.stdout:1/571: creat d14/d19/d1d/d45/d4f/d86/fbb x:0 0 0 2024-04-02T04:42:43.622 INFO:tasks.workunit.client.0.smithi016.stdout:1/572: chown d14/d41/f93 93 1 2024-04-02T04:42:43.622 INFO:tasks.workunit.client.1.smithi027.stdout:2/53: creat d1/df/f11 x:0 0 0 2024-04-02T04:42:43.623 INFO:tasks.workunit.client.1.smithi027.stdout:2/54: read d1/f6 [8325685,92770] 0 2024-04-02T04:42:43.624 INFO:tasks.workunit.client.0.smithi016.stdout:5/567: dread d3/df/d18/f42 [0,4194304] 0 2024-04-02T04:42:43.626 INFO:tasks.workunit.client.0.smithi016.stdout:0/564: sync 2024-04-02T04:42:43.626 INFO:tasks.workunit.client.0.smithi016.stdout:8/576: sync 2024-04-02T04:42:43.626 INFO:tasks.workunit.client.0.smithi016.stdout:3/685: sync 2024-04-02T04:42:43.626 INFO:tasks.workunit.client.0.smithi016.stdout:7/569: sync 2024-04-02T04:42:43.627 INFO:tasks.workunit.client.0.smithi016.stdout:5/568: dread d3/df/f68 [0,4194304] 0 2024-04-02T04:42:43.627 INFO:tasks.workunit.client.1.smithi027.stdout:4/51: sync 2024-04-02T04:42:43.628 INFO:tasks.workunit.client.1.smithi027.stdout:4/52: stat l7 0 2024-04-02T04:42:43.630 INFO:tasks.workunit.client.1.smithi027.stdout:1/32: dread d0/f6 [0,4194304] 0 2024-04-02T04:42:43.634 INFO:tasks.workunit.client.0.smithi016.stdout:1/573: symlink d14/d19/lbc 0 2024-04-02T04:42:43.634 INFO:tasks.workunit.client.0.smithi016.stdout:1/574: chown d14/d41/f93 919 1 2024-04-02T04:42:43.634 INFO:tasks.workunit.client.1.smithi027.stdout:0/82: getdents d4/d16 0 2024-04-02T04:42:43.635 INFO:tasks.workunit.client.0.smithi016.stdout:7/570: write d0/f9c [4355584,19332] 0 2024-04-02T04:42:43.635 INFO:tasks.workunit.client.1.smithi027.stdout:0/83: fsync f2 0 2024-04-02T04:42:43.636 INFO:tasks.workunit.client.0.smithi016.stdout:7/571: dread - d0/d46/d37/dbe/fbc zero size 2024-04-02T04:42:43.637 INFO:tasks.workunit.client.1.smithi027.stdout:1/33: dwrite d0/f9 [0,4194304] 0 2024-04-02T04:42:43.637 INFO:tasks.workunit.client.0.smithi016.stdout:0/565: creat d0/d16/d23/d3e/fc6 x:0 0 0 2024-04-02T04:42:43.639 INFO:tasks.workunit.client.1.smithi027.stdout:2/55: mkdir d1/d7/d12 0 2024-04-02T04:42:43.640 INFO:tasks.workunit.client.0.smithi016.stdout:0/566: read d0/d16/d24/d4e/d8d/d71/f7d [565829,2140] 0 2024-04-02T04:42:43.640 INFO:tasks.workunit.client.0.smithi016.stdout:5/569: stat d3/df/d18/f43 0 2024-04-02T04:42:43.640 INFO:tasks.workunit.client.1.smithi027.stdout:2/56: write d1/df/f11 [1044719,62916] 0 2024-04-02T04:42:43.641 INFO:tasks.workunit.client.0.smithi016.stdout:0/567: write d0/d16/d2e/d37/fae [128497,63214] 0 2024-04-02T04:42:43.641 INFO:tasks.workunit.client.1.smithi027.stdout:0/84: dread d4/fa [0,4194304] 0 2024-04-02T04:42:43.644 INFO:tasks.workunit.client.1.smithi027.stdout:4/53: creat f8 x:0 0 0 2024-04-02T04:42:43.647 INFO:tasks.workunit.client.0.smithi016.stdout:9/670: rename d0/d3/d31 to d0/d9/db/dc9 0 2024-04-02T04:42:43.647 INFO:tasks.workunit.client.1.smithi027.stdout:7/49: mkdir d2/d7/da/dc 0 2024-04-02T04:42:43.650 INFO:tasks.workunit.client.0.smithi016.stdout:3/686: creat d2/d15/d60/fe5 x:0 0 0 2024-04-02T04:42:43.651 INFO:tasks.workunit.client.1.smithi027.stdout:4/54: dread f5 [0,4194304] 0 2024-04-02T04:42:43.654 INFO:tasks.workunit.client.0.smithi016.stdout:7/572: creat d0/d46/d37/dbe/dae/d9d/fcb x:0 0 0 2024-04-02T04:42:43.654 INFO:tasks.workunit.client.0.smithi016.stdout:5/570: dread d3/f6 [0,4194304] 0 2024-04-02T04:42:43.655 INFO:tasks.workunit.client.0.smithi016.stdout:0/568: mkdir d0/d16/d24/d4e/d8d/d71/dc7 0 2024-04-02T04:42:43.656 INFO:tasks.workunit.client.0.smithi016.stdout:8/577: truncate d1/d6/d36/d3f/d40/d45/fb7 1371535 0 2024-04-02T04:42:43.658 INFO:tasks.workunit.client.1.smithi027.stdout:9/50: getdents . 0 2024-04-02T04:42:43.659 INFO:tasks.workunit.client.0.smithi016.stdout:2/656: sync 2024-04-02T04:42:43.660 INFO:tasks.workunit.client.1.smithi027.stdout:1/34: rename d0/fc to d0/de/ff 0 2024-04-02T04:42:43.661 INFO:tasks.workunit.client.0.smithi016.stdout:2/657: chown d12/d9f/dac/f2f 197698348 1 2024-04-02T04:42:43.662 INFO:tasks.workunit.client.1.smithi027.stdout:9/51: dread fa [0,4194304] 0 2024-04-02T04:42:43.662 INFO:tasks.workunit.client.0.smithi016.stdout:0/569: dwrite d0/d16/d2e/d37/d66/faa [0,4194304] 0 2024-04-02T04:42:43.663 INFO:tasks.workunit.client.0.smithi016.stdout:2/658: write d12/d9f/dac/f44 [813519,93653] 0 2024-04-02T04:42:43.666 INFO:tasks.workunit.client.0.smithi016.stdout:2/659: chown d12/d9f/dac/f2f 0 1 2024-04-02T04:42:43.667 INFO:tasks.workunit.client.0.smithi016.stdout:8/578: dwrite d1/d6/d36/d3f/d77/fa7 [0,4194304] 0 2024-04-02T04:42:43.667 INFO:tasks.workunit.client.0.smithi016.stdout:3/687: creat d2/d2e/d36/d48/da2/d88/fe6 x:0 0 0 2024-04-02T04:42:43.668 INFO:tasks.workunit.client.1.smithi027.stdout:7/50: creat d2/d7/fd x:0 0 0 2024-04-02T04:42:43.668 INFO:tasks.workunit.client.0.smithi016.stdout:8/579: chown d1/d6/l9 0 1 2024-04-02T04:42:43.668 INFO:tasks.workunit.client.1.smithi027.stdout:7/51: stat d2/d7/da 0 2024-04-02T04:42:43.669 INFO:tasks.workunit.client.0.smithi016.stdout:7/573: readlink d0/d46/d37/d3a/d4c/l8f 0 2024-04-02T04:42:43.669 INFO:tasks.workunit.client.1.smithi027.stdout:7/52: truncate d2/d7/fd 863213 0 2024-04-02T04:42:43.671 INFO:tasks.workunit.client.0.smithi016.stdout:7/574: write d0/d46/d37/dbe/dae/d9d/fcb [153398,125185] 0 2024-04-02T04:42:43.671 INFO:tasks.workunit.client.1.smithi027.stdout:4/55: creat f9 x:0 0 0 2024-04-02T04:42:43.676 INFO:tasks.workunit.client.1.smithi027.stdout:7/53: dread f1 [0,4194304] 0 2024-04-02T04:42:43.680 INFO:tasks.workunit.client.1.smithi027.stdout:7/54: dread f1 [0,4194304] 0 2024-04-02T04:42:43.680 INFO:tasks.workunit.client.1.smithi027.stdout:7/55: rename d2 to d2/de 22 2024-04-02T04:42:43.682 INFO:tasks.workunit.client.0.smithi016.stdout:9/671: link d0/d9/db/dc9/d53/f73 d0/d4a/d75/d95/fca 0 2024-04-02T04:42:43.682 INFO:tasks.workunit.client.1.smithi027.stdout:1/35: symlink d0/de/l10 0 2024-04-02T04:42:43.683 INFO:tasks.workunit.client.1.smithi027.stdout:7/56: dread f1 [0,4194304] 0 2024-04-02T04:42:43.684 INFO:tasks.workunit.client.1.smithi027.stdout:7/57: fsync f1 0 2024-04-02T04:42:43.685 INFO:tasks.workunit.client.1.smithi027.stdout:9/52: write f5 [4254724,25012] 0 2024-04-02T04:42:43.686 INFO:tasks.workunit.client.1.smithi027.stdout:3/32: truncate d1/f2 97743 0 2024-04-02T04:42:43.686 INFO:tasks.workunit.client.1.smithi027.stdout:3/33: fsync d1/f8 0 2024-04-02T04:42:43.687 INFO:tasks.workunit.client.1.smithi027.stdout:2/57: link d1/f6 d1/f13 0 2024-04-02T04:42:43.688 INFO:tasks.workunit.client.1.smithi027.stdout:3/34: write d1/f8 [1980821,22308] 0 2024-04-02T04:42:43.688 INFO:tasks.workunit.client.1.smithi027.stdout:3/35: readlink d1/l6 0 2024-04-02T04:42:43.688 INFO:tasks.workunit.client.1.smithi027.stdout:3/36: rename d1 to d1/d9 22 2024-04-02T04:42:43.691 INFO:tasks.workunit.client.1.smithi027.stdout:0/85: unlink d4/fa 0 2024-04-02T04:42:43.693 INFO:tasks.workunit.client.0.smithi016.stdout:3/688: mkdir d2/d2e/d36/d48/da2/de7 0 2024-04-02T04:42:43.695 INFO:tasks.workunit.client.1.smithi027.stdout:4/56: rename f8 to fa 0 2024-04-02T04:42:43.700 INFO:tasks.workunit.client.1.smithi027.stdout:7/58: mkdir d2/df 0 2024-04-02T04:42:43.700 INFO:tasks.workunit.client.1.smithi027.stdout:7/59: fsync f1 0 2024-04-02T04:42:43.701 INFO:tasks.workunit.client.1.smithi027.stdout:7/60: readlink d2/l3 0 2024-04-02T04:42:43.702 INFO:tasks.workunit.client.0.smithi016.stdout:5/571: rename d3/df/d18/f76 to d3/d40/d72/fd0 0 2024-04-02T04:42:43.702 INFO:tasks.workunit.client.1.smithi027.stdout:6/58: getdents d13 0 2024-04-02T04:42:43.703 INFO:tasks.workunit.client.0.smithi016.stdout:8/580: dwrite d1/f3 [0,4194304] 0 2024-04-02T04:42:43.704 INFO:tasks.workunit.client.1.smithi027.stdout:9/53: creat f12 x:0 0 0 2024-04-02T04:42:43.707 INFO:tasks.workunit.client.1.smithi027.stdout:2/58: rmdir d1/d7 39 2024-04-02T04:42:43.712 INFO:tasks.workunit.client.1.smithi027.stdout:3/37: rename d1/c5 to d1/ca 0 2024-04-02T04:42:43.712 INFO:tasks.workunit.client.1.smithi027.stdout:9/54: dread f5 [0,4194304] 0 2024-04-02T04:42:43.713 INFO:tasks.workunit.client.1.smithi027.stdout:6/59: dwrite f7 [0,4194304] 0 2024-04-02T04:42:43.714 INFO:tasks.workunit.client.1.smithi027.stdout:0/86: mkdir d4/d16/d1b 0 2024-04-02T04:42:43.716 INFO:tasks.workunit.client.1.smithi027.stdout:0/87: write d4/d13/fd [2053948,94230] 0 2024-04-02T04:42:43.716 INFO:tasks.workunit.client.0.smithi016.stdout:7/575: symlink d0/d46/d37/d3a/lcc 0 2024-04-02T04:42:43.717 INFO:tasks.workunit.client.1.smithi027.stdout:0/88: chown d4/d13/fd 1529 1 2024-04-02T04:42:43.717 INFO:tasks.workunit.client.0.smithi016.stdout:7/576: fsync d0/d46/d16/d49/fca 0 2024-04-02T04:42:43.718 INFO:tasks.workunit.client.1.smithi027.stdout:9/55: dread fa [0,4194304] 0 2024-04-02T04:42:43.719 INFO:tasks.workunit.client.0.smithi016.stdout:6/659: creat d0/d9/d16/d56/fcb x:0 0 0 2024-04-02T04:42:43.722 INFO:tasks.workunit.client.1.smithi027.stdout:4/57: symlink lb 0 2024-04-02T04:42:43.724 INFO:tasks.workunit.client.1.smithi027.stdout:4/58: chown lb 135910287 1 2024-04-02T04:42:43.726 INFO:tasks.workunit.client.0.smithi016.stdout:8/581: rename d1/d70/d71/d74/d82/cb0 to d1/d70/d71/d74/d82/cc6 0 2024-04-02T04:42:43.726 INFO:tasks.workunit.client.1.smithi027.stdout:4/59: truncate f9 78879 0 2024-04-02T04:42:43.729 INFO:tasks.workunit.client.1.smithi027.stdout:0/89: dwrite d4/f11 [0,4194304] 0 2024-04-02T04:42:43.729 INFO:tasks.workunit.client.1.smithi027.stdout:9/56: dwrite f5 [8388608,4194304] 0 2024-04-02T04:42:43.733 INFO:tasks.workunit.client.0.smithi016.stdout:0/570: creat d0/fc8 x:0 0 0 2024-04-02T04:42:43.735 INFO:tasks.workunit.client.1.smithi027.stdout:2/59: chown d1/d7 3451381 1 2024-04-02T04:42:43.739 INFO:tasks.workunit.client.1.smithi027.stdout:6/60: symlink d13/l15 0 2024-04-02T04:42:43.740 INFO:tasks.workunit.client.0.smithi016.stdout:6/660: write d0/d9/d16/d56/d89/f42 [8762,106827] 0 2024-04-02T04:42:43.742 INFO:tasks.workunit.client.0.smithi016.stdout:3/689: rename d2/d2e/d36/d66/d7a/lba to d2/d15/d16/d8a/dd1/d6e/d6f/d78/le8 0 2024-04-02T04:42:43.745 INFO:tasks.workunit.client.1.smithi027.stdout:7/61: creat d2/df/f10 x:0 0 0 2024-04-02T04:42:43.747 INFO:tasks.workunit.client.0.smithi016.stdout:6/661: dwrite d0/d9/d16/f20 [0,4194304] 0 2024-04-02T04:42:43.753 INFO:tasks.workunit.client.1.smithi027.stdout:9/57: unlink ld 0 2024-04-02T04:42:43.756 INFO:tasks.workunit.client.1.smithi027.stdout:0/90: creat d4/d16/f1c x:0 0 0 2024-04-02T04:42:43.757 INFO:tasks.workunit.client.1.smithi027.stdout:0/91: write d4/f14 [795351,34038] 0 2024-04-02T04:42:43.758 INFO:tasks.workunit.client.0.smithi016.stdout:3/690: mknod d2/d15/d60/ce9 0 2024-04-02T04:42:43.759 INFO:tasks.workunit.client.1.smithi027.stdout:0/92: read d4/d13/fd [3563961,71140] 0 2024-04-02T04:42:43.761 INFO:tasks.workunit.client.1.smithi027.stdout:0/93: write d4/d16/f1c [162142,76824] 0 2024-04-02T04:42:43.761 INFO:tasks.workunit.client.0.smithi016.stdout:8/582: mknod d1/d6/d36/d3f/d40/d45/cc7 0 2024-04-02T04:42:43.763 INFO:tasks.workunit.client.1.smithi027.stdout:6/61: creat d13/f16 x:0 0 0 2024-04-02T04:42:43.763 INFO:tasks.workunit.client.0.smithi016.stdout:3/691: dwrite d2/d15/d1a/f47 [0,4194304] 0 2024-04-02T04:42:43.765 INFO:tasks.workunit.client.0.smithi016.stdout:9/672: getdents d0/d4a/d75 0 2024-04-02T04:42:43.765 INFO:tasks.workunit.client.1.smithi027.stdout:5/126: dwrite d0/f6 [0,4194304] 0 2024-04-02T04:42:43.765 INFO:tasks.workunit.client.0.smithi016.stdout:3/692: dread - d2/d15/d16/d8a/f93 zero size 2024-04-02T04:42:43.770 INFO:tasks.workunit.client.0.smithi016.stdout:6/662: fdatasync d0/d5c/f6c 0 2024-04-02T04:42:43.771 INFO:tasks.workunit.client.1.smithi027.stdout:6/62: dread f12 [0,4194304] 0 2024-04-02T04:42:43.776 INFO:tasks.workunit.client.1.smithi027.stdout:9/58: fsync fa 0 2024-04-02T04:42:43.776 INFO:tasks.workunit.client.0.smithi016.stdout:3/693: dwrite d2/d15/d60/fc9 [0,4194304] 0 2024-04-02T04:42:43.783 INFO:tasks.workunit.client.1.smithi027.stdout:2/60: link d1/d7/fc d1/d7/f14 0 2024-04-02T04:42:43.787 INFO:tasks.workunit.client.1.smithi027.stdout:8/44: dread f4 [0,4194304] 0 2024-04-02T04:42:43.788 INFO:tasks.workunit.client.0.smithi016.stdout:8/583: rmdir d1/d6/d36/d8e 39 2024-04-02T04:42:43.790 INFO:tasks.workunit.client.0.smithi016.stdout:8/584: stat d1/d6/d36/d3f/d40/d45/fbc 0 2024-04-02T04:42:43.791 INFO:tasks.workunit.client.1.smithi027.stdout:0/94: creat d4/f1d x:0 0 0 2024-04-02T04:42:43.792 INFO:tasks.workunit.client.0.smithi016.stdout:9/673: chown d0/d9/db/dc9/d53/c91 28208886 1 2024-04-02T04:42:43.792 INFO:tasks.workunit.client.0.smithi016.stdout:9/674: chown d0/d3/d97/dac 28765701 1 2024-04-02T04:42:43.792 INFO:tasks.workunit.client.0.smithi016.stdout:4/612: truncate d5/d16/f55 3916196 0 2024-04-02T04:42:43.794 INFO:tasks.workunit.client.0.smithi016.stdout:9/675: dread d0/d4e/d50/f85 [0,4194304] 0 2024-04-02T04:42:43.795 INFO:tasks.workunit.client.0.smithi016.stdout:9/676: stat d0/d4a/d99/d46/f87 0 2024-04-02T04:42:43.797 INFO:tasks.workunit.client.1.smithi027.stdout:7/62: mknod d2/d7/da/dc/c11 0 2024-04-02T04:42:43.797 INFO:tasks.workunit.client.1.smithi027.stdout:0/95: dwrite f1 [4194304,4194304] 0 2024-04-02T04:42:43.799 INFO:tasks.workunit.client.1.smithi027.stdout:6/63: creat d13/f17 x:0 0 0 2024-04-02T04:42:43.800 INFO:tasks.workunit.client.0.smithi016.stdout:6/663: creat d0/d6e/fcc x:0 0 0 2024-04-02T04:42:43.802 INFO:tasks.workunit.client.1.smithi027.stdout:0/96: dread f1 [4194304,4194304] 0 2024-04-02T04:42:43.803 INFO:tasks.workunit.client.0.smithi016.stdout:0/571: creat d0/d16/d2e/d53/fc9 x:0 0 0 2024-04-02T04:42:43.806 INFO:tasks.workunit.client.0.smithi016.stdout:7/577: link d0/d46/d37/d3a/d84/d8a/l91 d0/d46/d37/dbe/dae/lcd 0 2024-04-02T04:42:43.807 INFO:tasks.workunit.client.0.smithi016.stdout:7/578: chown d0/d46/d37/d47/c6c 0 1 2024-04-02T04:42:43.809 INFO:tasks.workunit.client.0.smithi016.stdout:8/585: creat d1/d31/d67/fc8 x:0 0 0 2024-04-02T04:42:43.812 INFO:tasks.workunit.client.1.smithi027.stdout:8/45: symlink d5/la 0 2024-04-02T04:42:43.813 INFO:tasks.workunit.client.0.smithi016.stdout:4/613: chown d5/d1d/c30 86111 1 2024-04-02T04:42:43.813 INFO:tasks.workunit.client.0.smithi016.stdout:4/614: dread - d5/d17/d76/f8b zero size 2024-04-02T04:42:43.814 INFO:tasks.workunit.client.0.smithi016.stdout:4/615: chown d5/d17/d18/d6d/d8e/laf 0 1 2024-04-02T04:42:43.816 INFO:tasks.workunit.client.0.smithi016.stdout:9/677: rename d0/d4e/d50/d67/c8c to d0/d4e/d50/d63/d6e/d4b/d58/d6a/ccb 0 2024-04-02T04:42:43.819 INFO:tasks.workunit.client.1.smithi027.stdout:7/63: creat d2/d7/f12 x:0 0 0 2024-04-02T04:42:43.820 INFO:tasks.workunit.client.0.smithi016.stdout:0/572: rmdir d0/d16/d2e/d53/d9d 39 2024-04-02T04:42:43.822 INFO:tasks.workunit.client.1.smithi027.stdout:6/64: rename f3 to d13/f18 0 2024-04-02T04:42:43.823 INFO:tasks.workunit.client.0.smithi016.stdout:8/586: fdatasync d1/d70/f88 0 2024-04-02T04:42:43.827 INFO:tasks.workunit.client.0.smithi016.stdout:1/575: truncate d14/f48 4187692 0 2024-04-02T04:42:43.827 INFO:tasks.workunit.client.1.smithi027.stdout:2/61: mknod d1/db/c15 0 2024-04-02T04:42:43.829 INFO:tasks.workunit.client.0.smithi016.stdout:4/616: rmdir d5/d17/d18/d6d/da4 39 2024-04-02T04:42:43.834 INFO:tasks.workunit.client.0.smithi016.stdout:6/664: symlink d0/lcd 0 2024-04-02T04:42:43.835 INFO:tasks.workunit.client.1.smithi027.stdout:1/36: rmdir d0 39 2024-04-02T04:42:43.839 INFO:tasks.workunit.client.1.smithi027.stdout:6/65: symlink d13/l19 0 2024-04-02T04:42:43.840 INFO:tasks.workunit.client.1.smithi027.stdout:0/97: mknod d4/d16/d1b/c1e 0 2024-04-02T04:42:43.843 INFO:tasks.workunit.client.0.smithi016.stdout:2/660: dwrite d12/d35/d3f/f34 [4194304,4194304] 0 2024-04-02T04:42:43.843 INFO:tasks.workunit.client.1.smithi027.stdout:4/60: rename fa to fc 0 2024-04-02T04:42:43.845 INFO:tasks.workunit.client.1.smithi027.stdout:8/46: mkdir d5/d8/db 0 2024-04-02T04:42:43.846 INFO:tasks.workunit.client.1.smithi027.stdout:8/47: rename d5/d8 to d5/d8/db/dc 22 2024-04-02T04:42:43.848 INFO:tasks.workunit.client.1.smithi027.stdout:9/59: fsync f12 0 2024-04-02T04:42:43.848 INFO:tasks.workunit.client.1.smithi027.stdout:5/127: link d0/df/d11/d13/c18 d0/d1d/c36 0 2024-04-02T04:42:43.850 INFO:tasks.workunit.client.0.smithi016.stdout:4/617: mknod d5/d16/d88/db0/cb8 0 2024-04-02T04:42:43.854 INFO:tasks.workunit.client.0.smithi016.stdout:4/618: dread d5/d1d/d1e/f9a [0,4194304] 0 2024-04-02T04:42:43.855 INFO:tasks.workunit.client.1.smithi027.stdout:7/64: unlink d2/d7/fd 0 2024-04-02T04:42:43.855 INFO:tasks.workunit.client.1.smithi027.stdout:7/65: chown d2/d7 1357 1 2024-04-02T04:42:43.856 INFO:tasks.workunit.client.1.smithi027.stdout:7/66: chown d2/df 0 1 2024-04-02T04:42:43.859 INFO:tasks.workunit.client.0.smithi016.stdout:7/579: rename d0/d46/d37/dbe/dae/d9d/fcb to d0/d46/dbf/fce 0 2024-04-02T04:42:43.860 INFO:tasks.workunit.client.0.smithi016.stdout:5/572: write d3/df/d18/d2f/f62 [185666,81657] 0 2024-04-02T04:42:43.860 INFO:tasks.workunit.client.0.smithi016.stdout:1/576: dread d14/d19/d1e/d5e/fac [0,4194304] 0 2024-04-02T04:42:43.861 INFO:tasks.workunit.client.1.smithi027.stdout:6/66: creat d13/f1a x:0 0 0 2024-04-02T04:42:43.861 INFO:tasks.workunit.client.1.smithi027.stdout:6/67: stat c10 0 2024-04-02T04:42:43.862 INFO:tasks.workunit.client.0.smithi016.stdout:9/678: symlink d0/d9/d4d/d78/da6/dc1/lcc 0 2024-04-02T04:42:43.864 INFO:tasks.workunit.client.0.smithi016.stdout:4/619: dread d5/d8/ff [0,4194304] 0 2024-04-02T04:42:43.864 INFO:tasks.workunit.client.0.smithi016.stdout:1/577: dread d14/d19/d1e/d5e/fac [0,4194304] 0 2024-04-02T04:42:43.866 INFO:tasks.workunit.client.0.smithi016.stdout:3/694: getdents d2/d15/d16/d8a/dd1/d6e/d6f/db0 0 2024-04-02T04:42:43.867 INFO:tasks.workunit.client.1.smithi027.stdout:4/61: rename l6 to ld 0 2024-04-02T04:42:43.870 INFO:tasks.workunit.client.0.smithi016.stdout:2/661: creat d12/d70/fe8 x:0 0 0 2024-04-02T04:42:43.872 INFO:tasks.workunit.client.1.smithi027.stdout:9/60: unlink c2 0 2024-04-02T04:42:43.873 INFO:tasks.workunit.client.1.smithi027.stdout:5/128: mkdir d0/df/d11/d37 0 2024-04-02T04:42:43.874 INFO:tasks.workunit.client.1.smithi027.stdout:5/129: readlink d0/l1 0 2024-04-02T04:42:43.875 INFO:tasks.workunit.client.1.smithi027.stdout:4/62: dwrite f5 [0,4194304] 0 2024-04-02T04:42:43.879 INFO:tasks.workunit.client.1.smithi027.stdout:7/67: creat d2/f13 x:0 0 0 2024-04-02T04:42:43.880 INFO:tasks.workunit.client.1.smithi027.stdout:7/68: rename d2/d7/da to d2/d7/da/d14 22 2024-04-02T04:42:43.880 INFO:tasks.workunit.client.0.smithi016.stdout:5/573: write d3/df/d18/f89 [3687485,32668] 0 2024-04-02T04:42:43.881 INFO:tasks.workunit.client.1.smithi027.stdout:7/69: dread - d2/f13 zero size 2024-04-02T04:42:43.881 INFO:tasks.workunit.client.1.smithi027.stdout:1/37: mknod d0/de/c11 0 2024-04-02T04:42:43.883 INFO:tasks.workunit.client.0.smithi016.stdout:9/679: symlink d0/d4e/d50/d63/d6e/d4b/lcd 0 2024-04-02T04:42:43.886 INFO:tasks.workunit.client.0.smithi016.stdout:6/665: fsync d0/d9/df/f10 0 2024-04-02T04:42:43.889 INFO:tasks.workunit.client.0.smithi016.stdout:9/680: dwrite d0/d4e/d50/d63/d6e/f77 [0,4194304] 0 2024-04-02T04:42:43.889 INFO:tasks.workunit.client.1.smithi027.stdout:8/48: mkdir d5/d8/db/dd 0 2024-04-02T04:42:43.892 INFO:tasks.workunit.client.0.smithi016.stdout:1/578: creat d14/d19/d1d/d45/fbd x:0 0 0 2024-04-02T04:42:43.893 INFO:tasks.workunit.client.0.smithi016.stdout:1/579: dread - d14/da9/f9b zero size 2024-04-02T04:42:43.894 INFO:tasks.workunit.client.0.smithi016.stdout:1/580: truncate d14/d19/d1d/d45/f92 618141 0 2024-04-02T04:42:43.895 INFO:tasks.workunit.client.1.smithi027.stdout:7/70: dwrite d2/d7/f12 [0,4194304] 0 2024-04-02T04:42:43.895 INFO:tasks.workunit.client.1.smithi027.stdout:9/61: creat f13 x:0 0 0 2024-04-02T04:42:43.897 INFO:tasks.workunit.client.1.smithi027.stdout:2/62: dwrite d1/f6 [4194304,4194304] 0 2024-04-02T04:42:43.899 INFO:tasks.workunit.client.1.smithi027.stdout:9/62: write f5 [10711378,26033] 0 2024-04-02T04:42:43.899 INFO:tasks.workunit.client.1.smithi027.stdout:7/71: stat d2/l3 0 2024-04-02T04:42:43.902 INFO:tasks.workunit.client.0.smithi016.stdout:0/573: link d0/d4f/cb6 d0/d16/d23/d3e/db3/cca 0 2024-04-02T04:42:43.903 INFO:tasks.workunit.client.1.smithi027.stdout:2/63: fsync d1/df/f11 0 2024-04-02T04:42:43.904 INFO:tasks.workunit.client.1.smithi027.stdout:6/68: dwrite d13/f18 [0,4194304] 0 2024-04-02T04:42:43.905 INFO:tasks.workunit.client.1.smithi027.stdout:4/63: symlink le 0 2024-04-02T04:42:43.905 INFO:tasks.workunit.client.1.smithi027.stdout:2/64: chown d1/d7/l10 206084824 1 2024-04-02T04:42:43.905 INFO:tasks.workunit.client.1.smithi027.stdout:6/69: chown d13/l15 13909 1 2024-04-02T04:42:43.907 INFO:tasks.workunit.client.0.smithi016.stdout:2/662: mkdir d12/d35/d3f/d2b/de9 0 2024-04-02T04:42:43.907 INFO:tasks.workunit.client.1.smithi027.stdout:2/65: stat d1/dd 0 2024-04-02T04:42:43.912 INFO:tasks.workunit.client.1.smithi027.stdout:1/38: fdatasync d0/de/ff 0 2024-04-02T04:42:43.914 INFO:tasks.workunit.client.1.smithi027.stdout:7/72: dwrite d2/d7/f12 [0,4194304] 0 2024-04-02T04:42:43.915 INFO:tasks.workunit.client.1.smithi027.stdout:1/39: fdatasync d0/f9 0 2024-04-02T04:42:43.916 INFO:tasks.workunit.client.1.smithi027.stdout:1/40: chown d0/l3 743 1 2024-04-02T04:42:43.921 INFO:tasks.workunit.client.0.smithi016.stdout:5/574: dwrite d3/df/f22 [4194304,4194304] 0 2024-04-02T04:42:43.926 INFO:tasks.workunit.client.1.smithi027.stdout:9/63: rename f13 to f14 0 2024-04-02T04:42:43.936 INFO:tasks.workunit.client.0.smithi016.stdout:3/695: symlink d2/d15/d16/lea 0 2024-04-02T04:42:43.936 INFO:tasks.workunit.client.0.smithi016.stdout:6/666: dwrite d0/d9/f33 [0,4194304] 0 2024-04-02T04:42:43.936 INFO:tasks.workunit.client.1.smithi027.stdout:9/64: dwrite f12 [0,4194304] 0 2024-04-02T04:42:43.937 INFO:tasks.workunit.client.0.smithi016.stdout:6/667: stat d0/d5c/c88 0 2024-04-02T04:42:43.938 INFO:tasks.workunit.client.0.smithi016.stdout:0/574: fsync d0/d16/f7a 0 2024-04-02T04:42:43.940 INFO:tasks.workunit.client.1.smithi027.stdout:4/64: creat ff x:0 0 0 2024-04-02T04:42:43.941 INFO:tasks.workunit.client.0.smithi016.stdout:0/575: chown d0/d16/d23/f3f 18769706 1 2024-04-02T04:42:43.941 INFO:tasks.workunit.client.0.smithi016.stdout:6/668: write d0/d9/d16/d56/d89/d67/d6f/f7f [1331774,23197] 0 2024-04-02T04:42:43.944 INFO:tasks.workunit.client.1.smithi027.stdout:6/70: mkdir d13/d1b 0 2024-04-02T04:42:43.947 INFO:tasks.workunit.client.1.smithi027.stdout:2/66: rmdir d1/df 39 2024-04-02T04:42:43.949 INFO:tasks.workunit.client.1.smithi027.stdout:3/38: link d1/ca d1/cb 0 2024-04-02T04:42:43.950 INFO:tasks.workunit.client.1.smithi027.stdout:3/39: chown d1/f7 41403923 1 2024-04-02T04:42:43.950 INFO:tasks.workunit.client.1.smithi027.stdout:0/98: rmdir d4/d16 39 2024-04-02T04:42:43.951 INFO:tasks.workunit.client.0.smithi016.stdout:0/576: dwrite d0/d16/d2e/d53/f7c [0,4194304] 0 2024-04-02T04:42:43.951 INFO:tasks.workunit.client.1.smithi027.stdout:0/99: chown d4/f14 1946419 1 2024-04-02T04:42:43.952 INFO:tasks.workunit.client.1.smithi027.stdout:0/100: write d4/f14 [629186,91512] 0 2024-04-02T04:42:43.953 INFO:tasks.workunit.client.0.smithi016.stdout:3/696: rmdir d2/d15/d16/d8a 39 2024-04-02T04:42:43.959 INFO:tasks.workunit.client.0.smithi016.stdout:0/577: dwrite d0/d16/d2e/f85 [0,4194304] 0 2024-04-02T04:42:43.960 INFO:tasks.workunit.client.1.smithi027.stdout:1/41: mknod d0/c12 0 2024-04-02T04:42:43.964 INFO:tasks.workunit.client.0.smithi016.stdout:8/587: dwrite d1/d6/f10 [0,4194304] 0 2024-04-02T04:42:43.967 INFO:tasks.workunit.client.0.smithi016.stdout:8/588: write d1/d6/d36/d3f/d40/d49/f44 [435155,103470] 0 2024-04-02T04:42:43.968 INFO:tasks.workunit.client.0.smithi016.stdout:0/578: dwrite d0/d16/d24/d4e/d8d/fc2 [0,4194304] 0 2024-04-02T04:42:43.970 INFO:tasks.workunit.client.0.smithi016.stdout:0/579: read d0/d16/d2e/d53/f7c [1703298,22453] 0 2024-04-02T04:42:43.970 INFO:tasks.workunit.client.1.smithi027.stdout:1/42: dwrite d0/de/ff [0,4194304] 0 2024-04-02T04:42:43.970 INFO:tasks.workunit.client.0.smithi016.stdout:4/620: rename d5/d17/c38 to d5/d17/d18/cb9 0 2024-04-02T04:42:43.970 INFO:tasks.workunit.client.0.smithi016.stdout:9/681: rename d0/d4e/d50/d63/d6e to d0/d4e/d50/d63/d6e/d4b/d58/d6a/dce 22 2024-04-02T04:42:43.983 INFO:tasks.workunit.client.0.smithi016.stdout:2/663: creat d12/d35/fea x:0 0 0 2024-04-02T04:42:43.985 INFO:tasks.workunit.client.0.smithi016.stdout:4/621: dread d5/d17/d18/f7f [0,4194304] 0 2024-04-02T04:42:43.986 INFO:tasks.workunit.client.1.smithi027.stdout:4/65: rename l7 to l10 0 2024-04-02T04:42:43.987 INFO:tasks.workunit.client.0.smithi016.stdout:6/669: mkdir d0/d9/d16/d56/d89/dc5/dce 0 2024-04-02T04:42:43.988 INFO:tasks.workunit.client.1.smithi027.stdout:6/71: mknod d13/c1c 0 2024-04-02T04:42:43.991 INFO:tasks.workunit.client.1.smithi027.stdout:2/67: mknod d1/db/c16 0 2024-04-02T04:42:43.992 INFO:tasks.workunit.client.0.smithi016.stdout:2/664: dwrite d12/d9f/dac/d76/f92 [0,4194304] 0 2024-04-02T04:42:43.995 INFO:tasks.workunit.client.0.smithi016.stdout:7/580: dwrite d0/d28/f39 [0,4194304] 0 2024-04-02T04:42:43.996 INFO:tasks.workunit.client.1.smithi027.stdout:3/40: fdatasync d1/f7 0 2024-04-02T04:42:44.002 INFO:tasks.workunit.client.0.smithi016.stdout:1/581: rename d14/l2f to d14/d19/d1d/d45/lbe 0 2024-04-02T04:42:44.006 INFO:tasks.workunit.client.0.smithi016.stdout:3/697: mknod d2/d15/d16/d8a/dcd/ceb 0 2024-04-02T04:42:44.011 INFO:tasks.workunit.client.1.smithi027.stdout:1/43: symlink d0/l13 0 2024-04-02T04:42:44.013 INFO:tasks.workunit.client.1.smithi027.stdout:1/44: write d0/f7 [469993,43916] 0 2024-04-02T04:42:44.014 INFO:tasks.workunit.client.1.smithi027.stdout:1/45: chown d0/l8 138360 1 2024-04-02T04:42:44.015 INFO:tasks.workunit.client.1.smithi027.stdout:4/66: creat f11 x:0 0 0 2024-04-02T04:42:44.017 INFO:tasks.workunit.client.1.smithi027.stdout:4/67: write f5 [5230494,65311] 0 2024-04-02T04:42:44.018 INFO:tasks.workunit.client.1.smithi027.stdout:7/73: dwrite d2/d7/f12 [4194304,4194304] 0 2024-04-02T04:42:44.023 INFO:tasks.workunit.client.0.smithi016.stdout:7/581: unlink d0/d46/l5f 0 2024-04-02T04:42:44.025 INFO:tasks.workunit.client.1.smithi027.stdout:5/130: dwrite d0/f33 [0,4194304] 0 2024-04-02T04:42:44.027 INFO:tasks.workunit.client.1.smithi027.stdout:5/131: truncate d0/d1d/f31 313938 0 2024-04-02T04:42:44.029 INFO:tasks.workunit.client.0.smithi016.stdout:0/580: creat d0/d16/d24/d4e/d8d/d71/dc7/fcb x:0 0 0 2024-04-02T04:42:44.034 INFO:tasks.workunit.client.1.smithi027.stdout:0/101: symlink d4/d16/d1b/l1f 0 2024-04-02T04:42:44.035 INFO:tasks.workunit.client.1.smithi027.stdout:0/102: read - d4/f1d zero size 2024-04-02T04:42:44.036 INFO:tasks.workunit.client.1.smithi027.stdout:8/49: link l1 d5/d8/db/le 0 2024-04-02T04:42:44.037 INFO:tasks.workunit.client.1.smithi027.stdout:8/50: stat d5/d8/db 0 2024-04-02T04:42:44.040 INFO:tasks.workunit.client.0.smithi016.stdout:5/575: rename d3/df/d18/d26/d2c to d3/d4c/dd1 0 2024-04-02T04:42:44.041 INFO:tasks.workunit.client.1.smithi027.stdout:8/51: dread f4 [0,4194304] 0 2024-04-02T04:42:44.042 INFO:tasks.workunit.client.1.smithi027.stdout:0/103: dwrite d4/f11 [0,4194304] 0 2024-04-02T04:42:44.042 INFO:tasks.workunit.client.1.smithi027.stdout:1/46: creat d0/f14 x:0 0 0 2024-04-02T04:42:44.045 INFO:tasks.workunit.client.1.smithi027.stdout:1/47: fdatasync d0/fb 0 2024-04-02T04:42:44.046 INFO:tasks.workunit.client.0.smithi016.stdout:3/698: readlink d2/d15/d16/l41 0 2024-04-02T04:42:44.046 INFO:tasks.workunit.client.0.smithi016.stdout:3/699: chown d2/d15/d16/d8a/dd1 157608 1 2024-04-02T04:42:44.046 INFO:tasks.workunit.client.0.smithi016.stdout:3/700: dread - d2/d2e/d36/d48/da2/d88/fe6 zero size 2024-04-02T04:42:44.048 INFO:tasks.workunit.client.0.smithi016.stdout:4/622: unlink d5/d17/f35 0 2024-04-02T04:42:44.048 INFO:tasks.workunit.client.0.smithi016.stdout:4/623: readlink d5/d1d/d6c/l9b 0 2024-04-02T04:42:44.049 INFO:tasks.workunit.client.0.smithi016.stdout:4/624: dread - d5/d8/d43/f6e zero size 2024-04-02T04:42:44.049 INFO:tasks.workunit.client.1.smithi027.stdout:4/68: creat f12 x:0 0 0 2024-04-02T04:42:44.051 INFO:tasks.workunit.client.0.smithi016.stdout:6/670: creat d0/d9/d16/d56/d59/da1/fcf x:0 0 0 2024-04-02T04:42:44.052 INFO:tasks.workunit.client.1.smithi027.stdout:0/104: dwrite d4/f6 [0,4194304] 0 2024-04-02T04:42:44.053 INFO:tasks.workunit.client.0.smithi016.stdout:6/671: write d0/d9/d16/d56/fbf [444839,122673] 0 2024-04-02T04:42:44.053 INFO:tasks.workunit.client.1.smithi027.stdout:0/105: chown d4/d16/d1b/c1e 22819868 1 2024-04-02T04:42:44.053 INFO:tasks.workunit.client.1.smithi027.stdout:7/74: symlink d2/l15 0 2024-04-02T04:42:44.056 INFO:tasks.workunit.client.0.smithi016.stdout:9/682: creat d0/d9/fcf x:0 0 0 2024-04-02T04:42:44.057 INFO:tasks.workunit.client.1.smithi027.stdout:9/65: dwrite fa [0,4194304] 0 2024-04-02T04:42:44.060 INFO:tasks.workunit.client.1.smithi027.stdout:4/69: dread f2 [4194304,4194304] 0 2024-04-02T04:42:44.060 INFO:tasks.workunit.client.1.smithi027.stdout:4/70: dread - f12 zero size 2024-04-02T04:42:44.061 INFO:tasks.workunit.client.1.smithi027.stdout:4/71: read f9 [19079,12358] 0 2024-04-02T04:42:44.062 INFO:tasks.workunit.client.0.smithi016.stdout:0/581: creat d0/d16/d24/d4e/fcc x:0 0 0 2024-04-02T04:42:44.063 INFO:tasks.workunit.client.0.smithi016.stdout:0/582: write d0/d16/d24/d4e/d8d/fc2 [2241648,51073] 0 2024-04-02T04:42:44.065 INFO:tasks.workunit.client.0.smithi016.stdout:8/589: rename d1/dc0 to d1/d6/d36/d4f/dc9 0 2024-04-02T04:42:44.069 INFO:tasks.workunit.client.0.smithi016.stdout:8/590: write d1/d6/d36/d3f/f80 [865821,25743] 0 2024-04-02T04:42:44.069 INFO:tasks.workunit.client.0.smithi016.stdout:3/701: rmdir d2/d15/d16/d8a/dd1/d6e/d6f/db0 39 2024-04-02T04:42:44.069 INFO:tasks.workunit.client.1.smithi027.stdout:3/41: write d1/f7 [98874,81852] 0 2024-04-02T04:42:44.071 INFO:tasks.workunit.client.1.smithi027.stdout:3/42: read d1/f8 [1907264,19830] 0 2024-04-02T04:42:44.072 INFO:tasks.workunit.client.0.smithi016.stdout:1/582: dwrite d14/d19/d20/f3b [0,4194304] 0 2024-04-02T04:42:44.073 INFO:tasks.workunit.client.0.smithi016.stdout:6/672: dread - d0/d9/df/f74 zero size 2024-04-02T04:42:44.073 INFO:tasks.workunit.client.0.smithi016.stdout:3/702: truncate d2/d2e/fd8 183249 0 2024-04-02T04:42:44.074 INFO:tasks.workunit.client.1.smithi027.stdout:0/106: creat d4/d16/d1b/f20 x:0 0 0 2024-04-02T04:42:44.077 INFO:tasks.workunit.client.1.smithi027.stdout:7/75: mkdir d2/d7/da/dc/d16 0 2024-04-02T04:42:44.078 INFO:tasks.workunit.client.1.smithi027.stdout:7/76: chown d2/l5 0 1 2024-04-02T04:42:44.081 INFO:tasks.workunit.client.1.smithi027.stdout:5/132: truncate d0/df/d11/d13/f1e 2549863 0 2024-04-02T04:42:44.082 INFO:tasks.workunit.client.1.smithi027.stdout:5/133: stat d0/df/d29 0 2024-04-02T04:42:44.083 INFO:tasks.workunit.client.1.smithi027.stdout:5/134: write d0/df/d11/d13/f26 [29123,15693] 0 2024-04-02T04:42:44.085 INFO:tasks.workunit.client.1.smithi027.stdout:5/135: read d0/f24 [12892,83707] 0 2024-04-02T04:42:44.086 INFO:tasks.workunit.client.0.smithi016.stdout:5/576: rename d3/d4c/dd1/fa5 to d3/d40/d4f/d6b/d2a/d6d/d8f/fd2 0 2024-04-02T04:42:44.086 INFO:tasks.workunit.client.1.smithi027.stdout:5/136: write d0/d5/f22 [524987,88412] 0 2024-04-02T04:42:44.088 INFO:tasks.workunit.client.0.smithi016.stdout:0/583: rmdir d0/d16/d2e/d36/d9f 39 2024-04-02T04:42:44.090 INFO:tasks.workunit.client.1.smithi027.stdout:4/72: unlink c3 0 2024-04-02T04:42:44.090 INFO:tasks.workunit.client.1.smithi027.stdout:1/48: fsync d0/f14 0 2024-04-02T04:42:44.091 INFO:tasks.workunit.client.1.smithi027.stdout:1/49: readlink d0/l3 0 2024-04-02T04:42:44.092 INFO:tasks.workunit.client.1.smithi027.stdout:1/50: dread d0/f6 [0,4194304] 0 2024-04-02T04:42:44.096 INFO:tasks.workunit.client.0.smithi016.stdout:1/583: unlink d14/d19/d50/fb9 0 2024-04-02T04:42:44.098 INFO:tasks.workunit.client.1.smithi027.stdout:2/68: getdents d1/d7 0 2024-04-02T04:42:44.099 INFO:tasks.workunit.client.1.smithi027.stdout:2/69: chown d1/db/c16 204020 1 2024-04-02T04:42:44.099 INFO:tasks.workunit.client.1.smithi027.stdout:0/107: dread f1 [4194304,4194304] 0 2024-04-02T04:42:44.101 INFO:tasks.workunit.client.1.smithi027.stdout:7/77: mkdir d2/d17 0 2024-04-02T04:42:44.106 INFO:tasks.workunit.client.0.smithi016.stdout:4/625: dwrite d5/de/f79 [0,4194304] 0 2024-04-02T04:42:44.109 INFO:tasks.workunit.client.0.smithi016.stdout:6/673: rename d0/d9/d16/f47 to d0/d9/d16/d46/dbe/fd0 0 2024-04-02T04:42:44.109 INFO:tasks.workunit.client.1.smithi027.stdout:8/52: dwrite f4 [0,4194304] 0 2024-04-02T04:42:44.112 INFO:tasks.workunit.client.1.smithi027.stdout:2/70: dwrite d1/f13 [4194304,4194304] 0 2024-04-02T04:42:44.114 INFO:tasks.workunit.client.1.smithi027.stdout:2/71: write d1/f13 [5593452,21215] 0 2024-04-02T04:42:44.116 INFO:tasks.workunit.client.1.smithi027.stdout:2/72: readlink d1/l9 0 2024-04-02T04:42:44.116 INFO:tasks.workunit.client.1.smithi027.stdout:4/73: creat f13 x:0 0 0 2024-04-02T04:42:44.118 INFO:tasks.workunit.client.1.smithi027.stdout:2/73: write d1/d7/f14 [7344488,120199] 0 2024-04-02T04:42:44.119 INFO:tasks.workunit.client.1.smithi027.stdout:2/74: chown d1/db/c15 0 1 2024-04-02T04:42:44.122 INFO:tasks.workunit.client.0.smithi016.stdout:1/584: dread - d14/d19/d20/f61 zero size 2024-04-02T04:42:44.124 INFO:tasks.workunit.client.1.smithi027.stdout:1/51: truncate d0/f6 1366806 0 2024-04-02T04:42:44.124 INFO:tasks.workunit.client.1.smithi027.stdout:1/52: stat d0/l3 0 2024-04-02T04:42:44.125 INFO:tasks.workunit.client.0.smithi016.stdout:3/703: mkdir d2/d2e/d36/d48/da2/de7/dec 0 2024-04-02T04:42:44.125 INFO:tasks.workunit.client.1.smithi027.stdout:1/53: truncate d0/fd 497747 0 2024-04-02T04:42:44.126 INFO:tasks.workunit.client.1.smithi027.stdout:3/43: link d1/f8 d1/fc 0 2024-04-02T04:42:44.126 INFO:tasks.workunit.client.1.smithi027.stdout:1/54: chown d0/de/l10 443453 1 2024-04-02T04:42:44.129 INFO:tasks.workunit.client.1.smithi027.stdout:9/66: creat f15 x:0 0 0 2024-04-02T04:42:44.130 INFO:tasks.workunit.client.1.smithi027.stdout:7/78: creat d2/d7/f18 x:0 0 0 2024-04-02T04:42:44.131 INFO:tasks.workunit.client.0.smithi016.stdout:4/626: dread d5/d1d/f37 [0,4194304] 0 2024-04-02T04:42:44.131 INFO:tasks.workunit.client.1.smithi027.stdout:7/79: fdatasync d2/df/f10 0 2024-04-02T04:42:44.134 INFO:tasks.workunit.client.0.smithi016.stdout:6/674: creat d0/d9/d16/d46/dbe/fd1 x:0 0 0 2024-04-02T04:42:44.135 INFO:tasks.workunit.client.0.smithi016.stdout:6/675: chown d0/d9/d16/d46/d5e/l71 9508 1 2024-04-02T04:42:44.137 INFO:tasks.workunit.client.1.smithi027.stdout:5/137: creat d0/df/d2f/f38 x:0 0 0 2024-04-02T04:42:44.138 INFO:tasks.workunit.client.1.smithi027.stdout:5/138: fdatasync d0/df/d11/d13/f26 0 2024-04-02T04:42:44.140 INFO:tasks.workunit.client.1.smithi027.stdout:8/53: mkdir d5/d8/db/df 0 2024-04-02T04:42:44.141 INFO:tasks.workunit.client.0.smithi016.stdout:8/591: creat d1/d6/d36/d3f/d40/fca x:0 0 0 2024-04-02T04:42:44.143 INFO:tasks.workunit.client.0.smithi016.stdout:1/585: creat d14/d19/d1e/d71/fbf x:0 0 0 2024-04-02T04:42:44.144 INFO:tasks.workunit.client.0.smithi016.stdout:1/586: fdatasync d14/d41/d43/f6f 0 2024-04-02T04:42:44.144 INFO:tasks.workunit.client.1.smithi027.stdout:4/74: creat f14 x:0 0 0 2024-04-02T04:42:44.145 INFO:tasks.workunit.client.1.smithi027.stdout:4/75: dread - ff zero size 2024-04-02T04:42:44.145 INFO:tasks.workunit.client.1.smithi027.stdout:4/76: rmdir - no directory 2024-04-02T04:42:44.145 INFO:tasks.workunit.client.0.smithi016.stdout:9/683: getdents d0/d9/d4d/d78 0 2024-04-02T04:42:44.150 INFO:tasks.workunit.client.1.smithi027.stdout:2/75: rename d1/d7 to d1/dd/d17 0 2024-04-02T04:42:44.151 INFO:tasks.workunit.client.0.smithi016.stdout:5/577: creat d3/fd3 x:0 0 0 2024-04-02T04:42:44.151 INFO:tasks.workunit.client.0.smithi016.stdout:5/578: chown d3/d40/d4f/fc6 352191 1 2024-04-02T04:42:44.152 INFO:tasks.workunit.client.1.smithi027.stdout:4/77: dwrite f5 [0,4194304] 0 2024-04-02T04:42:44.154 INFO:tasks.workunit.client.1.smithi027.stdout:3/44: mknod d1/cd 0 2024-04-02T04:42:44.154 INFO:tasks.workunit.client.0.smithi016.stdout:0/584: truncate d0/d16/d2e/d37/d66/faa 902655 0 2024-04-02T04:42:44.155 INFO:tasks.workunit.client.0.smithi016.stdout:0/585: dread - d0/d48/f58 zero size 2024-04-02T04:42:44.156 INFO:tasks.workunit.client.0.smithi016.stdout:4/627: symlink d5/d1d/d2d/lba 0 2024-04-02T04:42:44.157 INFO:tasks.workunit.client.1.smithi027.stdout:0/108: mkdir d4/d21 0 2024-04-02T04:42:44.158 INFO:tasks.workunit.client.1.smithi027.stdout:0/109: chown l0 135 1 2024-04-02T04:42:44.159 INFO:tasks.workunit.client.1.smithi027.stdout:9/67: dread - f14 zero size 2024-04-02T04:42:44.160 INFO:tasks.workunit.client.1.smithi027.stdout:7/80: symlink d2/d7/da/l19 0 2024-04-02T04:42:44.161 INFO:tasks.workunit.client.0.smithi016.stdout:1/587: fsync d14/d19/d20/f7b 0 2024-04-02T04:42:44.161 INFO:tasks.workunit.client.0.smithi016.stdout:1/588: chown d14/d19 754872 1 2024-04-02T04:42:44.162 INFO:tasks.workunit.client.0.smithi016.stdout:1/589: write d14/d19/f4b [411022,79257] 0 2024-04-02T04:42:44.163 INFO:tasks.workunit.client.1.smithi027.stdout:0/110: dread f2 [8388608,4194304] 0 2024-04-02T04:42:44.164 INFO:tasks.workunit.client.1.smithi027.stdout:0/111: dread - d4/fb zero size 2024-04-02T04:42:44.166 INFO:tasks.workunit.client.1.smithi027.stdout:7/81: dread f1 [0,4194304] 0 2024-04-02T04:42:44.166 INFO:tasks.workunit.client.1.smithi027.stdout:5/139: rmdir d0 39 2024-04-02T04:42:44.169 INFO:tasks.workunit.client.0.smithi016.stdout:3/704: link d2/d15/d16/d8a/dd1/fbc d2/d2e/d36/d66/d90/fed 0 2024-04-02T04:42:44.170 INFO:tasks.workunit.client.0.smithi016.stdout:5/579: mknod d3/df/d18/d2f/cd4 0 2024-04-02T04:42:44.172 INFO:tasks.workunit.client.0.smithi016.stdout:4/628: mknod d5/d1d/d6c/cbb 0 2024-04-02T04:42:44.175 INFO:tasks.workunit.client.1.smithi027.stdout:1/55: rename d0/l2 to d0/de/l15 0 2024-04-02T04:42:44.176 INFO:tasks.workunit.client.0.smithi016.stdout:8/592: mknod d1/d6/d36/d3f/d40/d49/d4a/d4c/dad/ccb 0 2024-04-02T04:42:44.177 INFO:tasks.workunit.client.0.smithi016.stdout:8/593: readlink d1/d6/d36/d3f/d40/d49/d38/l93 0 2024-04-02T04:42:44.179 INFO:tasks.workunit.client.1.smithi027.stdout:1/56: dread d0/f9 [0,4194304] 0 2024-04-02T04:42:44.180 INFO:tasks.workunit.client.1.smithi027.stdout:1/57: read d0/f7 [318928,76628] 0 2024-04-02T04:42:44.182 INFO:tasks.workunit.client.1.smithi027.stdout:4/78: rename f5 to f15 0 2024-04-02T04:42:44.182 INFO:tasks.workunit.client.1.smithi027.stdout:4/79: write ff [542699,40875] 0 2024-04-02T04:42:44.183 INFO:tasks.workunit.client.0.smithi016.stdout:9/684: symlink d0/d9/ld0 0 2024-04-02T04:42:44.184 INFO:tasks.workunit.client.0.smithi016.stdout:8/594: dwrite d1/d6/d36/d3f/d40/fb6 [0,4194304] 0 2024-04-02T04:42:44.184 INFO:tasks.workunit.client.1.smithi027.stdout:3/45: mkdir d1/de 0 2024-04-02T04:42:44.185 INFO:tasks.workunit.client.0.smithi016.stdout:9/685: chown d0/d3/l24 0 1 2024-04-02T04:42:44.186 INFO:tasks.workunit.client.0.smithi016.stdout:8/595: write d1/d6/d36/d3f/d40/d49/d4a/fa8 [1143403,89776] 0 2024-04-02T04:42:44.188 INFO:tasks.workunit.client.0.smithi016.stdout:8/596: dread - d1/d6/d36/d3f/d40/d45/fb9 zero size 2024-04-02T04:42:44.189 INFO:tasks.workunit.client.0.smithi016.stdout:3/705: fsync d2/d15/d1a/f53 0 2024-04-02T04:42:44.191 INFO:tasks.workunit.client.0.smithi016.stdout:5/580: mkdir d3/df/dd5 0 2024-04-02T04:42:44.195 INFO:tasks.workunit.client.0.smithi016.stdout:4/629: creat d5/de/fbc x:0 0 0 2024-04-02T04:42:44.195 INFO:tasks.workunit.client.0.smithi016.stdout:8/597: dread d1/d6/d36/d3f/d40/d49/d4a/fa8 [0,4194304] 0 2024-04-02T04:42:44.196 INFO:tasks.workunit.client.1.smithi027.stdout:5/140: fsync d0/f6 0 2024-04-02T04:42:44.198 INFO:tasks.workunit.client.0.smithi016.stdout:4/630: dread d5/d17/d18/da0/fa7 [0,4194304] 0 2024-04-02T04:42:44.201 INFO:tasks.workunit.client.1.smithi027.stdout:8/54: creat d5/d8/db/dd/f10 x:0 0 0 2024-04-02T04:42:44.202 INFO:tasks.workunit.client.1.smithi027.stdout:8/55: write d5/d8/db/dd/f10 [512851,110454] 0 2024-04-02T04:42:44.203 INFO:tasks.workunit.client.1.smithi027.stdout:8/56: write f4 [4900501,33100] 0 2024-04-02T04:42:44.203 INFO:tasks.workunit.client.1.smithi027.stdout:2/76: getdents d1/dd/d17/d12 0 2024-04-02T04:42:44.211 INFO:tasks.workunit.client.1.smithi027.stdout:1/58: creat d0/f16 x:0 0 0 2024-04-02T04:42:44.211 INFO:tasks.workunit.client.0.smithi016.stdout:1/590: dwrite d14/d19/f69 [0,4194304] 0 2024-04-02T04:42:44.216 INFO:tasks.workunit.client.1.smithi027.stdout:9/68: symlink l16 0 2024-04-02T04:42:44.218 INFO:tasks.workunit.client.0.smithi016.stdout:6/676: link d0/d9/d16/cad d0/d9/cd2 0 2024-04-02T04:42:44.219 INFO:tasks.workunit.client.1.smithi027.stdout:0/112: dwrite d4/d13/f18 [0,4194304] 0 2024-04-02T04:42:44.222 INFO:tasks.workunit.client.1.smithi027.stdout:1/59: dwrite d0/f16 [0,4194304] 0 2024-04-02T04:42:44.223 INFO:tasks.workunit.client.1.smithi027.stdout:5/141: symlink d0/df/l39 0 2024-04-02T04:42:44.224 INFO:tasks.workunit.client.1.smithi027.stdout:1/60: dread - d0/f14 zero size 2024-04-02T04:42:44.226 INFO:tasks.workunit.client.1.smithi027.stdout:1/61: dread - d0/f14 zero size 2024-04-02T04:42:44.226 INFO:tasks.workunit.client.1.smithi027.stdout:7/82: getdents d2/d7/da/dc/d16 0 2024-04-02T04:42:44.227 INFO:tasks.workunit.client.1.smithi027.stdout:7/83: fdatasync d2/f13 0 2024-04-02T04:42:44.228 INFO:tasks.workunit.client.0.smithi016.stdout:9/686: mkdir d0/d3/dd1 0 2024-04-02T04:42:44.229 INFO:tasks.workunit.client.0.smithi016.stdout:9/687: fsync d0/d3/f81 0 2024-04-02T04:42:44.231 INFO:tasks.workunit.client.1.smithi027.stdout:7/84: dread d2/d7/f12 [0,4194304] 0 2024-04-02T04:42:44.237 INFO:tasks.workunit.client.0.smithi016.stdout:3/706: mkdir d2/d2e/dee 0 2024-04-02T04:42:44.238 INFO:tasks.workunit.client.0.smithi016.stdout:3/707: chown d2/d15/d70/dcf 188249 1 2024-04-02T04:42:44.240 INFO:tasks.workunit.client.1.smithi027.stdout:3/46: mkdir d1/de/df 0 2024-04-02T04:42:44.240 INFO:tasks.workunit.client.1.smithi027.stdout:2/77: dwrite d1/df/f11 [0,4194304] 0 2024-04-02T04:42:44.244 INFO:tasks.workunit.client.0.smithi016.stdout:8/598: mkdir d1/d6/d36/d3f/d40/d45/dcc 0 2024-04-02T04:42:44.248 INFO:tasks.workunit.client.1.smithi027.stdout:3/47: dread d1/fc [0,4194304] 0 2024-04-02T04:42:44.251 INFO:tasks.workunit.client.0.smithi016.stdout:4/631: creat d5/d17/d18/d6d/da4/fbd x:0 0 0 2024-04-02T04:42:44.251 INFO:tasks.workunit.client.1.smithi027.stdout:3/48: dread d1/f8 [0,4194304] 0 2024-04-02T04:42:44.252 INFO:tasks.workunit.client.1.smithi027.stdout:3/49: chown d1/de/df 172971 1 2024-04-02T04:42:44.253 INFO:tasks.workunit.client.1.smithi027.stdout:5/142: rename d0/d5/f2e to d0/df/d11/d37/f3a 0 2024-04-02T04:42:44.256 INFO:tasks.workunit.client.1.smithi027.stdout:1/62: dread d0/f6 [0,4194304] 0 2024-04-02T04:42:44.256 INFO:tasks.workunit.client.0.smithi016.stdout:9/688: unlink d0/d4a/l52 0 2024-04-02T04:42:44.260 INFO:tasks.workunit.client.1.smithi027.stdout:7/85: rmdir d2/df 39 2024-04-02T04:42:44.262 INFO:tasks.workunit.client.0.smithi016.stdout:3/708: rename d2/d15/d16/d8a/dd1/fa6 to d2/d2e/d36/d66/d90/fef 0 2024-04-02T04:42:44.263 INFO:tasks.workunit.client.1.smithi027.stdout:1/63: dwrite d0/f16 [0,4194304] 0 2024-04-02T04:42:44.264 INFO:tasks.workunit.client.0.smithi016.stdout:3/709: write d2/d2e/d36/d48/da2/f92 [846053,123225] 0 2024-04-02T04:42:44.265 INFO:tasks.workunit.client.0.smithi016.stdout:3/710: chown d2/d15/d16/d8a/lbb 104731312 1 2024-04-02T04:42:44.266 INFO:tasks.workunit.client.0.smithi016.stdout:3/711: stat d2/d2e/d36/d66/d90/fed 0 2024-04-02T04:42:44.293 INFO:tasks.workunit.client.0.smithi016.stdout:7/582: sync 2024-04-02T04:42:44.293 INFO:tasks.workunit.client.0.smithi016.stdout:2/665: sync 2024-04-02T04:42:44.293 INFO:tasks.workunit.client.0.smithi016.stdout:2/666: stat d12/d35/d3f/f93 0 2024-04-02T04:42:44.336 INFO:tasks.workunit.client.0.smithi016.stdout:9/689: rename d0/d4e/d50/d63/d70 to d0/d4e/d50/d63/dd2 0 2024-04-02T04:42:44.336 INFO:tasks.workunit.client.1.smithi027.stdout:2/78: mknod d1/dd/c18 0 2024-04-02T04:42:44.343 INFO:tasks.workunit.client.1.smithi027.stdout:2/79: dwrite d1/f13 [4194304,4194304] 0 2024-04-02T04:42:44.343 INFO:tasks.workunit.client.0.smithi016.stdout:8/599: creat d1/d70/d71/d9e/fcd x:0 0 0 2024-04-02T04:42:44.344 INFO:tasks.workunit.client.1.smithi027.stdout:2/80: read d1/df/f11 [2257229,21791] 0 2024-04-02T04:42:44.344 INFO:tasks.workunit.client.1.smithi027.stdout:0/113: mknod d4/c22 0 2024-04-02T04:42:44.345 INFO:tasks.workunit.client.1.smithi027.stdout:2/81: write d1/f6 [8297268,58894] 0 2024-04-02T04:42:44.347 INFO:tasks.workunit.client.0.smithi016.stdout:2/667: rmdir d12/d35/d3b/d42/d67/d6a 39 2024-04-02T04:42:44.348 INFO:tasks.workunit.client.0.smithi016.stdout:2/668: write d12/d35/d3f/d56/fdd [470776,99050] 0 2024-04-02T04:42:44.349 INFO:tasks.workunit.client.1.smithi027.stdout:3/50: stat d1/cb 0 2024-04-02T04:42:44.352 INFO:tasks.workunit.client.1.smithi027.stdout:5/143: fdatasync d0/f24 0 2024-04-02T04:42:44.352 INFO:tasks.workunit.client.1.smithi027.stdout:5/144: chown d0/df 95468 1 2024-04-02T04:42:44.355 INFO:tasks.workunit.client.1.smithi027.stdout:3/51: dread d1/f7 [0,4194304] 0 2024-04-02T04:42:44.361 INFO:tasks.workunit.client.1.smithi027.stdout:7/86: rename d2/l15 to d2/d7/da/dc/d16/l1a 0 2024-04-02T04:42:44.363 INFO:tasks.workunit.client.1.smithi027.stdout:4/80: getdents . 0 2024-04-02T04:42:44.364 INFO:tasks.workunit.client.1.smithi027.stdout:1/64: creat d0/de/f17 x:0 0 0 2024-04-02T04:42:44.366 INFO:tasks.workunit.client.1.smithi027.stdout:9/69: creat f17 x:0 0 0 2024-04-02T04:42:44.367 INFO:tasks.workunit.client.0.smithi016.stdout:6/677: link d0/d5c/f6c d0/d9/d16/d56/d59/da1/fd3 0 2024-04-02T04:42:44.369 INFO:tasks.workunit.client.0.smithi016.stdout:9/690: fdatasync d0/d4e/d50/d63/d6e/d4b/fc0 0 2024-04-02T04:42:44.370 INFO:tasks.workunit.client.0.smithi016.stdout:4/632: rename d5/d17/c3d to d5/d1d/d1e/cbe 0 2024-04-02T04:42:44.371 INFO:tasks.workunit.client.1.smithi027.stdout:2/82: symlink d1/db/l19 0 2024-04-02T04:42:44.374 INFO:tasks.workunit.client.0.smithi016.stdout:7/583: mknod d0/d46/d37/d3a/da9/ccf 0 2024-04-02T04:42:44.376 INFO:tasks.workunit.client.1.smithi027.stdout:5/145: chown d0/df/c30 1152 1 2024-04-02T04:42:44.379 INFO:tasks.workunit.client.1.smithi027.stdout:5/146: dread d0/df/d11/d37/f3a [0,4194304] 0 2024-04-02T04:42:44.379 INFO:tasks.workunit.client.1.smithi027.stdout:3/52: creat d1/f10 x:0 0 0 2024-04-02T04:42:44.379 INFO:tasks.workunit.client.0.smithi016.stdout:2/669: dwrite d12/d35/d3b/d42/d59/fba [0,4194304] 0 2024-04-02T04:42:44.380 INFO:tasks.workunit.client.1.smithi027.stdout:5/147: write d0/df/d11/f1f [4831210,94620] 0 2024-04-02T04:42:44.382 INFO:tasks.workunit.client.1.smithi027.stdout:7/87: mkdir d2/d7/da/dc/d1b 0 2024-04-02T04:42:44.384 INFO:tasks.workunit.client.1.smithi027.stdout:1/65: mknod d0/de/c18 0 2024-04-02T04:42:44.385 INFO:tasks.workunit.client.0.smithi016.stdout:7/584: creat d0/d46/d37/d3a/d84/fd0 x:0 0 0 2024-04-02T04:42:44.387 INFO:tasks.workunit.client.0.smithi016.stdout:0/586: sync 2024-04-02T04:42:44.389 INFO:tasks.workunit.client.0.smithi016.stdout:9/691: mkdir d0/d4a/d99/dd3 0 2024-04-02T04:42:44.393 INFO:tasks.workunit.client.1.smithi027.stdout:2/83: symlink d1/db/l1a 0 2024-04-02T04:42:44.394 INFO:tasks.workunit.client.1.smithi027.stdout:7/88: dwrite f1 [0,4194304] 0 2024-04-02T04:42:44.395 INFO:tasks.workunit.client.0.smithi016.stdout:8/600: rename d1/l60 to d1/d6/d36/d8e/lce 0 2024-04-02T04:42:44.396 INFO:tasks.workunit.client.0.smithi016.stdout:4/633: dwrite d5/d1d/f21 [0,4194304] 0 2024-04-02T04:42:44.398 INFO:tasks.workunit.client.1.smithi027.stdout:7/89: dread - d2/d7/f18 zero size 2024-04-02T04:42:44.399 INFO:tasks.workunit.client.0.smithi016.stdout:7/585: readlink d0/d46/l63 0 2024-04-02T04:42:44.401 INFO:tasks.workunit.client.1.smithi027.stdout:1/66: dwrite d0/f14 [0,4194304] 0 2024-04-02T04:42:44.403 INFO:tasks.workunit.client.1.smithi027.stdout:6/72: sync 2024-04-02T04:42:44.404 INFO:tasks.workunit.client.1.smithi027.stdout:9/70: dwrite f14 [0,4194304] 0 2024-04-02T04:42:44.405 INFO:tasks.workunit.client.0.smithi016.stdout:8/601: dwrite d1/d31/d67/d8f/f9f [0,4194304] 0 2024-04-02T04:42:44.406 INFO:tasks.workunit.client.0.smithi016.stdout:8/602: dread - d1/d70/d71/d74/fa6 zero size 2024-04-02T04:42:44.407 INFO:tasks.workunit.client.1.smithi027.stdout:1/67: chown d0/f5 23049611 1 2024-04-02T04:42:44.413 INFO:tasks.workunit.client.1.smithi027.stdout:5/148: mkdir d0/df/d2f/d3b 0 2024-04-02T04:42:44.417 INFO:tasks.workunit.client.1.smithi027.stdout:0/114: truncate d4/f9 24686 0 2024-04-02T04:42:44.418 INFO:tasks.workunit.client.1.smithi027.stdout:5/149: chown d0/d1a/d20 39476 1 2024-04-02T04:42:44.419 INFO:tasks.workunit.client.0.smithi016.stdout:4/634: readlink d5/d17/l52 0 2024-04-02T04:42:44.420 INFO:tasks.workunit.client.1.smithi027.stdout:4/81: getdents . 0 2024-04-02T04:42:44.420 INFO:tasks.workunit.client.0.smithi016.stdout:4/635: truncate d5/de/fbc 32951 0 2024-04-02T04:42:44.421 INFO:tasks.workunit.client.1.smithi027.stdout:2/84: dwrite d1/dd/d17/fc [4194304,4194304] 0 2024-04-02T04:42:44.422 INFO:tasks.workunit.client.0.smithi016.stdout:7/586: mknod d0/d28/cd1 0 2024-04-02T04:42:44.424 INFO:tasks.workunit.client.0.smithi016.stdout:0/587: write d0/d16/d2e/d37/d66/f70 [4965509,44348] 0 2024-04-02T04:42:44.425 INFO:tasks.workunit.client.0.smithi016.stdout:6/678: getdents d0/d78 0 2024-04-02T04:42:44.428 INFO:tasks.workunit.client.0.smithi016.stdout:2/670: truncate d12/d35/d3b/d42/d59/f88 1079983 0 2024-04-02T04:42:44.432 INFO:tasks.workunit.client.1.smithi027.stdout:6/73: dwrite fc [0,4194304] 0 2024-04-02T04:42:44.433 INFO:tasks.workunit.client.1.smithi027.stdout:7/90: write d2/d7/f12 [7816346,83713] 0 2024-04-02T04:42:44.436 INFO:tasks.workunit.client.1.smithi027.stdout:6/74: dread - d13/f17 zero size 2024-04-02T04:42:44.437 INFO:tasks.workunit.client.1.smithi027.stdout:1/68: symlink d0/l19 0 2024-04-02T04:42:44.442 INFO:tasks.workunit.client.0.smithi016.stdout:4/636: chown d5/d17/d76/d7e/c8a 112460 1 2024-04-02T04:42:44.442 INFO:tasks.workunit.client.0.smithi016.stdout:4/637: fsync d5/d1d/f78 0 2024-04-02T04:42:44.444 INFO:tasks.workunit.client.1.smithi027.stdout:3/53: dwrite d1/f7 [0,4194304] 0 2024-04-02T04:42:44.444 INFO:tasks.workunit.client.0.smithi016.stdout:7/587: rename d0/cc9 to d0/d46/d37/d3a/d4c/cd2 0 2024-04-02T04:42:44.446 INFO:tasks.workunit.client.0.smithi016.stdout:0/588: symlink d0/d16/d24/d4e/lcd 0 2024-04-02T04:42:44.449 INFO:tasks.workunit.client.1.smithi027.stdout:5/150: dwrite d0/df/d2f/f38 [0,4194304] 0 2024-04-02T04:42:44.449 INFO:tasks.workunit.client.1.smithi027.stdout:4/82: creat f16 x:0 0 0 2024-04-02T04:42:44.449 INFO:tasks.workunit.client.1.smithi027.stdout:0/115: dwrite d4/f11 [0,4194304] 0 2024-04-02T04:42:44.452 INFO:tasks.workunit.client.0.smithi016.stdout:9/692: link d0/d4e/d50/d63/dd2/l9e d0/d3/dd1/ld4 0 2024-04-02T04:42:44.454 INFO:tasks.workunit.client.1.smithi027.stdout:0/116: fsync d4/d16/d1b/f20 0 2024-04-02T04:42:44.454 INFO:tasks.workunit.client.1.smithi027.stdout:4/83: dread f9 [0,4194304] 0 2024-04-02T04:42:44.457 INFO:tasks.workunit.client.0.smithi016.stdout:4/638: fdatasync d5/d1d/f21 0 2024-04-02T04:42:44.458 INFO:tasks.workunit.client.1.smithi027.stdout:2/85: readlink d1/la 0 2024-04-02T04:42:44.459 INFO:tasks.workunit.client.1.smithi027.stdout:8/57: sync 2024-04-02T04:42:44.460 INFO:tasks.workunit.client.1.smithi027.stdout:0/117: read d4/f14 [680666,15934] 0 2024-04-02T04:42:44.460 INFO:tasks.workunit.client.0.smithi016.stdout:3/712: sync 2024-04-02T04:42:44.460 INFO:tasks.workunit.client.0.smithi016.stdout:1/591: sync 2024-04-02T04:42:44.460 INFO:tasks.workunit.client.0.smithi016.stdout:5/581: sync 2024-04-02T04:42:44.461 INFO:tasks.workunit.client.0.smithi016.stdout:6/679: rename d0/d9/d16/d56/d89/d67/d6f/lb6 to d0/d9/d16/d56/da4/ld4 0 2024-04-02T04:42:44.462 INFO:tasks.workunit.client.1.smithi027.stdout:6/75: dwrite d13/f18 [4194304,4194304] 0 2024-04-02T04:42:44.467 INFO:tasks.workunit.client.0.smithi016.stdout:7/588: creat d0/d46/d52/d82/fd3 x:0 0 0 2024-04-02T04:42:44.468 INFO:tasks.workunit.client.1.smithi027.stdout:6/76: chown d13/c1c 7070 1 2024-04-02T04:42:44.468 INFO:tasks.workunit.client.1.smithi027.stdout:3/54: dread d1/fc [0,4194304] 0 2024-04-02T04:42:44.468 INFO:tasks.workunit.client.1.smithi027.stdout:6/77: chown d13/f16 2 1 2024-04-02T04:42:44.468 INFO:tasks.workunit.client.1.smithi027.stdout:1/69: rmdir d0/de 39 2024-04-02T04:42:44.468 INFO:tasks.workunit.client.1.smithi027.stdout:1/70: write d0/f9 [1877701,27559] 0 2024-04-02T04:42:44.468 INFO:tasks.workunit.client.0.smithi016.stdout:0/589: mknod d0/d16/cce 0 2024-04-02T04:42:44.474 INFO:tasks.workunit.client.1.smithi027.stdout:2/86: unlink d1/la 0 2024-04-02T04:42:44.477 INFO:tasks.workunit.client.0.smithi016.stdout:3/713: dread d2/d15/f80 [0,4194304] 0 2024-04-02T04:42:44.482 INFO:tasks.workunit.client.0.smithi016.stdout:2/671: truncate d12/d35/d3b/d42/f65 3101050 0 2024-04-02T04:42:44.483 INFO:tasks.workunit.client.0.smithi016.stdout:2/672: write d12/d35/f5c [2586448,60979] 0 2024-04-02T04:42:44.484 INFO:tasks.workunit.client.0.smithi016.stdout:2/673: chown d12/d35/d3b/d42/c89 2 1 2024-04-02T04:42:44.485 INFO:tasks.workunit.client.0.smithi016.stdout:5/582: creat d3/df/d18/d2f/fd6 x:0 0 0 2024-04-02T04:42:44.488 INFO:tasks.workunit.client.0.smithi016.stdout:6/680: write d0/d9/df/fc4 [3634113,40284] 0 2024-04-02T04:42:44.488 INFO:tasks.workunit.client.0.smithi016.stdout:0/590: creat d0/d4f/daf/fcf x:0 0 0 2024-04-02T04:42:44.488 INFO:tasks.workunit.client.0.smithi016.stdout:8/603: sync 2024-04-02T04:42:44.492 INFO:tasks.workunit.client.1.smithi027.stdout:7/91: truncate f1 3452096 0 2024-04-02T04:42:44.493 INFO:tasks.workunit.client.0.smithi016.stdout:9/693: dwrite d0/d9/db/dc9/d53/fc5 [4194304,4194304] 0 2024-04-02T04:42:44.493 INFO:tasks.workunit.client.1.smithi027.stdout:2/87: symlink d1/db/l1b 0 2024-04-02T04:42:44.495 INFO:tasks.workunit.client.0.smithi016.stdout:4/639: symlink d5/d1d/lbf 0 2024-04-02T04:42:44.496 INFO:tasks.workunit.client.0.smithi016.stdout:5/583: dwrite d3/d4c/dd1/fa3 [0,4194304] 0 2024-04-02T04:42:44.497 INFO:tasks.workunit.client.0.smithi016.stdout:4/640: readlink d5/l5c 0 2024-04-02T04:42:44.498 INFO:tasks.workunit.client.0.smithi016.stdout:9/694: chown d0/d9/db/d54/l80 12912 1 2024-04-02T04:42:44.498 INFO:tasks.workunit.client.1.smithi027.stdout:5/151: mknod d0/df/d2f/d3b/c3c 0 2024-04-02T04:42:44.498 INFO:tasks.workunit.client.0.smithi016.stdout:1/592: getdents d14/d19/d1e/d6d 0 2024-04-02T04:42:44.501 INFO:tasks.workunit.client.0.smithi016.stdout:2/674: symlink d12/d26/leb 0 2024-04-02T04:42:44.501 INFO:tasks.workunit.client.0.smithi016.stdout:9/695: readlink d0/d4e/d50/d67/lc4 0 2024-04-02T04:42:44.501 INFO:tasks.workunit.client.0.smithi016.stdout:1/593: readlink d14/d19/d1d/d29/d39/l99 0 2024-04-02T04:42:44.504 INFO:tasks.workunit.client.0.smithi016.stdout:7/589: getdents d0/d46/da2/dc8 0 2024-04-02T04:42:44.505 INFO:tasks.workunit.client.0.smithi016.stdout:3/714: dwrite d2/d2e/d36/d66/d90/fef [0,4194304] 0 2024-04-02T04:42:44.505 INFO:tasks.workunit.client.0.smithi016.stdout:7/590: chown d0/f32 172893 1 2024-04-02T04:42:44.506 INFO:tasks.workunit.client.1.smithi027.stdout:4/84: truncate f2 6979901 0 2024-04-02T04:42:44.506 INFO:tasks.workunit.client.0.smithi016.stdout:4/641: dread d5/f67 [0,4194304] 0 2024-04-02T04:42:44.509 INFO:tasks.workunit.client.0.smithi016.stdout:4/642: fdatasync d5/d17/d18/d1f/f98 0 2024-04-02T04:42:44.512 INFO:tasks.workunit.client.1.smithi027.stdout:3/55: mknod d1/de/df/c11 0 2024-04-02T04:42:44.514 INFO:tasks.workunit.client.1.smithi027.stdout:6/78: creat d13/d1b/f1d x:0 0 0 2024-04-02T04:42:44.515 INFO:tasks.workunit.client.0.smithi016.stdout:6/681: creat d0/d9/d16/d46/dbe/fd5 x:0 0 0 2024-04-02T04:42:44.516 INFO:tasks.workunit.client.1.smithi027.stdout:1/71: symlink d0/de/l1a 0 2024-04-02T04:42:44.520 INFO:tasks.workunit.client.0.smithi016.stdout:0/591: write d0/d16/d2e/d53/f7c [5037033,126214] 0 2024-04-02T04:42:44.521 INFO:tasks.workunit.client.1.smithi027.stdout:9/71: link c9 c18 0 2024-04-02T04:42:44.523 INFO:tasks.workunit.client.1.smithi027.stdout:5/152: creat d0/df/d2f/d3b/f3d x:0 0 0 2024-04-02T04:42:44.524 INFO:tasks.workunit.client.0.smithi016.stdout:1/594: mkdir d14/d19/d1e/d71/dc0 0 2024-04-02T04:42:44.525 INFO:tasks.workunit.client.1.smithi027.stdout:4/85: mknod c17 0 2024-04-02T04:42:44.525 INFO:tasks.workunit.client.1.smithi027.stdout:4/86: readlink lb 0 2024-04-02T04:42:44.525 INFO:tasks.workunit.client.1.smithi027.stdout:4/87: stat fc 0 2024-04-02T04:42:44.525 INFO:tasks.workunit.client.1.smithi027.stdout:4/88: dread - f12 zero size 2024-04-02T04:42:44.526 INFO:tasks.workunit.client.0.smithi016.stdout:2/675: symlink d12/d35/d3b/d42/d8d/lec 0 2024-04-02T04:42:44.527 INFO:tasks.workunit.client.1.smithi027.stdout:0/118: creat d4/f23 x:0 0 0 2024-04-02T04:42:44.529 INFO:tasks.workunit.client.0.smithi016.stdout:9/696: mkdir d0/d4e/d50/d63/d6e/d4b/d58/dd5 0 2024-04-02T04:42:44.530 INFO:tasks.workunit.client.1.smithi027.stdout:0/119: write d4/f23 [55789,41820] 0 2024-04-02T04:42:44.532 INFO:tasks.workunit.client.0.smithi016.stdout:3/715: creat d2/d15/d1a/ff0 x:0 0 0 2024-04-02T04:42:44.532 INFO:tasks.workunit.client.0.smithi016.stdout:3/716: fdatasync d2/f26 0 2024-04-02T04:42:44.533 INFO:tasks.workunit.client.1.smithi027.stdout:6/79: creat d13/d1b/f1e x:0 0 0 2024-04-02T04:42:44.534 INFO:tasks.workunit.client.0.smithi016.stdout:7/591: truncate d0/d21/f36 386346 0 2024-04-02T04:42:44.534 INFO:tasks.workunit.client.1.smithi027.stdout:6/80: write fa [1567890,130479] 0 2024-04-02T04:42:44.535 INFO:tasks.workunit.client.1.smithi027.stdout:6/81: chown d13/l19 13 1 2024-04-02T04:42:44.536 INFO:tasks.workunit.client.0.smithi016.stdout:5/584: dwrite d3/df/f1f [0,4194304] 0 2024-04-02T04:42:44.537 INFO:tasks.workunit.client.0.smithi016.stdout:5/585: fsync d3/df/f57 0 2024-04-02T04:42:44.538 INFO:tasks.workunit.client.0.smithi016.stdout:5/586: fsync d3/d40/d4f/d9c/fc7 0 2024-04-02T04:42:44.538 INFO:tasks.workunit.client.0.smithi016.stdout:4/643: rename d5/d1d/d2d/f4d to d5/d17/d76/fc0 0 2024-04-02T04:42:44.538 INFO:tasks.workunit.client.0.smithi016.stdout:5/587: readlink d3/d5d/lb2 0 2024-04-02T04:42:44.541 INFO:tasks.workunit.client.1.smithi027.stdout:9/72: chown f5 159335 1 2024-04-02T04:42:44.546 INFO:tasks.workunit.client.1.smithi027.stdout:4/89: creat f18 x:0 0 0 2024-04-02T04:42:44.547 INFO:tasks.workunit.client.0.smithi016.stdout:1/595: truncate d14/d19/d1e/f23 712410 0 2024-04-02T04:42:44.548 INFO:tasks.workunit.client.0.smithi016.stdout:1/596: readlink d14/d19/d50/la3 0 2024-04-02T04:42:44.550 INFO:tasks.workunit.client.0.smithi016.stdout:2/676: creat d12/d35/d3f/d2b/fed x:0 0 0 2024-04-02T04:42:44.551 INFO:tasks.workunit.client.1.smithi027.stdout:9/73: dwrite f5 [0,4194304] 0 2024-04-02T04:42:44.551 INFO:tasks.workunit.client.0.smithi016.stdout:2/677: chown d12/d35/d3b/fc7 161257 1 2024-04-02T04:42:44.557 INFO:tasks.workunit.client.1.smithi027.stdout:6/82: creat d13/d1b/f1f x:0 0 0 2024-04-02T04:42:44.559 INFO:tasks.workunit.client.1.smithi027.stdout:2/88: truncate d1/f6 7293725 0 2024-04-02T04:42:44.562 INFO:tasks.workunit.client.1.smithi027.stdout:3/56: write d1/f2 [283341,24506] 0 2024-04-02T04:42:44.564 INFO:tasks.workunit.client.0.smithi016.stdout:2/678: dread d12/d35/d3b/f46 [0,4194304] 0 2024-04-02T04:42:44.565 INFO:tasks.workunit.client.1.smithi027.stdout:5/153: fsync d0/df/d11/d13/f1e 0 2024-04-02T04:42:44.567 INFO:tasks.workunit.client.0.smithi016.stdout:2/679: write d12/d35/d3f/d56/f81 [881136,41457] 0 2024-04-02T04:42:44.568 INFO:tasks.workunit.client.0.smithi016.stdout:6/682: rename d0/d9/cd2 to d0/d9/d16/d56/d95/cd6 0 2024-04-02T04:42:44.569 INFO:tasks.workunit.client.1.smithi027.stdout:4/90: mknod c19 0 2024-04-02T04:42:44.569 INFO:tasks.workunit.client.1.smithi027.stdout:4/91: readlink l4 0 2024-04-02T04:42:44.570 INFO:tasks.workunit.client.0.smithi016.stdout:2/680: chown d12/d26/f2c 240213 1 2024-04-02T04:42:44.571 INFO:tasks.workunit.client.0.smithi016.stdout:2/681: chown d12/d26/l85 663517 1 2024-04-02T04:42:44.571 INFO:tasks.workunit.client.0.smithi016.stdout:9/697: dwrite d0/d9/db/dc9/d53/f73 [0,4194304] 0 2024-04-02T04:42:44.572 INFO:tasks.workunit.client.1.smithi027.stdout:7/92: write f1 [779752,69183] 0 2024-04-02T04:42:44.572 INFO:tasks.workunit.client.0.smithi016.stdout:4/644: chown d5/d1d/d2d/c47 99 1 2024-04-02T04:42:44.573 INFO:tasks.workunit.client.0.smithi016.stdout:2/682: readlink d12/d35/d3f/d56/ld6 0 2024-04-02T04:42:44.574 INFO:tasks.workunit.client.0.smithi016.stdout:4/645: write d5/f60 [325451,108309] 0 2024-04-02T04:42:44.576 INFO:tasks.workunit.client.0.smithi016.stdout:0/592: fsync d0/d16/fb8 0 2024-04-02T04:42:44.578 INFO:tasks.workunit.client.0.smithi016.stdout:0/593: write d0/d16/d2e/f85 [5113792,56308] 0 2024-04-02T04:42:44.585 INFO:tasks.workunit.client.0.smithi016.stdout:7/592: mknod d0/d46/d37/d3a/d43/d96/d9b/cd4 0 2024-04-02T04:42:44.585 INFO:tasks.workunit.client.0.smithi016.stdout:0/594: dwrite d0/d16/d2e/d37/d66/f70 [0,4194304] 0 2024-04-02T04:42:44.590 INFO:tasks.workunit.client.0.smithi016.stdout:5/588: rename d3/d40/d4f/d6b/d2a/d6d/d8f to d3/d4c/dc4/dd7 0 2024-04-02T04:42:44.592 INFO:tasks.workunit.client.0.smithi016.stdout:6/683: mknod d0/d6e/cd7 0 2024-04-02T04:42:44.595 INFO:tasks.workunit.client.0.smithi016.stdout:8/604: getdents d1/d70/d71/d9e 0 2024-04-02T04:42:44.596 INFO:tasks.workunit.client.0.smithi016.stdout:9/698: mkdir d0/d4a/d99/d46/dd6 0 2024-04-02T04:42:44.600 INFO:tasks.workunit.client.0.smithi016.stdout:1/597: truncate d14/f28 1296277 0 2024-04-02T04:42:44.602 INFO:tasks.workunit.client.0.smithi016.stdout:3/717: creat d2/d2e/ff1 x:0 0 0 2024-04-02T04:42:44.602 INFO:tasks.workunit.client.1.smithi027.stdout:6/83: creat d13/d1b/f20 x:0 0 0 2024-04-02T04:42:44.603 INFO:tasks.workunit.client.0.smithi016.stdout:7/593: mkdir d0/d46/d52/dd5 0 2024-04-02T04:42:44.603 INFO:tasks.workunit.client.0.smithi016.stdout:1/598: chown d14/d41/d43/f96 345 1 2024-04-02T04:42:44.604 INFO:tasks.workunit.client.0.smithi016.stdout:8/605: dwrite d1/d70/d71/d74/fc3 [0,4194304] 0 2024-04-02T04:42:44.604 INFO:tasks.workunit.client.0.smithi016.stdout:9/699: dread d0/f5d [0,4194304] 0 2024-04-02T04:42:44.604 INFO:tasks.workunit.client.1.smithi027.stdout:5/154: fdatasync d0/f24 0 2024-04-02T04:42:44.605 INFO:tasks.workunit.client.1.smithi027.stdout:6/84: read - d13/d1b/f1f zero size 2024-04-02T04:42:44.607 INFO:tasks.workunit.client.1.smithi027.stdout:7/93: dwrite f1 [0,4194304] 0 2024-04-02T04:42:44.611 INFO:tasks.workunit.client.1.smithi027.stdout:4/92: write f9 [82530,23793] 0 2024-04-02T04:42:44.611 INFO:tasks.workunit.client.0.smithi016.stdout:2/683: mkdir d12/d35/d3b/d42/d67/d6a/dee 0 2024-04-02T04:42:44.614 INFO:tasks.workunit.client.1.smithi027.stdout:4/93: read - f14 zero size 2024-04-02T04:42:44.615 INFO:tasks.workunit.client.0.smithi016.stdout:4/646: mknod d5/d17/d76/d7e/cc1 0 2024-04-02T04:42:44.616 INFO:tasks.workunit.client.1.smithi027.stdout:2/89: fdatasync d1/f13 0 2024-04-02T04:42:44.616 INFO:tasks.workunit.client.0.smithi016.stdout:4/647: chown d5/d8/lc 882743 1 2024-04-02T04:42:44.616 INFO:tasks.workunit.client.0.smithi016.stdout:4/648: stat d5/l27 0 2024-04-02T04:42:44.617 INFO:tasks.workunit.client.0.smithi016.stdout:0/595: symlink d0/d16/d24/ld0 0 2024-04-02T04:42:44.617 INFO:tasks.workunit.client.1.smithi027.stdout:4/94: chown f15 0 1 2024-04-02T04:42:44.618 INFO:tasks.workunit.client.1.smithi027.stdout:6/85: symlink d13/d1b/l21 0 2024-04-02T04:42:44.619 INFO:tasks.workunit.client.0.smithi016.stdout:3/718: mkdir d2/d15/d1a/d45/df2 0 2024-04-02T04:42:44.619 INFO:tasks.workunit.client.1.smithi027.stdout:0/120: dwrite d4/f9 [0,4194304] 0 2024-04-02T04:42:44.621 INFO:tasks.workunit.client.1.smithi027.stdout:3/57: dwrite d1/f8 [0,4194304] 0 2024-04-02T04:42:44.622 INFO:tasks.workunit.client.0.smithi016.stdout:5/589: symlink d3/d40/d4f/ld8 0 2024-04-02T04:42:44.623 INFO:tasks.workunit.client.0.smithi016.stdout:5/590: chown d3/d4c/dc4/dd7/fd2 21 1 2024-04-02T04:42:44.623 INFO:tasks.workunit.client.1.smithi027.stdout:6/86: write ff [3608403,86458] 0 2024-04-02T04:42:44.625 INFO:tasks.workunit.client.1.smithi027.stdout:5/155: rename d0/d5/f22 to d0/df/d11/d13/f3e 0 2024-04-02T04:42:44.627 INFO:tasks.workunit.client.1.smithi027.stdout:7/94: mkdir d2/d1c 0 2024-04-02T04:42:44.631 INFO:tasks.workunit.client.0.smithi016.stdout:8/606: unlink d1/d6/f57 0 2024-04-02T04:42:44.632 INFO:tasks.workunit.client.1.smithi027.stdout:4/95: mknod c1a 0 2024-04-02T04:42:44.632 INFO:tasks.workunit.client.1.smithi027.stdout:2/90: dwrite d1/dd/d17/fc [0,4194304] 0 2024-04-02T04:42:44.632 INFO:tasks.workunit.client.0.smithi016.stdout:8/607: fsync d1/d6/d36/f99 0 2024-04-02T04:42:44.633 INFO:tasks.workunit.client.1.smithi027.stdout:2/91: stat d1/db/l1b 0 2024-04-02T04:42:44.633 INFO:tasks.workunit.client.0.smithi016.stdout:1/599: mkdir d14/d41/d43/d68/dc1 0 2024-04-02T04:42:44.634 INFO:tasks.workunit.client.0.smithi016.stdout:1/600: readlink d14/d19/d50/l8f 0 2024-04-02T04:42:44.638 INFO:tasks.workunit.client.1.smithi027.stdout:6/87: fdatasync ff 0 2024-04-02T04:42:44.639 INFO:tasks.workunit.client.0.smithi016.stdout:9/700: mknod d0/d4e/d50/d63/d6e/d4b/db2/dba/cd7 0 2024-04-02T04:42:44.640 INFO:tasks.workunit.client.0.smithi016.stdout:9/701: write d0/d4a/d75/d95/fca [2420109,116357] 0 2024-04-02T04:42:44.640 INFO:tasks.workunit.client.1.smithi027.stdout:0/121: creat d4/d13/f24 x:0 0 0 2024-04-02T04:42:44.644 INFO:tasks.workunit.client.1.smithi027.stdout:2/92: dwrite d1/f6 [0,4194304] 0 2024-04-02T04:42:44.645 INFO:tasks.workunit.client.0.smithi016.stdout:2/684: rename d12/d35/d3f/d2b/da3/fb9 to d12/d35/d3f/d2b/fef 0 2024-04-02T04:42:44.646 INFO:tasks.workunit.client.1.smithi027.stdout:6/88: dread d13/f18 [4194304,4194304] 0 2024-04-02T04:42:44.653 INFO:tasks.workunit.client.0.smithi016.stdout:3/719: rmdir d2/d2e/d36/d48/da2/de7 39 2024-04-02T04:42:44.655 INFO:tasks.workunit.client.1.smithi027.stdout:6/89: dwrite fa [0,4194304] 0 2024-04-02T04:42:44.655 INFO:tasks.workunit.client.0.smithi016.stdout:6/684: truncate d0/d9/df/fc4 1299002 0 2024-04-02T04:42:44.657 INFO:tasks.workunit.client.0.smithi016.stdout:6/685: read d0/d9/d16/d56/d59/f96 [388872,120714] 0 2024-04-02T04:42:44.662 INFO:tasks.workunit.client.0.smithi016.stdout:8/608: symlink d1/d6/d36/d4f/dc9/lcf 0 2024-04-02T04:42:44.666 INFO:tasks.workunit.client.1.smithi027.stdout:2/93: creat d1/dd/f1c x:0 0 0 2024-04-02T04:42:44.666 INFO:tasks.workunit.client.0.smithi016.stdout:7/594: dwrite d0/d46/d37/dbe/f7b [0,4194304] 0 2024-04-02T04:42:44.668 INFO:tasks.workunit.client.0.smithi016.stdout:9/702: mkdir d0/d4a/da9/dd8 0 2024-04-02T04:42:44.669 INFO:tasks.workunit.client.0.smithi016.stdout:9/703: chown d0/d9/d4d/da8/fb8 190409515 1 2024-04-02T04:42:44.671 INFO:tasks.workunit.client.1.smithi027.stdout:8/58: sync 2024-04-02T04:42:44.671 INFO:tasks.workunit.client.1.smithi027.stdout:1/72: sync 2024-04-02T04:42:44.673 INFO:tasks.workunit.client.1.smithi027.stdout:1/73: stat d0 0 2024-04-02T04:42:44.675 INFO:tasks.workunit.client.0.smithi016.stdout:5/591: write d3/df/f1e [3301575,90969] 0 2024-04-02T04:42:44.677 INFO:tasks.workunit.client.0.smithi016.stdout:0/596: mknod d0/d16/d2e/d36/cd1 0 2024-04-02T04:42:44.677 INFO:tasks.workunit.client.1.smithi027.stdout:5/156: dwrite d0/df/d11/d13/f1e [0,4194304] 0 2024-04-02T04:42:44.678 INFO:tasks.workunit.client.1.smithi027.stdout:6/90: rmdir d13/d1b 39 2024-04-02T04:42:44.679 INFO:tasks.workunit.client.1.smithi027.stdout:6/91: write ff [76624,65061] 0 2024-04-02T04:42:44.681 INFO:tasks.workunit.client.1.smithi027.stdout:1/74: dread d0/f16 [0,4194304] 0 2024-04-02T04:42:44.687 INFO:tasks.workunit.client.1.smithi027.stdout:7/95: truncate f1 1296518 0 2024-04-02T04:42:44.689 INFO:tasks.workunit.client.0.smithi016.stdout:1/601: creat d14/d19/d1e/d71/dc0/fc2 x:0 0 0 2024-04-02T04:42:44.690 INFO:tasks.workunit.client.0.smithi016.stdout:1/602: fdatasync d14/d19/d1d/f7d 0 2024-04-02T04:42:44.691 INFO:tasks.workunit.client.1.smithi027.stdout:2/94: creat d1/db/f1d x:0 0 0 2024-04-02T04:42:44.691 INFO:tasks.workunit.client.1.smithi027.stdout:2/95: dread - d1/db/f1d zero size 2024-04-02T04:42:44.692 INFO:tasks.workunit.client.1.smithi027.stdout:8/59: symlink d5/d8/l11 0 2024-04-02T04:42:44.693 INFO:tasks.workunit.client.1.smithi027.stdout:2/96: truncate d1/db/f1d 390092 0 2024-04-02T04:42:44.694 INFO:tasks.workunit.client.0.smithi016.stdout:7/595: creat d0/d28/d64/fd6 x:0 0 0 2024-04-02T04:42:44.695 INFO:tasks.workunit.client.1.smithi027.stdout:6/92: unlink cb 0 2024-04-02T04:42:44.699 INFO:tasks.workunit.client.1.smithi027.stdout:1/75: creat d0/de/f1b x:0 0 0 2024-04-02T04:42:44.701 INFO:tasks.workunit.client.0.smithi016.stdout:2/685: link d12/d9f/dac/f9d d12/d35/d3f/d4d/dde/ff0 0 2024-04-02T04:42:44.702 INFO:tasks.workunit.client.0.smithi016.stdout:2/686: read d12/f4c [1872617,35583] 0 2024-04-02T04:42:44.703 INFO:tasks.workunit.client.0.smithi016.stdout:4/649: creat d5/d17/fc2 x:0 0 0 2024-04-02T04:42:44.705 INFO:tasks.workunit.client.0.smithi016.stdout:5/592: creat d3/d40/db6/fd9 x:0 0 0 2024-04-02T04:42:44.705 INFO:tasks.workunit.client.1.smithi027.stdout:3/58: getdents d1 0 2024-04-02T04:42:44.706 INFO:tasks.workunit.client.0.smithi016.stdout:5/593: chown d3/d40 208619 1 2024-04-02T04:42:44.706 INFO:tasks.workunit.client.1.smithi027.stdout:3/59: fsync d1/f7 0 2024-04-02T04:42:44.706 INFO:tasks.workunit.client.0.smithi016.stdout:0/597: fdatasync d0/d16/d24/f7b 0 2024-04-02T04:42:44.707 INFO:tasks.workunit.client.1.smithi027.stdout:0/122: link d4/d13/c12 d4/d21/c25 0 2024-04-02T04:42:44.709 INFO:tasks.workunit.client.1.smithi027.stdout:8/60: symlink d5/l12 0 2024-04-02T04:42:44.710 INFO:tasks.workunit.client.0.smithi016.stdout:4/650: dwrite d5/d17/d18/d1f/f90 [4194304,4194304] 0 2024-04-02T04:42:44.710 INFO:tasks.workunit.client.1.smithi027.stdout:3/60: dread d1/f7 [0,4194304] 0 2024-04-02T04:42:44.712 INFO:tasks.workunit.client.1.smithi027.stdout:0/123: dread d4/f9 [0,4194304] 0 2024-04-02T04:42:44.713 INFO:tasks.workunit.client.1.smithi027.stdout:0/124: chown d4/f19 66791284 1 2024-04-02T04:42:44.715 INFO:tasks.workunit.client.0.smithi016.stdout:0/598: dwrite d0/d4f/daf/fb1 [0,4194304] 0 2024-04-02T04:42:44.715 INFO:tasks.workunit.client.1.smithi027.stdout:2/97: mknod d1/dd/d17/c1e 0 2024-04-02T04:42:44.718 INFO:tasks.workunit.client.1.smithi027.stdout:2/98: write d1/dd/d17/f14 [2742281,38815] 0 2024-04-02T04:42:44.719 INFO:tasks.workunit.client.1.smithi027.stdout:9/74: sync 2024-04-02T04:42:44.721 INFO:tasks.workunit.client.1.smithi027.stdout:0/125: dread d4/d13/f18 [0,4194304] 0 2024-04-02T04:42:44.725 INFO:tasks.workunit.client.1.smithi027.stdout:1/76: fsync d0/f16 0 2024-04-02T04:42:44.726 INFO:tasks.workunit.client.0.smithi016.stdout:1/603: creat d14/d19/d1e/d5e/fc3 x:0 0 0 2024-04-02T04:42:44.729 INFO:tasks.workunit.client.1.smithi027.stdout:9/75: dwrite f17 [0,4194304] 0 2024-04-02T04:42:44.729 INFO:tasks.workunit.client.1.smithi027.stdout:5/157: rename d0/df/d11/d13/c14 to d0/d1a/d20/c3f 0 2024-04-02T04:42:44.730 INFO:tasks.workunit.client.0.smithi016.stdout:6/686: truncate d0/d9/d16/d46/dbe/fd0 4162050 0 2024-04-02T04:42:44.733 INFO:tasks.workunit.client.0.smithi016.stdout:8/609: fsync d1/d6/d36/d8e/f9c 0 2024-04-02T04:42:44.733 INFO:tasks.workunit.client.0.smithi016.stdout:1/604: dwrite d14/d41/d43/f6f [0,4194304] 0 2024-04-02T04:42:44.735 INFO:tasks.workunit.client.1.smithi027.stdout:4/96: getdents . 0 2024-04-02T04:42:44.735 INFO:tasks.workunit.client.0.smithi016.stdout:8/610: read d1/d6/d36/f37 [2450455,46245] 0 2024-04-02T04:42:44.736 INFO:tasks.workunit.client.1.smithi027.stdout:4/97: read ff [396103,115975] 0 2024-04-02T04:42:44.738 INFO:tasks.workunit.client.1.smithi027.stdout:4/98: truncate f16 377172 0 2024-04-02T04:42:44.738 INFO:tasks.workunit.client.1.smithi027.stdout:4/99: stat f15 0 2024-04-02T04:42:44.739 INFO:tasks.workunit.client.0.smithi016.stdout:7/596: rmdir d0/d46 39 2024-04-02T04:42:44.740 INFO:tasks.workunit.client.0.smithi016.stdout:8/611: dwrite d1/d70/d71/d74/fc3 [0,4194304] 0 2024-04-02T04:42:44.743 INFO:tasks.workunit.client.0.smithi016.stdout:9/704: mknod d0/d3/d97/dac/cd9 0 2024-04-02T04:42:44.744 INFO:tasks.workunit.client.0.smithi016.stdout:9/705: stat d0/d4a/d99/d46 0 2024-04-02T04:42:44.744 INFO:tasks.workunit.client.0.smithi016.stdout:2/687: truncate d12/d26/f2c 1631689 0 2024-04-02T04:42:44.752 INFO:tasks.workunit.client.0.smithi016.stdout:5/594: symlink d3/d40/d4f/d6b/d63/lda 0 2024-04-02T04:42:44.755 INFO:tasks.workunit.client.0.smithi016.stdout:3/720: creat d2/d15/d16/d57/ff3 x:0 0 0 2024-04-02T04:42:44.755 INFO:tasks.workunit.client.1.smithi027.stdout:3/61: dwrite d1/f7 [4194304,4194304] 0 2024-04-02T04:42:44.758 INFO:tasks.workunit.client.1.smithi027.stdout:6/93: symlink d13/l22 0 2024-04-02T04:42:44.764 INFO:tasks.workunit.client.1.smithi027.stdout:0/126: symlink d4/d13/l26 0 2024-04-02T04:42:44.767 INFO:tasks.workunit.client.0.smithi016.stdout:6/687: mkdir d0/d9/d16/dd8 0 2024-04-02T04:42:44.768 INFO:tasks.workunit.client.0.smithi016.stdout:6/688: stat d0/d9/d16/d56/da4/cbc 0 2024-04-02T04:42:44.768 INFO:tasks.workunit.client.0.smithi016.stdout:6/689: stat d0/f41 0 2024-04-02T04:42:44.770 INFO:tasks.workunit.client.0.smithi016.stdout:1/605: readlink d14/d19/d1d/d45/lbe 0 2024-04-02T04:42:44.771 INFO:tasks.workunit.client.1.smithi027.stdout:9/76: write fa [3662475,101852] 0 2024-04-02T04:42:44.771 INFO:tasks.workunit.client.0.smithi016.stdout:1/606: fdatasync d14/d19/d1d/d45/d4f/faa 0 2024-04-02T04:42:44.771 INFO:tasks.workunit.client.1.smithi027.stdout:5/158: write d0/df/d11/d13/f3e [1043386,10309] 0 2024-04-02T04:42:44.775 INFO:tasks.workunit.client.1.smithi027.stdout:4/100: rename f18 to f1b 0 2024-04-02T04:42:44.777 INFO:tasks.workunit.client.0.smithi016.stdout:2/688: symlink d12/d9f/dac/d76/da0/lf1 0 2024-04-02T04:42:44.777 INFO:tasks.workunit.client.0.smithi016.stdout:2/689: chown d12/c9a 90 1 2024-04-02T04:42:44.778 INFO:tasks.workunit.client.1.smithi027.stdout:8/61: symlink d5/d7/l13 0 2024-04-02T04:42:44.783 INFO:tasks.workunit.client.1.smithi027.stdout:3/62: rename d1/l4 to d1/de/l12 0 2024-04-02T04:42:44.784 INFO:tasks.workunit.client.0.smithi016.stdout:2/690: dwrite d12/d35/d3b/d42/d8d/fcd [0,4194304] 0 2024-04-02T04:42:44.785 INFO:tasks.workunit.client.1.smithi027.stdout:4/101: dwrite f14 [0,4194304] 0 2024-04-02T04:42:44.786 INFO:tasks.workunit.client.1.smithi027.stdout:3/63: write d1/f8 [3011584,57047] 0 2024-04-02T04:42:44.787 INFO:tasks.workunit.client.1.smithi027.stdout:6/94: write f12 [2489383,72854] 0 2024-04-02T04:42:44.787 INFO:tasks.workunit.client.1.smithi027.stdout:5/159: dread d0/df/d11/d13/f3e [0,4194304] 0 2024-04-02T04:42:44.790 INFO:tasks.workunit.client.1.smithi027.stdout:2/99: creat d1/dd/d17/d12/f1f x:0 0 0 2024-04-02T04:42:44.794 INFO:tasks.workunit.client.1.smithi027.stdout:5/160: dread d0/df/d11/d13/f26 [0,4194304] 0 2024-04-02T04:42:44.803 INFO:tasks.workunit.client.0.smithi016.stdout:0/599: truncate d0/d16/d23/d3e/f98 699017 0 2024-04-02T04:42:44.804 INFO:tasks.workunit.client.0.smithi016.stdout:0/600: chown d0/d16/d24/d4e/d8d/f75 24764 1 2024-04-02T04:42:44.804 INFO:tasks.workunit.client.0.smithi016.stdout:9/706: mkdir d0/d9/d4d/dda 0 2024-04-02T04:42:44.804 INFO:tasks.workunit.client.0.smithi016.stdout:0/601: chown d0/d16/d24/d62/d9c/f5b 9116337 1 2024-04-02T04:42:44.806 INFO:tasks.workunit.client.0.smithi016.stdout:9/707: write d0/d4a/d99/d46/fb7 [5222827,108494] 0 2024-04-02T04:42:44.807 INFO:tasks.workunit.client.0.smithi016.stdout:5/595: mknod d3/d40/d4f/d6b/cdb 0 2024-04-02T04:42:44.809 INFO:tasks.workunit.client.1.smithi027.stdout:0/127: rename d4/d13/fe to d4/d16/f27 0 2024-04-02T04:42:44.810 INFO:tasks.workunit.client.0.smithi016.stdout:3/721: creat d2/d2e/d36/d48/da2/de7/ff4 x:0 0 0 2024-04-02T04:42:44.810 INFO:tasks.workunit.client.1.smithi027.stdout:7/96: sync 2024-04-02T04:42:44.811 INFO:tasks.workunit.client.1.smithi027.stdout:7/97: fdatasync d2/d7/f12 0 2024-04-02T04:42:44.813 INFO:tasks.workunit.client.1.smithi027.stdout:4/102: symlink l1c 0 2024-04-02T04:42:44.814 INFO:tasks.workunit.client.0.smithi016.stdout:1/607: dwrite d14/d19/f60 [0,4194304] 0 2024-04-02T04:42:44.814 INFO:tasks.workunit.client.0.smithi016.stdout:4/651: creat d5/d17/d18/d6d/fc3 x:0 0 0 2024-04-02T04:42:44.815 INFO:tasks.workunit.client.1.smithi027.stdout:3/64: mkdir d1/de/df/d13 0 2024-04-02T04:42:44.816 INFO:tasks.workunit.client.1.smithi027.stdout:3/65: readlink d1/l6 0 2024-04-02T04:42:44.817 INFO:tasks.workunit.client.0.smithi016.stdout:2/691: dread d12/d35/d3f/f7a [0,4194304] 0 2024-04-02T04:42:44.817 INFO:tasks.workunit.client.0.smithi016.stdout:1/608: dread - d14/d19/d1e/d71/dc0/fc2 zero size 2024-04-02T04:42:44.819 INFO:tasks.workunit.client.0.smithi016.stdout:3/722: dwrite d2/fab [0,4194304] 0 2024-04-02T04:42:44.820 INFO:tasks.workunit.client.1.smithi027.stdout:6/95: write d13/d1b/f1f [693867,116109] 0 2024-04-02T04:42:44.821 INFO:tasks.workunit.client.1.smithi027.stdout:6/96: chown fa 468327 1 2024-04-02T04:42:44.821 INFO:tasks.workunit.client.0.smithi016.stdout:3/723: write d2/d15/d1a/d45/f8f [1503719,115488] 0 2024-04-02T04:42:44.823 INFO:tasks.workunit.client.1.smithi027.stdout:7/98: dwrite d2/d7/f18 [0,4194304] 0 2024-04-02T04:42:44.825 INFO:tasks.workunit.client.0.smithi016.stdout:7/597: unlink d0/d46/d16/d49/cb9 0 2024-04-02T04:42:44.825 INFO:tasks.workunit.client.0.smithi016.stdout:7/598: readlink d0/d46/l1e 0 2024-04-02T04:42:44.828 INFO:tasks.workunit.client.1.smithi027.stdout:5/161: creat d0/df/d11/d37/f40 x:0 0 0 2024-04-02T04:42:44.834 INFO:tasks.workunit.client.1.smithi027.stdout:7/99: dwrite d2/d7/f12 [4194304,4194304] 0 2024-04-02T04:42:44.838 INFO:tasks.workunit.client.1.smithi027.stdout:8/62: link f4 d5/d7/f14 0 2024-04-02T04:42:44.842 INFO:tasks.workunit.client.0.smithi016.stdout:4/652: creat d5/d16/d3f/fc4 x:0 0 0 2024-04-02T04:42:44.843 INFO:tasks.workunit.client.1.smithi027.stdout:8/63: dread f4 [0,4194304] 0 2024-04-02T04:42:44.844 INFO:tasks.workunit.client.1.smithi027.stdout:2/100: rename d1/dd/d17/le to d1/dd/d17/d12/l20 0 2024-04-02T04:42:44.846 INFO:tasks.workunit.client.1.smithi027.stdout:4/103: fsync ff 0 2024-04-02T04:42:44.847 INFO:tasks.workunit.client.1.smithi027.stdout:8/64: dread d5/d7/f14 [0,4194304] 0 2024-04-02T04:42:44.848 INFO:tasks.workunit.client.0.smithi016.stdout:0/602: dwrite d0/d16/d2e/d36/f8b [0,4194304] 0 2024-04-02T04:42:44.848 INFO:tasks.workunit.client.1.smithi027.stdout:8/65: truncate d5/d7/f14 5187496 0 2024-04-02T04:42:44.850 INFO:tasks.workunit.client.0.smithi016.stdout:0/603: chown d0/d16/d24/d62/d9c/c77 875826 1 2024-04-02T04:42:44.850 INFO:tasks.workunit.client.1.smithi027.stdout:3/66: creat d1/de/df/f14 x:0 0 0 2024-04-02T04:42:44.852 INFO:tasks.workunit.client.0.smithi016.stdout:3/724: creat d2/d15/d70/ff5 x:0 0 0 2024-04-02T04:42:44.853 INFO:tasks.workunit.client.0.smithi016.stdout:0/604: write d0/d16/f54 [722644,47797] 0 2024-04-02T04:42:44.853 INFO:tasks.workunit.client.1.smithi027.stdout:1/77: getdents d0 0 2024-04-02T04:42:44.854 INFO:tasks.workunit.client.0.smithi016.stdout:1/609: dwrite d14/d19/d20/f3f [0,4194304] 0 2024-04-02T04:42:44.856 INFO:tasks.workunit.client.0.smithi016.stdout:7/599: mkdir d0/d46/d37/d3a/d4c/dd7 0 2024-04-02T04:42:44.857 INFO:tasks.workunit.client.1.smithi027.stdout:3/67: dread d1/fc [0,4194304] 0 2024-04-02T04:42:44.858 INFO:tasks.workunit.client.1.smithi027.stdout:6/97: creat d13/d1b/f23 x:0 0 0 2024-04-02T04:42:44.863 INFO:tasks.workunit.client.1.smithi027.stdout:1/78: dwrite d0/f5 [0,4194304] 0 2024-04-02T04:42:44.863 INFO:tasks.workunit.client.1.smithi027.stdout:9/77: rename c18 to c19 0 2024-04-02T04:42:44.863 INFO:tasks.workunit.client.0.smithi016.stdout:0/605: dwrite d0/d16/d24/d62/d9c/d8c/f92 [0,4194304] 0 2024-04-02T04:42:44.866 INFO:tasks.workunit.client.0.smithi016.stdout:7/600: dwrite d0/d21/f57 [0,4194304] 0 2024-04-02T04:42:44.868 INFO:tasks.workunit.client.1.smithi027.stdout:9/78: dread fa [0,4194304] 0 2024-04-02T04:42:44.871 INFO:tasks.workunit.client.1.smithi027.stdout:2/101: rename d1/l4 to d1/dd/d17/l21 0 2024-04-02T04:42:44.876 INFO:tasks.workunit.client.0.smithi016.stdout:2/692: truncate d12/d35/d3f/d2b/fc1 3616469 0 2024-04-02T04:42:44.877 INFO:tasks.workunit.client.0.smithi016.stdout:2/693: fsync d12/d35/d3f/f61 0 2024-04-02T04:42:44.878 INFO:tasks.workunit.client.0.smithi016.stdout:2/694: truncate d12/d35/d3f/f4b 4937346 0 2024-04-02T04:42:44.879 INFO:tasks.workunit.client.1.smithi027.stdout:4/104: creat f1d x:0 0 0 2024-04-02T04:42:44.880 INFO:tasks.workunit.client.1.smithi027.stdout:4/105: chown ff 76982 1 2024-04-02T04:42:44.888 INFO:tasks.workunit.client.1.smithi027.stdout:4/106: dwrite f12 [0,4194304] 0 2024-04-02T04:42:44.892 INFO:tasks.workunit.client.1.smithi027.stdout:3/68: mknod d1/de/df/c15 0 2024-04-02T04:42:44.893 INFO:tasks.workunit.client.1.smithi027.stdout:3/69: write d1/f7 [7552841,40396] 0 2024-04-02T04:42:44.895 INFO:tasks.workunit.client.0.smithi016.stdout:5/596: rename d3/d40/d4f/d6b/d2a to d3/df/d18/d26/ddc 0 2024-04-02T04:42:44.896 INFO:tasks.workunit.client.1.smithi027.stdout:8/66: write d5/d7/f14 [2181908,110554] 0 2024-04-02T04:42:44.897 INFO:tasks.workunit.client.1.smithi027.stdout:6/98: mkdir d13/d1b/d24 0 2024-04-02T04:42:44.897 INFO:tasks.workunit.client.1.smithi027.stdout:8/67: write f4 [4244391,126615] 0 2024-04-02T04:42:44.899 INFO:tasks.workunit.client.1.smithi027.stdout:6/99: write d13/d1b/f1f [1315149,4494] 0 2024-04-02T04:42:44.901 INFO:tasks.workunit.client.0.smithi016.stdout:5/597: dwrite d3/f5b [0,4194304] 0 2024-04-02T04:42:44.901 INFO:tasks.workunit.client.1.smithi027.stdout:6/100: fdatasync d13/d1b/f23 0 2024-04-02T04:42:44.904 INFO:tasks.workunit.client.1.smithi027.stdout:8/68: dread d5/d7/f14 [0,4194304] 0 2024-04-02T04:42:44.904 INFO:tasks.workunit.client.1.smithi027.stdout:5/162: truncate d0/f6 4034121 0 2024-04-02T04:42:44.906 INFO:tasks.workunit.client.1.smithi027.stdout:5/163: stat d0/d1a/d20 0 2024-04-02T04:42:44.911 INFO:tasks.workunit.client.1.smithi027.stdout:1/79: rename d0/l13 to d0/l1c 0 2024-04-02T04:42:44.913 INFO:tasks.workunit.client.0.smithi016.stdout:5/598: dread d3/f59 [4194304,4194304] 0 2024-04-02T04:42:44.915 INFO:tasks.workunit.client.0.smithi016.stdout:3/725: rename d2/d15/d16/d8a/dd1/d6e/cca to d2/d15/d16/d8a/dcd/cf6 0 2024-04-02T04:42:44.916 INFO:tasks.workunit.client.0.smithi016.stdout:3/726: fdatasync d2/d2e/d36/d48/da2/f7b 0 2024-04-02T04:42:44.918 INFO:tasks.workunit.client.0.smithi016.stdout:3/727: readlink d2/d15/d16/d57/d96/lda 0 2024-04-02T04:42:44.919 INFO:tasks.workunit.client.1.smithi027.stdout:2/102: unlink d1/df/f11 0 2024-04-02T04:42:44.919 INFO:tasks.workunit.client.0.smithi016.stdout:7/601: symlink d0/d46/d37/d3a/ld8 0 2024-04-02T04:42:44.920 INFO:tasks.workunit.client.1.smithi027.stdout:7/100: write f1 [1567424,29972] 0 2024-04-02T04:42:44.921 INFO:tasks.workunit.client.0.smithi016.stdout:2/695: dwrite d12/d35/d3b/f63 [0,4194304] 0 2024-04-02T04:42:44.925 INFO:tasks.workunit.client.1.smithi027.stdout:7/101: dread d2/d7/f12 [4194304,4194304] 0 2024-04-02T04:42:44.927 INFO:tasks.workunit.client.1.smithi027.stdout:7/102: write d2/d7/f18 [565897,119048] 0 2024-04-02T04:42:44.927 INFO:tasks.workunit.client.1.smithi027.stdout:7/103: fdatasync f1 0 2024-04-02T04:42:44.928 INFO:tasks.workunit.client.0.smithi016.stdout:1/610: rmdir d14/da9/dae 0 2024-04-02T04:42:44.929 INFO:tasks.workunit.client.1.smithi027.stdout:4/107: write f1b [712144,59520] 0 2024-04-02T04:42:44.931 INFO:tasks.workunit.client.0.smithi016.stdout:7/602: dwrite d0/d28/f39 [4194304,4194304] 0 2024-04-02T04:42:44.931 INFO:tasks.workunit.client.0.smithi016.stdout:5/599: creat d3/df/d18/d26/d70/fdd x:0 0 0 2024-04-02T04:42:44.932 INFO:tasks.workunit.client.0.smithi016.stdout:0/606: rename d0/d4f/daf/fcf to d0/d16/d24/d4e/d8d/d71/dc7/fd2 0 2024-04-02T04:42:44.934 INFO:tasks.workunit.client.0.smithi016.stdout:1/611: dread - d14/d19/d1d/d45/d4f/d86/fa6 zero size 2024-04-02T04:42:44.936 INFO:tasks.workunit.client.0.smithi016.stdout:2/696: dwrite d12/d26/f48 [0,4194304] 0 2024-04-02T04:42:44.936 INFO:tasks.workunit.client.0.smithi016.stdout:2/697: fdatasync f9 0 2024-04-02T04:42:44.938 INFO:tasks.workunit.client.1.smithi027.stdout:0/128: sync 2024-04-02T04:42:44.943 INFO:tasks.workunit.client.0.smithi016.stdout:0/607: dwrite d0/d4f/f95 [0,4194304] 0 2024-04-02T04:42:44.943 INFO:tasks.workunit.client.1.smithi027.stdout:5/164: read d0/df/d11/d13/f3e [643862,33784] 0 2024-04-02T04:42:44.945 INFO:tasks.workunit.client.0.smithi016.stdout:0/608: readlink d0/d16/d2e/d37/l81 0 2024-04-02T04:42:44.945 INFO:tasks.workunit.client.0.smithi016.stdout:5/600: mkdir d3/df/d18/d26/ddc/dde 0 2024-04-02T04:42:44.945 INFO:tasks.workunit.client.0.smithi016.stdout:2/698: dwrite d12/d35/d3b/d42/d59/f9e [0,4194304] 0 2024-04-02T04:42:44.947 INFO:tasks.workunit.client.1.smithi027.stdout:5/165: dread d0/df/d11/d13/f3e [0,4194304] 0 2024-04-02T04:42:44.947 INFO:tasks.workunit.client.0.smithi016.stdout:0/609: stat d0/d16/d23 0 2024-04-02T04:42:44.947 INFO:tasks.workunit.client.0.smithi016.stdout:7/603: mknod d0/d46/d37/d47/cd9 0 2024-04-02T04:42:44.953 INFO:tasks.workunit.client.1.smithi027.stdout:1/80: dwrite d0/f5 [0,4194304] 0 2024-04-02T04:42:44.957 INFO:tasks.workunit.client.1.smithi027.stdout:9/79: mkdir d1a 0 2024-04-02T04:42:44.960 INFO:tasks.workunit.client.1.smithi027.stdout:2/103: chown d1/dd/d17/l21 131366559 1 2024-04-02T04:42:44.962 INFO:tasks.workunit.client.1.smithi027.stdout:2/104: chown d1/dd/d17/d12/f1f 248 1 2024-04-02T04:42:44.963 INFO:tasks.workunit.client.1.smithi027.stdout:9/80: dread f17 [0,4194304] 0 2024-04-02T04:42:44.970 INFO:tasks.workunit.client.1.smithi027.stdout:7/104: truncate d2/df/f10 505032 0 2024-04-02T04:42:44.971 INFO:tasks.workunit.client.0.smithi016.stdout:5/601: rmdir d3/df/d18/d2f 39 2024-04-02T04:42:44.971 INFO:tasks.workunit.client.0.smithi016.stdout:5/602: fsync d3/f4 0 2024-04-02T04:42:44.973 INFO:tasks.workunit.client.1.smithi027.stdout:4/108: rename ff to f1e 0 2024-04-02T04:42:44.973 INFO:tasks.workunit.client.0.smithi016.stdout:0/610: mknod d0/d4f/cd3 0 2024-04-02T04:42:44.974 INFO:tasks.workunit.client.0.smithi016.stdout:0/611: write d0/d16/d24/fc5 [631008,77641] 0 2024-04-02T04:42:44.975 INFO:tasks.workunit.client.1.smithi027.stdout:3/70: creat d1/de/df/d13/f16 x:0 0 0 2024-04-02T04:42:44.976 INFO:tasks.workunit.client.0.smithi016.stdout:7/604: rename d0/d46/d37/dbe/dae/dc1 to d0/d46/da2/dc8/dda 0 2024-04-02T04:42:44.980 INFO:tasks.workunit.client.0.smithi016.stdout:3/728: creat d2/d15/ff7 x:0 0 0 2024-04-02T04:42:44.980 INFO:tasks.workunit.client.1.smithi027.stdout:6/101: mkdir d13/d1b/d24/d25 0 2024-04-02T04:42:44.983 INFO:tasks.workunit.client.0.smithi016.stdout:7/605: dwrite d0/d46/f9a [0,4194304] 0 2024-04-02T04:42:44.983 INFO:tasks.workunit.client.1.smithi027.stdout:1/81: fsync d0/f5 0 2024-04-02T04:42:44.985 INFO:tasks.workunit.client.0.smithi016.stdout:5/603: creat d3/d4c/dc4/dd7/fdf x:0 0 0 2024-04-02T04:42:44.987 INFO:tasks.workunit.client.0.smithi016.stdout:0/612: mknod d0/d16/d23/d3e/cd4 0 2024-04-02T04:42:44.988 INFO:tasks.workunit.client.1.smithi027.stdout:5/166: mkdir d0/df/d11/d41 0 2024-04-02T04:42:44.989 INFO:tasks.workunit.client.1.smithi027.stdout:5/167: fsync d0/d1d/f35 0 2024-04-02T04:42:44.990 INFO:tasks.workunit.client.1.smithi027.stdout:5/168: read d0/f33 [620133,66470] 0 2024-04-02T04:42:44.992 INFO:tasks.workunit.client.1.smithi027.stdout:8/69: truncate d5/d8/db/dd/f10 221231 0 2024-04-02T04:42:44.992 INFO:tasks.workunit.client.0.smithi016.stdout:3/729: stat d2/d15/d16/d8a/dd1/d6e/d6f/db0/de2 0 2024-04-02T04:42:44.993 INFO:tasks.workunit.client.0.smithi016.stdout:1/612: dwrite d14/f28 [0,4194304] 0 2024-04-02T04:42:44.995 INFO:tasks.workunit.client.0.smithi016.stdout:7/606: rename d0/d28/cad to d0/d46/d37/d3a/d84/d8a/cdb 0 2024-04-02T04:42:44.995 INFO:tasks.workunit.client.0.smithi016.stdout:1/613: dread - d14/d19/d1e/d5e/fc3 zero size 2024-04-02T04:42:44.996 INFO:tasks.workunit.client.0.smithi016.stdout:0/613: creat d0/d4f/daf/fd5 x:0 0 0 2024-04-02T04:42:44.996 INFO:tasks.workunit.client.0.smithi016.stdout:7/607: readlink d0/l10 0 2024-04-02T04:42:44.999 INFO:tasks.workunit.client.1.smithi027.stdout:0/129: mkdir d4/d21/d28 0 2024-04-02T04:42:45.001 INFO:tasks.workunit.client.0.smithi016.stdout:2/699: dwrite d12/d35/d3f/d2b/fc1 [0,4194304] 0 2024-04-02T04:42:45.001 INFO:tasks.workunit.client.1.smithi027.stdout:1/82: truncate d0/f14 5220743 0 2024-04-02T04:42:45.002 INFO:tasks.workunit.client.1.smithi027.stdout:1/83: write d0/f5 [3792868,122674] 0 2024-04-02T04:42:45.005 INFO:tasks.workunit.client.0.smithi016.stdout:7/608: truncate d0/d46/d37/d47/f5c 5730504 0 2024-04-02T04:42:45.005 INFO:tasks.workunit.client.0.smithi016.stdout:3/730: dread d2/d2e/d36/d66/d90/fef [0,4194304] 0 2024-04-02T04:42:45.005 INFO:tasks.workunit.client.0.smithi016.stdout:7/609: fdatasync d0/d46/d52/fc4 0 2024-04-02T04:42:45.007 INFO:tasks.workunit.client.0.smithi016.stdout:1/614: rmdir d14/d19/d1e/d71/d9c 39 2024-04-02T04:42:45.009 INFO:tasks.workunit.client.1.smithi027.stdout:2/105: symlink d1/l22 0 2024-04-02T04:42:45.012 INFO:tasks.workunit.client.0.smithi016.stdout:3/731: dwrite d2/d15/d16/f54 [0,4194304] 0 2024-04-02T04:42:45.012 INFO:tasks.workunit.client.1.smithi027.stdout:9/81: mkdir d1a/d1b 0 2024-04-02T04:42:45.018 INFO:tasks.workunit.client.0.smithi016.stdout:7/610: mknod d0/d87/cdc 0 2024-04-02T04:42:45.019 INFO:tasks.workunit.client.0.smithi016.stdout:3/732: dwrite d2/f26 [0,4194304] 0 2024-04-02T04:42:45.019 INFO:tasks.workunit.client.1.smithi027.stdout:0/130: creat d4/d16/d1b/f29 x:0 0 0 2024-04-02T04:42:45.019 INFO:tasks.workunit.client.0.smithi016.stdout:7/611: chown d0/d46/d37/lc3 9 1 2024-04-02T04:42:45.020 INFO:tasks.workunit.client.0.smithi016.stdout:3/733: stat d2/d15/d1a/d45/lae 0 2024-04-02T04:42:45.021 INFO:tasks.workunit.client.1.smithi027.stdout:0/131: stat d4/fb 0 2024-04-02T04:42:45.022 INFO:tasks.workunit.client.1.smithi027.stdout:0/132: dread - d4/d16/d1b/f29 zero size 2024-04-02T04:42:45.022 INFO:tasks.workunit.client.0.smithi016.stdout:3/734: dread - d2/d15/d70/ff5 zero size 2024-04-02T04:42:45.023 INFO:tasks.workunit.client.0.smithi016.stdout:1/615: rename fd to d14/d41/d43/d68/dc1/fc4 0 2024-04-02T04:42:45.023 INFO:tasks.workunit.client.1.smithi027.stdout:0/133: dread - d4/f1d zero size 2024-04-02T04:42:45.026 INFO:tasks.workunit.client.0.smithi016.stdout:1/616: dread d14/d19/d20/f3b [0,4194304] 0 2024-04-02T04:42:45.027 INFO:tasks.workunit.client.0.smithi016.stdout:8/612: sync 2024-04-02T04:42:45.027 INFO:tasks.workunit.client.1.smithi027.stdout:4/109: dwrite f2 [4194304,4194304] 0 2024-04-02T04:42:45.028 INFO:tasks.workunit.client.1.smithi027.stdout:5/169: mknod d0/df/d11/d41/c42 0 2024-04-02T04:42:45.030 INFO:tasks.workunit.client.1.smithi027.stdout:1/84: chown d0/c12 583174 1 2024-04-02T04:42:45.035 INFO:tasks.workunit.client.1.smithi027.stdout:2/106: mkdir d1/dd/d23 0 2024-04-02T04:42:45.040 INFO:tasks.workunit.client.1.smithi027.stdout:6/102: link d13/l15 d13/d1b/l26 0 2024-04-02T04:42:45.042 INFO:tasks.workunit.client.1.smithi027.stdout:6/103: chown d13/d1b/f1d 29526 1 2024-04-02T04:42:45.043 INFO:tasks.workunit.client.0.smithi016.stdout:0/614: truncate d0/d16/f54 2638443 0 2024-04-02T04:42:45.043 INFO:tasks.workunit.client.0.smithi016.stdout:0/615: chown d0/d16/d2e/f85 2811 1 2024-04-02T04:42:45.044 INFO:tasks.workunit.client.1.smithi027.stdout:6/104: truncate d13/f16 1004618 0 2024-04-02T04:42:45.046 INFO:tasks.workunit.client.0.smithi016.stdout:3/735: creat d2/d15/d16/d8a/dd1/d6e/d6f/db0/de2/ff8 x:0 0 0 2024-04-02T04:42:45.046 INFO:tasks.workunit.client.1.smithi027.stdout:9/82: dwrite f12 [0,4194304] 0 2024-04-02T04:42:45.047 INFO:tasks.workunit.client.1.smithi027.stdout:9/83: truncate f12 4251423 0 2024-04-02T04:42:45.047 INFO:tasks.workunit.client.0.smithi016.stdout:3/736: read - d2/d15/d16/d8a/dd1/d6e/d6f/db0/de2/ff8 zero size 2024-04-02T04:42:45.048 INFO:tasks.workunit.client.1.smithi027.stdout:4/110: rename l4 to l1f 0 2024-04-02T04:42:45.048 INFO:tasks.workunit.client.0.smithi016.stdout:7/612: write d0/d46/f1d [3038454,8189] 0 2024-04-02T04:42:45.050 INFO:tasks.workunit.client.0.smithi016.stdout:1/617: rename d14/d41/l67 to d14/d19/d50/lc5 0 2024-04-02T04:42:45.055 INFO:tasks.workunit.client.0.smithi016.stdout:3/737: dwrite d2/d2e/fd7 [0,4194304] 0 2024-04-02T04:42:45.056 INFO:tasks.workunit.client.0.smithi016.stdout:7/613: creat d0/d46/d52/daf/fdd x:0 0 0 2024-04-02T04:42:45.057 INFO:tasks.workunit.client.0.smithi016.stdout:7/614: readlink d0/l10 0 2024-04-02T04:42:45.057 INFO:tasks.workunit.client.1.smithi027.stdout:1/85: dwrite d0/de/ff [0,4194304] 0 2024-04-02T04:42:45.058 INFO:tasks.workunit.client.0.smithi016.stdout:3/738: dread - d2/d15/d16/d8a/f93 zero size 2024-04-02T04:42:45.059 INFO:tasks.workunit.client.0.smithi016.stdout:2/700: getdents d12/d9f/dac/d4f 0 2024-04-02T04:42:45.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:44 smithi016 ceph-mon[28969]: pgmap v21: 65 pgs: 65 active+clean; 1.2 GiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 35 MiB/s rd, 162 MiB/s wr, 401 op/s 2024-04-02T04:42:45.062 INFO:tasks.workunit.client.1.smithi027.stdout:9/84: dread f17 [0,4194304] 0 2024-04-02T04:42:45.063 INFO:tasks.workunit.client.1.smithi027.stdout:2/107: creat d1/dd/d23/f24 x:0 0 0 2024-04-02T04:42:45.064 INFO:tasks.workunit.client.1.smithi027.stdout:2/108: fsync d1/db/f1d 0 2024-04-02T04:42:45.065 INFO:tasks.workunit.client.0.smithi016.stdout:7/615: creat d0/d46/d37/dbe/fde x:0 0 0 2024-04-02T04:42:45.066 INFO:tasks.workunit.client.1.smithi027.stdout:6/105: mkdir d13/d1b/d24/d25/d27 0 2024-04-02T04:42:45.066 INFO:tasks.workunit.client.1.smithi027.stdout:2/109: dread d1/db/f1d [0,4194304] 0 2024-04-02T04:42:45.070 INFO:tasks.workunit.client.1.smithi027.stdout:6/106: write d13/d1b/f20 [16189,4482] 0 2024-04-02T04:42:45.070 INFO:tasks.workunit.client.1.smithi027.stdout:2/110: write d1/dd/d23/f24 [649453,76913] 0 2024-04-02T04:42:45.071 INFO:tasks.workunit.client.0.smithi016.stdout:2/701: symlink d12/d9f/lf2 0 2024-04-02T04:42:45.071 INFO:tasks.workunit.client.0.smithi016.stdout:2/702: readlink d12/d35/d3f/d2b/l7d 0 2024-04-02T04:42:45.072 INFO:tasks.workunit.client.1.smithi027.stdout:5/170: link d0/df/d2f/d3b/c3c d0/df/d11/c43 0 2024-04-02T04:42:45.078 INFO:tasks.workunit.client.1.smithi027.stdout:0/134: getdents d4/d21 0 2024-04-02T04:42:45.080 INFO:tasks.workunit.client.1.smithi027.stdout:5/171: dwrite d0/df/d11/d13/f28 [0,4194304] 0 2024-04-02T04:42:45.083 INFO:tasks.workunit.client.1.smithi027.stdout:5/172: chown d0/df/d11/d13/f26 16 1 2024-04-02T04:42:45.086 INFO:tasks.workunit.client.1.smithi027.stdout:2/111: mknod d1/dd/d17/c25 0 2024-04-02T04:42:45.088 INFO:tasks.workunit.client.1.smithi027.stdout:9/85: creat d1a/d1b/f1c x:0 0 0 2024-04-02T04:42:45.088 INFO:tasks.workunit.client.1.smithi027.stdout:2/112: fdatasync d1/f13 0 2024-04-02T04:42:45.089 INFO:tasks.workunit.client.1.smithi027.stdout:5/173: dread - d0/df/d11/d37/f40 zero size 2024-04-02T04:42:45.090 INFO:tasks.workunit.client.1.smithi027.stdout:8/70: truncate d5/d8/db/dd/f10 390991 0 2024-04-02T04:42:45.092 INFO:tasks.workunit.client.1.smithi027.stdout:0/135: dwrite d4/f1d [0,4194304] 0 2024-04-02T04:42:45.095 INFO:tasks.workunit.client.1.smithi027.stdout:9/86: mkdir d1a/d1d 0 2024-04-02T04:42:45.096 INFO:tasks.workunit.client.1.smithi027.stdout:6/107: dwrite d13/f18 [0,4194304] 0 2024-04-02T04:42:45.098 INFO:tasks.workunit.client.1.smithi027.stdout:1/86: link d0/fb d0/f1d 0 2024-04-02T04:42:45.102 INFO:tasks.workunit.client.1.smithi027.stdout:5/174: fdatasync d0/df/d11/d13/f3e 0 2024-04-02T04:42:45.104 INFO:tasks.workunit.client.1.smithi027.stdout:8/71: mkdir d5/d8/d15 0 2024-04-02T04:42:45.106 INFO:tasks.workunit.client.1.smithi027.stdout:0/136: rename d4/d13/fd to d4/d13/f2a 0 2024-04-02T04:42:45.106 INFO:tasks.workunit.client.1.smithi027.stdout:0/137: chown d4/c8 27321 1 2024-04-02T04:42:45.107 INFO:tasks.workunit.client.1.smithi027.stdout:0/138: write d4/f6 [620439,130397] 0 2024-04-02T04:42:45.108 INFO:tasks.workunit.client.1.smithi027.stdout:9/87: fsync fa 0 2024-04-02T04:42:45.110 INFO:tasks.workunit.client.1.smithi027.stdout:6/108: unlink ff 0 2024-04-02T04:42:45.115 INFO:tasks.workunit.client.1.smithi027.stdout:5/175: mknod d0/d1d/c44 0 2024-04-02T04:42:45.115 INFO:tasks.workunit.client.1.smithi027.stdout:5/176: chown d0/df/d11/d13 6347 1 2024-04-02T04:42:45.116 INFO:tasks.workunit.client.1.smithi027.stdout:8/72: readlink d5/d8/db/le 0 2024-04-02T04:42:45.117 INFO:tasks.workunit.client.1.smithi027.stdout:5/177: write d0/df/d2f/d3b/f3d [451614,39552] 0 2024-04-02T04:42:45.118 INFO:tasks.workunit.client.1.smithi027.stdout:5/178: fsync d0/df/d2f/f38 0 2024-04-02T04:42:45.121 INFO:tasks.workunit.client.1.smithi027.stdout:9/88: symlink d1a/d1b/l1e 0 2024-04-02T04:42:45.123 INFO:tasks.workunit.client.1.smithi027.stdout:5/179: dread d0/df/d11/d13/f28 [0,4194304] 0 2024-04-02T04:42:45.124 INFO:tasks.workunit.client.1.smithi027.stdout:6/109: mkdir d13/d1b/d24/d28 0 2024-04-02T04:42:45.124 INFO:tasks.workunit.client.1.smithi027.stdout:6/110: truncate f7 4394552 0 2024-04-02T04:42:45.125 INFO:tasks.workunit.client.1.smithi027.stdout:0/139: rename d4/d13 to d4/d16/d2b 0 2024-04-02T04:42:45.126 INFO:tasks.workunit.client.1.smithi027.stdout:1/87: unlink d0/de/l15 0 2024-04-02T04:42:45.128 INFO:tasks.workunit.client.1.smithi027.stdout:2/113: rmdir d1/df 0 2024-04-02T04:42:45.130 INFO:tasks.workunit.client.1.smithi027.stdout:8/73: truncate d5/d7/f14 1382844 0 2024-04-02T04:42:45.133 INFO:tasks.workunit.client.1.smithi027.stdout:0/140: dwrite d4/d16/d1b/f20 [0,4194304] 0 2024-04-02T04:42:45.134 INFO:tasks.workunit.client.1.smithi027.stdout:9/89: creat d1a/f1f x:0 0 0 2024-04-02T04:42:45.136 INFO:tasks.workunit.client.1.smithi027.stdout:5/180: mknod d0/df/d11/c45 0 2024-04-02T04:42:45.138 INFO:tasks.workunit.client.0.smithi016.stdout:7/616: dwrite d0/d46/d16/f8e [0,4194304] 0 2024-04-02T04:42:45.139 INFO:tasks.workunit.client.0.smithi016.stdout:1/618: truncate d14/d19/f6b 3160841 0 2024-04-02T04:42:45.139 INFO:tasks.workunit.client.1.smithi027.stdout:2/114: dwrite d1/f6 [4194304,4194304] 0 2024-04-02T04:42:45.140 INFO:tasks.workunit.client.0.smithi016.stdout:8/613: truncate d1/d6/d36/d3f/d40/d49/f2a 3228944 0 2024-04-02T04:42:45.140 INFO:tasks.workunit.client.0.smithi016.stdout:7/617: readlink d0/l88 0 2024-04-02T04:42:45.140 INFO:tasks.workunit.client.1.smithi027.stdout:2/115: read d1/db/f1d [365948,18674] 0 2024-04-02T04:42:45.143 INFO:tasks.workunit.client.0.smithi016.stdout:8/614: write d1/d6/d36/d3f/d40/fb6 [4774891,46427] 0 2024-04-02T04:42:45.143 INFO:tasks.workunit.client.1.smithi027.stdout:9/90: dwrite f5 [4194304,4194304] 0 2024-04-02T04:42:45.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:44 smithi027 ceph-mon[37550]: pgmap v21: 65 pgs: 65 active+clean; 1.2 GiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 35 MiB/s rd, 162 MiB/s wr, 401 op/s 2024-04-02T04:42:45.147 INFO:tasks.workunit.client.1.smithi027.stdout:8/74: rename d5/d8/db/le to d5/l16 0 2024-04-02T04:42:45.148 INFO:tasks.workunit.client.0.smithi016.stdout:0/616: getdents d0/d16/d23/d3e 0 2024-04-02T04:42:45.149 INFO:tasks.workunit.client.0.smithi016.stdout:0/617: chown d0/d16/d23/l4d 94555 1 2024-04-02T04:42:45.158 INFO:tasks.workunit.client.1.smithi027.stdout:0/141: creat d4/d16/d2b/f2c x:0 0 0 2024-04-02T04:42:45.159 INFO:tasks.workunit.client.1.smithi027.stdout:0/142: fsync d4/d16/d1b/f29 0 2024-04-02T04:42:45.160 INFO:tasks.workunit.client.0.smithi016.stdout:1/619: mknod d14/d19/d1d/d45/d4f/d86/cc6 0 2024-04-02T04:42:45.161 INFO:tasks.workunit.client.1.smithi027.stdout:0/143: truncate d4/fb 1012258 0 2024-04-02T04:42:45.161 INFO:tasks.workunit.client.1.smithi027.stdout:6/111: creat d13/d1b/d24/d28/f29 x:0 0 0 2024-04-02T04:42:45.164 INFO:tasks.workunit.client.1.smithi027.stdout:1/88: mkdir d0/d1e 0 2024-04-02T04:42:45.165 INFO:tasks.workunit.client.1.smithi027.stdout:5/181: mknod d0/df/d11/c46 0 2024-04-02T04:42:45.167 INFO:tasks.workunit.client.1.smithi027.stdout:0/144: dwrite d4/f6 [0,4194304] 0 2024-04-02T04:42:45.168 INFO:tasks.workunit.client.1.smithi027.stdout:0/145: fsync d4/d16/d1b/f20 0 2024-04-02T04:42:45.169 INFO:tasks.workunit.client.1.smithi027.stdout:2/116: creat d1/dd/d23/f26 x:0 0 0 2024-04-02T04:42:45.171 INFO:tasks.workunit.client.0.smithi016.stdout:2/703: link d12/d35/d3b/fc7 d12/d70/ff3 0 2024-04-02T04:42:45.174 INFO:tasks.workunit.client.0.smithi016.stdout:7/618: rename d0/d46/d37/d3a/d43/d96/d9b to d0/d46/d37/d3a/d43/d96/ddf 0 2024-04-02T04:42:45.178 INFO:tasks.workunit.client.1.smithi027.stdout:2/117: dread d1/f6 [4194304,4194304] 0 2024-04-02T04:42:45.178 INFO:tasks.workunit.client.1.smithi027.stdout:0/146: dwrite d4/f6 [4194304,4194304] 0 2024-04-02T04:42:45.179 INFO:tasks.workunit.client.0.smithi016.stdout:1/620: rename f7 to d14/d19/d1d/d45/d4f/d86/db4/fc7 0 2024-04-02T04:42:45.181 INFO:tasks.workunit.client.1.smithi027.stdout:6/112: creat d13/d1b/d24/d28/f2a x:0 0 0 2024-04-02T04:42:45.181 INFO:tasks.workunit.client.1.smithi027.stdout:1/89: creat d0/de/f1f x:0 0 0 2024-04-02T04:42:45.181 INFO:tasks.workunit.client.0.smithi016.stdout:7/619: dwrite d0/d46/d37/d3a/d4c/fb0 [0,4194304] 0 2024-04-02T04:42:45.182 INFO:tasks.workunit.client.0.smithi016.stdout:2/704: mknod d12/d35/d3f/d56/d66/cf4 0 2024-04-02T04:42:45.183 INFO:tasks.workunit.client.1.smithi027.stdout:5/182: chown d0/df/c2d 49 1 2024-04-02T04:42:45.190 INFO:tasks.workunit.client.1.smithi027.stdout:9/91: getdents d1a/d1d 0 2024-04-02T04:42:45.190 INFO:tasks.workunit.client.0.smithi016.stdout:1/621: unlink d14/d19/d1d/f1f 0 2024-04-02T04:42:45.191 INFO:tasks.workunit.client.0.smithi016.stdout:2/705: dwrite d12/d26/f48 [4194304,4194304] 0 2024-04-02T04:42:45.194 INFO:tasks.workunit.client.1.smithi027.stdout:2/118: unlink d1/db/c16 0 2024-04-02T04:42:45.195 INFO:tasks.workunit.client.0.smithi016.stdout:4/653: sync 2024-04-02T04:42:45.195 INFO:tasks.workunit.client.0.smithi016.stdout:9/708: sync 2024-04-02T04:42:45.195 INFO:tasks.workunit.client.0.smithi016.stdout:6/690: sync 2024-04-02T04:42:45.195 INFO:tasks.workunit.client.0.smithi016.stdout:8/615: link d1/d31/d67/f79 d1/d6/d36/d3f/d77/fd0 0 2024-04-02T04:42:45.195 INFO:tasks.workunit.client.1.smithi027.stdout:9/92: dread fa [0,4194304] 0 2024-04-02T04:42:45.196 INFO:tasks.workunit.client.1.smithi027.stdout:9/93: dread - f15 zero size 2024-04-02T04:42:45.196 INFO:tasks.workunit.client.0.smithi016.stdout:6/691: chown d0/d9/df/fb9 2850755 1 2024-04-02T04:42:45.197 INFO:tasks.workunit.client.0.smithi016.stdout:9/709: chown d0/d3/d97/dac 1050502 1 2024-04-02T04:42:45.198 INFO:tasks.workunit.client.1.smithi027.stdout:6/113: creat d13/d1b/f2b x:0 0 0 2024-04-02T04:42:45.199 INFO:tasks.workunit.client.0.smithi016.stdout:9/710: write d0/d9/f76 [337789,123531] 0 2024-04-02T04:42:45.199 INFO:tasks.workunit.client.0.smithi016.stdout:7/620: getdents d0/d46/d16/dc2 0 2024-04-02T04:42:45.199 INFO:tasks.workunit.client.1.smithi027.stdout:6/114: chown d13/d1b/f23 3 1 2024-04-02T04:42:45.206 INFO:tasks.workunit.client.0.smithi016.stdout:4/654: creat d5/d1d/d5f/d65/d6f/fc5 x:0 0 0 2024-04-02T04:42:45.207 INFO:tasks.workunit.client.1.smithi027.stdout:2/119: symlink d1/dd/d17/l27 0 2024-04-02T04:42:45.208 INFO:tasks.workunit.client.0.smithi016.stdout:1/622: dwrite d14/d19/d20/f24 [0,4194304] 0 2024-04-02T04:42:45.209 INFO:tasks.workunit.client.1.smithi027.stdout:2/120: dread d1/db/f1d [0,4194304] 0 2024-04-02T04:42:45.210 INFO:tasks.workunit.client.1.smithi027.stdout:7/105: sync 2024-04-02T04:42:45.210 INFO:tasks.workunit.client.1.smithi027.stdout:7/106: chown d2/f13 1271 1 2024-04-02T04:42:45.210 INFO:tasks.workunit.client.1.smithi027.stdout:7/107: chown d2/d7/da/dc/d1b 14016 1 2024-04-02T04:42:45.211 INFO:tasks.workunit.client.0.smithi016.stdout:8/616: unlink d1/d6/d36/d3f/d40/d49/f2b 0 2024-04-02T04:42:45.211 INFO:tasks.workunit.client.1.smithi027.stdout:7/108: write f1 [696391,126654] 0 2024-04-02T04:42:45.212 INFO:tasks.workunit.client.1.smithi027.stdout:7/109: truncate f1 2285484 0 2024-04-02T04:42:45.213 INFO:tasks.workunit.client.0.smithi016.stdout:6/692: dread d0/f40 [0,4194304] 0 2024-04-02T04:42:45.215 INFO:tasks.workunit.client.0.smithi016.stdout:9/711: creat d0/d9/db/dc9/d53/fdb x:0 0 0 2024-04-02T04:42:45.218 INFO:tasks.workunit.client.0.smithi016.stdout:7/621: rename d0/d46/c14 to d0/d46/d37/ce0 0 2024-04-02T04:42:45.224 INFO:tasks.workunit.client.0.smithi016.stdout:2/706: symlink d12/d35/d3b/lf5 0 2024-04-02T04:42:45.225 INFO:tasks.workunit.client.0.smithi016.stdout:2/707: truncate d12/d35/d3b/d42/d67/d6a/fe6 291631 0 2024-04-02T04:42:45.226 INFO:tasks.workunit.client.1.smithi027.stdout:8/75: getdents d5/d8/db/dd 0 2024-04-02T04:42:45.227 INFO:tasks.workunit.client.0.smithi016.stdout:4/655: creat d5/d1d/d1e/da5/fc6 x:0 0 0 2024-04-02T04:42:45.228 INFO:tasks.workunit.client.0.smithi016.stdout:1/623: creat d14/d19/d50/fc8 x:0 0 0 2024-04-02T04:42:45.229 INFO:tasks.workunit.client.0.smithi016.stdout:4/656: dread d5/d1d/d5f/d65/d6f/f93 [0,4194304] 0 2024-04-02T04:42:45.229 INFO:tasks.workunit.client.0.smithi016.stdout:4/657: readlink d5/d1d/d6c/l68 0 2024-04-02T04:42:45.234 INFO:tasks.workunit.client.0.smithi016.stdout:6/693: write d0/f73 [700282,115827] 0 2024-04-02T04:42:45.234 INFO:tasks.workunit.client.0.smithi016.stdout:9/712: creat d0/d9/db/d54/fdc x:0 0 0 2024-04-02T04:42:45.236 INFO:tasks.workunit.client.0.smithi016.stdout:9/713: write d0/d4a/d75/d95/fca [2207725,2134] 0 2024-04-02T04:42:45.239 INFO:tasks.workunit.client.1.smithi027.stdout:0/147: dwrite d4/f14 [0,4194304] 0 2024-04-02T04:42:45.240 INFO:tasks.workunit.client.0.smithi016.stdout:7/622: dwrite d0/d46/d16/d83/f93 [0,4194304] 0 2024-04-02T04:42:45.242 INFO:tasks.workunit.client.0.smithi016.stdout:7/623: dread - d0/d46/d52/f80 zero size 2024-04-02T04:42:45.243 INFO:tasks.workunit.client.0.smithi016.stdout:7/624: write d0/d46/d16/fc5 [505392,98775] 0 2024-04-02T04:42:45.244 INFO:tasks.workunit.client.0.smithi016.stdout:7/625: fdatasync d0/d46/d16/d83/fa8 0 2024-04-02T04:42:45.244 INFO:tasks.workunit.client.1.smithi027.stdout:0/148: dread d4/f1d [0,4194304] 0 2024-04-02T04:42:45.246 INFO:tasks.workunit.client.0.smithi016.stdout:1/624: rename d14/da9/c98 to d14/d19/d1d/d45/d4f/cc9 0 2024-04-02T04:42:45.250 INFO:tasks.workunit.client.0.smithi016.stdout:7/626: dwrite d0/d46/d37/d3a/d4c/fa0 [0,4194304] 0 2024-04-02T04:42:45.251 INFO:tasks.workunit.client.1.smithi027.stdout:2/121: link d1/db/l1a d1/dd/d17/d12/l28 0 2024-04-02T04:42:45.256 INFO:tasks.workunit.client.1.smithi027.stdout:7/110: rename d2/d7/da/dc/d1b to d2/d1d 0 2024-04-02T04:42:45.257 INFO:tasks.workunit.client.1.smithi027.stdout:9/94: dwrite fa [0,4194304] 0 2024-04-02T04:42:45.258 INFO:tasks.workunit.client.1.smithi027.stdout:9/95: chown d1a/d1b/f1c 1424865 1 2024-04-02T04:42:45.261 INFO:tasks.workunit.client.0.smithi016.stdout:9/714: fsync d0/d9/d4d/f82 0 2024-04-02T04:42:45.263 INFO:tasks.workunit.client.1.smithi027.stdout:6/115: getdents d13/d1b 0 2024-04-02T04:42:45.264 INFO:tasks.workunit.client.1.smithi027.stdout:5/183: truncate d0/df/d11/f1f 1931986 0 2024-04-02T04:42:45.264 INFO:tasks.workunit.client.1.smithi027.stdout:6/116: fsync fc 0 2024-04-02T04:42:45.266 INFO:tasks.workunit.client.1.smithi027.stdout:0/149: mkdir d4/d16/d2b/d2d 0 2024-04-02T04:42:45.267 INFO:tasks.workunit.client.1.smithi027.stdout:6/117: fdatasync d13/d1b/f23 0 2024-04-02T04:42:45.268 INFO:tasks.workunit.client.1.smithi027.stdout:6/118: chown f12 63 1 2024-04-02T04:42:45.269 INFO:tasks.workunit.client.1.smithi027.stdout:9/96: dwrite f15 [0,4194304] 0 2024-04-02T04:42:45.269 INFO:tasks.workunit.client.0.smithi016.stdout:8/617: write d1/d6/d36/d3f/d40/d49/fab [286918,103769] 0 2024-04-02T04:42:45.271 INFO:tasks.workunit.client.1.smithi027.stdout:2/122: mkdir d1/dd/d17/d29 0 2024-04-02T04:42:45.272 INFO:tasks.workunit.client.1.smithi027.stdout:2/123: write d1/dd/d23/f24 [1663774,12361] 0 2024-04-02T04:42:45.278 INFO:tasks.workunit.client.1.smithi027.stdout:7/111: creat d2/d7/da/f1e x:0 0 0 2024-04-02T04:42:45.278 INFO:tasks.workunit.client.1.smithi027.stdout:8/76: write f4 [1090492,114357] 0 2024-04-02T04:42:45.278 INFO:tasks.workunit.client.1.smithi027.stdout:7/112: write d2/d7/f18 [3932252,46684] 0 2024-04-02T04:42:45.279 INFO:tasks.workunit.client.0.smithi016.stdout:5/604: sync 2024-04-02T04:42:45.280 INFO:tasks.workunit.client.1.smithi027.stdout:8/77: dread d5/d7/f14 [0,4194304] 0 2024-04-02T04:42:45.282 INFO:tasks.workunit.client.0.smithi016.stdout:4/658: link d5/d1d/d1e/da5/faa d5/d1d/d5f/d65/fc7 0 2024-04-02T04:42:45.287 INFO:tasks.workunit.client.0.smithi016.stdout:6/694: symlink d0/d9/d16/d56/ld9 0 2024-04-02T04:42:45.289 INFO:tasks.workunit.client.0.smithi016.stdout:4/659: dwrite d5/d17/d18/d6d/da4/fad [0,4194304] 0 2024-04-02T04:42:45.289 INFO:tasks.workunit.client.1.smithi027.stdout:5/184: unlink d0/f24 0 2024-04-02T04:42:45.294 INFO:tasks.workunit.client.0.smithi016.stdout:2/708: rmdir d12/daf 0 2024-04-02T04:42:45.295 INFO:tasks.workunit.client.1.smithi027.stdout:9/97: mkdir d1a/d1b/d20 0 2024-04-02T04:42:45.295 INFO:tasks.workunit.client.0.smithi016.stdout:1/625: write d14/d19/d1d/f30 [2517143,54984] 0 2024-04-02T04:42:45.296 INFO:tasks.workunit.client.0.smithi016.stdout:6/695: dwrite d0/f9b [0,4194304] 0 2024-04-02T04:42:45.297 INFO:tasks.workunit.client.0.smithi016.stdout:2/709: read f8 [1096066,15263] 0 2024-04-02T04:42:45.297 INFO:tasks.workunit.client.0.smithi016.stdout:1/626: truncate d14/f83 776422 0 2024-04-02T04:42:45.298 INFO:tasks.workunit.client.0.smithi016.stdout:5/605: mkdir d3/d40/db6/de0 0 2024-04-02T04:42:45.298 INFO:tasks.workunit.client.0.smithi016.stdout:6/696: stat d0/d9/d16/d56/f2c 0 2024-04-02T04:42:45.300 INFO:tasks.workunit.client.0.smithi016.stdout:7/627: unlink d0/d46/d37/d3a/d84/d8a/l91 0 2024-04-02T04:42:45.301 INFO:tasks.workunit.client.0.smithi016.stdout:7/628: stat d0/d46/d37/d3a/da9 0 2024-04-02T04:42:45.302 INFO:tasks.workunit.client.1.smithi027.stdout:9/98: dwrite f5 [8388608,4194304] 0 2024-04-02T04:42:45.309 INFO:tasks.workunit.client.0.smithi016.stdout:2/710: dread d12/d35/d3f/d56/dab/f3c [0,4194304] 0 2024-04-02T04:42:45.309 INFO:tasks.workunit.client.1.smithi027.stdout:7/113: stat d2/d7/da/dc/d16/l1a 0 2024-04-02T04:42:45.312 INFO:tasks.workunit.client.1.smithi027.stdout:8/78: creat d5/d8/f17 x:0 0 0 2024-04-02T04:42:45.318 INFO:tasks.workunit.client.0.smithi016.stdout:9/715: dwrite d0/d4e/d50/f85 [0,4194304] 0 2024-04-02T04:42:45.319 INFO:tasks.workunit.client.1.smithi027.stdout:0/150: truncate d4/d16/d1b/f20 2095914 0 2024-04-02T04:42:45.319 INFO:tasks.workunit.client.0.smithi016.stdout:9/716: chown d0/d4e/d50/d63/d6e/f44 597 1 2024-04-02T04:42:45.319 INFO:tasks.workunit.client.0.smithi016.stdout:9/717: chown d0/d4e/d50/d67/lc4 2 1 2024-04-02T04:42:45.320 INFO:tasks.workunit.client.1.smithi027.stdout:6/119: symlink d13/d1b/d24/d25/d27/l2c 0 2024-04-02T04:42:45.322 INFO:tasks.workunit.client.0.smithi016.stdout:6/697: fsync d0/d9/df/f1b 0 2024-04-02T04:42:45.325 INFO:tasks.workunit.client.1.smithi027.stdout:2/124: mknod d1/c2a 0 2024-04-02T04:42:45.326 INFO:tasks.workunit.client.0.smithi016.stdout:4/660: dwrite d5/d17/d76/f7d [0,4194304] 0 2024-04-02T04:42:45.326 INFO:tasks.workunit.client.0.smithi016.stdout:7/629: rmdir d0/d46/d52 39 2024-04-02T04:42:45.329 INFO:tasks.workunit.client.1.smithi027.stdout:3/71: sync 2024-04-02T04:42:45.329 INFO:tasks.workunit.client.1.smithi027.stdout:4/111: sync 2024-04-02T04:42:45.330 INFO:tasks.workunit.client.1.smithi027.stdout:3/72: chown d1/f7 14180917 1 2024-04-02T04:42:45.333 INFO:tasks.workunit.client.1.smithi027.stdout:7/114: mkdir d2/d7/da/dc/d1f 0 2024-04-02T04:42:45.338 INFO:tasks.workunit.client.1.smithi027.stdout:0/151: creat d4/d21/f2e x:0 0 0 2024-04-02T04:42:45.339 INFO:tasks.workunit.client.1.smithi027.stdout:2/125: read d1/dd/d17/fc [945037,51670] 0 2024-04-02T04:42:45.343 INFO:tasks.workunit.client.1.smithi027.stdout:9/99: rename l8 to d1a/d1b/l21 0 2024-04-02T04:42:45.344 INFO:tasks.workunit.client.1.smithi027.stdout:6/120: rename d13/d1b/d24/d25 to d13/d1b/d24/d25/d27/d2d 22 2024-04-02T04:42:45.346 INFO:tasks.workunit.client.1.smithi027.stdout:6/121: read d13/d1b/f1f [1049207,112504] 0 2024-04-02T04:42:45.346 INFO:tasks.workunit.client.1.smithi027.stdout:4/112: rename l1c to l20 0 2024-04-02T04:42:45.347 INFO:tasks.workunit.client.1.smithi027.stdout:4/113: chown fc 76238 1 2024-04-02T04:42:45.348 INFO:tasks.workunit.client.1.smithi027.stdout:3/73: creat d1/de/df/f17 x:0 0 0 2024-04-02T04:42:45.349 INFO:tasks.workunit.client.0.smithi016.stdout:5/606: dread d3/d40/d4f/d6b/f47 [0,4194304] 0 2024-04-02T04:42:45.351 INFO:tasks.workunit.client.1.smithi027.stdout:7/115: creat d2/d1d/f20 x:0 0 0 2024-04-02T04:42:45.353 INFO:tasks.workunit.client.1.smithi027.stdout:5/185: creat d0/f47 x:0 0 0 2024-04-02T04:42:45.353 INFO:tasks.workunit.client.1.smithi027.stdout:7/116: dread - d2/d7/da/f1e zero size 2024-04-02T04:42:45.353 INFO:tasks.workunit.client.1.smithi027.stdout:0/152: creat d4/d16/d2b/f2f x:0 0 0 2024-04-02T04:42:45.354 INFO:tasks.workunit.client.1.smithi027.stdout:8/79: dwrite d5/d8/db/dd/f10 [0,4194304] 0 2024-04-02T04:42:45.356 INFO:tasks.workunit.client.1.smithi027.stdout:9/100: unlink f5 0 2024-04-02T04:42:45.358 INFO:tasks.workunit.client.1.smithi027.stdout:9/101: write d1a/d1b/f1c [997683,123340] 0 2024-04-02T04:42:45.358 INFO:tasks.workunit.client.1.smithi027.stdout:7/117: dread f1 [0,4194304] 0 2024-04-02T04:42:45.360 INFO:tasks.workunit.client.1.smithi027.stdout:2/126: rename d1/dd/d23 to d1/dd/d17/d2b 0 2024-04-02T04:42:45.365 INFO:tasks.workunit.client.1.smithi027.stdout:6/122: mknod d13/d1b/c2e 0 2024-04-02T04:42:45.367 INFO:tasks.workunit.client.1.smithi027.stdout:9/102: dwrite d1a/d1b/f1c [0,4194304] 0 2024-04-02T04:42:45.367 INFO:tasks.workunit.client.1.smithi027.stdout:4/114: write fc [973046,57958] 0 2024-04-02T04:42:45.369 INFO:tasks.workunit.client.1.smithi027.stdout:4/115: write f9 [1071863,40303] 0 2024-04-02T04:42:45.380 INFO:tasks.workunit.client.1.smithi027.stdout:1/90: sync 2024-04-02T04:42:45.380 INFO:tasks.workunit.client.1.smithi027.stdout:8/80: mknod d5/d7/c18 0 2024-04-02T04:42:45.386 INFO:tasks.workunit.client.1.smithi027.stdout:0/153: rename d4/c22 to d4/d16/d2b/d2d/c30 0 2024-04-02T04:42:45.388 INFO:tasks.workunit.client.1.smithi027.stdout:2/127: creat d1/dd/d17/f2c x:0 0 0 2024-04-02T04:42:45.391 INFO:tasks.workunit.client.1.smithi027.stdout:6/123: symlink d13/d1b/l2f 0 2024-04-02T04:42:45.396 INFO:tasks.workunit.client.1.smithi027.stdout:5/186: write d0/f33 [50506,5862] 0 2024-04-02T04:42:45.397 INFO:tasks.workunit.client.1.smithi027.stdout:4/116: creat f21 x:0 0 0 2024-04-02T04:42:45.397 INFO:tasks.workunit.client.1.smithi027.stdout:4/117: chown f11 199276570 1 2024-04-02T04:42:45.398 INFO:tasks.workunit.client.1.smithi027.stdout:4/118: readlink l10 0 2024-04-02T04:42:45.398 INFO:tasks.workunit.client.1.smithi027.stdout:4/119: read - f21 zero size 2024-04-02T04:42:45.400 INFO:tasks.workunit.client.0.smithi016.stdout:0/618: sync 2024-04-02T04:42:45.400 INFO:tasks.workunit.client.0.smithi016.stdout:3/739: sync 2024-04-02T04:42:45.401 INFO:tasks.workunit.client.0.smithi016.stdout:3/740: chown d2/d15/d60/fe5 72933 1 2024-04-02T04:42:45.401 INFO:tasks.workunit.client.0.smithi016.stdout:2/711: creat d12/d35/d3b/d42/d59/d91/ff6 x:0 0 0 2024-04-02T04:42:45.404 INFO:tasks.workunit.client.1.smithi027.stdout:7/118: write d2/d7/f12 [1791688,17440] 0 2024-04-02T04:42:45.404 INFO:tasks.workunit.client.1.smithi027.stdout:1/91: readlink d0/la 0 2024-04-02T04:42:45.407 INFO:tasks.workunit.client.1.smithi027.stdout:0/154: mknod d4/d16/d1b/c31 0 2024-04-02T04:42:45.410 INFO:tasks.workunit.client.1.smithi027.stdout:2/128: mknod d1/dd/d17/c2d 0 2024-04-02T04:42:45.411 INFO:tasks.workunit.client.0.smithi016.stdout:3/741: dread d2/d2e/f8d [0,4194304] 0 2024-04-02T04:42:45.411 INFO:tasks.workunit.client.1.smithi027.stdout:6/124: rmdir d13/d1b/d24/d25 39 2024-04-02T04:42:45.412 INFO:tasks.workunit.client.0.smithi016.stdout:3/742: chown d2/d15/d16/d8a/dd1/d8b/c8e 107861683 1 2024-04-02T04:42:45.412 INFO:tasks.workunit.client.1.smithi027.stdout:0/155: dread d4/f11 [0,4194304] 0 2024-04-02T04:42:45.413 INFO:tasks.workunit.client.1.smithi027.stdout:9/103: mknod d1a/d1d/c22 0 2024-04-02T04:42:45.416 INFO:tasks.workunit.client.1.smithi027.stdout:5/187: mknod d0/df/d11/d37/c48 0 2024-04-02T04:42:45.418 INFO:tasks.workunit.client.1.smithi027.stdout:5/188: chown d0/df/d2f/d3b 299 1 2024-04-02T04:42:45.419 INFO:tasks.workunit.client.0.smithi016.stdout:7/630: mkdir d0/d46/d37/dbe/de1 0 2024-04-02T04:42:45.419 INFO:tasks.workunit.client.0.smithi016.stdout:7/631: chown d0/d46/d37/d3a/d4c/l67 7966171 1 2024-04-02T04:42:45.420 INFO:tasks.workunit.client.1.smithi027.stdout:0/156: dwrite d4/d16/d2b/f2f [0,4194304] 0 2024-04-02T04:42:45.420 INFO:tasks.workunit.client.1.smithi027.stdout:4/120: symlink l22 0 2024-04-02T04:42:45.421 INFO:tasks.workunit.client.0.smithi016.stdout:5/607: creat d3/d4c/dc4/fe1 x:0 0 0 2024-04-02T04:42:45.423 INFO:tasks.workunit.client.1.smithi027.stdout:3/74: link d1/f8 d1/f18 0 2024-04-02T04:42:45.425 INFO:tasks.workunit.client.1.smithi027.stdout:3/75: write d1/f7 [6211615,104189] 0 2024-04-02T04:42:45.428 INFO:tasks.workunit.client.0.smithi016.stdout:0/619: mkdir d0/d16/dd6 0 2024-04-02T04:42:45.428 INFO:tasks.workunit.client.1.smithi027.stdout:4/121: dwrite f12 [0,4194304] 0 2024-04-02T04:42:45.431 INFO:tasks.workunit.client.0.smithi016.stdout:6/698: dread d0/d9/df/fc4 [0,4194304] 0 2024-04-02T04:42:45.432 INFO:tasks.workunit.client.0.smithi016.stdout:2/712: creat d12/d35/d3f/d56/d66/ff7 x:0 0 0 2024-04-02T04:42:45.433 INFO:tasks.workunit.client.0.smithi016.stdout:8/618: sync 2024-04-02T04:42:45.434 INFO:tasks.workunit.client.0.smithi016.stdout:1/627: dwrite d14/f48 [0,4194304] 0 2024-04-02T04:42:45.436 INFO:tasks.workunit.client.1.smithi027.stdout:5/189: creat d0/d1a/f49 x:0 0 0 2024-04-02T04:42:45.439 INFO:tasks.workunit.client.1.smithi027.stdout:1/92: symlink d0/d1e/l20 0 2024-04-02T04:42:45.443 INFO:tasks.workunit.client.1.smithi027.stdout:7/119: link f1 d2/d7/da/dc/d1f/f21 0 2024-04-02T04:42:45.444 INFO:tasks.workunit.client.0.smithi016.stdout:9/718: creat d0/d9/d4d/fdd x:0 0 0 2024-04-02T04:42:45.445 INFO:tasks.workunit.client.0.smithi016.stdout:2/713: mkdir d12/d9f/dac/d76/da0/de7/df8 0 2024-04-02T04:42:45.446 INFO:tasks.workunit.client.0.smithi016.stdout:2/714: readlink d12/d35/d3b/d42/d8d/lce 0 2024-04-02T04:42:45.447 INFO:tasks.workunit.client.0.smithi016.stdout:1/628: mknod d14/d19/d1d/d29/d39/d46/cca 0 2024-04-02T04:42:45.448 INFO:tasks.workunit.client.0.smithi016.stdout:1/629: fsync d14/da9/f9b 0 2024-04-02T04:42:45.450 INFO:tasks.workunit.client.0.smithi016.stdout:7/632: dread d0/d46/d37/d3a/f7d [0,4194304] 0 2024-04-02T04:42:45.452 INFO:tasks.workunit.client.0.smithi016.stdout:3/743: mkdir d2/d15/d16/d8a/dd1/d6e/d6f/df9 0 2024-04-02T04:42:45.453 INFO:tasks.workunit.client.0.smithi016.stdout:4/661: creat d5/d17/d18/fc8 x:0 0 0 2024-04-02T04:42:45.454 INFO:tasks.workunit.client.0.smithi016.stdout:4/662: write d5/d1d/f21 [2814046,104774] 0 2024-04-02T04:42:45.454 INFO:tasks.workunit.client.1.smithi027.stdout:6/125: rename d13/d1b/d24/d25/d27/l2c to d13/d1b/d24/d25/d27/l30 0 2024-04-02T04:42:45.455 INFO:tasks.workunit.client.0.smithi016.stdout:4/663: write d5/d16/d88/f96 [291913,129708] 0 2024-04-02T04:42:45.458 INFO:tasks.workunit.client.0.smithi016.stdout:0/620: symlink d0/d16/d2e/d53/ld7 0 2024-04-02T04:42:45.459 INFO:tasks.workunit.client.0.smithi016.stdout:0/621: write d0/f67 [4913193,75429] 0 2024-04-02T04:42:45.460 INFO:tasks.workunit.client.1.smithi027.stdout:5/190: stat d0/df/d11/c43 0 2024-04-02T04:42:45.461 INFO:tasks.workunit.client.1.smithi027.stdout:1/93: chown d0/fd 364854 1 2024-04-02T04:42:45.465 INFO:tasks.workunit.client.0.smithi016.stdout:6/699: dread d0/d9/d16/f29 [0,4194304] 0 2024-04-02T04:42:45.465 INFO:tasks.workunit.client.0.smithi016.stdout:5/608: write d3/d4c/dd1/f8d [644549,34637] 0 2024-04-02T04:42:45.466 INFO:tasks.workunit.client.0.smithi016.stdout:9/719: dwrite d0/d4e/d50/d63/d6e/d4b/d58/d6a/fbc [4194304,4194304] 0 2024-04-02T04:42:45.466 INFO:tasks.workunit.client.1.smithi027.stdout:2/129: link d1/dd/d17/l10 d1/dd/d17/l2e 0 2024-04-02T04:42:45.467 INFO:tasks.workunit.client.0.smithi016.stdout:6/700: write d0/f3f [1295147,6165] 0 2024-04-02T04:42:45.467 INFO:tasks.workunit.client.0.smithi016.stdout:5/609: chown d3/df/d18/d26/d70 0 1 2024-04-02T04:42:45.468 INFO:tasks.workunit.client.1.smithi027.stdout:2/130: chown d1/dd/d17/c25 4146635 1 2024-04-02T04:42:45.469 INFO:tasks.workunit.client.0.smithi016.stdout:6/701: write d0/f51 [241045,129142] 0 2024-04-02T04:42:45.469 INFO:tasks.workunit.client.0.smithi016.stdout:5/610: write d3/fad [126088,83630] 0 2024-04-02T04:42:45.470 INFO:tasks.workunit.client.1.smithi027.stdout:0/157: creat d4/f32 x:0 0 0 2024-04-02T04:42:45.471 INFO:tasks.workunit.client.1.smithi027.stdout:5/191: symlink d0/df/d29/l4a 0 2024-04-02T04:42:45.472 INFO:tasks.workunit.client.1.smithi027.stdout:5/192: dread - d0/df/d11/d37/f40 zero size 2024-04-02T04:42:45.473 INFO:tasks.workunit.client.0.smithi016.stdout:3/744: readlink d2/d15/d16/d8a/lb1 0 2024-04-02T04:42:45.492 INFO:tasks.workunit.client.0.smithi016.stdout:9/720: dread d0/d4a/d75/d8d/f94 [0,4194304] 0 2024-04-02T04:42:45.493 INFO:tasks.workunit.client.0.smithi016.stdout:9/721: write d0/d4e/d50/d67/fb6 [732995,30442] 0 2024-04-02T04:42:45.493 INFO:tasks.workunit.client.0.smithi016.stdout:9/722: dread - d0/d4e/fae zero size 2024-04-02T04:42:45.504 INFO:tasks.workunit.client.1.smithi027.stdout:9/104: truncate fa 3068577 0 2024-04-02T04:42:45.505 INFO:tasks.workunit.client.1.smithi027.stdout:9/105: chown d1a/d1b/d20 3096 1 2024-04-02T04:42:45.507 INFO:tasks.workunit.client.1.smithi027.stdout:7/120: creat d2/d1c/f22 x:0 0 0 2024-04-02T04:42:45.507 INFO:tasks.workunit.client.1.smithi027.stdout:7/121: chown d2/d1c/f22 135395110 1 2024-04-02T04:42:45.508 INFO:tasks.workunit.client.0.smithi016.stdout:0/622: dread d0/d16/d24/d62/d9c/f5b [0,4194304] 0 2024-04-02T04:42:45.510 INFO:tasks.workunit.client.0.smithi016.stdout:8/619: truncate d1/d70/f84 1393033 0 2024-04-02T04:42:45.511 INFO:tasks.workunit.client.0.smithi016.stdout:2/715: mknod d12/d35/d3b/d42/cf9 0 2024-04-02T04:42:45.511 INFO:tasks.workunit.client.1.smithi027.stdout:7/122: dread d2/d7/f12 [0,4194304] 0 2024-04-02T04:42:45.511 INFO:tasks.workunit.client.1.smithi027.stdout:7/123: read - d2/d1d/f20 zero size 2024-04-02T04:42:45.515 INFO:tasks.workunit.client.1.smithi027.stdout:0/158: write d4/f9 [2262301,105569] 0 2024-04-02T04:42:45.519 INFO:tasks.workunit.client.1.smithi027.stdout:1/94: mknod d0/c21 0 2024-04-02T04:42:45.522 INFO:tasks.workunit.client.1.smithi027.stdout:3/76: getdents d1/de/df/d13 0 2024-04-02T04:42:45.525 INFO:tasks.workunit.client.1.smithi027.stdout:9/106: symlink d1a/d1b/l23 0 2024-04-02T04:42:45.526 INFO:tasks.workunit.client.1.smithi027.stdout:9/107: readlink d1a/d1b/l23 0 2024-04-02T04:42:45.526 INFO:tasks.workunit.client.0.smithi016.stdout:6/702: mknod d0/d9/d16/cda 0 2024-04-02T04:42:45.528 INFO:tasks.workunit.client.1.smithi027.stdout:2/131: dwrite d1/dd/d17/f14 [0,4194304] 0 2024-04-02T04:42:45.529 INFO:tasks.workunit.client.0.smithi016.stdout:5/611: creat d3/df/d18/d26/ddc/fe2 x:0 0 0 2024-04-02T04:42:45.529 INFO:tasks.workunit.client.0.smithi016.stdout:5/612: chown d3/d40/d4f/d6b 213711949 1 2024-04-02T04:42:45.530 INFO:tasks.workunit.client.0.smithi016.stdout:2/716: dread f9 [0,4194304] 0 2024-04-02T04:42:45.531 INFO:tasks.workunit.client.0.smithi016.stdout:2/717: stat d12/d35/c99 0 2024-04-02T04:42:45.531 INFO:tasks.workunit.client.1.smithi027.stdout:4/122: getdents . 0 2024-04-02T04:42:45.533 INFO:tasks.workunit.client.0.smithi016.stdout:3/745: chown d2/d15/d16/d8a/dd1/ca0 4446 1 2024-04-02T04:42:45.535 INFO:tasks.workunit.client.0.smithi016.stdout:3/746: dread - d2/d15/d16/d8a/dd1/d6e/d6f/db0/f9e zero size 2024-04-02T04:42:45.536 INFO:tasks.workunit.client.0.smithi016.stdout:0/623: unlink d0/d16/d24/c46 0 2024-04-02T04:42:45.538 INFO:tasks.workunit.client.0.smithi016.stdout:8/620: creat d1/d31/d67/fd1 x:0 0 0 2024-04-02T04:42:45.539 INFO:tasks.workunit.client.1.smithi027.stdout:1/95: rmdir d0/de 39 2024-04-02T04:42:45.539 INFO:tasks.workunit.client.1.smithi027.stdout:1/96: chown d0/f14 256361621 1 2024-04-02T04:42:45.540 INFO:tasks.workunit.client.0.smithi016.stdout:1/630: rmdir d14/d19/d1e/d5e/db0 0 2024-04-02T04:42:45.542 INFO:tasks.workunit.client.0.smithi016.stdout:4/664: write d5/d17/d18/d25/f6b [1032616,95833] 0 2024-04-02T04:42:45.542 INFO:tasks.workunit.client.0.smithi016.stdout:4/665: readlink d5/d1d/lbf 0 2024-04-02T04:42:45.543 INFO:tasks.workunit.client.1.smithi027.stdout:5/193: write d0/df/d11/f1f [2230711,121937] 0 2024-04-02T04:42:45.548 INFO:tasks.workunit.client.1.smithi027.stdout:9/108: creat d1a/d1d/f24 x:0 0 0 2024-04-02T04:42:45.549 INFO:tasks.workunit.client.0.smithi016.stdout:7/633: creat d0/d46/d37/dbe/dae/d9d/fe2 x:0 0 0 2024-04-02T04:42:45.550 INFO:tasks.workunit.client.0.smithi016.stdout:7/634: chown d0/d21/f57 79303 1 2024-04-02T04:42:45.550 INFO:tasks.workunit.client.1.smithi027.stdout:4/123: creat f23 x:0 0 0 2024-04-02T04:42:45.552 INFO:tasks.workunit.client.0.smithi016.stdout:0/624: write d0/d16/d24/d4e/d8d/f75 [7494939,44022] 0 2024-04-02T04:42:45.553 INFO:tasks.workunit.client.1.smithi027.stdout:0/159: symlink d4/l33 0 2024-04-02T04:42:45.555 INFO:tasks.workunit.client.0.smithi016.stdout:8/621: chown d1/d6/d36/d3f/d77/fd0 2091308 1 2024-04-02T04:42:45.556 INFO:tasks.workunit.client.1.smithi027.stdout:1/97: fsync d0/de/f1b 0 2024-04-02T04:42:45.557 INFO:tasks.workunit.client.0.smithi016.stdout:5/613: getdents d3/df/d86 0 2024-04-02T04:42:45.557 INFO:tasks.workunit.client.1.smithi027.stdout:5/194: mknod d0/d1d/c4b 0 2024-04-02T04:42:45.558 INFO:tasks.workunit.client.1.smithi027.stdout:5/195: readlink d0/d1a/l27 0 2024-04-02T04:42:45.559 INFO:tasks.workunit.client.1.smithi027.stdout:5/196: fdatasync d0/df/d11/d13/f1e 0 2024-04-02T04:42:45.559 INFO:tasks.workunit.client.0.smithi016.stdout:7/635: stat d0/d46/d52/d82/fd3 0 2024-04-02T04:42:45.560 INFO:tasks.workunit.client.0.smithi016.stdout:0/625: creat d0/d16/d24/d4e/d8d/d71/fd8 x:0 0 0 2024-04-02T04:42:45.563 INFO:tasks.workunit.client.0.smithi016.stdout:8/622: read d1/d6/d36/f37 [2571411,86183] 0 2024-04-02T04:42:45.564 INFO:tasks.workunit.client.0.smithi016.stdout:8/623: readlink d1/d6/l9 0 2024-04-02T04:42:45.564 INFO:tasks.workunit.client.0.smithi016.stdout:7/636: dread d0/d46/d37/dbe/f7b [0,4194304] 0 2024-04-02T04:42:45.564 INFO:tasks.workunit.client.1.smithi027.stdout:9/109: rename d1a/d1b/l23 to d1a/d1b/d20/l25 0 2024-04-02T04:42:45.565 INFO:tasks.workunit.client.0.smithi016.stdout:7/637: write d0/d46/d52/f61 [1692117,57259] 0 2024-04-02T04:42:45.566 INFO:tasks.workunit.client.0.smithi016.stdout:5/614: rmdir d3/d40 39 2024-04-02T04:42:45.567 INFO:tasks.workunit.client.1.smithi027.stdout:4/124: mkdir d24 0 2024-04-02T04:42:45.568 INFO:tasks.workunit.client.0.smithi016.stdout:5/615: fsync d3/f28 0 2024-04-02T04:42:45.568 INFO:tasks.workunit.client.1.smithi027.stdout:4/125: fsync f1d 0 2024-04-02T04:42:45.568 INFO:tasks.workunit.client.1.smithi027.stdout:4/126: chown f14 27 1 2024-04-02T04:42:45.569 INFO:tasks.workunit.client.1.smithi027.stdout:4/127: chown c1a 126 1 2024-04-02T04:42:45.569 INFO:tasks.workunit.client.0.smithi016.stdout:4/666: dread d5/d16/f55 [0,4194304] 0 2024-04-02T04:42:45.570 INFO:tasks.workunit.client.1.smithi027.stdout:0/160: symlink d4/d16/d2b/d2d/l34 0 2024-04-02T04:42:45.572 INFO:tasks.workunit.client.0.smithi016.stdout:3/747: rename d2/d15/d16/d8a/dd1/d6e/d6f/db0/lad to d2/d2e/d36/d48/da2/lfa 0 2024-04-02T04:42:45.574 INFO:tasks.workunit.client.1.smithi027.stdout:2/132: truncate d1/dd/d17/fc 7465000 0 2024-04-02T04:42:45.575 INFO:tasks.workunit.client.0.smithi016.stdout:0/626: unlink d0/d16/d2e/d53/fc9 0 2024-04-02T04:42:45.575 INFO:tasks.workunit.client.0.smithi016.stdout:8/624: dwrite d1/d31/d67/fd1 [0,4194304] 0 2024-04-02T04:42:45.575 INFO:tasks.workunit.client.1.smithi027.stdout:2/133: chown d1/db/c15 3985602 1 2024-04-02T04:42:45.575 INFO:tasks.workunit.client.1.smithi027.stdout:1/98: symlink d0/d1e/l22 0 2024-04-02T04:42:45.577 INFO:tasks.workunit.client.0.smithi016.stdout:2/718: dwrite d12/d35/d3f/fb6 [0,4194304] 0 2024-04-02T04:42:45.578 INFO:tasks.workunit.client.1.smithi027.stdout:5/197: symlink d0/df/d2f/d3b/l4c 0 2024-04-02T04:42:45.578 INFO:tasks.workunit.client.1.smithi027.stdout:5/198: fdatasync d0/df/d2f/f38 0 2024-04-02T04:42:45.583 INFO:tasks.workunit.client.1.smithi027.stdout:7/124: getdents d2/d7 0 2024-04-02T04:42:45.584 INFO:tasks.workunit.client.1.smithi027.stdout:7/125: fdatasync d2/d1c/f22 0 2024-04-02T04:42:45.585 INFO:tasks.workunit.client.0.smithi016.stdout:2/719: dwrite d12/d35/d3f/d2b/fc1 [0,4194304] 0 2024-04-02T04:42:45.585 INFO:tasks.workunit.client.1.smithi027.stdout:4/128: rename f1b to d24/f25 0 2024-04-02T04:42:45.586 INFO:tasks.workunit.client.1.smithi027.stdout:9/110: dread fa [0,4194304] 0 2024-04-02T04:42:45.586 INFO:tasks.workunit.client.1.smithi027.stdout:9/111: fsync f12 0 2024-04-02T04:42:45.586 INFO:tasks.workunit.client.1.smithi027.stdout:4/129: write f12 [809582,79585] 0 2024-04-02T04:42:45.588 INFO:tasks.workunit.client.1.smithi027.stdout:4/130: chown f15 19 1 2024-04-02T04:42:45.588 INFO:tasks.workunit.client.0.smithi016.stdout:5/616: truncate d3/df/d18/f42 4593199 0 2024-04-02T04:42:45.589 INFO:tasks.workunit.client.1.smithi027.stdout:3/77: getdents d1 0 2024-04-02T04:42:45.594 INFO:tasks.workunit.client.1.smithi027.stdout:2/134: mkdir d1/dd/d17/d2b/d2f 0 2024-04-02T04:42:45.597 INFO:tasks.workunit.client.0.smithi016.stdout:1/631: rename d14/d41 to d14/d19/d1d/d29/d39/d46/dcb 0 2024-04-02T04:42:45.599 INFO:tasks.workunit.client.1.smithi027.stdout:1/99: creat d0/de/f23 x:0 0 0 2024-04-02T04:42:45.601 INFO:tasks.workunit.client.1.smithi027.stdout:5/199: symlink d0/d5/l4d 0 2024-04-02T04:42:45.602 INFO:tasks.workunit.client.0.smithi016.stdout:3/748: mkdir d2/d2e/d36/d66/d90/dfb 0 2024-04-02T04:42:45.603 INFO:tasks.workunit.client.0.smithi016.stdout:7/638: write d0/d46/d16/d49/fa [3266061,34904] 0 2024-04-02T04:42:45.606 INFO:tasks.workunit.client.1.smithi027.stdout:8/81: sync 2024-04-02T04:42:45.606 INFO:tasks.workunit.client.1.smithi027.stdout:6/126: sync 2024-04-02T04:42:45.607 INFO:tasks.workunit.client.0.smithi016.stdout:8/625: symlink d1/d70/d71/d74/ld2 0 2024-04-02T04:42:45.607 INFO:tasks.workunit.client.0.smithi016.stdout:4/667: write d5/d8/ff [3067540,13171] 0 2024-04-02T04:42:45.608 INFO:tasks.workunit.client.1.smithi027.stdout:7/126: mknod d2/d1d/c23 0 2024-04-02T04:42:45.612 INFO:tasks.workunit.client.0.smithi016.stdout:2/720: mkdir d12/d9f/dac/d4f/dfa 0 2024-04-02T04:42:45.613 INFO:tasks.workunit.client.1.smithi027.stdout:9/112: creat d1a/f26 x:0 0 0 2024-04-02T04:42:45.614 INFO:tasks.workunit.client.1.smithi027.stdout:0/161: truncate d4/d16/d1b/f20 1869440 0 2024-04-02T04:42:45.614 INFO:tasks.workunit.client.1.smithi027.stdout:0/162: readlink d4/d16/d2b/l26 0 2024-04-02T04:42:45.616 INFO:tasks.workunit.client.0.smithi016.stdout:5/617: unlink d3/d4c/fa9 0 2024-04-02T04:42:45.617 INFO:tasks.workunit.client.0.smithi016.stdout:2/721: dwrite d12/d35/d3f/f4b [0,4194304] 0 2024-04-02T04:42:45.617 INFO:tasks.workunit.client.1.smithi027.stdout:4/131: write d24/f25 [326248,67926] 0 2024-04-02T04:42:45.618 INFO:tasks.workunit.client.0.smithi016.stdout:2/722: dread - d12/f96 zero size 2024-04-02T04:42:45.618 INFO:tasks.workunit.client.0.smithi016.stdout:1/632: mknod d14/d19/d1d/d45/d4f/ccc 0 2024-04-02T04:42:45.619 INFO:tasks.workunit.client.1.smithi027.stdout:3/78: readlink d1/de/l12 0 2024-04-02T04:42:45.619 INFO:tasks.workunit.client.0.smithi016.stdout:1/633: truncate d14/f83 1741316 0 2024-04-02T04:42:45.623 INFO:tasks.workunit.client.1.smithi027.stdout:4/132: dread f9 [0,4194304] 0 2024-04-02T04:42:45.624 INFO:tasks.workunit.client.1.smithi027.stdout:1/100: rmdir d0 39 2024-04-02T04:42:45.625 INFO:tasks.workunit.client.0.smithi016.stdout:7/639: mkdir d0/d46/da2/dc8/dda/de3 0 2024-04-02T04:42:45.625 INFO:tasks.workunit.client.0.smithi016.stdout:7/640: write d0/d46/f9a [914410,123170] 0 2024-04-02T04:42:45.628 INFO:tasks.workunit.client.1.smithi027.stdout:6/127: mknod d13/d1b/c31 0 2024-04-02T04:42:45.630 INFO:tasks.workunit.client.1.smithi027.stdout:7/127: rename d2/d1d/c23 to d2/d7/c24 0 2024-04-02T04:42:45.632 INFO:tasks.workunit.client.1.smithi027.stdout:9/113: creat d1a/d1d/f27 x:0 0 0 2024-04-02T04:42:45.633 INFO:tasks.workunit.client.0.smithi016.stdout:5/618: chown d3/df/d18/d26/d70/c97 7367644 1 2024-04-02T04:42:45.635 INFO:tasks.workunit.client.1.smithi027.stdout:3/79: truncate d1/f18 3622882 0 2024-04-02T04:42:45.636 INFO:tasks.workunit.client.0.smithi016.stdout:3/749: link d2/d15/ff7 d2/d2e/dee/ffc 0 2024-04-02T04:42:45.638 INFO:tasks.workunit.client.1.smithi027.stdout:4/133: creat d24/f26 x:0 0 0 2024-04-02T04:42:45.640 INFO:tasks.workunit.client.1.smithi027.stdout:1/101: fdatasync d0/de/f1f 0 2024-04-02T04:42:45.641 INFO:tasks.workunit.client.1.smithi027.stdout:1/102: read - d0/de/f23 zero size 2024-04-02T04:42:45.641 INFO:tasks.workunit.client.0.smithi016.stdout:4/668: mknod d5/d17/cc9 0 2024-04-02T04:42:45.648 INFO:tasks.workunit.client.1.smithi027.stdout:6/128: mknod d13/d1b/c32 0 2024-04-02T04:42:45.650 INFO:tasks.workunit.client.0.smithi016.stdout:2/723: mknod d12/d35/d3f/cfb 0 2024-04-02T04:42:45.651 INFO:tasks.workunit.client.1.smithi027.stdout:7/128: rename d2/d7/da/dc/d1f/f21 to d2/d1d/f25 0 2024-04-02T04:42:45.652 INFO:tasks.workunit.client.0.smithi016.stdout:9/723: sync 2024-04-02T04:42:45.652 INFO:tasks.workunit.client.0.smithi016.stdout:6/703: sync 2024-04-02T04:42:45.652 INFO:tasks.workunit.client.0.smithi016.stdout:5/619: symlink d3/df/d86/le3 0 2024-04-02T04:42:45.652 INFO:tasks.workunit.client.0.smithi016.stdout:9/724: readlink d0/d3/l2f 0 2024-04-02T04:42:45.654 INFO:tasks.workunit.client.1.smithi027.stdout:0/163: symlink d4/l35 0 2024-04-02T04:42:45.654 INFO:tasks.workunit.client.1.smithi027.stdout:0/164: chown d4/f32 0 1 2024-04-02T04:42:45.657 INFO:tasks.workunit.client.0.smithi016.stdout:0/627: getdents d0/d16/d24/d4e/d8d 0 2024-04-02T04:42:45.658 INFO:tasks.workunit.client.1.smithi027.stdout:3/80: creat d1/de/df/f19 x:0 0 0 2024-04-02T04:42:45.659 INFO:tasks.workunit.client.0.smithi016.stdout:1/634: dwrite d14/d19/d1d/d29/f51 [4194304,4194304] 0 2024-04-02T04:42:45.664 INFO:tasks.workunit.client.1.smithi027.stdout:4/134: write f9 [2092150,3150] 0 2024-04-02T04:42:45.667 INFO:tasks.workunit.client.0.smithi016.stdout:8/626: rename d1/d70/d71/f94 to d1/d6/d36/d3f/d40/fd3 0 2024-04-02T04:42:45.669 INFO:tasks.workunit.client.0.smithi016.stdout:2/724: mkdir d12/d9f/dac/d76/da0/de7/dfc 0 2024-04-02T04:42:45.673 INFO:tasks.workunit.client.0.smithi016.stdout:9/725: mkdir d0/d3/d97/dde 0 2024-04-02T04:42:45.674 INFO:tasks.workunit.client.1.smithi027.stdout:6/129: rmdir d13 39 2024-04-02T04:42:45.678 INFO:tasks.workunit.client.0.smithi016.stdout:0/628: rmdir d0/d16/d2e/d53/da4 39 2024-04-02T04:42:45.678 INFO:tasks.workunit.client.0.smithi016.stdout:9/726: dwrite d0/d4e/d50/d63/d6e/f44 [0,4194304] 0 2024-04-02T04:42:45.679 INFO:tasks.workunit.client.1.smithi027.stdout:7/129: rename d2/d7/da/l19 to d2/d7/da/dc/d1f/l26 0 2024-04-02T04:42:45.683 INFO:tasks.workunit.client.0.smithi016.stdout:7/641: link d0/d46/d16/d49/c4f d0/d46/d16/d49/ce4 0 2024-04-02T04:42:45.685 INFO:tasks.workunit.client.1.smithi027.stdout:3/81: mkdir d1/d1a 0 2024-04-02T04:42:45.687 INFO:tasks.workunit.client.0.smithi016.stdout:7/642: dread d0/d46/d37/dbe/f7b [0,4194304] 0 2024-04-02T04:42:45.690 INFO:tasks.workunit.client.0.smithi016.stdout:4/669: rename d5/d1d/d69 to d5/d17/d76/dca 0 2024-04-02T04:42:45.690 INFO:tasks.workunit.client.1.smithi027.stdout:2/135: dread d1/dd/d17/f14 [4194304,4194304] 0 2024-04-02T04:42:45.691 INFO:tasks.workunit.client.0.smithi016.stdout:2/725: chown d12/d9f/dac/l8b 11 1 2024-04-02T04:42:45.692 INFO:tasks.workunit.client.1.smithi027.stdout:4/135: creat d24/f27 x:0 0 0 2024-04-02T04:42:45.696 INFO:tasks.workunit.client.1.smithi027.stdout:5/200: getdents d0 0 2024-04-02T04:42:45.697 INFO:tasks.workunit.client.1.smithi027.stdout:8/82: getdents d5/d7 0 2024-04-02T04:42:45.698 INFO:tasks.workunit.client.1.smithi027.stdout:4/136: dwrite d24/f26 [0,4194304] 0 2024-04-02T04:42:45.699 INFO:tasks.workunit.client.0.smithi016.stdout:0/629: readlink d0/d4f/lb7 0 2024-04-02T04:42:45.700 INFO:tasks.workunit.client.1.smithi027.stdout:4/137: write fc [75077,104616] 0 2024-04-02T04:42:45.702 INFO:tasks.workunit.client.0.smithi016.stdout:9/727: write d0/d33/f36 [7399689,104570] 0 2024-04-02T04:42:45.703 INFO:tasks.workunit.client.1.smithi027.stdout:6/130: chown d13/d1b/f1e 20 1 2024-04-02T04:42:45.707 INFO:tasks.workunit.client.0.smithi016.stdout:7/643: rmdir d0/d46/d52 39 2024-04-02T04:42:45.708 INFO:tasks.workunit.client.0.smithi016.stdout:7/644: dread - d0/d46/d16/d49/fca zero size 2024-04-02T04:42:45.708 INFO:tasks.workunit.client.1.smithi027.stdout:9/114: link l16 d1a/d1b/d20/l28 0 2024-04-02T04:42:45.708 INFO:tasks.workunit.client.1.smithi027.stdout:9/115: dread - d1a/f26 zero size 2024-04-02T04:42:45.709 INFO:tasks.workunit.client.0.smithi016.stdout:7/645: truncate d0/d46/d37/d47/f6b 1865431 0 2024-04-02T04:42:45.709 INFO:tasks.workunit.client.0.smithi016.stdout:6/704: rename d0/d9/d16/d46/dbe/fd5 to d0/d9/d16/d46/d5e/fdb 0 2024-04-02T04:42:45.712 INFO:tasks.workunit.client.1.smithi027.stdout:3/82: creat d1/de/df/d13/f1b x:0 0 0 2024-04-02T04:42:45.712 INFO:tasks.workunit.client.0.smithi016.stdout:8/627: symlink d1/d6/d36/ld4 0 2024-04-02T04:42:45.713 INFO:tasks.workunit.client.0.smithi016.stdout:8/628: read - d1/d6/d36/d3f/d40/d45/fb9 zero size 2024-04-02T04:42:45.714 INFO:tasks.workunit.client.1.smithi027.stdout:1/103: link d0/c21 d0/d1e/c24 0 2024-04-02T04:42:45.714 INFO:tasks.workunit.client.1.smithi027.stdout:1/104: dread - d0/de/f23 zero size 2024-04-02T04:42:45.715 INFO:tasks.workunit.client.0.smithi016.stdout:8/629: write d1/d31/d67/fc8 [487314,48000] 0 2024-04-02T04:42:45.716 INFO:tasks.workunit.client.0.smithi016.stdout:8/630: write d1/d6/d36/d3f/d40/d49/fab [986375,65583] 0 2024-04-02T04:42:45.716 INFO:tasks.workunit.client.0.smithi016.stdout:4/670: read d5/d1d/d6c/f56 [1427410,64323] 0 2024-04-02T04:42:45.718 INFO:tasks.workunit.client.1.smithi027.stdout:5/201: creat d0/df/d11/d37/f4e x:0 0 0 2024-04-02T04:42:45.722 INFO:tasks.workunit.client.1.smithi027.stdout:8/83: truncate d5/d7/f14 545599 0 2024-04-02T04:42:45.722 INFO:tasks.workunit.client.1.smithi027.stdout:8/84: chown d5 8687 1 2024-04-02T04:42:45.724 INFO:tasks.workunit.client.0.smithi016.stdout:8/631: dwrite d1/d6/d36/d3f/d40/d49/f44 [0,4194304] 0 2024-04-02T04:42:45.724 INFO:tasks.workunit.client.0.smithi016.stdout:9/728: creat d0/d9/db/dc9/fdf x:0 0 0 2024-04-02T04:42:45.724 INFO:tasks.workunit.client.1.smithi027.stdout:4/138: creat d24/f28 x:0 0 0 2024-04-02T04:42:45.725 INFO:tasks.workunit.client.0.smithi016.stdout:0/630: dwrite d0/d16/d2e/d36/d9f/fa0 [4194304,4194304] 0 2024-04-02T04:42:45.726 INFO:tasks.workunit.client.0.smithi016.stdout:9/729: write d0/d4e/d50/d63/d6e/f44 [4282328,82997] 0 2024-04-02T04:42:45.726 INFO:tasks.workunit.client.0.smithi016.stdout:1/635: link c13 d14/d19/d1e/d6d/ccd 0 2024-04-02T04:42:45.728 INFO:tasks.workunit.client.1.smithi027.stdout:7/130: creat d2/d17/f27 x:0 0 0 2024-04-02T04:42:45.728 INFO:tasks.workunit.client.0.smithi016.stdout:7/646: symlink d0/d46/dbf/le5 0 2024-04-02T04:42:45.729 INFO:tasks.workunit.client.1.smithi027.stdout:9/116: mkdir d1a/d1d/d29 0 2024-04-02T04:42:45.730 INFO:tasks.workunit.client.0.smithi016.stdout:7/647: stat d0/d46/d37/d3a/d84/d8a 0 2024-04-02T04:42:45.730 INFO:tasks.workunit.client.1.smithi027.stdout:0/165: link d4/d16/d2b/lf d4/d16/d1b/l36 0 2024-04-02T04:42:45.731 INFO:tasks.workunit.client.1.smithi027.stdout:0/166: stat d4/c8 0 2024-04-02T04:42:45.733 INFO:tasks.workunit.client.0.smithi016.stdout:5/620: link d3/df/d18/d26/d48/cc8 d3/df/ce4 0 2024-04-02T04:42:45.735 INFO:tasks.workunit.client.1.smithi027.stdout:1/105: mkdir d0/d1e/d25 0 2024-04-02T04:42:45.736 INFO:tasks.workunit.client.0.smithi016.stdout:3/750: link d2/d15/d70/f79 d2/d15/ffd 0 2024-04-02T04:42:45.737 INFO:tasks.workunit.client.0.smithi016.stdout:3/751: stat d2/d15/d16/d8a/dd1/d6e/d6f/d78 0 2024-04-02T04:42:45.738 INFO:tasks.workunit.client.0.smithi016.stdout:3/752: stat d2/d2e/dee 0 2024-04-02T04:42:45.738 INFO:tasks.workunit.client.1.smithi027.stdout:5/202: unlink d0/df/d11/d13/c15 0 2024-04-02T04:42:45.740 INFO:tasks.workunit.client.1.smithi027.stdout:8/85: mknod d5/d7/c19 0 2024-04-02T04:42:45.741 INFO:tasks.workunit.client.1.smithi027.stdout:6/131: symlink d13/l33 0 2024-04-02T04:42:45.743 INFO:tasks.workunit.client.1.smithi027.stdout:7/131: unlink d2/l3 0 2024-04-02T04:42:45.744 INFO:tasks.workunit.client.1.smithi027.stdout:7/132: write d2/f13 [372133,42853] 0 2024-04-02T04:42:45.746 INFO:tasks.workunit.client.1.smithi027.stdout:7/133: dread - d2/d1d/f20 zero size 2024-04-02T04:42:45.747 INFO:tasks.workunit.client.1.smithi027.stdout:3/83: creat d1/d1a/f1c x:0 0 0 2024-04-02T04:42:45.748 INFO:tasks.workunit.client.1.smithi027.stdout:6/132: dwrite f7 [0,4194304] 0 2024-04-02T04:42:45.749 INFO:tasks.workunit.client.0.smithi016.stdout:0/631: rmdir d0/d16/d2e/d53 39 2024-04-02T04:42:45.750 INFO:tasks.workunit.client.0.smithi016.stdout:0/632: truncate d0/d16/d24/d62/f9b 142205 0 2024-04-02T04:42:45.752 INFO:tasks.workunit.client.0.smithi016.stdout:0/633: truncate d0/d16/d24/d4e/d8d/d71/dc7/fd2 764251 0 2024-04-02T04:42:45.754 INFO:tasks.workunit.client.1.smithi027.stdout:0/167: dread d4/d16/d2b/f18 [0,4194304] 0 2024-04-02T04:42:45.755 INFO:tasks.workunit.client.1.smithi027.stdout:0/168: write d4/d16/d2b/f2c [606018,85591] 0 2024-04-02T04:42:45.761 INFO:tasks.workunit.client.1.smithi027.stdout:8/86: rename d5/la to d5/d8/db/l1a 0 2024-04-02T04:42:45.764 INFO:tasks.workunit.client.1.smithi027.stdout:0/169: dwrite d4/d21/f2e [0,4194304] 0 2024-04-02T04:42:45.768 INFO:tasks.workunit.client.0.smithi016.stdout:3/753: fdatasync d2/d2e/d36/d66/d90/fef 0 2024-04-02T04:42:45.771 INFO:tasks.workunit.client.1.smithi027.stdout:9/117: write fa [3277957,14792] 0 2024-04-02T04:42:45.771 INFO:tasks.workunit.client.0.smithi016.stdout:8/632: write d1/d31/f46 [5493094,129406] 0 2024-04-02T04:42:45.773 INFO:tasks.workunit.client.0.smithi016.stdout:8/633: write d1/d70/d71/d74/fc3 [2244613,32461] 0 2024-04-02T04:42:45.776 INFO:tasks.workunit.client.1.smithi027.stdout:2/136: getdents d1 0 2024-04-02T04:42:45.780 INFO:tasks.workunit.client.0.smithi016.stdout:6/705: creat d0/d9/d16/d56/d89/d67/d6f/fdc x:0 0 0 2024-04-02T04:42:45.783 INFO:tasks.workunit.client.1.smithi027.stdout:1/106: dwrite d0/fb [0,4194304] 0 2024-04-02T04:42:45.783 INFO:tasks.workunit.client.1.smithi027.stdout:5/203: dwrite d0/df/d11/d13/f3e [0,4194304] 0 2024-04-02T04:42:45.784 INFO:tasks.workunit.client.1.smithi027.stdout:8/87: rename d5/c9 to d5/d8/db/c1b 0 2024-04-02T04:42:45.786 INFO:tasks.workunit.client.1.smithi027.stdout:0/170: creat d4/d21/f37 x:0 0 0 2024-04-02T04:42:45.786 INFO:tasks.workunit.client.0.smithi016.stdout:5/621: mknod d3/df/dd5/ce5 0 2024-04-02T04:42:45.787 INFO:tasks.workunit.client.1.smithi027.stdout:0/171: chown d4/d16/d2b/c12 71 1 2024-04-02T04:42:45.788 INFO:tasks.workunit.client.0.smithi016.stdout:0/634: dwrite d0/d16/fb8 [0,4194304] 0 2024-04-02T04:42:45.789 INFO:tasks.workunit.client.0.smithi016.stdout:5/622: write d3/df/d18/f89 [1296396,87581] 0 2024-04-02T04:42:45.789 INFO:tasks.workunit.client.0.smithi016.stdout:3/754: creat d2/d15/d16/d8a/dcd/ffe x:0 0 0 2024-04-02T04:42:45.790 INFO:tasks.workunit.client.0.smithi016.stdout:4/671: creat d5/d17/d76/d7e/fcb x:0 0 0 2024-04-02T04:42:45.791 INFO:tasks.workunit.client.1.smithi027.stdout:3/84: link d1/de/df/f14 d1/d1a/f1d 0 2024-04-02T04:42:45.793 INFO:tasks.workunit.client.1.smithi027.stdout:9/118: symlink d1a/l2a 0 2024-04-02T04:42:45.794 INFO:tasks.workunit.client.0.smithi016.stdout:9/730: link d0/d4e/d50/d63/dd2/l9e d0/d4a/d75/d95/d9a/le0 0 2024-04-02T04:42:45.794 INFO:tasks.workunit.client.1.smithi027.stdout:9/119: read f15 [183325,98638] 0 2024-04-02T04:42:45.795 INFO:tasks.workunit.client.0.smithi016.stdout:9/731: fdatasync d0/d4e/fae 0 2024-04-02T04:42:45.799 INFO:tasks.workunit.client.0.smithi016.stdout:1/636: rename d14/d19/d1d/d45/d4f/ccc to d14/d19/d1d/d29/cce 0 2024-04-02T04:42:45.800 INFO:tasks.workunit.client.0.smithi016.stdout:1/637: write d14/d19/fa7 [34694,57058] 0 2024-04-02T04:42:45.801 INFO:tasks.workunit.client.0.smithi016.stdout:6/706: mknod d0/d9/d16/d56/d59/cdd 0 2024-04-02T04:42:45.805 INFO:tasks.workunit.client.1.smithi027.stdout:5/204: creat d0/f4f x:0 0 0 2024-04-02T04:42:45.806 INFO:tasks.workunit.client.0.smithi016.stdout:7/648: creat d0/d46/fe6 x:0 0 0 2024-04-02T04:42:45.807 INFO:tasks.workunit.client.1.smithi027.stdout:1/107: dwrite d0/f16 [4194304,4194304] 0 2024-04-02T04:42:45.808 INFO:tasks.workunit.client.0.smithi016.stdout:2/726: getdents d12/d35/d3b/d73 0 2024-04-02T04:42:45.808 INFO:tasks.workunit.client.1.smithi027.stdout:1/108: dread - d0/de/f1f zero size 2024-04-02T04:42:45.810 INFO:tasks.workunit.client.1.smithi027.stdout:1/109: write d0/de/f1b [27611,75417] 0 2024-04-02T04:42:45.810 INFO:tasks.workunit.client.0.smithi016.stdout:1/638: dwrite d14/d19/d1e/d5e/fb1 [0,4194304] 0 2024-04-02T04:42:45.811 INFO:tasks.workunit.client.1.smithi027.stdout:3/85: mknod d1/de/c1e 0 2024-04-02T04:42:45.814 INFO:tasks.workunit.client.1.smithi027.stdout:6/133: creat d13/d1b/d24/f34 x:0 0 0 2024-04-02T04:42:45.819 INFO:tasks.workunit.client.0.smithi016.stdout:5/623: creat d3/df/d86/fe6 x:0 0 0 2024-04-02T04:42:45.820 INFO:tasks.workunit.client.1.smithi027.stdout:9/120: creat d1a/d1b/d20/f2b x:0 0 0 2024-04-02T04:42:45.822 INFO:tasks.workunit.client.0.smithi016.stdout:4/672: unlink d5/d16/d88/db0/lb6 0 2024-04-02T04:42:45.823 INFO:tasks.workunit.client.0.smithi016.stdout:9/732: rmdir d0/d9/db/dc9/d53/d7e 39 2024-04-02T04:42:45.825 INFO:tasks.workunit.client.1.smithi027.stdout:0/172: mkdir d4/d38 0 2024-04-02T04:42:45.825 INFO:tasks.workunit.client.0.smithi016.stdout:8/634: dwrite d1/d6/d36/d3f/d77/fa7 [4194304,4194304] 0 2024-04-02T04:42:45.826 INFO:tasks.workunit.client.0.smithi016.stdout:8/635: readlink d1/l7 0 2024-04-02T04:42:45.829 INFO:tasks.workunit.client.1.smithi027.stdout:2/137: dwrite d1/f13 [4194304,4194304] 0 2024-04-02T04:42:45.829 INFO:tasks.workunit.client.1.smithi027.stdout:7/134: getdents d2/d7/da 0 2024-04-02T04:42:45.830 INFO:tasks.workunit.client.0.smithi016.stdout:6/707: creat d0/d9/d16/d56/da4/fde x:0 0 0 2024-04-02T04:42:45.835 INFO:tasks.workunit.client.0.smithi016.stdout:0/635: dread d0/f19 [0,4194304] 0 2024-04-02T04:42:45.838 INFO:tasks.workunit.client.1.smithi027.stdout:2/138: dwrite d1/f6 [0,4194304] 0 2024-04-02T04:42:45.839 INFO:tasks.workunit.client.0.smithi016.stdout:2/727: rename d12/d35/d3b/d73 to d12/d35/d3b/d42/d59/d91/db2/dfd 0 2024-04-02T04:42:45.839 INFO:tasks.workunit.client.1.smithi027.stdout:2/139: dread - d1/dd/d17/d12/f1f zero size 2024-04-02T04:42:45.840 INFO:tasks.workunit.client.1.smithi027.stdout:2/140: write d1/dd/d17/f2c [291011,3233] 0 2024-04-02T04:42:45.841 INFO:tasks.workunit.client.1.smithi027.stdout:6/134: creat d13/d1b/d24/d28/f35 x:0 0 0 2024-04-02T04:42:45.850 INFO:tasks.workunit.client.0.smithi016.stdout:4/673: rmdir d5/d1d/d5f/d95 39 2024-04-02T04:42:45.853 INFO:tasks.workunit.client.0.smithi016.stdout:9/733: mkdir d0/d9/db/dc9/de1 0 2024-04-02T04:42:45.854 INFO:tasks.workunit.client.1.smithi027.stdout:0/173: unlink d4/f32 0 2024-04-02T04:42:45.858 INFO:tasks.workunit.client.1.smithi027.stdout:7/135: mknod d2/d1d/c28 0 2024-04-02T04:42:45.863 INFO:tasks.workunit.client.1.smithi027.stdout:3/86: truncate d1/f18 107296 0 2024-04-02T04:42:45.863 INFO:tasks.workunit.client.0.smithi016.stdout:3/755: dwrite d2/d15/ffd [0,4194304] 0 2024-04-02T04:42:45.865 INFO:tasks.workunit.client.0.smithi016.stdout:0/636: mknod d0/d16/d24/d4e/d8d/d71/dc7/cd9 0 2024-04-02T04:42:45.865 INFO:tasks.workunit.client.1.smithi027.stdout:8/88: dwrite f4 [0,4194304] 0 2024-04-02T04:42:45.867 INFO:tasks.workunit.client.1.smithi027.stdout:9/121: unlink c10 0 2024-04-02T04:42:45.868 INFO:tasks.workunit.client.0.smithi016.stdout:2/728: rename d12/d26/l85 to d12/d70/lfe 0 2024-04-02T04:42:45.869 INFO:tasks.workunit.client.1.smithi027.stdout:8/89: truncate d5/d8/f17 909540 0 2024-04-02T04:42:45.869 INFO:tasks.workunit.client.0.smithi016.stdout:3/756: rename d2 to d2/d15/d1a/d45/df2/dff 22 2024-04-02T04:42:45.870 INFO:tasks.workunit.client.1.smithi027.stdout:9/122: dread - d1a/f1f zero size 2024-04-02T04:42:45.871 INFO:tasks.workunit.client.0.smithi016.stdout:3/757: chown d2/d2e/fd8 133500379 1 2024-04-02T04:42:45.872 INFO:tasks.workunit.client.1.smithi027.stdout:9/123: chown d1a/d1d/d29 5358189 1 2024-04-02T04:42:45.873 INFO:tasks.workunit.client.1.smithi027.stdout:3/87: dwrite d1/d1a/f1c [0,4194304] 0 2024-04-02T04:42:45.874 INFO:tasks.workunit.client.0.smithi016.stdout:6/708: getdents d0/d9/d16/dd8 0 2024-04-02T04:42:45.876 INFO:tasks.workunit.client.0.smithi016.stdout:7/649: creat d0/fe7 x:0 0 0 2024-04-02T04:42:45.877 INFO:tasks.workunit.client.0.smithi016.stdout:2/729: dwrite d12/d35/d3f/d56/fcc [0,4194304] 0 2024-04-02T04:42:45.882 INFO:tasks.workunit.client.0.smithi016.stdout:5/624: creat d3/d40/db6/de0/fe7 x:0 0 0 2024-04-02T04:42:45.882 INFO:tasks.workunit.client.1.smithi027.stdout:0/174: mknod d4/d21/d28/c39 0 2024-04-02T04:42:45.883 INFO:tasks.workunit.client.1.smithi027.stdout:0/175: dread - d4/f19 zero size 2024-04-02T04:42:45.885 INFO:tasks.workunit.client.0.smithi016.stdout:9/734: creat d0/d4a/d99/dd3/fe2 x:0 0 0 2024-04-02T04:42:45.886 INFO:tasks.workunit.client.0.smithi016.stdout:9/735: chown d0/d3/d97 1 1 2024-04-02T04:42:45.887 INFO:tasks.workunit.client.1.smithi027.stdout:8/90: unlink d5/d7/l13 0 2024-04-02T04:42:45.894 INFO:tasks.workunit.client.1.smithi027.stdout:8/91: dread d5/d7/f14 [0,4194304] 0 2024-04-02T04:42:45.895 INFO:tasks.workunit.client.0.smithi016.stdout:8/636: creat d1/d6/d36/d3f/d40/d49/fd5 x:0 0 0 2024-04-02T04:42:45.896 INFO:tasks.workunit.client.1.smithi027.stdout:5/205: write d0/f6 [3508612,115909] 0 2024-04-02T04:42:45.896 INFO:tasks.workunit.client.1.smithi027.stdout:8/92: dread d5/d7/f14 [0,4194304] 0 2024-04-02T04:42:45.897 INFO:tasks.workunit.client.0.smithi016.stdout:1/639: getdents d14 0 2024-04-02T04:42:45.897 INFO:tasks.workunit.client.1.smithi027.stdout:6/135: truncate d13/f18 214091 0 2024-04-02T04:42:45.898 INFO:tasks.workunit.client.1.smithi027.stdout:6/136: stat d13/d1b 0 2024-04-02T04:42:45.899 INFO:tasks.workunit.client.0.smithi016.stdout:8/637: dwrite d1/d6/d36/f99 [0,4194304] 0 2024-04-02T04:42:45.900 INFO:tasks.workunit.client.0.smithi016.stdout:7/650: chown d0/d46/d52/f80 20 1 2024-04-02T04:42:45.901 INFO:tasks.workunit.client.0.smithi016.stdout:8/638: fsync d1/d6/d36/fa3 0 2024-04-02T04:42:45.901 INFO:tasks.workunit.client.0.smithi016.stdout:2/730: creat d12/d35/d3b/d42/d59/d91/fff x:0 0 0 2024-04-02T04:42:45.904 INFO:tasks.workunit.client.1.smithi027.stdout:0/176: creat d4/d16/d2b/f3a x:0 0 0 2024-04-02T04:42:45.907 INFO:tasks.workunit.client.0.smithi016.stdout:7/651: dwrite d0/d46/d37/dbe/dae/d9d/fe2 [0,4194304] 0 2024-04-02T04:42:45.913 INFO:tasks.workunit.client.1.smithi027.stdout:9/124: dwrite d1a/d1b/f1c [4194304,4194304] 0 2024-04-02T04:42:45.913 INFO:tasks.workunit.client.1.smithi027.stdout:7/136: link d2/d7/da/dc/c11 d2/d7/da/dc/d1f/c29 0 2024-04-02T04:42:45.915 INFO:tasks.workunit.client.1.smithi027.stdout:7/137: read - d2/d1d/f20 zero size 2024-04-02T04:42:45.919 INFO:tasks.workunit.client.0.smithi016.stdout:7/652: dwrite d0/d46/d37/faa [0,4194304] 0 2024-04-02T04:42:45.919 INFO:tasks.workunit.client.0.smithi016.stdout:1/640: chown d14/d19/d1e/d71/d9c 156154 1 2024-04-02T04:42:45.921 INFO:tasks.workunit.client.1.smithi027.stdout:1/110: link d0/l8 d0/l26 0 2024-04-02T04:42:45.922 INFO:tasks.workunit.client.0.smithi016.stdout:8/639: symlink d1/d6/d36/d4f/dc9/ld6 0 2024-04-02T04:42:45.923 INFO:tasks.workunit.client.0.smithi016.stdout:1/641: write d14/d19/d1d/d45/f92 [995617,35657] 0 2024-04-02T04:42:45.924 INFO:tasks.workunit.client.1.smithi027.stdout:0/177: mkdir d4/d16/d3b 0 2024-04-02T04:42:45.925 INFO:tasks.workunit.client.0.smithi016.stdout:8/640: fdatasync d1/d6/d36/d3f/d40/d49/fd5 0 2024-04-02T04:42:45.926 INFO:tasks.workunit.client.0.smithi016.stdout:8/641: chown d1/d31/l3c 7836 1 2024-04-02T04:42:45.927 INFO:tasks.workunit.client.1.smithi027.stdout:5/206: truncate d0/d5/f2c 2768470 0 2024-04-02T04:42:45.927 INFO:tasks.workunit.client.0.smithi016.stdout:0/637: dwrite d0/d16/f54 [0,4194304] 0 2024-04-02T04:42:45.928 INFO:tasks.workunit.client.0.smithi016.stdout:8/642: chown d1/d6/d86/fb8 964224 1 2024-04-02T04:42:45.929 INFO:tasks.workunit.client.0.smithi016.stdout:5/625: mknod d3/df/d18/ce8 0 2024-04-02T04:42:45.930 INFO:tasks.workunit.client.0.smithi016.stdout:9/736: symlink d0/d3/dd1/le3 0 2024-04-02T04:42:45.932 INFO:tasks.workunit.client.1.smithi027.stdout:9/125: unlink f14 0 2024-04-02T04:42:45.933 INFO:tasks.workunit.client.1.smithi027.stdout:9/126: fdatasync fa 0 2024-04-02T04:42:45.938 INFO:tasks.workunit.client.1.smithi027.stdout:8/93: symlink d5/d8/d15/l1c 0 2024-04-02T04:42:45.939 INFO:tasks.workunit.client.0.smithi016.stdout:6/709: creat d0/d9/d16/d56/d89/d67/fdf x:0 0 0 2024-04-02T04:42:45.941 INFO:tasks.workunit.client.1.smithi027.stdout:7/138: rename d2/d7/f12 to d2/d7/da/f2a 0 2024-04-02T04:42:45.943 INFO:tasks.workunit.client.1.smithi027.stdout:7/139: rename d2/d7/da to d2/d7/da/dc/d1f/d2b 22 2024-04-02T04:42:45.950 INFO:tasks.workunit.client.1.smithi027.stdout:7/140: dwrite d2/df/f10 [0,4194304] 0 2024-04-02T04:42:45.954 INFO:tasks.workunit.client.1.smithi027.stdout:0/178: unlink d4/d16/d2b/lf 0 2024-04-02T04:42:45.955 INFO:tasks.workunit.client.0.smithi016.stdout:2/731: write d12/d35/d3b/d42/d59/f88 [1268390,44013] 0 2024-04-02T04:42:45.956 INFO:tasks.workunit.client.0.smithi016.stdout:2/732: chown d12/d35/d3f/d4d/f97 36114 1 2024-04-02T04:42:45.957 INFO:tasks.workunit.client.0.smithi016.stdout:5/626: mkdir d3/df/d86/de9 0 2024-04-02T04:42:45.957 INFO:tasks.workunit.client.0.smithi016.stdout:5/627: fsync d3/d4c/dc4/fe1 0 2024-04-02T04:42:45.960 INFO:tasks.workunit.client.0.smithi016.stdout:3/758: getdents d2/d15/d1a 0 2024-04-02T04:42:45.961 INFO:tasks.workunit.client.0.smithi016.stdout:6/710: write d0/d9/d16/f2a [463680,11775] 0 2024-04-02T04:42:45.961 INFO:tasks.workunit.client.0.smithi016.stdout:3/759: dread d2/d2e/fd8 [0,4194304] 0 2024-04-02T04:42:45.962 INFO:tasks.workunit.client.0.smithi016.stdout:3/760: fsync d2/d15/d60/fe5 0 2024-04-02T04:42:45.963 INFO:tasks.workunit.client.1.smithi027.stdout:3/88: write d1/fc [82155,15622] 0 2024-04-02T04:42:45.964 INFO:tasks.workunit.client.1.smithi027.stdout:3/89: read d1/f8 [1938,62684] 0 2024-04-02T04:42:45.965 INFO:tasks.workunit.client.0.smithi016.stdout:3/761: read d2/f7 [147224,54270] 0 2024-04-02T04:42:45.965 INFO:tasks.workunit.client.1.smithi027.stdout:3/90: read d1/f18 [45223,39196] 0 2024-04-02T04:42:45.967 INFO:tasks.workunit.client.1.smithi027.stdout:8/94: dwrite d5/d7/f14 [0,4194304] 0 2024-04-02T04:42:45.968 INFO:tasks.workunit.client.1.smithi027.stdout:8/95: fsync d5/d8/f17 0 2024-04-02T04:42:45.969 INFO:tasks.workunit.client.0.smithi016.stdout:8/643: mknod d1/d6/d36/d3f/d40/d49/cd7 0 2024-04-02T04:42:45.972 INFO:tasks.workunit.client.1.smithi027.stdout:3/91: dread d1/f7 [4194304,4194304] 0 2024-04-02T04:42:45.973 INFO:tasks.workunit.client.0.smithi016.stdout:2/733: readlink d12/d35/d3b/d42/d59/l8a 0 2024-04-02T04:42:45.973 INFO:tasks.workunit.client.0.smithi016.stdout:2/734: fdatasync d12/d35/d3b/f63 0 2024-04-02T04:42:45.974 INFO:tasks.workunit.client.1.smithi027.stdout:3/92: dread - d1/de/df/f19 zero size 2024-04-02T04:42:45.974 INFO:tasks.workunit.client.0.smithi016.stdout:2/735: chown d12/d35/d3b/f57 421 1 2024-04-02T04:42:45.975 INFO:tasks.workunit.client.1.smithi027.stdout:3/93: dread - d1/de/df/f17 zero size 2024-04-02T04:42:45.976 INFO:tasks.workunit.client.1.smithi027.stdout:7/141: write d2/df/f10 [5076342,42158] 0 2024-04-02T04:42:45.976 INFO:tasks.workunit.client.1.smithi027.stdout:3/94: write d1/fc [731904,108483] 0 2024-04-02T04:42:45.976 INFO:tasks.workunit.client.0.smithi016.stdout:9/737: creat d0/d4a/d75/d95/d9a/fe4 x:0 0 0 2024-04-02T04:42:45.977 INFO:tasks.workunit.client.0.smithi016.stdout:5/628: mknod d3/df/d93/cea 0 2024-04-02T04:42:45.977 INFO:tasks.workunit.client.1.smithi027.stdout:3/95: truncate d1/f8 1798628 0 2024-04-02T04:42:45.981 INFO:tasks.workunit.client.1.smithi027.stdout:1/111: truncate d0/f5 1902532 0 2024-04-02T04:42:45.982 INFO:tasks.workunit.client.1.smithi027.stdout:7/142: fdatasync d2/df/f10 0 2024-04-02T04:42:45.984 INFO:tasks.workunit.client.0.smithi016.stdout:6/711: mknod d0/d9/d16/d56/da4/ce0 0 2024-04-02T04:42:45.985 INFO:tasks.workunit.client.0.smithi016.stdout:7/653: link d0/d46/d37/d47/c6c d0/d28/d64/ce8 0 2024-04-02T04:42:45.985 INFO:tasks.workunit.client.0.smithi016.stdout:6/712: dread - d0/d9/d16/d56/d89/f85 zero size 2024-04-02T04:42:45.985 INFO:tasks.workunit.client.0.smithi016.stdout:7/654: stat d0 0 2024-04-02T04:42:45.985 INFO:tasks.workunit.client.0.smithi016.stdout:7/655: dread - d0/d46/d52/d82/fbb zero size 2024-04-02T04:42:45.986 INFO:tasks.workunit.client.0.smithi016.stdout:3/762: rmdir d2/d15/d60 39 2024-04-02T04:42:45.986 INFO:tasks.workunit.client.0.smithi016.stdout:7/656: chown d0/d28/d64/c68 395 1 2024-04-02T04:42:45.989 INFO:tasks.workunit.client.0.smithi016.stdout:8/644: dread d1/d6/d36/d8e/f9c [4194304,4194304] 0 2024-04-02T04:42:45.991 INFO:tasks.workunit.client.0.smithi016.stdout:8/645: chown d1/d6/d36/d3f/d40/d49/d4a/f98 5 1 2024-04-02T04:42:45.991 INFO:tasks.workunit.client.1.smithi027.stdout:5/207: dwrite d0/df/d2f/d3b/f3d [0,4194304] 0 2024-04-02T04:42:45.996 INFO:tasks.workunit.client.0.smithi016.stdout:2/736: creat d12/d35/d3b/d42/d59/d91/db2/f100 x:0 0 0 2024-04-02T04:42:45.999 INFO:tasks.workunit.client.0.smithi016.stdout:9/738: symlink d0/d9/d4d/da8/le5 0 2024-04-02T04:42:45.999 INFO:tasks.workunit.client.0.smithi016.stdout:9/739: fdatasync d0/d9/db/f43 0 2024-04-02T04:42:46.001 INFO:tasks.workunit.client.1.smithi027.stdout:8/96: creat d5/d8/db/dd/f1d x:0 0 0 2024-04-02T04:42:46.003 INFO:tasks.workunit.client.1.smithi027.stdout:6/137: link c10 d13/d1b/d24/c36 0 2024-04-02T04:42:46.003 INFO:tasks.workunit.client.1.smithi027.stdout:8/97: read d5/d8/db/dd/f10 [856005,71542] 0 2024-04-02T04:42:46.003 INFO:tasks.workunit.client.0.smithi016.stdout:1/642: getdents d14/d19/daf 0 2024-04-02T04:42:46.004 INFO:tasks.workunit.client.0.smithi016.stdout:1/643: chown fa 10151 1 2024-04-02T04:42:46.004 INFO:tasks.workunit.client.0.smithi016.stdout:6/713: symlink d0/d9/d16/d56/d89/d67/d6d/le1 0 2024-04-02T04:42:46.005 INFO:tasks.workunit.client.0.smithi016.stdout:7/657: creat d0/d21/fe9 x:0 0 0 2024-04-02T04:42:46.007 INFO:tasks.workunit.client.1.smithi027.stdout:4/139: sync 2024-04-02T04:42:46.008 INFO:tasks.workunit.client.0.smithi016.stdout:8/646: creat d1/d6/d86/fd8 x:0 0 0 2024-04-02T04:42:46.014 INFO:tasks.workunit.client.0.smithi016.stdout:7/658: rmdir d0/d28/d64 39 2024-04-02T04:42:46.016 INFO:tasks.workunit.client.1.smithi027.stdout:5/208: dwrite d0/df/d11/d13/f1e [0,4194304] 0 2024-04-02T04:42:46.017 INFO:tasks.workunit.client.1.smithi027.stdout:0/179: truncate d4/f9 3009998 0 2024-04-02T04:42:46.018 INFO:tasks.workunit.client.0.smithi016.stdout:8/647: fdatasync d1/d6/d36/d3f/d77/f8a 0 2024-04-02T04:42:46.019 INFO:tasks.workunit.client.0.smithi016.stdout:8/648: chown d1/d6/d36/d4f/d5d/l73 60 1 2024-04-02T04:42:46.019 INFO:tasks.workunit.client.1.smithi027.stdout:0/180: write d4/d16/d2b/f2c [261478,120583] 0 2024-04-02T04:42:46.021 INFO:tasks.workunit.client.1.smithi027.stdout:8/98: stat d5/d8/db/c1b 0 2024-04-02T04:42:46.021 INFO:tasks.workunit.client.1.smithi027.stdout:0/181: dread - d4/d16/d2b/f3a zero size 2024-04-02T04:42:46.023 INFO:tasks.workunit.client.0.smithi016.stdout:5/629: rename d3/d4c/dd1/ccf to d3/ceb 0 2024-04-02T04:42:46.025 INFO:tasks.workunit.client.1.smithi027.stdout:6/138: symlink d13/d1b/d24/d25/l37 0 2024-04-02T04:42:46.026 INFO:tasks.workunit.client.1.smithi027.stdout:6/139: chown d13/d1b/l26 193084 1 2024-04-02T04:42:46.027 INFO:tasks.workunit.client.0.smithi016.stdout:1/644: mkdir d14/d19/d1e/d71/dcf 0 2024-04-02T04:42:46.028 INFO:tasks.workunit.client.1.smithi027.stdout:0/182: dwrite f2 [4194304,4194304] 0 2024-04-02T04:42:46.029 INFO:tasks.workunit.client.1.smithi027.stdout:3/96: link d1/de/df/f19 d1/de/df/f1f 0 2024-04-02T04:42:46.029 INFO:tasks.workunit.client.1.smithi027.stdout:3/97: chown d1/f8 3361 1 2024-04-02T04:42:46.030 INFO:tasks.workunit.client.0.smithi016.stdout:5/630: dwrite d3/df/d18/d26/d70/faf [0,4194304] 0 2024-04-02T04:42:46.034 INFO:tasks.workunit.client.1.smithi027.stdout:5/209: fsync d0/d1d/f31 0 2024-04-02T04:42:46.034 INFO:tasks.workunit.client.0.smithi016.stdout:9/740: dwrite d0/d9/f2e [0,4194304] 0 2024-04-02T04:42:46.039 INFO:tasks.workunit.client.1.smithi027.stdout:8/99: rename d5/d8/l11 to d5/l1e 0 2024-04-02T04:42:46.039 INFO:tasks.workunit.client.0.smithi016.stdout:7/659: dwrite d0/d46/d52/daf/fc7 [0,4194304] 0 2024-04-02T04:42:46.040 INFO:tasks.workunit.client.0.smithi016.stdout:1/645: mknod d14/d19/d1e/d71/d9c/cd0 0 2024-04-02T04:42:46.040 INFO:tasks.workunit.client.1.smithi027.stdout:8/100: fsync d5/d8/db/dd/f1d 0 2024-04-02T04:42:46.041 INFO:tasks.workunit.client.1.smithi027.stdout:6/140: symlink d13/d1b/d24/d28/l38 0 2024-04-02T04:42:46.042 INFO:tasks.workunit.client.1.smithi027.stdout:6/141: dread - d13/d1b/f23 zero size 2024-04-02T04:42:46.048 INFO:tasks.workunit.client.0.smithi016.stdout:5/631: rmdir d3/d40 39 2024-04-02T04:42:46.050 INFO:tasks.workunit.client.1.smithi027.stdout:3/98: mkdir d1/d1a/d20 0 2024-04-02T04:42:46.050 INFO:tasks.workunit.client.1.smithi027.stdout:8/101: dwrite d5/d7/f14 [4194304,4194304] 0 2024-04-02T04:42:46.051 INFO:tasks.workunit.client.0.smithi016.stdout:8/649: mknod d1/d6/d36/d3f/d40/d49/cd9 0 2024-04-02T04:42:46.052 INFO:tasks.workunit.client.0.smithi016.stdout:8/650: readlink d1/d31/l3c 0 2024-04-02T04:42:46.054 INFO:tasks.workunit.client.0.smithi016.stdout:5/632: dwrite d3/df/d18/d26/f7a [4194304,4194304] 0 2024-04-02T04:42:46.062 INFO:tasks.workunit.client.0.smithi016.stdout:9/741: creat d0/d4a/d99/d46/fe6 x:0 0 0 2024-04-02T04:42:46.065 INFO:tasks.workunit.client.1.smithi027.stdout:0/183: write f1 [7430948,59993] 0 2024-04-02T04:42:46.066 INFO:tasks.workunit.client.0.smithi016.stdout:1/646: truncate d14/f56 1395540 0 2024-04-02T04:42:46.067 INFO:tasks.workunit.client.0.smithi016.stdout:6/714: link d0/f84 d0/d9/fe2 0 2024-04-02T04:42:46.069 INFO:tasks.workunit.client.0.smithi016.stdout:6/715: stat d0/d9/d16/d46/d5e 0 2024-04-02T04:42:46.070 INFO:tasks.workunit.client.0.smithi016.stdout:7/660: dwrite d0/d28/d64/fd6 [0,4194304] 0 2024-04-02T04:42:46.072 INFO:tasks.workunit.client.0.smithi016.stdout:7/661: dread - d0/d46/d16/d49/f6e zero size 2024-04-02T04:42:46.075 INFO:tasks.workunit.client.0.smithi016.stdout:3/763: rename d2/d2e/d36/d48/fd4 to d2/d2e/f100 0 2024-04-02T04:42:46.076 INFO:tasks.workunit.client.1.smithi027.stdout:8/102: write d5/d8/db/dd/f10 [4949206,2762] 0 2024-04-02T04:42:46.076 INFO:tasks.workunit.client.0.smithi016.stdout:3/764: write d2/d2e/f30 [775597,29781] 0 2024-04-02T04:42:46.076 INFO:tasks.workunit.client.0.smithi016.stdout:7/662: dread d0/d46/d52/daf/fc7 [0,4194304] 0 2024-04-02T04:42:46.077 INFO:tasks.workunit.client.1.smithi027.stdout:8/103: write d5/d7/f14 [3908726,83884] 0 2024-04-02T04:42:46.078 INFO:tasks.workunit.client.0.smithi016.stdout:5/633: write d3/d40/d4f/d67/f78 [2307220,6760] 0 2024-04-02T04:42:46.078 INFO:tasks.workunit.client.1.smithi027.stdout:1/112: link d0/de/l1a d0/l27 0 2024-04-02T04:42:46.079 INFO:tasks.workunit.client.1.smithi027.stdout:1/113: fsync d0/fd 0 2024-04-02T04:42:46.080 INFO:tasks.workunit.client.0.smithi016.stdout:8/651: creat d1/d6/d36/d3f/d40/d49/d4a/d4c/dad/fda x:0 0 0 2024-04-02T04:42:46.081 INFO:tasks.workunit.client.1.smithi027.stdout:7/143: link d2/d7/da/f2a d2/d7/da/dc/d16/f2c 0 2024-04-02T04:42:46.082 INFO:tasks.workunit.client.0.smithi016.stdout:8/652: chown d1/d6/d36/d3f/d40/d49/lbf 38389 1 2024-04-02T04:42:46.083 INFO:tasks.workunit.client.0.smithi016.stdout:9/742: mknod d0/d4e/d50/ce7 0 2024-04-02T04:42:46.085 INFO:tasks.workunit.client.0.smithi016.stdout:1/647: creat d14/da9/fd1 x:0 0 0 2024-04-02T04:42:46.089 INFO:tasks.workunit.client.1.smithi027.stdout:1/114: dwrite d0/de/f1f [0,4194304] 0 2024-04-02T04:42:46.090 INFO:tasks.workunit.client.1.smithi027.stdout:0/184: rename d4/f19 to d4/d16/d2b/d2d/f3c 0 2024-04-02T04:42:46.091 INFO:tasks.workunit.client.0.smithi016.stdout:6/716: truncate d0/d9/d16/d56/d59/da1/fd3 92532 0 2024-04-02T04:42:46.098 INFO:tasks.workunit.client.1.smithi027.stdout:7/144: dwrite d2/f13 [0,4194304] 0 2024-04-02T04:42:46.098 INFO:tasks.workunit.client.1.smithi027.stdout:6/142: truncate fc 3038202 0 2024-04-02T04:42:46.099 INFO:tasks.workunit.client.1.smithi027.stdout:7/145: dread d2/d1d/f25 [0,4194304] 0 2024-04-02T04:42:46.107 INFO:tasks.workunit.client.0.smithi016.stdout:3/765: write d2/d15/f31 [954831,81924] 0 2024-04-02T04:42:46.108 INFO:tasks.workunit.client.0.smithi016.stdout:3/766: dread - d2/d15/d70/ff5 zero size 2024-04-02T04:42:46.113 INFO:tasks.workunit.client.1.smithi027.stdout:5/210: link d0/df/d2f/d3b/c3c d0/df/d11/c50 0 2024-04-02T04:42:46.114 INFO:tasks.workunit.client.1.smithi027.stdout:0/185: dread f1 [0,4194304] 0 2024-04-02T04:42:46.122 INFO:tasks.workunit.client.1.smithi027.stdout:2/141: sync 2024-04-02T04:42:46.124 INFO:tasks.workunit.client.1.smithi027.stdout:3/99: rmdir d1/d1a/d20 0 2024-04-02T04:42:46.126 INFO:tasks.workunit.client.0.smithi016.stdout:7/663: write d0/d46/fc [3147734,90302] 0 2024-04-02T04:42:46.127 INFO:tasks.workunit.client.0.smithi016.stdout:4/674: sync 2024-04-02T04:42:46.128 INFO:tasks.workunit.client.1.smithi027.stdout:8/104: link d5/d8/f17 d5/d8/db/dd/f1f 0 2024-04-02T04:42:46.130 INFO:tasks.workunit.client.1.smithi027.stdout:8/105: stat d5/d8/d15/l1c 0 2024-04-02T04:42:46.130 INFO:tasks.workunit.client.1.smithi027.stdout:2/142: dwrite d1/dd/d17/d12/f1f [0,4194304] 0 2024-04-02T04:42:46.130 INFO:tasks.workunit.client.1.smithi027.stdout:8/106: readlink d5/l6 0 2024-04-02T04:42:46.131 INFO:tasks.workunit.client.1.smithi027.stdout:5/211: truncate d0/d1a/f49 835706 0 2024-04-02T04:42:46.132 INFO:tasks.workunit.client.0.smithi016.stdout:8/653: link d1/d70/d71/fb3 d1/d6/d36/d8e/fdb 0 2024-04-02T04:42:46.132 INFO:tasks.workunit.client.1.smithi027.stdout:8/107: stat d5/d8/db/dd 0 2024-04-02T04:42:46.136 INFO:tasks.workunit.client.1.smithi027.stdout:1/115: creat d0/d1e/d25/f28 x:0 0 0 2024-04-02T04:42:46.137 INFO:tasks.workunit.client.0.smithi016.stdout:1/648: creat d14/d19/d1d/d29/d4c/fd2 x:0 0 0 2024-04-02T04:42:46.139 INFO:tasks.workunit.client.1.smithi027.stdout:2/143: dwrite d1/f6 [0,4194304] 0 2024-04-02T04:42:46.140 INFO:tasks.workunit.client.1.smithi027.stdout:6/143: link d13/d1b/d24/d28/f2a d13/d1b/d24/d28/f39 0 2024-04-02T04:42:46.142 INFO:tasks.workunit.client.1.smithi027.stdout:6/144: write f12 [3191705,51843] 0 2024-04-02T04:42:46.143 INFO:tasks.workunit.client.1.smithi027.stdout:6/145: write f7 [1505592,112435] 0 2024-04-02T04:42:46.144 INFO:tasks.workunit.client.0.smithi016.stdout:2/737: rename d12/d9f/dac to d12/d35/d101 0 2024-04-02T04:42:46.146 INFO:tasks.workunit.client.0.smithi016.stdout:2/738: write d12/d35/d3b/d42/d59/fba [2150604,28259] 0 2024-04-02T04:42:46.148 INFO:tasks.workunit.client.0.smithi016.stdout:6/717: dread d0/d9/d16/f8d [0,4194304] 0 2024-04-02T04:42:46.149 INFO:tasks.workunit.client.0.smithi016.stdout:6/718: stat d0/d9/d16/d56/d89/d67/fdf 0 2024-04-02T04:42:46.150 INFO:tasks.workunit.client.0.smithi016.stdout:3/767: creat d2/d2e/d36/d48/da2/dcb/f101 x:0 0 0 2024-04-02T04:42:46.150 INFO:tasks.workunit.client.0.smithi016.stdout:9/743: truncate d0/d4a/f9c 2070892 0 2024-04-02T04:42:46.151 INFO:tasks.workunit.client.0.smithi016.stdout:9/744: dread - d0/d4a/d99/dd3/fe2 zero size 2024-04-02T04:42:46.151 INFO:tasks.workunit.client.0.smithi016.stdout:5/634: rmdir d3/df/d18/d26/ddc/dde 0 2024-04-02T04:42:46.154 INFO:tasks.workunit.client.1.smithi027.stdout:2/144: fsync d1/dd/d17/fc 0 2024-04-02T04:42:46.156 INFO:tasks.workunit.client.1.smithi027.stdout:0/186: mkdir d4/d3d 0 2024-04-02T04:42:46.156 INFO:tasks.workunit.client.0.smithi016.stdout:9/745: dwrite d0/d9/f2e [0,4194304] 0 2024-04-02T04:42:46.158 INFO:tasks.workunit.client.0.smithi016.stdout:9/746: chown d0/d4a/d99/d46/l64 1469997 1 2024-04-02T04:42:46.158 INFO:tasks.workunit.client.0.smithi016.stdout:1/649: creat d14/d19/d1d/d45/d4f/d86/db4/fd3 x:0 0 0 2024-04-02T04:42:46.161 INFO:tasks.workunit.client.1.smithi027.stdout:8/108: mknod d5/d8/d15/c20 0 2024-04-02T04:42:46.165 INFO:tasks.workunit.client.1.smithi027.stdout:4/140: sync 2024-04-02T04:42:46.165 INFO:tasks.workunit.client.1.smithi027.stdout:9/127: sync 2024-04-02T04:42:46.165 INFO:tasks.workunit.client.1.smithi027.stdout:4/141: read f16 [180770,74087] 0 2024-04-02T04:42:46.166 INFO:tasks.workunit.client.0.smithi016.stdout:4/675: rename d5/d1d/d5f/d65 to d5/d17/d18/d6d/d8e/d9c/dcc 0 2024-04-02T04:42:46.167 INFO:tasks.workunit.client.1.smithi027.stdout:9/128: truncate d1a/d1d/f27 86077 0 2024-04-02T04:42:46.168 INFO:tasks.workunit.client.1.smithi027.stdout:5/212: mkdir d0/df/d2f/d3b/d51 0 2024-04-02T04:42:46.170 INFO:tasks.workunit.client.1.smithi027.stdout:4/142: dread f14 [0,4194304] 0 2024-04-02T04:42:46.174 INFO:tasks.workunit.client.0.smithi016.stdout:3/768: mkdir d2/d15/d70/dcf/d102 0 2024-04-02T04:42:46.175 INFO:tasks.workunit.client.0.smithi016.stdout:3/769: write d2/d2e/d36/d48/da2/d88/fe6 [207683,114260] 0 2024-04-02T04:42:46.176 INFO:tasks.workunit.client.1.smithi027.stdout:5/213: dwrite d0/df/d11/d13/f1e [0,4194304] 0 2024-04-02T04:42:46.178 INFO:tasks.workunit.client.1.smithi027.stdout:4/143: dwrite f21 [0,4194304] 0 2024-04-02T04:42:46.179 INFO:tasks.workunit.client.1.smithi027.stdout:4/144: write f16 [511364,93834] 0 2024-04-02T04:42:46.182 INFO:tasks.workunit.client.0.smithi016.stdout:5/635: read f1 [1129400,52276] 0 2024-04-02T04:42:46.183 INFO:tasks.workunit.client.0.smithi016.stdout:6/719: write d0/d9/fe2 [263857,128677] 0 2024-04-02T04:42:46.184 INFO:tasks.workunit.client.0.smithi016.stdout:7/664: unlink d0/d46/d16/d49/c74 0 2024-04-02T04:42:46.185 INFO:tasks.workunit.client.0.smithi016.stdout:7/665: write d0/d46/d16/d49/fa [1268267,45114] 0 2024-04-02T04:42:46.186 INFO:tasks.workunit.client.0.smithi016.stdout:7/666: chown d0/d46/d37/d3a/d4c 18997871 1 2024-04-02T04:42:46.187 INFO:tasks.workunit.client.0.smithi016.stdout:8/654: mknod d1/d6/d36/d3f/d40/d45/dcc/cdc 0 2024-04-02T04:42:46.193 INFO:tasks.workunit.client.0.smithi016.stdout:4/676: truncate d5/d1d/d1e/f36 1042812 0 2024-04-02T04:42:46.194 INFO:tasks.workunit.client.1.smithi027.stdout:8/109: creat d5/d8/db/dd/f21 x:0 0 0 2024-04-02T04:42:46.194 INFO:tasks.workunit.client.1.smithi027.stdout:8/110: stat d5/d7 0 2024-04-02T04:42:46.197 INFO:tasks.workunit.client.1.smithi027.stdout:8/111: dread d5/d8/db/dd/f1f [0,4194304] 0 2024-04-02T04:42:46.200 INFO:tasks.workunit.client.0.smithi016.stdout:4/677: dread d5/d17/d18/d1f/f98 [0,4194304] 0 2024-04-02T04:42:46.201 INFO:tasks.workunit.client.0.smithi016.stdout:3/770: creat d2/d2e/d36/d48/da2/d88/f103 x:0 0 0 2024-04-02T04:42:46.202 INFO:tasks.workunit.client.0.smithi016.stdout:4/678: stat d5/d17/d18/d6d/d8e/d9c/dcc/d6f/f93 0 2024-04-02T04:42:46.202 INFO:tasks.workunit.client.1.smithi027.stdout:9/129: mknod d1a/d1b/d20/c2c 0 2024-04-02T04:42:46.202 INFO:tasks.workunit.client.1.smithi027.stdout:9/130: chown d1a/d1d/f24 90897 1 2024-04-02T04:42:46.203 INFO:tasks.workunit.client.1.smithi027.stdout:9/131: dread - d1a/f1f zero size 2024-04-02T04:42:46.203 INFO:tasks.workunit.client.0.smithi016.stdout:5/636: unlink d3/df/d18/d26/f4b 0 2024-04-02T04:42:46.205 INFO:tasks.workunit.client.0.smithi016.stdout:5/637: truncate d3/df/d86/fe6 216338 0 2024-04-02T04:42:46.206 INFO:tasks.workunit.client.0.smithi016.stdout:6/720: dread - d0/d9/d16/f3c zero size 2024-04-02T04:42:46.206 INFO:tasks.workunit.client.0.smithi016.stdout:3/771: dread d2/d2e/d36/d48/da2/f92 [0,4194304] 0 2024-04-02T04:42:46.210 INFO:tasks.workunit.client.1.smithi027.stdout:1/116: unlink d0/d1e/d25/f28 0 2024-04-02T04:42:46.212 INFO:tasks.workunit.client.0.smithi016.stdout:8/655: unlink d1/d6/d36/d3f/d40/d49/d4a/f50 0 2024-04-02T04:42:46.214 INFO:tasks.workunit.client.1.smithi027.stdout:5/214: readlink d0/l4 0 2024-04-02T04:42:46.219 INFO:tasks.workunit.client.1.smithi027.stdout:4/145: mknod d24/c29 0 2024-04-02T04:42:46.222 INFO:tasks.workunit.client.1.smithi027.stdout:7/146: getdents d2 0 2024-04-02T04:42:46.224 INFO:tasks.workunit.client.1.smithi027.stdout:4/146: dread f12 [0,4194304] 0 2024-04-02T04:42:46.224 INFO:tasks.workunit.client.1.smithi027.stdout:7/147: truncate d2/d1d/f20 241873 0 2024-04-02T04:42:46.224 INFO:tasks.workunit.client.1.smithi027.stdout:4/147: fsync d24/f27 0 2024-04-02T04:42:46.226 INFO:tasks.workunit.client.1.smithi027.stdout:0/187: write d4/d16/d1b/f29 [418527,24369] 0 2024-04-02T04:42:46.228 INFO:tasks.workunit.client.0.smithi016.stdout:1/650: rename d14/d19/d50/l8f to d14/d19/d1e/d71/ld4 0 2024-04-02T04:42:46.228 INFO:tasks.workunit.client.0.smithi016.stdout:1/651: stat d14/d19/d1d/d29 0 2024-04-02T04:42:46.229 INFO:tasks.workunit.client.0.smithi016.stdout:1/652: readlink d14/d19/lbc 0 2024-04-02T04:42:46.229 INFO:tasks.workunit.client.0.smithi016.stdout:4/679: mkdir d5/d17/dcd 0 2024-04-02T04:42:46.230 INFO:tasks.workunit.client.0.smithi016.stdout:5/638: rmdir d3/df/d18/d2f 39 2024-04-02T04:42:46.231 INFO:tasks.workunit.client.0.smithi016.stdout:5/639: fdatasync d3/d40/d4f/fc6 0 2024-04-02T04:42:46.232 INFO:tasks.workunit.client.1.smithi027.stdout:9/132: symlink d1a/d1b/d20/l2d 0 2024-04-02T04:42:46.233 INFO:tasks.workunit.client.0.smithi016.stdout:8/656: symlink d1/d6/d36/d3f/ldd 0 2024-04-02T04:42:46.234 INFO:tasks.workunit.client.0.smithi016.stdout:1/653: dwrite d14/d19/f60 [0,4194304] 0 2024-04-02T04:42:46.236 INFO:tasks.workunit.client.0.smithi016.stdout:9/747: rename d0/d9/db/ce to d0/d9/db/dc9/ce8 0 2024-04-02T04:42:46.238 INFO:tasks.workunit.client.1.smithi027.stdout:6/146: rename d13/d1b/f1d to d13/d1b/d24/f3a 0 2024-04-02T04:42:46.240 INFO:tasks.workunit.client.1.smithi027.stdout:3/100: getdents d1/de/df 0 2024-04-02T04:42:46.240 INFO:tasks.workunit.client.0.smithi016.stdout:5/640: creat d3/d4c/dc4/dd7/fec x:0 0 0 2024-04-02T04:42:46.241 INFO:tasks.workunit.client.1.smithi027.stdout:3/101: dread - d1/f10 zero size 2024-04-02T04:42:46.243 INFO:tasks.workunit.client.1.smithi027.stdout:7/148: creat d2/d7/da/dc/d16/f2d x:0 0 0 2024-04-02T04:42:46.244 INFO:tasks.workunit.client.0.smithi016.stdout:0/638: sync 2024-04-02T04:42:46.244 INFO:tasks.workunit.client.0.smithi016.stdout:1/654: mknod d14/d19/d1d/d29/d39/d4d/cd5 0 2024-04-02T04:42:46.246 INFO:tasks.workunit.client.1.smithi027.stdout:8/112: creat d5/d8/db/df/f22 x:0 0 0 2024-04-02T04:42:46.248 INFO:tasks.workunit.client.0.smithi016.stdout:4/680: creat d5/d17/dcd/fce x:0 0 0 2024-04-02T04:42:46.249 INFO:tasks.workunit.client.1.smithi027.stdout:8/113: truncate d5/d8/db/dd/f1d 1035938 0 2024-04-02T04:42:46.249 INFO:tasks.workunit.client.1.smithi027.stdout:8/114: fsync d5/d8/db/dd/f21 0 2024-04-02T04:42:46.250 INFO:tasks.workunit.client.0.smithi016.stdout:5/641: rmdir d3/d4c/dd1 39 2024-04-02T04:42:46.250 INFO:tasks.workunit.client.0.smithi016.stdout:3/772: dread d2/d15/d1a/f47 [0,4194304] 0 2024-04-02T04:42:46.256 INFO:tasks.workunit.client.1.smithi027.stdout:2/145: getdents d1/dd/d17/d12 0 2024-04-02T04:42:46.258 INFO:tasks.workunit.client.0.smithi016.stdout:0/639: dread d0/d16/d23/d3e/f43 [0,4194304] 0 2024-04-02T04:42:46.260 INFO:tasks.workunit.client.1.smithi027.stdout:7/149: symlink d2/d1d/l2e 0 2024-04-02T04:42:46.260 INFO:tasks.workunit.client.1.smithi027.stdout:2/146: dread d1/dd/d17/d2b/f24 [0,4194304] 0 2024-04-02T04:42:46.262 INFO:tasks.workunit.client.0.smithi016.stdout:4/681: rmdir d5/d1d/d1e/da5 39 2024-04-02T04:42:46.263 INFO:tasks.workunit.client.1.smithi027.stdout:9/133: symlink d1a/d1d/d29/l2e 0 2024-04-02T04:42:46.263 INFO:tasks.workunit.client.0.smithi016.stdout:8/657: link d1/d6/d36/d8e/fdb d1/d31/fde 0 2024-04-02T04:42:46.264 INFO:tasks.workunit.client.1.smithi027.stdout:9/134: fsync d1a/d1b/d20/f2b 0 2024-04-02T04:42:46.264 INFO:tasks.workunit.client.1.smithi027.stdout:9/135: write d1a/d1b/f1c [7093640,99408] 0 2024-04-02T04:42:46.265 INFO:tasks.workunit.client.1.smithi027.stdout:8/115: mknod d5/d8/c23 0 2024-04-02T04:42:46.266 INFO:tasks.workunit.client.1.smithi027.stdout:9/136: dread d1a/d1d/f27 [0,4194304] 0 2024-04-02T04:42:46.266 INFO:tasks.workunit.client.0.smithi016.stdout:5/642: creat d3/df/d93/fed x:0 0 0 2024-04-02T04:42:46.268 INFO:tasks.workunit.client.1.smithi027.stdout:1/117: rename d0/f9 to d0/d1e/d25/f29 0 2024-04-02T04:42:46.271 INFO:tasks.workunit.client.1.smithi027.stdout:6/147: mknod d13/c3b 0 2024-04-02T04:42:46.272 INFO:tasks.workunit.client.1.smithi027.stdout:6/148: write d13/d1b/f1e [53032,92726] 0 2024-04-02T04:42:46.274 INFO:tasks.workunit.client.1.smithi027.stdout:0/188: creat d4/d16/d1b/f3e x:0 0 0 2024-04-02T04:42:46.278 INFO:tasks.workunit.client.0.smithi016.stdout:7/667: rename d0/d28 to d0/d46/d16/dc2/dea 0 2024-04-02T04:42:46.280 INFO:tasks.workunit.client.0.smithi016.stdout:4/682: mkdir d5/d17/d18/d6d/d8e/d9c/dcc/d6f/dcf 0 2024-04-02T04:42:46.281 INFO:tasks.workunit.client.0.smithi016.stdout:4/683: chown d5/f67 981 1 2024-04-02T04:42:46.283 INFO:tasks.workunit.client.0.smithi016.stdout:8/658: mknod d1/d31/cdf 0 2024-04-02T04:42:46.284 INFO:tasks.workunit.client.0.smithi016.stdout:4/684: dread d5/d17/d76/f7d [0,4194304] 0 2024-04-02T04:42:46.284 INFO:tasks.workunit.client.1.smithi027.stdout:9/137: chown le 3 1 2024-04-02T04:42:46.285 INFO:tasks.workunit.client.0.smithi016.stdout:4/685: chown d5/d16/d88/db0 0 1 2024-04-02T04:42:46.285 INFO:tasks.workunit.client.0.smithi016.stdout:4/686: stat d5/d17/d76/f8b 0 2024-04-02T04:42:46.286 INFO:tasks.workunit.client.0.smithi016.stdout:4/687: write d5/d1d/f78 [4858931,128788] 0 2024-04-02T04:42:46.288 INFO:tasks.workunit.client.1.smithi027.stdout:3/102: rename d1/f2 to d1/de/df/d13/f21 0 2024-04-02T04:42:46.288 INFO:tasks.workunit.client.1.smithi027.stdout:3/103: readlink d1/de/l12 0 2024-04-02T04:42:46.289 INFO:tasks.workunit.client.1.smithi027.stdout:3/104: dread - d1/d1a/f1d zero size 2024-04-02T04:42:46.290 INFO:tasks.workunit.client.1.smithi027.stdout:1/118: mknod d0/de/c2a 0 2024-04-02T04:42:46.290 INFO:tasks.workunit.client.1.smithi027.stdout:3/105: truncate d1/f10 572027 0 2024-04-02T04:42:46.292 INFO:tasks.workunit.client.0.smithi016.stdout:4/688: dwrite d5/d17/d76/d7e/f81 [0,4194304] 0 2024-04-02T04:42:46.294 INFO:tasks.workunit.client.1.smithi027.stdout:6/149: mkdir d13/d1b/d24/d28/d3c 0 2024-04-02T04:42:46.295 INFO:tasks.workunit.client.0.smithi016.stdout:0/640: truncate d0/d16/f7a 2833971 0 2024-04-02T04:42:46.303 INFO:tasks.workunit.client.1.smithi027.stdout:2/147: truncate d1/db/f1d 65571 0 2024-04-02T04:42:46.305 INFO:tasks.workunit.client.0.smithi016.stdout:5/643: link d3/d4c/dc4/fe1 d3/dbd/fee 0 2024-04-02T04:42:46.305 INFO:tasks.workunit.client.1.smithi027.stdout:2/148: dread d1/f13 [0,4194304] 0 2024-04-02T04:42:46.306 INFO:tasks.workunit.client.1.smithi027.stdout:9/138: fdatasync f15 0 2024-04-02T04:42:46.307 INFO:tasks.workunit.client.0.smithi016.stdout:9/748: truncate d0/d4e/d50/d63/d6e/d4b/d58/d6a/f9f 2569879 0 2024-04-02T04:42:46.307 INFO:tasks.workunit.client.0.smithi016.stdout:9/749: chown d0/d3/d39 399451875 1 2024-04-02T04:42:46.313 INFO:tasks.workunit.client.1.smithi027.stdout:9/139: dwrite d1a/d1b/f1c [4194304,4194304] 0 2024-04-02T04:42:46.313 INFO:tasks.workunit.client.1.smithi027.stdout:7/150: rename d2/d17 to d2/d7/da/dc/d16/d2f 0 2024-04-02T04:42:46.314 INFO:tasks.workunit.client.0.smithi016.stdout:9/750: dread d0/d4a/d99/d46/f92 [0,4194304] 0 2024-04-02T04:42:46.315 INFO:tasks.workunit.client.0.smithi016.stdout:9/751: chown d0/d9/db/dc9/de1 1160752 1 2024-04-02T04:42:46.319 INFO:tasks.workunit.client.0.smithi016.stdout:2/739: sync 2024-04-02T04:42:46.320 INFO:tasks.workunit.client.1.smithi027.stdout:3/106: creat d1/d1a/f22 x:0 0 0 2024-04-02T04:42:46.321 INFO:tasks.workunit.client.1.smithi027.stdout:9/140: dwrite d1a/f26 [0,4194304] 0 2024-04-02T04:42:46.321 INFO:tasks.workunit.client.0.smithi016.stdout:1/655: write d14/d19/d20/f38 [7600475,27336] 0 2024-04-02T04:42:46.321 INFO:tasks.workunit.client.0.smithi016.stdout:0/641: creat d0/d16/d24/d4e/d8d/d71/dc7/fda x:0 0 0 2024-04-02T04:42:46.322 INFO:tasks.workunit.client.1.smithi027.stdout:1/119: mkdir d0/d1e/d25/d2b 0 2024-04-02T04:42:46.322 INFO:tasks.workunit.client.1.smithi027.stdout:3/107: dread d1/f10 [0,4194304] 0 2024-04-02T04:42:46.323 INFO:tasks.workunit.client.0.smithi016.stdout:0/642: truncate d0/d16/f54 4780515 0 2024-04-02T04:42:46.324 INFO:tasks.workunit.client.0.smithi016.stdout:0/643: dread - d0/d48/fb0 zero size 2024-04-02T04:42:46.327 INFO:tasks.workunit.client.1.smithi027.stdout:6/150: unlink fa 0 2024-04-02T04:42:46.332 INFO:tasks.workunit.client.0.smithi016.stdout:3/773: truncate d2/d15/d16/d8a/dd1/d6e/d6f/db0/f83 2017383 0 2024-04-02T04:42:46.334 INFO:tasks.workunit.client.1.smithi027.stdout:2/149: write d1/dd/d17/d2b/f24 [1735957,85674] 0 2024-04-02T04:42:46.342 INFO:tasks.workunit.client.1.smithi027.stdout:8/116: rename f4 to d5/d7/f24 0 2024-04-02T04:42:46.346 INFO:tasks.workunit.client.0.smithi016.stdout:8/659: dwrite d1/f15 [0,4194304] 0 2024-04-02T04:42:46.350 INFO:tasks.workunit.client.0.smithi016.stdout:4/689: write d5/d16/d88/f8c [251632,12869] 0 2024-04-02T04:42:46.352 INFO:tasks.workunit.client.0.smithi016.stdout:4/690: chown d5/d16/d3f/fc4 2906 1 2024-04-02T04:42:46.360 INFO:tasks.workunit.client.0.smithi016.stdout:6/721: sync 2024-04-02T04:42:46.367 INFO:tasks.workunit.client.1.smithi027.stdout:2/150: rename d1/dd/d17/d2b/f24 to d1/db/f30 0 2024-04-02T04:42:46.369 INFO:tasks.workunit.client.1.smithi027.stdout:8/117: truncate d5/d8/db/dd/f1f 955870 0 2024-04-02T04:42:46.376 INFO:tasks.workunit.client.1.smithi027.stdout:8/118: dwrite d5/d8/db/dd/f1d [0,4194304] 0 2024-04-02T04:42:46.380 INFO:tasks.workunit.client.1.smithi027.stdout:9/141: write d1a/d1d/f27 [833802,40901] 0 2024-04-02T04:42:46.381 INFO:tasks.workunit.client.1.smithi027.stdout:7/151: dread d2/d7/da/dc/d16/f2c [0,4194304] 0 2024-04-02T04:42:46.382 INFO:tasks.workunit.client.1.smithi027.stdout:0/189: link d4/d16/d1b/f20 d4/d3d/f3f 0 2024-04-02T04:42:46.383 INFO:tasks.workunit.client.1.smithi027.stdout:0/190: read d4/fb [1009546,80646] 0 2024-04-02T04:42:46.388 INFO:tasks.workunit.client.1.smithi027.stdout:2/151: rmdir d1/dd/d17/d12 39 2024-04-02T04:42:46.390 INFO:tasks.workunit.client.1.smithi027.stdout:6/151: write d13/d1b/f1f [1217718,21858] 0 2024-04-02T04:42:46.394 INFO:tasks.workunit.client.1.smithi027.stdout:1/120: link d0/de/c11 d0/d1e/d25/c2c 0 2024-04-02T04:42:46.396 INFO:tasks.workunit.client.1.smithi027.stdout:1/121: write d0/f14 [6011740,58700] 0 2024-04-02T04:42:46.397 INFO:tasks.workunit.client.1.smithi027.stdout:8/119: write d5/d7/f24 [5670611,15681] 0 2024-04-02T04:42:46.398 INFO:tasks.workunit.client.1.smithi027.stdout:6/152: dwrite d13/d1b/d24/f34 [0,4194304] 0 2024-04-02T04:42:46.399 INFO:tasks.workunit.client.1.smithi027.stdout:3/108: link d1/de/l12 d1/l23 0 2024-04-02T04:42:46.400 INFO:tasks.workunit.client.1.smithi027.stdout:8/120: write d5/d7/f24 [2570229,5968] 0 2024-04-02T04:42:46.401 INFO:tasks.workunit.client.1.smithi027.stdout:3/109: truncate d1/d1a/f1c 4554299 0 2024-04-02T04:42:46.402 INFO:tasks.workunit.client.1.smithi027.stdout:3/110: fdatasync d1/de/df/f17 0 2024-04-02T04:42:46.403 INFO:tasks.workunit.client.1.smithi027.stdout:3/111: stat d1/de/df/f19 0 2024-04-02T04:42:46.404 INFO:tasks.workunit.client.1.smithi027.stdout:3/112: chown d1/de/c1e 29550729 1 2024-04-02T04:42:46.405 INFO:tasks.workunit.client.1.smithi027.stdout:9/142: mknod d1a/d1b/c2f 0 2024-04-02T04:42:46.406 INFO:tasks.workunit.client.1.smithi027.stdout:3/113: write d1/de/df/f17 [499236,46765] 0 2024-04-02T04:42:46.409 INFO:tasks.workunit.client.1.smithi027.stdout:7/152: write d2/d1d/f20 [1042031,29461] 0 2024-04-02T04:42:46.411 INFO:tasks.workunit.client.1.smithi027.stdout:7/153: read d2/d7/da/f2a [7428763,29823] 0 2024-04-02T04:42:46.413 INFO:tasks.workunit.client.1.smithi027.stdout:9/143: dwrite d1a/f26 [0,4194304] 0 2024-04-02T04:42:46.413 INFO:tasks.workunit.client.1.smithi027.stdout:0/191: creat d4/d21/d28/f40 x:0 0 0 2024-04-02T04:42:46.414 INFO:tasks.workunit.client.1.smithi027.stdout:0/192: dread - d4/d16/f17 zero size 2024-04-02T04:42:46.422 INFO:tasks.workunit.client.1.smithi027.stdout:1/122: mknod d0/d1e/c2d 0 2024-04-02T04:42:46.424 INFO:tasks.workunit.client.0.smithi016.stdout:9/752: rmdir d0/d9/db/d54 39 2024-04-02T04:42:46.426 INFO:tasks.workunit.client.1.smithi027.stdout:6/153: mkdir d13/d1b/d24/d28/d3d 0 2024-04-02T04:42:46.428 INFO:tasks.workunit.client.1.smithi027.stdout:8/121: mknod d5/d8/c25 0 2024-04-02T04:42:46.429 INFO:tasks.workunit.client.0.smithi016.stdout:7/668: chown d0/d46/dbf/fce 696 1 2024-04-02T04:42:46.432 INFO:tasks.workunit.client.1.smithi027.stdout:6/154: dwrite d13/d1b/f1f [0,4194304] 0 2024-04-02T04:42:46.433 INFO:tasks.workunit.client.0.smithi016.stdout:5/644: getdents d3/d40/d4f/d67/dbf 0 2024-04-02T04:42:46.434 INFO:tasks.workunit.client.1.smithi027.stdout:3/114: rmdir d1/de 39 2024-04-02T04:42:46.439 INFO:tasks.workunit.client.1.smithi027.stdout:7/154: creat d2/d7/da/dc/f30 x:0 0 0 2024-04-02T04:42:46.440 INFO:tasks.workunit.client.1.smithi027.stdout:7/155: chown f1 196580369 1 2024-04-02T04:42:46.441 INFO:tasks.workunit.client.1.smithi027.stdout:7/156: write d2/d7/da/dc/d16/f2d [310951,35162] 0 2024-04-02T04:42:46.445 INFO:tasks.workunit.client.0.smithi016.stdout:8/660: rename d1/d31/d67/fb5 to d1/d6/d36/d3f/d40/d63/dc5/fe0 0 2024-04-02T04:42:46.447 INFO:tasks.workunit.client.0.smithi016.stdout:2/740: write d12/d70/ff3 [591787,16462] 0 2024-04-02T04:42:46.448 INFO:tasks.workunit.client.0.smithi016.stdout:6/722: truncate d0/d9/d16/f63 3475767 0 2024-04-02T04:42:46.450 INFO:tasks.workunit.client.0.smithi016.stdout:0/644: write d0/d16/d2e/d37/f3b [4296929,89388] 0 2024-04-02T04:42:46.451 INFO:tasks.workunit.client.1.smithi027.stdout:5/215: sync 2024-04-02T04:42:46.451 INFO:tasks.workunit.client.1.smithi027.stdout:6/155: creat d13/d1b/d24/d25/d27/f3e x:0 0 0 2024-04-02T04:42:46.451 INFO:tasks.workunit.client.0.smithi016.stdout:8/661: dwrite d1/d31/d67/d8f/fbe [0,4194304] 0 2024-04-02T04:42:46.453 INFO:tasks.workunit.client.0.smithi016.stdout:8/662: truncate d1/d6/d36/d3f/fb4 253480 0 2024-04-02T04:42:46.454 INFO:tasks.workunit.client.0.smithi016.stdout:9/753: mknod d0/d9/d4d/d78/da6/dc1/ce9 0 2024-04-02T04:42:46.455 INFO:tasks.workunit.client.1.smithi027.stdout:3/115: write d1/de/df/d13/f16 [291978,104249] 0 2024-04-02T04:42:46.455 INFO:tasks.workunit.client.1.smithi027.stdout:0/193: write d4/f11 [3320612,123523] 0 2024-04-02T04:42:46.457 INFO:tasks.workunit.client.0.smithi016.stdout:7/669: symlink d0/d46/d37/dbe/dae/d9d/leb 0 2024-04-02T04:42:46.459 INFO:tasks.workunit.client.0.smithi016.stdout:7/670: write d0/d46/d16/fc5 [733145,90563] 0 2024-04-02T04:42:46.460 INFO:tasks.workunit.client.1.smithi027.stdout:7/157: creat d2/d7/da/f31 x:0 0 0 2024-04-02T04:42:46.463 INFO:tasks.workunit.client.1.smithi027.stdout:0/194: dread d4/f14 [0,4194304] 0 2024-04-02T04:42:46.463 INFO:tasks.workunit.client.0.smithi016.stdout:1/656: link d14/d19/f60 d14/d19/d1d/d29/d4c/fd6 0 2024-04-02T04:42:46.464 INFO:tasks.workunit.client.1.smithi027.stdout:3/116: dwrite d1/d1a/f1c [0,4194304] 0 2024-04-02T04:42:46.464 INFO:tasks.workunit.client.0.smithi016.stdout:5/645: creat d3/df/dd5/fef x:0 0 0 2024-04-02T04:42:46.468 INFO:tasks.workunit.client.1.smithi027.stdout:5/216: rename d0/l4 to d0/df/d2f/d3b/d51/l52 0 2024-04-02T04:42:46.480 INFO:tasks.workunit.client.1.smithi027.stdout:2/152: getdents d1/dd 0 2024-04-02T04:42:46.481 INFO:tasks.workunit.client.0.smithi016.stdout:0/645: chown d0/d16/d2e/d53/c78 8 1 2024-04-02T04:42:46.481 INFO:tasks.workunit.client.0.smithi016.stdout:0/646: stat d0/d16/d23/c4a 0 2024-04-02T04:42:46.482 INFO:tasks.workunit.client.1.smithi027.stdout:2/153: dread d1/dd/d17/f2c [0,4194304] 0 2024-04-02T04:42:46.482 INFO:tasks.workunit.client.1.smithi027.stdout:3/117: creat d1/de/df/f24 x:0 0 0 2024-04-02T04:42:46.483 INFO:tasks.workunit.client.0.smithi016.stdout:8/663: creat d1/d70/d71/d74/fe1 x:0 0 0 2024-04-02T04:42:46.485 INFO:tasks.workunit.client.1.smithi027.stdout:8/122: write d5/d8/db/dd/f1f [1900157,24436] 0 2024-04-02T04:42:46.489 INFO:tasks.workunit.client.0.smithi016.stdout:7/671: rmdir d0/d46/d52 39 2024-04-02T04:42:46.490 INFO:tasks.workunit.client.0.smithi016.stdout:1/657: dread d14/d19/d20/f38 [4194304,4194304] 0 2024-04-02T04:42:46.492 INFO:tasks.workunit.client.0.smithi016.stdout:6/723: write d0/d9/f32 [4926137,55804] 0 2024-04-02T04:42:46.496 INFO:tasks.workunit.client.0.smithi016.stdout:0/647: dread d0/d16/d2e/f85 [0,4194304] 0 2024-04-02T04:42:46.502 INFO:tasks.workunit.client.0.smithi016.stdout:2/741: mkdir d12/d35/d3f/d2b/de9/d102 0 2024-04-02T04:42:46.504 INFO:tasks.workunit.client.0.smithi016.stdout:8/664: read - d1/d6/d36/d3f/d40/d49/d38/f6f zero size 2024-04-02T04:42:46.508 INFO:tasks.workunit.client.0.smithi016.stdout:1/658: symlink d14/d19/d1d/d29/d39/d4d/ld7 0 2024-04-02T04:42:46.509 INFO:tasks.workunit.client.0.smithi016.stdout:1/659: write d14/d19/d1d/d29/d4c/fd2 [234538,45713] 0 2024-04-02T04:42:46.509 INFO:tasks.workunit.client.0.smithi016.stdout:6/724: dread - d0/d9/df/fa6 zero size 2024-04-02T04:42:46.511 INFO:tasks.workunit.client.0.smithi016.stdout:5/646: creat d3/d40/d4f/d67/dbf/ff0 x:0 0 0 2024-04-02T04:42:46.513 INFO:tasks.workunit.client.0.smithi016.stdout:4/691: getdents d5/d17/d18 0 2024-04-02T04:42:46.515 INFO:tasks.workunit.client.0.smithi016.stdout:9/754: creat d0/d9/db/fea x:0 0 0 2024-04-02T04:42:46.517 INFO:tasks.workunit.client.0.smithi016.stdout:9/755: dread - d0/d4a/d99/dd3/fe2 zero size 2024-04-02T04:42:46.517 INFO:tasks.workunit.client.0.smithi016.stdout:3/774: dread d2/d15/d16/d8a/dd1/d6e/d6f/db0/f83 [0,4194304] 0 2024-04-02T04:42:46.517 INFO:tasks.workunit.client.0.smithi016.stdout:7/672: mkdir d0/d46/d37/d3a/dec 0 2024-04-02T04:42:46.518 INFO:tasks.workunit.client.0.smithi016.stdout:3/775: dread - d2/d15/d16/d8a/dd1/fe3 zero size 2024-04-02T04:42:46.519 INFO:tasks.workunit.client.0.smithi016.stdout:1/660: symlink d14/d19/d1d/d29/d39/d46/ld8 0 2024-04-02T04:42:46.521 INFO:tasks.workunit.client.0.smithi016.stdout:0/648: mknod d0/d16/d2e/cdb 0 2024-04-02T04:42:46.522 INFO:tasks.workunit.client.0.smithi016.stdout:6/725: dread d0/d9/d16/d56/d89/d67/d6f/f82 [0,4194304] 0 2024-04-02T04:42:46.522 INFO:tasks.workunit.client.0.smithi016.stdout:2/742: mkdir d12/d35/d3f/d2b/de9/d102/d103 0 2024-04-02T04:42:46.524 INFO:tasks.workunit.client.0.smithi016.stdout:4/692: read d5/f7 [1058640,101115] 0 2024-04-02T04:42:46.526 INFO:tasks.workunit.client.0.smithi016.stdout:9/756: unlink d0/l5e 0 2024-04-02T04:42:46.529 INFO:tasks.workunit.client.0.smithi016.stdout:1/661: symlink d14/d19/d1d/d45/d4f/d86/ld9 0 2024-04-02T04:42:46.530 INFO:tasks.workunit.client.0.smithi016.stdout:5/647: symlink d3/df/d18/d26/lf1 0 2024-04-02T04:42:46.532 INFO:tasks.workunit.client.0.smithi016.stdout:0/649: write d0/d16/d24/d4e/d8d/d71/f7d [284580,25887] 0 2024-04-02T04:42:46.533 INFO:tasks.workunit.client.0.smithi016.stdout:6/726: mkdir d0/d9/d16/de3 0 2024-04-02T04:42:46.534 INFO:tasks.workunit.client.0.smithi016.stdout:2/743: mkdir d12/d35/d3b/d42/d59/d104 0 2024-04-02T04:42:46.538 INFO:tasks.workunit.client.0.smithi016.stdout:9/757: symlink d0/d4a/d75/d8d/leb 0 2024-04-02T04:42:46.538 INFO:tasks.workunit.client.0.smithi016.stdout:0/650: dwrite d0/d16/f80 [0,4194304] 0 2024-04-02T04:42:46.539 INFO:tasks.workunit.client.0.smithi016.stdout:3/776: mknod d2/d2e/dee/c104 0 2024-04-02T04:42:46.543 INFO:tasks.workunit.client.0.smithi016.stdout:6/727: dwrite d0/d9/d16/d56/d89/d67/fdf [0,4194304] 0 2024-04-02T04:42:46.543 INFO:tasks.workunit.client.0.smithi016.stdout:5/648: dread - d3/d40/d4f/d6b/f9e zero size 2024-04-02T04:42:46.547 INFO:tasks.workunit.client.0.smithi016.stdout:8/665: rename d1/f19 to d1/d6/d36/d3f/d40/d45/fe2 0 2024-04-02T04:42:46.547 INFO:tasks.workunit.client.0.smithi016.stdout:7/673: write d0/d46/d37/d3a/d4c/fb0 [4527964,9305] 0 2024-04-02T04:42:46.548 INFO:tasks.workunit.client.0.smithi016.stdout:4/693: symlink d5/d1d/d5f/ld0 0 2024-04-02T04:42:46.566 INFO:tasks.workunit.client.1.smithi027.stdout:9/144: getdents d1a/d1d 0 2024-04-02T04:42:46.567 INFO:tasks.workunit.client.1.smithi027.stdout:6/156: write d13/f18 [373903,9468] 0 2024-04-02T04:42:46.567 INFO:tasks.workunit.client.1.smithi027.stdout:3/118: creat d1/de/df/d13/f25 x:0 0 0 2024-04-02T04:42:46.569 INFO:tasks.workunit.client.1.smithi027.stdout:8/123: mknod d5/d7/c26 0 2024-04-02T04:42:46.570 INFO:tasks.workunit.client.1.smithi027.stdout:6/157: dread - d13/d1b/d24/d28/f2a zero size 2024-04-02T04:42:46.570 INFO:tasks.workunit.client.1.smithi027.stdout:2/154: stat d1/dd/d17/d12/l28 0 2024-04-02T04:42:46.570 INFO:tasks.workunit.client.1.smithi027.stdout:3/119: chown d1/de/df/c11 170529 1 2024-04-02T04:42:46.570 INFO:tasks.workunit.client.1.smithi027.stdout:9/145: creat d1a/d1b/d20/f30 x:0 0 0 2024-04-02T04:42:46.572 INFO:tasks.workunit.client.0.smithi016.stdout:1/662: dread d14/d19/d1d/d29/f3d [0,4194304] 0 2024-04-02T04:42:46.572 INFO:tasks.workunit.client.1.smithi027.stdout:6/158: write d13/f18 [971698,76636] 0 2024-04-02T04:42:46.572 INFO:tasks.workunit.client.1.smithi027.stdout:7/158: getdents d2/d1d 0 2024-04-02T04:42:46.574 INFO:tasks.workunit.client.1.smithi027.stdout:5/217: truncate d0/df/d11/d13/f26 944410 0 2024-04-02T04:42:46.575 INFO:tasks.workunit.client.1.smithi027.stdout:6/159: dread d13/f16 [0,4194304] 0 2024-04-02T04:42:46.575 INFO:tasks.workunit.client.0.smithi016.stdout:8/666: creat d1/d31/fe3 x:0 0 0 2024-04-02T04:42:46.577 INFO:tasks.workunit.client.1.smithi027.stdout:9/146: mknod d1a/d1b/d20/c31 0 2024-04-02T04:42:46.577 INFO:tasks.workunit.client.0.smithi016.stdout:3/777: dwrite d2/d2e/d36/d48/da2/dcb/f101 [0,4194304] 0 2024-04-02T04:42:46.577 INFO:tasks.workunit.client.1.smithi027.stdout:9/147: truncate d1a/f26 4253289 0 2024-04-02T04:42:46.578 INFO:tasks.workunit.client.1.smithi027.stdout:2/155: mknod d1/dd/d17/d2b/c31 0 2024-04-02T04:42:46.579 INFO:tasks.workunit.client.0.smithi016.stdout:6/728: dwrite d0/d9/d16/f2a [8388608,4194304] 0 2024-04-02T04:42:46.579 INFO:tasks.workunit.client.0.smithi016.stdout:2/744: creat d12/d35/f105 x:0 0 0 2024-04-02T04:42:46.580 INFO:tasks.workunit.client.1.smithi027.stdout:7/159: write d2/f13 [1025149,119246] 0 2024-04-02T04:42:46.582 INFO:tasks.workunit.client.0.smithi016.stdout:7/674: fdatasync d0/d46/d52/d82/fb3 0 2024-04-02T04:42:46.582 INFO:tasks.workunit.client.1.smithi027.stdout:3/120: truncate d1/f7 2679288 0 2024-04-02T04:42:46.583 INFO:tasks.workunit.client.0.smithi016.stdout:6/729: chown d0/d9/d16/cda 1962744669 1 2024-04-02T04:42:46.583 INFO:tasks.workunit.client.1.smithi027.stdout:3/121: fdatasync d1/de/df/d13/f1b 0 2024-04-02T04:42:46.584 INFO:tasks.workunit.client.0.smithi016.stdout:1/663: symlink d14/d19/daf/lda 0 2024-04-02T04:42:46.584 INFO:tasks.workunit.client.0.smithi016.stdout:4/694: dread d5/d17/d18/d1f/f70 [0,4194304] 0 2024-04-02T04:42:46.585 INFO:tasks.workunit.client.1.smithi027.stdout:5/218: creat d0/df/d2f/d3b/d51/f53 x:0 0 0 2024-04-02T04:42:46.585 INFO:tasks.workunit.client.0.smithi016.stdout:2/745: dread - d12/d9f/fb8 zero size 2024-04-02T04:42:46.586 INFO:tasks.workunit.client.0.smithi016.stdout:5/649: creat d3/df/d18/ff2 x:0 0 0 2024-04-02T04:42:46.587 INFO:tasks.workunit.client.0.smithi016.stdout:1/664: dread - d14/d19/d7a/fab zero size 2024-04-02T04:42:46.587 INFO:tasks.workunit.client.1.smithi027.stdout:6/160: unlink d13/d1b/d24/f34 0 2024-04-02T04:42:46.592 INFO:tasks.workunit.client.1.smithi027.stdout:6/161: dread d13/d1b/f1f [0,4194304] 0 2024-04-02T04:42:46.593 INFO:tasks.workunit.client.1.smithi027.stdout:2/156: creat d1/db/f32 x:0 0 0 2024-04-02T04:42:46.594 INFO:tasks.workunit.client.0.smithi016.stdout:4/695: dwrite d5/f60 [0,4194304] 0 2024-04-02T04:42:46.595 INFO:tasks.workunit.client.0.smithi016.stdout:4/696: fsync d5/d17/d18/d6d/da4/fad 0 2024-04-02T04:42:46.601 INFO:tasks.workunit.client.0.smithi016.stdout:1/665: dread d14/d19/d1e/d5e/fac [0,4194304] 0 2024-04-02T04:42:46.602 INFO:tasks.workunit.client.0.smithi016.stdout:7/675: readlink d0/d46/d37/dbe/dae/lab 0 2024-04-02T04:42:46.603 INFO:tasks.workunit.client.0.smithi016.stdout:1/666: truncate d14/d19/f69 5062886 0 2024-04-02T04:42:46.604 INFO:tasks.workunit.client.0.smithi016.stdout:7/676: dread - d0/d46/d37/dbe/fbc zero size 2024-04-02T04:42:46.606 INFO:tasks.workunit.client.0.smithi016.stdout:1/667: truncate d14/d19/d1d/d45/d4f/d86/fa6 999905 0 2024-04-02T04:42:46.610 INFO:tasks.workunit.client.0.smithi016.stdout:2/746: dread - d12/d35/d101/d76/da0/de7/fbd zero size 2024-04-02T04:42:46.611 INFO:tasks.workunit.client.1.smithi027.stdout:5/219: mknod d0/df/d11/d41/c54 0 2024-04-02T04:42:46.615 INFO:tasks.workunit.client.0.smithi016.stdout:3/778: dwrite d2/fa [0,4194304] 0 2024-04-02T04:42:46.619 INFO:tasks.workunit.client.0.smithi016.stdout:4/697: creat d5/d16/d88/db0/fd1 x:0 0 0 2024-04-02T04:42:46.620 INFO:tasks.workunit.client.0.smithi016.stdout:5/650: dread d3/df/d18/d26/ddc/f5f [0,4194304] 0 2024-04-02T04:42:46.622 INFO:tasks.workunit.client.1.smithi027.stdout:6/162: creat d13/d1b/d24/d25/d27/f3f x:0 0 0 2024-04-02T04:42:46.623 INFO:tasks.workunit.client.1.smithi027.stdout:4/148: sync 2024-04-02T04:42:46.623 INFO:tasks.workunit.client.0.smithi016.stdout:6/730: write d0/d9/f7a [518191,78036] 0 2024-04-02T04:42:46.624 INFO:tasks.workunit.client.1.smithi027.stdout:4/149: dread - d24/f28 zero size 2024-04-02T04:42:46.624 INFO:tasks.workunit.client.1.smithi027.stdout:4/150: readlink l10 0 2024-04-02T04:42:46.625 INFO:tasks.workunit.client.1.smithi027.stdout:9/148: truncate d1a/d1b/f1c 8066166 0 2024-04-02T04:42:46.627 INFO:tasks.workunit.client.1.smithi027.stdout:8/124: getdents d5/d8/db 0 2024-04-02T04:42:46.627 INFO:tasks.workunit.client.0.smithi016.stdout:8/667: link d1/d6/f2e d1/d70/d71/d74/d82/fe4 0 2024-04-02T04:42:46.628 INFO:tasks.workunit.client.1.smithi027.stdout:3/122: symlink d1/l26 0 2024-04-02T04:42:46.631 INFO:tasks.workunit.client.1.smithi027.stdout:5/220: fsync d0/f33 0 2024-04-02T04:42:46.632 INFO:tasks.workunit.client.1.smithi027.stdout:2/157: symlink d1/dd/d17/l33 0 2024-04-02T04:42:46.633 INFO:tasks.workunit.client.1.smithi027.stdout:8/125: dwrite d5/d7/f14 [0,4194304] 0 2024-04-02T04:42:46.633 INFO:tasks.workunit.client.0.smithi016.stdout:7/677: dread d0/d46/d52/d82/f99 [0,4194304] 0 2024-04-02T04:42:46.634 INFO:tasks.workunit.client.0.smithi016.stdout:7/678: read - d0/d46/d37/dbe/dae/d9d/fa3 zero size 2024-04-02T04:42:46.635 INFO:tasks.workunit.client.1.smithi027.stdout:6/163: symlink d13/d1b/d24/d28/l40 0 2024-04-02T04:42:46.636 INFO:tasks.workunit.client.1.smithi027.stdout:6/164: write d13/f14 [273104,14576] 0 2024-04-02T04:42:46.637 INFO:tasks.workunit.client.1.smithi027.stdout:6/165: fdatasync f7 0 2024-04-02T04:42:46.639 INFO:tasks.workunit.client.1.smithi027.stdout:6/166: write d13/d1b/d24/d28/f29 [421561,123983] 0 2024-04-02T04:42:46.640 INFO:tasks.workunit.client.1.smithi027.stdout:6/167: stat d13/l33 0 2024-04-02T04:42:46.641 INFO:tasks.workunit.client.0.smithi016.stdout:2/747: mkdir d12/d35/d101/d76/da0/de7/dfc/d106 0 2024-04-02T04:42:46.642 INFO:tasks.workunit.client.1.smithi027.stdout:4/151: rename d24/f26 to d24/f2a 0 2024-04-02T04:42:46.644 INFO:tasks.workunit.client.1.smithi027.stdout:9/149: creat d1a/f32 x:0 0 0 2024-04-02T04:42:46.646 INFO:tasks.workunit.client.0.smithi016.stdout:2/748: dread d12/d35/d3f/d56/f81 [0,4194304] 0 2024-04-02T04:42:46.647 INFO:tasks.workunit.client.0.smithi016.stdout:2/749: chown d12/d35 4 1 2024-04-02T04:42:46.647 INFO:tasks.workunit.client.1.smithi027.stdout:9/150: truncate d1a/d1d/f27 1329227 0 2024-04-02T04:42:46.648 INFO:tasks.workunit.client.1.smithi027.stdout:6/168: dwrite d13/d1b/f20 [0,4194304] 0 2024-04-02T04:42:46.648 INFO:tasks.workunit.client.1.smithi027.stdout:6/169: chown d13/f17 269416516 1 2024-04-02T04:42:46.649 INFO:tasks.workunit.client.0.smithi016.stdout:4/698: truncate d5/d17/d18/d1f/f2a 598585 0 2024-04-02T04:42:46.649 INFO:tasks.workunit.client.0.smithi016.stdout:2/750: write d12/d35/d3b/d42/d59/d91/fff [148840,129725] 0 2024-04-02T04:42:46.650 INFO:tasks.workunit.client.1.smithi027.stdout:3/123: readlink d1/de/l12 0 2024-04-02T04:42:46.651 INFO:tasks.workunit.client.1.smithi027.stdout:6/170: write d13/d1b/d24/d25/d27/f3f [652262,66017] 0 2024-04-02T04:42:46.652 INFO:tasks.workunit.client.1.smithi027.stdout:6/171: truncate d13/d1b/f1f 4735218 0 2024-04-02T04:42:46.654 INFO:tasks.workunit.client.0.smithi016.stdout:6/731: dwrite d0/d9/d16/d56/d59/f86 [0,4194304] 0 2024-04-02T04:42:46.658 INFO:tasks.workunit.client.0.smithi016.stdout:5/651: truncate d3/f36 3498461 0 2024-04-02T04:42:46.658 INFO:tasks.workunit.client.0.smithi016.stdout:3/779: write d2/d15/f23 [2556450,68136] 0 2024-04-02T04:42:46.660 INFO:tasks.workunit.client.0.smithi016.stdout:2/751: dwrite d12/d35/d3f/d56/dab/f37 [0,4194304] 0 2024-04-02T04:42:46.661 INFO:tasks.workunit.client.0.smithi016.stdout:3/780: truncate d2/d2e/d36/d66/d90/fdf 580767 0 2024-04-02T04:42:46.663 INFO:tasks.workunit.client.0.smithi016.stdout:8/668: symlink d1/d6/d36/d3f/d40/le5 0 2024-04-02T04:42:46.665 INFO:tasks.workunit.client.1.smithi027.stdout:9/151: mknod d1a/c33 0 2024-04-02T04:42:46.667 INFO:tasks.workunit.client.0.smithi016.stdout:4/699: unlink d5/d8/f8f 0 2024-04-02T04:42:46.670 INFO:tasks.workunit.client.0.smithi016.stdout:8/669: dwrite d1/d70/d71/d74/fa6 [0,4194304] 0 2024-04-02T04:42:46.674 INFO:tasks.workunit.client.1.smithi027.stdout:5/221: mkdir d0/d5/d55 0 2024-04-02T04:42:46.676 INFO:tasks.workunit.client.1.smithi027.stdout:7/160: getdents d2/df 0 2024-04-02T04:42:46.683 INFO:tasks.workunit.client.0.smithi016.stdout:2/752: rename d12/d35/d3b/d42/d67/d6a to d12/d9f/d107 0 2024-04-02T04:42:46.685 INFO:tasks.workunit.client.0.smithi016.stdout:3/781: creat d2/d15/d1a/d45/f105 x:0 0 0 2024-04-02T04:42:46.686 INFO:tasks.workunit.client.0.smithi016.stdout:3/782: write d2/d15/d16/d8a/f98 [284709,60209] 0 2024-04-02T04:42:46.687 INFO:tasks.workunit.client.1.smithi027.stdout:2/158: rename d1/dd/d17/c1e to d1/dd/d17/c34 0 2024-04-02T04:42:46.690 INFO:tasks.workunit.client.0.smithi016.stdout:7/679: truncate d0/d46/f2c 1914697 0 2024-04-02T04:42:46.691 INFO:tasks.workunit.client.1.smithi027.stdout:5/222: dwrite d0/d1d/f31 [0,4194304] 0 2024-04-02T04:42:46.691 INFO:tasks.workunit.client.1.smithi027.stdout:7/161: symlink d2/d7/l32 0 2024-04-02T04:42:46.694 INFO:tasks.workunit.client.1.smithi027.stdout:9/152: rename c4 to d1a/d1b/d20/c34 0 2024-04-02T04:42:46.695 INFO:tasks.workunit.client.1.smithi027.stdout:9/153: chown d1a/d1d 378430200 1 2024-04-02T04:42:46.695 INFO:tasks.workunit.client.0.smithi016.stdout:8/670: rmdir d1/d70/d71/d9e 39 2024-04-02T04:42:46.696 INFO:tasks.workunit.client.1.smithi027.stdout:9/154: dread - d1a/d1b/d20/f2b zero size 2024-04-02T04:42:46.697 INFO:tasks.workunit.client.0.smithi016.stdout:5/652: dwrite d3/d40/d4f/d6b/f47 [0,4194304] 0 2024-04-02T04:42:46.698 INFO:tasks.workunit.client.0.smithi016.stdout:7/680: dwrite d0/d46/f2f [0,4194304] 0 2024-04-02T04:42:46.700 INFO:tasks.workunit.client.1.smithi027.stdout:6/172: creat d13/d1b/d24/f41 x:0 0 0 2024-04-02T04:42:46.700 INFO:tasks.workunit.client.0.smithi016.stdout:8/671: dread - d1/d6/d36/d3f/d40/d63/dc5/fe0 zero size 2024-04-02T04:42:46.702 INFO:tasks.workunit.client.1.smithi027.stdout:2/159: symlink d1/l35 0 2024-04-02T04:42:46.704 INFO:tasks.workunit.client.1.smithi027.stdout:5/223: mknod d0/df/d11/c56 0 2024-04-02T04:42:46.707 INFO:tasks.workunit.client.1.smithi027.stdout:5/224: write d0/df/d11/f1f [3093788,71387] 0 2024-04-02T04:42:46.710 INFO:tasks.workunit.client.1.smithi027.stdout:7/162: mkdir d2/d7/da/dc/d1f/d33 0 2024-04-02T04:42:46.711 INFO:tasks.workunit.client.0.smithi016.stdout:3/783: mknod d2/d15/d16/d8a/dd1/d6e/d6f/db0/c106 0 2024-04-02T04:42:46.711 INFO:tasks.workunit.client.1.smithi027.stdout:7/163: truncate d2/d1d/f20 1462324 0 2024-04-02T04:42:46.711 INFO:tasks.workunit.client.1.smithi027.stdout:9/155: creat d1a/d1b/f35 x:0 0 0 2024-04-02T04:42:46.711 INFO:tasks.workunit.client.1.smithi027.stdout:3/124: getdents d1/de/df/d13 0 2024-04-02T04:42:46.712 INFO:tasks.workunit.client.1.smithi027.stdout:3/125: read d1/de/df/f17 [446630,22035] 0 2024-04-02T04:42:46.715 INFO:tasks.workunit.client.0.smithi016.stdout:6/732: creat d0/fe4 x:0 0 0 2024-04-02T04:42:46.715 INFO:tasks.workunit.client.0.smithi016.stdout:6/733: chown d0/d78 1107469 1 2024-04-02T04:42:46.716 INFO:tasks.workunit.client.1.smithi027.stdout:2/160: rmdir d1 39 2024-04-02T04:42:46.717 INFO:tasks.workunit.client.0.smithi016.stdout:5/653: rmdir d3/d40/d72 39 2024-04-02T04:42:46.720 INFO:tasks.workunit.client.1.smithi027.stdout:5/225: mknod d0/d1d/c57 0 2024-04-02T04:42:46.722 INFO:tasks.workunit.client.0.smithi016.stdout:4/700: rename f4 to d5/d17/dcd/fd2 0 2024-04-02T04:42:46.724 INFO:tasks.workunit.client.0.smithi016.stdout:4/701: write d5/d16/d88/f96 [785824,54001] 0 2024-04-02T04:42:46.726 INFO:tasks.workunit.client.1.smithi027.stdout:1/123: sync 2024-04-02T04:42:46.727 INFO:tasks.workunit.client.0.smithi016.stdout:4/702: chown d5/d17/d18/d6d/d8e/laf 76 1 2024-04-02T04:42:46.731 INFO:tasks.workunit.client.0.smithi016.stdout:7/681: write d0/d46/d37/d3a/f7d [500012,60812] 0 2024-04-02T04:42:46.731 INFO:tasks.workunit.client.0.smithi016.stdout:6/734: creat d0/d9/d16/d56/d89/d67/d6f/d8a/fe5 x:0 0 0 2024-04-02T04:42:46.733 INFO:tasks.workunit.client.1.smithi027.stdout:7/164: write d2/d1d/f25 [996334,77149] 0 2024-04-02T04:42:46.735 INFO:tasks.workunit.client.0.smithi016.stdout:5/654: mknod d3/d40/d4f/d9c/cf3 0 2024-04-02T04:42:46.737 INFO:tasks.workunit.client.0.smithi016.stdout:8/672: dwrite d1/d6/d36/d3f/d40/d63/dc5/fe0 [0,4194304] 0 2024-04-02T04:42:46.738 INFO:tasks.workunit.client.1.smithi027.stdout:7/165: dread d2/d1d/f25 [0,4194304] 0 2024-04-02T04:42:46.738 INFO:tasks.workunit.client.0.smithi016.stdout:3/784: dread d2/fab [0,4194304] 0 2024-04-02T04:42:46.739 INFO:tasks.workunit.client.0.smithi016.stdout:5/655: read - d3/d40/d4f/d6b/f9e zero size 2024-04-02T04:42:46.739 INFO:tasks.workunit.client.1.smithi027.stdout:9/156: dwrite f15 [0,4194304] 0 2024-04-02T04:42:46.740 INFO:tasks.workunit.client.1.smithi027.stdout:9/157: fsync d1a/d1d/f24 0 2024-04-02T04:42:46.741 INFO:tasks.workunit.client.1.smithi027.stdout:1/124: unlink d0/de/f1f 0 2024-04-02T04:42:46.742 INFO:tasks.workunit.client.1.smithi027.stdout:6/173: truncate d13/d1b/f1f 4125795 0 2024-04-02T04:42:46.743 INFO:tasks.workunit.client.1.smithi027.stdout:6/174: fsync d13/d1b/d24/d25/d27/f3f 0 2024-04-02T04:42:46.744 INFO:tasks.workunit.client.0.smithi016.stdout:8/673: dwrite d1/d6/d36/f99 [0,4194304] 0 2024-04-02T04:42:46.749 INFO:tasks.workunit.client.0.smithi016.stdout:8/674: stat d1/d6/d36/d3f/d40/d45/fb9 0 2024-04-02T04:42:46.750 INFO:tasks.workunit.client.0.smithi016.stdout:2/753: link d12/d35/d3b/d42/cad d12/d35/d101/d76/dcf/c108 0 2024-04-02T04:42:46.750 INFO:tasks.workunit.client.1.smithi027.stdout:2/161: symlink d1/dd/d17/d12/l36 0 2024-04-02T04:42:46.751 INFO:tasks.workunit.client.1.smithi027.stdout:7/166: rmdir d2/df 39 2024-04-02T04:42:46.755 INFO:tasks.workunit.client.1.smithi027.stdout:9/158: symlink d1a/d1d/d29/l36 0 2024-04-02T04:42:46.756 INFO:tasks.workunit.client.1.smithi027.stdout:9/159: fsync d1a/d1b/d20/f30 0 2024-04-02T04:42:46.757 INFO:tasks.workunit.client.1.smithi027.stdout:9/160: readlink d1a/d1d/d29/l36 0 2024-04-02T04:42:46.758 INFO:tasks.workunit.client.0.smithi016.stdout:7/682: rename d0/d46/d37/dbe/f7b to d0/d21/fed 0 2024-04-02T04:42:46.760 INFO:tasks.workunit.client.1.smithi027.stdout:1/125: mkdir d0/d1e/d2e 0 2024-04-02T04:42:46.760 INFO:tasks.workunit.client.1.smithi027.stdout:1/126: chown d0/d1e/d25 50887719 1 2024-04-02T04:42:46.763 INFO:tasks.workunit.client.0.smithi016.stdout:2/754: rmdir d12/d35/d101/d76 39 2024-04-02T04:42:46.767 INFO:tasks.workunit.client.0.smithi016.stdout:8/675: truncate d1/d6/d36/f37 468036 0 2024-04-02T04:42:46.768 INFO:tasks.workunit.client.1.smithi027.stdout:3/126: link d1/de/df/c15 d1/de/df/c27 0 2024-04-02T04:42:46.771 INFO:tasks.workunit.client.0.smithi016.stdout:3/785: creat d2/d15/d1a/dd5/f107 x:0 0 0 2024-04-02T04:42:46.772 INFO:tasks.workunit.client.0.smithi016.stdout:3/786: chown d2/d2e/d36/d48/fe1 1260543481 1 2024-04-02T04:42:46.774 INFO:tasks.workunit.client.0.smithi016.stdout:7/683: rename d0/d46/le to d0/d46/da2/lee 0 2024-04-02T04:42:46.775 INFO:tasks.workunit.client.1.smithi027.stdout:5/226: dwrite d0/d5/f2c [0,4194304] 0 2024-04-02T04:42:46.776 INFO:tasks.workunit.client.1.smithi027.stdout:2/162: creat d1/dd/f37 x:0 0 0 2024-04-02T04:42:46.778 INFO:tasks.workunit.client.0.smithi016.stdout:6/735: creat d0/d9/d16/d56/fe6 x:0 0 0 2024-04-02T04:42:46.780 INFO:tasks.workunit.client.0.smithi016.stdout:9/758: sync 2024-04-02T04:42:46.781 INFO:tasks.workunit.client.0.smithi016.stdout:7/684: dwrite d0/d46/fc [0,4194304] 0 2024-04-02T04:42:46.783 INFO:tasks.workunit.client.1.smithi027.stdout:1/127: creat d0/d1e/f2f x:0 0 0 2024-04-02T04:42:46.784 INFO:tasks.workunit.client.0.smithi016.stdout:6/736: dread - d0/d9/d16/d56/d89/fc7 zero size 2024-04-02T04:42:46.786 INFO:tasks.workunit.client.0.smithi016.stdout:5/656: write d3/df/d18/d26/d48/f49 [1938695,42387] 0 2024-04-02T04:42:46.789 INFO:tasks.workunit.client.1.smithi027.stdout:5/227: creat d0/df/d11/f58 x:0 0 0 2024-04-02T04:42:46.790 INFO:tasks.workunit.client.1.smithi027.stdout:5/228: readlink d0/df/d2f/d3b/l4c 0 2024-04-02T04:42:46.791 INFO:tasks.workunit.client.0.smithi016.stdout:9/759: dwrite d0/d9/f2e [0,4194304] 0 2024-04-02T04:42:46.794 INFO:tasks.workunit.client.0.smithi016.stdout:7/685: mknod d0/d21/cef 0 2024-04-02T04:42:46.795 INFO:tasks.workunit.client.0.smithi016.stdout:6/737: unlink d0/d9/d16/d56/d89/f85 0 2024-04-02T04:42:46.799 INFO:tasks.workunit.client.1.smithi027.stdout:6/175: rmdir d13/d1b/d24/d28/d3d 0 2024-04-02T04:42:46.800 INFO:tasks.workunit.client.1.smithi027.stdout:8/126: sync 2024-04-02T04:42:46.800 INFO:tasks.workunit.client.1.smithi027.stdout:0/195: sync 2024-04-02T04:42:46.801 INFO:tasks.workunit.client.1.smithi027.stdout:6/176: chown d13/d1b/d24/d28/f39 99 1 2024-04-02T04:42:46.802 INFO:tasks.workunit.client.1.smithi027.stdout:8/127: read d5/d8/db/dd/f1d [1524863,99141] 0 2024-04-02T04:42:46.802 INFO:tasks.workunit.client.1.smithi027.stdout:0/196: fdatasync d4/d16/d2b/f24 0 2024-04-02T04:42:46.803 INFO:tasks.workunit.client.0.smithi016.stdout:9/760: dwrite d0/d9/db/dc9/fdf [0,4194304] 0 2024-04-02T04:42:46.804 INFO:tasks.workunit.client.1.smithi027.stdout:7/167: symlink d2/d1c/l34 0 2024-04-02T04:42:46.805 INFO:tasks.workunit.client.1.smithi027.stdout:8/128: write d5/d8/f17 [1343887,21387] 0 2024-04-02T04:42:46.809 INFO:tasks.workunit.client.0.smithi016.stdout:6/738: rename d0/d9/d16/d56/l9d to d0/d9/d16/d56/d89/db4/le7 0 2024-04-02T04:42:46.811 INFO:tasks.workunit.client.1.smithi027.stdout:5/229: dwrite d0/df/d11/d37/f3a [0,4194304] 0 2024-04-02T04:42:46.814 INFO:tasks.workunit.client.0.smithi016.stdout:5/657: creat d3/d40/d4f/ff4 x:0 0 0 2024-04-02T04:42:46.819 INFO:tasks.workunit.client.0.smithi016.stdout:9/761: rename d0/d9/ld0 to d0/d4e/d50/d63/d6e/d4b/d58/dd5/lec 0 2024-04-02T04:42:46.821 INFO:tasks.workunit.client.0.smithi016.stdout:2/755: write d12/d35/d3b/d42/d67/fb3 [781305,8473] 0 2024-04-02T04:42:46.822 INFO:tasks.workunit.client.1.smithi027.stdout:6/177: dwrite d13/d1b/d24/f3a [0,4194304] 0 2024-04-02T04:42:46.823 INFO:tasks.workunit.client.0.smithi016.stdout:6/739: fdatasync d0/f5d 0 2024-04-02T04:42:46.826 INFO:tasks.workunit.client.0.smithi016.stdout:5/658: creat d3/d4c/dc4/ff5 x:0 0 0 2024-04-02T04:42:46.826 INFO:tasks.workunit.client.0.smithi016.stdout:7/686: dread d0/d21/f65 [4194304,4194304] 0 2024-04-02T04:42:46.826 INFO:tasks.workunit.client.1.smithi027.stdout:1/128: rmdir d0/d1e 39 2024-04-02T04:42:46.827 INFO:tasks.workunit.client.1.smithi027.stdout:4/152: sync 2024-04-02T04:42:46.827 INFO:tasks.workunit.client.1.smithi027.stdout:4/153: readlink ld 0 2024-04-02T04:42:46.829 INFO:tasks.workunit.client.1.smithi027.stdout:9/161: getdents d1a/d1b/d20 0 2024-04-02T04:42:46.834 INFO:tasks.workunit.client.1.smithi027.stdout:7/168: fsync d2/d7/da/dc/f30 0 2024-04-02T04:42:46.835 INFO:tasks.workunit.client.1.smithi027.stdout:8/129: mkdir d5/d7/d27 0 2024-04-02T04:42:46.835 INFO:tasks.workunit.client.0.smithi016.stdout:3/787: dwrite d2/d15/d16/d8a/dd1/d8b/f9a [0,4194304] 0 2024-04-02T04:42:46.837 INFO:tasks.workunit.client.0.smithi016.stdout:8/676: write d1/f1c [1586578,87199] 0 2024-04-02T04:42:46.837 INFO:tasks.workunit.client.1.smithi027.stdout:4/154: dwrite f23 [0,4194304] 0 2024-04-02T04:42:46.837 INFO:tasks.workunit.client.1.smithi027.stdout:7/169: stat d2/d7/da/dc/d16 0 2024-04-02T04:42:46.839 INFO:tasks.workunit.client.0.smithi016.stdout:8/677: dread d1/d31/f7e [0,4194304] 0 2024-04-02T04:42:46.839 INFO:tasks.workunit.client.1.smithi027.stdout:7/170: chown d2/d1c/f22 2239 1 2024-04-02T04:42:46.842 INFO:tasks.workunit.client.1.smithi027.stdout:2/163: getdents d1/dd/d17/d12 0 2024-04-02T04:42:46.842 INFO:tasks.workunit.client.1.smithi027.stdout:3/127: dwrite d1/f18 [0,4194304] 0 2024-04-02T04:42:46.844 INFO:tasks.workunit.client.1.smithi027.stdout:2/164: read - d1/dd/d17/d2b/f26 zero size 2024-04-02T04:42:46.846 INFO:tasks.workunit.client.0.smithi016.stdout:6/740: rename d0/d9/d16/f5b to d0/d9/d16/d56/d59/da1/fe8 0 2024-04-02T04:42:46.847 INFO:tasks.workunit.client.1.smithi027.stdout:2/165: dread d1/db/f30 [0,4194304] 0 2024-04-02T04:42:46.849 INFO:tasks.workunit.client.0.smithi016.stdout:8/678: dwrite d1/d6/d36/d3f/d40/d49/d4a/d4c/fba [0,4194304] 0 2024-04-02T04:42:46.852 INFO:tasks.workunit.client.0.smithi016.stdout:7/687: creat d0/d46/d37/ff0 x:0 0 0 2024-04-02T04:42:46.855 INFO:tasks.workunit.client.0.smithi016.stdout:8/679: dread d1/d6/d36/d3f/d40/d63/dc5/fe0 [0,4194304] 0 2024-04-02T04:42:46.861 INFO:tasks.workunit.client.0.smithi016.stdout:7/688: dread d0/d46/d16/dc2/dea/d64/fd6 [0,4194304] 0 2024-04-02T04:42:46.863 INFO:tasks.workunit.client.1.smithi027.stdout:9/162: rename d1a/d1b to d1a/d1d/d29/d37 0 2024-04-02T04:42:46.864 INFO:tasks.workunit.client.1.smithi027.stdout:0/197: dwrite d4/d16/d2b/f2a [4194304,4194304] 0 2024-04-02T04:42:46.867 INFO:tasks.workunit.client.0.smithi016.stdout:0/651: sync 2024-04-02T04:42:46.869 INFO:tasks.workunit.client.0.smithi016.stdout:3/788: creat d2/d15/d1a/f108 x:0 0 0 2024-04-02T04:42:46.869 INFO:tasks.workunit.client.1.smithi027.stdout:4/155: fdatasync d24/f2a 0 2024-04-02T04:42:46.871 INFO:tasks.workunit.client.0.smithi016.stdout:2/756: dread d12/d35/d3b/d42/d8d/fcd [0,4194304] 0 2024-04-02T04:42:46.873 INFO:tasks.workunit.client.1.smithi027.stdout:7/171: unlink d2/d7/cb 0 2024-04-02T04:42:46.874 INFO:tasks.workunit.client.1.smithi027.stdout:7/172: chown d2/d7/da/dc/d16/d2f/f27 114 1 2024-04-02T04:42:46.875 INFO:tasks.workunit.client.1.smithi027.stdout:3/128: symlink d1/d1a/l28 0 2024-04-02T04:42:46.877 INFO:tasks.workunit.client.1.smithi027.stdout:2/166: readlink d1/dd/d17/l2e 0 2024-04-02T04:42:46.879 INFO:tasks.workunit.client.1.smithi027.stdout:5/230: creat d0/d1a/d20/f59 x:0 0 0 2024-04-02T04:42:46.880 INFO:tasks.workunit.client.0.smithi016.stdout:9/762: truncate d0/d9/db/dc9/fa0 234825 0 2024-04-02T04:42:46.881 INFO:tasks.workunit.client.1.smithi027.stdout:6/178: write d13/d1b/d24/d28/f2a [989160,127804] 0 2024-04-02T04:42:46.884 INFO:tasks.workunit.client.0.smithi016.stdout:8/680: rmdir d1/d6/d36/d3f/d40 39 2024-04-02T04:42:46.886 INFO:tasks.workunit.client.1.smithi027.stdout:1/129: fsync d0/f7 0 2024-04-02T04:42:46.888 INFO:tasks.workunit.client.1.smithi027.stdout:9/163: chown cb 3257196 1 2024-04-02T04:42:46.891 INFO:tasks.workunit.client.0.smithi016.stdout:3/789: creat d2/d2e/d36/d48/da2/de7/f109 x:0 0 0 2024-04-02T04:42:46.892 INFO:tasks.workunit.client.1.smithi027.stdout:0/198: symlink d4/d16/d2b/d2d/l41 0 2024-04-02T04:42:46.893 INFO:tasks.workunit.client.0.smithi016.stdout:3/790: write d2/d15/d16/d8a/dd1/d6e/d6f/db0/de2/ff8 [637130,121598] 0 2024-04-02T04:42:46.894 INFO:tasks.workunit.client.0.smithi016.stdout:3/791: write d2/d15/d16/f44 [3122814,28551] 0 2024-04-02T04:42:46.898 INFO:tasks.workunit.client.1.smithi027.stdout:7/173: mkdir d2/d7/da/d35 0 2024-04-02T04:42:46.899 INFO:tasks.workunit.client.0.smithi016.stdout:5/659: creat d3/d4c/ff6 x:0 0 0 2024-04-02T04:42:46.899 INFO:tasks.workunit.client.1.smithi027.stdout:3/129: rmdir d1 39 2024-04-02T04:42:46.901 INFO:tasks.workunit.client.1.smithi027.stdout:2/167: creat d1/dd/d17/d12/f38 x:0 0 0 2024-04-02T04:42:46.902 INFO:tasks.workunit.client.1.smithi027.stdout:2/168: truncate d1/db/f32 845514 0 2024-04-02T04:42:46.905 INFO:tasks.workunit.client.0.smithi016.stdout:9/763: mknod d0/d4a/d75/d95/d9a/ced 0 2024-04-02T04:42:46.906 INFO:tasks.workunit.client.0.smithi016.stdout:6/741: write d0/d9/f9c [265077,90231] 0 2024-04-02T04:42:46.907 INFO:tasks.workunit.client.1.smithi027.stdout:6/179: mkdir d13/d1b/d24/d25/d42 0 2024-04-02T04:42:46.907 INFO:tasks.workunit.client.0.smithi016.stdout:9/764: chown d0/d4a/d99/f37 527033711 1 2024-04-02T04:42:46.908 INFO:tasks.workunit.client.1.smithi027.stdout:2/169: dread d1/dd/d17/fc [4194304,4194304] 0 2024-04-02T04:42:46.908 INFO:tasks.workunit.client.1.smithi027.stdout:4/156: dwrite f12 [0,4194304] 0 2024-04-02T04:42:46.909 INFO:tasks.workunit.client.0.smithi016.stdout:7/689: mknod d0/d87/cf1 0 2024-04-02T04:42:46.910 INFO:tasks.workunit.client.1.smithi027.stdout:6/180: truncate d13/d1b/d24/d28/f39 1808842 0 2024-04-02T04:42:46.914 INFO:tasks.workunit.client.0.smithi016.stdout:0/652: mknod d0/d16/d2e/d53/da4/cdc 0 2024-04-02T04:42:46.914 INFO:tasks.workunit.client.1.smithi027.stdout:9/164: rmdir d1a/d1d 39 2024-04-02T04:42:46.914 INFO:tasks.workunit.client.0.smithi016.stdout:9/765: dwrite d0/d4e/d50/d63/d6e/f44 [0,4194304] 0 2024-04-02T04:42:46.915 INFO:tasks.workunit.client.0.smithi016.stdout:9/766: fsync d0/f2d 0 2024-04-02T04:42:46.916 INFO:tasks.workunit.client.1.smithi027.stdout:4/157: dwrite f11 [0,4194304] 0 2024-04-02T04:42:46.916 INFO:tasks.workunit.client.0.smithi016.stdout:5/660: mknod d3/d40/d4f/d6b/d45/cf7 0 2024-04-02T04:42:46.917 INFO:tasks.workunit.client.1.smithi027.stdout:5/231: unlink d0/df/d11/d13/c17 0 2024-04-02T04:42:46.920 INFO:tasks.workunit.client.1.smithi027.stdout:3/130: dread d1/de/df/d13/f21 [0,4194304] 0 2024-04-02T04:42:46.926 INFO:tasks.workunit.client.0.smithi016.stdout:7/690: dread - d0/fe7 zero size 2024-04-02T04:42:46.927 INFO:tasks.workunit.client.0.smithi016.stdout:3/792: symlink d2/l10a 0 2024-04-02T04:42:46.927 INFO:tasks.workunit.client.0.smithi016.stdout:7/691: chown d0/d46/d16/dc2/dea/d64/fd6 10 1 2024-04-02T04:42:46.928 INFO:tasks.workunit.client.0.smithi016.stdout:3/793: dread - d2/d15/f24 zero size 2024-04-02T04:42:46.929 INFO:tasks.workunit.client.0.smithi016.stdout:9/767: creat d0/d9/d4d/da8/fee x:0 0 0 2024-04-02T04:42:46.930 INFO:tasks.workunit.client.0.smithi016.stdout:3/794: dread - d2/d15/f24 zero size 2024-04-02T04:42:46.932 INFO:tasks.workunit.client.0.smithi016.stdout:0/653: mkdir d0/d16/dd6/ddd 0 2024-04-02T04:42:46.935 INFO:tasks.workunit.client.0.smithi016.stdout:0/654: fsync d0/d16/d2e/d37/f3b 0 2024-04-02T04:42:46.942 INFO:tasks.workunit.client.0.smithi016.stdout:0/655: creat d0/d16/d23/d3e/db3/fde x:0 0 0 2024-04-02T04:42:46.942 INFO:tasks.workunit.client.0.smithi016.stdout:7/692: dwrite d0/d46/fe6 [0,4194304] 0 2024-04-02T04:42:46.945 INFO:tasks.workunit.client.0.smithi016.stdout:5/661: dwrite d3/df/d18/d2f/fd6 [0,4194304] 0 2024-04-02T04:42:46.952 INFO:tasks.workunit.client.0.smithi016.stdout:6/742: write d0/f37 [6143246,122708] 0 2024-04-02T04:42:46.953 INFO:tasks.workunit.client.0.smithi016.stdout:5/662: stat d3/d40/d72/fd0 0 2024-04-02T04:42:46.955 INFO:tasks.workunit.client.0.smithi016.stdout:6/743: fdatasync d0/d9/df/f1b 0 2024-04-02T04:42:46.961 INFO:tasks.workunit.client.0.smithi016.stdout:3/795: write d2/d15/d1a/f53 [637662,36910] 0 2024-04-02T04:42:46.964 INFO:tasks.workunit.client.0.smithi016.stdout:8/681: dwrite d1/d6/f2e [0,4194304] 0 2024-04-02T04:42:46.965 INFO:tasks.workunit.client.1.smithi027.stdout:0/199: creat d4/d16/d3b/f42 x:0 0 0 2024-04-02T04:42:46.965 INFO:tasks.workunit.client.1.smithi027.stdout:7/174: mkdir d2/d7/da/dc/d1f/d33/d36 0 2024-04-02T04:42:46.966 INFO:tasks.workunit.client.1.smithi027.stdout:5/232: mknod d0/d1d/c5a 0 2024-04-02T04:42:46.966 INFO:tasks.workunit.client.0.smithi016.stdout:6/744: link d0/d5c/fb1 d0/d9/d16/dd8/fe9 0 2024-04-02T04:42:46.968 INFO:tasks.workunit.client.1.smithi027.stdout:4/158: dwrite f21 [0,4194304] 0 2024-04-02T04:42:46.969 INFO:tasks.workunit.client.0.smithi016.stdout:3/796: mkdir d2/d15/d10b 0 2024-04-02T04:42:46.970 INFO:tasks.workunit.client.0.smithi016.stdout:5/663: write d3/f36 [289883,85331] 0 2024-04-02T04:42:46.971 INFO:tasks.workunit.client.1.smithi027.stdout:7/175: write d2/d7/da/f31 [1045003,61576] 0 2024-04-02T04:42:46.973 INFO:tasks.workunit.client.0.smithi016.stdout:3/797: chown d2/d15/d16/f54 3 1 2024-04-02T04:42:46.973 INFO:tasks.workunit.client.1.smithi027.stdout:3/131: rename d1/de/c1e to d1/d1a/c29 0 2024-04-02T04:42:46.975 INFO:tasks.workunit.client.1.smithi027.stdout:0/200: rmdir d4/d21/d28 39 2024-04-02T04:42:46.975 INFO:tasks.workunit.client.0.smithi016.stdout:0/656: dwrite d0/d16/f7a [0,4194304] 0 2024-04-02T04:42:46.976 INFO:tasks.workunit.client.1.smithi027.stdout:1/130: dwrite d0/f5 [0,4194304] 0 2024-04-02T04:42:46.976 INFO:tasks.workunit.client.0.smithi016.stdout:3/798: dread - d2/d15/d70/ff5 zero size 2024-04-02T04:42:46.978 INFO:tasks.workunit.client.0.smithi016.stdout:7/693: dwrite d0/d46/d16/dc2/dea/f53 [0,4194304] 0 2024-04-02T04:42:46.978 INFO:tasks.workunit.client.0.smithi016.stdout:8/682: mknod d1/d6/d36/d3f/d40/d63/ce6 0 2024-04-02T04:42:46.979 INFO:tasks.workunit.client.1.smithi027.stdout:5/233: creat d0/df/d2f/d3b/f5b x:0 0 0 2024-04-02T04:42:46.979 INFO:tasks.workunit.client.0.smithi016.stdout:6/745: read d0/d9/d16/d56/d89/f3b [1813894,16469] 0 2024-04-02T04:42:46.980 INFO:tasks.workunit.client.0.smithi016.stdout:0/657: fdatasync d0/d16/d24/d4e/d8d/f75 0 2024-04-02T04:42:46.981 INFO:tasks.workunit.client.1.smithi027.stdout:4/159: truncate f1e 1581984 0 2024-04-02T04:42:46.990 INFO:tasks.workunit.client.1.smithi027.stdout:9/165: creat d1a/f38 x:0 0 0 2024-04-02T04:42:46.991 INFO:tasks.workunit.client.0.smithi016.stdout:3/799: symlink d2/d15/d1a/d45/l10c 0 2024-04-02T04:42:46.993 INFO:tasks.workunit.client.0.smithi016.stdout:3/800: dread - d2/d15/d16/d8a/dd1/fbc zero size 2024-04-02T04:42:46.994 INFO:tasks.workunit.client.0.smithi016.stdout:3/801: dread - d2/d15/f24 zero size 2024-04-02T04:42:46.994 INFO:tasks.workunit.client.0.smithi016.stdout:3/802: stat d2/d15/d1a/f53 0 2024-04-02T04:42:46.995 INFO:tasks.workunit.client.0.smithi016.stdout:4/703: sync 2024-04-02T04:42:46.995 INFO:tasks.workunit.client.0.smithi016.stdout:1/668: sync 2024-04-02T04:42:46.996 INFO:tasks.workunit.client.0.smithi016.stdout:6/746: rmdir d0/d9/d16/d56/d89/d67/d6d 39 2024-04-02T04:42:46.997 INFO:tasks.workunit.client.1.smithi027.stdout:9/166: dwrite d1a/f1f [0,4194304] 0 2024-04-02T04:42:46.999 INFO:tasks.workunit.client.0.smithi016.stdout:5/664: dread d3/d4c/dd1/f8d [0,4194304] 0 2024-04-02T04:42:47.002 INFO:tasks.workunit.client.0.smithi016.stdout:7/694: creat d0/d46/d16/dc2/ff2 x:0 0 0 2024-04-02T04:42:47.003 INFO:tasks.workunit.client.0.smithi016.stdout:5/665: stat d3/df/d18/d26/d48/f49 0 2024-04-02T04:42:47.003 INFO:tasks.workunit.client.1.smithi027.stdout:4/160: creat d24/f2b x:0 0 0 2024-04-02T04:42:47.004 INFO:tasks.workunit.client.1.smithi027.stdout:4/161: chown d24/f28 192 1 2024-04-02T04:42:47.004 INFO:tasks.workunit.client.0.smithi016.stdout:6/747: dread d0/d9/d16/f23 [0,4194304] 0 2024-04-02T04:42:47.004 INFO:tasks.workunit.client.1.smithi027.stdout:2/170: getdents d1/dd 0 2024-04-02T04:42:47.005 INFO:tasks.workunit.client.0.smithi016.stdout:7/695: read d0/d46/d37/d47/f5c [2571166,119349] 0 2024-04-02T04:42:47.006 INFO:tasks.workunit.client.0.smithi016.stdout:3/803: dread - d2/d15/d1a/d45/fa4 zero size 2024-04-02T04:42:47.007 INFO:tasks.workunit.client.0.smithi016.stdout:4/704: mknod d5/d1d/d2d/cd3 0 2024-04-02T04:42:47.008 INFO:tasks.workunit.client.0.smithi016.stdout:3/804: fsync d2/d15/fb7 0 2024-04-02T04:42:47.009 INFO:tasks.workunit.client.0.smithi016.stdout:1/669: rename d14/d19/d1d/d29/d39/d46/dcb/d43/f77 to d14/fdb 0 2024-04-02T04:42:47.010 INFO:tasks.workunit.client.0.smithi016.stdout:0/658: dread d0/d16/d2e/d53/d6f/f74 [0,4194304] 0 2024-04-02T04:42:47.011 INFO:tasks.workunit.client.1.smithi027.stdout:0/201: symlink d4/d16/d1b/l43 0 2024-04-02T04:42:47.011 INFO:tasks.workunit.client.1.smithi027.stdout:8/130: sync 2024-04-02T04:42:47.013 INFO:tasks.workunit.client.1.smithi027.stdout:5/234: mkdir d0/df/d5c 0 2024-04-02T04:42:47.014 INFO:tasks.workunit.client.0.smithi016.stdout:4/705: dwrite d5/d17/d18/d6d/f74 [0,4194304] 0 2024-04-02T04:42:47.015 INFO:tasks.workunit.client.1.smithi027.stdout:5/235: truncate d0/df/d2f/d3b/d51/f53 184485 0 2024-04-02T04:42:47.015 INFO:tasks.workunit.client.1.smithi027.stdout:2/171: unlink d1/f6 0 2024-04-02T04:42:47.016 INFO:tasks.workunit.client.1.smithi027.stdout:7/176: getdents d2/df 0 2024-04-02T04:42:47.017 INFO:tasks.workunit.client.1.smithi027.stdout:2/172: rename d1 to d1/dd/d17/d29/d39 22 2024-04-02T04:42:47.021 INFO:tasks.workunit.client.1.smithi027.stdout:3/132: creat d1/f2a x:0 0 0 2024-04-02T04:42:47.026 INFO:tasks.workunit.client.0.smithi016.stdout:8/683: getdents d1/d6/d36/d3f/d40/d49/d38 0 2024-04-02T04:42:47.028 INFO:tasks.workunit.client.0.smithi016.stdout:4/706: unlink d5/d17/l52 0 2024-04-02T04:42:47.031 INFO:tasks.workunit.client.0.smithi016.stdout:6/748: rename d0/d9/d16/d56/d59/da1/fe8 to d0/d9/d16/d56/da4/fea 0 2024-04-02T04:42:47.032 INFO:tasks.workunit.client.0.smithi016.stdout:5/666: write d3/f77 [902780,90335] 0 2024-04-02T04:42:47.033 INFO:tasks.workunit.client.0.smithi016.stdout:8/684: creat d1/d6/d36/d4f/d5d/fe7 x:0 0 0 2024-04-02T04:42:47.035 INFO:tasks.workunit.client.1.smithi027.stdout:5/236: symlink d0/d1d/l5d 0 2024-04-02T04:42:47.038 INFO:tasks.workunit.client.1.smithi027.stdout:7/177: creat d2/d1c/f37 x:0 0 0 2024-04-02T04:42:47.039 INFO:tasks.workunit.client.0.smithi016.stdout:1/670: fsync d14/d19/d1e/d71/fbf 0 2024-04-02T04:42:47.039 INFO:tasks.workunit.client.1.smithi027.stdout:2/173: symlink d1/dd/l3a 0 2024-04-02T04:42:47.041 INFO:tasks.workunit.client.0.smithi016.stdout:0/659: rename d0/d16/d24/l27 to d0/d16/d23/d3e/db3/ldf 0 2024-04-02T04:42:47.042 INFO:tasks.workunit.client.0.smithi016.stdout:0/660: write d0/d16/d24/d4e/d8d/d71/f72 [338790,17691] 0 2024-04-02T04:42:47.046 INFO:tasks.workunit.client.1.smithi027.stdout:9/167: creat d1a/f39 x:0 0 0 2024-04-02T04:42:47.048 INFO:tasks.workunit.client.0.smithi016.stdout:8/685: creat d1/d6/d36/d3f/d40/d49/d4a/d4c/dad/fe8 x:0 0 0 2024-04-02T04:42:47.049 INFO:tasks.workunit.client.1.smithi027.stdout:0/202: rename d4/d21/f37 to d4/d21/d28/f44 0 2024-04-02T04:42:47.051 INFO:tasks.workunit.client.1.smithi027.stdout:6/181: sync 2024-04-02T04:42:47.052 INFO:tasks.workunit.client.1.smithi027.stdout:9/168: dwrite f12 [0,4194304] 0 2024-04-02T04:42:47.054 INFO:tasks.workunit.client.0.smithi016.stdout:6/749: dread d0/d9/d16/d56/f25 [0,4194304] 0 2024-04-02T04:42:47.055 INFO:tasks.workunit.client.0.smithi016.stdout:6/750: chown d0/d9/d16/d46/f7b 283568 1 2024-04-02T04:42:47.055 INFO:tasks.workunit.client.1.smithi027.stdout:3/133: write d1/de/df/f17 [567903,119249] 0 2024-04-02T04:42:47.057 INFO:tasks.workunit.client.0.smithi016.stdout:1/671: dread d14/d19/d1d/d29/d39/d46/dcb/d43/f6f [0,4194304] 0 2024-04-02T04:42:47.057 INFO:tasks.workunit.client.1.smithi027.stdout:5/237: symlink d0/d1a/d20/l5e 0 2024-04-02T04:42:47.061 INFO:tasks.workunit.client.0.smithi016.stdout:5/667: symlink d3/df/d18/d26/lf8 0 2024-04-02T04:42:47.063 INFO:tasks.workunit.client.0.smithi016.stdout:6/751: dwrite d0/f3f [0,4194304] 0 2024-04-02T04:42:47.065 INFO:tasks.workunit.client.0.smithi016.stdout:4/707: link d5/d1d/d2d/c47 d5/d17/d18/d6d/d8e/cd4 0 2024-04-02T04:42:47.066 INFO:tasks.workunit.client.0.smithi016.stdout:5/668: write d3/faa [200591,66758] 0 2024-04-02T04:42:47.068 INFO:tasks.workunit.client.0.smithi016.stdout:1/672: mknod d14/d19/d1e/d5e/cdc 0 2024-04-02T04:42:47.068 INFO:tasks.workunit.client.0.smithi016.stdout:5/669: readlink d3/df/d86/le3 0 2024-04-02T04:42:47.069 INFO:tasks.workunit.client.1.smithi027.stdout:1/131: getdents d0/d1e/d25 0 2024-04-02T04:42:47.070 INFO:tasks.workunit.client.0.smithi016.stdout:5/670: write d3/d4c/ff6 [586801,51054] 0 2024-04-02T04:42:47.072 INFO:tasks.workunit.client.0.smithi016.stdout:5/671: fsync d3/df/f1f 0 2024-04-02T04:42:47.072 INFO:tasks.workunit.client.1.smithi027.stdout:9/169: write d1a/d1d/d29/d37/d20/f2b [563904,59766] 0 2024-04-02T04:42:47.073 INFO:tasks.workunit.client.1.smithi027.stdout:0/203: read d4/d16/d1b/f20 [700774,106415] 0 2024-04-02T04:42:47.073 INFO:tasks.workunit.client.0.smithi016.stdout:8/686: mkdir d1/d6/d36/d3f/d40/de9 0 2024-04-02T04:42:47.073 INFO:tasks.workunit.client.1.smithi027.stdout:0/204: fsync d4/f6 0 2024-04-02T04:42:47.077 INFO:tasks.workunit.client.0.smithi016.stdout:4/708: mkdir d5/de/dd5 0 2024-04-02T04:42:47.078 INFO:tasks.workunit.client.1.smithi027.stdout:8/131: link d5/d8/c25 d5/d8/db/dd/c28 0 2024-04-02T04:42:47.079 INFO:tasks.workunit.client.1.smithi027.stdout:9/170: dwrite f15 [0,4194304] 0 2024-04-02T04:42:47.079 INFO:tasks.workunit.client.1.smithi027.stdout:3/134: symlink d1/d1a/l2b 0 2024-04-02T04:42:47.080 INFO:tasks.workunit.client.1.smithi027.stdout:3/135: dread - d1/de/df/f14 zero size 2024-04-02T04:42:47.081 INFO:tasks.workunit.client.0.smithi016.stdout:6/752: mknod d0/d9/d16/d56/d89/d67/d6f/d8a/ceb 0 2024-04-02T04:42:47.081 INFO:tasks.workunit.client.1.smithi027.stdout:3/136: fdatasync d1/fc 0 2024-04-02T04:42:47.082 INFO:tasks.workunit.client.1.smithi027.stdout:5/238: write d0/f47 [47849,110196] 0 2024-04-02T04:42:47.082 INFO:tasks.workunit.client.1.smithi027.stdout:7/178: truncate d2/d1d/f25 964826 0 2024-04-02T04:42:47.083 INFO:tasks.workunit.client.1.smithi027.stdout:3/137: chown d1/de/df/d13/f25 68521 1 2024-04-02T04:42:47.083 INFO:tasks.workunit.client.0.smithi016.stdout:1/673: fdatasync d14/d19/d1d/d29/d39/d46/dcb/d43/d68/dc1/fc4 0 2024-04-02T04:42:47.084 INFO:tasks.workunit.client.1.smithi027.stdout:4/162: getdents d24 0 2024-04-02T04:42:47.084 INFO:tasks.workunit.client.1.smithi027.stdout:7/179: write d2/d7/da/f31 [1034792,86320] 0 2024-04-02T04:42:47.087 INFO:tasks.workunit.client.0.smithi016.stdout:5/672: write d3/df/d18/d26/ddc/d6d/fa7 [6600182,22574] 0 2024-04-02T04:42:47.092 INFO:tasks.workunit.client.1.smithi027.stdout:0/205: symlink d4/d16/d3b/l45 0 2024-04-02T04:42:47.093 INFO:tasks.workunit.client.1.smithi027.stdout:0/206: write d4/d16/d2b/f2a [7503311,29294] 0 2024-04-02T04:42:47.094 INFO:tasks.workunit.client.0.smithi016.stdout:1/674: unlink d14/d19/d1e/c21 0 2024-04-02T04:42:47.096 INFO:tasks.workunit.client.0.smithi016.stdout:1/675: write d14/d19/d1d/d29/d39/d46/dcb/d43/f96 [533660,53861] 0 2024-04-02T04:42:47.096 INFO:tasks.workunit.client.1.smithi027.stdout:8/132: creat d5/d8/db/f29 x:0 0 0 2024-04-02T04:42:47.097 INFO:tasks.workunit.client.1.smithi027.stdout:9/171: dread f17 [0,4194304] 0 2024-04-02T04:42:47.098 INFO:tasks.workunit.client.0.smithi016.stdout:4/709: mkdir d5/d17/d18/d6d/d91/dd6 0 2024-04-02T04:42:47.098 INFO:tasks.workunit.client.1.smithi027.stdout:8/133: chown d5/d8/db/dd/c28 229 1 2024-04-02T04:42:47.101 INFO:tasks.workunit.client.0.smithi016.stdout:5/673: mkdir d3/d40/d4f/d67/dbf/df9 0 2024-04-02T04:42:47.102 INFO:tasks.workunit.client.0.smithi016.stdout:0/661: link d0/d48/l5e d0/d16/d2e/d37/le0 0 2024-04-02T04:42:47.102 INFO:tasks.workunit.client.0.smithi016.stdout:0/662: chown d0/d16/d24/l1d 112 1 2024-04-02T04:42:47.104 INFO:tasks.workunit.client.0.smithi016.stdout:0/663: write d0/d16/d24/d4e/d8d/d71/dc7/fda [286028,114404] 0 2024-04-02T04:42:47.104 INFO:tasks.workunit.client.0.smithi016.stdout:6/753: link d0/d9/d16/d56/d59/f86 d0/d5c/fec 0 2024-04-02T04:42:47.105 INFO:tasks.workunit.client.1.smithi027.stdout:1/132: getdents d0/d1e/d25/d2b 0 2024-04-02T04:42:47.105 INFO:tasks.workunit.client.0.smithi016.stdout:0/664: readlink d0/d16/d24/d62/d9c/d8c/lc4 0 2024-04-02T04:42:47.107 INFO:tasks.workunit.client.0.smithi016.stdout:0/665: chown d0/d16/d24/d62/d9c/c77 0 1 2024-04-02T04:42:47.107 INFO:tasks.workunit.client.0.smithi016.stdout:8/687: creat d1/d6/d36/d3f/d40/d49/fea x:0 0 0 2024-04-02T04:42:47.110 INFO:tasks.workunit.client.1.smithi027.stdout:6/182: link d13/d1b/c31 d13/d1b/d24/d28/d3c/c43 0 2024-04-02T04:42:47.110 INFO:tasks.workunit.client.0.smithi016.stdout:0/666: dread d0/d16/d23/d3e/f43 [0,4194304] 0 2024-04-02T04:42:47.114 INFO:tasks.workunit.client.0.smithi016.stdout:5/674: chown d3/df/d18/d26/d48/f61 30850 1 2024-04-02T04:42:47.117 INFO:tasks.workunit.client.1.smithi027.stdout:2/174: rmdir d1/dd/d17/d12 39 2024-04-02T04:42:47.118 INFO:tasks.workunit.client.1.smithi027.stdout:8/134: symlink d5/d8/db/l2a 0 2024-04-02T04:42:47.122 INFO:tasks.workunit.client.0.smithi016.stdout:1/676: mkdir d14/d19/d1e/d71/ddd 0 2024-04-02T04:42:47.122 INFO:tasks.workunit.client.1.smithi027.stdout:8/135: dread - d5/d8/db/df/f22 zero size 2024-04-02T04:42:47.122 INFO:tasks.workunit.client.1.smithi027.stdout:4/163: getdents d24 0 2024-04-02T04:42:47.125 INFO:tasks.workunit.client.1.smithi027.stdout:7/180: rename d2/c9 to d2/d7/da/dc/d1f/d33/c38 0 2024-04-02T04:42:47.125 INFO:tasks.workunit.client.1.smithi027.stdout:8/136: dread d5/d8/db/dd/f1f [0,4194304] 0 2024-04-02T04:42:47.126 INFO:tasks.workunit.client.1.smithi027.stdout:7/181: dread - d2/d1c/f37 zero size 2024-04-02T04:42:47.126 INFO:tasks.workunit.client.0.smithi016.stdout:5/675: write d3/df/f68 [1325523,46668] 0 2024-04-02T04:42:47.127 INFO:tasks.workunit.client.1.smithi027.stdout:7/182: chown d2/d7/da/f2a 209 1 2024-04-02T04:42:47.127 INFO:tasks.workunit.client.1.smithi027.stdout:0/207: mknod d4/d16/d1b/c46 0 2024-04-02T04:42:47.128 INFO:tasks.workunit.client.1.smithi027.stdout:7/183: fdatasync d2/d7/f18 0 2024-04-02T04:42:47.129 INFO:tasks.workunit.client.0.smithi016.stdout:8/688: mkdir d1/d6/d36/d3f/d40/de9/deb 0 2024-04-02T04:42:47.130 INFO:tasks.workunit.client.0.smithi016.stdout:1/677: symlink d14/d19/d1d/d45/d4f/d86/db4/lde 0 2024-04-02T04:42:47.131 INFO:tasks.workunit.client.0.smithi016.stdout:6/754: dwrite d0/d9/d16/d56/f11 [0,4194304] 0 2024-04-02T04:42:47.131 INFO:tasks.workunit.client.1.smithi027.stdout:4/164: creat d24/f2c x:0 0 0 2024-04-02T04:42:47.132 INFO:tasks.workunit.client.0.smithi016.stdout:8/689: write d1/d31/fe3 [482964,79337] 0 2024-04-02T04:42:47.135 INFO:tasks.workunit.client.1.smithi027.stdout:8/137: creat d5/d7/f2b x:0 0 0 2024-04-02T04:42:47.136 INFO:tasks.workunit.client.0.smithi016.stdout:4/710: getdents d5/d17/d18/d6d/d8e 0 2024-04-02T04:42:47.137 INFO:tasks.workunit.client.1.smithi027.stdout:2/175: chown d1/dd/d17/d2b/d2f 50 1 2024-04-02T04:42:47.138 INFO:tasks.workunit.client.1.smithi027.stdout:4/165: dwrite f21 [0,4194304] 0 2024-04-02T04:42:47.138 INFO:tasks.workunit.client.0.smithi016.stdout:5/676: dwrite d3/d40/d4f/ff4 [0,4194304] 0 2024-04-02T04:42:47.140 INFO:tasks.workunit.client.0.smithi016.stdout:0/667: link d0/d16/d2e/d37/la9 d0/d16/d24/d62/d9c/le1 0 2024-04-02T04:42:47.141 INFO:tasks.workunit.client.0.smithi016.stdout:5/677: write d3/df/d93/fed [851699,71671] 0 2024-04-02T04:42:47.142 INFO:tasks.workunit.client.1.smithi027.stdout:0/208: rename d4/d16/d1b/f29 to d4/d3d/f47 0 2024-04-02T04:42:47.143 INFO:tasks.workunit.client.0.smithi016.stdout:6/755: dwrite d0/d9/fe2 [0,4194304] 0 2024-04-02T04:42:47.143 INFO:tasks.workunit.client.0.smithi016.stdout:0/668: fdatasync d0/d16/d23/d3e/db3/fde 0 2024-04-02T04:42:47.144 INFO:tasks.workunit.client.0.smithi016.stdout:2/757: sync 2024-04-02T04:42:47.146 INFO:tasks.workunit.client.0.smithi016.stdout:8/690: creat d1/d6/d86/fec x:0 0 0 2024-04-02T04:42:47.149 INFO:tasks.workunit.client.0.smithi016.stdout:4/711: unlink d5/l5c 0 2024-04-02T04:42:47.150 INFO:tasks.workunit.client.1.smithi027.stdout:1/133: creat d0/f30 x:0 0 0 2024-04-02T04:42:47.154 INFO:tasks.workunit.client.0.smithi016.stdout:0/669: dwrite d0/d4f/daf/fd5 [0,4194304] 0 2024-04-02T04:42:47.157 INFO:tasks.workunit.client.1.smithi027.stdout:4/166: rmdir d24 39 2024-04-02T04:42:47.161 INFO:tasks.workunit.client.0.smithi016.stdout:8/691: dwrite d1/d6/d36/f37 [0,4194304] 0 2024-04-02T04:42:47.162 INFO:tasks.workunit.client.1.smithi027.stdout:4/167: dread f21 [0,4194304] 0 2024-04-02T04:42:47.166 INFO:tasks.workunit.client.0.smithi016.stdout:6/756: rmdir d0/d9/d16/d56/d89/d67/d6f/d7e 39 2024-04-02T04:42:47.167 INFO:tasks.workunit.client.1.smithi027.stdout:2/176: rename d1/dd to d1/db/d3b 0 2024-04-02T04:42:47.170 INFO:tasks.workunit.client.1.smithi027.stdout:1/134: write d0/d1e/d25/f29 [2564172,3534] 0 2024-04-02T04:42:47.170 INFO:tasks.workunit.client.1.smithi027.stdout:1/135: stat d0/de/l1a 0 2024-04-02T04:42:47.171 INFO:tasks.workunit.client.0.smithi016.stdout:8/692: dread d1/d6/d36/f37 [0,4194304] 0 2024-04-02T04:42:47.171 INFO:tasks.workunit.client.0.smithi016.stdout:4/712: mkdir d5/d8/d43/dd7 0 2024-04-02T04:42:47.171 INFO:tasks.workunit.client.1.smithi027.stdout:1/136: read - d0/de/f23 zero size 2024-04-02T04:42:47.173 INFO:tasks.workunit.client.0.smithi016.stdout:4/713: write d5/d17/d18/d6d/fc3 [303955,113881] 0 2024-04-02T04:42:47.180 INFO:tasks.workunit.client.0.smithi016.stdout:2/758: dread d12/d35/d101/d76/f92 [0,4194304] 0 2024-04-02T04:42:47.181 INFO:tasks.workunit.client.0.smithi016.stdout:5/678: dread d3/d4c/dd1/f52 [0,4194304] 0 2024-04-02T04:42:47.182 INFO:tasks.workunit.client.0.smithi016.stdout:5/679: chown d3/df/d93/cea 310 1 2024-04-02T04:42:47.236 INFO:tasks.workunit.client.0.smithi016.stdout:1/678: link d14/d19/d1d/d29/f3d d14/d19/fdf 0 2024-04-02T04:42:47.238 INFO:tasks.workunit.client.0.smithi016.stdout:1/679: write d14/d19/d1e/d71/dc0/fc2 [312348,116143] 0 2024-04-02T04:42:47.243 INFO:tasks.workunit.client.0.smithi016.stdout:0/670: symlink d0/d16/d24/d4e/d8d/d71/le2 0 2024-04-02T04:42:47.245 INFO:tasks.workunit.client.1.smithi027.stdout:2/177: fsync d1/db/d3b/d17/d12/f38 0 2024-04-02T04:42:47.246 INFO:tasks.workunit.client.1.smithi027.stdout:2/178: read - d1/db/d3b/f37 zero size 2024-04-02T04:42:47.247 INFO:tasks.workunit.client.0.smithi016.stdout:0/671: dread d0/d16/d23/d3e/f43 [0,4194304] 0 2024-04-02T04:42:47.250 INFO:tasks.workunit.client.1.smithi027.stdout:1/137: creat d0/d1e/d25/f31 x:0 0 0 2024-04-02T04:42:47.252 INFO:tasks.workunit.client.1.smithi027.stdout:1/138: dread d0/fd [0,4194304] 0 2024-04-02T04:42:47.254 INFO:tasks.workunit.client.0.smithi016.stdout:8/693: rmdir d1/d6/d36/d3f/d40/de9 39 2024-04-02T04:42:47.254 INFO:tasks.workunit.client.1.smithi027.stdout:5/239: sync 2024-04-02T04:42:47.255 INFO:tasks.workunit.client.1.smithi027.stdout:5/240: dread - d0/df/d11/f58 zero size 2024-04-02T04:42:47.256 INFO:tasks.workunit.client.1.smithi027.stdout:0/209: write d4/d16/d1b/f20 [230274,63464] 0 2024-04-02T04:42:47.256 INFO:tasks.workunit.client.1.smithi027.stdout:5/241: fdatasync d0/df/d11/d37/f3a 0 2024-04-02T04:42:47.257 INFO:tasks.workunit.client.1.smithi027.stdout:5/242: write d0/d1a/d20/f59 [109040,35912] 0 2024-04-02T04:42:47.259 INFO:tasks.workunit.client.0.smithi016.stdout:2/759: stat d12/d35/d3b/d42/d67/c6b 0 2024-04-02T04:42:47.260 INFO:tasks.workunit.client.1.smithi027.stdout:4/168: mkdir d24/d2d 0 2024-04-02T04:42:47.261 INFO:tasks.workunit.client.0.smithi016.stdout:8/694: dwrite d1/d6/d36/d8e/f9c [0,4194304] 0 2024-04-02T04:42:47.261 INFO:tasks.workunit.client.0.smithi016.stdout:8/695: stat d1/d31/f7e 0 2024-04-02T04:42:47.264 INFO:tasks.workunit.client.1.smithi027.stdout:0/210: dwrite d4/d16/d2b/f3a [0,4194304] 0 2024-04-02T04:42:47.264 INFO:tasks.workunit.client.1.smithi027.stdout:2/179: dread d1/db/d3b/d17/f2c [0,4194304] 0 2024-04-02T04:42:47.267 INFO:tasks.workunit.client.1.smithi027.stdout:8/138: getdents d5/d8/db/dd 0 2024-04-02T04:42:47.269 INFO:tasks.workunit.client.1.smithi027.stdout:8/139: write d5/d8/db/dd/f10 [935046,107456] 0 2024-04-02T04:42:47.271 INFO:tasks.workunit.client.0.smithi016.stdout:6/757: mkdir d0/d9/d16/d56/d89/d67/d6f/d7e/ded 0 2024-04-02T04:42:47.271 INFO:tasks.workunit.client.0.smithi016.stdout:9/768: sync 2024-04-02T04:42:47.274 INFO:tasks.workunit.client.0.smithi016.stdout:0/672: mkdir d0/d16/d2e/d53/da4/de3 0 2024-04-02T04:42:47.274 INFO:tasks.workunit.client.1.smithi027.stdout:4/169: rmdir d24 39 2024-04-02T04:42:47.275 INFO:tasks.workunit.client.1.smithi027.stdout:4/170: truncate f11 5217589 0 2024-04-02T04:42:47.278 INFO:tasks.workunit.client.1.smithi027.stdout:1/139: mknod d0/d1e/d25/d2b/c32 0 2024-04-02T04:42:47.279 INFO:tasks.workunit.client.0.smithi016.stdout:8/696: mkdir d1/d6/d36/d4f/ded 0 2024-04-02T04:42:47.280 INFO:tasks.workunit.client.0.smithi016.stdout:1/680: mkdir d14/d19/d1e/d71/de0 0 2024-04-02T04:42:47.281 INFO:tasks.workunit.client.1.smithi027.stdout:2/180: symlink d1/db/d3b/l3c 0 2024-04-02T04:42:47.281 INFO:tasks.workunit.client.0.smithi016.stdout:4/714: dwrite d5/d8/fb [0,4194304] 0 2024-04-02T04:42:47.281 INFO:tasks.workunit.client.0.smithi016.stdout:9/769: mkdir d0/d4e/d50/d63/dd2/def 0 2024-04-02T04:42:47.285 INFO:tasks.workunit.client.1.smithi027.stdout:2/181: dread d1/db/d3b/d17/d12/f1f [0,4194304] 0 2024-04-02T04:42:47.285 INFO:tasks.workunit.client.0.smithi016.stdout:0/673: mknod d0/d4f/d76/ce4 0 2024-04-02T04:42:47.289 INFO:tasks.workunit.client.1.smithi027.stdout:5/243: write d0/df/d11/d13/f28 [508729,51811] 0 2024-04-02T04:42:47.290 INFO:tasks.workunit.client.0.smithi016.stdout:5/680: creat d3/d4c/ffa x:0 0 0 2024-04-02T04:42:47.290 INFO:tasks.workunit.client.0.smithi016.stdout:5/681: read d3/df/d18/d2f/fd6 [1450404,85662] 0 2024-04-02T04:42:47.291 INFO:tasks.workunit.client.1.smithi027.stdout:5/244: write d0/df/d2f/d3b/f5b [91192,32245] 0 2024-04-02T04:42:47.292 INFO:tasks.workunit.client.0.smithi016.stdout:5/682: truncate d3/df/d18/fc3 1648947 0 2024-04-02T04:42:47.292 INFO:tasks.workunit.client.1.smithi027.stdout:1/140: rename d0/c21 to d0/de/c33 0 2024-04-02T04:42:47.293 INFO:tasks.workunit.client.0.smithi016.stdout:5/683: write d3/df/fb9 [443791,8368] 0 2024-04-02T04:42:47.297 INFO:tasks.workunit.client.0.smithi016.stdout:1/681: dread d14/fb6 [0,4194304] 0 2024-04-02T04:42:47.297 INFO:tasks.workunit.client.1.smithi027.stdout:0/211: link d4/f23 d4/d21/d28/f48 0 2024-04-02T04:42:47.298 INFO:tasks.workunit.client.1.smithi027.stdout:4/171: dwrite d24/f2b [0,4194304] 0 2024-04-02T04:42:47.299 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:47 smithi016 ceph-mon[28969]: pgmap v22: 65 pgs: 65 active+clean; 1.6 GiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 55 MiB/s rd, 209 MiB/s wr, 451 op/s 2024-04-02T04:42:47.300 INFO:tasks.workunit.client.1.smithi027.stdout:0/212: fdatasync d4/d16/d2b/f2f 0 2024-04-02T04:42:47.301 INFO:tasks.workunit.client.1.smithi027.stdout:9/172: sync 2024-04-02T04:42:47.301 INFO:tasks.workunit.client.1.smithi027.stdout:3/138: sync 2024-04-02T04:42:47.301 INFO:tasks.workunit.client.0.smithi016.stdout:9/770: truncate d0/f5d 2323498 0 2024-04-02T04:42:47.302 INFO:tasks.workunit.client.1.smithi027.stdout:3/139: chown d1 0 1 2024-04-02T04:42:47.304 INFO:tasks.workunit.client.1.smithi027.stdout:1/141: dwrite d0/fb [0,4194304] 0 2024-04-02T04:42:47.305 INFO:tasks.workunit.client.1.smithi027.stdout:8/140: symlink d5/d7/d27/l2c 0 2024-04-02T04:42:47.306 INFO:tasks.workunit.client.0.smithi016.stdout:4/715: dread d5/d17/d18/da0/fa7 [0,4194304] 0 2024-04-02T04:42:47.306 INFO:tasks.workunit.client.0.smithi016.stdout:4/716: chown d5/d17/d76/d7e 22 1 2024-04-02T04:42:47.309 INFO:tasks.workunit.client.0.smithi016.stdout:9/771: dwrite d0/d9/db/dc9/d53/fa3 [0,4194304] 0 2024-04-02T04:42:47.310 INFO:tasks.workunit.client.0.smithi016.stdout:6/758: write d0/d9/d16/d56/d89/d67/d6f/f82 [755015,68809] 0 2024-04-02T04:42:47.310 INFO:tasks.workunit.client.1.smithi027.stdout:3/140: read d1/de/df/f17 [561215,49469] 0 2024-04-02T04:42:47.312 INFO:tasks.workunit.client.1.smithi027.stdout:5/245: chown d0/d1d/c36 6036 1 2024-04-02T04:42:47.313 INFO:tasks.workunit.client.1.smithi027.stdout:9/173: dread f17 [0,4194304] 0 2024-04-02T04:42:47.314 INFO:tasks.workunit.client.0.smithi016.stdout:2/760: rename d12/c9a to d12/d35/d101/c109 0 2024-04-02T04:42:47.315 INFO:tasks.workunit.client.1.smithi027.stdout:3/141: dread - d1/de/df/d13/f1b zero size 2024-04-02T04:42:47.316 INFO:tasks.workunit.client.1.smithi027.stdout:0/213: dwrite d4/f11 [0,4194304] 0 2024-04-02T04:42:47.316 INFO:tasks.workunit.client.1.smithi027.stdout:5/246: truncate d0/f47 701533 0 2024-04-02T04:42:47.319 INFO:tasks.workunit.client.0.smithi016.stdout:1/682: fdatasync d14/f28 0 2024-04-02T04:42:47.320 INFO:tasks.workunit.client.0.smithi016.stdout:9/772: dwrite d0/d3/f81 [0,4194304] 0 2024-04-02T04:42:47.322 INFO:tasks.workunit.client.1.smithi027.stdout:4/172: rename c19 to d24/c2e 0 2024-04-02T04:42:47.323 INFO:tasks.workunit.client.1.smithi027.stdout:4/173: stat d24/f25 0 2024-04-02T04:42:47.324 INFO:tasks.workunit.client.1.smithi027.stdout:1/142: dwrite d0/f30 [0,4194304] 0 2024-04-02T04:42:47.330 INFO:tasks.workunit.client.0.smithi016.stdout:4/717: creat d5/d17/dcd/fd8 x:0 0 0 2024-04-02T04:42:47.331 INFO:tasks.workunit.client.1.smithi027.stdout:3/142: dwrite d1/d1a/f1c [0,4194304] 0 2024-04-02T04:42:47.332 INFO:tasks.workunit.client.0.smithi016.stdout:5/684: dwrite d3/df/d18/fac [0,4194304] 0 2024-04-02T04:42:47.333 INFO:tasks.workunit.client.0.smithi016.stdout:6/759: mknod d0/d9/d16/d46/dbe/cee 0 2024-04-02T04:42:47.334 INFO:tasks.workunit.client.1.smithi027.stdout:8/141: dread d5/d8/db/dd/f1f [0,4194304] 0 2024-04-02T04:42:47.336 INFO:tasks.workunit.client.1.smithi027.stdout:8/142: write d5/d7/f2b [584190,99572] 0 2024-04-02T04:42:47.336 INFO:tasks.workunit.client.1.smithi027.stdout:2/182: write d1/db/d3b/d17/f2c [1247646,86585] 0 2024-04-02T04:42:47.337 INFO:tasks.workunit.client.0.smithi016.stdout:0/674: dread d0/d16/d24/f2a [0,4194304] 0 2024-04-02T04:42:47.337 INFO:tasks.workunit.client.1.smithi027.stdout:8/143: chown d5/l6 1979292726 1 2024-04-02T04:42:47.339 INFO:tasks.workunit.client.1.smithi027.stdout:2/183: read - d1/db/d3b/f37 zero size 2024-04-02T04:42:47.344 INFO:tasks.workunit.client.1.smithi027.stdout:9/174: mkdir d1a/d1d/d29/d37/d3a 0 2024-04-02T04:42:47.344 INFO:tasks.workunit.client.0.smithi016.stdout:6/760: dwrite d0/d9/df/f4d [0,4194304] 0 2024-04-02T04:42:47.347 INFO:tasks.workunit.client.1.smithi027.stdout:5/247: rename d0/d5/l4d to d0/d5/l5f 0 2024-04-02T04:42:47.351 INFO:tasks.workunit.client.1.smithi027.stdout:4/174: creat d24/f2f x:0 0 0 2024-04-02T04:42:47.357 INFO:tasks.workunit.client.1.smithi027.stdout:0/214: dwrite d4/f14 [0,4194304] 0 2024-04-02T04:42:47.358 INFO:tasks.workunit.client.1.smithi027.stdout:0/215: readlink d4/d16/d2b/l26 0 2024-04-02T04:42:47.358 INFO:tasks.workunit.client.1.smithi027.stdout:6/183: sync 2024-04-02T04:42:47.359 INFO:tasks.workunit.client.1.smithi027.stdout:0/216: chown d4/d16/f17 45532 1 2024-04-02T04:42:47.361 INFO:tasks.workunit.client.1.smithi027.stdout:8/144: mknod d5/d8/db/dd/c2d 0 2024-04-02T04:42:47.362 INFO:tasks.workunit.client.1.smithi027.stdout:8/145: read - d5/d8/db/dd/f21 zero size 2024-04-02T04:42:47.362 INFO:tasks.workunit.client.1.smithi027.stdout:7/184: sync 2024-04-02T04:42:47.365 INFO:tasks.workunit.client.1.smithi027.stdout:9/175: mknod d1a/d1d/d29/d37/c3b 0 2024-04-02T04:42:47.365 INFO:tasks.workunit.client.1.smithi027.stdout:0/217: dread d4/d16/d2b/f2c [0,4194304] 0 2024-04-02T04:42:47.365 INFO:tasks.workunit.client.1.smithi027.stdout:9/176: fdatasync d1a/f39 0 2024-04-02T04:42:47.366 INFO:tasks.workunit.client.1.smithi027.stdout:5/248: symlink d0/d5/l60 0 2024-04-02T04:42:47.369 INFO:tasks.workunit.client.1.smithi027.stdout:7/185: dread d2/d7/da/f2a [0,4194304] 0 2024-04-02T04:42:47.370 INFO:tasks.workunit.client.1.smithi027.stdout:1/143: rename d0/l19 to d0/d1e/l34 0 2024-04-02T04:42:47.371 INFO:tasks.workunit.client.1.smithi027.stdout:5/249: truncate d0/df/d2f/d3b/f5b 423907 0 2024-04-02T04:42:47.372 INFO:tasks.workunit.client.1.smithi027.stdout:9/177: dread d1a/d1d/f27 [0,4194304] 0 2024-04-02T04:42:47.379 INFO:tasks.workunit.client.1.smithi027.stdout:1/144: dwrite d0/f14 [4194304,4194304] 0 2024-04-02T04:42:47.379 INFO:tasks.workunit.client.1.smithi027.stdout:8/146: creat d5/d8/f2e x:0 0 0 2024-04-02T04:42:47.381 INFO:tasks.workunit.client.1.smithi027.stdout:8/147: truncate d5/d8/db/df/f22 426170 0 2024-04-02T04:42:47.382 INFO:tasks.workunit.client.1.smithi027.stdout:0/218: mkdir d4/d3d/d49 0 2024-04-02T04:42:47.382 INFO:tasks.workunit.client.1.smithi027.stdout:8/148: dread - d5/d8/f2e zero size 2024-04-02T04:42:47.383 INFO:tasks.workunit.client.1.smithi027.stdout:0/219: read d4/d16/d2b/f2a [7036196,82591] 0 2024-04-02T04:42:47.384 INFO:tasks.workunit.client.1.smithi027.stdout:8/149: chown d5/d7/d27/l2c 0 1 2024-04-02T04:42:47.386 INFO:tasks.workunit.client.0.smithi016.stdout:2/761: dwrite d12/d35/d101/f9d [0,4194304] 0 2024-04-02T04:42:47.387 INFO:tasks.workunit.client.1.smithi027.stdout:0/220: dread d4/d3d/f47 [0,4194304] 0 2024-04-02T04:42:47.391 INFO:tasks.workunit.client.0.smithi016.stdout:1/683: dwrite d14/d19/f82 [8388608,4194304] 0 2024-04-02T04:42:47.392 INFO:tasks.workunit.client.1.smithi027.stdout:4/175: rename l22 to d24/l30 0 2024-04-02T04:42:47.392 INFO:tasks.workunit.client.0.smithi016.stdout:1/684: chown d14/d19/d20 2 1 2024-04-02T04:42:47.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:47 smithi027 ceph-mon[37550]: pgmap v22: 65 pgs: 65 active+clean; 1.6 GiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 55 MiB/s rd, 209 MiB/s wr, 451 op/s 2024-04-02T04:42:47.396 INFO:tasks.workunit.client.1.smithi027.stdout:5/250: mkdir d0/d1d/d61 0 2024-04-02T04:42:47.397 INFO:tasks.workunit.client.1.smithi027.stdout:5/251: fsync d0/df/d11/d13/f26 0 2024-04-02T04:42:47.398 INFO:tasks.workunit.client.1.smithi027.stdout:5/252: fsync d0/d1a/f49 0 2024-04-02T04:42:47.399 INFO:tasks.workunit.client.1.smithi027.stdout:9/178: mkdir d1a/d1d/d3c 0 2024-04-02T04:42:47.402 INFO:tasks.workunit.client.1.smithi027.stdout:6/184: truncate fc 3819345 0 2024-04-02T04:42:47.407 INFO:tasks.workunit.client.1.smithi027.stdout:1/145: symlink d0/de/l35 0 2024-04-02T04:42:47.408 INFO:tasks.workunit.client.1.smithi027.stdout:1/146: chown d0/de/f23 7077 1 2024-04-02T04:42:47.410 INFO:tasks.workunit.client.1.smithi027.stdout:2/184: write d1/db/d3b/d17/fc [9187113,49964] 0 2024-04-02T04:42:47.411 INFO:tasks.workunit.client.1.smithi027.stdout:2/185: write d1/f13 [6734930,77391] 0 2024-04-02T04:42:47.414 INFO:tasks.workunit.client.1.smithi027.stdout:3/143: link d1/ca d1/c2c 0 2024-04-02T04:42:47.416 INFO:tasks.workunit.client.1.smithi027.stdout:7/186: rename d2/d1d/c28 to d2/d7/da/dc/d1f/c39 0 2024-04-02T04:42:47.425 INFO:tasks.workunit.client.1.smithi027.stdout:6/185: unlink d13/d1b/d24/d28/f29 0 2024-04-02T04:42:47.433 INFO:tasks.workunit.client.0.smithi016.stdout:3/805: sync 2024-04-02T04:42:47.433 INFO:tasks.workunit.client.0.smithi016.stdout:7/696: sync 2024-04-02T04:42:47.434 INFO:tasks.workunit.client.1.smithi027.stdout:3/144: creat d1/de/df/d13/f2d x:0 0 0 2024-04-02T04:42:47.437 INFO:tasks.workunit.client.1.smithi027.stdout:7/187: mkdir d2/d7/da/dc/d16/d3a 0 2024-04-02T04:42:47.440 INFO:tasks.workunit.client.1.smithi027.stdout:4/176: creat d24/d2d/f31 x:0 0 0 2024-04-02T04:42:47.446 INFO:tasks.workunit.client.1.smithi027.stdout:9/179: symlink d1a/l3d 0 2024-04-02T04:42:47.456 INFO:tasks.workunit.client.1.smithi027.stdout:0/221: truncate d4/d16/d2b/f2a 282840 0 2024-04-02T04:42:47.457 INFO:tasks.workunit.client.1.smithi027.stdout:0/222: chown d4/d16/d1b/c46 45 1 2024-04-02T04:42:47.458 INFO:tasks.workunit.client.1.smithi027.stdout:3/145: dread d1/f10 [0,4194304] 0 2024-04-02T04:42:47.458 INFO:tasks.workunit.client.1.smithi027.stdout:3/146: chown d1/f8 500 1 2024-04-02T04:42:47.460 INFO:tasks.workunit.client.1.smithi027.stdout:1/147: rename d0/d1e/l34 to d0/d1e/d2e/l36 0 2024-04-02T04:42:47.461 INFO:tasks.workunit.client.1.smithi027.stdout:1/148: chown d0/d1e/f2f 115171 1 2024-04-02T04:42:47.465 INFO:tasks.workunit.client.1.smithi027.stdout:5/253: rmdir d0/df/d5c 0 2024-04-02T04:42:47.466 INFO:tasks.workunit.client.0.smithi016.stdout:8/697: sync 2024-04-02T04:42:47.467 INFO:tasks.workunit.client.1.smithi027.stdout:1/149: dwrite d0/f14 [0,4194304] 0 2024-04-02T04:42:47.467 INFO:tasks.workunit.client.1.smithi027.stdout:5/254: write d0/df/d11/f1f [2460006,99770] 0 2024-04-02T04:42:47.469 INFO:tasks.workunit.client.1.smithi027.stdout:8/150: link d5/l1e d5/d8/db/dd/l2f 0 2024-04-02T04:42:47.472 INFO:tasks.workunit.client.1.smithi027.stdout:8/151: fsync d5/d8/db/df/f22 0 2024-04-02T04:42:47.475 INFO:tasks.workunit.client.0.smithi016.stdout:9/773: fsync d0/f2c 0 2024-04-02T04:42:47.476 INFO:tasks.workunit.client.1.smithi027.stdout:1/150: dwrite d0/de/f17 [0,4194304] 0 2024-04-02T04:42:47.480 INFO:tasks.workunit.client.1.smithi027.stdout:7/188: rename d2/d7/da/f31 to d2/d1c/f3b 0 2024-04-02T04:42:47.484 INFO:tasks.workunit.client.0.smithi016.stdout:0/675: creat d0/d16/d24/d62/fe5 x:0 0 0 2024-04-02T04:42:47.485 INFO:tasks.workunit.client.0.smithi016.stdout:5/685: readlink d3/df/d18/d2f/la8 0 2024-04-02T04:42:47.491 INFO:tasks.workunit.client.0.smithi016.stdout:6/761: rename d0/d9/f33 to d0/d9/d16/d56/d89/d67/d6f/d7e/fef 0 2024-04-02T04:42:47.492 INFO:tasks.workunit.client.1.smithi027.stdout:4/177: write f21 [3588254,87570] 0 2024-04-02T04:42:47.492 INFO:tasks.workunit.client.0.smithi016.stdout:6/762: read d0/d78/fa0 [492963,66437] 0 2024-04-02T04:42:47.492 INFO:tasks.workunit.client.1.smithi027.stdout:2/186: getdents d1/db/d3b/d17/d12 0 2024-04-02T04:42:47.493 INFO:tasks.workunit.client.0.smithi016.stdout:6/763: chown d0/d9/d16/d56/d89/d67 17 1 2024-04-02T04:42:47.494 INFO:tasks.workunit.client.0.smithi016.stdout:6/764: chown d0/d9/d16/d46/f55 1430 1 2024-04-02T04:42:47.498 INFO:tasks.workunit.client.0.smithi016.stdout:6/765: dread d0/d9/d16/d56/d59/f86 [0,4194304] 0 2024-04-02T04:42:47.499 INFO:tasks.workunit.client.0.smithi016.stdout:7/697: creat d0/d46/d16/dc2/ff3 x:0 0 0 2024-04-02T04:42:47.500 INFO:tasks.workunit.client.0.smithi016.stdout:3/806: mkdir d2/d2e/d36/d66/d10d 0 2024-04-02T04:42:47.505 INFO:tasks.workunit.client.1.smithi027.stdout:0/223: rename d4/d16/d3b to d4/d16/d2b/d4a 0 2024-04-02T04:42:47.508 INFO:tasks.workunit.client.0.smithi016.stdout:9/774: creat d0/d3/d97/dac/ff0 x:0 0 0 2024-04-02T04:42:47.509 INFO:tasks.workunit.client.0.smithi016.stdout:9/775: chown d0/d3/d39/l66 13169300 1 2024-04-02T04:42:47.510 INFO:tasks.workunit.client.1.smithi027.stdout:6/186: getdents d13/d1b/d24/d28/d3c 0 2024-04-02T04:42:47.512 INFO:tasks.workunit.client.1.smithi027.stdout:7/189: dwrite d2/d1c/f3b [0,4194304] 0 2024-04-02T04:42:47.513 INFO:tasks.workunit.client.1.smithi027.stdout:5/255: creat d0/d1d/d61/f62 x:0 0 0 2024-04-02T04:42:47.515 INFO:tasks.workunit.client.1.smithi027.stdout:5/256: chown d0/d5 4106007 1 2024-04-02T04:42:47.515 INFO:tasks.workunit.client.1.smithi027.stdout:7/190: stat d2/d7/da/dc/d1f/c29 0 2024-04-02T04:42:47.516 INFO:tasks.workunit.client.0.smithi016.stdout:0/676: write d0/d16/d24/f2a [2552623,20383] 0 2024-04-02T04:42:47.516 INFO:tasks.workunit.client.0.smithi016.stdout:9/776: dwrite d0/d9/fcf [0,4194304] 0 2024-04-02T04:42:47.517 INFO:tasks.workunit.client.1.smithi027.stdout:4/178: creat d24/f32 x:0 0 0 2024-04-02T04:42:47.519 INFO:tasks.workunit.client.0.smithi016.stdout:5/686: mknod d3/d40/d72/cfb 0 2024-04-02T04:42:47.520 INFO:tasks.workunit.client.1.smithi027.stdout:7/191: truncate d2/d7/da/dc/f30 701296 0 2024-04-02T04:42:47.521 INFO:tasks.workunit.client.1.smithi027.stdout:8/152: link d5/d8/f2e d5/d8/db/f30 0 2024-04-02T04:42:47.521 INFO:tasks.workunit.client.1.smithi027.stdout:6/187: dwrite d13/d1b/f2b [0,4194304] 0 2024-04-02T04:42:47.523 INFO:tasks.workunit.client.0.smithi016.stdout:9/777: dwrite d0/d4e/fad [4194304,4194304] 0 2024-04-02T04:42:47.524 INFO:tasks.workunit.client.1.smithi027.stdout:3/147: creat d1/f2e x:0 0 0 2024-04-02T04:42:47.526 INFO:tasks.workunit.client.1.smithi027.stdout:3/148: stat d1/d1a/l2b 0 2024-04-02T04:42:47.532 INFO:tasks.workunit.client.1.smithi027.stdout:7/192: dwrite d2/d7/f18 [0,4194304] 0 2024-04-02T04:42:47.532 INFO:tasks.workunit.client.0.smithi016.stdout:1/685: unlink d14/d19/d1d/d29/d39/d46/l4a 0 2024-04-02T04:42:47.534 INFO:tasks.workunit.client.1.smithi027.stdout:5/257: creat d0/d5/f63 x:0 0 0 2024-04-02T04:42:47.536 INFO:tasks.workunit.client.1.smithi027.stdout:7/193: write d2/d7/da/dc/f30 [288999,123912] 0 2024-04-02T04:42:47.542 INFO:tasks.workunit.client.0.smithi016.stdout:7/698: dread - d0/d21/f86 zero size 2024-04-02T04:42:47.543 INFO:tasks.workunit.client.1.smithi027.stdout:5/258: dwrite d0/df/d11/d13/f28 [0,4194304] 0 2024-04-02T04:42:47.544 INFO:tasks.workunit.client.1.smithi027.stdout:8/153: readlink d5/l1e 0 2024-04-02T04:42:47.545 INFO:tasks.workunit.client.1.smithi027.stdout:5/259: dread - d0/df/d11/d37/f40 zero size 2024-04-02T04:42:47.545 INFO:tasks.workunit.client.0.smithi016.stdout:3/807: fsync d2/d2e/d36/d66/d90/fed 0 2024-04-02T04:42:47.546 INFO:tasks.workunit.client.1.smithi027.stdout:5/260: chown d0/d5/f2c 11775685 1 2024-04-02T04:42:47.546 INFO:tasks.workunit.client.0.smithi016.stdout:3/808: stat d2/d2e/f30 0 2024-04-02T04:42:47.547 INFO:tasks.workunit.client.0.smithi016.stdout:7/699: read d0/d46/d16/dc2/dea/f42 [3446177,126557] 0 2024-04-02T04:42:47.549 INFO:tasks.workunit.client.0.smithi016.stdout:7/700: write d0/d46/d37/d3a/d84/fd0 [325115,99877] 0 2024-04-02T04:42:47.562 INFO:tasks.workunit.client.1.smithi027.stdout:1/151: creat d0/f37 x:0 0 0 2024-04-02T04:42:47.562 INFO:tasks.workunit.client.1.smithi027.stdout:1/152: stat d0/de/l1a 0 2024-04-02T04:42:47.563 INFO:tasks.workunit.client.1.smithi027.stdout:1/153: write d0/f1d [3671453,121440] 0 2024-04-02T04:42:47.563 INFO:tasks.workunit.client.1.smithi027.stdout:6/188: rename d13/d1b/d24/d25/d27/f3e to d13/d1b/d24/d25/d27/f44 0 2024-04-02T04:42:47.564 INFO:tasks.workunit.client.1.smithi027.stdout:6/189: dread - d13/f17 zero size 2024-04-02T04:42:47.567 INFO:tasks.workunit.client.0.smithi016.stdout:4/718: link d5/d16/c80 d5/d17/d76/dca/cd9 0 2024-04-02T04:42:47.567 INFO:tasks.workunit.client.0.smithi016.stdout:2/762: write d12/d26/f2c [406642,30067] 0 2024-04-02T04:42:47.568 INFO:tasks.workunit.client.1.smithi027.stdout:3/149: mknod d1/de/c2f 0 2024-04-02T04:42:47.570 INFO:tasks.workunit.client.1.smithi027.stdout:0/224: write d4/fb [1993416,37189] 0 2024-04-02T04:42:47.571 INFO:tasks.workunit.client.0.smithi016.stdout:0/677: rmdir d0/d48 39 2024-04-02T04:42:47.573 INFO:tasks.workunit.client.0.smithi016.stdout:6/766: write d0/d9/d16/d56/d89/d67/d6f/d8a/fa5 [85623,131008] 0 2024-04-02T04:42:47.574 INFO:tasks.workunit.client.1.smithi027.stdout:7/194: symlink d2/d7/da/dc/d16/d2f/l3c 0 2024-04-02T04:42:47.576 INFO:tasks.workunit.client.1.smithi027.stdout:1/154: dwrite d0/fd [0,4194304] 0 2024-04-02T04:42:47.577 INFO:tasks.workunit.client.1.smithi027.stdout:6/190: dwrite d13/f14 [0,4194304] 0 2024-04-02T04:42:47.579 INFO:tasks.workunit.client.1.smithi027.stdout:4/179: link f13 d24/f33 0 2024-04-02T04:42:47.580 INFO:tasks.workunit.client.1.smithi027.stdout:6/191: fsync d13/d1b/d24/d25/d27/f3f 0 2024-04-02T04:42:47.580 INFO:tasks.workunit.client.1.smithi027.stdout:4/180: chown f12 932701871 1 2024-04-02T04:42:47.581 INFO:tasks.workunit.client.1.smithi027.stdout:1/155: truncate d0/d1e/f2f 535202 0 2024-04-02T04:42:47.581 INFO:tasks.workunit.client.1.smithi027.stdout:6/192: chown c8 243637 1 2024-04-02T04:42:47.585 INFO:tasks.workunit.client.1.smithi027.stdout:8/154: mkdir d5/d7/d27/d31 0 2024-04-02T04:42:47.586 INFO:tasks.workunit.client.1.smithi027.stdout:6/193: fdatasync d13/f17 0 2024-04-02T04:42:47.587 INFO:tasks.workunit.client.1.smithi027.stdout:5/261: creat d0/df/d2f/d3b/f64 x:0 0 0 2024-04-02T04:42:47.589 INFO:tasks.workunit.client.0.smithi016.stdout:1/686: dread d14/f83 [0,4194304] 0 2024-04-02T04:42:47.591 INFO:tasks.workunit.client.1.smithi027.stdout:0/225: symlink d4/d3d/l4b 0 2024-04-02T04:42:47.592 INFO:tasks.workunit.client.0.smithi016.stdout:3/809: unlink d2/d15/d16/d8a/dd1/d6e/d6f/db0/f62 0 2024-04-02T04:42:47.593 INFO:tasks.workunit.client.1.smithi027.stdout:7/195: dwrite d2/d7/da/dc/f30 [0,4194304] 0 2024-04-02T04:42:47.593 INFO:tasks.workunit.client.1.smithi027.stdout:1/156: dwrite d0/f30 [0,4194304] 0 2024-04-02T04:42:47.596 INFO:tasks.workunit.client.1.smithi027.stdout:3/150: rename d1/de/df/d13/f25 to d1/de/f30 0 2024-04-02T04:42:47.598 INFO:tasks.workunit.client.0.smithi016.stdout:4/719: mknod d5/d16/d88/db0/cda 0 2024-04-02T04:42:47.599 INFO:tasks.workunit.client.0.smithi016.stdout:4/720: dread - d5/d8/d43/f6e zero size 2024-04-02T04:42:47.602 INFO:tasks.workunit.client.0.smithi016.stdout:2/763: mkdir d12/d35/d3b/d42/d59/d91/d10a 0 2024-04-02T04:42:47.603 INFO:tasks.workunit.client.1.smithi027.stdout:4/181: truncate f16 785193 0 2024-04-02T04:42:47.605 INFO:tasks.workunit.client.0.smithi016.stdout:0/678: mkdir d0/d16/d24/d4e/de6 0 2024-04-02T04:42:47.606 INFO:tasks.workunit.client.0.smithi016.stdout:6/767: readlink d0/d9/la9 0 2024-04-02T04:42:47.606 INFO:tasks.workunit.client.1.smithi027.stdout:4/182: dread f14 [0,4194304] 0 2024-04-02T04:42:47.607 INFO:tasks.workunit.client.1.smithi027.stdout:5/262: mkdir d0/df/d11/d65 0 2024-04-02T04:42:47.608 INFO:tasks.workunit.client.1.smithi027.stdout:5/263: write d0/f47 [727568,79933] 0 2024-04-02T04:42:47.610 INFO:tasks.workunit.client.0.smithi016.stdout:9/778: creat d0/d3/d97/dde/ff1 x:0 0 0 2024-04-02T04:42:47.610 INFO:tasks.workunit.client.0.smithi016.stdout:9/779: chown d0/d9/db/dc9/f45 111505 1 2024-04-02T04:42:47.611 INFO:tasks.workunit.client.0.smithi016.stdout:9/780: readlink d0/d9/db/dc9/d53/l6b 0 2024-04-02T04:42:47.612 INFO:tasks.workunit.client.1.smithi027.stdout:0/226: dread d4/d16/d2b/f2a [0,4194304] 0 2024-04-02T04:42:47.612 INFO:tasks.workunit.client.0.smithi016.stdout:9/781: fdatasync d0/d9/db/dc9/d53/f73 0 2024-04-02T04:42:47.613 INFO:tasks.workunit.client.0.smithi016.stdout:9/782: chown d0/d9/d4d/f82 3434312 1 2024-04-02T04:42:47.615 INFO:tasks.workunit.client.1.smithi027.stdout:8/155: rename d5/d8/db/f30 to d5/d7/d27/d31/f32 0 2024-04-02T04:42:47.617 INFO:tasks.workunit.client.0.smithi016.stdout:1/687: truncate d14/d19/d1d/d29/f6a 2776278 0 2024-04-02T04:42:47.620 INFO:tasks.workunit.client.0.smithi016.stdout:3/810: symlink d2/d15/d7e/l10e 0 2024-04-02T04:42:47.623 INFO:tasks.workunit.client.0.smithi016.stdout:8/698: getdents d1/d6/d36 0 2024-04-02T04:42:47.625 INFO:tasks.workunit.client.0.smithi016.stdout:4/721: rename d5/de/la2 to d5/d17/d18/d6d/d8e/ldb 0 2024-04-02T04:42:47.628 INFO:tasks.workunit.client.0.smithi016.stdout:2/764: write d12/d35/d3f/f39 [765973,115025] 0 2024-04-02T04:42:47.628 INFO:tasks.workunit.client.0.smithi016.stdout:6/768: chown d0/d5c/f6c 6239319 1 2024-04-02T04:42:47.629 INFO:tasks.workunit.client.0.smithi016.stdout:2/765: write d12/d35/d3b/d42/fdf [207738,99866] 0 2024-04-02T04:42:47.635 INFO:tasks.workunit.client.0.smithi016.stdout:2/766: dwrite d12/f43 [0,4194304] 0 2024-04-02T04:42:47.638 INFO:tasks.workunit.client.0.smithi016.stdout:0/679: rename d0/l6 to d0/d16/d2e/d36/d9f/le7 0 2024-04-02T04:42:47.642 INFO:tasks.workunit.client.0.smithi016.stdout:5/687: getdents d3/d5d 0 2024-04-02T04:42:47.643 INFO:tasks.workunit.client.0.smithi016.stdout:5/688: stat d3/df/fb9 0 2024-04-02T04:42:47.644 INFO:tasks.workunit.client.0.smithi016.stdout:5/689: write d3/df/f1f [3084526,111348] 0 2024-04-02T04:42:47.645 INFO:tasks.workunit.client.0.smithi016.stdout:5/690: read d3/d4c/dd1/f52 [1467182,75220] 0 2024-04-02T04:42:47.645 INFO:tasks.workunit.client.0.smithi016.stdout:3/811: mkdir d2/d15/d10f 0 2024-04-02T04:42:47.647 INFO:tasks.workunit.client.0.smithi016.stdout:5/691: write d3/df/d18/d26/ddc/d6d/fa7 [12667945,79586] 0 2024-04-02T04:42:47.650 INFO:tasks.workunit.client.0.smithi016.stdout:0/680: creat d0/d16/fe8 x:0 0 0 2024-04-02T04:42:47.651 INFO:tasks.workunit.client.0.smithi016.stdout:0/681: chown d0/d16/d2e/d37/d66/f70 11 1 2024-04-02T04:42:47.653 INFO:tasks.workunit.client.1.smithi027.stdout:6/194: mknod d13/d1b/d24/c45 0 2024-04-02T04:42:47.653 INFO:tasks.workunit.client.1.smithi027.stdout:4/183: mkdir d24/d2d/d34 0 2024-04-02T04:42:47.654 INFO:tasks.workunit.client.0.smithi016.stdout:1/688: dwrite d14/d19/fdf [0,4194304] 0 2024-04-02T04:42:47.655 INFO:tasks.workunit.client.1.smithi027.stdout:4/184: dread - d24/f2f zero size 2024-04-02T04:42:47.655 INFO:tasks.workunit.client.1.smithi027.stdout:6/195: chown d13/d1b/d24/d25/d27/f44 0 1 2024-04-02T04:42:47.656 INFO:tasks.workunit.client.0.smithi016.stdout:8/699: link d1/d31/fe3 d1/d6/d36/d3f/d40/d45/fee 0 2024-04-02T04:42:47.657 INFO:tasks.workunit.client.1.smithi027.stdout:1/157: write d0/f6 [886780,129401] 0 2024-04-02T04:42:47.658 INFO:tasks.workunit.client.1.smithi027.stdout:6/196: chown d13/c1c 44591 1 2024-04-02T04:42:47.659 INFO:tasks.workunit.client.1.smithi027.stdout:8/156: dread d5/d7/f2b [0,4194304] 0 2024-04-02T04:42:47.659 INFO:tasks.workunit.client.1.smithi027.stdout:1/158: chown d0/d1e/c2d 4169091 1 2024-04-02T04:42:47.659 INFO:tasks.workunit.client.0.smithi016.stdout:8/700: truncate d1/d6/d36/d3f/d40/d49/fab 1612180 0 2024-04-02T04:42:47.662 INFO:tasks.workunit.client.0.smithi016.stdout:5/692: mkdir d3/d40/db6/dfc 0 2024-04-02T04:42:47.664 INFO:tasks.workunit.client.1.smithi027.stdout:4/185: creat d24/f35 x:0 0 0 2024-04-02T04:42:47.665 INFO:tasks.workunit.client.1.smithi027.stdout:6/197: dread d13/f14 [0,4194304] 0 2024-04-02T04:42:47.666 INFO:tasks.workunit.client.1.smithi027.stdout:4/186: chown d24/f35 79569 1 2024-04-02T04:42:47.666 INFO:tasks.workunit.client.0.smithi016.stdout:4/722: dwrite d5/d17/d18/d25/f3b [0,4194304] 0 2024-04-02T04:42:47.667 INFO:tasks.workunit.client.1.smithi027.stdout:6/198: chown d13/d1b/l21 25430425 1 2024-04-02T04:42:47.667 INFO:tasks.workunit.client.1.smithi027.stdout:6/199: stat d13/d1b/f1e 0 2024-04-02T04:42:47.668 INFO:tasks.workunit.client.0.smithi016.stdout:8/701: dwrite d1/d6/d86/fec [0,4194304] 0 2024-04-02T04:42:47.668 INFO:tasks.workunit.client.1.smithi027.stdout:1/159: creat d0/de/f38 x:0 0 0 2024-04-02T04:42:47.669 INFO:tasks.workunit.client.1.smithi027.stdout:1/160: write d0/fd [176374,16779] 0 2024-04-02T04:42:47.671 INFO:tasks.workunit.client.1.smithi027.stdout:5/264: creat d0/f66 x:0 0 0 2024-04-02T04:42:47.673 INFO:tasks.workunit.client.1.smithi027.stdout:8/157: fdatasync d5/d7/d27/d31/f32 0 2024-04-02T04:42:47.675 INFO:tasks.workunit.client.0.smithi016.stdout:2/767: creat d12/d35/d3b/d42/d67/f10b x:0 0 0 2024-04-02T04:42:47.675 INFO:tasks.workunit.client.1.smithi027.stdout:0/227: dread d4/f1d [0,4194304] 0 2024-04-02T04:42:47.677 INFO:tasks.workunit.client.0.smithi016.stdout:9/783: rename d0/d4a/d99/d46/f92 to d0/d9/db/ff2 0 2024-04-02T04:42:47.677 INFO:tasks.workunit.client.0.smithi016.stdout:4/723: dwrite d5/d1d/f78 [0,4194304] 0 2024-04-02T04:42:47.679 INFO:tasks.workunit.client.1.smithi027.stdout:4/187: write f15 [5724012,5957] 0 2024-04-02T04:42:47.679 INFO:tasks.workunit.client.0.smithi016.stdout:4/724: write d5/d17/d18/fb3 [25044,12100] 0 2024-04-02T04:42:47.684 INFO:tasks.workunit.client.1.smithi027.stdout:6/200: mkdir d13/d1b/d24/d28/d46 0 2024-04-02T04:42:47.685 INFO:tasks.workunit.client.0.smithi016.stdout:9/784: dwrite d0/d9/f2e [0,4194304] 0 2024-04-02T04:42:47.689 INFO:tasks.workunit.client.0.smithi016.stdout:0/682: symlink d0/le9 0 2024-04-02T04:42:47.690 INFO:tasks.workunit.client.0.smithi016.stdout:6/769: getdents d0/d9/d16/d56/db0 0 2024-04-02T04:42:47.691 INFO:tasks.workunit.client.0.smithi016.stdout:0/683: write d0/d16/d24/d4e/d8d/d71/f72 [4282200,91573] 0 2024-04-02T04:42:47.693 INFO:tasks.workunit.client.0.smithi016.stdout:0/684: dread d0/d16/d24/d62/f9b [0,4194304] 0 2024-04-02T04:42:47.693 INFO:tasks.workunit.client.1.smithi027.stdout:5/265: creat d0/df/d2f/f67 x:0 0 0 2024-04-02T04:42:47.695 INFO:tasks.workunit.client.1.smithi027.stdout:5/266: write d0/df/d11/d37/f4e [315939,55771] 0 2024-04-02T04:42:47.700 INFO:tasks.workunit.client.1.smithi027.stdout:8/158: creat d5/d8/f33 x:0 0 0 2024-04-02T04:42:47.700 INFO:tasks.workunit.client.0.smithi016.stdout:3/812: link d2/d15/f24 d2/d2e/d36/d66/d90/dfb/f110 0 2024-04-02T04:42:47.701 INFO:tasks.workunit.client.0.smithi016.stdout:3/813: write d2/d2e/d36/d48/fe1 [277670,33905] 0 2024-04-02T04:42:47.702 INFO:tasks.workunit.client.1.smithi027.stdout:7/196: rename d2/d7/da/dc/d1f/c39 to d2/c3d 0 2024-04-02T04:42:47.704 INFO:tasks.workunit.client.1.smithi027.stdout:0/228: mkdir d4/d16/d2b/d2d/d4c 0 2024-04-02T04:42:47.709 INFO:tasks.workunit.client.0.smithi016.stdout:5/693: mkdir d3/df/d18/dfd 0 2024-04-02T04:42:47.710 INFO:tasks.workunit.client.0.smithi016.stdout:4/725: dread d5/d17/d18/d1f/f2a [0,4194304] 0 2024-04-02T04:42:47.711 INFO:tasks.workunit.client.0.smithi016.stdout:4/726: readlink d5/d17/d18/d1f/l73 0 2024-04-02T04:42:47.712 INFO:tasks.workunit.client.0.smithi016.stdout:9/785: unlink d0/d4a/d99/c6f 0 2024-04-02T04:42:47.712 INFO:tasks.workunit.client.0.smithi016.stdout:8/702: dread d1/d31/d67/d8f/fbe [0,4194304] 0 2024-04-02T04:42:47.712 INFO:tasks.workunit.client.0.smithi016.stdout:4/727: write d5/d16/d88/db0/fd1 [84284,92623] 0 2024-04-02T04:42:47.713 INFO:tasks.workunit.client.1.smithi027.stdout:5/267: chown d0/f6 470340770 1 2024-04-02T04:42:47.713 INFO:tasks.workunit.client.1.smithi027.stdout:5/268: chown d0/df/d11 1590 1 2024-04-02T04:42:47.714 INFO:tasks.workunit.client.0.smithi016.stdout:1/689: fdatasync d14/d19/d1d/d29/f6a 0 2024-04-02T04:42:47.715 INFO:tasks.workunit.client.1.smithi027.stdout:1/161: truncate d0/de/f17 1107087 0 2024-04-02T04:42:47.715 INFO:tasks.workunit.client.0.smithi016.stdout:1/690: stat d14/d19/f82 0 2024-04-02T04:42:47.716 INFO:tasks.workunit.client.0.smithi016.stdout:8/703: truncate d1/d6/d36/d3f/d40/fca 413372 0 2024-04-02T04:42:47.721 INFO:tasks.workunit.client.1.smithi027.stdout:4/188: rename c1a to d24/c36 0 2024-04-02T04:42:47.722 INFO:tasks.workunit.client.1.smithi027.stdout:4/189: stat f15 0 2024-04-02T04:42:47.722 INFO:tasks.workunit.client.0.smithi016.stdout:4/728: dwrite d5/d17/d18/d6d/d8e/d9c/dcc/fa8 [0,4194304] 0 2024-04-02T04:42:47.723 INFO:tasks.workunit.client.1.smithi027.stdout:4/190: dread - d24/f32 zero size 2024-04-02T04:42:47.725 INFO:tasks.workunit.client.0.smithi016.stdout:1/691: dwrite d14/d19/d7a/fab [0,4194304] 0 2024-04-02T04:42:47.725 INFO:tasks.workunit.client.1.smithi027.stdout:7/197: mknod d2/d7/da/dc/d16/d2f/c3e 0 2024-04-02T04:42:47.726 INFO:tasks.workunit.client.0.smithi016.stdout:0/685: rmdir d0/d16/d2e/d36 39 2024-04-02T04:42:47.728 INFO:tasks.workunit.client.1.smithi027.stdout:6/201: mknod d13/d1b/d24/d25/d42/c47 0 2024-04-02T04:42:47.733 INFO:tasks.workunit.client.1.smithi027.stdout:8/159: dwrite d5/d8/db/dd/f1f [0,4194304] 0 2024-04-02T04:42:47.733 INFO:tasks.workunit.client.0.smithi016.stdout:1/692: dwrite d14/d19/d1d/d29/d39/d46/dcb/d43/f96 [0,4194304] 0 2024-04-02T04:42:47.734 INFO:tasks.workunit.client.0.smithi016.stdout:6/770: dwrite d0/d9/d16/d56/f4f [0,4194304] 0 2024-04-02T04:42:47.738 INFO:tasks.workunit.client.0.smithi016.stdout:1/693: fsync d14/d19/d1e/d5e/fc3 0 2024-04-02T04:42:47.739 INFO:tasks.workunit.client.1.smithi027.stdout:9/180: sync 2024-04-02T04:42:47.739 INFO:tasks.workunit.client.1.smithi027.stdout:9/181: readlink d1a/d1d/d29/d37/l1e 0 2024-04-02T04:42:47.740 INFO:tasks.workunit.client.1.smithi027.stdout:7/198: mkdir d2/d7/da/dc/d16/d3f 0 2024-04-02T04:42:47.742 INFO:tasks.workunit.client.0.smithi016.stdout:5/694: creat d3/df/d18/d26/d70/ffe x:0 0 0 2024-04-02T04:42:47.745 INFO:tasks.workunit.client.1.smithi027.stdout:8/160: dwrite d5/d7/f14 [0,4194304] 0 2024-04-02T04:42:47.745 INFO:tasks.workunit.client.0.smithi016.stdout:9/786: rename d0/d4a/d99/d46/f87 to d0/d4a/da9/ff3 0 2024-04-02T04:42:47.748 INFO:tasks.workunit.client.1.smithi027.stdout:6/202: mkdir d13/d1b/d48 0 2024-04-02T04:42:47.748 INFO:tasks.workunit.client.0.smithi016.stdout:7/701: sync 2024-04-02T04:42:47.753 INFO:tasks.workunit.client.0.smithi016.stdout:5/695: dwrite d3/df/d86/fe6 [0,4194304] 0 2024-04-02T04:42:47.753 INFO:tasks.workunit.client.1.smithi027.stdout:5/269: mkdir d0/d68 0 2024-04-02T04:42:47.755 INFO:tasks.workunit.client.1.smithi027.stdout:5/270: truncate d0/d1a/f49 1327334 0 2024-04-02T04:42:47.757 INFO:tasks.workunit.client.0.smithi016.stdout:3/814: write d2/d15/f4b [6521344,90913] 0 2024-04-02T04:42:47.760 INFO:tasks.workunit.client.0.smithi016.stdout:4/729: rmdir d5/d17/d18/da0 39 2024-04-02T04:42:47.762 INFO:tasks.workunit.client.1.smithi027.stdout:4/191: link d24/f2c d24/d2d/d34/f37 0 2024-04-02T04:42:47.763 INFO:tasks.workunit.client.0.smithi016.stdout:5/696: dwrite d3/df/d18/d26/d70/ffe [0,4194304] 0 2024-04-02T04:42:47.764 INFO:tasks.workunit.client.0.smithi016.stdout:0/686: creat d0/d16/d2e/d53/d6f/fea x:0 0 0 2024-04-02T04:42:47.764 INFO:tasks.workunit.client.1.smithi027.stdout:7/199: write d2/f13 [3180328,102800] 0 2024-04-02T04:42:47.765 INFO:tasks.workunit.client.1.smithi027.stdout:9/182: rename d1a/d1d/d29/d37/d20/c2c to d1a/d1d/d29/d37/d20/c3e 0 2024-04-02T04:42:47.766 INFO:tasks.workunit.client.0.smithi016.stdout:0/687: chown d0/d16/d23/f3f 30763 1 2024-04-02T04:42:47.767 INFO:tasks.workunit.client.0.smithi016.stdout:2/768: truncate d12/d70/f75 2242384 0 2024-04-02T04:42:47.769 INFO:tasks.workunit.client.1.smithi027.stdout:6/203: rmdir d13/d1b/d24/d25/d27 39 2024-04-02T04:42:47.772 INFO:tasks.workunit.client.1.smithi027.stdout:0/229: truncate d4/fb 988987 0 2024-04-02T04:42:47.775 INFO:tasks.workunit.client.1.smithi027.stdout:4/192: mkdir d24/d38 0 2024-04-02T04:42:47.775 INFO:tasks.workunit.client.1.smithi027.stdout:4/193: stat d24/f33 0 2024-04-02T04:42:47.777 INFO:tasks.workunit.client.1.smithi027.stdout:6/204: dwrite d13/d1b/d24/f3a [0,4194304] 0 2024-04-02T04:42:47.778 INFO:tasks.workunit.client.1.smithi027.stdout:7/200: read f1 [808205,118836] 0 2024-04-02T04:42:47.779 INFO:tasks.workunit.client.0.smithi016.stdout:8/704: write d1/d6/d36/d3f/d40/d49/d4a/fa8 [240433,114120] 0 2024-04-02T04:42:47.783 INFO:tasks.workunit.client.1.smithi027.stdout:9/183: creat d1a/d1d/d29/d37/d20/f3f x:0 0 0 2024-04-02T04:42:47.787 INFO:tasks.workunit.client.1.smithi027.stdout:8/161: dread d5/d7/f24 [4194304,4194304] 0 2024-04-02T04:42:47.787 INFO:tasks.workunit.client.1.smithi027.stdout:7/201: dwrite d2/df/f10 [0,4194304] 0 2024-04-02T04:42:47.788 INFO:tasks.workunit.client.1.smithi027.stdout:1/162: truncate d0/de/ff 3605687 0 2024-04-02T04:42:47.789 INFO:tasks.workunit.client.1.smithi027.stdout:7/202: chown d2/d7/da/d35 1 1 2024-04-02T04:42:47.789 INFO:tasks.workunit.client.0.smithi016.stdout:7/702: creat d0/d46/d37/dbe/ff4 x:0 0 0 2024-04-02T04:42:47.792 INFO:tasks.workunit.client.1.smithi027.stdout:4/194: creat d24/f39 x:0 0 0 2024-04-02T04:42:47.795 INFO:tasks.workunit.client.0.smithi016.stdout:9/787: rename d0/d4e/l7b to d0/d4a/da9/dd8/lf4 0 2024-04-02T04:42:47.797 INFO:tasks.workunit.client.1.smithi027.stdout:6/205: unlink d13/d1b/l21 0 2024-04-02T04:42:47.800 INFO:tasks.workunit.client.1.smithi027.stdout:9/184: symlink d1a/d1d/l40 0 2024-04-02T04:42:47.801 INFO:tasks.workunit.client.0.smithi016.stdout:3/815: mknod d2/d15/d70/c111 0 2024-04-02T04:42:47.802 INFO:tasks.workunit.client.0.smithi016.stdout:3/816: write d2/d15/d16/d8a/dd1/d6e/d6f/db0/de2/fe4 [193126,260] 0 2024-04-02T04:42:47.804 INFO:tasks.workunit.client.0.smithi016.stdout:3/817: dread d2/d2e/d36/d66/d90/fdf [0,4194304] 0 2024-04-02T04:42:47.804 INFO:tasks.workunit.client.0.smithi016.stdout:4/730: truncate d5/d1d/d1e/f9d 361160 0 2024-04-02T04:42:47.805 INFO:tasks.workunit.client.1.smithi027.stdout:6/206: dwrite d13/d1b/d24/f41 [0,4194304] 0 2024-04-02T04:42:47.805 INFO:tasks.workunit.client.0.smithi016.stdout:6/771: write d0/d9/df/f1b [1406613,52474] 0 2024-04-02T04:42:47.806 INFO:tasks.workunit.client.0.smithi016.stdout:3/818: fdatasync d2/d2e/d36/d48/da2/de7/f109 0 2024-04-02T04:42:47.806 INFO:tasks.workunit.client.1.smithi027.stdout:1/163: creat d0/d1e/d25/f39 x:0 0 0 2024-04-02T04:42:47.807 INFO:tasks.workunit.client.1.smithi027.stdout:1/164: write d0/d1e/f2f [1574016,110776] 0 2024-04-02T04:42:47.810 INFO:tasks.workunit.client.0.smithi016.stdout:0/688: unlink d0/d16/d2e/d53/ld7 0 2024-04-02T04:42:47.811 INFO:tasks.workunit.client.1.smithi027.stdout:8/162: rename d5/d7/c26 to d5/d8/db/df/c34 0 2024-04-02T04:42:47.813 INFO:tasks.workunit.client.1.smithi027.stdout:7/203: creat d2/d1c/f40 x:0 0 0 2024-04-02T04:42:47.815 INFO:tasks.workunit.client.0.smithi016.stdout:1/694: mknod d14/d19/d1e/d71/dcf/ce1 0 2024-04-02T04:42:47.818 INFO:tasks.workunit.client.1.smithi027.stdout:8/163: dwrite d5/d8/f33 [0,4194304] 0 2024-04-02T04:42:47.819 INFO:tasks.workunit.client.0.smithi016.stdout:7/703: mknod d0/d46/dbf/cf5 0 2024-04-02T04:42:47.820 INFO:tasks.workunit.client.1.smithi027.stdout:8/164: fsync d5/d8/db/dd/f21 0 2024-04-02T04:42:47.827 INFO:tasks.workunit.client.0.smithi016.stdout:2/769: rename d12/d35/d101/d76/lc6 to d12/d35/d101/d76/da0/dbe/l10c 0 2024-04-02T04:42:47.828 INFO:tasks.workunit.client.0.smithi016.stdout:2/770: readlink d12/d35/d3f/d56/la8 0 2024-04-02T04:42:47.831 INFO:tasks.workunit.client.1.smithi027.stdout:9/185: symlink d1a/d1d/d29/d37/d3a/l41 0 2024-04-02T04:42:47.832 INFO:tasks.workunit.client.1.smithi027.stdout:9/186: chown l16 18779 1 2024-04-02T04:42:47.834 INFO:tasks.workunit.client.0.smithi016.stdout:2/771: dwrite d12/d9f/fc9 [0,4194304] 0 2024-04-02T04:42:47.835 INFO:tasks.workunit.client.1.smithi027.stdout:0/230: getdents d4/d16/d2b/d4a 0 2024-04-02T04:42:47.837 INFO:tasks.workunit.client.0.smithi016.stdout:6/772: creat d0/d9/d16/d56/d89/d67/d6f/d7e/ff0 x:0 0 0 2024-04-02T04:42:47.838 INFO:tasks.workunit.client.0.smithi016.stdout:6/773: write d0/d9/df/f1b [573789,46278] 0 2024-04-02T04:42:47.838 INFO:tasks.workunit.client.1.smithi027.stdout:0/231: dread d4/f23 [0,4194304] 0 2024-04-02T04:42:47.840 INFO:tasks.workunit.client.0.smithi016.stdout:6/774: write d0/d9/d16/d56/d89/d67/d6f/f90 [270413,37129] 0 2024-04-02T04:42:47.841 INFO:tasks.workunit.client.1.smithi027.stdout:8/165: dread d5/d8/db/dd/f10 [0,4194304] 0 2024-04-02T04:42:47.841 INFO:tasks.workunit.client.1.smithi027.stdout:6/207: symlink d13/l49 0 2024-04-02T04:42:47.843 INFO:tasks.workunit.client.1.smithi027.stdout:8/166: stat d5/d7/d27/d31 0 2024-04-02T04:42:47.844 INFO:tasks.workunit.client.1.smithi027.stdout:1/165: truncate d0/de/f17 1936872 0 2024-04-02T04:42:47.845 INFO:tasks.workunit.client.1.smithi027.stdout:8/167: chown d5/d8/db/dd/f1d 5534 1 2024-04-02T04:42:47.847 INFO:tasks.workunit.client.1.smithi027.stdout:9/187: creat d1a/d1d/f42 x:0 0 0 2024-04-02T04:42:47.851 INFO:tasks.workunit.client.1.smithi027.stdout:9/188: stat d1a/d1d/d29/d37/d20/l28 0 2024-04-02T04:42:47.851 INFO:tasks.workunit.client.1.smithi027.stdout:6/208: dwrite d13/d1b/d24/d28/f35 [0,4194304] 0 2024-04-02T04:42:47.852 INFO:tasks.workunit.client.1.smithi027.stdout:9/189: chown d1a/l2a 78918 1 2024-04-02T04:42:47.855 INFO:tasks.workunit.client.1.smithi027.stdout:1/166: dwrite d0/f6 [0,4194304] 0 2024-04-02T04:42:47.863 INFO:tasks.workunit.client.1.smithi027.stdout:1/167: dwrite d0/d1e/d25/f39 [0,4194304] 0 2024-04-02T04:42:47.864 INFO:tasks.workunit.client.0.smithi016.stdout:8/705: truncate d1/d6/d36/f99 2016602 0 2024-04-02T04:42:47.870 INFO:tasks.workunit.client.1.smithi027.stdout:6/209: unlink d13/l33 0 2024-04-02T04:42:47.874 INFO:tasks.workunit.client.0.smithi016.stdout:2/772: creat d12/d35/d101/d76/da0/dbe/f10d x:0 0 0 2024-04-02T04:42:47.874 INFO:tasks.workunit.client.1.smithi027.stdout:0/232: creat d4/d16/d2b/d2d/d4c/f4d x:0 0 0 2024-04-02T04:42:47.878 INFO:tasks.workunit.client.0.smithi016.stdout:3/819: mknod d2/d15/d16/d8a/dd1/c112 0 2024-04-02T04:42:47.879 INFO:tasks.workunit.client.0.smithi016.stdout:3/820: stat d2/d15/d16/d8a/dcd 0 2024-04-02T04:42:47.881 INFO:tasks.workunit.client.0.smithi016.stdout:6/775: unlink d0/d9/d16/d56/d89/d67/d6f/d8a/fe5 0 2024-04-02T04:42:47.883 INFO:tasks.workunit.client.1.smithi027.stdout:1/168: creat d0/d1e/f3a x:0 0 0 2024-04-02T04:42:47.884 INFO:tasks.workunit.client.0.smithi016.stdout:0/689: mkdir d0/d16/d24/deb 0 2024-04-02T04:42:47.885 INFO:tasks.workunit.client.1.smithi027.stdout:8/168: write d5/d7/d27/d31/f32 [513195,65562] 0 2024-04-02T04:42:47.886 INFO:tasks.workunit.client.1.smithi027.stdout:2/187: sync 2024-04-02T04:42:47.886 INFO:tasks.workunit.client.1.smithi027.stdout:3/151: sync 2024-04-02T04:42:47.887 INFO:tasks.workunit.client.1.smithi027.stdout:2/188: readlink d1/db/d3b/l3c 0 2024-04-02T04:42:47.887 INFO:tasks.workunit.client.0.smithi016.stdout:0/690: dread d0/d16/d2e/d37/fae [0,4194304] 0 2024-04-02T04:42:47.888 INFO:tasks.workunit.client.0.smithi016.stdout:1/695: mkdir d14/d19/d20/de2 0 2024-04-02T04:42:47.889 INFO:tasks.workunit.client.0.smithi016.stdout:1/696: chown d14/d19/d1d/d29/d39/l99 30279 1 2024-04-02T04:42:47.890 INFO:tasks.workunit.client.1.smithi027.stdout:9/190: symlink d1a/l43 0 2024-04-02T04:42:47.893 INFO:tasks.workunit.client.1.smithi027.stdout:9/191: write d1a/f26 [72497,34976] 0 2024-04-02T04:42:47.893 INFO:tasks.workunit.client.1.smithi027.stdout:9/192: readlink d1a/d1d/d29/d37/d3a/l41 0 2024-04-02T04:42:47.893 INFO:tasks.workunit.client.1.smithi027.stdout:1/169: mkdir d0/d1e/d2e/d3b 0 2024-04-02T04:42:47.893 INFO:tasks.workunit.client.0.smithi016.stdout:1/697: fsync d14/d19/f69 0 2024-04-02T04:42:47.893 INFO:tasks.workunit.client.0.smithi016.stdout:3/821: dread d2/d15/d16/f54 [0,4194304] 0 2024-04-02T04:42:47.893 INFO:tasks.workunit.client.0.smithi016.stdout:3/822: fsync d2/d15/d1a/fc3 0 2024-04-02T04:42:47.893 INFO:tasks.workunit.client.0.smithi016.stdout:7/704: mkdir d0/d46/d16/df6 0 2024-04-02T04:42:47.893 INFO:tasks.workunit.client.0.smithi016.stdout:7/705: chown d0/d46/d37 146079705 1 2024-04-02T04:42:47.894 INFO:tasks.workunit.client.0.smithi016.stdout:9/788: link d0/d9/db/dc9/f45 d0/d4a/d99/dd3/ff5 0 2024-04-02T04:42:47.894 INFO:tasks.workunit.client.0.smithi016.stdout:7/706: chown d0/d46/d16/d49/c79 17 1 2024-04-02T04:42:47.895 INFO:tasks.workunit.client.0.smithi016.stdout:3/823: dread d2/d15/fb4 [0,4194304] 0 2024-04-02T04:42:47.897 INFO:tasks.workunit.client.0.smithi016.stdout:4/731: mknod d5/d17/d18/d25/cdc 0 2024-04-02T04:42:47.903 INFO:tasks.workunit.client.1.smithi027.stdout:8/169: dwrite d5/d8/db/dd/f10 [0,4194304] 0 2024-04-02T04:42:47.904 INFO:tasks.workunit.client.1.smithi027.stdout:8/170: readlink d5/d8/db/dd/l2f 0 2024-04-02T04:42:47.905 INFO:tasks.workunit.client.1.smithi027.stdout:8/171: stat d5/d7/f14 0 2024-04-02T04:42:47.906 INFO:tasks.workunit.client.1.smithi027.stdout:8/172: stat d5/d8/d15 0 2024-04-02T04:42:47.907 INFO:tasks.workunit.client.0.smithi016.stdout:5/697: getdents d3/d40/d4f 0 2024-04-02T04:42:47.907 INFO:tasks.workunit.client.1.smithi027.stdout:8/173: read d5/d8/f33 [543000,33886] 0 2024-04-02T04:42:47.907 INFO:tasks.workunit.client.1.smithi027.stdout:3/152: symlink d1/l31 0 2024-04-02T04:42:47.908 INFO:tasks.workunit.client.1.smithi027.stdout:3/153: chown d1/de/df/f19 24705691 1 2024-04-02T04:42:47.911 INFO:tasks.workunit.client.1.smithi027.stdout:6/210: creat d13/d1b/d24/f4a x:0 0 0 2024-04-02T04:42:47.912 INFO:tasks.workunit.client.1.smithi027.stdout:8/174: dread d5/d7/f14 [4194304,4194304] 0 2024-04-02T04:42:47.914 INFO:tasks.workunit.client.1.smithi027.stdout:8/175: chown d5/d7/d27/d31/f32 120869 1 2024-04-02T04:42:47.916 INFO:tasks.workunit.client.1.smithi027.stdout:0/233: write d4/f9 [399198,103142] 0 2024-04-02T04:42:47.918 INFO:tasks.workunit.client.0.smithi016.stdout:1/698: mkdir d14/d19/d1d/d45/d4f/d86/db4/de3 0 2024-04-02T04:42:47.922 INFO:tasks.workunit.client.1.smithi027.stdout:8/176: dwrite d5/d8/db/dd/f21 [0,4194304] 0 2024-04-02T04:42:47.922 INFO:tasks.workunit.client.1.smithi027.stdout:9/193: dread d1a/d1d/d29/d37/f1c [4194304,4194304] 0 2024-04-02T04:42:47.924 INFO:tasks.workunit.client.1.smithi027.stdout:9/194: dread - d1a/d1d/d29/d37/f35 zero size 2024-04-02T04:42:47.925 INFO:tasks.workunit.client.0.smithi016.stdout:9/789: dread - d0/d9/db/d54/fdc zero size 2024-04-02T04:42:47.926 INFO:tasks.workunit.client.0.smithi016.stdout:0/691: dwrite d0/d16/f31 [4194304,4194304] 0 2024-04-02T04:42:47.926 INFO:tasks.workunit.client.1.smithi027.stdout:2/189: write d1/db/f1d [942832,39073] 0 2024-04-02T04:42:47.927 INFO:tasks.workunit.client.0.smithi016.stdout:9/790: write d0/d9/d4d/da8/fb8 [757098,2095] 0 2024-04-02T04:42:47.929 INFO:tasks.workunit.client.1.smithi027.stdout:9/195: dread f12 [0,4194304] 0 2024-04-02T04:42:47.932 INFO:tasks.workunit.client.1.smithi027.stdout:1/170: truncate d0/d1e/d25/f39 1388782 0 2024-04-02T04:42:47.934 INFO:tasks.workunit.client.0.smithi016.stdout:8/706: creat d1/d70/d71/d9e/fef x:0 0 0 2024-04-02T04:42:47.935 INFO:tasks.workunit.client.0.smithi016.stdout:8/707: fdatasync d1/d6/d36/d3f/fb4 0 2024-04-02T04:42:47.936 INFO:tasks.workunit.client.1.smithi027.stdout:0/234: mkdir d4/d21/d28/d4e 0 2024-04-02T04:42:47.937 INFO:tasks.workunit.client.0.smithi016.stdout:8/708: truncate d1/d6/d36/d3f/fb4 746755 0 2024-04-02T04:42:47.938 INFO:tasks.workunit.client.1.smithi027.stdout:8/177: rmdir d5/d7/d27 39 2024-04-02T04:42:47.945 INFO:tasks.workunit.client.1.smithi027.stdout:6/211: symlink d13/d1b/d24/d28/d46/l4b 0 2024-04-02T04:42:47.945 INFO:tasks.workunit.client.1.smithi027.stdout:6/212: chown d13/d1b/f23 425 1 2024-04-02T04:42:47.946 INFO:tasks.workunit.client.0.smithi016.stdout:1/699: rmdir d14/d19/d1e/d71 39 2024-04-02T04:42:47.946 INFO:tasks.workunit.client.1.smithi027.stdout:1/171: mkdir d0/d1e/d25/d3c 0 2024-04-02T04:42:47.947 INFO:tasks.workunit.client.0.smithi016.stdout:7/707: symlink d0/lf7 0 2024-04-02T04:42:47.950 INFO:tasks.workunit.client.1.smithi027.stdout:0/235: fdatasync d4/d16/d2b/f18 0 2024-04-02T04:42:47.953 INFO:tasks.workunit.client.1.smithi027.stdout:8/178: creat d5/d8/db/f35 x:0 0 0 2024-04-02T04:42:47.954 INFO:tasks.workunit.client.1.smithi027.stdout:1/172: dwrite d0/d1e/d25/f31 [0,4194304] 0 2024-04-02T04:42:47.955 INFO:tasks.workunit.client.0.smithi016.stdout:9/791: readlink d0/d9/db/l68 0 2024-04-02T04:42:47.956 INFO:tasks.workunit.client.1.smithi027.stdout:3/154: creat d1/f32 x:0 0 0 2024-04-02T04:42:47.958 INFO:tasks.workunit.client.0.smithi016.stdout:3/824: symlink d2/d15/d10b/l113 0 2024-04-02T04:42:47.960 INFO:tasks.workunit.client.1.smithi027.stdout:2/190: symlink d1/db/d3b/d17/d2b/d2f/l3d 0 2024-04-02T04:42:47.961 INFO:tasks.workunit.client.0.smithi016.stdout:6/776: dwrite d0/d9/fc2 [0,4194304] 0 2024-04-02T04:42:47.961 INFO:tasks.workunit.client.1.smithi027.stdout:2/191: fsync d1/db/f32 0 2024-04-02T04:42:47.962 INFO:tasks.workunit.client.0.smithi016.stdout:6/777: fsync d0/d9/d16/d56/fe6 0 2024-04-02T04:42:47.964 INFO:tasks.workunit.client.0.smithi016.stdout:8/709: rmdir d1/d6/d36/d4f/d5d 39 2024-04-02T04:42:47.966 INFO:tasks.workunit.client.1.smithi027.stdout:3/155: dwrite d1/f32 [0,4194304] 0 2024-04-02T04:42:47.968 INFO:tasks.workunit.client.1.smithi027.stdout:3/156: read - d1/f2e zero size 2024-04-02T04:42:47.968 INFO:tasks.workunit.client.1.smithi027.stdout:9/196: rename l7 to d1a/d1d/d3c/l44 0 2024-04-02T04:42:47.970 INFO:tasks.workunit.client.1.smithi027.stdout:9/197: fdatasync d1a/f38 0 2024-04-02T04:42:47.971 INFO:tasks.workunit.client.1.smithi027.stdout:9/198: dread - d1a/f39 zero size 2024-04-02T04:42:47.979 INFO:tasks.workunit.client.0.smithi016.stdout:7/708: chown d0/d46/l63 10 1 2024-04-02T04:42:47.980 INFO:tasks.workunit.client.1.smithi027.stdout:6/213: write d13/d1b/d24/d25/d27/f44 [710746,17363] 0 2024-04-02T04:42:47.980 INFO:tasks.workunit.client.0.smithi016.stdout:0/692: getdents d0/d16/d24/d4e/de6 0 2024-04-02T04:42:47.981 INFO:tasks.workunit.client.0.smithi016.stdout:7/709: write d0/d46/d37/dbe/dae/d9d/fa3 [6336,38041] 0 2024-04-02T04:42:47.982 INFO:tasks.workunit.client.0.smithi016.stdout:9/792: rename d0/d9/db/dc9/d53/f7a to d0/d4e/d50/d67/ff6 0 2024-04-02T04:42:47.983 INFO:tasks.workunit.client.1.smithi027.stdout:2/192: mknod d1/db/d3b/d17/d2b/c3e 0 2024-04-02T04:42:47.983 INFO:tasks.workunit.client.0.smithi016.stdout:9/793: chown d0/d9/f2e 28780939 1 2024-04-02T04:42:47.984 INFO:tasks.workunit.client.1.smithi027.stdout:6/214: dread d13/d1b/d24/f41 [0,4194304] 0 2024-04-02T04:42:47.985 INFO:tasks.workunit.client.1.smithi027.stdout:2/193: dread - d1/db/d3b/d17/d2b/f26 zero size 2024-04-02T04:42:47.986 INFO:tasks.workunit.client.1.smithi027.stdout:8/179: rename d5/d8/c25 to d5/d8/db/dd/c36 0 2024-04-02T04:42:47.987 INFO:tasks.workunit.client.1.smithi027.stdout:5/271: sync 2024-04-02T04:42:47.989 INFO:tasks.workunit.client.0.smithi016.stdout:8/710: creat d1/d31/ff0 x:0 0 0 2024-04-02T04:42:47.989 INFO:tasks.workunit.client.1.smithi027.stdout:9/199: symlink d1a/d1d/l45 0 2024-04-02T04:42:47.990 INFO:tasks.workunit.client.1.smithi027.stdout:9/200: stat d1a/d1d/d3c 0 2024-04-02T04:42:47.991 INFO:tasks.workunit.client.0.smithi016.stdout:4/732: symlink d5/d17/d18/d6d/da4/db7/ldd 0 2024-04-02T04:42:47.991 INFO:tasks.workunit.client.1.smithi027.stdout:9/201: fsync d1a/f26 0 2024-04-02T04:42:47.992 INFO:tasks.workunit.client.0.smithi016.stdout:2/773: link d12/d35/d101/d76/da0/dbe/l10c d12/d35/d101/d76/dcf/l10e 0 2024-04-02T04:42:47.992 INFO:tasks.workunit.client.1.smithi027.stdout:6/215: dwrite f7 [0,4194304] 0 2024-04-02T04:42:47.992 INFO:tasks.workunit.client.1.smithi027.stdout:9/202: dread - d1a/f38 zero size 2024-04-02T04:42:47.997 INFO:tasks.workunit.client.1.smithi027.stdout:6/216: dread d13/d1b/f2b [0,4194304] 0 2024-04-02T04:42:47.997 INFO:tasks.workunit.client.0.smithi016.stdout:1/700: getdents d14/d19/d7a/da2 0 2024-04-02T04:42:47.998 INFO:tasks.workunit.client.1.smithi027.stdout:2/194: chown d1/db/d3b/d17/c34 8230949 1 2024-04-02T04:42:48.000 INFO:tasks.workunit.client.0.smithi016.stdout:7/710: rmdir d0/d46/d37/dbe 39 2024-04-02T04:42:48.002 INFO:tasks.workunit.client.1.smithi027.stdout:0/236: write d4/f23 [545145,38203] 0 2024-04-02T04:42:48.002 INFO:tasks.workunit.client.1.smithi027.stdout:7/204: sync 2024-04-02T04:42:48.002 INFO:tasks.workunit.client.1.smithi027.stdout:4/195: sync 2024-04-02T04:42:48.004 INFO:tasks.workunit.client.1.smithi027.stdout:3/157: rename d1/de/df/c11 to d1/de/c33 0 2024-04-02T04:42:48.005 INFO:tasks.workunit.client.1.smithi027.stdout:7/205: dread f1 [0,4194304] 0 2024-04-02T04:42:48.005 INFO:tasks.workunit.client.0.smithi016.stdout:5/698: write d3/d40/d4f/f75 [987687,109023] 0 2024-04-02T04:42:48.006 INFO:tasks.workunit.client.0.smithi016.stdout:5/699: stat d3/d40/f79 0 2024-04-02T04:42:48.009 INFO:tasks.workunit.client.1.smithi027.stdout:8/180: creat d5/d8/d15/f37 x:0 0 0 2024-04-02T04:42:48.009 INFO:tasks.workunit.client.1.smithi027.stdout:3/158: dread d1/f18 [0,4194304] 0 2024-04-02T04:42:48.010 INFO:tasks.workunit.client.0.smithi016.stdout:3/825: link d2/d2e/d36/d48/fe1 d2/d2e/d36/d66/d7a/f114 0 2024-04-02T04:42:48.013 INFO:tasks.workunit.client.0.smithi016.stdout:6/778: symlink d0/d9/d16/d56/d89/dc5/dce/lf1 0 2024-04-02T04:42:48.016 INFO:tasks.workunit.client.1.smithi027.stdout:1/173: link d0/f37 d0/f3d 0 2024-04-02T04:42:48.018 INFO:tasks.workunit.client.1.smithi027.stdout:9/203: creat d1a/d1d/d29/f46 x:0 0 0 2024-04-02T04:42:48.020 INFO:tasks.workunit.client.0.smithi016.stdout:2/774: creat d12/d35/d101/d76/da0/de7/f10f x:0 0 0 2024-04-02T04:42:48.022 INFO:tasks.workunit.client.0.smithi016.stdout:2/775: write d12/d70/ff3 [4120,88562] 0 2024-04-02T04:42:48.024 INFO:tasks.workunit.client.0.smithi016.stdout:1/701: unlink d14/d19/f69 0 2024-04-02T04:42:48.024 INFO:tasks.workunit.client.1.smithi027.stdout:8/181: dread d5/d8/db/dd/f1d [0,4194304] 0 2024-04-02T04:42:48.025 INFO:tasks.workunit.client.1.smithi027.stdout:6/217: symlink d13/d1b/d24/d28/d46/l4c 0 2024-04-02T04:42:48.025 INFO:tasks.workunit.client.1.smithi027.stdout:8/182: write d5/d8/db/dd/f1f [650001,70189] 0 2024-04-02T04:42:48.028 INFO:tasks.workunit.client.1.smithi027.stdout:2/195: rmdir d1 39 2024-04-02T04:42:48.031 INFO:tasks.workunit.client.0.smithi016.stdout:9/794: dwrite d0/d3/f40 [0,4194304] 0 2024-04-02T04:42:48.032 INFO:tasks.workunit.client.0.smithi016.stdout:0/693: truncate d0/d4f/f9e 323886 0 2024-04-02T04:42:48.033 INFO:tasks.workunit.client.1.smithi027.stdout:0/237: unlink d4/d16/d2b/f2c 0 2024-04-02T04:42:48.034 INFO:tasks.workunit.client.0.smithi016.stdout:5/700: dread - d3/df/f65 zero size 2024-04-02T04:42:48.035 INFO:tasks.workunit.client.0.smithi016.stdout:7/711: dwrite d0/d46/d52/f80 [0,4194304] 0 2024-04-02T04:42:48.036 INFO:tasks.workunit.client.1.smithi027.stdout:7/206: unlink d2/d1d/f20 0 2024-04-02T04:42:48.037 INFO:tasks.workunit.client.0.smithi016.stdout:7/712: read - d0/d46/d52/d82/fb3 zero size 2024-04-02T04:42:48.038 INFO:tasks.workunit.client.0.smithi016.stdout:4/733: write d5/de/f8d [808408,46909] 0 2024-04-02T04:42:48.042 INFO:tasks.workunit.client.1.smithi027.stdout:1/174: mkdir d0/d1e/d2e/d3e 0 2024-04-02T04:42:48.043 INFO:tasks.workunit.client.0.smithi016.stdout:7/713: read d0/d46/d16/d83/f93 [1218589,80055] 0 2024-04-02T04:42:48.045 INFO:tasks.workunit.client.0.smithi016.stdout:6/779: readlink d0/d9/d16/d56/d89/d67/d6d/le1 0 2024-04-02T04:42:48.046 INFO:tasks.workunit.client.0.smithi016.stdout:8/711: mknod d1/d6/d36/d4f/ded/cf1 0 2024-04-02T04:42:48.047 INFO:tasks.workunit.client.1.smithi027.stdout:6/218: mknod d13/d1b/d24/d28/c4d 0 2024-04-02T04:42:48.047 INFO:tasks.workunit.client.1.smithi027.stdout:6/219: readlink d13/l15 0 2024-04-02T04:42:48.048 INFO:tasks.workunit.client.1.smithi027.stdout:8/183: symlink d5/d8/d15/l38 0 2024-04-02T04:42:48.049 INFO:tasks.workunit.client.0.smithi016.stdout:0/694: dwrite d0/d16/f31 [0,4194304] 0 2024-04-02T04:42:48.051 INFO:tasks.workunit.client.1.smithi027.stdout:5/272: creat d0/f69 x:0 0 0 2024-04-02T04:42:48.051 INFO:tasks.workunit.client.1.smithi027.stdout:2/196: write d1/db/d3b/d17/fc [9477610,25493] 0 2024-04-02T04:42:48.052 INFO:tasks.workunit.client.1.smithi027.stdout:0/238: creat d4/d16/d2b/f4f x:0 0 0 2024-04-02T04:42:48.055 INFO:tasks.workunit.client.1.smithi027.stdout:7/207: truncate d2/d7/da/dc/d16/f2c 144108 0 2024-04-02T04:42:48.057 INFO:tasks.workunit.client.1.smithi027.stdout:7/208: truncate d2/d1c/f22 425160 0 2024-04-02T04:42:48.057 INFO:tasks.workunit.client.0.smithi016.stdout:9/795: creat d0/d3/d97/ff7 x:0 0 0 2024-04-02T04:42:48.058 INFO:tasks.workunit.client.1.smithi027.stdout:6/220: dread d13/d1b/d24/d25/d27/f44 [0,4194304] 0 2024-04-02T04:42:48.059 INFO:tasks.workunit.client.0.smithi016.stdout:9/796: chown d0/d9/d4d/da8 339584 1 2024-04-02T04:42:48.059 INFO:tasks.workunit.client.0.smithi016.stdout:5/701: mknod d3/df/d86/cff 0 2024-04-02T04:42:48.061 INFO:tasks.workunit.client.0.smithi016.stdout:5/702: fsync d3/df/d18/d26/d70/ffe 0 2024-04-02T04:42:48.061 INFO:tasks.workunit.client.0.smithi016.stdout:4/734: rmdir d5/d1d/d6c 39 2024-04-02T04:42:48.063 INFO:tasks.workunit.client.1.smithi027.stdout:6/221: dread f7 [0,4194304] 0 2024-04-02T04:42:48.065 INFO:tasks.workunit.client.0.smithi016.stdout:4/735: chown d5/d17/d18/d6d/da4/fbd 3218657 1 2024-04-02T04:42:48.066 INFO:tasks.workunit.client.1.smithi027.stdout:5/273: rename d0/df/d11/c43 to d0/d1d/d61/c6a 0 2024-04-02T04:42:48.067 INFO:tasks.workunit.client.0.smithi016.stdout:3/826: rename d2/d15/d16/d8a/dd1/fbc to d2/d15/d60/f115 0 2024-04-02T04:42:48.067 INFO:tasks.workunit.client.1.smithi027.stdout:5/274: fsync d0/df/d11/d13/f26 0 2024-04-02T04:42:48.069 INFO:tasks.workunit.client.0.smithi016.stdout:5/703: dwrite d3/df/d86/fe6 [0,4194304] 0 2024-04-02T04:42:48.069 INFO:tasks.workunit.client.1.smithi027.stdout:2/197: truncate d1/db/d3b/d17/d12/f1f 1989311 0 2024-04-02T04:42:48.069 INFO:tasks.workunit.client.1.smithi027.stdout:5/275: fsync d0/df/d2f/d3b/f3d 0 2024-04-02T04:42:48.070 INFO:tasks.workunit.client.0.smithi016.stdout:3/827: chown d2/d15/d16/d57/ff3 262 1 2024-04-02T04:42:48.070 INFO:tasks.workunit.client.1.smithi027.stdout:5/276: chown d0/df/d2f/f38 127094239 1 2024-04-02T04:42:48.071 INFO:tasks.workunit.client.1.smithi027.stdout:5/277: read - d0/df/d11/d37/f40 zero size 2024-04-02T04:42:48.073 INFO:tasks.workunit.client.1.smithi027.stdout:5/278: write d0/d5/f63 [1001853,3300] 0 2024-04-02T04:42:48.075 INFO:tasks.workunit.client.0.smithi016.stdout:6/780: mkdir d0/d9/d16/d56/db0/df2 0 2024-04-02T04:42:48.077 INFO:tasks.workunit.client.1.smithi027.stdout:7/209: creat d2/d7/da/dc/d16/d2f/f41 x:0 0 0 2024-04-02T04:42:48.078 INFO:tasks.workunit.client.1.smithi027.stdout:9/204: link d1a/d1d/d29/d37/d20/c34 d1a/d1d/c47 0 2024-04-02T04:42:48.082 INFO:tasks.workunit.client.0.smithi016.stdout:1/702: creat d14/d19/d7a/da2/fe4 x:0 0 0 2024-04-02T04:42:48.085 INFO:tasks.workunit.client.1.smithi027.stdout:0/239: creat d4/d3d/d49/f50 x:0 0 0 2024-04-02T04:42:48.087 INFO:tasks.workunit.client.1.smithi027.stdout:6/222: mknod d13/d1b/d24/d25/d42/c4e 0 2024-04-02T04:42:48.088 INFO:tasks.workunit.client.1.smithi027.stdout:0/240: dread d4/d16/f1c [0,4194304] 0 2024-04-02T04:42:48.090 INFO:tasks.workunit.client.0.smithi016.stdout:5/704: dread d3/df/d18/d26/ddc/f5f [0,4194304] 0 2024-04-02T04:42:48.091 INFO:tasks.workunit.client.0.smithi016.stdout:5/705: chown d3/d40/d4f/d6b/d45 1 1 2024-04-02T04:42:48.093 INFO:tasks.workunit.client.1.smithi027.stdout:2/198: fdatasync d1/db/d3b/d17/f14 0 2024-04-02T04:42:48.093 INFO:tasks.workunit.client.1.smithi027.stdout:6/223: dread d13/d1b/f1e [0,4194304] 0 2024-04-02T04:42:48.094 INFO:tasks.workunit.client.1.smithi027.stdout:2/199: chown d1/l9 462 1 2024-04-02T04:42:48.094 INFO:tasks.workunit.client.0.smithi016.stdout:4/736: chown d5/d17/d18/f7f 114 1 2024-04-02T04:42:48.096 INFO:tasks.workunit.client.1.smithi027.stdout:5/279: chown d0/df/d2f/d3b/d51/l52 10255 1 2024-04-02T04:42:48.101 INFO:tasks.workunit.client.0.smithi016.stdout:0/695: creat d0/d48/fec x:0 0 0 2024-04-02T04:42:48.101 INFO:tasks.workunit.client.1.smithi027.stdout:1/175: write d0/d1e/d25/f39 [1986040,82297] 0 2024-04-02T04:42:48.102 INFO:tasks.workunit.client.0.smithi016.stdout:0/696: chown d0/d16/d2e/d53/d6f/f88 660312486 1 2024-04-02T04:42:48.105 INFO:tasks.workunit.client.0.smithi016.stdout:8/712: creat d1/d6/d36/d3f/d40/de9/ff2 x:0 0 0 2024-04-02T04:42:48.108 INFO:tasks.workunit.client.0.smithi016.stdout:1/703: fsync d14/d19/f60 0 2024-04-02T04:42:48.108 INFO:tasks.workunit.client.1.smithi027.stdout:0/241: rmdir d4/d16/d2b/d4a 39 2024-04-02T04:42:48.110 INFO:tasks.workunit.client.1.smithi027.stdout:0/242: chown d4/d16/d1b/l43 36208 1 2024-04-02T04:42:48.111 INFO:tasks.workunit.client.0.smithi016.stdout:5/706: rmdir d3/df/d18/d26 39 2024-04-02T04:42:48.113 INFO:tasks.workunit.client.0.smithi016.stdout:4/737: creat d5/d16/d88/fde x:0 0 0 2024-04-02T04:42:48.116 INFO:tasks.workunit.client.1.smithi027.stdout:9/205: symlink d1a/l48 0 2024-04-02T04:42:48.117 INFO:tasks.workunit.client.1.smithi027.stdout:9/206: chown d1a/f38 188 1 2024-04-02T04:42:48.120 INFO:tasks.workunit.client.0.smithi016.stdout:6/781: dwrite d0/d9/df/f10 [0,4194304] 0 2024-04-02T04:42:48.123 INFO:tasks.workunit.client.0.smithi016.stdout:2/776: getdents d12/d35/d3b/d42 0 2024-04-02T04:42:48.129 INFO:tasks.workunit.client.1.smithi027.stdout:2/200: truncate d1/db/d3b/d17/f14 7703871 0 2024-04-02T04:42:48.131 INFO:tasks.workunit.client.1.smithi027.stdout:5/280: mkdir d0/d68/d6b 0 2024-04-02T04:42:48.133 INFO:tasks.workunit.client.1.smithi027.stdout:1/176: dwrite d0/f3d [0,4194304] 0 2024-04-02T04:42:48.134 INFO:tasks.workunit.client.0.smithi016.stdout:4/738: write d5/d1d/f61 [602283,106255] 0 2024-04-02T04:42:48.134 INFO:tasks.workunit.client.1.smithi027.stdout:1/177: chown d0/c12 63 1 2024-04-02T04:42:48.136 INFO:tasks.workunit.client.1.smithi027.stdout:1/178: chown d0/d1e/l22 7293 1 2024-04-02T04:42:48.139 INFO:tasks.workunit.client.0.smithi016.stdout:7/714: rename d0/d46/d37/d3a to d0/d46/d37/dbe/dae/df8 0 2024-04-02T04:42:48.141 INFO:tasks.workunit.client.1.smithi027.stdout:5/281: dwrite d0/df/d2f/f67 [0,4194304] 0 2024-04-02T04:42:48.145 INFO:tasks.workunit.client.1.smithi027.stdout:9/207: mknod d1a/d1d/d29/d37/d3a/c49 0 2024-04-02T04:42:48.146 INFO:tasks.workunit.client.0.smithi016.stdout:6/782: symlink d0/d9/d16/d56/db0/lf3 0 2024-04-02T04:42:48.148 INFO:tasks.workunit.client.0.smithi016.stdout:2/777: rmdir d12/d35/d101/d76/da0/de7 39 2024-04-02T04:42:48.148 INFO:tasks.workunit.client.1.smithi027.stdout:0/243: getdents d4/d38 0 2024-04-02T04:42:48.151 INFO:tasks.workunit.client.0.smithi016.stdout:1/704: symlink d14/d19/d1e/d71/dcf/le5 0 2024-04-02T04:42:48.153 INFO:tasks.workunit.client.1.smithi027.stdout:9/208: dwrite d1a/d1d/d29/d37/d20/f3f [0,4194304] 0 2024-04-02T04:42:48.153 INFO:tasks.workunit.client.0.smithi016.stdout:9/797: getdents d0/d9/db 0 2024-04-02T04:42:48.154 INFO:tasks.workunit.client.0.smithi016.stdout:9/798: chown d0/d9/db/l15 33857 1 2024-04-02T04:42:48.154 INFO:tasks.workunit.client.0.smithi016.stdout:4/739: dread d5/d17/d18/da0/fa7 [0,4194304] 0 2024-04-02T04:42:48.155 INFO:tasks.workunit.client.0.smithi016.stdout:9/799: chown d0/d3/d97/dde/ff1 228540 1 2024-04-02T04:42:48.157 INFO:tasks.workunit.client.0.smithi016.stdout:3/828: rename d2/d15/d16/f2c to d2/d15/d16/d8a/dcd/f116 0 2024-04-02T04:42:48.161 INFO:tasks.workunit.client.0.smithi016.stdout:7/715: mkdir d0/d46/d16/d49/df9 0 2024-04-02T04:42:48.167 INFO:tasks.workunit.client.0.smithi016.stdout:8/713: creat d1/d6/d36/d3f/d40/ff3 x:0 0 0 2024-04-02T04:42:48.167 INFO:tasks.workunit.client.0.smithi016.stdout:7/716: dwrite d0/d46/d16/dc2/dea/d64/f8b [0,4194304] 0 2024-04-02T04:42:48.173 INFO:tasks.workunit.client.0.smithi016.stdout:2/778: creat d12/d35/d3b/d42/d59/d91/db2/f110 x:0 0 0 2024-04-02T04:42:48.173 INFO:tasks.workunit.client.0.smithi016.stdout:2/779: readlink d12/d35/d3f/l6f 0 2024-04-02T04:42:48.176 INFO:tasks.workunit.client.0.smithi016.stdout:1/705: unlink d14/d19/fa7 0 2024-04-02T04:42:48.177 INFO:tasks.workunit.client.1.smithi027.stdout:7/210: write d2/d7/da/dc/d16/f2c [422239,52909] 0 2024-04-02T04:42:48.177 INFO:tasks.workunit.client.1.smithi027.stdout:1/179: symlink d0/d1e/d2e/d3b/l3f 0 2024-04-02T04:42:48.178 INFO:tasks.workunit.client.1.smithi027.stdout:7/211: fsync d2/f13 0 2024-04-02T04:42:48.180 INFO:tasks.workunit.client.1.smithi027.stdout:9/209: rename le to d1a/d1d/d29/d37/l4a 0 2024-04-02T04:42:48.184 INFO:tasks.workunit.client.1.smithi027.stdout:0/244: dwrite d4/f1d [4194304,4194304] 0 2024-04-02T04:42:48.185 INFO:tasks.workunit.client.1.smithi027.stdout:5/282: creat d0/df/d11/d65/f6c x:0 0 0 2024-04-02T04:42:48.186 INFO:tasks.workunit.client.1.smithi027.stdout:5/283: write d0/f33 [338538,122432] 0 2024-04-02T04:42:48.187 INFO:tasks.workunit.client.0.smithi016.stdout:6/783: dwrite d0/d5c/f76 [0,4194304] 0 2024-04-02T04:42:48.192 INFO:tasks.workunit.client.0.smithi016.stdout:9/800: creat d0/d4a/d75/d95/d9a/ff8 x:0 0 0 2024-04-02T04:42:48.193 INFO:tasks.workunit.client.0.smithi016.stdout:9/801: fdatasync d0/d4e/fae 0 2024-04-02T04:42:48.195 INFO:tasks.workunit.client.0.smithi016.stdout:5/707: rename d3/df/d93/lb5 to d3/df/d18/dfd/l100 0 2024-04-02T04:42:48.197 INFO:tasks.workunit.client.0.smithi016.stdout:3/829: fsync d2/d2e/d36/d48/fe1 0 2024-04-02T04:42:48.198 INFO:tasks.workunit.client.1.smithi027.stdout:2/201: dwrite d1/db/d3b/d17/d12/f1f [0,4194304] 0 2024-04-02T04:42:48.199 INFO:tasks.workunit.client.1.smithi027.stdout:2/202: fsync d1/db/d3b/d17/d12/f38 0 2024-04-02T04:42:48.200 INFO:tasks.workunit.client.1.smithi027.stdout:1/180: symlink d0/d1e/d25/l40 0 2024-04-02T04:42:48.202 INFO:tasks.workunit.client.0.smithi016.stdout:4/740: dwrite d5/d16/f55 [0,4194304] 0 2024-04-02T04:42:48.205 INFO:tasks.workunit.client.1.smithi027.stdout:9/210: creat d1a/d1d/f4b x:0 0 0 2024-04-02T04:42:48.208 INFO:tasks.workunit.client.0.smithi016.stdout:8/714: creat d1/d6/d36/d3f/d40/d45/dcc/ff4 x:0 0 0 2024-04-02T04:42:48.210 INFO:tasks.workunit.client.0.smithi016.stdout:8/715: fsync d1/d6/d36/d3f/fb4 0 2024-04-02T04:42:48.211 INFO:tasks.workunit.client.0.smithi016.stdout:8/716: dread - d1/d70/d71/d9e/fcd zero size 2024-04-02T04:42:48.211 INFO:tasks.workunit.client.1.smithi027.stdout:5/284: unlink d0/d1d/c25 0 2024-04-02T04:42:48.215 INFO:tasks.workunit.client.1.smithi027.stdout:2/203: stat d1/db/d3b/d17/d29 0 2024-04-02T04:42:48.216 INFO:tasks.workunit.client.1.smithi027.stdout:7/212: symlink d2/d7/da/d35/l42 0 2024-04-02T04:42:48.217 INFO:tasks.workunit.client.0.smithi016.stdout:6/784: read - d0/d9/d16/d46/d5e/f8f zero size 2024-04-02T04:42:48.217 INFO:tasks.workunit.client.1.smithi027.stdout:2/204: dread - d1/db/d3b/f37 zero size 2024-04-02T04:42:48.218 INFO:tasks.workunit.client.1.smithi027.stdout:1/181: rename d0/de/l10 to d0/d1e/d25/d3c/l41 0 2024-04-02T04:42:48.220 INFO:tasks.workunit.client.1.smithi027.stdout:9/211: mkdir d1a/d1d/d29/d37/d4c 0 2024-04-02T04:42:48.221 INFO:tasks.workunit.client.1.smithi027.stdout:0/245: rmdir d4/d16/d2b/d4a 39 2024-04-02T04:42:48.222 INFO:tasks.workunit.client.1.smithi027.stdout:5/285: dwrite d0/d1a/d20/f59 [0,4194304] 0 2024-04-02T04:42:48.222 INFO:tasks.workunit.client.1.smithi027.stdout:0/246: dread - d4/d16/d1b/f3e zero size 2024-04-02T04:42:48.226 INFO:tasks.workunit.client.0.smithi016.stdout:5/708: write d3/df/d18/d2f/f62 [1772505,56637] 0 2024-04-02T04:42:48.230 INFO:tasks.workunit.client.1.smithi027.stdout:7/213: unlink d2/d1c/f22 0 2024-04-02T04:42:48.231 INFO:tasks.workunit.client.1.smithi027.stdout:9/212: dwrite d1a/d1d/d29/d37/d20/f3f [0,4194304] 0 2024-04-02T04:42:48.232 INFO:tasks.workunit.client.0.smithi016.stdout:3/830: symlink d2/d15/d16/d8a/dd1/d8b/dd2/l117 0 2024-04-02T04:42:48.232 INFO:tasks.workunit.client.1.smithi027.stdout:9/213: read - d1a/f38 zero size 2024-04-02T04:42:48.233 INFO:tasks.workunit.client.0.smithi016.stdout:3/831: write d2/d2e/d36/d48/da2/de7/f109 [772497,107890] 0 2024-04-02T04:42:48.235 INFO:tasks.workunit.client.0.smithi016.stdout:3/832: chown d2/d15/d10b 98938 1 2024-04-02T04:42:48.235 INFO:tasks.workunit.client.0.smithi016.stdout:0/697: link d0/d16/d2e/d53/c33 d0/d16/d2e/d53/d9d/ced 0 2024-04-02T04:42:48.235 INFO:tasks.workunit.client.0.smithi016.stdout:3/833: chown d2/d2e/d36/d66/d7a/cb2 0 1 2024-04-02T04:42:48.236 INFO:tasks.workunit.client.1.smithi027.stdout:0/247: dwrite d4/d21/f2e [4194304,4194304] 0 2024-04-02T04:42:48.241 INFO:tasks.workunit.client.0.smithi016.stdout:3/834: chown d2/d2e/d36/d66/d90/faf 124452218 1 2024-04-02T04:42:48.242 INFO:tasks.workunit.client.1.smithi027.stdout:5/286: mkdir d0/d1d/d61/d6d 0 2024-04-02T04:42:48.245 INFO:tasks.workunit.client.0.smithi016.stdout:0/698: dwrite d0/d16/d2e/d53/d6f/fea [0,4194304] 0 2024-04-02T04:42:48.246 INFO:tasks.workunit.client.0.smithi016.stdout:0/699: chown d0/d16/d23/c4a 2168 1 2024-04-02T04:42:48.247 INFO:tasks.workunit.client.0.smithi016.stdout:2/780: mkdir d12/d35/d3b/d111 0 2024-04-02T04:42:48.248 INFO:tasks.workunit.client.0.smithi016.stdout:0/700: fsync d0/d16/d2e/d37/f4b 0 2024-04-02T04:42:48.249 INFO:tasks.workunit.client.0.smithi016.stdout:2/781: readlink d12/d35/d3f/d56/la8 0 2024-04-02T04:42:48.249 INFO:tasks.workunit.client.0.smithi016.stdout:8/717: creat d1/d70/ff5 x:0 0 0 2024-04-02T04:42:48.249 INFO:tasks.workunit.client.0.smithi016.stdout:2/782: readlink d12/d35/d101/d76/da0/lf1 0 2024-04-02T04:42:48.250 INFO:tasks.workunit.client.1.smithi027.stdout:5/287: dwrite d0/df/d2f/d3b/f64 [0,4194304] 0 2024-04-02T04:42:48.251 INFO:tasks.workunit.client.0.smithi016.stdout:2/783: truncate d12/d35/d3f/d4d/f97 5013502 0 2024-04-02T04:42:48.254 INFO:tasks.workunit.client.1.smithi027.stdout:2/205: unlink d1/db/d3b/d17/l8 0 2024-04-02T04:42:48.255 INFO:tasks.workunit.client.0.smithi016.stdout:6/785: chown d0/d9/c4a 2 1 2024-04-02T04:42:48.255 INFO:tasks.workunit.client.1.smithi027.stdout:2/206: fsync d1/db/f32 0 2024-04-02T04:42:48.257 INFO:tasks.workunit.client.1.smithi027.stdout:9/214: mknod d1a/d1d/d29/d37/c4d 0 2024-04-02T04:42:48.258 INFO:tasks.workunit.client.1.smithi027.stdout:9/215: write d1a/d1d/d29/f46 [1036900,22657] 0 2024-04-02T04:42:48.260 INFO:tasks.workunit.client.1.smithi027.stdout:1/182: creat d0/d1e/d2e/d3e/f42 x:0 0 0 2024-04-02T04:42:48.268 INFO:tasks.workunit.client.0.smithi016.stdout:6/786: dread d0/d6e/f81 [0,4194304] 0 2024-04-02T04:42:48.268 INFO:tasks.workunit.client.1.smithi027.stdout:5/288: mknod d0/d5/c6e 0 2024-04-02T04:42:48.270 INFO:tasks.workunit.client.0.smithi016.stdout:7/717: link d0/d46/d37/dbe/dae/df8/fb6 d0/d46/da2/dc8/dda/ffa 0 2024-04-02T04:42:48.272 INFO:tasks.workunit.client.1.smithi027.stdout:6/224: sync 2024-04-02T04:42:48.272 INFO:tasks.workunit.client.1.smithi027.stdout:4/196: sync 2024-04-02T04:42:48.272 INFO:tasks.workunit.client.1.smithi027.stdout:8/184: sync 2024-04-02T04:42:48.272 INFO:tasks.workunit.client.1.smithi027.stdout:3/159: sync 2024-04-02T04:42:48.274 INFO:tasks.workunit.client.1.smithi027.stdout:4/197: write d24/f35 [1037195,124840] 0 2024-04-02T04:42:48.274 INFO:tasks.workunit.client.1.smithi027.stdout:1/183: readlink d0/d1e/d25/d3c/l41 0 2024-04-02T04:42:48.276 INFO:tasks.workunit.client.1.smithi027.stdout:8/185: stat d5/d8/d15/l1c 0 2024-04-02T04:42:48.276 INFO:tasks.workunit.client.0.smithi016.stdout:3/835: mknod d2/d15/d1a/dd5/c118 0 2024-04-02T04:42:48.281 INFO:tasks.workunit.client.1.smithi027.stdout:5/289: unlink d0/df/d2f/f38 0 2024-04-02T04:42:48.281 INFO:tasks.workunit.client.1.smithi027.stdout:2/207: dwrite d1/db/f30 [0,4194304] 0 2024-04-02T04:42:48.283 INFO:tasks.workunit.client.1.smithi027.stdout:8/186: write d5/d8/db/dd/f10 [4824219,104144] 0 2024-04-02T04:42:48.286 INFO:tasks.workunit.client.1.smithi027.stdout:4/198: dread d24/f35 [0,4194304] 0 2024-04-02T04:42:48.286 INFO:tasks.workunit.client.1.smithi027.stdout:5/290: fsync d0/df/d11/d13/f1e 0 2024-04-02T04:42:48.288 INFO:tasks.workunit.client.0.smithi016.stdout:1/706: creat d14/d19/fe6 x:0 0 0 2024-04-02T04:42:48.288 INFO:tasks.workunit.client.1.smithi027.stdout:9/216: symlink d1a/l4e 0 2024-04-02T04:42:48.288 INFO:tasks.workunit.client.1.smithi027.stdout:3/160: dwrite d1/d1a/f22 [0,4194304] 0 2024-04-02T04:42:48.289 INFO:tasks.workunit.client.1.smithi027.stdout:8/187: read - d5/d8/db/f29 zero size 2024-04-02T04:42:48.290 INFO:tasks.workunit.client.0.smithi016.stdout:2/784: creat d12/d35/d3b/d42/f112 x:0 0 0 2024-04-02T04:42:48.293 INFO:tasks.workunit.client.0.smithi016.stdout:2/785: dread d12/d9f/fc9 [0,4194304] 0 2024-04-02T04:42:48.293 INFO:tasks.workunit.client.1.smithi027.stdout:0/248: link d4/d3d/l4b d4/d16/d2b/d2d/d4c/l51 0 2024-04-02T04:42:48.294 INFO:tasks.workunit.client.1.smithi027.stdout:7/214: write f1 [706844,78756] 0 2024-04-02T04:42:48.294 INFO:tasks.workunit.client.1.smithi027.stdout:2/208: truncate d1/db/d3b/f1c 460100 0 2024-04-02T04:42:48.296 INFO:tasks.workunit.client.0.smithi016.stdout:5/709: symlink d3/d40/d4f/d6b/l101 0 2024-04-02T04:42:48.297 INFO:tasks.workunit.client.1.smithi027.stdout:0/249: stat d4/d3d/f3f 0 2024-04-02T04:42:48.308 INFO:tasks.workunit.client.1.smithi027.stdout:4/199: dwrite f12 [0,4194304] 0 2024-04-02T04:42:48.309 INFO:tasks.workunit.client.1.smithi027.stdout:5/291: mkdir d0/d1a/d6f 0 2024-04-02T04:42:48.310 INFO:tasks.workunit.client.1.smithi027.stdout:9/217: dwrite d1a/d1d/d29/d37/f35 [0,4194304] 0 2024-04-02T04:42:48.312 INFO:tasks.workunit.client.1.smithi027.stdout:8/188: dwrite d5/d8/db/df/f22 [0,4194304] 0 2024-04-02T04:42:48.314 INFO:tasks.workunit.client.1.smithi027.stdout:6/225: dread fc [0,4194304] 0 2024-04-02T04:42:48.316 INFO:tasks.workunit.client.1.smithi027.stdout:6/226: write d13/d1b/d24/d28/f2a [26333,35055] 0 2024-04-02T04:42:48.349 INFO:tasks.workunit.client.1.smithi027.stdout:7/215: rmdir d2/d7/da/d35 39 2024-04-02T04:42:48.361 INFO:tasks.workunit.client.1.smithi027.stdout:7/216: dread d2/f13 [0,4194304] 0 2024-04-02T04:42:48.363 INFO:tasks.workunit.client.1.smithi027.stdout:7/217: dread d2/d1d/f25 [0,4194304] 0 2024-04-02T04:42:48.365 INFO:tasks.workunit.client.1.smithi027.stdout:7/218: truncate d2/d7/da/dc/d16/d2f/f41 1030753 0 2024-04-02T04:42:48.365 INFO:tasks.workunit.client.1.smithi027.stdout:7/219: fsync d2/f13 0 2024-04-02T04:42:48.367 INFO:tasks.workunit.client.1.smithi027.stdout:7/220: write d2/f13 [2134232,113960] 0 2024-04-02T04:42:48.367 INFO:tasks.workunit.client.1.smithi027.stdout:7/221: chown d2/d7/da/dc/f30 245 1 2024-04-02T04:42:48.368 INFO:tasks.workunit.client.1.smithi027.stdout:7/222: dread - d2/d1c/f40 zero size 2024-04-02T04:42:48.369 INFO:tasks.workunit.client.1.smithi027.stdout:7/223: stat d2/d1c/f37 0 2024-04-02T04:42:48.369 INFO:tasks.workunit.client.1.smithi027.stdout:7/224: stat d2/d1c/f40 0 2024-04-02T04:42:48.371 INFO:tasks.workunit.client.1.smithi027.stdout:7/225: write d2/d7/da/f1e [369885,70997] 0 2024-04-02T04:42:48.372 INFO:tasks.workunit.client.1.smithi027.stdout:7/226: write d2/d1c/f3b [1070566,21192] 0 2024-04-02T04:42:48.383 INFO:tasks.workunit.client.1.smithi027.stdout:3/161: truncate d1/de/df/f14 84947 0 2024-04-02T04:42:48.388 INFO:tasks.workunit.client.1.smithi027.stdout:0/250: write d4/d16/d2b/d2d/f3c [823755,28913] 0 2024-04-02T04:42:48.389 INFO:tasks.workunit.client.1.smithi027.stdout:3/162: dwrite d1/f8 [0,4194304] 0 2024-04-02T04:42:48.391 INFO:tasks.workunit.client.1.smithi027.stdout:3/163: fsync d1/f32 0 2024-04-02T04:42:48.392 INFO:tasks.workunit.client.0.smithi016.stdout:7/718: mknod d0/d46/d52/daf/cfb 0 2024-04-02T04:42:48.398 INFO:tasks.workunit.client.1.smithi027.stdout:4/200: unlink f9 0 2024-04-02T04:42:48.402 INFO:tasks.workunit.client.0.smithi016.stdout:9/802: truncate d0/d9/fcf 2633068 0 2024-04-02T04:42:48.403 INFO:tasks.workunit.client.1.smithi027.stdout:9/218: creat d1a/d1d/d29/f4f x:0 0 0 2024-04-02T04:42:48.404 INFO:tasks.workunit.client.0.smithi016.stdout:4/741: truncate d5/d17/d18/d1f/f98 347438 0 2024-04-02T04:42:48.404 INFO:tasks.workunit.client.1.smithi027.stdout:9/219: chown d1a/d1d/d29/d37/d20 3418357 1 2024-04-02T04:42:48.404 INFO:tasks.workunit.client.0.smithi016.stdout:3/836: creat d2/d2e/d36/d48/da2/de7/f119 x:0 0 0 2024-04-02T04:42:48.405 INFO:tasks.workunit.client.1.smithi027.stdout:5/292: symlink d0/df/d11/d41/l70 0 2024-04-02T04:42:48.407 INFO:tasks.workunit.client.0.smithi016.stdout:2/786: mknod d12/d9f/d107/c113 0 2024-04-02T04:42:48.408 INFO:tasks.workunit.client.0.smithi016.stdout:3/837: readlink d2/le 0 2024-04-02T04:42:48.408 INFO:tasks.workunit.client.0.smithi016.stdout:1/707: creat d14/d19/d1e/d5e/fe7 x:0 0 0 2024-04-02T04:42:48.409 INFO:tasks.workunit.client.1.smithi027.stdout:8/189: rename d5/d8/db/dd/f21 to d5/d8/d15/f39 0 2024-04-02T04:42:48.411 INFO:tasks.workunit.client.0.smithi016.stdout:5/710: dwrite d3/f4 [0,4194304] 0 2024-04-02T04:42:48.411 INFO:tasks.workunit.client.0.smithi016.stdout:2/787: chown d12/d35/d3f/f61 89692590 1 2024-04-02T04:42:48.413 INFO:tasks.workunit.client.0.smithi016.stdout:2/788: chown d12/d35/d101/d76/da0/lf1 12560 1 2024-04-02T04:42:48.413 INFO:tasks.workunit.client.1.smithi027.stdout:8/190: dread d5/d8/db/dd/f1d [0,4194304] 0 2024-04-02T04:42:48.413 INFO:tasks.workunit.client.0.smithi016.stdout:2/789: chown d12/d35/d101/d4f/dfa 10647 1 2024-04-02T04:42:48.416 INFO:tasks.workunit.client.1.smithi027.stdout:6/227: mknod d13/d1b/c4f 0 2024-04-02T04:42:48.435 INFO:tasks.workunit.client.1.smithi027.stdout:3/164: write d1/de/f30 [112362,66575] 0 2024-04-02T04:42:48.438 INFO:tasks.workunit.client.1.smithi027.stdout:4/201: creat d24/f3a x:0 0 0 2024-04-02T04:42:48.441 INFO:tasks.workunit.client.1.smithi027.stdout:9/220: mkdir d1a/d1d/d29/d50 0 2024-04-02T04:42:48.444 INFO:tasks.workunit.client.1.smithi027.stdout:5/293: symlink d0/df/d11/d41/l71 0 2024-04-02T04:42:48.449 INFO:tasks.workunit.client.1.smithi027.stdout:8/191: write d5/d7/f2b [507984,60554] 0 2024-04-02T04:42:48.455 INFO:tasks.workunit.client.1.smithi027.stdout:7/227: creat d2/d7/da/dc/d16/d3f/f43 x:0 0 0 2024-04-02T04:42:48.456 INFO:tasks.workunit.client.1.smithi027.stdout:7/228: truncate d2/d1c/f37 1002938 0 2024-04-02T04:42:48.456 INFO:tasks.workunit.client.1.smithi027.stdout:7/229: truncate d2/d1c/f40 612706 0 2024-04-02T04:42:48.458 INFO:tasks.workunit.client.1.smithi027.stdout:2/209: creat d1/db/d3b/d17/f3f x:0 0 0 2024-04-02T04:42:48.460 INFO:tasks.workunit.client.1.smithi027.stdout:7/230: dread d2/f13 [0,4194304] 0 2024-04-02T04:42:48.462 INFO:tasks.workunit.client.1.smithi027.stdout:7/231: read d2/d7/da/dc/d16/f2c [167329,65715] 0 2024-04-02T04:42:48.463 INFO:tasks.workunit.client.1.smithi027.stdout:7/232: chown d2/d7/da/dc/d1f 24401 1 2024-04-02T04:42:48.463 INFO:tasks.workunit.client.1.smithi027.stdout:3/165: creat d1/de/df/d13/f34 x:0 0 0 2024-04-02T04:42:48.464 INFO:tasks.workunit.client.1.smithi027.stdout:7/233: read d2/d1c/f40 [421081,122076] 0 2024-04-02T04:42:48.466 INFO:tasks.workunit.client.1.smithi027.stdout:7/234: read d2/d7/da/dc/d16/f2c [260007,23782] 0 2024-04-02T04:42:48.466 INFO:tasks.workunit.client.1.smithi027.stdout:4/202: rmdir d24/d2d 39 2024-04-02T04:42:48.470 INFO:tasks.workunit.client.1.smithi027.stdout:9/221: mkdir d1a/d1d/d29/d51 0 2024-04-02T04:42:48.472 INFO:tasks.workunit.client.1.smithi027.stdout:5/294: mkdir d0/d1d/d61/d72 0 2024-04-02T04:42:48.476 INFO:tasks.workunit.client.1.smithi027.stdout:8/192: rmdir d5/d7 39 2024-04-02T04:42:48.494 INFO:tasks.workunit.client.1.smithi027.stdout:7/235: mkdir d2/d7/da/dc/d16/d3f/d44 0 2024-04-02T04:42:48.496 INFO:tasks.workunit.client.1.smithi027.stdout:9/222: unlink d1a/f32 0 2024-04-02T04:42:48.497 INFO:tasks.workunit.client.1.smithi027.stdout:9/223: write fa [4335390,54929] 0 2024-04-02T04:42:48.513 INFO:tasks.workunit.client.1.smithi027.stdout:9/224: rmdir d1a/d1d/d3c 39 2024-04-02T04:42:48.516 INFO:tasks.workunit.client.1.smithi027.stdout:5/295: link d0/f47 d0/d1a/d20/f73 0 2024-04-02T04:42:48.519 INFO:tasks.workunit.client.1.smithi027.stdout:9/225: dwrite fa [0,4194304] 0 2024-04-02T04:42:48.525 INFO:tasks.workunit.client.0.smithi016.stdout:6/787: truncate d0/d9/d16/d56/d89/d67/d6f/f7f 1224220 0 2024-04-02T04:42:48.528 INFO:tasks.workunit.client.1.smithi027.stdout:8/193: creat d5/d7/d27/f3a x:0 0 0 2024-04-02T04:42:48.528 INFO:tasks.workunit.client.1.smithi027.stdout:8/194: dread - d5/d8/d15/f37 zero size 2024-04-02T04:42:48.530 INFO:tasks.workunit.client.1.smithi027.stdout:9/226: mknod d1a/d1d/d29/d37/d3a/c52 0 2024-04-02T04:42:48.534 INFO:tasks.workunit.client.1.smithi027.stdout:5/296: mkdir d0/d68/d6b/d74 0 2024-04-02T04:42:48.534 INFO:tasks.workunit.client.1.smithi027.stdout:1/184: truncate d0/d1e/d25/f39 1252831 0 2024-04-02T04:42:48.535 INFO:tasks.workunit.client.1.smithi027.stdout:5/297: write d0/f69 [513371,53208] 0 2024-04-02T04:42:48.536 INFO:tasks.workunit.client.1.smithi027.stdout:1/185: dread - d0/de/f23 zero size 2024-04-02T04:42:48.536 INFO:tasks.workunit.client.1.smithi027.stdout:5/298: chown d0/d1d/d61 1194 1 2024-04-02T04:42:48.537 INFO:tasks.workunit.client.1.smithi027.stdout:5/299: rename d0/d68/d6b to d0/d68/d6b/d75 22 2024-04-02T04:42:48.538 INFO:tasks.workunit.client.1.smithi027.stdout:8/195: creat d5/d8/db/dd/f3b x:0 0 0 2024-04-02T04:42:48.539 INFO:tasks.workunit.client.0.smithi016.stdout:0/701: sync 2024-04-02T04:42:48.541 INFO:tasks.workunit.client.1.smithi027.stdout:1/186: symlink d0/d1e/d25/d3c/l43 0 2024-04-02T04:42:48.542 INFO:tasks.workunit.client.0.smithi016.stdout:0/702: dread d0/d4f/daf/fd5 [0,4194304] 0 2024-04-02T04:42:48.553 INFO:tasks.workunit.client.1.smithi027.stdout:9/227: creat d1a/d1d/f53 x:0 0 0 2024-04-02T04:42:48.557 INFO:tasks.workunit.client.0.smithi016.stdout:8/718: rmdir d1/d6/d36/d3f/d40/de9/deb 0 2024-04-02T04:42:48.558 INFO:tasks.workunit.client.0.smithi016.stdout:8/719: stat d1/d6/d86/fec 0 2024-04-02T04:42:48.559 INFO:tasks.workunit.client.1.smithi027.stdout:9/228: creat d1a/d1d/d29/d37/d3a/f54 x:0 0 0 2024-04-02T04:42:48.561 INFO:tasks.workunit.client.1.smithi027.stdout:8/196: link l1 d5/d8/l3c 0 2024-04-02T04:42:48.566 INFO:tasks.workunit.client.1.smithi027.stdout:8/197: symlink d5/d7/d27/l3d 0 2024-04-02T04:42:48.566 INFO:tasks.workunit.client.1.smithi027.stdout:8/198: stat d5/l6 0 2024-04-02T04:42:48.569 INFO:tasks.workunit.client.0.smithi016.stdout:1/708: creat d14/d19/d1e/d6d/fe8 x:0 0 0 2024-04-02T04:42:48.570 INFO:tasks.workunit.client.0.smithi016.stdout:3/838: dread - d2/d15/d16/d8a/dd1/d6e/fbd zero size 2024-04-02T04:42:48.571 INFO:tasks.workunit.client.1.smithi027.stdout:8/199: creat d5/d8/d15/f3e x:0 0 0 2024-04-02T04:42:48.572 INFO:tasks.workunit.client.1.smithi027.stdout:8/200: write d5/d8/db/f35 [72555,19260] 0 2024-04-02T04:42:48.574 INFO:tasks.workunit.client.1.smithi027.stdout:8/201: write d5/d7/d27/f3a [78438,10683] 0 2024-04-02T04:42:48.575 INFO:tasks.workunit.client.1.smithi027.stdout:8/202: dread - d5/d8/d15/f3e zero size 2024-04-02T04:42:48.578 INFO:tasks.workunit.client.0.smithi016.stdout:6/788: creat d0/d9/d16/ff4 x:0 0 0 2024-04-02T04:42:48.579 INFO:tasks.workunit.client.1.smithi027.stdout:9/229: link d1a/d1d/d29/d37/l4a d1a/d1d/d29/d50/l55 0 2024-04-02T04:42:48.580 INFO:tasks.workunit.client.0.smithi016.stdout:6/789: write d0/d9/d16/d46/dbe/fd1 [11161,28223] 0 2024-04-02T04:42:48.580 INFO:tasks.workunit.client.0.smithi016.stdout:6/790: dread - d0/fe4 zero size 2024-04-02T04:42:48.581 INFO:tasks.workunit.client.0.smithi016.stdout:0/703: symlink d0/d48/lee 0 2024-04-02T04:42:48.583 INFO:tasks.workunit.client.1.smithi027.stdout:9/230: mkdir d1a/d1d/d29/d56 0 2024-04-02T04:42:48.584 INFO:tasks.workunit.client.0.smithi016.stdout:7/719: mknod d0/d46/d37/dbe/de1/cfc 0 2024-04-02T04:42:48.587 INFO:tasks.workunit.client.1.smithi027.stdout:9/231: dread d1a/d1d/d29/d37/f35 [0,4194304] 0 2024-04-02T04:42:48.591 INFO:tasks.workunit.client.0.smithi016.stdout:1/709: write d14/d19/d20/f62 [939334,121873] 0 2024-04-02T04:42:48.591 INFO:tasks.workunit.client.0.smithi016.stdout:8/720: dwrite d1/d6/d36/f37 [0,4194304] 0 2024-04-02T04:42:48.593 INFO:tasks.workunit.client.1.smithi027.stdout:0/251: dwrite d4/d16/f1c [0,4194304] 0 2024-04-02T04:42:48.595 INFO:tasks.workunit.client.0.smithi016.stdout:5/711: truncate d3/df/d18/f83 1314283 0 2024-04-02T04:42:48.601 INFO:tasks.workunit.client.0.smithi016.stdout:6/791: mkdir d0/d5c/df5 0 2024-04-02T04:42:48.603 INFO:tasks.workunit.client.1.smithi027.stdout:9/232: creat d1a/f57 x:0 0 0 2024-04-02T04:42:48.603 INFO:tasks.workunit.client.1.smithi027.stdout:0/252: dwrite d4/d16/d2b/f2f [4194304,4194304] 0 2024-04-02T04:42:48.605 INFO:tasks.workunit.client.1.smithi027.stdout:9/233: rename d1a/d1d/d29/d37/d4c to d1a/d1d/d29/d37/d4c/d58 22 2024-04-02T04:42:48.606 INFO:tasks.workunit.client.1.smithi027.stdout:9/234: stat d1a/f1f 0 2024-04-02T04:42:48.610 INFO:tasks.workunit.client.0.smithi016.stdout:7/720: dread d0/d46/d16/dc2/dea/d64/fd6 [0,4194304] 0 2024-04-02T04:42:48.611 INFO:tasks.workunit.client.1.smithi027.stdout:0/253: dread - d4/d16/d2b/d4a/f42 zero size 2024-04-02T04:42:48.612 INFO:tasks.workunit.client.1.smithi027.stdout:0/254: chown d4/d16/d2b/f24 0 1 2024-04-02T04:42:48.613 INFO:tasks.workunit.client.0.smithi016.stdout:1/710: rename d14/d19/d1d/d45/d4f/d86/ld9 to d14/d19/d20/de2/le9 0 2024-04-02T04:42:48.618 INFO:tasks.workunit.client.0.smithi016.stdout:2/790: creat d12/d35/d101/f114 x:0 0 0 2024-04-02T04:42:48.619 INFO:tasks.workunit.client.0.smithi016.stdout:2/791: read - d12/d35/d101/f114 zero size 2024-04-02T04:42:48.619 INFO:tasks.workunit.client.1.smithi027.stdout:0/255: mknod d4/d38/c52 0 2024-04-02T04:42:48.622 INFO:tasks.workunit.client.1.smithi027.stdout:0/256: mkdir d4/d16/d53 0 2024-04-02T04:42:48.624 INFO:tasks.workunit.client.0.smithi016.stdout:8/721: write d1/d70/d71/d74/fa6 [5141165,16020] 0 2024-04-02T04:42:48.627 INFO:tasks.workunit.client.0.smithi016.stdout:7/721: creat d0/d46/d37/dbe/dae/df8/d43/d96/ffd x:0 0 0 2024-04-02T04:42:48.628 INFO:tasks.workunit.client.1.smithi027.stdout:6/228: write d13/d1b/f20 [4994370,57905] 0 2024-04-02T04:42:48.633 INFO:tasks.workunit.client.1.smithi027.stdout:6/229: truncate fc 1525797 0 2024-04-02T04:42:48.636 INFO:tasks.workunit.client.1.smithi027.stdout:6/230: dread d13/f16 [0,4194304] 0 2024-04-02T04:42:48.638 INFO:tasks.workunit.client.1.smithi027.stdout:2/210: rmdir d1/db/d3b/d17 39 2024-04-02T04:42:48.642 INFO:tasks.workunit.client.0.smithi016.stdout:1/711: creat d14/d19/d1e/d71/de0/fea x:0 0 0 2024-04-02T04:42:48.643 INFO:tasks.workunit.client.0.smithi016.stdout:8/722: dwrite d1/d31/d67/d8f/fbe [0,4194304] 0 2024-04-02T04:42:48.644 INFO:tasks.workunit.client.1.smithi027.stdout:6/231: mknod d13/d1b/d24/c50 0 2024-04-02T04:42:48.647 INFO:tasks.workunit.client.1.smithi027.stdout:2/211: chown d1/db/d3b/d17/f14 0 1 2024-04-02T04:42:48.648 INFO:tasks.workunit.client.0.smithi016.stdout:0/704: link d0/d16/d24/c30 d0/d16/d2e/d53/da4/de3/cef 0 2024-04-02T04:42:48.649 INFO:tasks.workunit.client.0.smithi016.stdout:1/712: dwrite d14/d19/d1d/d45/f92 [0,4194304] 0 2024-04-02T04:42:48.650 INFO:tasks.workunit.client.0.smithi016.stdout:7/722: creat d0/d46/d52/dd5/ffe x:0 0 0 2024-04-02T04:42:48.650 INFO:tasks.workunit.client.0.smithi016.stdout:0/705: dread - d0/d16/d24/d4e/d8d/d71/dc7/fcb zero size 2024-04-02T04:42:48.652 INFO:tasks.workunit.client.0.smithi016.stdout:1/713: dread - d14/d19/d1d/d45/d4f/d86/fbb zero size 2024-04-02T04:42:48.653 INFO:tasks.workunit.client.1.smithi027.stdout:6/232: rename d13/d1b/d24/d28/f39 to d13/d1b/d24/d25/d27/f51 0 2024-04-02T04:42:48.654 INFO:tasks.workunit.client.0.smithi016.stdout:1/714: write d14/d19/d1e/d71/fbf [279811,116421] 0 2024-04-02T04:42:48.656 INFO:tasks.workunit.client.1.smithi027.stdout:2/212: creat d1/f40 x:0 0 0 2024-04-02T04:42:48.657 INFO:tasks.workunit.client.1.smithi027.stdout:2/213: read - d1/db/d3b/d17/d12/f38 zero size 2024-04-02T04:42:48.660 INFO:tasks.workunit.client.0.smithi016.stdout:0/706: dwrite d0/d16/d24/f7b [4194304,4194304] 0 2024-04-02T04:42:48.663 INFO:tasks.workunit.client.1.smithi027.stdout:2/214: creat d1/db/d3b/d17/d12/f41 x:0 0 0 2024-04-02T04:42:48.663 INFO:tasks.workunit.client.0.smithi016.stdout:7/723: fsync d0/d46/d16/d49/f11 0 2024-04-02T04:42:48.667 INFO:tasks.workunit.client.1.smithi027.stdout:2/215: fsync d1/db/d3b/d17/f14 0 2024-04-02T04:42:48.668 INFO:tasks.workunit.client.0.smithi016.stdout:7/724: write d0/d46/d37/dbe/dae/df8/f7d [772254,55399] 0 2024-04-02T04:42:48.669 INFO:tasks.workunit.client.0.smithi016.stdout:1/715: dwrite d14/d19/d20/f3f [0,4194304] 0 2024-04-02T04:42:48.672 INFO:tasks.workunit.client.0.smithi016.stdout:8/723: link d1/d6/d36/d3f/d40/ff3 d1/d6/d36/d8e/ff6 0 2024-04-02T04:42:48.672 INFO:tasks.workunit.client.0.smithi016.stdout:2/792: mknod d12/d35/d3b/d42/d59/d104/c115 0 2024-04-02T04:42:48.675 INFO:tasks.workunit.client.1.smithi027.stdout:2/216: symlink d1/db/d3b/d17/d2b/d2f/l42 0 2024-04-02T04:42:48.675 INFO:tasks.workunit.client.0.smithi016.stdout:8/724: truncate d1/d6/d36/d3f/d40/d45/dcc/ff4 900167 0 2024-04-02T04:42:48.676 INFO:tasks.workunit.client.1.smithi027.stdout:2/217: truncate d1/f40 396951 0 2024-04-02T04:42:48.677 INFO:tasks.workunit.client.0.smithi016.stdout:0/707: rename d0/d16/d2e/d53/da4/cdc to d0/d16/dd6/ddd/cf0 0 2024-04-02T04:42:48.679 INFO:tasks.workunit.client.0.smithi016.stdout:1/716: truncate d14/d19/d1d/d29/d39/d46/dcb/f47 338520 0 2024-04-02T04:42:48.680 INFO:tasks.workunit.client.1.smithi027.stdout:2/218: dread d1/db/f1d [0,4194304] 0 2024-04-02T04:42:48.681 INFO:tasks.workunit.client.1.smithi027.stdout:3/166: write d1/de/df/f19 [562026,7853] 0 2024-04-02T04:42:48.682 INFO:tasks.workunit.client.1.smithi027.stdout:2/219: dread - d1/db/d3b/f37 zero size 2024-04-02T04:42:48.682 INFO:tasks.workunit.client.0.smithi016.stdout:2/793: dread - d12/d35/d3b/d42/d67/f10b zero size 2024-04-02T04:42:48.683 INFO:tasks.workunit.client.0.smithi016.stdout:2/794: read d12/d35/d3f/d4d/dde/ff0 [2562722,34148] 0 2024-04-02T04:42:48.683 INFO:tasks.workunit.client.0.smithi016.stdout:2/795: stat d12/d35/d3b/d42/d59/d91/db2 0 2024-04-02T04:42:48.685 INFO:tasks.workunit.client.0.smithi016.stdout:2/796: dread - d12/d35/d101/f114 zero size 2024-04-02T04:42:48.685 INFO:tasks.workunit.client.1.smithi027.stdout:3/167: creat d1/de/df/d13/f35 x:0 0 0 2024-04-02T04:42:48.686 INFO:tasks.workunit.client.0.smithi016.stdout:0/708: rmdir d0/d16/d2e/d53/da4/de3 39 2024-04-02T04:42:48.687 INFO:tasks.workunit.client.1.smithi027.stdout:3/168: write d1/de/df/d13/f35 [381896,20490] 0 2024-04-02T04:42:48.693 INFO:tasks.workunit.client.1.smithi027.stdout:3/169: dwrite d1/f8 [0,4194304] 0 2024-04-02T04:42:48.696 INFO:tasks.workunit.client.0.smithi016.stdout:2/797: creat d12/d35/d3b/d42/d59/f116 x:0 0 0 2024-04-02T04:42:48.699 INFO:tasks.workunit.client.1.smithi027.stdout:3/170: dread d1/f32 [0,4194304] 0 2024-04-02T04:42:48.700 INFO:tasks.workunit.client.0.smithi016.stdout:1/717: dread d14/d19/d1e/d5e/fac [0,4194304] 0 2024-04-02T04:42:48.701 INFO:tasks.workunit.client.0.smithi016.stdout:0/709: rename d0/d16/d24/d4e/d8d/d71/dc7 to d0/d16/d24/d4e/d8d/d71/df1 0 2024-04-02T04:42:48.701 INFO:tasks.workunit.client.0.smithi016.stdout:1/718: dread - d14/d19/d50/fc8 zero size 2024-04-02T04:42:48.702 INFO:tasks.workunit.client.0.smithi016.stdout:0/710: chown d0/d16/d2e/d53/d9d/ced 76781 1 2024-04-02T04:42:48.703 INFO:tasks.workunit.client.0.smithi016.stdout:2/798: read - d12/d35/d101/d76/da0/de7/fbd zero size 2024-04-02T04:42:48.704 INFO:tasks.workunit.client.0.smithi016.stdout:2/799: write d12/d26/f48 [354104,42230] 0 2024-04-02T04:42:48.704 INFO:tasks.workunit.client.1.smithi027.stdout:3/171: rename d1/f18 to d1/de/df/d13/f36 0 2024-04-02T04:42:48.706 INFO:tasks.workunit.client.0.smithi016.stdout:2/800: chown d12/d9f/d107/dee 3 1 2024-04-02T04:42:48.706 INFO:tasks.workunit.client.1.smithi027.stdout:3/172: read - d1/f2a zero size 2024-04-02T04:42:48.706 INFO:tasks.workunit.client.1.smithi027.stdout:3/173: stat d1/d1a/l28 0 2024-04-02T04:42:48.707 INFO:tasks.workunit.client.0.smithi016.stdout:1/719: chown d14/d19/d50/c8e 4384686 1 2024-04-02T04:42:48.713 INFO:tasks.workunit.client.0.smithi016.stdout:0/711: symlink d0/d16/d2e/d53/da4/lf2 0 2024-04-02T04:42:48.713 INFO:tasks.workunit.client.0.smithi016.stdout:0/712: chown d0/fc 29 1 2024-04-02T04:42:48.714 INFO:tasks.workunit.client.0.smithi016.stdout:2/801: mknod d12/d35/d101/d4f/dfa/c117 0 2024-04-02T04:42:48.715 INFO:tasks.workunit.client.0.smithi016.stdout:0/713: write d0/d16/d2e/d53/f7c [813232,8126] 0 2024-04-02T04:42:48.716 INFO:tasks.workunit.client.0.smithi016.stdout:2/802: write d12/d35/d3b/d42/d59/d91/fff [444964,102644] 0 2024-04-02T04:42:48.717 INFO:tasks.workunit.client.0.smithi016.stdout:2/803: write d12/d70/fe8 [1026307,126671] 0 2024-04-02T04:42:48.718 INFO:tasks.workunit.client.0.smithi016.stdout:0/714: unlink d0/d48/f58 0 2024-04-02T04:42:48.719 INFO:tasks.workunit.client.1.smithi027.stdout:4/203: dwrite f1e [0,4194304] 0 2024-04-02T04:42:48.724 INFO:tasks.workunit.client.0.smithi016.stdout:0/715: mknod d0/d4f/d76/cf3 0 2024-04-02T04:42:48.728 INFO:tasks.workunit.client.0.smithi016.stdout:2/804: link d12/d35/d3b/d42/fdf d12/d35/d3f/d56/d66/f118 0 2024-04-02T04:42:48.733 INFO:tasks.workunit.client.0.smithi016.stdout:2/805: chown d12/d35/d3b/d42/d8d/l8f 194 1 2024-04-02T04:42:48.733 INFO:tasks.workunit.client.1.smithi027.stdout:5/300: mkdir d0/df/d76 0 2024-04-02T04:42:48.734 INFO:tasks.workunit.client.1.smithi027.stdout:4/204: getdents d24/d2d 0 2024-04-02T04:42:48.735 INFO:tasks.workunit.client.0.smithi016.stdout:5/712: dread d3/df/d18/d2f/f3f [0,4194304] 0 2024-04-02T04:42:48.738 INFO:tasks.workunit.client.1.smithi027.stdout:5/301: truncate d0/df/d2f/d3b/f3d 3369970 0 2024-04-02T04:42:48.739 INFO:tasks.workunit.client.1.smithi027.stdout:5/302: write d0/df/d2f/d3b/f5b [1425509,14680] 0 2024-04-02T04:42:48.743 INFO:tasks.workunit.client.0.smithi016.stdout:5/713: rename d3/d4c to d3/d40/d4f/d6b/d45/da6/d102 0 2024-04-02T04:42:48.744 INFO:tasks.workunit.client.0.smithi016.stdout:5/714: write d3/d40/d4f/d6b/f47 [4453474,127638] 0 2024-04-02T04:42:48.747 INFO:tasks.workunit.client.1.smithi027.stdout:5/303: dwrite d0/f4f [0,4194304] 0 2024-04-02T04:42:48.748 INFO:tasks.workunit.client.0.smithi016.stdout:9/803: sync 2024-04-02T04:42:48.750 INFO:tasks.workunit.client.0.smithi016.stdout:5/715: chown d3/df/d18/d26/d70/ffe 1 1 2024-04-02T04:42:48.753 INFO:tasks.workunit.client.1.smithi027.stdout:5/304: truncate d0/f47 178569 0 2024-04-02T04:42:48.759 INFO:tasks.workunit.client.1.smithi027.stdout:9/235: rmdir d1a/d1d/d29/d37/d3a 39 2024-04-02T04:42:48.762 INFO:tasks.workunit.client.1.smithi027.stdout:9/236: mknod d1a/d1d/d29/d56/c59 0 2024-04-02T04:42:48.765 INFO:tasks.workunit.client.0.smithi016.stdout:4/742: dwrite d5/d17/d18/d6d/d8e/d9c/dcc/d6f/f84 [0,4194304] 0 2024-04-02T04:42:48.766 INFO:tasks.workunit.client.0.smithi016.stdout:4/743: truncate d5/d17/dcd/fd8 187897 0 2024-04-02T04:42:48.767 INFO:tasks.workunit.client.1.smithi027.stdout:9/237: creat d1a/d1d/d29/d37/f5a x:0 0 0 2024-04-02T04:42:48.767 INFO:tasks.workunit.client.0.smithi016.stdout:4/744: fdatasync d5/d17/d18/d6d/f74 0 2024-04-02T04:42:48.771 INFO:tasks.workunit.client.1.smithi027.stdout:5/305: dread d0/d1d/f35 [0,4194304] 0 2024-04-02T04:42:48.772 INFO:tasks.workunit.client.0.smithi016.stdout:4/745: dread d5/d17/d18/d6d/d8e/d9c/dcc/d6f/f84 [0,4194304] 0 2024-04-02T04:42:48.774 INFO:tasks.workunit.client.0.smithi016.stdout:4/746: readlink d5/d17/d18/d6d/da4/db7/ldd 0 2024-04-02T04:42:48.774 INFO:tasks.workunit.client.1.smithi027.stdout:9/238: rename d1a/d1d/d29/d37/d20/l25 to d1a/d1d/d29/d37/d4c/l5b 0 2024-04-02T04:42:48.776 INFO:tasks.workunit.client.1.smithi027.stdout:9/239: chown d1a/d1d/d29/d37/d20/l2d 5 1 2024-04-02T04:42:48.778 INFO:tasks.workunit.client.1.smithi027.stdout:9/240: write d1a/d1d/d29/f4f [430510,28648] 0 2024-04-02T04:42:48.778 INFO:tasks.workunit.client.0.smithi016.stdout:9/804: fdatasync d0/d9/fcf 0 2024-04-02T04:42:48.779 INFO:tasks.workunit.client.1.smithi027.stdout:5/306: dwrite d0/df/d11/f1f [0,4194304] 0 2024-04-02T04:42:48.781 INFO:tasks.workunit.client.0.smithi016.stdout:7/725: sync 2024-04-02T04:42:48.781 INFO:tasks.workunit.client.0.smithi016.stdout:3/839: write d2/d15/d16/d8a/dd1/d6e/fbd [557935,55920] 0 2024-04-02T04:42:48.783 INFO:tasks.workunit.client.0.smithi016.stdout:3/840: chown d2/d15/d70/dcf/fdc 91 1 2024-04-02T04:42:48.784 INFO:tasks.workunit.client.0.smithi016.stdout:9/805: write d0/d9/db/dc9/d53/fc5 [4621975,44936] 0 2024-04-02T04:42:48.786 INFO:tasks.workunit.client.1.smithi027.stdout:5/307: dread d0/d1d/f35 [0,4194304] 0 2024-04-02T04:42:48.787 INFO:tasks.workunit.client.1.smithi027.stdout:9/241: dread f12 [0,4194304] 0 2024-04-02T04:42:48.789 INFO:tasks.workunit.client.0.smithi016.stdout:9/806: write d0/d4a/d99/f48 [3388192,39406] 0 2024-04-02T04:42:48.794 INFO:tasks.workunit.client.1.smithi027.stdout:5/308: rename d0/df/d11/d13/c18 to d0/df/d11/d65/c77 0 2024-04-02T04:42:48.797 INFO:tasks.workunit.client.0.smithi016.stdout:4/747: link d5/d17/d18/d1f/f59 d5/d1d/d5f/d95/fdf 0 2024-04-02T04:42:48.799 INFO:tasks.workunit.client.0.smithi016.stdout:9/807: mkdir d0/d4a/d75/d95/d9b/db4/df9 0 2024-04-02T04:42:48.801 INFO:tasks.workunit.client.0.smithi016.stdout:4/748: creat d5/d17/d18/d6d/d8e/fe0 x:0 0 0 2024-04-02T04:42:48.802 INFO:tasks.workunit.client.1.smithi027.stdout:5/309: symlink d0/df/l78 0 2024-04-02T04:42:48.803 INFO:tasks.workunit.client.1.smithi027.stdout:9/242: dwrite d1a/d1d/d29/d37/d3a/f54 [0,4194304] 0 2024-04-02T04:42:48.809 INFO:tasks.workunit.client.0.smithi016.stdout:6/792: truncate d0/f3f 7503972 0 2024-04-02T04:42:48.812 INFO:tasks.workunit.client.1.smithi027.stdout:0/257: dwrite d4/fb [0,4194304] 0 2024-04-02T04:42:48.812 INFO:tasks.workunit.client.0.smithi016.stdout:9/808: truncate d0/d4a/f9c 2437795 0 2024-04-02T04:42:48.813 INFO:tasks.workunit.client.0.smithi016.stdout:9/809: chown d0/d4e/d50/ca2 24 1 2024-04-02T04:42:48.814 INFO:tasks.workunit.client.1.smithi027.stdout:5/310: dwrite d0/df/d11/d13/f3e [0,4194304] 0 2024-04-02T04:42:48.814 INFO:tasks.workunit.client.1.smithi027.stdout:0/258: fsync d4/f23 0 2024-04-02T04:42:48.815 INFO:tasks.workunit.client.0.smithi016.stdout:9/810: chown d0/d4e/d50/d63/d6e/d4b/d58/d6a 1861 1 2024-04-02T04:42:48.816 INFO:tasks.workunit.client.1.smithi027.stdout:5/311: fsync d0/f33 0 2024-04-02T04:42:48.818 INFO:tasks.workunit.client.0.smithi016.stdout:6/793: creat d0/d9/d16/d56/d89/ff6 x:0 0 0 2024-04-02T04:42:48.818 INFO:tasks.workunit.client.1.smithi027.stdout:5/312: write d0/df/d11/d13/f1e [5151517,7762] 0 2024-04-02T04:42:48.819 INFO:tasks.workunit.client.0.smithi016.stdout:6/794: readlink d0/d9/d16/d56/d89/db4/lbb 0 2024-04-02T04:42:48.820 INFO:tasks.workunit.client.1.smithi027.stdout:5/313: write d0/df/d11/d37/f3a [4791489,76689] 0 2024-04-02T04:42:48.821 INFO:tasks.workunit.client.1.smithi027.stdout:5/314: read d0/df/d11/d37/f4e [337770,74661] 0 2024-04-02T04:42:48.823 INFO:tasks.workunit.client.0.smithi016.stdout:9/811: rename d0/d4e/f61 to d0/d4e/d50/d63/d6e/d4b/d58/dd5/ffa 0 2024-04-02T04:42:48.825 INFO:tasks.workunit.client.0.smithi016.stdout:6/795: dread d0/d9/df/f4d [0,4194304] 0 2024-04-02T04:42:48.825 INFO:tasks.workunit.client.1.smithi027.stdout:9/243: dread d1a/f1f [0,4194304] 0 2024-04-02T04:42:48.826 INFO:tasks.workunit.client.1.smithi027.stdout:0/259: creat d4/d21/d28/f54 x:0 0 0 2024-04-02T04:42:48.827 INFO:tasks.workunit.client.1.smithi027.stdout:6/233: write d13/d1b/d24/d25/d27/f44 [508062,123808] 0 2024-04-02T04:42:48.830 INFO:tasks.workunit.client.0.smithi016.stdout:9/812: creat d0/d9/db/d54/ffb x:0 0 0 2024-04-02T04:42:48.831 INFO:tasks.workunit.client.0.smithi016.stdout:6/796: dread d0/d9/df/f4d [0,4194304] 0 2024-04-02T04:42:48.834 INFO:tasks.workunit.client.1.smithi027.stdout:9/244: dwrite d1a/f57 [0,4194304] 0 2024-04-02T04:42:48.835 INFO:tasks.workunit.client.1.smithi027.stdout:6/234: dread d13/d1b/d24/f41 [0,4194304] 0 2024-04-02T04:42:48.836 INFO:tasks.workunit.client.1.smithi027.stdout:9/245: chown d1a/d1d/d29/l36 119246 1 2024-04-02T04:42:48.837 INFO:tasks.workunit.client.0.smithi016.stdout:6/797: rmdir d0/d9/d16/d46/d5e 39 2024-04-02T04:42:48.838 INFO:tasks.workunit.client.1.smithi027.stdout:6/235: truncate d13/d1b/f23 53473 0 2024-04-02T04:42:48.839 INFO:tasks.workunit.client.0.smithi016.stdout:9/813: symlink d0/d4e/d50/d63/d6e/lfc 0 2024-04-02T04:42:48.842 INFO:tasks.workunit.client.1.smithi027.stdout:9/246: symlink d1a/d1d/d29/d37/d4c/l5c 0 2024-04-02T04:42:48.843 INFO:tasks.workunit.client.1.smithi027.stdout:9/247: truncate d1a/f57 5193513 0 2024-04-02T04:42:48.844 INFO:tasks.workunit.client.1.smithi027.stdout:9/248: read - d1a/d1d/f53 zero size 2024-04-02T04:42:48.845 INFO:tasks.workunit.client.1.smithi027.stdout:9/249: truncate d1a/d1d/f42 540300 0 2024-04-02T04:42:48.846 INFO:tasks.workunit.client.1.smithi027.stdout:9/250: write d1a/f38 [837776,35435] 0 2024-04-02T04:42:48.848 INFO:tasks.workunit.client.1.smithi027.stdout:9/251: fsync f17 0 2024-04-02T04:42:48.849 INFO:tasks.workunit.client.1.smithi027.stdout:9/252: chown d1a/d1d/d29/d37/d3a/c49 491 1 2024-04-02T04:42:48.850 INFO:tasks.workunit.client.1.smithi027.stdout:2/220: truncate d1/db/d3b/d17/d12/f1f 583732 0 2024-04-02T04:42:48.851 INFO:tasks.workunit.client.1.smithi027.stdout:2/221: chown d1/c2a 15 1 2024-04-02T04:42:48.851 INFO:tasks.workunit.client.0.smithi016.stdout:8/725: write d1/d6/d36/d3f/d40/d45/fe2 [1517934,60138] 0 2024-04-02T04:42:48.852 INFO:tasks.workunit.client.0.smithi016.stdout:8/726: readlink d1/d6/d36/ld4 0 2024-04-02T04:42:48.856 INFO:tasks.workunit.client.0.smithi016.stdout:8/727: symlink d1/d70/d71/lf7 0 2024-04-02T04:42:48.858 INFO:tasks.workunit.client.0.smithi016.stdout:1/720: write fa [1468177,15237] 0 2024-04-02T04:42:48.858 INFO:tasks.workunit.client.1.smithi027.stdout:3/174: truncate d1/d1a/f22 4083459 0 2024-04-02T04:42:48.861 INFO:tasks.workunit.client.0.smithi016.stdout:0/716: write d0/fe [2930841,68761] 0 2024-04-02T04:42:48.864 INFO:tasks.workunit.client.0.smithi016.stdout:8/728: dwrite d1/d70/d71/d9e/fcd [0,4194304] 0 2024-04-02T04:42:48.865 INFO:tasks.workunit.client.0.smithi016.stdout:0/717: write d0/d16/f94 [318994,83497] 0 2024-04-02T04:42:48.871 INFO:tasks.workunit.client.0.smithi016.stdout:8/729: chown d1/d6/d36/d3f 4986 1 2024-04-02T04:42:48.871 INFO:tasks.workunit.client.0.smithi016.stdout:2/806: dwrite d12/d35/fca [0,4194304] 0 2024-04-02T04:42:48.872 INFO:tasks.workunit.client.0.smithi016.stdout:2/807: stat d12/d35/d3b/fc7 0 2024-04-02T04:42:48.874 INFO:tasks.workunit.client.1.smithi027.stdout:7/236: sync 2024-04-02T04:42:48.877 INFO:tasks.workunit.client.0.smithi016.stdout:5/716: truncate d3/d40/d4f/d6b/d45/da6/d102/ff6 374441 0 2024-04-02T04:42:48.877 INFO:tasks.workunit.client.0.smithi016.stdout:1/721: link d14/d19/d1d/d29/d39/d4d/ld7 d14/d19/d1d/d45/d4f/leb 0 2024-04-02T04:42:48.891 INFO:tasks.workunit.client.0.smithi016.stdout:0/718: getdents d0/d16 0 2024-04-02T04:42:48.894 INFO:tasks.workunit.client.0.smithi016.stdout:2/808: link d12/d35/d3b/d42/f65 d12/d35/d3b/d42/d59/d91/db2/dfd/f119 0 2024-04-02T04:42:48.896 INFO:tasks.workunit.client.0.smithi016.stdout:5/717: getdents d3/d40/d4f/d6b/d45/da6 0 2024-04-02T04:42:48.897 INFO:tasks.workunit.client.0.smithi016.stdout:1/722: getdents d14/d19/d1d/d45/d4f/d86 0 2024-04-02T04:42:48.898 INFO:tasks.workunit.client.0.smithi016.stdout:4/749: dwrite d5/d8/fd [0,4194304] 0 2024-04-02T04:42:48.899 INFO:tasks.workunit.client.0.smithi016.stdout:2/809: write d12/d35/d3f/d56/fb7 [2304826,15393] 0 2024-04-02T04:42:48.901 INFO:tasks.workunit.client.0.smithi016.stdout:1/723: chown d14/d19/d1e/f5d 2501363 1 2024-04-02T04:42:48.903 INFO:tasks.workunit.client.0.smithi016.stdout:5/718: rename d3/d40/d4f/d67/f6a to d3/d5d/f103 0 2024-04-02T04:42:48.903 INFO:tasks.workunit.client.0.smithi016.stdout:1/724: readlink d14/d19/d1e/d5e/la4 0 2024-04-02T04:42:48.903 INFO:tasks.workunit.client.0.smithi016.stdout:0/719: dwrite d0/d16/d24/d4e/d8d/d71/fd8 [0,4194304] 0 2024-04-02T04:42:48.903 INFO:tasks.workunit.client.0.smithi016.stdout:6/798: write d0/d6e/f8b [4627438,72886] 0 2024-04-02T04:42:48.906 INFO:tasks.workunit.client.0.smithi016.stdout:9/814: write d0/d9/db/dc9/f57 [1587525,62460] 0 2024-04-02T04:42:48.906 INFO:tasks.workunit.client.0.smithi016.stdout:5/719: chown d3/df/d18/f89 199936 1 2024-04-02T04:42:48.910 INFO:tasks.workunit.client.0.smithi016.stdout:2/810: symlink d12/d35/d3b/d42/d59/d91/d10a/l11a 0 2024-04-02T04:42:48.910 INFO:tasks.workunit.client.0.smithi016.stdout:4/750: creat d5/d17/d76/d7e/fe1 x:0 0 0 2024-04-02T04:42:48.912 INFO:tasks.workunit.client.0.smithi016.stdout:9/815: unlink d0/d9/d4d/da8/le5 0 2024-04-02T04:42:48.916 INFO:tasks.workunit.client.0.smithi016.stdout:5/720: mkdir d3/d40/d4f/d67/dbf/d104 0 2024-04-02T04:42:48.917 INFO:tasks.workunit.client.1.smithi027.stdout:2/222: write d1/db/d3b/d17/f14 [5315535,96603] 0 2024-04-02T04:42:48.918 INFO:tasks.workunit.client.0.smithi016.stdout:4/751: write d5/d17/d18/fc8 [313724,34198] 0 2024-04-02T04:42:48.918 INFO:tasks.workunit.client.1.smithi027.stdout:2/223: stat d1/db/l1a 0 2024-04-02T04:42:48.920 INFO:tasks.workunit.client.0.smithi016.stdout:2/811: creat d12/d35/d101/d4f/dfa/f11b x:0 0 0 2024-04-02T04:42:48.920 INFO:tasks.workunit.client.0.smithi016.stdout:9/816: mknod d0/d4a/d75/d95/cfd 0 2024-04-02T04:42:48.921 INFO:tasks.workunit.client.0.smithi016.stdout:2/812: chown d12/f7b 1961 1 2024-04-02T04:42:48.921 INFO:tasks.workunit.client.0.smithi016.stdout:5/721: dread d3/d40/d4f/d6b/d45/fab [0,4194304] 0 2024-04-02T04:42:48.924 INFO:tasks.workunit.client.1.smithi027.stdout:2/224: mknod d1/db/d3b/d17/d2b/c43 0 2024-04-02T04:42:48.932 INFO:tasks.workunit.client.1.smithi027.stdout:7/237: truncate d2/d7/da/dc/f30 1203069 0 2024-04-02T04:42:48.940 INFO:tasks.workunit.client.1.smithi027.stdout:2/225: write d1/db/d3b/d17/d12/f1f [950309,118097] 0 2024-04-02T04:42:48.945 INFO:tasks.workunit.client.1.smithi027.stdout:2/226: rename d1/db/d3b/d17/l10 to d1/db/d3b/d17/d12/l44 0 2024-04-02T04:42:48.953 INFO:tasks.workunit.client.1.smithi027.stdout:2/227: truncate d1/db/f30 4184945 0 2024-04-02T04:42:48.957 INFO:tasks.workunit.client.1.smithi027.stdout:2/228: getdents d1/db/d3b 0 2024-04-02T04:42:48.959 INFO:tasks.workunit.client.1.smithi027.stdout:2/229: stat d1/db/d3b/d17/c2d 0 2024-04-02T04:42:48.961 INFO:tasks.workunit.client.1.smithi027.stdout:2/230: symlink d1/db/d3b/d17/d29/l45 0 2024-04-02T04:42:48.961 INFO:tasks.workunit.client.1.smithi027.stdout:2/231: readlink d1/db/d3b/d17/d12/l28 0 2024-04-02T04:42:48.969 INFO:tasks.workunit.client.1.smithi027.stdout:2/232: dread d1/db/f30 [0,4194304] 0 2024-04-02T04:42:48.972 INFO:tasks.workunit.client.1.smithi027.stdout:2/233: link d1/db/f32 d1/f46 0 2024-04-02T04:42:48.973 INFO:tasks.workunit.client.1.smithi027.stdout:2/234: symlink d1/db/d3b/d17/d12/l47 0 2024-04-02T04:42:48.974 INFO:tasks.workunit.client.0.smithi016.stdout:7/726: sync 2024-04-02T04:42:48.974 INFO:tasks.workunit.client.0.smithi016.stdout:3/841: sync 2024-04-02T04:42:48.978 INFO:tasks.workunit.client.0.smithi016.stdout:2/813: dwrite d12/d35/d3f/d56/fcc [0,4194304] 0 2024-04-02T04:42:48.979 INFO:tasks.workunit.client.1.smithi027.stdout:2/235: dwrite d1/db/d3b/d17/f14 [4194304,4194304] 0 2024-04-02T04:42:48.981 INFO:tasks.workunit.client.0.smithi016.stdout:4/752: write d5/d1d/d1e/f9a [4088978,114940] 0 2024-04-02T04:42:48.982 INFO:tasks.workunit.client.1.smithi027.stdout:2/236: chown d1/db/d3b/d17/l27 0 1 2024-04-02T04:42:48.982 INFO:tasks.workunit.client.0.smithi016.stdout:9/817: write d0/d3/fa7 [377721,33103] 0 2024-04-02T04:42:48.985 INFO:tasks.workunit.client.0.smithi016.stdout:6/799: rename d0/d9/d16/d56/d89/d67/d6f/f7f to d0/d9/d16/de3/ff7 0 2024-04-02T04:42:48.987 INFO:tasks.workunit.client.0.smithi016.stdout:3/842: truncate d2/d15/d1a/d45/fa4 324643 0 2024-04-02T04:42:48.989 INFO:tasks.workunit.client.0.smithi016.stdout:8/730: dwrite d1/d6/d36/d3f/d77/fae [0,4194304] 0 2024-04-02T04:42:48.991 INFO:tasks.workunit.client.0.smithi016.stdout:4/753: creat d5/d17/d18/d6d/d91/fe2 x:0 0 0 2024-04-02T04:42:48.992 INFO:tasks.workunit.client.0.smithi016.stdout:9/818: mkdir d0/d3/d39/dfe 0 2024-04-02T04:42:48.992 INFO:tasks.workunit.client.0.smithi016.stdout:3/843: stat d2/d15/d1a/dd5/c118 0 2024-04-02T04:42:48.993 INFO:tasks.workunit.client.0.smithi016.stdout:3/844: chown d2/d2e/d36/d66/d10d 7352486 1 2024-04-02T04:42:48.995 INFO:tasks.workunit.client.0.smithi016.stdout:8/731: truncate d1/d6/d36/d3f/f80 1729532 0 2024-04-02T04:42:49.078 INFO:tasks.workunit.client.0.smithi016.stdout:5/722: rename d3/faa to d3/d40/d4f/d6b/d45/da6/d102/dc4/f105 0 2024-04-02T04:42:49.078 INFO:tasks.workunit.client.0.smithi016.stdout:9/819: write d0/d9/db/d54/fdc [416255,65077] 0 2024-04-02T04:42:49.081 INFO:tasks.workunit.client.1.smithi027.stdout:2/237: link d1/db/d3b/d17/d2b/d2f/l3d d1/db/d3b/d17/d12/l48 0 2024-04-02T04:42:49.085 INFO:tasks.workunit.client.1.smithi027.stdout:2/238: creat d1/db/f49 x:0 0 0 2024-04-02T04:42:49.085 INFO:tasks.workunit.client.1.smithi027.stdout:2/239: unlink d1/f46 0 2024-04-02T04:42:49.085 INFO:tasks.workunit.client.1.smithi027.stdout:2/240: write d1/db/d3b/d17/d12/f38 [772370,35828] 0 2024-04-02T04:42:49.085 INFO:tasks.workunit.client.1.smithi027.stdout:2/241: dwrite d1/db/d3b/d17/fc [8388608,4194304] 0 2024-04-02T04:42:49.085 INFO:tasks.workunit.client.1.smithi027.stdout:2/242: chown d1/db/d3b/d17/d12 862 1 2024-04-02T04:42:49.085 INFO:tasks.workunit.client.1.smithi027.stdout:2/243: creat d1/db/d3b/d17/f4a x:0 0 0 2024-04-02T04:42:49.085 INFO:tasks.workunit.client.1.smithi027.stdout:2/244: rename d1/db/d3b/d17/d12/l47 to d1/db/d3b/d17/l4b 0 2024-04-02T04:42:49.085 INFO:tasks.workunit.client.1.smithi027.stdout:2/245: link d1/db/d3b/d17/d2b/f26 d1/db/d3b/f4c 0 2024-04-02T04:42:49.085 INFO:tasks.workunit.client.1.smithi027.stdout:2/246: creat d1/f4d x:0 0 0 2024-04-02T04:42:49.085 INFO:tasks.workunit.client.1.smithi027.stdout:1/187: sync 2024-04-02T04:42:49.086 INFO:tasks.workunit.client.1.smithi027.stdout:8/203: sync 2024-04-02T04:42:49.086 INFO:tasks.workunit.client.1.smithi027.stdout:8/204: chown d5/d7/c19 3 1 2024-04-02T04:42:49.086 INFO:tasks.workunit.client.1.smithi027.stdout:1/188: symlink d0/de/l44 0 2024-04-02T04:42:49.086 INFO:tasks.workunit.client.1.smithi027.stdout:8/205: dread d5/d8/f33 [0,4194304] 0 2024-04-02T04:42:49.089 INFO:tasks.workunit.client.0.smithi016.stdout:8/732: dwrite d1/d31/f46 [4194304,4194304] 0 2024-04-02T04:42:49.089 INFO:tasks.workunit.client.0.smithi016.stdout:3/845: rmdir d2/d15/d1a/dd5 39 2024-04-02T04:42:49.089 INFO:tasks.workunit.client.0.smithi016.stdout:2/814: write d12/d35/d101/d76/fbb [25501,112570] 0 2024-04-02T04:42:49.089 INFO:tasks.workunit.client.0.smithi016.stdout:2/815: fsync d12/f96 0 2024-04-02T04:42:49.089 INFO:tasks.workunit.client.0.smithi016.stdout:6/800: dread d0/d9/d16/d46/dbe/fd0 [0,4194304] 0 2024-04-02T04:42:49.089 INFO:tasks.workunit.client.0.smithi016.stdout:1/725: dwrite d14/d19/d1d/d29/d39/d46/dcb/f47 [0,4194304] 0 2024-04-02T04:42:49.089 INFO:tasks.workunit.client.0.smithi016.stdout:6/801: readlink d0/d9/lc6 0 2024-04-02T04:42:49.089 INFO:tasks.workunit.client.0.smithi016.stdout:7/727: rename d0/d46/d16/dc2/dea/cd1 to d0/d46/d37/dbe/dae/df8/d84/d8a/cff 0 2024-04-02T04:42:49.089 INFO:tasks.workunit.client.0.smithi016.stdout:8/733: read - d1/d70/d71/d74/fbd zero size 2024-04-02T04:42:49.089 INFO:tasks.workunit.client.0.smithi016.stdout:8/734: dread - d1/d6/d36/d3f/d40/d45/fb9 zero size 2024-04-02T04:42:49.089 INFO:tasks.workunit.client.0.smithi016.stdout:3/846: fdatasync d2/f9 0 2024-04-02T04:42:49.090 INFO:tasks.workunit.client.0.smithi016.stdout:3/847: write d2/d15/d60/fe5 [142314,95527] 0 2024-04-02T04:42:49.090 INFO:tasks.workunit.client.0.smithi016.stdout:1/726: rename d14/d19/d1d/d45/lbe to d14/d19/d1e/d6d/lec 0 2024-04-02T04:42:49.090 INFO:tasks.workunit.client.0.smithi016.stdout:8/735: creat d1/d31/d67/d8f/ff8 x:0 0 0 2024-04-02T04:42:49.090 INFO:tasks.workunit.client.0.smithi016.stdout:4/754: link d5/d17/d18/d25/f41 d5/d17/d76/fe3 0 2024-04-02T04:42:49.090 INFO:tasks.workunit.client.0.smithi016.stdout:7/728: dwrite d0/d46/f30 [4194304,4194304] 0 2024-04-02T04:42:49.090 INFO:tasks.workunit.client.0.smithi016.stdout:8/736: stat d1/d6/d36/d3f/d40/d49/d4a/d4c/dad 0 2024-04-02T04:42:49.090 INFO:tasks.workunit.client.0.smithi016.stdout:7/729: stat d0/d46/d16/d49/fa 0 2024-04-02T04:42:49.090 INFO:tasks.workunit.client.0.smithi016.stdout:5/723: link d3/d40/d4f/d6b/fa2 d3/df/d18/d26/f106 0 2024-04-02T04:42:49.090 INFO:tasks.workunit.client.0.smithi016.stdout:1/727: symlink d14/d19/d1e/d71/led 0 2024-04-02T04:42:49.090 INFO:tasks.workunit.client.0.smithi016.stdout:4/755: creat d5/d8/d43/dd7/fe4 x:0 0 0 2024-04-02T04:42:49.090 INFO:tasks.workunit.client.0.smithi016.stdout:6/802: dread d0/d9/fe2 [0,4194304] 0 2024-04-02T04:42:49.090 INFO:tasks.workunit.client.0.smithi016.stdout:6/803: chown d0/f84 28703 1 2024-04-02T04:42:49.090 INFO:tasks.workunit.client.0.smithi016.stdout:7/730: unlink d0/d21/fe9 0 2024-04-02T04:42:49.090 INFO:tasks.workunit.client.0.smithi016.stdout:5/724: rename d3/df/fbb to d3/d40/d4f/d6b/d45/da6/d102/dd1/f107 0 2024-04-02T04:42:49.090 INFO:tasks.workunit.client.0.smithi016.stdout:9/820: truncate d0/d4a/d99/f48 3660833 0 2024-04-02T04:42:49.091 INFO:tasks.workunit.client.0.smithi016.stdout:5/725: read d3/df/f14 [30992,72457] 0 2024-04-02T04:42:49.091 INFO:tasks.workunit.client.0.smithi016.stdout:9/821: write d0/d4a/d75/d95/d9a/fe4 [428756,66682] 0 2024-04-02T04:42:49.091 INFO:tasks.workunit.client.0.smithi016.stdout:5/726: write d3/d40/d72/f7d [7191657,68789] 0 2024-04-02T04:42:49.091 INFO:tasks.workunit.client.0.smithi016.stdout:1/728: symlink d14/d19/d1e/d71/dcf/lee 0 2024-04-02T04:42:49.091 INFO:tasks.workunit.client.0.smithi016.stdout:7/731: dwrite d0/d46/d37/ff0 [0,4194304] 0 2024-04-02T04:42:49.091 INFO:tasks.workunit.client.0.smithi016.stdout:4/756: mknod d5/d17/d18/da0/ce5 0 2024-04-02T04:42:49.091 INFO:tasks.workunit.client.0.smithi016.stdout:8/737: write d1/d6/d36/d3f/d40/d63/f66 [530903,52232] 0 2024-04-02T04:42:49.091 INFO:tasks.workunit.client.0.smithi016.stdout:4/757: write d5/d1d/d1e/f3c [5996008,22290] 0 2024-04-02T04:42:49.091 INFO:tasks.workunit.client.0.smithi016.stdout:3/848: getdents d2/d15/d16/d57 0 2024-04-02T04:42:49.091 INFO:tasks.workunit.client.0.smithi016.stdout:8/738: dread d1/d6/d36/d3f/d40/d45/fe2 [0,4194304] 0 2024-04-02T04:42:49.091 INFO:tasks.workunit.client.0.smithi016.stdout:1/729: creat d14/d19/d1d/d29/d39/d46/dcb/d43/fef x:0 0 0 2024-04-02T04:42:49.091 INFO:tasks.workunit.client.0.smithi016.stdout:7/732: rename d0/d46/d37/d47/fb4 to d0/d46/d37/dbe/f100 0 2024-04-02T04:42:49.091 INFO:tasks.workunit.client.0.smithi016.stdout:1/730: dwrite d14/d19/d1d/d45/f92 [0,4194304] 0 2024-04-02T04:42:49.091 INFO:tasks.workunit.client.0.smithi016.stdout:1/731: readlink l9 0 2024-04-02T04:42:49.091 INFO:tasks.workunit.client.0.smithi016.stdout:1/732: write d14/d19/d1d/d29/d39/d46/dcb/d43/fef [241571,44119] 0 2024-04-02T04:42:49.095 INFO:tasks.workunit.client.1.smithi027.stdout:2/247: dread d1/db/d3b/d17/fc [0,4194304] 0 2024-04-02T04:42:49.096 INFO:tasks.workunit.client.1.smithi027.stdout:2/248: readlink d1/db/d3b/d17/d2b/d2f/l42 0 2024-04-02T04:42:49.108 INFO:tasks.workunit.client.0.smithi016.stdout:2/816: sync 2024-04-02T04:42:49.108 INFO:tasks.workunit.client.0.smithi016.stdout:0/720: sync 2024-04-02T04:42:49.118 INFO:tasks.workunit.client.0.smithi016.stdout:2/817: dread d12/d35/d3f/d56/f94 [0,4194304] 0 2024-04-02T04:42:49.118 INFO:tasks.workunit.client.0.smithi016.stdout:2/818: stat d12/d35/d101/d76/da0/de7/dfc/d106 0 2024-04-02T04:42:49.118 INFO:tasks.workunit.client.0.smithi016.stdout:2/819: readlink d12/d35/d3b/d42/d59/d91/d10a/l11a 0 2024-04-02T04:42:49.166 INFO:tasks.workunit.client.0.smithi016.stdout:6/804: creat d0/d9/d16/d46/d5e/ff8 x:0 0 0 2024-04-02T04:42:49.173 INFO:tasks.workunit.client.1.smithi027.stdout:8/206: mknod d5/d8/c3f 0 2024-04-02T04:42:49.174 INFO:tasks.workunit.client.0.smithi016.stdout:6/805: write d0/d6e/f8b [4306547,6662] 0 2024-04-02T04:42:49.174 INFO:tasks.workunit.client.1.smithi027.stdout:1/189: symlink d0/l45 0 2024-04-02T04:42:49.175 INFO:tasks.workunit.client.1.smithi027.stdout:8/207: dwrite d5/d7/d27/f3a [0,4194304] 0 2024-04-02T04:42:49.175 INFO:tasks.workunit.client.1.smithi027.stdout:2/249: creat d1/db/d3b/d17/d2b/f4e x:0 0 0 2024-04-02T04:42:49.175 INFO:tasks.workunit.client.0.smithi016.stdout:4/758: creat d5/d17/d18/d6d/da4/fe6 x:0 0 0 2024-04-02T04:42:49.176 INFO:tasks.workunit.client.0.smithi016.stdout:9/822: dwrite d0/d9/db/dc9/fa4 [0,4194304] 0 2024-04-02T04:42:49.177 INFO:tasks.workunit.client.0.smithi016.stdout:6/806: dwrite d0/d9/f36 [0,4194304] 0 2024-04-02T04:42:49.177 INFO:tasks.workunit.client.0.smithi016.stdout:4/759: chown d5/d17/d18/d6d/d8e/c9f 0 1 2024-04-02T04:42:49.178 INFO:tasks.workunit.client.1.smithi027.stdout:8/208: dread d5/d8/f17 [0,4194304] 0 2024-04-02T04:42:49.178 INFO:tasks.workunit.client.0.smithi016.stdout:6/807: chown d0/d9/d16/d56/c79 2848 1 2024-04-02T04:42:49.179 INFO:tasks.workunit.client.1.smithi027.stdout:8/209: truncate d5/d8/d15/f37 858150 0 2024-04-02T04:42:49.181 INFO:tasks.workunit.client.0.smithi016.stdout:8/739: fsync d1/d70/d71/d74/fbd 0 2024-04-02T04:42:49.187 INFO:tasks.workunit.client.0.smithi016.stdout:5/727: rename d3/df/d18/d26/ddc to d3/d40/d4f/d6b/d45/da6/d102/dc4/d108 0 2024-04-02T04:42:49.190 INFO:tasks.workunit.client.0.smithi016.stdout:7/733: creat d0/d46/d37/dbe/dae/df8/d43/d96/ddf/f101 x:0 0 0 2024-04-02T04:42:49.190 INFO:tasks.workunit.client.0.smithi016.stdout:6/808: dwrite d0/d9/d16/d56/fe6 [0,4194304] 0 2024-04-02T04:42:49.191 INFO:tasks.workunit.client.0.smithi016.stdout:0/721: chown d0/d16/d2e/d37/d66/faa 9518 1 2024-04-02T04:42:49.191 INFO:tasks.workunit.client.1.smithi027.stdout:2/250: creat d1/db/d3b/d17/d29/f4f x:0 0 0 2024-04-02T04:42:49.191 INFO:tasks.workunit.client.1.smithi027.stdout:8/210: creat d5/d8/db/dd/f40 x:0 0 0 2024-04-02T04:42:49.191 INFO:tasks.workunit.client.1.smithi027.stdout:1/190: mknod d0/c46 0 2024-04-02T04:42:49.191 INFO:tasks.workunit.client.1.smithi027.stdout:1/191: chown d0/d1e/d2e/d3e 917172 1 2024-04-02T04:42:49.192 INFO:tasks.workunit.client.1.smithi027.stdout:1/192: chown d0/d1e/f3a 1427 1 2024-04-02T04:42:49.192 INFO:tasks.workunit.client.1.smithi027.stdout:4/205: sync 2024-04-02T04:42:49.192 INFO:tasks.workunit.client.1.smithi027.stdout:1/193: read d0/d1e/f2f [21243,18901] 0 2024-04-02T04:42:49.192 INFO:tasks.workunit.client.0.smithi016.stdout:2/820: mkdir d12/d70/d11c 0 2024-04-02T04:42:49.194 INFO:tasks.workunit.client.1.smithi027.stdout:1/194: write d0/f1d [1740631,7713] 0 2024-04-02T04:42:49.197 INFO:tasks.workunit.client.1.smithi027.stdout:8/211: rename d5/d7/c18 to d5/d7/d27/d31/c41 0 2024-04-02T04:42:49.202 INFO:tasks.workunit.client.1.smithi027.stdout:2/251: mkdir d1/db/d3b/d17/d50 0 2024-04-02T04:42:49.204 INFO:tasks.workunit.client.1.smithi027.stdout:8/212: dwrite d5/d8/db/dd/f10 [0,4194304] 0 2024-04-02T04:42:49.206 INFO:tasks.workunit.client.0.smithi016.stdout:9/823: fsync d0/f7 0 2024-04-02T04:42:49.210 INFO:tasks.workunit.client.1.smithi027.stdout:2/252: fsync d1/db/d3b/d17/fc 0 2024-04-02T04:42:49.215 INFO:tasks.workunit.client.0.smithi016.stdout:4/760: rename d5/d17/d76/c89 to d5/d17/d18/d6d/d8e/d9c/dcc/d6f/dcf/ce7 0 2024-04-02T04:42:49.223 INFO:tasks.workunit.client.1.smithi027.stdout:1/195: getdents d0/d1e/d2e/d3b 0 2024-04-02T04:42:49.224 INFO:tasks.workunit.client.0.smithi016.stdout:6/809: truncate d0/d9/d16/d56/d95/fab 12854 0 2024-04-02T04:42:49.224 INFO:tasks.workunit.client.0.smithi016.stdout:6/810: write d0/d9/d16/d56/f4f [1729974,63692] 0 2024-04-02T04:42:49.229 INFO:tasks.workunit.client.1.smithi027.stdout:8/213: link d5/l12 d5/d8/l42 0 2024-04-02T04:42:49.229 INFO:tasks.workunit.client.1.smithi027.stdout:1/196: dread d0/d1e/d25/f31 [0,4194304] 0 2024-04-02T04:42:49.229 INFO:tasks.workunit.client.1.smithi027.stdout:8/214: readlink d5/l6 0 2024-04-02T04:42:49.229 INFO:tasks.workunit.client.1.smithi027.stdout:8/215: chown d5/d8/db/df/f22 180257851 1 2024-04-02T04:42:49.230 INFO:tasks.workunit.client.0.smithi016.stdout:3/849: link d2/l10 d2/d2e/d36/d48/da2/d88/l11a 0 2024-04-02T04:42:49.230 INFO:tasks.workunit.client.1.smithi027.stdout:8/216: read d5/d7/f14 [1109676,80361] 0 2024-04-02T04:42:49.231 INFO:tasks.workunit.client.1.smithi027.stdout:2/253: creat d1/db/d3b/f51 x:0 0 0 2024-04-02T04:42:49.234 INFO:tasks.workunit.client.1.smithi027.stdout:1/197: mkdir d0/d1e/d25/d2b/d47 0 2024-04-02T04:42:49.235 INFO:tasks.workunit.client.0.smithi016.stdout:9/824: creat d0/d4a/d75/fff x:0 0 0 2024-04-02T04:42:49.237 INFO:tasks.workunit.client.1.smithi027.stdout:8/217: unlink d5/d8/db/dd/f40 0 2024-04-02T04:42:49.238 INFO:tasks.workunit.client.0.smithi016.stdout:3/850: dread d2/f9 [0,4194304] 0 2024-04-02T04:42:49.238 INFO:tasks.workunit.client.0.smithi016.stdout:2/821: dwrite d12/d35/d101/d4f/f5f [0,4194304] 0 2024-04-02T04:42:49.244 INFO:tasks.workunit.client.0.smithi016.stdout:0/722: unlink d0/d4f/cb6 0 2024-04-02T04:42:49.245 INFO:tasks.workunit.client.1.smithi027.stdout:8/218: mkdir d5/d8/db/dd/d43 0 2024-04-02T04:42:49.247 INFO:tasks.workunit.client.1.smithi027.stdout:2/254: creat d1/db/d3b/d17/d2b/f52 x:0 0 0 2024-04-02T04:42:49.247 INFO:tasks.workunit.client.0.smithi016.stdout:8/740: dwrite d1/ff [0,4194304] 0 2024-04-02T04:42:49.248 INFO:tasks.workunit.client.1.smithi027.stdout:1/198: rename d0/d1e/d25/f39 to d0/d1e/d25/d3c/f48 0 2024-04-02T04:42:49.249 INFO:tasks.workunit.client.1.smithi027.stdout:1/199: chown d0/f7 1101049 1 2024-04-02T04:42:49.251 INFO:tasks.workunit.client.1.smithi027.stdout:2/255: readlink d1/db/d3b/d17/l33 0 2024-04-02T04:42:49.252 INFO:tasks.workunit.client.1.smithi027.stdout:2/256: fdatasync d1/db/d3b/f51 0 2024-04-02T04:42:49.261 INFO:tasks.workunit.client.0.smithi016.stdout:7/734: dwrite d0/d46/f2c [0,4194304] 0 2024-04-02T04:42:49.264 INFO:tasks.workunit.client.0.smithi016.stdout:5/728: truncate d3/d40/d4f/d6b/d45/da6/d102/dd1/fa3 3133463 0 2024-04-02T04:42:49.265 INFO:tasks.workunit.client.1.smithi027.stdout:8/219: rmdir d5/d8/db/dd/d43 0 2024-04-02T04:42:49.270 INFO:tasks.workunit.client.1.smithi027.stdout:8/220: mknod d5/d8/c44 0 2024-04-02T04:42:49.270 INFO:tasks.workunit.client.0.smithi016.stdout:7/735: dwrite d0/d46/d16/dc2/ff3 [0,4194304] 0 2024-04-02T04:42:49.272 INFO:tasks.workunit.client.0.smithi016.stdout:2/822: read d12/d35/d3b/d42/d8d/fa6 [1435750,93307] 0 2024-04-02T04:42:49.273 INFO:tasks.workunit.client.1.smithi027.stdout:2/257: rmdir d1/db/d3b/d17/d50 0 2024-04-02T04:42:49.275 INFO:tasks.workunit.client.0.smithi016.stdout:5/729: dwrite d3/f5b [0,4194304] 0 2024-04-02T04:42:49.276 INFO:tasks.workunit.client.1.smithi027.stdout:8/221: unlink d5/d8/db/f35 0 2024-04-02T04:42:49.277 INFO:tasks.workunit.client.0.smithi016.stdout:2/823: truncate d12/d35/d3b/d42/d59/d91/fff 558598 0 2024-04-02T04:42:49.277 INFO:tasks.workunit.client.0.smithi016.stdout:0/723: dread d0/d4f/f95 [0,4194304] 0 2024-04-02T04:42:49.278 INFO:tasks.workunit.client.1.smithi027.stdout:2/258: creat d1/db/d3b/d17/d2b/f53 x:0 0 0 2024-04-02T04:42:49.278 INFO:tasks.workunit.client.1.smithi027.stdout:1/200: write d0/f7 [15885,105044] 0 2024-04-02T04:42:49.279 INFO:tasks.workunit.client.1.smithi027.stdout:2/259: readlink d1/db/d3b/d17/l33 0 2024-04-02T04:42:49.280 INFO:tasks.workunit.client.1.smithi027.stdout:1/201: truncate d0/de/f38 924588 0 2024-04-02T04:42:49.280 INFO:tasks.workunit.client.0.smithi016.stdout:9/825: mkdir d0/d4e/d50/d63/d100 0 2024-04-02T04:42:49.281 INFO:tasks.workunit.client.1.smithi027.stdout:1/202: chown d0/d1e/f3a 25 1 2024-04-02T04:42:49.295 INFO:tasks.workunit.client.0.smithi016.stdout:3/851: creat d2/d2e/d36/d66/d90/dfb/f11b x:0 0 0 2024-04-02T04:42:49.296 INFO:tasks.workunit.client.0.smithi016.stdout:3/852: chown d2/d15/d16/d8a/dcd/ffe 99 1 2024-04-02T04:42:49.298 INFO:tasks.workunit.client.0.smithi016.stdout:1/733: sync 2024-04-02T04:42:49.301 INFO:tasks.workunit.client.0.smithi016.stdout:7/736: unlink d0/d46/d16/dc2/dea/l2e 0 2024-04-02T04:42:49.303 INFO:tasks.workunit.client.1.smithi027.stdout:2/260: write d1/f13 [9669103,63702] 0 2024-04-02T04:42:49.304 INFO:tasks.workunit.client.1.smithi027.stdout:5/315: sync 2024-04-02T04:42:49.304 INFO:tasks.workunit.client.1.smithi027.stdout:8/222: link d5/l12 d5/d7/d27/d31/l45 0 2024-04-02T04:42:49.305 INFO:tasks.workunit.client.1.smithi027.stdout:2/261: truncate d1/db/d3b/d17/d2b/f4e 655683 0 2024-04-02T04:42:49.306 INFO:tasks.workunit.client.0.smithi016.stdout:6/811: creat d0/d9/d16/d56/d89/d67/ff9 x:0 0 0 2024-04-02T04:42:49.306 INFO:tasks.workunit.client.1.smithi027.stdout:2/262: readlink d1/db/l1b 0 2024-04-02T04:42:49.307 INFO:tasks.workunit.client.0.smithi016.stdout:6/812: write d0/d9/d16/d56/d59/da1/fcf [197536,117364] 0 2024-04-02T04:42:49.307 INFO:tasks.workunit.client.1.smithi027.stdout:1/203: truncate d0/f16 4330700 0 2024-04-02T04:42:49.309 INFO:tasks.workunit.client.0.smithi016.stdout:0/724: creat d0/d16/d2e/d37/d66/ff4 x:0 0 0 2024-04-02T04:42:49.310 INFO:tasks.workunit.client.1.smithi027.stdout:5/316: dread d0/d1a/d20/f59 [0,4194304] 0 2024-04-02T04:42:49.310 INFO:tasks.workunit.client.0.smithi016.stdout:0/725: dread - d0/d16/d23/d3e/fc6 zero size 2024-04-02T04:42:49.311 INFO:tasks.workunit.client.1.smithi027.stdout:8/223: creat d5/d8/db/dd/f46 x:0 0 0 2024-04-02T04:42:49.316 INFO:tasks.workunit.client.0.smithi016.stdout:9/826: dread d0/f5d [0,4194304] 0 2024-04-02T04:42:49.316 INFO:tasks.workunit.client.1.smithi027.stdout:1/204: creat d0/d1e/d2e/f49 x:0 0 0 2024-04-02T04:42:49.318 INFO:tasks.workunit.client.0.smithi016.stdout:3/853: mkdir d2/d15/d70/dcf/d11c 0 2024-04-02T04:42:49.319 INFO:tasks.workunit.client.0.smithi016.stdout:1/734: unlink d14/d19/d1d/d29/d39/d46/cca 0 2024-04-02T04:42:49.320 INFO:tasks.workunit.client.0.smithi016.stdout:1/735: readlink l9 0 2024-04-02T04:42:49.321 INFO:tasks.workunit.client.1.smithi027.stdout:5/317: creat d0/d1d/f79 x:0 0 0 2024-04-02T04:42:49.322 INFO:tasks.workunit.client.1.smithi027.stdout:5/318: write d0/df/d2f/f67 [3890754,114798] 0 2024-04-02T04:42:49.324 INFO:tasks.workunit.client.1.smithi027.stdout:5/319: write d0/d1d/f35 [2868353,45074] 0 2024-04-02T04:42:49.326 INFO:tasks.workunit.client.0.smithi016.stdout:5/730: mknod d3/df/d18/d26/c109 0 2024-04-02T04:42:49.327 INFO:tasks.workunit.client.0.smithi016.stdout:5/731: write d3/df/d18/d26/d48/f49 [1020311,81483] 0 2024-04-02T04:42:49.327 INFO:tasks.workunit.client.0.smithi016.stdout:5/732: chown d3/df/dd5 223767168 1 2024-04-02T04:42:49.327 INFO:tasks.workunit.client.1.smithi027.stdout:1/205: creat d0/d1e/d2e/f4a x:0 0 0 2024-04-02T04:42:49.329 INFO:tasks.workunit.client.1.smithi027.stdout:1/206: read d0/f1d [663483,29809] 0 2024-04-02T04:42:49.330 INFO:tasks.workunit.client.1.smithi027.stdout:1/207: write d0/f3d [304975,38642] 0 2024-04-02T04:42:49.331 INFO:tasks.workunit.client.0.smithi016.stdout:0/726: fdatasync d0/d16/d24/fa1 0 2024-04-02T04:42:49.333 INFO:tasks.workunit.client.0.smithi016.stdout:4/761: getdents d5/d1d/d1e/da5 0 2024-04-02T04:42:49.337 INFO:tasks.workunit.client.0.smithi016.stdout:9/827: truncate d0/d9/db/dc9/d53/f9d 792254 0 2024-04-02T04:42:49.338 INFO:tasks.workunit.client.0.smithi016.stdout:4/762: dwrite d5/d17/d76/d7e/fcb [0,4194304] 0 2024-04-02T04:42:49.339 INFO:tasks.workunit.client.1.smithi027.stdout:8/224: rename d5/d8/db/dd/f46 to d5/d8/f47 0 2024-04-02T04:42:49.340 INFO:tasks.workunit.client.0.smithi016.stdout:4/763: truncate d5/d1d/f61 905014 0 2024-04-02T04:42:49.341 INFO:tasks.workunit.client.0.smithi016.stdout:8/741: getdents d1/d6/d36/d3f/d40/d49/d4a/d4c/dad 0 2024-04-02T04:42:49.342 INFO:tasks.workunit.client.1.smithi027.stdout:9/253: sync 2024-04-02T04:42:49.342 INFO:tasks.workunit.client.1.smithi027.stdout:6/236: sync 2024-04-02T04:42:49.342 INFO:tasks.workunit.client.1.smithi027.stdout:7/238: sync 2024-04-02T04:42:49.342 INFO:tasks.workunit.client.1.smithi027.stdout:0/260: sync 2024-04-02T04:42:49.343 INFO:tasks.workunit.client.1.smithi027.stdout:4/206: sync 2024-04-02T04:42:49.343 INFO:tasks.workunit.client.1.smithi027.stdout:3/175: sync 2024-04-02T04:42:49.343 INFO:tasks.workunit.client.1.smithi027.stdout:2/263: getdents d1/db/d3b/d17/d12 0 2024-04-02T04:42:49.344 INFO:tasks.workunit.client.0.smithi016.stdout:5/733: creat d3/d40/db6/f10a x:0 0 0 2024-04-02T04:42:49.345 INFO:tasks.workunit.client.0.smithi016.stdout:5/734: fsync d3/df/f1e 0 2024-04-02T04:42:49.346 INFO:tasks.workunit.client.1.smithi027.stdout:1/208: rename d0/de/l44 to d0/d1e/d2e/d3e/l4b 0 2024-04-02T04:42:49.350 INFO:tasks.workunit.client.0.smithi016.stdout:5/735: dwrite d3/f28 [0,4194304] 0 2024-04-02T04:42:49.351 INFO:tasks.workunit.client.1.smithi027.stdout:8/225: mkdir d5/d8/d48 0 2024-04-02T04:42:49.355 INFO:tasks.workunit.client.0.smithi016.stdout:3/854: creat d2/d15/d16/d35/f11d x:0 0 0 2024-04-02T04:42:49.362 INFO:tasks.workunit.client.1.smithi027.stdout:3/176: unlink d1/de/df/d13/f16 0 2024-04-02T04:42:49.363 INFO:tasks.workunit.client.1.smithi027.stdout:9/254: dwrite d1a/d1d/d29/d37/f35 [0,4194304] 0 2024-04-02T04:42:49.364 INFO:tasks.workunit.client.1.smithi027.stdout:9/255: stat d1a/d1d/d29/d56/c59 0 2024-04-02T04:42:49.364 INFO:tasks.workunit.client.0.smithi016.stdout:1/736: mkdir d14/d19/d1d/d29/d39/df0 0 2024-04-02T04:42:49.365 INFO:tasks.workunit.client.1.smithi027.stdout:4/207: truncate d24/f35 1060625 0 2024-04-02T04:42:49.365 INFO:tasks.workunit.client.1.smithi027.stdout:9/256: read d1a/f38 [479165,71618] 0 2024-04-02T04:42:49.366 INFO:tasks.workunit.client.0.smithi016.stdout:3/855: dread d2/d15/d70/f79 [0,4194304] 0 2024-04-02T04:42:49.367 INFO:tasks.workunit.client.0.smithi016.stdout:8/742: creat d1/d6/d86/ff9 x:0 0 0 2024-04-02T04:42:49.367 INFO:tasks.workunit.client.1.smithi027.stdout:4/208: dread - d24/d2d/d34/f37 zero size 2024-04-02T04:42:49.372 INFO:tasks.workunit.client.1.smithi027.stdout:5/320: rename d0/df/d11/d37/f3a to d0/df/d29/f7a 0 2024-04-02T04:42:49.373 INFO:tasks.workunit.client.1.smithi027.stdout:2/264: rename d1/db to d1/db/d3b/d17/d2b/d54 22 2024-04-02T04:42:49.373 INFO:tasks.workunit.client.1.smithi027.stdout:6/237: dwrite d13/f16 [0,4194304] 0 2024-04-02T04:42:49.373 INFO:tasks.workunit.client.1.smithi027.stdout:1/209: mknod d0/d1e/d25/d3c/c4c 0 2024-04-02T04:42:49.374 INFO:tasks.workunit.client.1.smithi027.stdout:5/321: chown d0/d1a/f49 297 1 2024-04-02T04:42:49.376 INFO:tasks.workunit.client.0.smithi016.stdout:2/824: getdents d12/d35/d3b/d42/d59/d91/db2/dfd 0 2024-04-02T04:42:49.376 INFO:tasks.workunit.client.0.smithi016.stdout:2/825: fsync d12/f43 0 2024-04-02T04:42:49.377 INFO:tasks.workunit.client.1.smithi027.stdout:8/226: symlink d5/d8/l49 0 2024-04-02T04:42:49.377 INFO:tasks.workunit.client.1.smithi027.stdout:5/322: readlink d0/df/d11/d41/l70 0 2024-04-02T04:42:49.382 INFO:tasks.workunit.client.1.smithi027.stdout:0/261: creat d4/d21/d28/d4e/f55 x:0 0 0 2024-04-02T04:42:49.382 INFO:tasks.workunit.client.1.smithi027.stdout:1/210: truncate d0/d1e/d2e/d3e/f42 432729 0 2024-04-02T04:42:49.382 INFO:tasks.workunit.client.1.smithi027.stdout:8/227: chown d5/d8/d15/f37 324 1 2024-04-02T04:42:49.382 INFO:tasks.workunit.client.1.smithi027.stdout:8/228: readlink d5/d8/db/dd/l2f 0 2024-04-02T04:42:49.382 INFO:tasks.workunit.client.0.smithi016.stdout:0/727: mkdir d0/d16/d24/deb/df5 0 2024-04-02T04:42:49.383 INFO:tasks.workunit.client.0.smithi016.stdout:9/828: mknod d0/d3/d39/dfe/c101 0 2024-04-02T04:42:49.383 INFO:tasks.workunit.client.0.smithi016.stdout:9/829: readlink d0/d4e/d50/d63/dd2/lbd 0 2024-04-02T04:42:49.384 INFO:tasks.workunit.client.0.smithi016.stdout:9/830: write d0/d4a/d99/d46/fb7 [1268913,76886] 0 2024-04-02T04:42:49.385 INFO:tasks.workunit.client.1.smithi027.stdout:7/239: creat d2/d7/da/dc/d1f/d33/d36/f45 x:0 0 0 2024-04-02T04:42:49.394 INFO:tasks.workunit.client.1.smithi027.stdout:2/265: rmdir d1/db/d3b/d17/d29 39 2024-04-02T04:42:49.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:49 smithi027 ceph-mon[37550]: pgmap v23: 65 pgs: 65 active+clean; 1.6 GiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 50 MiB/s rd, 173 MiB/s wr, 353 op/s 2024-04-02T04:42:49.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:49 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:49.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:49 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:49.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:49 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-02T04:42:49.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:49 smithi027 ceph-mon[37550]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-02T04:42:49.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:49 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:42:49.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:49 smithi027 ceph-mon[37550]: Upgrade: Need to upgrade myself (mgr.smithi027.kjzcsc) 2024-04-02T04:42:49.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:49 smithi027 ceph-mon[37550]: Upgrade: Need to upgrade myself (mgr.smithi027.kjzcsc) 2024-04-02T04:42:49.397 INFO:tasks.workunit.client.1.smithi027.stdout:5/323: symlink d0/d1a/l7b 0 2024-04-02T04:42:49.399 INFO:tasks.workunit.client.1.smithi027.stdout:6/238: rename d13/d1b/d24/d28/l40 to d13/d1b/d24/d28/d3c/l52 0 2024-04-02T04:42:49.404 INFO:tasks.workunit.client.1.smithi027.stdout:0/262: creat d4/d16/d2b/d2d/f56 x:0 0 0 2024-04-02T04:42:49.409 INFO:tasks.workunit.client.1.smithi027.stdout:4/209: mknod d24/d38/c3b 0 2024-04-02T04:42:49.413 INFO:tasks.workunit.client.1.smithi027.stdout:7/240: symlink d2/d7/da/dc/d1f/d33/l46 0 2024-04-02T04:42:49.413 INFO:tasks.workunit.client.1.smithi027.stdout:7/241: stat d2/d7/da/dc/d1f/d33/l46 0 2024-04-02T04:42:49.418 INFO:tasks.workunit.client.1.smithi027.stdout:2/266: dread d1/db/f32 [0,4194304] 0 2024-04-02T04:42:49.419 INFO:tasks.workunit.client.1.smithi027.stdout:2/267: fsync d1/db/d3b/d17/fc 0 2024-04-02T04:42:49.421 INFO:tasks.workunit.client.1.smithi027.stdout:5/324: dread d0/df/d11/d37/f4e [0,4194304] 0 2024-04-02T04:42:49.422 INFO:tasks.workunit.client.1.smithi027.stdout:5/325: write d0/d1a/f49 [1135819,52188] 0 2024-04-02T04:42:49.423 INFO:tasks.workunit.client.1.smithi027.stdout:5/326: chown d0/d5/c6e 10782 1 2024-04-02T04:42:49.424 INFO:tasks.workunit.client.1.smithi027.stdout:0/263: mknod d4/d38/c57 0 2024-04-02T04:42:49.425 INFO:tasks.workunit.client.1.smithi027.stdout:5/327: write d0/d5/f63 [50905,113149] 0 2024-04-02T04:42:49.431 INFO:tasks.workunit.client.1.smithi027.stdout:0/264: dread d4/fb [0,4194304] 0 2024-04-02T04:42:49.432 INFO:tasks.workunit.client.1.smithi027.stdout:7/242: creat d2/d7/da/dc/d1f/f47 x:0 0 0 2024-04-02T04:42:49.433 INFO:tasks.workunit.client.1.smithi027.stdout:5/328: dwrite d0/df/d11/d13/f26 [0,4194304] 0 2024-04-02T04:42:49.435 INFO:tasks.workunit.client.1.smithi027.stdout:5/329: write d0/df/d11/d65/f6c [41785,38443] 0 2024-04-02T04:42:49.438 INFO:tasks.workunit.client.1.smithi027.stdout:5/330: read d0/df/d11/d13/f1e [3273424,89837] 0 2024-04-02T04:42:49.440 INFO:tasks.workunit.client.1.smithi027.stdout:9/257: truncate d1a/d1d/d29/f4f 53102 0 2024-04-02T04:42:49.442 INFO:tasks.workunit.client.1.smithi027.stdout:0/265: dwrite d4/d16/d2b/f3a [0,4194304] 0 2024-04-02T04:42:49.442 INFO:tasks.workunit.client.1.smithi027.stdout:9/258: write d1a/f39 [469396,36286] 0 2024-04-02T04:42:49.443 INFO:tasks.workunit.client.1.smithi027.stdout:0/266: readlink d4/l1a 0 2024-04-02T04:42:49.443 INFO:tasks.workunit.client.0.smithi016.stdout:0/728: creat d0/d16/d24/d4e/d8d/ff6 x:0 0 0 2024-04-02T04:42:49.443 INFO:tasks.workunit.client.0.smithi016.stdout:1/737: truncate d14/f28 2727246 0 2024-04-02T04:42:49.444 INFO:tasks.workunit.client.0.smithi016.stdout:3/856: mknod d2/d15/d1a/c11e 0 2024-04-02T04:42:49.444 INFO:tasks.workunit.client.0.smithi016.stdout:4/764: rename d5/d16/d88/db0/fd1 to d5/d1d/d5f/fe8 0 2024-04-02T04:42:49.453 INFO:tasks.workunit.client.1.smithi027.stdout:1/211: dwrite d0/de/f17 [0,4194304] 0 2024-04-02T04:42:49.454 INFO:tasks.workunit.client.1.smithi027.stdout:1/212: truncate d0/de/f1b 742910 0 2024-04-02T04:42:49.462 INFO:tasks.workunit.client.1.smithi027.stdout:7/243: creat d2/d7/f48 x:0 0 0 2024-04-02T04:42:49.466 INFO:tasks.workunit.client.1.smithi027.stdout:7/244: dread d2/d7/da/f1e [0,4194304] 0 2024-04-02T04:42:49.472 INFO:tasks.workunit.client.1.smithi027.stdout:0/267: creat d4/d38/f58 x:0 0 0 2024-04-02T04:42:49.473 INFO:tasks.workunit.client.1.smithi027.stdout:0/268: fdatasync d4/d38/f58 0 2024-04-02T04:42:49.474 INFO:tasks.workunit.client.1.smithi027.stdout:2/268: mknod d1/db/d3b/d17/c55 0 2024-04-02T04:42:49.474 INFO:tasks.workunit.client.1.smithi027.stdout:2/269: chown d1/db/d3b/d17/d12 0 1 2024-04-02T04:42:49.476 INFO:tasks.workunit.client.0.smithi016.stdout:2/826: unlink d12/d35/d3b/d42/c58 0 2024-04-02T04:42:49.477 INFO:tasks.workunit.client.0.smithi016.stdout:8/743: truncate d1/d6/d36/d8e/f9c 2095413 0 2024-04-02T04:42:49.478 INFO:tasks.workunit.client.1.smithi027.stdout:1/213: rename d0/d1e/d2e/f49 to d0/de/f4d 0 2024-04-02T04:42:49.480 INFO:tasks.workunit.client.0.smithi016.stdout:0/729: dread - d0/fc8 zero size 2024-04-02T04:42:49.480 INFO:tasks.workunit.client.0.smithi016.stdout:3/857: dread d2/d15/d16/d8a/dd1/d6e/fbd [0,4194304] 0 2024-04-02T04:42:49.481 INFO:tasks.workunit.client.1.smithi027.stdout:8/229: getdents d5/d8/db/df 0 2024-04-02T04:42:49.483 INFO:tasks.workunit.client.1.smithi027.stdout:4/210: creat d24/f3c x:0 0 0 2024-04-02T04:42:49.490 INFO:tasks.workunit.client.1.smithi027.stdout:5/331: link d0/f66 d0/d1d/f7c 0 2024-04-02T04:42:49.491 INFO:tasks.workunit.client.0.smithi016.stdout:2/827: unlink d12/d9f/d107/lae 0 2024-04-02T04:42:49.492 INFO:tasks.workunit.client.1.smithi027.stdout:1/214: dread d0/f14 [4194304,4194304] 0 2024-04-02T04:42:49.493 INFO:tasks.workunit.client.1.smithi027.stdout:1/215: chown d0/d1e/l20 2573316 1 2024-04-02T04:42:49.493 INFO:tasks.workunit.client.1.smithi027.stdout:9/259: mknod d1a/d1d/d29/d51/c5d 0 2024-04-02T04:42:49.494 INFO:tasks.workunit.client.1.smithi027.stdout:1/216: write d0/d1e/d2e/d3e/f42 [964873,46166] 0 2024-04-02T04:42:49.496 INFO:tasks.workunit.client.1.smithi027.stdout:0/269: unlink d4/d16/d2b/f4f 0 2024-04-02T04:42:49.497 INFO:tasks.workunit.client.0.smithi016.stdout:4/765: truncate d5/d17/dcd/fd2 2483940 0 2024-04-02T04:42:49.499 INFO:tasks.workunit.client.1.smithi027.stdout:2/270: fdatasync d1/db/d3b/f37 0 2024-04-02T04:42:49.500 INFO:tasks.workunit.client.0.smithi016.stdout:8/744: mkdir d1/d6/d36/d4f/dc9/dfa 0 2024-04-02T04:42:49.501 INFO:tasks.workunit.client.0.smithi016.stdout:8/745: write d1/d70/d71/d74/fc3 [890696,59450] 0 2024-04-02T04:42:49.503 INFO:tasks.workunit.client.0.smithi016.stdout:8/746: write d1/d6/d36/d3f/d40/d49/f44 [3956518,17738] 0 2024-04-02T04:42:49.505 INFO:tasks.workunit.client.1.smithi027.stdout:0/270: dread d4/d16/d1b/f20 [0,4194304] 0 2024-04-02T04:42:49.505 INFO:tasks.workunit.client.1.smithi027.stdout:7/245: rename d2/d7/da/dc/d1f/d33/l46 to d2/d1c/l49 0 2024-04-02T04:42:49.507 INFO:tasks.workunit.client.1.smithi027.stdout:5/332: dread d0/df/d11/d13/f28 [0,4194304] 0 2024-04-02T04:42:49.507 INFO:tasks.workunit.client.0.smithi016.stdout:3/858: mknod d2/d15/d60/c11f 0 2024-04-02T04:42:49.508 INFO:tasks.workunit.client.1.smithi027.stdout:5/333: dread - d0/d1d/f79 zero size 2024-04-02T04:42:49.509 INFO:tasks.workunit.client.0.smithi016.stdout:3/859: fsync d2/d15/d1a/fc3 0 2024-04-02T04:42:49.509 INFO:tasks.workunit.client.0.smithi016.stdout:8/747: dwrite d1/d6/d36/fa3 [0,4194304] 0 2024-04-02T04:42:49.509 INFO:tasks.workunit.client.0.smithi016.stdout:7/737: sync 2024-04-02T04:42:49.511 INFO:tasks.workunit.client.1.smithi027.stdout:8/230: mkdir d5/d8/db/dd/d4a 0 2024-04-02T04:42:49.514 INFO:tasks.workunit.client.1.smithi027.stdout:7/246: dwrite d2/d7/da/dc/d1f/d33/d36/f45 [0,4194304] 0 2024-04-02T04:42:49.515 INFO:tasks.workunit.client.0.smithi016.stdout:7/738: dwrite d0/d46/f2c [0,4194304] 0 2024-04-02T04:42:49.524 INFO:tasks.workunit.client.1.smithi027.stdout:9/260: readlink d1a/d1d/d29/d37/l21 0 2024-04-02T04:42:49.526 INFO:tasks.workunit.client.1.smithi027.stdout:9/261: chown d1a/d1d/d29/d37/d4c/l5c 56126244 1 2024-04-02T04:42:49.529 INFO:tasks.workunit.client.0.smithi016.stdout:0/730: write d0/d4f/daf/fd5 [381542,37888] 0 2024-04-02T04:42:49.530 INFO:tasks.workunit.client.0.smithi016.stdout:0/731: chown d0/d16/d2e/d37/f4b 4 1 2024-04-02T04:42:49.531 INFO:tasks.workunit.client.0.smithi016.stdout:0/732: chown d0/f67 22805 1 2024-04-02T04:42:49.536 INFO:tasks.workunit.client.1.smithi027.stdout:2/271: rename d1/db/f1d to d1/f56 0 2024-04-02T04:42:49.537 INFO:tasks.workunit.client.0.smithi016.stdout:9/831: fsync d0/d4a/d75/d8d/f94 0 2024-04-02T04:42:49.539 INFO:tasks.workunit.client.0.smithi016.stdout:9/832: dread - d0/d3/d97/dde/ff1 zero size 2024-04-02T04:42:49.539 INFO:tasks.workunit.client.0.smithi016.stdout:9/833: chown d0/d3/d97 5 1 2024-04-02T04:42:49.540 INFO:tasks.workunit.client.0.smithi016.stdout:9/834: chown d0/d9/db/dc9/d53/c91 89317 1 2024-04-02T04:42:49.541 INFO:tasks.workunit.client.1.smithi027.stdout:0/271: symlink d4/d21/d28/l59 0 2024-04-02T04:42:49.541 INFO:tasks.workunit.client.1.smithi027.stdout:0/272: truncate d4/d3d/d49/f50 981195 0 2024-04-02T04:42:49.541 INFO:tasks.workunit.client.1.smithi027.stdout:0/273: dread - d4/d16/d1b/f3e zero size 2024-04-02T04:42:49.544 INFO:tasks.workunit.client.0.smithi016.stdout:8/748: unlink d1/d6/d36/d3f/d40/d49/l58 0 2024-04-02T04:42:49.545 INFO:tasks.workunit.client.0.smithi016.stdout:8/749: chown d1/d6/d36/d3f/d40/d49/f43 32152 1 2024-04-02T04:42:49.545 INFO:tasks.workunit.client.0.smithi016.stdout:1/738: creat d14/d19/ff1 x:0 0 0 2024-04-02T04:42:49.548 INFO:tasks.workunit.client.0.smithi016.stdout:7/739: creat d0/d46/da2/dc8/dda/f102 x:0 0 0 2024-04-02T04:42:49.549 INFO:tasks.workunit.client.0.smithi016.stdout:2/828: fsync d12/d35/d3b/d42/d8d/fa6 0 2024-04-02T04:42:49.549 INFO:tasks.workunit.client.0.smithi016.stdout:7/740: chown d0/d46/d52/fc4 95 1 2024-04-02T04:42:49.551 INFO:tasks.workunit.client.1.smithi027.stdout:7/247: creat d2/d7/da/dc/d16/f4a x:0 0 0 2024-04-02T04:42:49.552 INFO:tasks.workunit.client.1.smithi027.stdout:7/248: readlink d2/d1d/l2e 0 2024-04-02T04:42:49.553 INFO:tasks.workunit.client.0.smithi016.stdout:1/739: dwrite d14/d19/d1d/d45/d4f/d86/fa6 [0,4194304] 0 2024-04-02T04:42:49.555 INFO:tasks.workunit.client.0.smithi016.stdout:7/741: dread d0/d46/f2c [0,4194304] 0 2024-04-02T04:42:49.556 INFO:tasks.workunit.client.0.smithi016.stdout:3/860: mkdir d2/d2e/d36/d48/da2/d120 0 2024-04-02T04:42:49.558 INFO:tasks.workunit.client.1.smithi027.stdout:4/211: truncate fc 1009515 0 2024-04-02T04:42:49.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:49 smithi016 ceph-mon[28969]: pgmap v23: 65 pgs: 65 active+clean; 1.6 GiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 50 MiB/s rd, 173 MiB/s wr, 353 op/s 2024-04-02T04:42:49.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:49 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:49.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:49 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:49.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:49 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-02T04:42:49.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:49 smithi016 ceph-mon[28969]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-02T04:42:49.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:49 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:42:49.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:49 smithi016 ceph-mon[28969]: Upgrade: Need to upgrade myself (mgr.smithi027.kjzcsc) 2024-04-02T04:42:49.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:49 smithi016 ceph-mon[28969]: Upgrade: Need to upgrade myself (mgr.smithi027.kjzcsc) 2024-04-02T04:42:49.562 INFO:tasks.workunit.client.1.smithi027.stdout:2/272: mkdir d1/db/d57 0 2024-04-02T04:42:49.564 INFO:tasks.workunit.client.1.smithi027.stdout:2/273: dread - d1/db/d3b/d17/d2b/f53 zero size 2024-04-02T04:42:49.564 INFO:tasks.workunit.client.0.smithi016.stdout:4/766: rename d5/d8/ff to d5/d17/d76/d7e/fe9 0 2024-04-02T04:42:49.565 INFO:tasks.workunit.client.1.smithi027.stdout:0/274: chown d4/d16/d1b/c31 54 1 2024-04-02T04:42:49.568 INFO:tasks.workunit.client.0.smithi016.stdout:0/733: symlink d0/d16/d24/deb/df5/lf7 0 2024-04-02T04:42:49.574 INFO:tasks.workunit.client.0.smithi016.stdout:7/742: rmdir d0/d46/d37/dbe/dae/df8/da9 39 2024-04-02T04:42:49.574 INFO:tasks.workunit.client.1.smithi027.stdout:3/177: sync 2024-04-02T04:42:49.575 INFO:tasks.workunit.client.0.smithi016.stdout:1/740: dwrite d14/f48 [0,4194304] 0 2024-04-02T04:42:49.575 INFO:tasks.workunit.client.1.smithi027.stdout:1/217: rmdir d0/d1e/d25/d2b/d47 0 2024-04-02T04:42:49.577 INFO:tasks.workunit.client.1.smithi027.stdout:4/212: dread f16 [0,4194304] 0 2024-04-02T04:42:49.580 INFO:tasks.workunit.client.0.smithi016.stdout:9/835: rename d0/l55 to d0/d4e/d50/d63/d6e/d4b/d58/l102 0 2024-04-02T04:42:49.582 INFO:tasks.workunit.client.1.smithi027.stdout:0/275: rename d4/f1d to d4/d16/d53/f5a 0 2024-04-02T04:42:49.583 INFO:tasks.workunit.client.0.smithi016.stdout:4/767: dread - d5/d17/d76/f8b zero size 2024-04-02T04:42:49.585 INFO:tasks.workunit.client.1.smithi027.stdout:8/231: truncate d5/d7/f24 5198934 0 2024-04-02T04:42:49.588 INFO:tasks.workunit.client.1.smithi027.stdout:7/249: link d2/d7/da/dc/d16/d2f/f27 d2/d1c/f4b 0 2024-04-02T04:42:49.588 INFO:tasks.workunit.client.1.smithi027.stdout:5/334: dwrite d0/d1a/d20/f73 [0,4194304] 0 2024-04-02T04:42:49.589 INFO:tasks.workunit.client.1.smithi027.stdout:5/335: chown d0/d68 1083 1 2024-04-02T04:42:49.590 INFO:tasks.workunit.client.1.smithi027.stdout:9/262: truncate d1a/f57 254998 0 2024-04-02T04:42:49.595 INFO:tasks.workunit.client.0.smithi016.stdout:2/829: rename d12/d35/d3b/d42/f112 to d12/d35/d3f/d2b/de9/d102/d103/f11d 0 2024-04-02T04:42:49.600 INFO:tasks.workunit.client.1.smithi027.stdout:2/274: mkdir d1/db/d3b/d58 0 2024-04-02T04:42:49.601 INFO:tasks.workunit.client.0.smithi016.stdout:2/830: dwrite d12/d35/d3f/f61 [0,4194304] 0 2024-04-02T04:42:49.601 INFO:tasks.workunit.client.1.smithi027.stdout:2/275: write d1/db/d3b/d17/d12/f38 [915682,6231] 0 2024-04-02T04:42:49.602 INFO:tasks.workunit.client.0.smithi016.stdout:4/768: rmdir d5/d16/d88/db0 39 2024-04-02T04:42:49.603 INFO:tasks.workunit.client.1.smithi027.stdout:4/213: creat d24/d38/f3d x:0 0 0 2024-04-02T04:42:49.604 INFO:tasks.workunit.client.1.smithi027.stdout:4/214: fsync d24/d2d/f31 0 2024-04-02T04:42:49.605 INFO:tasks.workunit.client.1.smithi027.stdout:4/215: dread - d24/f27 zero size 2024-04-02T04:42:49.609 INFO:tasks.workunit.client.0.smithi016.stdout:0/734: write d0/fc [141226,50093] 0 2024-04-02T04:42:49.611 INFO:tasks.workunit.client.1.smithi027.stdout:0/276: truncate d4/d16/d2b/f18 1232024 0 2024-04-02T04:42:49.615 INFO:tasks.workunit.client.0.smithi016.stdout:9/836: write d0/d9/db/d54/fb3 [713379,99991] 0 2024-04-02T04:42:49.617 INFO:tasks.workunit.client.1.smithi027.stdout:7/250: dread d2/d7/da/f1e [0,4194304] 0 2024-04-02T04:42:49.618 INFO:tasks.workunit.client.1.smithi027.stdout:8/232: dread d5/d8/f17 [0,4194304] 0 2024-04-02T04:42:49.619 INFO:tasks.workunit.client.1.smithi027.stdout:5/336: rename d0/df/l78 to d0/df/d11/d37/l7d 0 2024-04-02T04:42:49.623 INFO:tasks.workunit.client.0.smithi016.stdout:7/743: link d0/d46/d16/f62 d0/d46/d37/dbe/dae/df8/d84/d8a/f103 0 2024-04-02T04:42:49.627 INFO:tasks.workunit.client.1.smithi027.stdout:3/178: symlink d1/l37 0 2024-04-02T04:42:49.628 INFO:tasks.workunit.client.0.smithi016.stdout:6/813: sync 2024-04-02T04:42:49.628 INFO:tasks.workunit.client.0.smithi016.stdout:5/736: sync 2024-04-02T04:42:49.629 INFO:tasks.workunit.client.1.smithi027.stdout:1/218: symlink d0/l4e 0 2024-04-02T04:42:49.633 INFO:tasks.workunit.client.1.smithi027.stdout:2/276: rmdir d1 39 2024-04-02T04:42:49.635 INFO:tasks.workunit.client.1.smithi027.stdout:4/216: creat d24/d38/f3e x:0 0 0 2024-04-02T04:42:49.635 INFO:tasks.workunit.client.1.smithi027.stdout:4/217: chown d24/f2f 11951035 1 2024-04-02T04:42:49.636 INFO:tasks.workunit.client.1.smithi027.stdout:4/218: write f11 [854310,73284] 0 2024-04-02T04:42:49.637 INFO:tasks.workunit.client.0.smithi016.stdout:6/814: dwrite d0/d9/d16/d56/d59/da1/fcf [0,4194304] 0 2024-04-02T04:42:49.637 INFO:tasks.workunit.client.0.smithi016.stdout:4/769: write d5/d17/d76/fe3 [4344511,107470] 0 2024-04-02T04:42:49.637 INFO:tasks.workunit.client.0.smithi016.stdout:0/735: dwrite d0/d4f/fa2 [0,4194304] 0 2024-04-02T04:42:49.641 INFO:tasks.workunit.client.0.smithi016.stdout:9/837: mknod d0/d4a/d75/c103 0 2024-04-02T04:42:49.643 INFO:tasks.workunit.client.0.smithi016.stdout:9/838: dread - d0/d3/d97/ff7 zero size 2024-04-02T04:42:49.644 INFO:tasks.workunit.client.1.smithi027.stdout:9/263: creat d1a/d1d/d3c/f5e x:0 0 0 2024-04-02T04:42:49.645 INFO:tasks.workunit.client.0.smithi016.stdout:9/839: write d0/d9/d4d/da8/fb8 [1520606,115076] 0 2024-04-02T04:42:49.647 INFO:tasks.workunit.client.0.smithi016.stdout:2/831: link d12/d35/d3f/f34 d12/d35/d3f/d2b/de9/d102/f11e 0 2024-04-02T04:42:49.647 INFO:tasks.workunit.client.0.smithi016.stdout:9/840: dread - d0/d9/db/d54/ffb zero size 2024-04-02T04:42:49.649 INFO:tasks.workunit.client.0.smithi016.stdout:2/832: fsync d12/d35/fca 0 2024-04-02T04:42:49.649 INFO:tasks.workunit.client.1.smithi027.stdout:3/179: dread d1/d1a/f1d [0,4194304] 0 2024-04-02T04:42:49.650 INFO:tasks.workunit.client.0.smithi016.stdout:0/736: stat d0/d16/d2e/d36 0 2024-04-02T04:42:49.651 INFO:tasks.workunit.client.1.smithi027.stdout:3/180: dread - d1/de/df/f24 zero size 2024-04-02T04:42:49.652 INFO:tasks.workunit.client.1.smithi027.stdout:2/277: stat d1/db/d57 0 2024-04-02T04:42:49.652 INFO:tasks.workunit.client.1.smithi027.stdout:7/251: dread d2/d7/da/dc/f30 [0,4194304] 0 2024-04-02T04:42:49.653 INFO:tasks.workunit.client.0.smithi016.stdout:6/815: mkdir d0/d9/d16/d56/d89/dfa 0 2024-04-02T04:42:49.656 INFO:tasks.workunit.client.0.smithi016.stdout:7/744: dread d0/f22 [0,4194304] 0 2024-04-02T04:42:49.657 INFO:tasks.workunit.client.0.smithi016.stdout:3/861: rename d2/d2e/d36/c3b to d2/d2e/d36/d48/da2/c121 0 2024-04-02T04:42:49.657 INFO:tasks.workunit.client.1.smithi027.stdout:6/239: sync 2024-04-02T04:42:49.657 INFO:tasks.workunit.client.0.smithi016.stdout:7/745: write d0/d46/d37/faa [4916456,10369] 0 2024-04-02T04:42:49.658 INFO:tasks.workunit.client.0.smithi016.stdout:7/746: stat d0/d46/d52/d82/lc0 0 2024-04-02T04:42:49.659 INFO:tasks.workunit.client.0.smithi016.stdout:7/747: chown d0/d46/d16/dc2/ff2 4091 1 2024-04-02T04:42:49.661 INFO:tasks.workunit.client.1.smithi027.stdout:8/233: write d5/d8/db/dd/f1d [1494873,95798] 0 2024-04-02T04:42:49.661 INFO:tasks.workunit.client.0.smithi016.stdout:7/748: write d0/d46/d37/dbe/dae/df8/d84/fd0 [700610,59227] 0 2024-04-02T04:42:49.661 INFO:tasks.workunit.client.0.smithi016.stdout:4/770: read - d5/d1d/d6c/f72 zero size 2024-04-02T04:42:49.662 INFO:tasks.workunit.client.1.smithi027.stdout:6/240: write d13/d1b/d24/d28/f35 [3580466,118376] 0 2024-04-02T04:42:49.663 INFO:tasks.workunit.client.1.smithi027.stdout:0/277: dwrite d4/d3d/f3f [0,4194304] 0 2024-04-02T04:42:49.664 INFO:tasks.workunit.client.0.smithi016.stdout:2/833: mkdir d12/d35/d3f/d2b/da3/db1/d11f 0 2024-04-02T04:42:49.664 INFO:tasks.workunit.client.1.smithi027.stdout:7/252: dwrite d2/d7/da/dc/d1f/d33/d36/f45 [0,4194304] 0 2024-04-02T04:42:49.665 INFO:tasks.workunit.client.0.smithi016.stdout:2/834: chown d12/d70/fe8 50385 1 2024-04-02T04:42:49.665 INFO:tasks.workunit.client.1.smithi027.stdout:0/278: fsync d4/d21/d28/f48 0 2024-04-02T04:42:49.665 INFO:tasks.workunit.client.1.smithi027.stdout:8/234: chown d5/d8/d15/f37 420708 1 2024-04-02T04:42:49.666 INFO:tasks.workunit.client.0.smithi016.stdout:2/835: write d12/f96 [389378,123024] 0 2024-04-02T04:42:49.666 INFO:tasks.workunit.client.0.smithi016.stdout:0/737: rmdir d0/d16/d24/d62/d9c 39 2024-04-02T04:42:49.667 INFO:tasks.workunit.client.0.smithi016.stdout:0/738: write d0/fe [3294163,1018] 0 2024-04-02T04:42:49.668 INFO:tasks.workunit.client.1.smithi027.stdout:9/264: mknod d1a/d1d/d29/d37/d4c/c5f 0 2024-04-02T04:42:49.668 INFO:tasks.workunit.client.0.smithi016.stdout:0/739: stat d0/d4f/daf/fb1 0 2024-04-02T04:42:49.669 INFO:tasks.workunit.client.1.smithi027.stdout:7/253: stat d2/d1d/f25 0 2024-04-02T04:42:49.670 INFO:tasks.workunit.client.0.smithi016.stdout:0/740: write d0/d16/f80 [878953,105508] 0 2024-04-02T04:42:49.671 INFO:tasks.workunit.client.1.smithi027.stdout:0/279: write d4/d21/d28/f48 [444098,52058] 0 2024-04-02T04:42:49.672 INFO:tasks.workunit.client.0.smithi016.stdout:0/741: readlink d0/d16/d2e/d37/l81 0 2024-04-02T04:42:49.672 INFO:tasks.workunit.client.1.smithi027.stdout:0/280: chown d4/d16/d2b/l26 20128060 1 2024-04-02T04:42:49.677 INFO:tasks.workunit.client.1.smithi027.stdout:0/281: dread d4/d16/d1b/f20 [0,4194304] 0 2024-04-02T04:42:49.679 INFO:tasks.workunit.client.0.smithi016.stdout:9/841: rename d0/d4a/d99/f37 to d0/d9/d4d/d78/da6/dc1/f104 0 2024-04-02T04:42:49.679 INFO:tasks.workunit.client.1.smithi027.stdout:0/282: read d4/f9 [1330376,39858] 0 2024-04-02T04:42:49.681 INFO:tasks.workunit.client.1.smithi027.stdout:5/337: write d0/df/d2f/d3b/f3d [4022485,78483] 0 2024-04-02T04:42:49.685 INFO:tasks.workunit.client.1.smithi027.stdout:5/338: dread d0/df/d2f/d3b/f64 [0,4194304] 0 2024-04-02T04:42:49.688 INFO:tasks.workunit.client.0.smithi016.stdout:7/749: mkdir d0/d21/d104 0 2024-04-02T04:42:49.689 INFO:tasks.workunit.client.1.smithi027.stdout:8/235: mknod d5/d8/d15/c4b 0 2024-04-02T04:42:49.690 INFO:tasks.workunit.client.0.smithi016.stdout:4/771: symlink d5/d17/d18/d6d/d8e/d9c/dcc/d6f/dcf/lea 0 2024-04-02T04:42:49.691 INFO:tasks.workunit.client.0.smithi016.stdout:4/772: fdatasync d5/d8/fb 0 2024-04-02T04:42:49.692 INFO:tasks.workunit.client.1.smithi027.stdout:7/254: unlink d2/d1d/l2e 0 2024-04-02T04:42:49.692 INFO:tasks.workunit.client.0.smithi016.stdout:4/773: fdatasync d5/d16/d88/f8c 0 2024-04-02T04:42:49.693 INFO:tasks.workunit.client.1.smithi027.stdout:7/255: write d2/d7/da/dc/d16/f4a [991204,54395] 0 2024-04-02T04:42:49.694 INFO:tasks.workunit.client.1.smithi027.stdout:9/265: creat d1a/d1d/d29/d51/f60 x:0 0 0 2024-04-02T04:42:49.695 INFO:tasks.workunit.client.0.smithi016.stdout:7/750: dwrite d0/d46/d16/fc5 [0,4194304] 0 2024-04-02T04:42:49.698 INFO:tasks.workunit.client.0.smithi016.stdout:0/742: mknod d0/d16/dd6/cf8 0 2024-04-02T04:42:49.699 INFO:tasks.workunit.client.0.smithi016.stdout:9/842: creat d0/d4e/d50/d67/f105 x:0 0 0 2024-04-02T04:42:49.700 INFO:tasks.workunit.client.1.smithi027.stdout:4/219: creat d24/f3f x:0 0 0 2024-04-02T04:42:49.702 INFO:tasks.workunit.client.0.smithi016.stdout:2/836: dwrite d12/d35/d3f/d2b/de9/d102/d103/f11d [0,4194304] 0 2024-04-02T04:42:49.702 INFO:tasks.workunit.client.0.smithi016.stdout:5/737: getdents d3/d40/d4f/d6b/d45/da6/d102/dc4 0 2024-04-02T04:42:49.705 INFO:tasks.workunit.client.0.smithi016.stdout:4/774: dwrite d5/d1d/d2d/f29 [0,4194304] 0 2024-04-02T04:42:49.706 INFO:tasks.workunit.client.0.smithi016.stdout:7/751: dread d0/d46/f30 [4194304,4194304] 0 2024-04-02T04:42:49.708 INFO:tasks.workunit.client.0.smithi016.stdout:9/843: read d0/d9/db/dc9/d53/f73 [334693,32225] 0 2024-04-02T04:42:49.709 INFO:tasks.workunit.client.0.smithi016.stdout:8/750: sync 2024-04-02T04:42:49.709 INFO:tasks.workunit.client.0.smithi016.stdout:1/741: sync 2024-04-02T04:42:49.711 INFO:tasks.workunit.client.1.smithi027.stdout:5/339: mkdir d0/d1d/d61/d7e 0 2024-04-02T04:42:49.711 INFO:tasks.workunit.client.0.smithi016.stdout:0/743: dwrite d0/d16/d2e/d36/f8b [0,4194304] 0 2024-04-02T04:42:49.712 INFO:tasks.workunit.client.0.smithi016.stdout:0/744: chown d0/d4f/d76/l89 5776658 1 2024-04-02T04:42:49.716 INFO:tasks.workunit.client.1.smithi027.stdout:6/241: link d13/d1b/d24/d28/f2a d13/d1b/d24/d28/d3c/f53 0 2024-04-02T04:42:49.718 INFO:tasks.workunit.client.1.smithi027.stdout:6/242: fsync d13/d1b/d24/f4a 0 2024-04-02T04:42:49.718 INFO:tasks.workunit.client.1.smithi027.stdout:7/256: dread d2/df/f10 [0,4194304] 0 2024-04-02T04:42:49.721 INFO:tasks.workunit.client.0.smithi016.stdout:9/844: dwrite d0/d9/db/dc9/fdf [0,4194304] 0 2024-04-02T04:42:49.723 INFO:tasks.workunit.client.1.smithi027.stdout:9/266: creat d1a/d1d/d29/d37/d20/f61 x:0 0 0 2024-04-02T04:42:49.724 INFO:tasks.workunit.client.1.smithi027.stdout:9/267: chown d1a/l2a 111804809 1 2024-04-02T04:42:49.730 INFO:tasks.workunit.client.0.smithi016.stdout:4/775: symlink d5/d17/d18/da0/leb 0 2024-04-02T04:42:49.731 INFO:tasks.workunit.client.0.smithi016.stdout:0/745: dread d0/f19 [0,4194304] 0 2024-04-02T04:42:49.731 INFO:tasks.workunit.client.0.smithi016.stdout:4/776: fdatasync d5/d1d/f61 0 2024-04-02T04:42:49.732 INFO:tasks.workunit.client.0.smithi016.stdout:5/738: write d3/d40/d4f/d6b/d45/da6/d102/dc4/d108/f5f [3004707,104403] 0 2024-04-02T04:42:49.735 INFO:tasks.workunit.client.0.smithi016.stdout:5/739: fdatasync d3/d40/d4f/d6b/d45/da6/d102/dc4/d108/d6d/fa7 0 2024-04-02T04:42:49.736 INFO:tasks.workunit.client.0.smithi016.stdout:1/742: mknod d14/d19/d20/de2/cf2 0 2024-04-02T04:42:49.737 INFO:tasks.workunit.client.1.smithi027.stdout:3/181: link d1/d1a/f22 d1/de/f38 0 2024-04-02T04:42:49.738 INFO:tasks.workunit.client.0.smithi016.stdout:8/751: mknod d1/d6/d36/d3f/d40/d45/dcc/cfb 0 2024-04-02T04:42:49.742 INFO:tasks.workunit.client.1.smithi027.stdout:7/257: stat d2/d7/da/dc/d1f/d33/c38 0 2024-04-02T04:42:49.743 INFO:tasks.workunit.client.0.smithi016.stdout:6/816: link d0/f3f d0/d5c/ffb 0 2024-04-02T04:42:49.747 INFO:tasks.workunit.client.0.smithi016.stdout:3/862: getdents d2/d15/d16/d8a/dcd 0 2024-04-02T04:42:49.749 INFO:tasks.workunit.client.0.smithi016.stdout:2/837: unlink d12/d35/d3b/d42/d59/d91/d10a/l11a 0 2024-04-02T04:42:49.752 INFO:tasks.workunit.client.1.smithi027.stdout:5/340: mknod d0/d1d/d61/d6d/c7f 0 2024-04-02T04:42:49.753 INFO:tasks.workunit.client.0.smithi016.stdout:9/845: dread d0/d4e/d50/f85 [0,4194304] 0 2024-04-02T04:42:49.754 INFO:tasks.workunit.client.0.smithi016.stdout:9/846: stat d0/d4a/d99/l49 0 2024-04-02T04:42:49.754 INFO:tasks.workunit.client.0.smithi016.stdout:9/847: chown d0/d9/db/lf 848037 1 2024-04-02T04:42:49.757 INFO:tasks.workunit.client.1.smithi027.stdout:8/236: dwrite d5/d7/f24 [0,4194304] 0 2024-04-02T04:42:49.760 INFO:tasks.workunit.client.0.smithi016.stdout:2/838: dread d12/d35/d3f/d56/dab/f3c [0,4194304] 0 2024-04-02T04:42:49.760 INFO:tasks.workunit.client.1.smithi027.stdout:6/243: mkdir d13/d1b/d24/d54 0 2024-04-02T04:42:49.761 INFO:tasks.workunit.client.1.smithi027.stdout:9/268: write f12 [3027740,126421] 0 2024-04-02T04:42:49.762 INFO:tasks.workunit.client.1.smithi027.stdout:8/237: write d5/d8/d15/f37 [267502,32104] 0 2024-04-02T04:42:49.762 INFO:tasks.workunit.client.1.smithi027.stdout:5/341: dwrite d0/d1d/d61/f62 [0,4194304] 0 2024-04-02T04:42:49.765 INFO:tasks.workunit.client.0.smithi016.stdout:5/740: creat d3/d40/d4f/d6b/d63/f10b x:0 0 0 2024-04-02T04:42:49.768 INFO:tasks.workunit.client.0.smithi016.stdout:1/743: rename d14/d19/d1e/c53 to d14/d19/d7a/da2/cf3 0 2024-04-02T04:42:49.770 INFO:tasks.workunit.client.0.smithi016.stdout:7/752: dwrite d0/d87/fb1 [0,4194304] 0 2024-04-02T04:42:49.770 INFO:tasks.workunit.client.1.smithi027.stdout:4/220: rename l20 to d24/l40 0 2024-04-02T04:42:49.770 INFO:tasks.workunit.client.1.smithi027.stdout:8/238: dwrite d5/d7/d27/d31/f32 [0,4194304] 0 2024-04-02T04:42:49.772 INFO:tasks.workunit.client.0.smithi016.stdout:7/753: stat d0/d46/d37/dbe/dae/df8/la6 0 2024-04-02T04:42:49.773 INFO:tasks.workunit.client.1.smithi027.stdout:0/283: getdents d4/d16/d2b/d2d/d4c 0 2024-04-02T04:42:49.776 INFO:tasks.workunit.client.0.smithi016.stdout:6/817: creat d0/d9/d16/d56/d89/db4/ffc x:0 0 0 2024-04-02T04:42:49.777 INFO:tasks.workunit.client.1.smithi027.stdout:7/258: write d2/d7/da/dc/d16/d2f/f27 [342761,116106] 0 2024-04-02T04:42:49.778 INFO:tasks.workunit.client.1.smithi027.stdout:3/182: dwrite d1/f10 [0,4194304] 0 2024-04-02T04:42:49.779 INFO:tasks.workunit.client.0.smithi016.stdout:3/863: mknod d2/d15/d1a/d45/da5/c122 0 2024-04-02T04:42:49.782 INFO:tasks.workunit.client.1.smithi027.stdout:9/269: readlink l6 0 2024-04-02T04:42:49.782 INFO:tasks.workunit.client.0.smithi016.stdout:0/746: symlink d0/d16/d24/d62/d9c/lf9 0 2024-04-02T04:42:49.784 INFO:tasks.workunit.client.1.smithi027.stdout:9/270: chown d1a/d1d/d29/d37/d20/f3f 845 1 2024-04-02T04:42:49.792 INFO:tasks.workunit.client.0.smithi016.stdout:5/741: dread - d3/d40/d4f/d67/fa0 zero size 2024-04-02T04:42:49.793 INFO:tasks.workunit.client.1.smithi027.stdout:8/239: creat d5/d8/db/df/f4c x:0 0 0 2024-04-02T04:42:49.794 INFO:tasks.workunit.client.1.smithi027.stdout:8/240: stat d5/d7/d27/l3d 0 2024-04-02T04:42:49.796 INFO:tasks.workunit.client.0.smithi016.stdout:8/752: fsync d1/f15 0 2024-04-02T04:42:49.799 INFO:tasks.workunit.client.0.smithi016.stdout:7/754: symlink d0/d87/l105 0 2024-04-02T04:42:49.800 INFO:tasks.workunit.client.1.smithi027.stdout:3/183: symlink d1/de/df/l39 0 2024-04-02T04:42:49.800 INFO:tasks.workunit.client.1.smithi027.stdout:3/184: readlink d1/d1a/l28 0 2024-04-02T04:42:49.801 INFO:tasks.workunit.client.1.smithi027.stdout:9/271: rename d1a/d1d/f27 to d1a/d1d/d29/d37/d4c/f62 0 2024-04-02T04:42:49.803 INFO:tasks.workunit.client.0.smithi016.stdout:3/864: rename d2/d15/d16/d8a/dd1/d8b/f9a to d2/d2e/d36/d66/f123 0 2024-04-02T04:42:49.805 INFO:tasks.workunit.client.1.smithi027.stdout:4/221: mkdir d24/d2d/d34/d41 0 2024-04-02T04:42:49.808 INFO:tasks.workunit.client.1.smithi027.stdout:8/241: symlink d5/d8/l4d 0 2024-04-02T04:42:49.810 INFO:tasks.workunit.client.1.smithi027.stdout:8/242: write d5/d8/db/df/f22 [5019255,1462] 0 2024-04-02T04:42:49.811 INFO:tasks.workunit.client.0.smithi016.stdout:9/848: write d0/d9/f19 [964153,45449] 0 2024-04-02T04:42:49.811 INFO:tasks.workunit.client.0.smithi016.stdout:4/777: truncate d5/d17/d18/d25/f3b 2468130 0 2024-04-02T04:42:49.811 INFO:tasks.workunit.client.0.smithi016.stdout:9/849: readlink d0/d9/db/d54/l80 0 2024-04-02T04:42:49.812 INFO:tasks.workunit.client.0.smithi016.stdout:8/753: mkdir d1/d6/d36/d3f/d40/d49/d4a/dfc 0 2024-04-02T04:42:49.813 INFO:tasks.workunit.client.0.smithi016.stdout:2/839: write d12/d35/d3b/f83 [4895265,37728] 0 2024-04-02T04:42:49.813 INFO:tasks.workunit.client.1.smithi027.stdout:3/185: mkdir d1/de/d3a 0 2024-04-02T04:42:49.816 INFO:tasks.workunit.client.1.smithi027.stdout:9/272: unlink d1a/l2a 0 2024-04-02T04:42:49.819 INFO:tasks.workunit.client.0.smithi016.stdout:5/742: dread d3/df/d18/f42 [0,4194304] 0 2024-04-02T04:42:49.819 INFO:tasks.workunit.client.1.smithi027.stdout:5/342: creat d0/df/f80 x:0 0 0 2024-04-02T04:42:49.819 INFO:tasks.workunit.client.1.smithi027.stdout:7/259: write f1 [419992,24858] 0 2024-04-02T04:42:49.820 INFO:tasks.workunit.client.1.smithi027.stdout:3/186: dread d1/de/df/d13/f21 [0,4194304] 0 2024-04-02T04:42:49.821 INFO:tasks.workunit.client.1.smithi027.stdout:3/187: readlink d1/l37 0 2024-04-02T04:42:49.821 INFO:tasks.workunit.client.1.smithi027.stdout:7/260: stat d2/d7/da/dc/d16/f2d 0 2024-04-02T04:42:49.822 INFO:tasks.workunit.client.1.smithi027.stdout:4/222: mknod d24/d38/c42 0 2024-04-02T04:42:49.824 INFO:tasks.workunit.client.0.smithi016.stdout:0/747: rename d0/d4f/d76/fa6 to d0/d16/d23/d3e/ffa 0 2024-04-02T04:42:49.826 INFO:tasks.workunit.client.1.smithi027.stdout:4/223: dread f12 [0,4194304] 0 2024-04-02T04:42:49.827 INFO:tasks.workunit.client.1.smithi027.stdout:0/284: creat d4/d16/d2b/f5b x:0 0 0 2024-04-02T04:42:49.827 INFO:tasks.workunit.client.1.smithi027.stdout:0/285: fdatasync d4/d16/d2b/f24 0 2024-04-02T04:42:49.828 INFO:tasks.workunit.client.0.smithi016.stdout:3/865: creat d2/d15/d1a/d45/da5/db6/f124 x:0 0 0 2024-04-02T04:42:49.829 INFO:tasks.workunit.client.0.smithi016.stdout:3/866: fsync d2/fa 0 2024-04-02T04:42:49.829 INFO:tasks.workunit.client.1.smithi027.stdout:6/244: truncate d13/d1b/d24/f41 3826762 0 2024-04-02T04:42:49.829 INFO:tasks.workunit.client.1.smithi027.stdout:9/273: dread f15 [0,4194304] 0 2024-04-02T04:42:49.831 INFO:tasks.workunit.client.1.smithi027.stdout:8/243: mknod d5/d7/d27/d31/c4e 0 2024-04-02T04:42:49.832 INFO:tasks.workunit.client.1.smithi027.stdout:8/244: fsync d5/d7/f2b 0 2024-04-02T04:42:49.835 INFO:tasks.workunit.client.0.smithi016.stdout:4/778: stat d5/d17/d18/c7c 0 2024-04-02T04:42:49.835 INFO:tasks.workunit.client.1.smithi027.stdout:2/278: sync 2024-04-02T04:42:49.835 INFO:tasks.workunit.client.1.smithi027.stdout:1/219: sync 2024-04-02T04:42:49.840 INFO:tasks.workunit.client.0.smithi016.stdout:8/754: creat d1/d6/d36/d3f/d40/d63/dc5/ffd x:0 0 0 2024-04-02T04:42:49.843 INFO:tasks.workunit.client.0.smithi016.stdout:7/755: mkdir d0/d46/d16/d49/df9/d106 0 2024-04-02T04:42:49.845 INFO:tasks.workunit.client.0.smithi016.stdout:6/818: link d0/d78/fa0 d0/d9/d16/ffd 0 2024-04-02T04:42:49.846 INFO:tasks.workunit.client.1.smithi027.stdout:0/286: creat d4/d16/d2b/d2d/d4c/f5c x:0 0 0 2024-04-02T04:42:49.847 INFO:tasks.workunit.client.1.smithi027.stdout:0/287: chown d4/c5 0 1 2024-04-02T04:42:49.847 INFO:tasks.workunit.client.1.smithi027.stdout:0/288: readlink d4/d16/d1b/l43 0 2024-04-02T04:42:49.849 INFO:tasks.workunit.client.1.smithi027.stdout:6/245: read f7 [676794,122154] 0 2024-04-02T04:42:49.854 INFO:tasks.workunit.client.0.smithi016.stdout:3/867: rmdir d2/d2e/d36/d66/d90 39 2024-04-02T04:42:49.856 INFO:tasks.workunit.client.1.smithi027.stdout:8/245: creat d5/d8/db/dd/f4f x:0 0 0 2024-04-02T04:42:49.857 INFO:tasks.workunit.client.0.smithi016.stdout:1/744: link d14/da9/f9b d14/d19/d1e/ff4 0 2024-04-02T04:42:49.859 INFO:tasks.workunit.client.0.smithi016.stdout:4/779: write d5/d17/d18/d6d/d8e/d9c/dcc/d6f/f84 [2462437,59753] 0 2024-04-02T04:42:49.860 INFO:tasks.workunit.client.0.smithi016.stdout:4/780: write d5/d16/d3f/fc4 [883987,65969] 0 2024-04-02T04:42:49.866 INFO:tasks.workunit.client.0.smithi016.stdout:2/840: mknod d12/d35/d101/d76/da0/c120 0 2024-04-02T04:42:49.869 INFO:tasks.workunit.client.0.smithi016.stdout:9/850: truncate d0/d4a/d99/d46/fb7 1498147 0 2024-04-02T04:42:49.869 INFO:tasks.workunit.client.0.smithi016.stdout:6/819: symlink d0/d9/d16/d56/d89/db4/lfe 0 2024-04-02T04:42:49.870 INFO:tasks.workunit.client.0.smithi016.stdout:6/820: fsync d0/d9/d16/d46/dbe/fd1 0 2024-04-02T04:42:49.875 INFO:tasks.workunit.client.1.smithi027.stdout:2/279: symlink d1/db/l59 0 2024-04-02T04:42:49.875 INFO:tasks.workunit.client.1.smithi027.stdout:1/220: mkdir d0/d1e/d2e/d3b/d4f 0 2024-04-02T04:42:49.876 INFO:tasks.workunit.client.1.smithi027.stdout:5/343: creat d0/d1d/d61/d72/f81 x:0 0 0 2024-04-02T04:42:49.878 INFO:tasks.workunit.client.1.smithi027.stdout:3/188: truncate d1/d1a/f22 3297240 0 2024-04-02T04:42:49.881 INFO:tasks.workunit.client.1.smithi027.stdout:8/246: dread d5/d8/f17 [0,4194304] 0 2024-04-02T04:42:49.884 INFO:tasks.workunit.client.1.smithi027.stdout:8/247: stat d5/d8/d48 0 2024-04-02T04:42:49.886 INFO:tasks.workunit.client.0.smithi016.stdout:5/743: fdatasync d3/d40/d4f/d6b/d45/da6/d102/ff6 0 2024-04-02T04:42:49.887 INFO:tasks.workunit.client.1.smithi027.stdout:9/274: dwrite d1a/d1d/d29/d37/f1c [4194304,4194304] 0 2024-04-02T04:42:49.888 INFO:tasks.workunit.client.1.smithi027.stdout:7/261: creat d2/f4c x:0 0 0 2024-04-02T04:42:49.888 INFO:tasks.workunit.client.0.smithi016.stdout:2/841: stat d12/d35/d3b/d42/d67/c7c 0 2024-04-02T04:42:49.889 INFO:tasks.workunit.client.0.smithi016.stdout:0/748: fsync d0/d16/d24/d4e/d8d/d71/df1/fd2 0 2024-04-02T04:42:49.889 INFO:tasks.workunit.client.0.smithi016.stdout:1/745: read d14/d19/d1e/f5d [1067346,33267] 0 2024-04-02T04:42:49.889 INFO:tasks.workunit.client.0.smithi016.stdout:0/749: stat d0/d16/d23/f3f 0 2024-04-02T04:42:49.889 INFO:tasks.workunit.client.0.smithi016.stdout:9/851: rename d0/d4e/d50/d63/dd2/l9e to d0/d4e/d50/d63/d6e/d4b/l106 0 2024-04-02T04:42:49.890 INFO:tasks.workunit.client.1.smithi027.stdout:1/221: dwrite d0/fd [0,4194304] 0 2024-04-02T04:42:49.891 INFO:tasks.workunit.client.1.smithi027.stdout:7/262: fdatasync d2/d7/da/dc/d16/d2f/f41 0 2024-04-02T04:42:49.892 INFO:tasks.workunit.client.1.smithi027.stdout:2/280: dwrite d1/db/d3b/d17/d12/f1f [0,4194304] 0 2024-04-02T04:42:49.893 INFO:tasks.workunit.client.0.smithi016.stdout:8/755: write d1/d6/d36/d8e/fdb [371732,41534] 0 2024-04-02T04:42:49.894 INFO:tasks.workunit.client.1.smithi027.stdout:2/281: dread - d1/db/d3b/d17/f3f zero size 2024-04-02T04:42:49.895 INFO:tasks.workunit.client.1.smithi027.stdout:3/189: mknod d1/d1a/c3b 0 2024-04-02T04:42:49.899 INFO:tasks.workunit.client.1.smithi027.stdout:3/190: truncate d1/de/df/d13/f1b 784822 0 2024-04-02T04:42:49.899 INFO:tasks.workunit.client.0.smithi016.stdout:5/744: rmdir d3/df 39 2024-04-02T04:42:49.901 INFO:tasks.workunit.client.1.smithi027.stdout:3/191: dread - d1/f2e zero size 2024-04-02T04:42:49.903 INFO:tasks.workunit.client.1.smithi027.stdout:0/289: unlink d4/d16/d1b/l1f 0 2024-04-02T04:42:49.904 INFO:tasks.workunit.client.1.smithi027.stdout:5/344: rename d0/d1d/c57 to d0/df/d76/c82 0 2024-04-02T04:42:49.906 INFO:tasks.workunit.client.0.smithi016.stdout:7/756: link d0/d46/d16/dc2/dea/c6d d0/d46/d52/d82/c107 0 2024-04-02T04:42:49.906 INFO:tasks.workunit.client.1.smithi027.stdout:2/282: dwrite d1/db/d3b/d17/d12/f38 [0,4194304] 0 2024-04-02T04:42:49.908 INFO:tasks.workunit.client.1.smithi027.stdout:5/345: read d0/f47 [4015571,68920] 0 2024-04-02T04:42:49.909 INFO:tasks.workunit.client.1.smithi027.stdout:8/248: creat d5/d7/d27/f50 x:0 0 0 2024-04-02T04:42:49.910 INFO:tasks.workunit.client.0.smithi016.stdout:2/842: fsync d12/f7b 0 2024-04-02T04:42:49.911 INFO:tasks.workunit.client.0.smithi016.stdout:2/843: stat d12/d9f 0 2024-04-02T04:42:49.911 INFO:tasks.workunit.client.0.smithi016.stdout:2/844: stat d12/d70/d11c 0 2024-04-02T04:42:49.912 INFO:tasks.workunit.client.0.smithi016.stdout:0/750: mkdir d0/d16/d24/d62/d9c/dfb 0 2024-04-02T04:42:49.913 INFO:tasks.workunit.client.0.smithi016.stdout:0/751: chown d0/f32 3140 1 2024-04-02T04:42:49.915 INFO:tasks.workunit.client.0.smithi016.stdout:1/746: mknod d14/d19/d1d/d29/d39/d46/dcb/d43/d68/cf5 0 2024-04-02T04:42:49.915 INFO:tasks.workunit.client.1.smithi027.stdout:1/222: creat d0/d1e/d25/f50 x:0 0 0 2024-04-02T04:42:49.916 INFO:tasks.workunit.client.0.smithi016.stdout:9/852: mknod d0/d33/c107 0 2024-04-02T04:42:49.920 INFO:tasks.workunit.client.0.smithi016.stdout:5/745: creat d3/d40/db6/de0/f10c x:0 0 0 2024-04-02T04:42:49.925 INFO:tasks.workunit.client.0.smithi016.stdout:0/752: creat d0/d16/d24/deb/ffc x:0 0 0 2024-04-02T04:42:49.926 INFO:tasks.workunit.client.1.smithi027.stdout:8/249: creat d5/d8/db/dd/f51 x:0 0 0 2024-04-02T04:42:49.927 INFO:tasks.workunit.client.1.smithi027.stdout:1/223: write d0/d1e/d25/f31 [3834152,57730] 0 2024-04-02T04:42:49.928 INFO:tasks.workunit.client.1.smithi027.stdout:8/250: write d5/d8/db/dd/f10 [1933705,67114] 0 2024-04-02T04:42:49.928 INFO:tasks.workunit.client.1.smithi027.stdout:7/263: mkdir d2/d7/da/dc/d16/d3a/d4d 0 2024-04-02T04:42:49.929 INFO:tasks.workunit.client.0.smithi016.stdout:5/746: dread - d3/d5d/f103 zero size 2024-04-02T04:42:49.929 INFO:tasks.workunit.client.1.smithi027.stdout:8/251: stat d5/d8/d15/f37 0 2024-04-02T04:42:49.930 INFO:tasks.workunit.client.0.smithi016.stdout:6/821: link d0/d5c/f6b d0/d5c/fff 0 2024-04-02T04:42:49.932 INFO:tasks.workunit.client.1.smithi027.stdout:0/290: mkdir d4/d5d 0 2024-04-02T04:42:49.932 INFO:tasks.workunit.client.1.smithi027.stdout:7/264: truncate d2/d7/da/dc/d1f/f47 266976 0 2024-04-02T04:42:49.934 INFO:tasks.workunit.client.1.smithi027.stdout:0/291: fsync d4/f14 0 2024-04-02T04:42:49.936 INFO:tasks.workunit.client.1.smithi027.stdout:2/283: creat d1/db/d3b/d17/d29/f5a x:0 0 0 2024-04-02T04:42:49.936 INFO:tasks.workunit.client.0.smithi016.stdout:2/845: symlink d12/d35/d3b/d111/l121 0 2024-04-02T04:42:49.937 INFO:tasks.workunit.client.1.smithi027.stdout:1/224: dwrite d0/f7 [0,4194304] 0 2024-04-02T04:42:49.939 INFO:tasks.workunit.client.0.smithi016.stdout:0/753: creat d0/d4f/ffd x:0 0 0 2024-04-02T04:42:49.940 INFO:tasks.workunit.client.1.smithi027.stdout:9/275: creat d1a/f63 x:0 0 0 2024-04-02T04:42:49.942 INFO:tasks.workunit.client.1.smithi027.stdout:9/276: dread - d1a/d1d/d29/d37/f5a zero size 2024-04-02T04:42:49.942 INFO:tasks.workunit.client.0.smithi016.stdout:1/747: symlink d14/d19/lf6 0 2024-04-02T04:42:49.944 INFO:tasks.workunit.client.1.smithi027.stdout:9/277: dread - d1a/d1d/f53 zero size 2024-04-02T04:42:49.944 INFO:tasks.workunit.client.0.smithi016.stdout:0/754: dwrite d0/d4f/daf/fb1 [4194304,4194304] 0 2024-04-02T04:42:49.946 INFO:tasks.workunit.client.1.smithi027.stdout:1/225: dwrite d0/f7 [0,4194304] 0 2024-04-02T04:42:49.956 INFO:tasks.workunit.client.1.smithi027.stdout:6/246: dwrite d13/d1b/d24/f41 [0,4194304] 0 2024-04-02T04:42:49.961 INFO:tasks.workunit.client.0.smithi016.stdout:2/846: unlink d12/d35/d3b/d42/d8d/fa6 0 2024-04-02T04:42:49.962 INFO:tasks.workunit.client.1.smithi027.stdout:0/292: fsync d4/d3d/f3f 0 2024-04-02T04:42:49.968 INFO:tasks.workunit.client.0.smithi016.stdout:1/748: unlink d14/d19/d1d/d29/d39/d46/dcb/f47 0 2024-04-02T04:42:49.968 INFO:tasks.workunit.client.0.smithi016.stdout:6/822: dread d0/f9b [0,4194304] 0 2024-04-02T04:42:49.969 INFO:tasks.workunit.client.1.smithi027.stdout:2/284: rename d1/db/l1b to d1/db/d3b/d17/d2b/l5b 0 2024-04-02T04:42:49.973 INFO:tasks.workunit.client.1.smithi027.stdout:4/224: sync 2024-04-02T04:42:49.974 INFO:tasks.workunit.client.0.smithi016.stdout:0/755: mknod d0/d16/dd6/ddd/cfe 0 2024-04-02T04:42:49.977 INFO:tasks.workunit.client.0.smithi016.stdout:8/756: getdents d1 0 2024-04-02T04:42:49.977 INFO:tasks.workunit.client.0.smithi016.stdout:8/757: fsync d1/d70/ff5 0 2024-04-02T04:42:49.979 INFO:tasks.workunit.client.1.smithi027.stdout:2/285: dwrite d1/db/d3b/d17/d29/f5a [0,4194304] 0 2024-04-02T04:42:49.980 INFO:tasks.workunit.client.1.smithi027.stdout:1/226: write d0/de/f4d [769739,126053] 0 2024-04-02T04:42:49.982 INFO:tasks.workunit.client.1.smithi027.stdout:2/286: write d1/f13 [7390011,19992] 0 2024-04-02T04:42:49.983 INFO:tasks.workunit.client.1.smithi027.stdout:3/192: dwrite d1/de/df/f14 [0,4194304] 0 2024-04-02T04:42:49.986 INFO:tasks.workunit.client.0.smithi016.stdout:7/757: truncate d0/d46/f1d 1935736 0 2024-04-02T04:42:49.990 INFO:tasks.workunit.client.1.smithi027.stdout:5/346: getdents d0/d1d/d61 0 2024-04-02T04:42:49.993 INFO:tasks.workunit.client.0.smithi016.stdout:9/853: link d0/d4a/d99/l26 d0/d4e/d50/d63/l108 0 2024-04-02T04:42:49.995 INFO:tasks.workunit.client.1.smithi027.stdout:4/225: rename d24/f2f to d24/d2d/f43 0 2024-04-02T04:42:49.996 INFO:tasks.workunit.client.1.smithi027.stdout:4/226: truncate d24/f27 937624 0 2024-04-02T04:42:49.997 INFO:tasks.workunit.client.1.smithi027.stdout:4/227: write d24/d38/f3e [1007873,56216] 0 2024-04-02T04:42:49.997 INFO:tasks.workunit.client.0.smithi016.stdout:0/756: creat d0/d16/dd6/ddd/fff x:0 0 0 2024-04-02T04:42:49.998 INFO:tasks.workunit.client.1.smithi027.stdout:4/228: dread - d24/f39 zero size 2024-04-02T04:42:49.999 INFO:tasks.workunit.client.1.smithi027.stdout:9/278: unlink cc 0 2024-04-02T04:42:50.000 INFO:tasks.workunit.client.1.smithi027.stdout:9/279: fdatasync d1a/d1d/d29/f46 0 2024-04-02T04:42:50.001 INFO:tasks.workunit.client.0.smithi016.stdout:8/758: fdatasync d1/d6/d86/fa9 0 2024-04-02T04:42:50.002 INFO:tasks.workunit.client.0.smithi016.stdout:8/759: read d1/d6/d36/d3f/fb4 [694812,81296] 0 2024-04-02T04:42:50.003 INFO:tasks.workunit.client.0.smithi016.stdout:0/757: dwrite d0/d16/d2e/d36/f8b [0,4194304] 0 2024-04-02T04:42:50.004 INFO:tasks.workunit.client.0.smithi016.stdout:5/747: truncate d3/df/d18/d2f/fb4 688900 0 2024-04-02T04:42:50.005 INFO:tasks.workunit.client.1.smithi027.stdout:1/227: creat d0/de/f51 x:0 0 0 2024-04-02T04:42:50.006 INFO:tasks.workunit.client.0.smithi016.stdout:5/748: read - d3/d40/d4f/d6b/d45/da6/d102/dc4/dd7/fdf zero size 2024-04-02T04:42:50.006 INFO:tasks.workunit.client.1.smithi027.stdout:7/265: dwrite d2/d7/da/dc/f30 [0,4194304] 0 2024-04-02T04:42:50.008 INFO:tasks.workunit.client.1.smithi027.stdout:7/266: dread - d2/d7/da/dc/d16/d3f/f43 zero size 2024-04-02T04:42:50.013 INFO:tasks.workunit.client.1.smithi027.stdout:7/267: dread d2/df/f10 [0,4194304] 0 2024-04-02T04:42:50.015 INFO:tasks.workunit.client.1.smithi027.stdout:7/268: write d2/d1c/f37 [259374,89317] 0 2024-04-02T04:42:50.015 INFO:tasks.workunit.client.0.smithi016.stdout:7/758: creat d0/d46/da2/dc8/f108 x:0 0 0 2024-04-02T04:42:50.016 INFO:tasks.workunit.client.1.smithi027.stdout:7/269: fdatasync d2/d7/da/dc/d1f/d33/d36/f45 0 2024-04-02T04:42:50.017 INFO:tasks.workunit.client.1.smithi027.stdout:2/287: mkdir d1/db/d3b/d17/d12/d5c 0 2024-04-02T04:42:50.019 INFO:tasks.workunit.client.1.smithi027.stdout:2/288: write d1/db/d3b/d17/d2b/f52 [613405,93416] 0 2024-04-02T04:42:50.020 INFO:tasks.workunit.client.0.smithi016.stdout:6/823: write d0/d9/d16/f20 [1460923,117349] 0 2024-04-02T04:42:50.021 INFO:tasks.workunit.client.0.smithi016.stdout:6/824: fsync d0/d9/d16/d46/dbe/fd1 0 2024-04-02T04:42:50.023 INFO:tasks.workunit.client.1.smithi027.stdout:0/293: symlink d4/d5d/l5e 0 2024-04-02T04:42:50.024 INFO:tasks.workunit.client.0.smithi016.stdout:5/749: dread d3/df/f22 [0,4194304] 0 2024-04-02T04:42:50.025 INFO:tasks.workunit.client.1.smithi027.stdout:0/294: chown d4/d16/d2b/d2d/l34 111130225 1 2024-04-02T04:42:50.026 INFO:tasks.workunit.client.1.smithi027.stdout:5/347: symlink d0/df/d76/l83 0 2024-04-02T04:42:50.026 INFO:tasks.workunit.client.0.smithi016.stdout:0/758: dread d0/d16/f54 [0,4194304] 0 2024-04-02T04:42:50.027 INFO:tasks.workunit.client.0.smithi016.stdout:9/854: mknod d0/d9/d4d/d78/da6/c109 0 2024-04-02T04:42:50.031 INFO:tasks.workunit.client.1.smithi027.stdout:4/229: rename d24/d2d/d34 to d24/d38/d44 0 2024-04-02T04:42:50.032 INFO:tasks.workunit.client.0.smithi016.stdout:2/847: truncate d12/d35/d3b/d42/d59/fba 3716936 0 2024-04-02T04:42:50.035 INFO:tasks.workunit.client.0.smithi016.stdout:5/750: dread d3/f5 [0,4194304] 0 2024-04-02T04:42:50.036 INFO:tasks.workunit.client.0.smithi016.stdout:5/751: write d3/df/f1e [4676269,103769] 0 2024-04-02T04:42:50.037 INFO:tasks.workunit.client.1.smithi027.stdout:4/230: dwrite f23 [0,4194304] 0 2024-04-02T04:42:50.038 INFO:tasks.workunit.client.0.smithi016.stdout:1/749: truncate d14/f48 2394979 0 2024-04-02T04:42:50.039 INFO:tasks.workunit.client.1.smithi027.stdout:9/280: dread d1a/d1d/d29/f4f [0,4194304] 0 2024-04-02T04:42:50.040 INFO:tasks.workunit.client.0.smithi016.stdout:1/750: write d14/d19/d1d/d45/d4f/d86/db4/fd3 [454761,23660] 0 2024-04-02T04:42:50.042 INFO:tasks.workunit.client.0.smithi016.stdout:7/759: unlink d0/lf7 0 2024-04-02T04:42:50.043 INFO:tasks.workunit.client.1.smithi027.stdout:9/281: dread d1a/f39 [0,4194304] 0 2024-04-02T04:42:50.044 INFO:tasks.workunit.client.0.smithi016.stdout:7/760: truncate d0/d46/d52/d82/fd3 1015597 0 2024-04-02T04:42:50.045 INFO:tasks.workunit.client.0.smithi016.stdout:7/761: readlink d0/d46/d16/l4b 0 2024-04-02T04:42:50.047 INFO:tasks.workunit.client.0.smithi016.stdout:6/825: mkdir d0/d9/d16/d56/d89/d100 0 2024-04-02T04:42:50.049 INFO:tasks.workunit.client.0.smithi016.stdout:6/826: truncate d0/d9/d16/d56/d89/fc7 14455 0 2024-04-02T04:42:50.053 INFO:tasks.workunit.client.0.smithi016.stdout:0/759: creat d0/d4f/f100 x:0 0 0 2024-04-02T04:42:50.055 INFO:tasks.workunit.client.1.smithi027.stdout:1/228: dread d0/d1e/d25/f29 [0,4194304] 0 2024-04-02T04:42:50.057 INFO:tasks.workunit.client.0.smithi016.stdout:9/855: write d0/d4e/d50/f85 [229317,130067] 0 2024-04-02T04:42:50.062 INFO:tasks.workunit.client.1.smithi027.stdout:6/247: link d13/d1b/d24/d25/d27/f51 d13/d1b/d24/d28/f55 0 2024-04-02T04:42:50.063 INFO:tasks.workunit.client.1.smithi027.stdout:6/248: write d13/d1b/f20 [2366579,6334] 0 2024-04-02T04:42:50.065 INFO:tasks.workunit.client.1.smithi027.stdout:5/348: rmdir d0/df/d11 39 2024-04-02T04:42:50.069 INFO:tasks.workunit.client.1.smithi027.stdout:0/295: rename d4/d16/d53 to d4/d21/d28/d4e/d5f 0 2024-04-02T04:42:50.073 INFO:tasks.workunit.client.1.smithi027.stdout:4/231: symlink d24/d2d/l45 0 2024-04-02T04:42:50.075 INFO:tasks.workunit.client.1.smithi027.stdout:3/193: dwrite d1/d1a/f22 [0,4194304] 0 2024-04-02T04:42:50.075 INFO:tasks.workunit.client.0.smithi016.stdout:7/762: mknod d0/d46/d16/d49/c109 0 2024-04-02T04:42:50.078 INFO:tasks.workunit.client.0.smithi016.stdout:6/827: creat d0/d9/d16/d56/da4/f101 x:0 0 0 2024-04-02T04:42:50.079 INFO:tasks.workunit.client.1.smithi027.stdout:8/252: link d5/l16 d5/d7/l52 0 2024-04-02T04:42:50.083 INFO:tasks.workunit.client.0.smithi016.stdout:0/760: mknod d0/d16/c101 0 2024-04-02T04:42:50.084 INFO:tasks.workunit.client.0.smithi016.stdout:0/761: fsync d0/d48/fec 0 2024-04-02T04:42:50.087 INFO:tasks.workunit.client.0.smithi016.stdout:5/752: mkdir d3/d40/d10d 0 2024-04-02T04:42:50.089 INFO:tasks.workunit.client.1.smithi027.stdout:5/349: creat d0/df/d29/f84 x:0 0 0 2024-04-02T04:42:50.096 INFO:tasks.workunit.client.0.smithi016.stdout:7/763: rmdir d0/d46/d16 39 2024-04-02T04:42:50.096 INFO:tasks.workunit.client.0.smithi016.stdout:7/764: stat d0/d46/d37/dbe/ff4 0 2024-04-02T04:42:50.097 INFO:tasks.workunit.client.0.smithi016.stdout:7/765: readlink d0/d46/d52/d82/lbd 0 2024-04-02T04:42:50.097 INFO:tasks.workunit.client.1.smithi027.stdout:5/350: dwrite d0/d5/f63 [0,4194304] 0 2024-04-02T04:42:50.097 INFO:tasks.workunit.client.1.smithi027.stdout:3/194: dread d1/f32 [0,4194304] 0 2024-04-02T04:42:50.099 INFO:tasks.workunit.client.1.smithi027.stdout:8/253: truncate d5/d8/f17 145996 0 2024-04-02T04:42:50.101 INFO:tasks.workunit.client.0.smithi016.stdout:6/828: unlink d0/d9/d16/d56/fe6 0 2024-04-02T04:42:50.101 INFO:tasks.workunit.client.0.smithi016.stdout:6/829: stat d0/d9/d16/d46/d5e/c9a 0 2024-04-02T04:42:50.103 INFO:tasks.workunit.client.0.smithi016.stdout:2/848: write d12/d35/d3f/d2b/de9/d102/f11e [3394697,63968] 0 2024-04-02T04:42:50.105 INFO:tasks.workunit.client.1.smithi027.stdout:7/270: creat d2/f4e x:0 0 0 2024-04-02T04:42:50.106 INFO:tasks.workunit.client.1.smithi027.stdout:7/271: chown d2/f4e 517589225 1 2024-04-02T04:42:50.107 INFO:tasks.workunit.client.1.smithi027.stdout:2/289: creat d1/db/d3b/d17/f5d x:0 0 0 2024-04-02T04:42:50.108 INFO:tasks.workunit.client.1.smithi027.stdout:6/249: mkdir d13/d1b/d24/d25/d56 0 2024-04-02T04:42:50.110 INFO:tasks.workunit.client.0.smithi016.stdout:8/760: link d1/d70/d71/d74/d82/fc2 d1/d6/d36/d3f/d40/d49/ffe 0 2024-04-02T04:42:50.111 INFO:tasks.workunit.client.0.smithi016.stdout:3/868: sync 2024-04-02T04:42:50.111 INFO:tasks.workunit.client.0.smithi016.stdout:4/781: sync 2024-04-02T04:42:50.112 INFO:tasks.workunit.client.1.smithi027.stdout:9/282: rename c11 to d1a/d1d/d29/d51/c64 0 2024-04-02T04:42:50.113 INFO:tasks.workunit.client.0.smithi016.stdout:3/869: fsync d2/d15/d1a/d45/f8f 0 2024-04-02T04:42:50.114 INFO:tasks.workunit.client.0.smithi016.stdout:4/782: write d5/d17/d76/fe3 [875019,33533] 0 2024-04-02T04:42:50.114 INFO:tasks.workunit.client.0.smithi016.stdout:7/766: chown d0/d46/d16/d49/c79 1298116 1 2024-04-02T04:42:50.115 INFO:tasks.workunit.client.0.smithi016.stdout:3/870: stat d2/d2e/d36/d66 0 2024-04-02T04:42:50.116 INFO:tasks.workunit.client.0.smithi016.stdout:1/751: dwrite d14/d19/d1e/f2b [0,4194304] 0 2024-04-02T04:42:50.118 INFO:tasks.workunit.client.1.smithi027.stdout:5/351: creat d0/df/d76/f85 x:0 0 0 2024-04-02T04:42:50.118 INFO:tasks.workunit.client.1.smithi027.stdout:1/229: truncate d0/f37 2942367 0 2024-04-02T04:42:50.123 INFO:tasks.workunit.client.1.smithi027.stdout:8/254: rmdir d5/d8 39 2024-04-02T04:42:50.125 INFO:tasks.workunit.client.1.smithi027.stdout:8/255: truncate d5/d7/d27/f50 643801 0 2024-04-02T04:42:50.128 INFO:tasks.workunit.client.0.smithi016.stdout:0/762: mkdir d0/d16/d2e/d53/da4/d102 0 2024-04-02T04:42:50.129 INFO:tasks.workunit.client.1.smithi027.stdout:2/290: unlink d1/db/f49 0 2024-04-02T04:42:50.131 INFO:tasks.workunit.client.0.smithi016.stdout:5/753: truncate d3/df/d18/f31 1262687 0 2024-04-02T04:42:50.134 INFO:tasks.workunit.client.1.smithi027.stdout:2/291: dread d1/f13 [0,4194304] 0 2024-04-02T04:42:50.135 INFO:tasks.workunit.client.0.smithi016.stdout:8/761: symlink d1/d6/d36/d3f/d40/d63/lff 0 2024-04-02T04:42:50.143 INFO:tasks.workunit.client.1.smithi027.stdout:7/272: dwrite d2/d7/da/f1e [0,4194304] 0 2024-04-02T04:42:50.146 INFO:tasks.workunit.client.1.smithi027.stdout:3/195: creat d1/de/d3a/f3c x:0 0 0 2024-04-02T04:42:50.151 INFO:tasks.workunit.client.1.smithi027.stdout:7/273: dwrite d2/d7/da/dc/f30 [0,4194304] 0 2024-04-02T04:42:50.153 INFO:tasks.workunit.client.1.smithi027.stdout:7/274: write d2/d7/da/dc/d16/d2f/f41 [1426983,120847] 0 2024-04-02T04:42:50.156 INFO:tasks.workunit.client.1.smithi027.stdout:2/292: symlink d1/db/d3b/d17/d2b/d2f/l5e 0 2024-04-02T04:42:50.156 INFO:tasks.workunit.client.1.smithi027.stdout:0/296: getdents d4/d16/d1b 0 2024-04-02T04:42:50.161 INFO:tasks.workunit.client.1.smithi027.stdout:3/196: creat d1/de/d3a/f3d x:0 0 0 2024-04-02T04:42:50.162 INFO:tasks.workunit.client.1.smithi027.stdout:8/256: mkdir d5/d8/db/d53 0 2024-04-02T04:42:50.166 INFO:tasks.workunit.client.1.smithi027.stdout:9/283: link d1a/l4e d1a/d1d/d29/d50/l65 0 2024-04-02T04:42:50.167 INFO:tasks.workunit.client.1.smithi027.stdout:9/284: truncate d1a/f63 382197 0 2024-04-02T04:42:50.167 INFO:tasks.workunit.client.1.smithi027.stdout:3/197: dwrite d1/d1a/f1c [0,4194304] 0 2024-04-02T04:42:50.168 INFO:tasks.workunit.client.1.smithi027.stdout:9/285: stat d1a/d1d/d29/d37/d20 0 2024-04-02T04:42:50.174 INFO:tasks.workunit.client.1.smithi027.stdout:6/250: getdents d13/d1b/d24/d25/d42 0 2024-04-02T04:42:50.175 INFO:tasks.workunit.client.1.smithi027.stdout:8/257: dread d5/d8/f17 [0,4194304] 0 2024-04-02T04:42:50.176 INFO:tasks.workunit.client.0.smithi016.stdout:3/871: creat d2/d2e/d36/d48/da2/d88/f125 x:0 0 0 2024-04-02T04:42:50.176 INFO:tasks.workunit.client.1.smithi027.stdout:2/293: creat d1/db/d3b/d58/f5f x:0 0 0 2024-04-02T04:42:50.176 INFO:tasks.workunit.client.0.smithi016.stdout:7/767: unlink d0/d46/d37/dbe/dae/df8/d43/f5e 0 2024-04-02T04:42:50.177 INFO:tasks.workunit.client.0.smithi016.stdout:9/856: getdents d0/d4e/d50/d63/d6e/d4b/db2/dba 0 2024-04-02T04:42:50.177 INFO:tasks.workunit.client.0.smithi016.stdout:2/849: mknod d12/c122 0 2024-04-02T04:42:50.178 INFO:tasks.workunit.client.0.smithi016.stdout:0/763: mknod d0/d4f/d76/c103 0 2024-04-02T04:42:50.179 INFO:tasks.workunit.client.0.smithi016.stdout:5/754: creat d3/d40/d4f/d67/dbf/f10e x:0 0 0 2024-04-02T04:42:50.182 INFO:tasks.workunit.client.0.smithi016.stdout:2/850: chown d12/d35/d3b/d42/d67/fb3 1703750 1 2024-04-02T04:42:50.183 INFO:tasks.workunit.client.0.smithi016.stdout:6/830: dwrite d0/d9/df/f74 [0,4194304] 0 2024-04-02T04:42:50.183 INFO:tasks.workunit.client.1.smithi027.stdout:0/297: creat d4/f60 x:0 0 0 2024-04-02T04:42:50.183 INFO:tasks.workunit.client.1.smithi027.stdout:3/198: dwrite d1/de/df/d13/f34 [0,4194304] 0 2024-04-02T04:42:50.184 INFO:tasks.workunit.client.0.smithi016.stdout:0/764: truncate d0/d16/d2e/d37/d66/ff4 594679 0 2024-04-02T04:42:50.186 INFO:tasks.workunit.client.1.smithi027.stdout:5/352: truncate d0/f6 1104877 0 2024-04-02T04:42:50.186 INFO:tasks.workunit.client.1.smithi027.stdout:9/286: dwrite d1a/d1d/d29/f46 [0,4194304] 0 2024-04-02T04:42:50.190 INFO:tasks.workunit.client.0.smithi016.stdout:7/768: creat d0/d46/d52/d82/f10a x:0 0 0 2024-04-02T04:42:50.191 INFO:tasks.workunit.client.1.smithi027.stdout:5/353: chown d0/d5 1022184 1 2024-04-02T04:42:50.192 INFO:tasks.workunit.client.1.smithi027.stdout:8/258: dwrite d5/d7/f14 [0,4194304] 0 2024-04-02T04:42:50.197 INFO:tasks.workunit.client.1.smithi027.stdout:7/275: write d2/d7/da/f2a [915172,21281] 0 2024-04-02T04:42:50.198 INFO:tasks.workunit.client.1.smithi027.stdout:1/230: chown d0/f3d 10088 1 2024-04-02T04:42:50.201 INFO:tasks.workunit.client.1.smithi027.stdout:7/276: fdatasync d2/d7/da/dc/d16/d3f/f43 0 2024-04-02T04:42:50.202 INFO:tasks.workunit.client.1.smithi027.stdout:6/251: symlink d13/d1b/d24/d25/d27/l57 0 2024-04-02T04:42:50.205 INFO:tasks.workunit.client.0.smithi016.stdout:9/857: chown d0/d4e/d50/d63/d6e/d4b/d58/d6a/ccb 67585854 1 2024-04-02T04:42:50.206 INFO:tasks.workunit.client.0.smithi016.stdout:8/762: dwrite d1/d31/f7e [0,4194304] 0 2024-04-02T04:42:50.207 INFO:tasks.workunit.client.1.smithi027.stdout:0/298: dwrite d4/d16/d2b/d2d/f3c [0,4194304] 0 2024-04-02T04:42:50.208 INFO:tasks.workunit.client.0.smithi016.stdout:9/858: truncate d0/d4a/d99/d46/fe6 589106 0 2024-04-02T04:42:50.208 INFO:tasks.workunit.client.0.smithi016.stdout:1/752: creat d14/d19/d1d/d29/ff7 x:0 0 0 2024-04-02T04:42:50.211 INFO:tasks.workunit.client.1.smithi027.stdout:5/354: dwrite d0/d1d/d61/f62 [0,4194304] 0 2024-04-02T04:42:50.213 INFO:tasks.workunit.client.1.smithi027.stdout:5/355: readlink d0/d5/l60 0 2024-04-02T04:42:50.218 INFO:tasks.workunit.client.1.smithi027.stdout:0/299: write d4/d16/d2b/d2d/d4c/f5c [995169,48405] 0 2024-04-02T04:42:50.218 INFO:tasks.workunit.client.1.smithi027.stdout:2/294: write d1/db/d3b/d17/fc [12779165,5962] 0 2024-04-02T04:42:50.221 INFO:tasks.workunit.client.0.smithi016.stdout:7/769: read d0/f9c [3590417,24251] 0 2024-04-02T04:42:50.221 INFO:tasks.workunit.client.0.smithi016.stdout:3/872: creat d2/d15/d16/d8a/dd1/d6e/d6f/df9/f126 x:0 0 0 2024-04-02T04:42:50.221 INFO:tasks.workunit.client.1.smithi027.stdout:7/277: mknod d2/d7/da/dc/d1f/c4f 0 2024-04-02T04:42:50.222 INFO:tasks.workunit.client.1.smithi027.stdout:6/252: chown d13/d1b/d24/d25/d27/l30 1831 1 2024-04-02T04:42:50.225 INFO:tasks.workunit.client.1.smithi027.stdout:5/356: creat d0/d1d/d61/d72/f86 x:0 0 0 2024-04-02T04:42:50.227 INFO:tasks.workunit.client.0.smithi016.stdout:8/763: rename d1/d31/d67/d8f to d1/d6/d36/d100 0 2024-04-02T04:42:50.227 INFO:tasks.workunit.client.0.smithi016.stdout:5/755: write d3/df/f22 [6341144,124030] 0 2024-04-02T04:42:50.228 INFO:tasks.workunit.client.1.smithi027.stdout:5/357: write d0/df/d2f/d3b/f3d [2904048,68956] 0 2024-04-02T04:42:50.230 INFO:tasks.workunit.client.1.smithi027.stdout:5/358: truncate d0/df/d76/f85 622620 0 2024-04-02T04:42:50.233 INFO:tasks.workunit.client.0.smithi016.stdout:9/859: unlink d0/d4a/d99/d46/lb1 0 2024-04-02T04:42:50.234 INFO:tasks.workunit.client.1.smithi027.stdout:8/259: fsync d5/d7/f14 0 2024-04-02T04:42:50.235 INFO:tasks.workunit.client.0.smithi016.stdout:1/753: symlink d14/d19/d1d/d29/d39/d46/dcb/d43/lf8 0 2024-04-02T04:42:50.237 INFO:tasks.workunit.client.1.smithi027.stdout:0/300: creat d4/d16/d2b/d4a/f61 x:0 0 0 2024-04-02T04:42:50.238 INFO:tasks.workunit.client.1.smithi027.stdout:0/301: chown d4/d38/c52 111947 1 2024-04-02T04:42:50.242 INFO:tasks.workunit.client.1.smithi027.stdout:9/287: dread f17 [0,4194304] 0 2024-04-02T04:42:50.243 INFO:tasks.workunit.client.1.smithi027.stdout:6/253: rmdir d13/d1b/d24 39 2024-04-02T04:42:50.244 INFO:tasks.workunit.client.1.smithi027.stdout:3/199: truncate d1/de/df/d13/f36 3069370 0 2024-04-02T04:42:50.244 INFO:tasks.workunit.client.0.smithi016.stdout:2/851: dwrite d12/d35/d101/f44 [0,4194304] 0 2024-04-02T04:42:50.246 INFO:tasks.workunit.client.0.smithi016.stdout:7/770: fdatasync d0/d46/f30 0 2024-04-02T04:42:50.248 INFO:tasks.workunit.client.0.smithi016.stdout:6/831: dwrite d0/d9/d16/ffd [0,4194304] 0 2024-04-02T04:42:50.250 INFO:tasks.workunit.client.0.smithi016.stdout:7/771: chown d0/d46/d37/dbe/dae/lab 5369 1 2024-04-02T04:42:50.250 INFO:tasks.workunit.client.0.smithi016.stdout:3/873: creat d2/d15/d60/f127 x:0 0 0 2024-04-02T04:42:50.251 INFO:tasks.workunit.client.0.smithi016.stdout:0/765: dwrite d0/d16/d24/d62/f9b [0,4194304] 0 2024-04-02T04:42:50.251 INFO:tasks.workunit.client.1.smithi027.stdout:3/200: dwrite d1/de/df/f19 [0,4194304] 0 2024-04-02T04:42:50.253 INFO:tasks.workunit.client.0.smithi016.stdout:6/832: chown d0/d9/d16/d56/fcb 92 1 2024-04-02T04:42:50.254 INFO:tasks.workunit.client.0.smithi016.stdout:7/772: write d0/d46/d16/dc2/ff2 [158117,111867] 0 2024-04-02T04:42:50.256 INFO:tasks.workunit.client.0.smithi016.stdout:7/773: stat d0/d46/d37/dbe/dae/df8/d84/fd0 0 2024-04-02T04:42:50.259 INFO:tasks.workunit.client.1.smithi027.stdout:1/231: truncate d0/f7 2929128 0 2024-04-02T04:42:50.260 INFO:tasks.workunit.client.0.smithi016.stdout:4/783: sync 2024-04-02T04:42:50.267 INFO:tasks.workunit.client.0.smithi016.stdout:1/754: creat d14/d19/d1d/d29/d39/d46/dcb/d43/d68/ff9 x:0 0 0 2024-04-02T04:42:50.268 INFO:tasks.workunit.client.0.smithi016.stdout:1/755: chown fa 4441774 1 2024-04-02T04:42:50.268 INFO:tasks.workunit.client.1.smithi027.stdout:0/302: creat d4/d21/d28/d4e/f62 x:0 0 0 2024-04-02T04:42:50.272 INFO:tasks.workunit.client.1.smithi027.stdout:7/278: unlink d2/l5 0 2024-04-02T04:42:50.273 INFO:tasks.workunit.client.1.smithi027.stdout:7/279: stat d2/f4e 0 2024-04-02T04:42:50.279 INFO:tasks.workunit.client.1.smithi027.stdout:6/254: read d13/d1b/f1f [466807,108747] 0 2024-04-02T04:42:50.280 INFO:tasks.workunit.client.1.smithi027.stdout:9/288: creat d1a/d1d/d29/d37/d4c/f66 x:0 0 0 2024-04-02T04:42:50.281 INFO:tasks.workunit.client.1.smithi027.stdout:6/255: chown d13/d1b/f2b 967 1 2024-04-02T04:42:50.282 INFO:tasks.workunit.client.1.smithi027.stdout:6/256: fdatasync f12 0 2024-04-02T04:42:50.286 INFO:tasks.workunit.client.1.smithi027.stdout:5/359: dwrite d0/f47 [0,4194304] 0 2024-04-02T04:42:50.286 INFO:tasks.workunit.client.1.smithi027.stdout:3/201: rmdir d1/de/df 39 2024-04-02T04:42:50.288 INFO:tasks.workunit.client.0.smithi016.stdout:6/833: creat d0/d6e/f102 x:0 0 0 2024-04-02T04:42:50.292 INFO:tasks.workunit.client.1.smithi027.stdout:4/232: sync 2024-04-02T04:42:50.294 INFO:tasks.workunit.client.0.smithi016.stdout:9/860: dread d0/d3/fbe [0,4194304] 0 2024-04-02T04:42:50.296 INFO:tasks.workunit.client.1.smithi027.stdout:2/295: creat d1/db/d3b/f60 x:0 0 0 2024-04-02T04:42:50.298 INFO:tasks.workunit.client.0.smithi016.stdout:4/784: mknod d5/d17/d18/da0/cec 0 2024-04-02T04:42:50.298 INFO:tasks.workunit.client.1.smithi027.stdout:0/303: mknod d4/d16/c63 0 2024-04-02T04:42:50.301 INFO:tasks.workunit.client.1.smithi027.stdout:1/232: dread d0/f30 [0,4194304] 0 2024-04-02T04:42:50.302 INFO:tasks.workunit.client.1.smithi027.stdout:7/280: mkdir d2/d7/da/dc/d16/d3f/d50 0 2024-04-02T04:42:50.303 INFO:tasks.workunit.client.1.smithi027.stdout:0/304: dread d4/d21/d28/f48 [0,4194304] 0 2024-04-02T04:42:50.305 INFO:tasks.workunit.client.1.smithi027.stdout:9/289: mknod d1a/d1d/d29/d56/c67 0 2024-04-02T04:42:50.305 INFO:tasks.workunit.client.1.smithi027.stdout:0/305: write d4/d21/f2e [5828014,2572] 0 2024-04-02T04:42:50.307 INFO:tasks.workunit.client.1.smithi027.stdout:0/306: read - d4/d16/d2b/d2d/f56 zero size 2024-04-02T04:42:50.310 INFO:tasks.workunit.client.0.smithi016.stdout:2/852: write d12/d35/d3f/f7e [1179704,109030] 0 2024-04-02T04:42:50.311 INFO:tasks.workunit.client.1.smithi027.stdout:3/202: mknod d1/d1a/c3e 0 2024-04-02T04:42:50.313 INFO:tasks.workunit.client.1.smithi027.stdout:7/281: dwrite d2/d7/da/dc/d16/d3f/f43 [0,4194304] 0 2024-04-02T04:42:50.313 INFO:tasks.workunit.client.0.smithi016.stdout:7/774: dread d0/d46/f1d [0,4194304] 0 2024-04-02T04:42:50.313 INFO:tasks.workunit.client.0.smithi016.stdout:6/834: symlink d0/d9/d16/d56/d89/dc5/l103 0 2024-04-02T04:42:50.314 INFO:tasks.workunit.client.1.smithi027.stdout:5/360: creat d0/d1a/d20/f87 x:0 0 0 2024-04-02T04:42:50.319 INFO:tasks.workunit.client.1.smithi027.stdout:8/260: link d5/d8/db/l1a d5/d8/db/l54 0 2024-04-02T04:42:50.321 INFO:tasks.workunit.client.0.smithi016.stdout:2/853: dwrite d12/d35/d3b/d42/d59/d91/db2/f110 [0,4194304] 0 2024-04-02T04:42:50.322 INFO:tasks.workunit.client.0.smithi016.stdout:5/756: write d3/d40/d4f/d6b/fa2 [3133680,99657] 0 2024-04-02T04:42:50.330 INFO:tasks.workunit.client.0.smithi016.stdout:1/756: dwrite d14/d19/d1e/d71/f78 [0,4194304] 0 2024-04-02T04:42:50.330 INFO:tasks.workunit.client.0.smithi016.stdout:3/874: link d2/d15/d16/d8a/dd1/fe3 d2/d15/d1a/f128 0 2024-04-02T04:42:50.331 INFO:tasks.workunit.client.0.smithi016.stdout:6/835: dread d0/d9/f32 [0,4194304] 0 2024-04-02T04:42:50.331 INFO:tasks.workunit.client.0.smithi016.stdout:1/757: dread - d14/d19/ff1 zero size 2024-04-02T04:42:50.332 INFO:tasks.workunit.client.0.smithi016.stdout:3/875: truncate d2/d2e/f100 253250 0 2024-04-02T04:42:50.336 INFO:tasks.workunit.client.0.smithi016.stdout:9/861: mknod d0/d9/db/dc9/d53/d7e/c10a 0 2024-04-02T04:42:50.339 INFO:tasks.workunit.client.1.smithi027.stdout:5/361: rmdir d0/d68/d6b 39 2024-04-02T04:42:50.339 INFO:tasks.workunit.client.1.smithi027.stdout:0/307: dwrite d4/d16/f17 [0,4194304] 0 2024-04-02T04:42:50.341 INFO:tasks.workunit.client.0.smithi016.stdout:5/757: rename d3/df/d86/de9 to d3/df/d18/dfd/d10f 0 2024-04-02T04:42:50.346 INFO:tasks.workunit.client.0.smithi016.stdout:0/766: getdents d0/d16/d24/d62 0 2024-04-02T04:42:50.346 INFO:tasks.workunit.client.0.smithi016.stdout:7/775: dread d0/d46/d16/d49/f11 [0,4194304] 0 2024-04-02T04:42:50.347 INFO:tasks.workunit.client.1.smithi027.stdout:3/203: rmdir d1 39 2024-04-02T04:42:50.348 INFO:tasks.workunit.client.1.smithi027.stdout:8/261: mknod d5/d8/d15/c55 0 2024-04-02T04:42:50.352 INFO:tasks.workunit.client.1.smithi027.stdout:1/233: write d0/f14 [4017103,33595] 0 2024-04-02T04:42:50.352 INFO:tasks.workunit.client.0.smithi016.stdout:6/836: creat d0/d9/d16/de3/f104 x:0 0 0 2024-04-02T04:42:50.354 INFO:tasks.workunit.client.0.smithi016.stdout:3/876: fdatasync d2/d15/fb4 0 2024-04-02T04:42:50.357 INFO:tasks.workunit.client.0.smithi016.stdout:8/764: write d1/d6/d36/d3f/d40/d49/f2a [2656047,67095] 0 2024-04-02T04:42:50.358 INFO:tasks.workunit.client.0.smithi016.stdout:8/765: chown d1/f15 16214 1 2024-04-02T04:42:50.359 INFO:tasks.workunit.client.0.smithi016.stdout:9/862: mkdir d0/d3/d97/dac/d10b 0 2024-04-02T04:42:50.360 INFO:tasks.workunit.client.0.smithi016.stdout:8/766: chown d1/d6/d36/d3f/d40/d45/dcc/cdc 104805800 1 2024-04-02T04:42:50.360 INFO:tasks.workunit.client.0.smithi016.stdout:9/863: chown d0/d9/db/d54/f8e 19603 1 2024-04-02T04:42:50.362 INFO:tasks.workunit.client.0.smithi016.stdout:1/758: mkdir d14/d19/d1d/d29/d39/d46/dcb/d43/d68/dc1/dfa 0 2024-04-02T04:42:50.364 INFO:tasks.workunit.client.0.smithi016.stdout:2/854: mknod d12/d35/d3b/c123 0 2024-04-02T04:42:50.365 INFO:tasks.workunit.client.1.smithi027.stdout:0/308: dread f2 [8388608,4194304] 0 2024-04-02T04:42:50.365 INFO:tasks.workunit.client.0.smithi016.stdout:2/855: stat d12/d35/d101/d76/dcf 0 2024-04-02T04:42:50.366 INFO:tasks.workunit.client.1.smithi027.stdout:0/309: write d4/d16/d2b/d2d/f3c [183823,91322] 0 2024-04-02T04:42:50.368 INFO:tasks.workunit.client.1.smithi027.stdout:0/310: write d4/d3d/d49/f50 [1608839,104340] 0 2024-04-02T04:42:50.369 INFO:tasks.workunit.client.1.smithi027.stdout:5/362: write d0/df/d11/d37/f40 [264006,55095] 0 2024-04-02T04:42:50.369 INFO:tasks.workunit.client.1.smithi027.stdout:0/311: dread - d4/d16/d1b/f3e zero size 2024-04-02T04:42:50.373 INFO:tasks.workunit.client.1.smithi027.stdout:8/262: mknod d5/d8/db/df/c56 0 2024-04-02T04:42:50.374 INFO:tasks.workunit.client.0.smithi016.stdout:6/837: mknod d0/d9/d16/d56/d95/c105 0 2024-04-02T04:42:50.376 INFO:tasks.workunit.client.1.smithi027.stdout:2/296: rename d1/db/d3b/d58 to d1/db/d3b/d61 0 2024-04-02T04:42:50.378 INFO:tasks.workunit.client.1.smithi027.stdout:2/297: truncate d1/db/d3b/d17/f3f 931212 0 2024-04-02T04:42:50.378 INFO:tasks.workunit.client.1.smithi027.stdout:1/234: mkdir d0/de/d52 0 2024-04-02T04:42:50.378 INFO:tasks.workunit.client.0.smithi016.stdout:7/776: write d0/d46/f2c [2916804,80207] 0 2024-04-02T04:42:50.378 INFO:tasks.workunit.client.0.smithi016.stdout:3/877: dread - d2/d15/d16/d8a/dd1/d8b/f9c zero size 2024-04-02T04:42:50.381 INFO:tasks.workunit.client.1.smithi027.stdout:9/290: rename d1a/l3d to d1a/d1d/d29/d37/l68 0 2024-04-02T04:42:50.382 INFO:tasks.workunit.client.1.smithi027.stdout:9/291: write d1a/f63 [1388187,1301] 0 2024-04-02T04:42:50.382 INFO:tasks.workunit.client.0.smithi016.stdout:4/785: getdents d5 0 2024-04-02T04:42:50.383 INFO:tasks.workunit.client.1.smithi027.stdout:6/257: rmdir d13/d1b/d24/d54 0 2024-04-02T04:42:50.385 INFO:tasks.workunit.client.0.smithi016.stdout:8/767: rmdir d1/d6/d36/d3f/d40/d63/dc5 39 2024-04-02T04:42:50.385 INFO:tasks.workunit.client.0.smithi016.stdout:7/777: dwrite d0/d46/d16/dc2/dea/f39 [4194304,4194304] 0 2024-04-02T04:42:50.387 INFO:tasks.workunit.client.0.smithi016.stdout:8/768: chown d1/d70/d71/d74/d82/fe4 1016031 1 2024-04-02T04:42:50.387 INFO:tasks.workunit.client.0.smithi016.stdout:9/864: mknod d0/d9/db/d54/c10c 0 2024-04-02T04:42:50.387 INFO:tasks.workunit.client.0.smithi016.stdout:8/769: chown d1/c17 260935554 1 2024-04-02T04:42:50.395 INFO:tasks.workunit.client.1.smithi027.stdout:4/233: getdents d24/d38/d44 0 2024-04-02T04:42:50.398 INFO:tasks.workunit.client.1.smithi027.stdout:6/258: dread f7 [0,4194304] 0 2024-04-02T04:42:50.399 INFO:tasks.workunit.client.1.smithi027.stdout:6/259: readlink le 0 2024-04-02T04:42:50.400 INFO:tasks.workunit.client.1.smithi027.stdout:3/204: creat d1/d1a/f3f x:0 0 0 2024-04-02T04:42:50.401 INFO:tasks.workunit.client.1.smithi027.stdout:8/263: mkdir d5/d8/db/d57 0 2024-04-02T04:42:50.402 INFO:tasks.workunit.client.1.smithi027.stdout:1/235: mknod d0/de/c53 0 2024-04-02T04:42:50.403 INFO:tasks.workunit.client.1.smithi027.stdout:1/236: write d0/de/f1b [392232,108145] 0 2024-04-02T04:42:50.403 INFO:tasks.workunit.client.0.smithi016.stdout:3/878: dread - d2/d15/d60/f115 zero size 2024-04-02T04:42:50.405 INFO:tasks.workunit.client.1.smithi027.stdout:2/298: creat d1/db/d3b/d17/d12/f62 x:0 0 0 2024-04-02T04:42:50.405 INFO:tasks.workunit.client.0.smithi016.stdout:5/758: write d3/df/d18/f98 [637348,86686] 0 2024-04-02T04:42:50.406 INFO:tasks.workunit.client.1.smithi027.stdout:2/299: write d1/db/d3b/d17/d2b/f52 [679970,69930] 0 2024-04-02T04:42:50.408 INFO:tasks.workunit.client.0.smithi016.stdout:4/786: rename d5/d16/c24 to d5/d1d/d5f/d95/ced 0 2024-04-02T04:42:50.410 INFO:tasks.workunit.client.0.smithi016.stdout:6/838: dread d0/f3f [4194304,4194304] 0 2024-04-02T04:42:50.412 INFO:tasks.workunit.client.0.smithi016.stdout:9/865: chown d0/d4e/d50/d63/d6e/d4b/l106 20 1 2024-04-02T04:42:50.413 INFO:tasks.workunit.client.1.smithi027.stdout:4/234: rmdir d24/d2d 39 2024-04-02T04:42:50.414 INFO:tasks.workunit.client.0.smithi016.stdout:8/770: read - d1/d6/d36/d3f/d40/d49/d38/fc4 zero size 2024-04-02T04:42:50.415 INFO:tasks.workunit.client.1.smithi027.stdout:0/312: symlink d4/d21/l64 0 2024-04-02T04:42:50.416 INFO:tasks.workunit.client.0.smithi016.stdout:1/759: fdatasync d14/fdb 0 2024-04-02T04:42:50.417 INFO:tasks.workunit.client.1.smithi027.stdout:9/292: write f17 [1852872,121404] 0 2024-04-02T04:42:50.422 INFO:tasks.workunit.client.0.smithi016.stdout:2/856: creat d12/d35/d101/d76/da0/de7/df8/f124 x:0 0 0 2024-04-02T04:42:50.423 INFO:tasks.workunit.client.1.smithi027.stdout:9/293: write d1a/d1d/f4b [683380,112081] 0 2024-04-02T04:42:50.423 INFO:tasks.workunit.client.1.smithi027.stdout:3/205: dread d1/de/df/d13/f1b [0,4194304] 0 2024-04-02T04:42:50.423 INFO:tasks.workunit.client.0.smithi016.stdout:0/767: creat d0/f104 x:0 0 0 2024-04-02T04:42:50.423 INFO:tasks.workunit.client.1.smithi027.stdout:3/206: write d1/f2a [688688,52176] 0 2024-04-02T04:42:50.424 INFO:tasks.workunit.client.0.smithi016.stdout:3/879: truncate d2/f7 247717 0 2024-04-02T04:42:50.425 INFO:tasks.workunit.client.0.smithi016.stdout:7/778: dread d0/f9c [0,4194304] 0 2024-04-02T04:42:50.426 INFO:tasks.workunit.client.0.smithi016.stdout:4/787: rmdir d5/d17/d76/dca 39 2024-04-02T04:42:50.426 INFO:tasks.workunit.client.1.smithi027.stdout:3/207: dread d1/de/f30 [0,4194304] 0 2024-04-02T04:42:50.427 INFO:tasks.workunit.client.0.smithi016.stdout:6/839: mknod d0/d9/d16/d56/da4/c106 0 2024-04-02T04:42:50.428 INFO:tasks.workunit.client.1.smithi027.stdout:7/282: rename d2/d7/c24 to d2/d1d/c51 0 2024-04-02T04:42:50.430 INFO:tasks.workunit.client.1.smithi027.stdout:7/283: truncate d2/d7/da/dc/d16/f4a 1133886 0 2024-04-02T04:42:50.430 INFO:tasks.workunit.client.0.smithi016.stdout:9/866: rename d0/f2d to d0/d4e/d50/d63/d6e/d4b/db2/f10d 0 2024-04-02T04:42:50.435 INFO:tasks.workunit.client.1.smithi027.stdout:5/363: link d0/d1a/l27 d0/d1d/d61/l88 0 2024-04-02T04:42:50.437 INFO:tasks.workunit.client.1.smithi027.stdout:9/294: mknod d1a/d1d/d29/d37/d3a/c69 0 2024-04-02T04:42:50.440 INFO:tasks.workunit.client.1.smithi027.stdout:6/260: mkdir d13/d1b/d48/d58 0 2024-04-02T04:42:50.440 INFO:tasks.workunit.client.0.smithi016.stdout:5/759: mknod d3/df/d18/d2f/c110 0 2024-04-02T04:42:50.440 INFO:tasks.workunit.client.0.smithi016.stdout:5/760: write d3/df/d18/d26/f7a [2058550,93921] 0 2024-04-02T04:42:50.444 INFO:tasks.workunit.client.1.smithi027.stdout:8/264: creat d5/d8/d48/f58 x:0 0 0 2024-04-02T04:42:50.449 INFO:tasks.workunit.client.0.smithi016.stdout:8/771: write d1/d6/d86/fa9 [1985179,111673] 0 2024-04-02T04:42:50.449 INFO:tasks.workunit.client.0.smithi016.stdout:1/760: write d14/d19/d1d/d29/d4c/f88 [994765,118545] 0 2024-04-02T04:42:50.455 INFO:tasks.workunit.client.1.smithi027.stdout:9/295: mknod d1a/d1d/d29/d37/d4c/c6a 0 2024-04-02T04:42:50.458 INFO:tasks.workunit.client.1.smithi027.stdout:6/261: creat d13/d1b/d24/d28/d46/f59 x:0 0 0 2024-04-02T04:42:50.462 INFO:tasks.workunit.client.1.smithi027.stdout:8/265: fsync d5/d8/f47 0 2024-04-02T04:42:50.467 INFO:tasks.workunit.client.1.smithi027.stdout:1/237: creat d0/f54 x:0 0 0 2024-04-02T04:42:50.468 INFO:tasks.workunit.client.1.smithi027.stdout:1/238: fsync d0/d1e/d25/f50 0 2024-04-02T04:42:50.469 INFO:tasks.workunit.client.1.smithi027.stdout:0/313: dwrite d4/d16/d2b/f2a [0,4194304] 0 2024-04-02T04:42:50.475 INFO:tasks.workunit.client.1.smithi027.stdout:2/300: link d1/db/d3b/d17/f3f d1/db/d3b/d17/f63 0 2024-04-02T04:42:50.476 INFO:tasks.workunit.client.1.smithi027.stdout:1/239: dwrite d0/fd [0,4194304] 0 2024-04-02T04:42:50.477 INFO:tasks.workunit.client.1.smithi027.stdout:2/301: stat d1/db/d3b/d17/d12/f62 0 2024-04-02T04:42:50.478 INFO:tasks.workunit.client.1.smithi027.stdout:1/240: readlink d0/d1e/d25/d3c/l41 0 2024-04-02T04:42:50.478 INFO:tasks.workunit.client.1.smithi027.stdout:5/364: creat d0/d1a/d6f/f89 x:0 0 0 2024-04-02T04:42:50.482 INFO:tasks.workunit.client.1.smithi027.stdout:6/262: truncate d13/d1b/d24/d28/f55 2640077 0 2024-04-02T04:42:50.483 INFO:tasks.workunit.client.1.smithi027.stdout:1/241: dread d0/fb [0,4194304] 0 2024-04-02T04:42:50.484 INFO:tasks.workunit.client.1.smithi027.stdout:8/266: symlink d5/d8/l59 0 2024-04-02T04:42:50.485 INFO:tasks.workunit.client.1.smithi027.stdout:8/267: chown d5/d8/l4d 169498 1 2024-04-02T04:42:50.486 INFO:tasks.workunit.client.1.smithi027.stdout:8/268: chown d5/d8/d15/f37 15497968 1 2024-04-02T04:42:50.492 INFO:tasks.workunit.client.1.smithi027.stdout:4/235: rename d24/d38/d44/f37 to d24/f46 0 2024-04-02T04:42:50.493 INFO:tasks.workunit.client.1.smithi027.stdout:0/314: creat d4/d16/d1b/f65 x:0 0 0 2024-04-02T04:42:50.495 INFO:tasks.workunit.client.1.smithi027.stdout:7/284: creat d2/f52 x:0 0 0 2024-04-02T04:42:50.497 INFO:tasks.workunit.client.1.smithi027.stdout:2/302: creat d1/db/d3b/d17/d2b/d2f/f64 x:0 0 0 2024-04-02T04:42:50.498 INFO:tasks.workunit.client.1.smithi027.stdout:2/303: dread - d1/db/d3b/f51 zero size 2024-04-02T04:42:50.502 INFO:tasks.workunit.client.1.smithi027.stdout:5/365: rmdir d0/df 39 2024-04-02T04:42:50.505 INFO:tasks.workunit.client.1.smithi027.stdout:8/269: dread d5/d8/db/df/f22 [0,4194304] 0 2024-04-02T04:42:50.506 INFO:tasks.workunit.client.1.smithi027.stdout:8/270: stat d5/l16 0 2024-04-02T04:42:50.507 INFO:tasks.workunit.client.1.smithi027.stdout:6/263: unlink d13/f18 0 2024-04-02T04:42:50.512 INFO:tasks.workunit.client.1.smithi027.stdout:3/208: getdents d1/de/df 0 2024-04-02T04:42:50.515 INFO:tasks.workunit.client.1.smithi027.stdout:4/236: mkdir d24/d38/d47 0 2024-04-02T04:42:50.517 INFO:tasks.workunit.client.1.smithi027.stdout:7/285: fdatasync d2/df/f10 0 2024-04-02T04:42:50.518 INFO:tasks.workunit.client.1.smithi027.stdout:7/286: chown d2/d7/da/dc/d1f/f47 1 1 2024-04-02T04:42:50.519 INFO:tasks.workunit.client.1.smithi027.stdout:2/304: creat d1/db/d3b/d17/d12/f65 x:0 0 0 2024-04-02T04:42:50.524 INFO:tasks.workunit.client.1.smithi027.stdout:7/287: dread d2/d1d/f25 [0,4194304] 0 2024-04-02T04:42:50.531 INFO:tasks.workunit.client.1.smithi027.stdout:7/288: dwrite d2/d7/da/dc/d16/d2f/f27 [0,4194304] 0 2024-04-02T04:42:50.536 INFO:tasks.workunit.client.1.smithi027.stdout:9/296: rename d1a/d1d/d29/d37/d20/c34 to d1a/d1d/d29/d37/d4c/c6b 0 2024-04-02T04:42:50.537 INFO:tasks.workunit.client.1.smithi027.stdout:9/297: write d1a/f63 [1079152,46033] 0 2024-04-02T04:42:50.541 INFO:tasks.workunit.client.0.smithi016.stdout:1/761: chown d14/d19/d1d/d45/d4f/d86/fbb 10092362 1 2024-04-02T04:42:50.542 INFO:tasks.workunit.client.0.smithi016.stdout:2/857: symlink d12/d35/l125 0 2024-04-02T04:42:50.542 INFO:tasks.workunit.client.0.smithi016.stdout:5/761: mkdir d3/d40/d4f/d67/dbf/d111 0 2024-04-02T04:42:50.543 INFO:tasks.workunit.client.0.smithi016.stdout:5/762: chown d3/d40/d4f/d6b/d45/da6/d102/dc4/f105 302678208 1 2024-04-02T04:42:50.547 INFO:tasks.workunit.client.1.smithi027.stdout:5/366: creat d0/d5/d55/f8a x:0 0 0 2024-04-02T04:42:50.552 INFO:tasks.workunit.client.0.smithi016.stdout:7/779: link d0/d46/d37/dbe/dae/d9d/fe2 d0/d46/d16/dc2/dea/d64/f10b 0 2024-04-02T04:42:50.552 INFO:tasks.workunit.client.1.smithi027.stdout:1/242: write d0/f7 [1616262,122233] 0 2024-04-02T04:42:50.555 INFO:tasks.workunit.client.1.smithi027.stdout:3/209: unlink d1/c2c 0 2024-04-02T04:42:50.558 INFO:tasks.workunit.client.1.smithi027.stdout:8/271: rename d5/d8/l59 to d5/d8/db/l5a 0 2024-04-02T04:42:50.559 INFO:tasks.workunit.client.1.smithi027.stdout:8/272: dread - d5/d8/db/f29 zero size 2024-04-02T04:42:50.563 INFO:tasks.workunit.client.1.smithi027.stdout:4/237: mkdir d24/d48 0 2024-04-02T04:42:50.563 INFO:tasks.workunit.client.1.smithi027.stdout:0/315: truncate d4/d16/f17 2380365 0 2024-04-02T04:42:50.567 INFO:tasks.workunit.client.0.smithi016.stdout:1/762: creat d14/d19/d1e/d5e/d75/ffb x:0 0 0 2024-04-02T04:42:50.569 INFO:tasks.workunit.client.1.smithi027.stdout:2/305: write d1/db/f32 [1606867,127778] 0 2024-04-02T04:42:50.570 INFO:tasks.workunit.client.0.smithi016.stdout:7/780: truncate d0/d46/d16/d49/f70 5019595 0 2024-04-02T04:42:50.571 INFO:tasks.workunit.client.1.smithi027.stdout:2/306: write d1/db/d3b/d17/d2b/f52 [1105929,129020] 0 2024-04-02T04:42:50.574 INFO:tasks.workunit.client.0.smithi016.stdout:0/768: rename d0/d16/d24/d62/d9c/f5b to d0/f105 0 2024-04-02T04:42:50.576 INFO:tasks.workunit.client.1.smithi027.stdout:2/307: dread - d1/db/d3b/d17/d2b/f26 zero size 2024-04-02T04:42:50.580 INFO:tasks.workunit.client.1.smithi027.stdout:6/264: rename d13/d1b/d24/d28/d46/l4c to d13/d1b/d48/d58/l5a 0 2024-04-02T04:42:50.580 INFO:tasks.workunit.client.0.smithi016.stdout:3/880: getdents d2/d15/d70/dcf 0 2024-04-02T04:42:50.582 INFO:tasks.workunit.client.1.smithi027.stdout:8/273: rmdir d5/d7/d27 39 2024-04-02T04:42:50.583 INFO:tasks.workunit.client.1.smithi027.stdout:8/274: chown d5/d8/d15/f39 397 1 2024-04-02T04:42:50.585 INFO:tasks.workunit.client.0.smithi016.stdout:1/763: mknod d14/da9/cfc 0 2024-04-02T04:42:50.590 INFO:tasks.workunit.client.0.smithi016.stdout:7/781: mknod d0/d46/d37/d47/c10c 0 2024-04-02T04:42:50.591 INFO:tasks.workunit.client.0.smithi016.stdout:5/763: write d3/f1a [4059919,97350] 0 2024-04-02T04:42:50.595 INFO:tasks.workunit.client.1.smithi027.stdout:5/367: dread d0/df/d2f/d3b/f3d [0,4194304] 0 2024-04-02T04:42:50.597 INFO:tasks.workunit.client.1.smithi027.stdout:1/243: write d0/f6 [4539217,85260] 0 2024-04-02T04:42:50.601 INFO:tasks.workunit.client.0.smithi016.stdout:9/867: link d0/d4a/d75/d95/cfd d0/d4e/d50/d63/d100/c10e 0 2024-04-02T04:42:50.603 INFO:tasks.workunit.client.0.smithi016.stdout:3/881: fdatasync d2/f3e 0 2024-04-02T04:42:50.604 INFO:tasks.workunit.client.0.smithi016.stdout:3/882: chown d2/l10a 164125 1 2024-04-02T04:42:50.606 INFO:tasks.workunit.client.0.smithi016.stdout:7/782: truncate d0/d46/d37/dbe/fde 552511 0 2024-04-02T04:42:50.609 INFO:tasks.workunit.client.0.smithi016.stdout:4/788: rename d5/d16/d88/db0 to d5/d17/d18/d6d/da4/db7/dee 0 2024-04-02T04:42:50.609 INFO:tasks.workunit.client.0.smithi016.stdout:0/769: write d0/d16/d24/d4e/d8d/fc2 [3456650,121321] 0 2024-04-02T04:42:50.610 INFO:tasks.workunit.client.0.smithi016.stdout:5/764: fdatasync d3/d40/d4f/d6b/d45/da6/d102/dc4/fce 0 2024-04-02T04:42:50.612 INFO:tasks.workunit.client.1.smithi027.stdout:9/298: rename d1a/d1d/d29/d37/c2f to d1a/d1d/d29/d50/c6c 0 2024-04-02T04:42:50.613 INFO:tasks.workunit.client.1.smithi027.stdout:9/299: fsync d1a/d1d/d29/d37/f35 0 2024-04-02T04:42:50.614 INFO:tasks.workunit.client.0.smithi016.stdout:8/772: creat d1/d6/d36/d4f/f101 x:0 0 0 2024-04-02T04:42:50.615 INFO:tasks.workunit.client.0.smithi016.stdout:8/773: stat d1/d6/d36/d3f/d40/d49/l26 0 2024-04-02T04:42:50.615 INFO:tasks.workunit.client.1.smithi027.stdout:8/275: truncate d5/d8/db/df/f22 3719622 0 2024-04-02T04:42:50.616 INFO:tasks.workunit.client.0.smithi016.stdout:9/868: chown d0/d4a/d99/d46/fb7 3 1 2024-04-02T04:42:50.617 INFO:tasks.workunit.client.1.smithi027.stdout:4/238: mkdir d24/d38/d44/d41/d49 0 2024-04-02T04:42:50.621 INFO:tasks.workunit.client.0.smithi016.stdout:1/764: symlink d14/d19/lfd 0 2024-04-02T04:42:50.623 INFO:tasks.workunit.client.0.smithi016.stdout:7/783: dread d0/d21/fed [0,4194304] 0 2024-04-02T04:42:50.623 INFO:tasks.workunit.client.0.smithi016.stdout:3/883: dread d2/d2e/f67 [0,4194304] 0 2024-04-02T04:42:50.626 INFO:tasks.workunit.client.1.smithi027.stdout:5/368: mknod d0/df/d11/d41/c8b 0 2024-04-02T04:42:50.626 INFO:tasks.workunit.client.1.smithi027.stdout:6/265: dwrite d13/f1a [0,4194304] 0 2024-04-02T04:42:50.626 INFO:tasks.workunit.client.0.smithi016.stdout:0/770: rmdir d0/d16/dd6/ddd 39 2024-04-02T04:42:50.627 INFO:tasks.workunit.client.1.smithi027.stdout:6/266: read - d13/d1b/d24/d28/d46/f59 zero size 2024-04-02T04:42:50.629 INFO:tasks.workunit.client.1.smithi027.stdout:3/210: creat d1/de/f40 x:0 0 0 2024-04-02T04:42:50.631 INFO:tasks.workunit.client.1.smithi027.stdout:3/211: dread - d1/d1a/f3f zero size 2024-04-02T04:42:50.631 INFO:tasks.workunit.client.0.smithi016.stdout:8/774: write d1/d6/d36/f37 [3675815,21415] 0 2024-04-02T04:42:50.632 INFO:tasks.workunit.client.1.smithi027.stdout:5/369: dread d0/df/d2f/d3b/f5b [0,4194304] 0 2024-04-02T04:42:50.632 INFO:tasks.workunit.client.1.smithi027.stdout:3/212: chown d1/f10 838224 1 2024-04-02T04:42:50.633 INFO:tasks.workunit.client.1.smithi027.stdout:3/213: fdatasync d1/de/f40 0 2024-04-02T04:42:50.635 INFO:tasks.workunit.client.1.smithi027.stdout:2/308: rename d1/db/d3b/d17/d12/f1f to d1/db/d3b/d61/f66 0 2024-04-02T04:42:50.640 INFO:tasks.workunit.client.1.smithi027.stdout:5/370: dwrite d0/d1d/f79 [0,4194304] 0 2024-04-02T04:42:50.641 INFO:tasks.workunit.client.1.smithi027.stdout:8/276: fdatasync d5/d7/d27/d31/f32 0 2024-04-02T04:42:50.642 INFO:tasks.workunit.client.0.smithi016.stdout:1/765: mknod d14/d19/d50/cfe 0 2024-04-02T04:42:50.644 INFO:tasks.workunit.client.0.smithi016.stdout:4/789: dwrite d5/d1d/d6c/f72 [0,4194304] 0 2024-04-02T04:42:50.645 INFO:tasks.workunit.client.0.smithi016.stdout:7/784: stat d0/l88 0 2024-04-02T04:42:50.645 INFO:tasks.workunit.client.0.smithi016.stdout:1/766: chown d14/d19/d1d/d29/ff7 106214206 1 2024-04-02T04:42:50.648 INFO:tasks.workunit.client.1.smithi027.stdout:1/244: mknod d0/c55 0 2024-04-02T04:42:50.652 INFO:tasks.workunit.client.0.smithi016.stdout:3/884: write d2/d2e/d36/d66/d90/fef [4612519,34504] 0 2024-04-02T04:42:50.653 INFO:tasks.workunit.client.1.smithi027.stdout:6/267: unlink d13/d1b/d24/d28/d46/l4b 0 2024-04-02T04:42:50.653 INFO:tasks.workunit.client.0.smithi016.stdout:7/785: dwrite d0/d46/f9a [4194304,4194304] 0 2024-04-02T04:42:50.655 INFO:tasks.workunit.client.0.smithi016.stdout:0/771: dread d0/d16/d2e/d37/f3b [0,4194304] 0 2024-04-02T04:42:50.655 INFO:tasks.workunit.client.0.smithi016.stdout:7/786: readlink d0/d46/d52/d82/lc0 0 2024-04-02T04:42:50.657 INFO:tasks.workunit.client.1.smithi027.stdout:3/214: symlink d1/de/d3a/l41 0 2024-04-02T04:42:50.665 INFO:tasks.workunit.client.1.smithi027.stdout:2/309: rename d1/db/d3b/d17/d2b/d2f/l5e to d1/db/d3b/d17/d29/l67 0 2024-04-02T04:42:50.667 INFO:tasks.workunit.client.1.smithi027.stdout:3/215: dwrite d1/de/df/d13/f35 [0,4194304] 0 2024-04-02T04:42:50.668 INFO:tasks.workunit.client.0.smithi016.stdout:4/790: mkdir d5/d17/dcd/def 0 2024-04-02T04:42:50.670 INFO:tasks.workunit.client.1.smithi027.stdout:5/371: mkdir d0/df/d11/d8c 0 2024-04-02T04:42:50.672 INFO:tasks.workunit.client.1.smithi027.stdout:8/277: dread d5/d8/db/dd/f10 [0,4194304] 0 2024-04-02T04:42:50.673 INFO:tasks.workunit.client.0.smithi016.stdout:1/767: unlink d14/d19/d1d/d29/d39/d46/dcb/d43/d68/ff9 0 2024-04-02T04:42:50.676 INFO:tasks.workunit.client.1.smithi027.stdout:0/316: getdents d4 0 2024-04-02T04:42:50.677 INFO:tasks.workunit.client.0.smithi016.stdout:3/885: creat d2/d15/d16/d57/d96/f129 x:0 0 0 2024-04-02T04:42:50.678 INFO:tasks.workunit.client.1.smithi027.stdout:1/245: mknod d0/d1e/d2e/d3e/c56 0 2024-04-02T04:42:50.679 INFO:tasks.workunit.client.1.smithi027.stdout:1/246: readlink d0/d1e/d25/d3c/l43 0 2024-04-02T04:42:50.679 INFO:tasks.workunit.client.0.smithi016.stdout:7/787: rmdir d0/d46/d52/d82 39 2024-04-02T04:42:50.681 INFO:tasks.workunit.client.0.smithi016.stdout:0/772: creat d0/d16/d2e/d53/da4/de3/f106 x:0 0 0 2024-04-02T04:42:50.683 INFO:tasks.workunit.client.0.smithi016.stdout:8/775: creat d1/d6/d36/d3f/d40/d49/d4a/dfc/f102 x:0 0 0 2024-04-02T04:42:50.684 INFO:tasks.workunit.client.0.smithi016.stdout:8/776: readlink d1/d6/d36/d4f/dc9/ld6 0 2024-04-02T04:42:50.686 INFO:tasks.workunit.client.1.smithi027.stdout:1/247: dwrite d0/d1e/f3a [0,4194304] 0 2024-04-02T04:42:50.686 INFO:tasks.workunit.client.0.smithi016.stdout:1/768: dread - d14/d19/d1d/d45/d4f/faa zero size 2024-04-02T04:42:50.687 INFO:tasks.workunit.client.1.smithi027.stdout:1/248: chown d0/d1e/f3a 1360557 1 2024-04-02T04:42:50.688 INFO:tasks.workunit.client.1.smithi027.stdout:5/372: fdatasync d0/df/d2f/d3b/f5b 0 2024-04-02T04:42:50.688 INFO:tasks.workunit.client.0.smithi016.stdout:5/765: getdents d3/d40/d4f/d6b/d63 0 2024-04-02T04:42:50.689 INFO:tasks.workunit.client.1.smithi027.stdout:5/373: fdatasync d0/df/d76/f85 0 2024-04-02T04:42:50.690 INFO:tasks.workunit.client.1.smithi027.stdout:8/278: mkdir d5/d7/d27/d31/d5b 0 2024-04-02T04:42:50.691 INFO:tasks.workunit.client.0.smithi016.stdout:3/886: read - d2/d15/d16/d8a/f93 zero size 2024-04-02T04:42:50.691 INFO:tasks.workunit.client.0.smithi016.stdout:3/887: stat d2/d15/d16/d8a/dd1/d8b/lc1 0 2024-04-02T04:42:50.694 INFO:tasks.workunit.client.1.smithi027.stdout:0/317: write d4/d16/d1b/f65 [238110,58073] 0 2024-04-02T04:42:50.695 INFO:tasks.workunit.client.1.smithi027.stdout:5/374: dread d0/df/d2f/d3b/f64 [0,4194304] 0 2024-04-02T04:42:50.696 INFO:tasks.workunit.client.0.smithi016.stdout:7/788: rename d0/d46/d16/dc2/dea/l3f to d0/d46/d37/dbe/dae/df8/dec/l10d 0 2024-04-02T04:42:50.699 INFO:tasks.workunit.client.0.smithi016.stdout:8/777: creat d1/f103 x:0 0 0 2024-04-02T04:42:50.701 INFO:tasks.workunit.client.0.smithi016.stdout:4/791: fdatasync d5/d1d/d1e/f36 0 2024-04-02T04:42:50.702 INFO:tasks.workunit.client.1.smithi027.stdout:2/310: mkdir d1/db/d3b/d17/d68 0 2024-04-02T04:42:50.708 INFO:tasks.workunit.client.1.smithi027.stdout:1/249: creat d0/d1e/d25/d2b/f57 x:0 0 0 2024-04-02T04:42:50.711 INFO:tasks.workunit.client.0.smithi016.stdout:1/769: rename d14/d19/d1e/d71/d9c/cba to d14/d19/d1e/d5e/cff 0 2024-04-02T04:42:50.711 INFO:tasks.workunit.client.0.smithi016.stdout:9/869: dwrite d0/d4e/d50/d63/d6e/d4b/d58/d6a/f9f [0,4194304] 0 2024-04-02T04:42:50.712 INFO:tasks.workunit.client.1.smithi027.stdout:8/279: mkdir d5/d8/d5c 0 2024-04-02T04:42:50.714 INFO:tasks.workunit.client.1.smithi027.stdout:8/280: truncate d5/d8/d15/f3e 964031 0 2024-04-02T04:42:50.719 INFO:tasks.workunit.client.1.smithi027.stdout:0/318: rename d4/d16/d2b to d4/d38/d66 0 2024-04-02T04:42:50.721 INFO:tasks.workunit.client.1.smithi027.stdout:1/250: dread d0/f5 [0,4194304] 0 2024-04-02T04:42:50.723 INFO:tasks.workunit.client.0.smithi016.stdout:6/840: sync 2024-04-02T04:42:50.723 INFO:tasks.workunit.client.0.smithi016.stdout:6/841: stat d0/d9/d16/d46/dbe 0 2024-04-02T04:42:50.724 INFO:tasks.workunit.client.0.smithi016.stdout:6/842: dread - d0/d9/d16/d56/d89/d67/d6f/fdc zero size 2024-04-02T04:42:50.725 INFO:tasks.workunit.client.0.smithi016.stdout:6/843: dread - d0/d9/d16/d46/d5e/ff8 zero size 2024-04-02T04:42:50.727 INFO:tasks.workunit.client.0.smithi016.stdout:6/844: write d0/d9/d16/d56/d89/d67/d6f/f82 [1491442,128426] 0 2024-04-02T04:42:50.727 INFO:tasks.workunit.client.1.smithi027.stdout:3/216: creat d1/de/f42 x:0 0 0 2024-04-02T04:42:50.730 INFO:tasks.workunit.client.0.smithi016.stdout:1/770: symlink d14/d19/d1d/d45/d4f/d86/db4/l100 0 2024-04-02T04:42:50.731 INFO:tasks.workunit.client.0.smithi016.stdout:9/870: mknod d0/d4e/d50/d63/d6e/d4b/d58/dd5/c10f 0 2024-04-02T04:42:50.733 INFO:tasks.workunit.client.1.smithi027.stdout:0/319: mknod d4/d38/d66/d2d/c67 0 2024-04-02T04:42:50.733 INFO:tasks.workunit.client.1.smithi027.stdout:6/268: write d13/d1b/d24/d28/f2a [438562,116003] 0 2024-04-02T04:42:50.736 INFO:tasks.workunit.client.1.smithi027.stdout:1/251: creat d0/de/f58 x:0 0 0 2024-04-02T04:42:50.738 INFO:tasks.workunit.client.0.smithi016.stdout:5/766: creat d3/df/d18/f112 x:0 0 0 2024-04-02T04:42:50.738 INFO:tasks.workunit.client.0.smithi016.stdout:3/888: write d2/d2e/d36/d66/d90/dfb/f110 [822852,98189] 0 2024-04-02T04:42:50.739 INFO:tasks.workunit.client.1.smithi027.stdout:2/311: mkdir d1/db/d3b/d17/d68/d69 0 2024-04-02T04:42:50.739 INFO:tasks.workunit.client.0.smithi016.stdout:5/767: stat d3/d40/d4f/d6b/d45/da6/d102/dc4/d108/fe2 0 2024-04-02T04:42:50.741 INFO:tasks.workunit.client.1.smithi027.stdout:6/269: dwrite d13/d1b/d24/d25/d27/f3f [0,4194304] 0 2024-04-02T04:42:50.741 INFO:tasks.workunit.client.1.smithi027.stdout:2/312: stat d1/db/d3b/d17/c55 0 2024-04-02T04:42:50.742 INFO:tasks.workunit.client.1.smithi027.stdout:3/217: creat d1/de/df/d13/f43 x:0 0 0 2024-04-02T04:42:50.743 INFO:tasks.workunit.client.1.smithi027.stdout:6/270: chown d13/d1b/d24/d28/d46/f59 4313647 1 2024-04-02T04:42:50.743 INFO:tasks.workunit.client.1.smithi027.stdout:3/218: stat d1/de/f40 0 2024-04-02T04:42:50.744 INFO:tasks.workunit.client.0.smithi016.stdout:6/845: creat d0/d6e/f107 x:0 0 0 2024-04-02T04:42:50.744 INFO:tasks.workunit.client.1.smithi027.stdout:6/271: stat d13/d1b/d24/d25/d27/l57 0 2024-04-02T04:42:50.746 INFO:tasks.workunit.client.0.smithi016.stdout:1/771: rmdir d14/d19/d1d/d29/d39/d46/dcb/d43/d68/dc1 39 2024-04-02T04:42:50.747 INFO:tasks.workunit.client.1.smithi027.stdout:8/281: getdents d5/d8/db/d53 0 2024-04-02T04:42:50.747 INFO:tasks.workunit.client.1.smithi027.stdout:5/375: write d0/df/d11/d37/f4e [1241382,13924] 0 2024-04-02T04:42:50.748 INFO:tasks.workunit.client.0.smithi016.stdout:9/871: symlink d0/d3/d97/dac/l110 0 2024-04-02T04:42:50.749 INFO:tasks.workunit.client.1.smithi027.stdout:8/282: dread - d5/d8/db/dd/f4f zero size 2024-04-02T04:42:50.749 INFO:tasks.workunit.client.1.smithi027.stdout:5/376: stat d0/df/d11/d41/c54 0 2024-04-02T04:42:50.751 INFO:tasks.workunit.client.0.smithi016.stdout:7/789: dwrite d0/d46/f30 [0,4194304] 0 2024-04-02T04:42:50.751 INFO:tasks.workunit.client.1.smithi027.stdout:1/252: mkdir d0/d1e/d25/d2b/d59 0 2024-04-02T04:42:50.752 INFO:tasks.workunit.client.0.smithi016.stdout:8/778: write d1/d70/d71/d9e/fcd [630734,88765] 0 2024-04-02T04:42:50.753 INFO:tasks.workunit.client.0.smithi016.stdout:4/792: creat d5/d1d/d1e/ff0 x:0 0 0 2024-04-02T04:42:50.756 INFO:tasks.workunit.client.0.smithi016.stdout:3/889: rename d2/d15/d60/f127 to d2/d2e/dee/f12a 0 2024-04-02T04:42:50.757 INFO:tasks.workunit.client.1.smithi027.stdout:3/219: mknod d1/de/df/d13/c44 0 2024-04-02T04:42:50.760 INFO:tasks.workunit.client.1.smithi027.stdout:6/272: readlink d13/d1b/d48/d58/l5a 0 2024-04-02T04:42:50.763 INFO:tasks.workunit.client.1.smithi027.stdout:6/273: write d13/d1b/d24/d28/f2a [3449380,127114] 0 2024-04-02T04:42:50.764 INFO:tasks.workunit.client.1.smithi027.stdout:6/274: fsync f12 0 2024-04-02T04:42:50.766 INFO:tasks.workunit.client.1.smithi027.stdout:6/275: chown d13/l49 376 1 2024-04-02T04:42:50.766 INFO:tasks.workunit.client.1.smithi027.stdout:3/220: dwrite d1/de/f38 [0,4194304] 0 2024-04-02T04:42:50.770 INFO:tasks.workunit.client.1.smithi027.stdout:8/283: unlink d5/d8/d48/f58 0 2024-04-02T04:42:50.774 INFO:tasks.workunit.client.1.smithi027.stdout:5/377: creat d0/df/d76/f8d x:0 0 0 2024-04-02T04:42:50.776 INFO:tasks.workunit.client.1.smithi027.stdout:1/253: mknod d0/d1e/c5a 0 2024-04-02T04:42:50.779 INFO:tasks.workunit.client.1.smithi027.stdout:1/254: read d0/f14 [4053898,87501] 0 2024-04-02T04:42:50.782 INFO:tasks.workunit.client.1.smithi027.stdout:1/255: read d0/f6 [1903722,62131] 0 2024-04-02T04:42:50.783 INFO:tasks.workunit.client.1.smithi027.stdout:1/256: stat d0/de/f4d 0 2024-04-02T04:42:50.783 INFO:tasks.workunit.client.1.smithi027.stdout:0/320: truncate d4/f11 3584026 0 2024-04-02T04:42:50.784 INFO:tasks.workunit.client.1.smithi027.stdout:0/321: fsync d4/d21/d28/f40 0 2024-04-02T04:42:50.785 INFO:tasks.workunit.client.0.smithi016.stdout:6/846: symlink d0/d9/d16/d46/l108 0 2024-04-02T04:42:50.785 INFO:tasks.workunit.client.0.smithi016.stdout:8/779: chown d1/d6/d36/d3f/d77/fa7 14215742 1 2024-04-02T04:42:50.787 INFO:tasks.workunit.client.1.smithi027.stdout:0/322: truncate d4/d38/d66/d2d/f56 449528 0 2024-04-02T04:42:50.787 INFO:tasks.workunit.client.1.smithi027.stdout:3/221: read d1/f8 [4572,34887] 0 2024-04-02T04:42:50.788 INFO:tasks.workunit.client.0.smithi016.stdout:3/890: dread d2/d2e/d36/d48/fe1 [0,4194304] 0 2024-04-02T04:42:50.790 INFO:tasks.workunit.client.1.smithi027.stdout:5/378: dread d0/df/d2f/f67 [0,4194304] 0 2024-04-02T04:42:50.791 INFO:tasks.workunit.client.1.smithi027.stdout:1/257: dread d0/de/f1b [0,4194304] 0 2024-04-02T04:42:50.793 INFO:tasks.workunit.client.0.smithi016.stdout:9/872: dread d0/d9/db/dc9/d53/f73 [0,4194304] 0 2024-04-02T04:42:50.794 INFO:tasks.workunit.client.0.smithi016.stdout:9/873: chown d0/d4e/d50/d63/d6e/c5c 37 1 2024-04-02T04:42:50.795 INFO:tasks.workunit.client.0.smithi016.stdout:8/780: truncate d1/d6/d36/d3f/d40/d49/d38/f90 2283864 0 2024-04-02T04:42:50.795 INFO:tasks.workunit.client.0.smithi016.stdout:9/874: chown d0/d4e/d50/d63/d6e/d4b/d58/d6a/ccb 16376080 1 2024-04-02T04:42:50.796 INFO:tasks.workunit.client.0.smithi016.stdout:5/768: getdents d3/d40/d4f/d6b/d45/da6/d102/dc4 0 2024-04-02T04:42:50.798 INFO:tasks.workunit.client.0.smithi016.stdout:4/793: dread d5/de/f14 [0,4194304] 0 2024-04-02T04:42:50.801 INFO:tasks.workunit.client.0.smithi016.stdout:6/847: write d0/d9/d16/d46/f7b [1198071,25063] 0 2024-04-02T04:42:50.802 INFO:tasks.workunit.client.0.smithi016.stdout:6/848: read - d0/d9/d16/d46/d5e/ff8 zero size 2024-04-02T04:42:50.803 INFO:tasks.workunit.client.1.smithi027.stdout:6/276: symlink d13/d1b/d24/d25/d56/l5b 0 2024-04-02T04:42:50.806 INFO:tasks.workunit.client.1.smithi027.stdout:8/284: dwrite d5/d8/f47 [0,4194304] 0 2024-04-02T04:42:50.808 INFO:tasks.workunit.client.0.smithi016.stdout:6/849: dwrite d0/d9/f36 [4194304,4194304] 0 2024-04-02T04:42:50.809 INFO:tasks.workunit.client.1.smithi027.stdout:0/323: unlink d4/d38/d66/d2d/l34 0 2024-04-02T04:42:50.813 INFO:tasks.workunit.client.1.smithi027.stdout:5/379: dread d0/df/d11/f1f [0,4194304] 0 2024-04-02T04:42:50.814 INFO:tasks.workunit.client.1.smithi027.stdout:6/277: dwrite d13/d1b/f20 [4194304,4194304] 0 2024-04-02T04:42:50.816 INFO:tasks.workunit.client.0.smithi016.stdout:5/769: rename d3/d40/d4f/d67/dbf/ff0 to d3/d40/d4f/d6b/d45/da6/d102/dc4/f113 0 2024-04-02T04:42:50.817 INFO:tasks.workunit.client.0.smithi016.stdout:3/891: write d2/d15/d16/d8a/dd1/d6e/fbd [258984,76268] 0 2024-04-02T04:42:50.819 INFO:tasks.workunit.client.0.smithi016.stdout:4/794: mkdir d5/d16/d3f/df1 0 2024-04-02T04:42:50.819 INFO:tasks.workunit.client.0.smithi016.stdout:3/892: chown d2/d15/d16/d8a/dd1/d8b/dd2/cdd 1737 1 2024-04-02T04:42:50.820 INFO:tasks.workunit.client.0.smithi016.stdout:4/795: write d5/d17/d18/d6d/d91/fe2 [506225,71597] 0 2024-04-02T04:42:50.824 INFO:tasks.workunit.client.0.smithi016.stdout:8/781: write d1/d6/d36/d3f/d40/d45/fee [1098666,55492] 0 2024-04-02T04:42:50.825 INFO:tasks.workunit.client.1.smithi027.stdout:6/278: dwrite d13/d1b/d24/d25/d27/f44 [0,4194304] 0 2024-04-02T04:42:50.826 INFO:tasks.workunit.client.0.smithi016.stdout:0/773: sync 2024-04-02T04:42:50.826 INFO:tasks.workunit.client.0.smithi016.stdout:2/858: sync 2024-04-02T04:42:50.829 INFO:tasks.workunit.client.0.smithi016.stdout:6/850: mknod d0/d6e/c109 0 2024-04-02T04:42:50.834 INFO:tasks.workunit.client.1.smithi027.stdout:8/285: dread d5/d8/d15/f39 [0,4194304] 0 2024-04-02T04:42:50.838 INFO:tasks.workunit.client.1.smithi027.stdout:0/324: write f1 [831990,85548] 0 2024-04-02T04:42:50.839 INFO:tasks.workunit.client.0.smithi016.stdout:4/796: mkdir d5/d1d/d5f/d95/df2 0 2024-04-02T04:42:50.844 INFO:tasks.workunit.client.0.smithi016.stdout:9/875: write d0/d9/db/dc9/d53/f9d [1463444,42184] 0 2024-04-02T04:42:50.845 INFO:tasks.workunit.client.1.smithi027.stdout:3/222: truncate d1/f32 2154075 0 2024-04-02T04:42:50.845 INFO:tasks.workunit.client.0.smithi016.stdout:2/859: unlink d12/d35/d101/d76/da0/c120 0 2024-04-02T04:42:50.847 INFO:tasks.workunit.client.1.smithi027.stdout:5/380: creat d0/df/d2f/d3b/d51/f8e x:0 0 0 2024-04-02T04:42:50.849 INFO:tasks.workunit.client.0.smithi016.stdout:0/774: fdatasync d0/d16/d2e/d37/d66/fa3 0 2024-04-02T04:42:50.849 INFO:tasks.workunit.client.0.smithi016.stdout:5/770: write d3/df/d18/f35 [3737947,46482] 0 2024-04-02T04:42:50.849 INFO:tasks.workunit.client.0.smithi016.stdout:0/775: readlink d0/d4f/d76/l89 0 2024-04-02T04:42:50.850 INFO:tasks.workunit.client.0.smithi016.stdout:4/797: dread d5/d17/d18/d6d/da4/fad [0,4194304] 0 2024-04-02T04:42:50.851 INFO:tasks.workunit.client.0.smithi016.stdout:6/851: mknod d0/d9/d16/d56/d89/d67/d6f/d8a/d91/c10a 0 2024-04-02T04:42:50.852 INFO:tasks.workunit.client.0.smithi016.stdout:6/852: readlink d0/d9/d16/d56/d89/db4/lfe 0 2024-04-02T04:42:50.852 INFO:tasks.workunit.client.1.smithi027.stdout:1/258: symlink d0/de/d52/l5b 0 2024-04-02T04:42:50.853 INFO:tasks.workunit.client.1.smithi027.stdout:1/259: stat d0/d1e/d2e/d3b 0 2024-04-02T04:42:50.855 INFO:tasks.workunit.client.0.smithi016.stdout:5/771: dwrite d3/df/f80 [0,4194304] 0 2024-04-02T04:42:50.857 INFO:tasks.workunit.client.0.smithi016.stdout:8/782: mknod d1/d6/d36/d3f/d40/d45/c104 0 2024-04-02T04:42:50.860 INFO:tasks.workunit.client.1.smithi027.stdout:1/260: dwrite d0/de/f17 [0,4194304] 0 2024-04-02T04:42:50.861 INFO:tasks.workunit.client.0.smithi016.stdout:9/876: creat d0/d4a/d75/d95/d9b/f111 x:0 0 0 2024-04-02T04:42:50.864 INFO:tasks.workunit.client.0.smithi016.stdout:2/860: rmdir d12/d35/d3b/d42/d59 39 2024-04-02T04:42:50.865 INFO:tasks.workunit.client.0.smithi016.stdout:2/861: chown d12/d35/d3f/d2b/f3d 1345 1 2024-04-02T04:42:50.871 INFO:tasks.workunit.client.1.smithi027.stdout:6/279: dwrite d13/d1b/f1e [0,4194304] 0 2024-04-02T04:42:50.871 INFO:tasks.workunit.client.0.smithi016.stdout:0/776: mkdir d0/d16/d24/d4e/d8d/d107 0 2024-04-02T04:42:50.873 INFO:tasks.workunit.client.0.smithi016.stdout:4/798: rmdir d5/d17/d18/d6d/d91 39 2024-04-02T04:42:50.875 INFO:tasks.workunit.client.1.smithi027.stdout:8/286: link d5/d7/d27/f50 d5/d8/d48/f5d 0 2024-04-02T04:42:50.880 INFO:tasks.workunit.client.1.smithi027.stdout:1/261: dread d0/f3d [0,4194304] 0 2024-04-02T04:42:50.882 INFO:tasks.workunit.client.0.smithi016.stdout:3/893: getdents d2/d15/d16/d57 0 2024-04-02T04:42:50.884 INFO:tasks.workunit.client.1.smithi027.stdout:6/280: dread d13/d1b/d24/d25/d27/f51 [0,4194304] 0 2024-04-02T04:42:50.884 INFO:tasks.workunit.client.0.smithi016.stdout:8/783: unlink d1/d6/d36/d3f/d40/d45/dcc/cdc 0 2024-04-02T04:42:50.886 INFO:tasks.workunit.client.0.smithi016.stdout:9/877: mkdir d0/d4e/d50/d63/d6e/d4b/d58/d6a/d112 0 2024-04-02T04:42:50.888 INFO:tasks.workunit.client.0.smithi016.stdout:9/878: dread - d0/d4a/d75/fff zero size 2024-04-02T04:42:50.889 INFO:tasks.workunit.client.0.smithi016.stdout:9/879: stat d0/d4a/d75/d95/cfd 0 2024-04-02T04:42:50.890 INFO:tasks.workunit.client.0.smithi016.stdout:3/894: dwrite d2/d15/d1a/d45/da5/db6/f124 [0,4194304] 0 2024-04-02T04:42:50.890 INFO:tasks.workunit.client.0.smithi016.stdout:9/880: readlink d0/d4a/d99/l49 0 2024-04-02T04:42:50.891 INFO:tasks.workunit.client.1.smithi027.stdout:3/223: link d1/d1a/f22 d1/f45 0 2024-04-02T04:42:50.894 INFO:tasks.workunit.client.0.smithi016.stdout:5/772: mkdir d3/d40/d4f/d67/dbf/d111/d114 0 2024-04-02T04:42:50.895 INFO:tasks.workunit.client.0.smithi016.stdout:4/799: dread d5/d8/f7b [0,4194304] 0 2024-04-02T04:42:50.896 INFO:tasks.workunit.client.0.smithi016.stdout:4/800: chown d5/d17/d18/d6d 317316109 1 2024-04-02T04:42:50.897 INFO:tasks.workunit.client.0.smithi016.stdout:1/772: sync 2024-04-02T04:42:50.897 INFO:tasks.workunit.client.0.smithi016.stdout:7/790: sync 2024-04-02T04:42:50.897 INFO:tasks.workunit.client.0.smithi016.stdout:8/784: rmdir d1/d6/d36/d3f/d40/d49/d38 39 2024-04-02T04:42:50.898 INFO:tasks.workunit.client.0.smithi016.stdout:1/773: fdatasync d14/d19/d1d/d45/d4f/d86/db4/fd3 0 2024-04-02T04:42:50.899 INFO:tasks.workunit.client.1.smithi027.stdout:6/281: symlink d13/d1b/d24/d25/d56/l5c 0 2024-04-02T04:42:50.899 INFO:tasks.workunit.client.0.smithi016.stdout:1/774: readlink d14/d19/d1e/d71/led 0 2024-04-02T04:42:50.900 INFO:tasks.workunit.client.0.smithi016.stdout:1/775: fsync d14/d19/d1e/d71/fbf 0 2024-04-02T04:42:50.900 INFO:tasks.workunit.client.1.smithi027.stdout:8/287: dread d5/d8/f33 [0,4194304] 0 2024-04-02T04:42:50.902 INFO:tasks.workunit.client.0.smithi016.stdout:1/776: stat d14/d19/d1d/d45 0 2024-04-02T04:42:50.902 INFO:tasks.workunit.client.0.smithi016.stdout:6/853: dread d0/d9/d16/d56/d89/f45 [0,4194304] 0 2024-04-02T04:42:50.902 INFO:tasks.workunit.client.0.smithi016.stdout:1/777: dread - d14/d19/ff1 zero size 2024-04-02T04:42:50.902 INFO:tasks.workunit.client.0.smithi016.stdout:2/862: rmdir d12/d35/d3b/d42/d59/d91 39 2024-04-02T04:42:50.902 INFO:tasks.workunit.client.0.smithi016.stdout:6/854: read - d0/faa zero size 2024-04-02T04:42:50.903 INFO:tasks.workunit.client.1.smithi027.stdout:0/325: getdents d4/d38/d66/d2d/d4c 0 2024-04-02T04:42:50.903 INFO:tasks.workunit.client.0.smithi016.stdout:6/855: fdatasync d0/d9/d16/d46/f7b 0 2024-04-02T04:42:50.904 INFO:tasks.workunit.client.0.smithi016.stdout:3/895: read d2/d15/f1b [4222303,33033] 0 2024-04-02T04:42:50.906 INFO:tasks.workunit.client.0.smithi016.stdout:5/773: mkdir d3/d40/db6/de0/d115 0 2024-04-02T04:42:50.907 INFO:tasks.workunit.client.1.smithi027.stdout:5/381: link d0/d1a/d20/c3f d0/df/d11/d37/c8f 0 2024-04-02T04:42:50.907 INFO:tasks.workunit.client.0.smithi016.stdout:5/774: fdatasync d3/d40/d4f/ff4 0 2024-04-02T04:42:50.908 INFO:tasks.workunit.client.1.smithi027.stdout:5/382: truncate d0/d1d/d61/d72/f86 111523 0 2024-04-02T04:42:50.908 INFO:tasks.workunit.client.0.smithi016.stdout:4/801: creat d5/d1d/d1e/da5/ff3 x:0 0 0 2024-04-02T04:42:50.908 INFO:tasks.workunit.client.0.smithi016.stdout:9/881: dread d0/d33/f38 [0,4194304] 0 2024-04-02T04:42:50.909 INFO:tasks.workunit.client.0.smithi016.stdout:9/882: chown d0/d4a/da9 714178555 1 2024-04-02T04:42:50.910 INFO:tasks.workunit.client.0.smithi016.stdout:4/802: write d5/d1d/d1e/da5/ff3 [89137,42991] 0 2024-04-02T04:42:50.910 INFO:tasks.workunit.client.1.smithi027.stdout:6/282: mknod d13/d1b/d48/d58/c5d 0 2024-04-02T04:42:50.910 INFO:tasks.workunit.client.1.smithi027.stdout:3/224: fsync d1/f45 0 2024-04-02T04:42:50.910 INFO:tasks.workunit.client.0.smithi016.stdout:9/883: write d0/d4e/fae [176824,123468] 0 2024-04-02T04:42:50.911 INFO:tasks.workunit.client.0.smithi016.stdout:4/803: chown d5/d17/d76/d7e/c8a 0 1 2024-04-02T04:42:50.912 INFO:tasks.workunit.client.0.smithi016.stdout:4/804: readlink d5/d17/d18/d1f/l3e 0 2024-04-02T04:42:50.912 INFO:tasks.workunit.client.0.smithi016.stdout:8/785: chown d1/d31/d67/fc8 28 1 2024-04-02T04:42:50.912 INFO:tasks.workunit.client.0.smithi016.stdout:9/884: chown d0/d4e/d50/d63/d6e/d4b/d58 29716 1 2024-04-02T04:42:50.917 INFO:tasks.workunit.client.1.smithi027.stdout:0/326: creat d4/d3d/f68 x:0 0 0 2024-04-02T04:42:50.919 INFO:tasks.workunit.client.0.smithi016.stdout:9/885: dwrite d0/d9/db/d54/ffb [0,4194304] 0 2024-04-02T04:42:50.919 INFO:tasks.workunit.client.0.smithi016.stdout:1/778: creat d14/da9/f101 x:0 0 0 2024-04-02T04:42:50.920 INFO:tasks.workunit.client.0.smithi016.stdout:1/779: chown d14/d19/d1d/d29/d39/d46/dcb/d43/f6f 7222 1 2024-04-02T04:42:50.921 INFO:tasks.workunit.client.1.smithi027.stdout:1/262: creat d0/f5c x:0 0 0 2024-04-02T04:42:50.922 INFO:tasks.workunit.client.0.smithi016.stdout:0/777: creat d0/d16/d2e/d37/f108 x:0 0 0 2024-04-02T04:42:50.923 INFO:tasks.workunit.client.0.smithi016.stdout:1/780: dread d14/d19/d1d/d29/d4c/fd2 [0,4194304] 0 2024-04-02T04:42:50.924 INFO:tasks.workunit.client.0.smithi016.stdout:2/863: mkdir d12/d9f/d107/d126 0 2024-04-02T04:42:50.926 INFO:tasks.workunit.client.0.smithi016.stdout:2/864: chown d12/d70/ff3 226052124 1 2024-04-02T04:42:50.927 INFO:tasks.workunit.client.1.smithi027.stdout:8/288: dwrite d5/d7/d27/f3a [4194304,4194304] 0 2024-04-02T04:42:50.927 INFO:tasks.workunit.client.0.smithi016.stdout:7/791: dwrite d0/d46/d52/fc4 [0,4194304] 0 2024-04-02T04:42:50.931 INFO:tasks.workunit.client.1.smithi027.stdout:0/327: mkdir d4/d69 0 2024-04-02T04:42:50.937 INFO:tasks.workunit.client.0.smithi016.stdout:4/805: truncate d5/d17/d18/d6d/d91/fe2 1280067 0 2024-04-02T04:42:50.940 INFO:tasks.workunit.client.1.smithi027.stdout:8/289: creat d5/d7/d27/d31/f5e x:0 0 0 2024-04-02T04:42:50.941 INFO:tasks.workunit.client.1.smithi027.stdout:8/290: write d5/d8/db/dd/f51 [785438,93912] 0 2024-04-02T04:42:50.944 INFO:tasks.workunit.client.0.smithi016.stdout:3/896: write d2/d15/f38 [559683,31777] 0 2024-04-02T04:42:50.945 INFO:tasks.workunit.client.0.smithi016.stdout:0/778: fdatasync d0/d16/d2e/f61 0 2024-04-02T04:42:50.947 INFO:tasks.workunit.client.0.smithi016.stdout:0/779: write d0/d16/d24/d4e/d8d/fc2 [4301522,94580] 0 2024-04-02T04:42:50.949 INFO:tasks.workunit.client.0.smithi016.stdout:6/856: creat d0/d9/d16/d56/d89/d100/f10b x:0 0 0 2024-04-02T04:42:50.950 INFO:tasks.workunit.client.0.smithi016.stdout:8/786: mkdir d1/d6/d36/d3f/d40/d63/d105 0 2024-04-02T04:42:50.950 INFO:tasks.workunit.client.0.smithi016.stdout:4/806: mkdir d5/de/df4 0 2024-04-02T04:42:50.952 INFO:tasks.workunit.client.0.smithi016.stdout:9/886: mkdir d0/d4a/d75/d113 0 2024-04-02T04:42:50.956 INFO:tasks.workunit.client.1.smithi027.stdout:5/383: truncate d0/df/d11/d13/f26 4129847 0 2024-04-02T04:42:50.959 INFO:tasks.workunit.client.1.smithi027.stdout:5/384: unlink d0/d1a/l7b 0 2024-04-02T04:42:50.962 INFO:tasks.workunit.client.1.smithi027.stdout:6/283: dwrite fc [0,4194304] 0 2024-04-02T04:42:50.962 INFO:tasks.workunit.client.1.smithi027.stdout:8/291: rename d5/d7 to d5/d8/db/dd/d5f 0 2024-04-02T04:42:50.964 INFO:tasks.workunit.client.1.smithi027.stdout:8/292: write d5/d8/db/dd/d5f/f14 [672836,107900] 0 2024-04-02T04:42:50.965 INFO:tasks.workunit.client.0.smithi016.stdout:0/780: creat d0/d16/d2e/d53/d9d/f109 x:0 0 0 2024-04-02T04:42:50.965 INFO:tasks.workunit.client.1.smithi027.stdout:3/225: truncate d1/d1a/f22 4466598 0 2024-04-02T04:42:50.966 INFO:tasks.workunit.client.0.smithi016.stdout:0/781: chown d0/d16/f94 1 1 2024-04-02T04:42:50.968 INFO:tasks.workunit.client.0.smithi016.stdout:7/792: mknod d0/d46/d37/dbe/dae/df8/c10e 0 2024-04-02T04:42:50.968 INFO:tasks.workunit.client.1.smithi027.stdout:5/385: rmdir d0/df/d11 39 2024-04-02T04:42:50.970 INFO:tasks.workunit.client.1.smithi027.stdout:0/328: rename d4/d3d/f68 to d4/d16/f6a 0 2024-04-02T04:42:50.971 INFO:tasks.workunit.client.0.smithi016.stdout:8/787: fsync d1/d6/d36/d3f/d77/fd0 0 2024-04-02T04:42:50.973 INFO:tasks.workunit.client.1.smithi027.stdout:6/284: symlink d13/d1b/d24/d28/d3c/l5e 0 2024-04-02T04:42:50.974 INFO:tasks.workunit.client.0.smithi016.stdout:1/781: dwrite d14/d19/d20/f24 [4194304,4194304] 0 2024-04-02T04:42:50.978 INFO:tasks.workunit.client.0.smithi016.stdout:9/887: creat d0/d4a/d75/d95/d9b/db4/f114 x:0 0 0 2024-04-02T04:42:50.979 INFO:tasks.workunit.client.0.smithi016.stdout:9/888: chown d0/d4a/d75/c103 171229236 1 2024-04-02T04:42:50.981 INFO:tasks.workunit.client.1.smithi027.stdout:5/386: readlink d0/df/l39 0 2024-04-02T04:42:50.983 INFO:tasks.workunit.client.0.smithi016.stdout:0/782: unlink d0/d4f/d76/la5 0 2024-04-02T04:42:50.984 INFO:tasks.workunit.client.1.smithi027.stdout:7/289: sync 2024-04-02T04:42:50.986 INFO:tasks.workunit.client.1.smithi027.stdout:7/290: write d2/d1c/f37 [1880601,33935] 0 2024-04-02T04:42:50.987 INFO:tasks.workunit.client.0.smithi016.stdout:6/857: symlink d0/d9/d16/d56/l10c 0 2024-04-02T04:42:50.991 INFO:tasks.workunit.client.0.smithi016.stdout:4/807: dread d5/d16/d3f/f64 [0,4194304] 0 2024-04-02T04:42:51.000 INFO:tasks.workunit.client.1.smithi027.stdout:3/226: dwrite d1/de/df/f17 [0,4194304] 0 2024-04-02T04:42:51.002 INFO:tasks.workunit.client.0.smithi016.stdout:9/889: truncate d0/d4a/d99/dd3/ff5 906610 0 2024-04-02T04:42:51.004 INFO:tasks.workunit.client.0.smithi016.stdout:8/788: write d1/d6/f21 [1437982,77740] 0 2024-04-02T04:42:51.006 INFO:tasks.workunit.client.0.smithi016.stdout:0/783: rename d0/d4f/d76/l89 to d0/d16/d2e/d53/da4/d102/l10a 0 2024-04-02T04:42:51.006 INFO:tasks.workunit.client.1.smithi027.stdout:6/285: dwrite f7 [0,4194304] 0 2024-04-02T04:42:51.008 INFO:tasks.workunit.client.0.smithi016.stdout:1/782: write d14/d19/d1d/d29/f3c [1228281,95110] 0 2024-04-02T04:42:51.009 INFO:tasks.workunit.client.0.smithi016.stdout:7/793: truncate d0/d46/d37/dbe/dae/df8/fb6 2495104 0 2024-04-02T04:42:51.010 INFO:tasks.workunit.client.1.smithi027.stdout:5/387: write d0/d1d/f35 [3795561,1951] 0 2024-04-02T04:42:51.012 INFO:tasks.workunit.client.1.smithi027.stdout:7/291: mkdir d2/d7/da/dc/d16/d3a/d53 0 2024-04-02T04:42:51.014 INFO:tasks.workunit.client.0.smithi016.stdout:4/808: symlink d5/d17/d18/d6d/d8e/d9c/dcc/d6f/dcf/lf5 0 2024-04-02T04:42:51.015 INFO:tasks.workunit.client.1.smithi027.stdout:8/293: rmdir d5/d8/db/d53 0 2024-04-02T04:42:51.015 INFO:tasks.workunit.client.1.smithi027.stdout:6/286: dwrite d13/d1b/f20 [4194304,4194304] 0 2024-04-02T04:42:51.018 INFO:tasks.workunit.client.0.smithi016.stdout:3/897: link d2/d15/ffd d2/d2e/d36/d48/da2/f12b 0 2024-04-02T04:42:51.022 INFO:tasks.workunit.client.0.smithi016.stdout:9/890: mkdir d0/d3/d97/dde/d115 0 2024-04-02T04:42:51.023 INFO:tasks.workunit.client.0.smithi016.stdout:8/789: creat d1/d6/d36/d3f/d77/f106 x:0 0 0 2024-04-02T04:42:51.025 INFO:tasks.workunit.client.1.smithi027.stdout:0/329: dwrite d4/d16/f17 [0,4194304] 0 2024-04-02T04:42:51.028 INFO:tasks.workunit.client.0.smithi016.stdout:1/783: mkdir d14/d19/d1e/d71/dcf/d102 0 2024-04-02T04:42:51.030 INFO:tasks.workunit.client.0.smithi016.stdout:7/794: creat d0/d21/f10f x:0 0 0 2024-04-02T04:42:51.036 INFO:tasks.workunit.client.1.smithi027.stdout:8/294: symlink d5/d8/l60 0 2024-04-02T04:42:51.037 INFO:tasks.workunit.client.1.smithi027.stdout:8/295: dread - d5/d8/db/df/f4c zero size 2024-04-02T04:42:51.037 INFO:tasks.workunit.client.0.smithi016.stdout:3/898: mkdir d2/d2e/d36/d66/d90/d12c 0 2024-04-02T04:42:51.039 INFO:tasks.workunit.client.1.smithi027.stdout:6/287: fsync d13/d1b/d24/d28/f2a 0 2024-04-02T04:42:51.043 INFO:tasks.workunit.client.1.smithi027.stdout:0/330: truncate d4/d21/d28/f44 375159 0 2024-04-02T04:42:51.045 INFO:tasks.workunit.client.1.smithi027.stdout:7/292: write d2/d1c/f40 [233859,109011] 0 2024-04-02T04:42:51.046 INFO:tasks.workunit.client.1.smithi027.stdout:7/293: read - d2/f52 zero size 2024-04-02T04:42:51.048 INFO:tasks.workunit.client.1.smithi027.stdout:7/294: read d2/d7/da/dc/d16/f2d [41579,5721] 0 2024-04-02T04:42:51.048 INFO:tasks.workunit.client.1.smithi027.stdout:8/296: mkdir d5/d8/db/dd/d5f/d27/d31/d61 0 2024-04-02T04:42:51.051 INFO:tasks.workunit.client.1.smithi027.stdout:6/288: chown d13/d1b/d24/d28/d3c/l52 0 1 2024-04-02T04:42:51.051 INFO:tasks.workunit.client.1.smithi027.stdout:6/289: readlink d13/d1b/l26 0 2024-04-02T04:42:51.052 INFO:tasks.workunit.client.1.smithi027.stdout:6/290: fdatasync d13/d1b/d24/f41 0 2024-04-02T04:42:51.052 INFO:tasks.workunit.client.0.smithi016.stdout:4/809: truncate d5/d1d/d1e/f9a 3986402 0 2024-04-02T04:42:51.052 INFO:tasks.workunit.client.1.smithi027.stdout:6/291: fsync fc 0 2024-04-02T04:42:51.053 INFO:tasks.workunit.client.0.smithi016.stdout:3/899: unlink d2/d15/d7e/l10e 0 2024-04-02T04:42:51.055 INFO:tasks.workunit.client.0.smithi016.stdout:8/790: creat d1/d6/d36/d3f/d40/d63/d105/f107 x:0 0 0 2024-04-02T04:42:51.056 INFO:tasks.workunit.client.0.smithi016.stdout:9/891: dwrite d0/d9/f3d [0,4194304] 0 2024-04-02T04:42:51.056 INFO:tasks.workunit.client.1.smithi027.stdout:5/388: link d0/d1a/d20/f59 d0/df/d11/d41/f90 0 2024-04-02T04:42:51.056 INFO:tasks.workunit.client.1.smithi027.stdout:5/389: readlink d0/d5/l60 0 2024-04-02T04:42:51.057 INFO:tasks.workunit.client.0.smithi016.stdout:7/795: dwrite d0/d46/d37/dbe/dae/df8/d43/d96/fba [0,4194304] 0 2024-04-02T04:42:51.057 INFO:tasks.workunit.client.0.smithi016.stdout:3/900: read - d2/d15/d16/d8a/dd1/d6e/d6f/db0/f9e zero size 2024-04-02T04:42:51.058 INFO:tasks.workunit.client.0.smithi016.stdout:8/791: write d1/d31/d67/fd1 [3681849,85466] 0 2024-04-02T04:42:51.059 INFO:tasks.workunit.client.0.smithi016.stdout:3/901: chown d2/d15/f23 8844 1 2024-04-02T04:42:51.059 INFO:tasks.workunit.client.1.smithi027.stdout:8/297: unlink d5/d8/c44 0 2024-04-02T04:42:51.060 INFO:tasks.workunit.client.0.smithi016.stdout:7/796: read d0/d46/d16/d49/f11 [3254023,65479] 0 2024-04-02T04:42:51.060 INFO:tasks.workunit.client.0.smithi016.stdout:3/902: fsync d2/d2e/d36/d48/da2/d88/f125 0 2024-04-02T04:42:51.060 INFO:tasks.workunit.client.0.smithi016.stdout:7/797: chown d0/d46/d52 13524499 1 2024-04-02T04:42:51.061 INFO:tasks.workunit.client.0.smithi016.stdout:7/798: stat d0/d46/d16/d49/df9 0 2024-04-02T04:42:51.063 INFO:tasks.workunit.client.1.smithi027.stdout:7/295: mknod d2/d7/da/dc/d16/d2f/c54 0 2024-04-02T04:42:51.064 INFO:tasks.workunit.client.0.smithi016.stdout:4/810: readlink d5/d17/d18/d6d/da4/lb5 0 2024-04-02T04:42:51.066 INFO:tasks.workunit.client.0.smithi016.stdout:4/811: fdatasync d5/d17/d18/d6d/d8e/d9c/dcc/d6f/f84 0 2024-04-02T04:42:51.068 INFO:tasks.workunit.client.0.smithi016.stdout:7/799: dwrite d0/d46/d37/dbe/ff4 [0,4194304] 0 2024-04-02T04:42:51.069 INFO:tasks.workunit.client.1.smithi027.stdout:8/298: dwrite d5/d8/d15/f37 [0,4194304] 0 2024-04-02T04:42:51.071 INFO:tasks.workunit.client.1.smithi027.stdout:8/299: chown d5/d8/db/dd/d5f/d27/d31/d61 2470 1 2024-04-02T04:42:51.072 INFO:tasks.workunit.client.0.smithi016.stdout:7/800: write d0/d46/d16/dc2/dea/f53 [5091668,93530] 0 2024-04-02T04:42:51.073 INFO:tasks.workunit.client.0.smithi016.stdout:1/784: getdents d14/d19/d1d/d29/d39/d46/dcb/d43/d68/dc1/dfa 0 2024-04-02T04:42:51.073 INFO:tasks.workunit.client.0.smithi016.stdout:7/801: chown d0/d46/d37/dbe/dae/df8/ld8 25 1 2024-04-02T04:42:51.074 INFO:tasks.workunit.client.1.smithi027.stdout:3/227: link d1/fc d1/f46 0 2024-04-02T04:42:51.077 INFO:tasks.workunit.client.1.smithi027.stdout:3/228: truncate d1/de/f42 328570 0 2024-04-02T04:42:51.079 INFO:tasks.workunit.client.0.smithi016.stdout:7/802: dread d0/d46/d16/f34 [0,4194304] 0 2024-04-02T04:42:51.079 INFO:tasks.workunit.client.0.smithi016.stdout:8/792: dread - d1/d6/d36/d3f/d40/d49/d38/fc4 zero size 2024-04-02T04:42:51.080 INFO:tasks.workunit.client.1.smithi027.stdout:6/292: dwrite d13/d1b/d24/d28/f2a [0,4194304] 0 2024-04-02T04:42:51.082 INFO:tasks.workunit.client.1.smithi027.stdout:6/293: read d13/f1a [3202426,116627] 0 2024-04-02T04:42:51.083 INFO:tasks.workunit.client.1.smithi027.stdout:0/331: unlink d4/d38/d66/f5b 0 2024-04-02T04:42:51.084 INFO:tasks.workunit.client.1.smithi027.stdout:0/332: fsync d4/d38/d66/d4a/f61 0 2024-04-02T04:42:51.087 INFO:tasks.workunit.client.1.smithi027.stdout:0/333: write d4/d38/d66/f2f [8732501,47677] 0 2024-04-02T04:42:51.088 INFO:tasks.workunit.client.0.smithi016.stdout:3/903: creat d2/d15/d1a/d45/da5/f12d x:0 0 0 2024-04-02T04:42:51.089 INFO:tasks.workunit.client.1.smithi027.stdout:5/390: creat d0/d1d/f91 x:0 0 0 2024-04-02T04:42:51.090 INFO:tasks.workunit.client.1.smithi027.stdout:5/391: write d0/d1d/f79 [1564805,32476] 0 2024-04-02T04:42:51.094 INFO:tasks.workunit.client.0.smithi016.stdout:4/812: fsync d5/d17/d18/fb3 0 2024-04-02T04:42:51.095 INFO:tasks.workunit.client.1.smithi027.stdout:7/296: rename f1 to d2/d7/da/dc/d1f/d33/f55 0 2024-04-02T04:42:51.098 INFO:tasks.workunit.client.1.smithi027.stdout:8/300: chown d5/d8/db/df/f22 293 1 2024-04-02T04:42:51.099 INFO:tasks.workunit.client.0.smithi016.stdout:9/892: creat d0/d4a/d75/d95/d9b/db4/df9/f116 x:0 0 0 2024-04-02T04:42:51.100 INFO:tasks.workunit.client.0.smithi016.stdout:9/893: write d0/d4e/d50/f85 [3883106,89627] 0 2024-04-02T04:42:51.104 INFO:tasks.workunit.client.1.smithi027.stdout:6/294: symlink d13/d1b/d48/d58/l5f 0 2024-04-02T04:42:51.105 INFO:tasks.workunit.client.0.smithi016.stdout:3/904: creat d2/d15/d16/d57/d96/f12e x:0 0 0 2024-04-02T04:42:51.107 INFO:tasks.workunit.client.0.smithi016.stdout:7/803: dwrite d0/fe7 [0,4194304] 0 2024-04-02T04:42:51.110 INFO:tasks.workunit.client.0.smithi016.stdout:4/813: creat d5/d17/d18/d6d/d8e/d9c/dcc/d6f/dcf/ff6 x:0 0 0 2024-04-02T04:42:51.112 INFO:tasks.workunit.client.0.smithi016.stdout:8/793: dread d1/d6/d86/fa9 [0,4194304] 0 2024-04-02T04:42:51.113 INFO:tasks.workunit.client.0.smithi016.stdout:8/794: dread - d1/d6/d36/d3f/d40/d49/d4a/d4c/dad/fe8 zero size 2024-04-02T04:42:51.114 INFO:tasks.workunit.client.1.smithi027.stdout:5/392: mknod d0/d1d/c92 0 2024-04-02T04:42:51.115 INFO:tasks.workunit.client.1.smithi027.stdout:9/300: sync 2024-04-02T04:42:51.116 INFO:tasks.workunit.client.1.smithi027.stdout:9/301: dread - d1a/d1d/d29/d37/f5a zero size 2024-04-02T04:42:51.117 INFO:tasks.workunit.client.1.smithi027.stdout:7/297: symlink d2/d7/da/dc/d1f/d33/d36/l56 0 2024-04-02T04:42:51.121 INFO:tasks.workunit.client.1.smithi027.stdout:3/229: symlink d1/de/l47 0 2024-04-02T04:42:51.122 INFO:tasks.workunit.client.1.smithi027.stdout:3/230: readlink d1/de/l12 0 2024-04-02T04:42:51.127 INFO:tasks.workunit.client.1.smithi027.stdout:0/334: mknod d4/d5d/c6b 0 2024-04-02T04:42:51.128 INFO:tasks.workunit.client.1.smithi027.stdout:0/335: chown d4/d21/d28/d4e/d5f 2809395 1 2024-04-02T04:42:51.129 INFO:tasks.workunit.client.1.smithi027.stdout:3/231: dwrite d1/de/d3a/f3d [0,4194304] 0 2024-04-02T04:42:51.131 INFO:tasks.workunit.client.1.smithi027.stdout:5/393: rename d0/d1a/d6f/f89 to d0/d1a/d20/f93 0 2024-04-02T04:42:51.132 INFO:tasks.workunit.client.0.smithi016.stdout:4/814: unlink d5/d17/d18/d1f/f63 0 2024-04-02T04:42:51.133 INFO:tasks.workunit.client.1.smithi027.stdout:5/394: truncate d0/d1a/d20/f87 288290 0 2024-04-02T04:42:51.134 INFO:tasks.workunit.client.1.smithi027.stdout:7/298: mkdir d2/d1d/d57 0 2024-04-02T04:42:51.134 INFO:tasks.workunit.client.0.smithi016.stdout:3/905: dread d2/fd [0,4194304] 0 2024-04-02T04:42:51.137 INFO:tasks.workunit.client.0.smithi016.stdout:1/785: link d14/d19/d1e/f5d d14/d19/d1e/d71/ddd/f103 0 2024-04-02T04:42:51.139 INFO:tasks.workunit.client.1.smithi027.stdout:8/301: write d5/d8/db/df/f22 [1343681,16356] 0 2024-04-02T04:42:51.145 INFO:tasks.workunit.client.0.smithi016.stdout:8/795: write d1/d6/f4b [1084887,29459] 0 2024-04-02T04:42:51.147 INFO:tasks.workunit.client.0.smithi016.stdout:7/804: write d0/d46/da2/dc8/dda/ffa [1085106,91314] 0 2024-04-02T04:42:51.149 INFO:tasks.workunit.client.1.smithi027.stdout:5/395: unlink d0/df/d29/c23 0 2024-04-02T04:42:51.150 INFO:tasks.workunit.client.0.smithi016.stdout:9/894: creat d0/d4a/d75/f117 x:0 0 0 2024-04-02T04:42:51.151 INFO:tasks.workunit.client.1.smithi027.stdout:7/299: rmdir d2/d7/da/dc/d16/d3a 39 2024-04-02T04:42:51.151 INFO:tasks.workunit.client.0.smithi016.stdout:7/805: truncate d0/d46/d37/dbe/dae/df8/d43/d96/ffd 252394 0 2024-04-02T04:42:51.152 INFO:tasks.workunit.client.0.smithi016.stdout:3/906: unlink d2/d15/d1a/d45/c6a 0 2024-04-02T04:42:51.154 INFO:tasks.workunit.client.0.smithi016.stdout:8/796: mkdir d1/d6/d36/d3f/d40/d49/d4a/d108 0 2024-04-02T04:42:51.155 INFO:tasks.workunit.client.0.smithi016.stdout:1/786: dwrite d14/d19/f60 [4194304,4194304] 0 2024-04-02T04:42:51.155 INFO:tasks.workunit.client.0.smithi016.stdout:3/907: stat d2/d2e/d36/d48/da2/d88 0 2024-04-02T04:42:51.156 INFO:tasks.workunit.client.1.smithi027.stdout:6/295: getdents d13/d1b/d24/d25 0 2024-04-02T04:42:51.156 INFO:tasks.workunit.client.0.smithi016.stdout:8/797: fsync d1/d6/d36/d3f/d77/fae 0 2024-04-02T04:42:51.160 INFO:tasks.workunit.client.0.smithi016.stdout:9/895: dwrite d0/d9/db/dc9/d53/fc5 [0,4194304] 0 2024-04-02T04:42:51.161 INFO:tasks.workunit.client.0.smithi016.stdout:9/896: readlink d0/l3f 0 2024-04-02T04:42:51.163 INFO:tasks.workunit.client.0.smithi016.stdout:1/787: dread d14/d19/d1e/d71/f78 [0,4194304] 0 2024-04-02T04:42:51.164 INFO:tasks.workunit.client.1.smithi027.stdout:5/396: dread d0/f6 [0,4194304] 0 2024-04-02T04:42:51.172 INFO:tasks.workunit.client.0.smithi016.stdout:1/788: dread d14/d19/d1d/f2c [0,4194304] 0 2024-04-02T04:42:51.173 INFO:tasks.workunit.client.0.smithi016.stdout:7/806: truncate d0/d21/f65 6456220 0 2024-04-02T04:42:51.178 INFO:tasks.workunit.client.1.smithi027.stdout:6/296: rename d13/d1b/d24/d28/f55 to d13/d1b/d24/d25/d42/f60 0 2024-04-02T04:42:51.179 INFO:tasks.workunit.client.1.smithi027.stdout:0/336: getdents d4/d21/d28 0 2024-04-02T04:42:51.181 INFO:tasks.workunit.client.1.smithi027.stdout:3/232: getdents d1 0 2024-04-02T04:42:51.182 INFO:tasks.workunit.client.1.smithi027.stdout:3/233: write d1/de/f40 [19007,29249] 0 2024-04-02T04:42:51.186 INFO:tasks.workunit.client.0.smithi016.stdout:8/798: write d1/d70/d71/d74/d82/fc2 [353751,45487] 0 2024-04-02T04:42:51.187 INFO:tasks.workunit.client.1.smithi027.stdout:8/302: getdents d5/d8/d48 0 2024-04-02T04:42:51.190 INFO:tasks.workunit.client.1.smithi027.stdout:5/397: dwrite d0/df/d2f/d3b/f3d [0,4194304] 0 2024-04-02T04:42:51.192 INFO:tasks.workunit.client.1.smithi027.stdout:7/300: creat d2/d7/da/dc/d16/d3a/d4d/f58 x:0 0 0 2024-04-02T04:42:51.194 INFO:tasks.workunit.client.0.smithi016.stdout:9/897: dread d0/d33/f36 [4194304,4194304] 0 2024-04-02T04:42:51.199 INFO:tasks.workunit.client.0.smithi016.stdout:8/799: mknod d1/d6/d36/d3f/d40/d63/d105/c109 0 2024-04-02T04:42:51.200 INFO:tasks.workunit.client.0.smithi016.stdout:8/800: truncate d1/d6/d36/d3f/d40/d49/ffe 649857 0 2024-04-02T04:42:51.200 INFO:tasks.workunit.client.1.smithi027.stdout:8/303: fsync d5/d8/db/dd/d5f/d27/f50 0 2024-04-02T04:42:51.202 INFO:tasks.workunit.client.0.smithi016.stdout:8/801: truncate d1/d6/d36/d3f/d40/d63/d105/f107 199999 0 2024-04-02T04:42:51.203 INFO:tasks.workunit.client.0.smithi016.stdout:8/802: truncate d1/d6/d36/d3f/d40/fd3 1046702 0 2024-04-02T04:42:51.206 INFO:tasks.workunit.client.0.smithi016.stdout:3/908: link d2/d15/d16/d8a/lb1 d2/d15/d70/dcf/d11c/l12f 0 2024-04-02T04:42:51.208 INFO:tasks.workunit.client.0.smithi016.stdout:1/789: link d14/d19/d1e/d71/fbf d14/d19/d1e/d71/f104 0 2024-04-02T04:42:51.209 INFO:tasks.workunit.client.0.smithi016.stdout:7/807: dread d0/d21/f57 [0,4194304] 0 2024-04-02T04:42:51.210 INFO:tasks.workunit.client.0.smithi016.stdout:8/803: write d1/ff [5472674,37018] 0 2024-04-02T04:42:51.210 INFO:tasks.workunit.client.0.smithi016.stdout:9/898: creat d0/d9/db/dc9/de1/f118 x:0 0 0 2024-04-02T04:42:51.211 INFO:tasks.workunit.client.0.smithi016.stdout:9/899: dread - d0/d4a/d75/fff zero size 2024-04-02T04:42:51.213 INFO:tasks.workunit.client.0.smithi016.stdout:3/909: rename d2/d15/d16/d8a/dd1/d6e/d6f/df9 to d2/d15/d16/d8a/dd1/d8b/d130 0 2024-04-02T04:42:51.215 INFO:tasks.workunit.client.0.smithi016.stdout:9/900: dread d0/d4a/da9/ff3 [0,4194304] 0 2024-04-02T04:42:51.218 INFO:tasks.workunit.client.1.smithi027.stdout:5/398: write d0/f66 [514990,92024] 0 2024-04-02T04:42:51.219 INFO:tasks.workunit.client.1.smithi027.stdout:8/304: creat d5/d8/db/d57/f62 x:0 0 0 2024-04-02T04:42:51.220 INFO:tasks.workunit.client.0.smithi016.stdout:7/808: mkdir d0/d46/d37/dbe/dae/d9d/d110 0 2024-04-02T04:42:51.220 INFO:tasks.workunit.client.1.smithi027.stdout:3/234: dwrite d1/f8 [0,4194304] 0 2024-04-02T04:42:51.221 INFO:tasks.workunit.client.1.smithi027.stdout:5/399: readlink d0/df/d2f/d3b/d51/l52 0 2024-04-02T04:42:51.221 INFO:tasks.workunit.client.1.smithi027.stdout:0/337: truncate d4/d16/f1c 2567633 0 2024-04-02T04:42:51.223 INFO:tasks.workunit.client.1.smithi027.stdout:0/338: stat d4/d21/d28/f48 0 2024-04-02T04:42:51.225 INFO:tasks.workunit.client.1.smithi027.stdout:0/339: write d4/d38/f58 [937673,29544] 0 2024-04-02T04:42:51.228 INFO:tasks.workunit.client.1.smithi027.stdout:0/340: truncate d4/d38/d66/f24 1040023 0 2024-04-02T04:42:51.228 INFO:tasks.workunit.client.1.smithi027.stdout:8/305: symlink d5/d8/db/dd/d5f/d27/l63 0 2024-04-02T04:42:51.230 INFO:tasks.workunit.client.1.smithi027.stdout:0/341: fdatasync d4/d38/d66/f2f 0 2024-04-02T04:42:51.231 INFO:tasks.workunit.client.1.smithi027.stdout:3/235: mknod d1/de/df/c48 0 2024-04-02T04:42:51.232 INFO:tasks.workunit.client.1.smithi027.stdout:5/400: dwrite d0/df/d11/d13/f1e [4194304,4194304] 0 2024-04-02T04:42:51.232 INFO:tasks.workunit.client.1.smithi027.stdout:8/306: write d5/d8/db/dd/f1d [4285257,74097] 0 2024-04-02T04:42:51.233 INFO:tasks.workunit.client.0.smithi016.stdout:3/910: creat d2/d15/d70/dcf/f131 x:0 0 0 2024-04-02T04:42:51.234 INFO:tasks.workunit.client.1.smithi027.stdout:3/236: write d1/de/df/f24 [43646,3942] 0 2024-04-02T04:42:51.235 INFO:tasks.workunit.client.0.smithi016.stdout:1/790: dread d14/d19/d1d/d29/f3c [0,4194304] 0 2024-04-02T04:42:51.236 INFO:tasks.workunit.client.1.smithi027.stdout:8/307: fsync d5/d8/db/dd/d5f/f2b 0 2024-04-02T04:42:51.237 INFO:tasks.workunit.client.1.smithi027.stdout:8/308: fdatasync d5/d8/d15/f37 0 2024-04-02T04:42:51.238 INFO:tasks.workunit.client.1.smithi027.stdout:8/309: read d5/d8/d48/f5d [187405,20140] 0 2024-04-02T04:42:51.242 INFO:tasks.workunit.client.1.smithi027.stdout:8/310: dread - d5/d8/db/f29 zero size 2024-04-02T04:42:51.242 INFO:tasks.workunit.client.1.smithi027.stdout:3/237: creat d1/d1a/f49 x:0 0 0 2024-04-02T04:42:51.243 INFO:tasks.workunit.client.0.smithi016.stdout:7/809: chown d0/d46/d37/d47/f5c 11323420 1 2024-04-02T04:42:51.243 INFO:tasks.workunit.client.1.smithi027.stdout:5/401: symlink d0/d5/l94 0 2024-04-02T04:42:51.244 INFO:tasks.workunit.client.0.smithi016.stdout:3/911: creat d2/d15/d1a/d45/f132 x:0 0 0 2024-04-02T04:42:51.245 INFO:tasks.workunit.client.1.smithi027.stdout:5/402: read d0/d1d/f7c [49956,72899] 0 2024-04-02T04:42:51.245 INFO:tasks.workunit.client.0.smithi016.stdout:1/791: unlink d14/d19/d1d/d29/d4c/fd2 0 2024-04-02T04:42:51.246 INFO:tasks.workunit.client.1.smithi027.stdout:5/403: truncate d0/d1d/f91 220672 0 2024-04-02T04:42:51.247 INFO:tasks.workunit.client.1.smithi027.stdout:7/301: link d2/d1d/f25 d2/d7/da/f59 0 2024-04-02T04:42:51.247 INFO:tasks.workunit.client.1.smithi027.stdout:5/404: write d0/df/d11/d13/f3e [724286,94175] 0 2024-04-02T04:42:51.249 INFO:tasks.workunit.client.1.smithi027.stdout:5/405: write d0/df/d2f/d3b/f5b [1954043,61068] 0 2024-04-02T04:42:51.250 INFO:tasks.workunit.client.0.smithi016.stdout:7/810: dwrite d0/d46/d37/dbe/fbc [0,4194304] 0 2024-04-02T04:42:51.250 INFO:tasks.workunit.client.1.smithi027.stdout:5/406: dread - d0/d5/d55/f8a zero size 2024-04-02T04:42:51.257 INFO:tasks.workunit.client.1.smithi027.stdout:2/313: sync 2024-04-02T04:42:51.257 INFO:tasks.workunit.client.1.smithi027.stdout:1/263: sync 2024-04-02T04:42:51.257 INFO:tasks.workunit.client.1.smithi027.stdout:4/239: sync 2024-04-02T04:42:51.258 INFO:tasks.workunit.client.1.smithi027.stdout:2/314: stat d1/db/d3b/d17/f3f 0 2024-04-02T04:42:51.258 INFO:tasks.workunit.client.1.smithi027.stdout:8/311: rename d5/d8/l4d to d5/d8/d5c/l64 0 2024-04-02T04:42:51.258 INFO:tasks.workunit.client.1.smithi027.stdout:1/264: truncate d0/de/f58 236271 0 2024-04-02T04:42:51.260 INFO:tasks.workunit.client.0.smithi016.stdout:1/792: mknod d14/d19/daf/c105 0 2024-04-02T04:42:51.266 INFO:tasks.workunit.client.1.smithi027.stdout:0/342: creat d4/d38/d66/f6c x:0 0 0 2024-04-02T04:42:51.267 INFO:tasks.workunit.client.0.smithi016.stdout:9/901: dwrite d0/d9/db/d54/f8e [0,4194304] 0 2024-04-02T04:42:51.269 INFO:tasks.workunit.client.0.smithi016.stdout:9/902: chown d0/d4a/d99/d46/fb7 362090 1 2024-04-02T04:42:51.272 INFO:tasks.workunit.client.0.smithi016.stdout:3/912: symlink d2/d2e/d36/d48/l133 0 2024-04-02T04:42:51.279 INFO:tasks.workunit.client.0.smithi016.stdout:8/804: getdents d1/d6/d36/d4f/d5d 0 2024-04-02T04:42:51.281 INFO:tasks.workunit.client.1.smithi027.stdout:3/238: rename d1/fc to d1/d1a/f4a 0 2024-04-02T04:42:51.282 INFO:tasks.workunit.client.1.smithi027.stdout:1/265: mkdir d0/d1e/d5d 0 2024-04-02T04:42:51.285 INFO:tasks.workunit.client.0.smithi016.stdout:3/913: creat d2/d15/d16/d8a/dd1/d6e/d6f/d78/f134 x:0 0 0 2024-04-02T04:42:51.285 INFO:tasks.workunit.client.0.smithi016.stdout:9/903: dread d0/d9/db/dc9/fa0 [0,4194304] 0 2024-04-02T04:42:51.286 INFO:tasks.workunit.client.0.smithi016.stdout:1/793: mkdir d14/d19/d1d/d29/d39/df0/d106 0 2024-04-02T04:42:51.286 INFO:tasks.workunit.client.1.smithi027.stdout:8/312: write d5/d8/d15/f39 [2930215,32458] 0 2024-04-02T04:42:51.286 INFO:tasks.workunit.client.1.smithi027.stdout:8/313: chown d5/d8/c3f 11 1 2024-04-02T04:42:51.286 INFO:tasks.workunit.client.0.smithi016.stdout:3/914: dread - d2/d15/fb7 zero size 2024-04-02T04:42:51.287 INFO:tasks.workunit.client.0.smithi016.stdout:8/805: dwrite d1/d70/d71/d74/d82/fc2 [0,4194304] 0 2024-04-02T04:42:51.287 INFO:tasks.workunit.client.0.smithi016.stdout:9/904: write d0/d3/f40 [3409567,68596] 0 2024-04-02T04:42:51.291 INFO:tasks.workunit.client.1.smithi027.stdout:0/343: symlink d4/d38/d66/d4a/l6d 0 2024-04-02T04:42:51.293 INFO:tasks.workunit.client.1.smithi027.stdout:0/344: write d4/d38/d66/d2d/f3c [2759973,2511] 0 2024-04-02T04:42:51.296 INFO:tasks.workunit.client.1.smithi027.stdout:4/240: symlink d24/l4a 0 2024-04-02T04:42:51.298 INFO:tasks.workunit.client.0.smithi016.stdout:1/794: mkdir d14/d19/d1e/d71/dc0/d107 0 2024-04-02T04:42:51.298 INFO:tasks.workunit.client.0.smithi016.stdout:1/795: stat d14/d19/d1e/d5e/fb1 0 2024-04-02T04:42:51.300 INFO:tasks.workunit.client.0.smithi016.stdout:3/915: rmdir d2/d2e/d36/d48/da2/d88 39 2024-04-02T04:42:51.302 INFO:tasks.workunit.client.0.smithi016.stdout:9/905: read d0/d4e/d50/d63/d6e/d4b/fc0 [369435,96177] 0 2024-04-02T04:42:51.304 INFO:tasks.workunit.client.0.smithi016.stdout:3/916: symlink d2/d2e/d36/d48/da2/l135 0 2024-04-02T04:42:51.305 INFO:tasks.workunit.client.0.smithi016.stdout:5/775: sync 2024-04-02T04:42:51.305 INFO:tasks.workunit.client.0.smithi016.stdout:2/865: sync 2024-04-02T04:42:51.307 INFO:tasks.workunit.client.1.smithi027.stdout:5/407: link d0/d5/l94 d0/d5/l95 0 2024-04-02T04:42:51.307 INFO:tasks.workunit.client.0.smithi016.stdout:5/776: chown d3/d40/d4f/d6b/d45/da6/d102/dc4/d108/d6d/fc0 0 1 2024-04-02T04:42:51.309 INFO:tasks.workunit.client.0.smithi016.stdout:3/917: rename d2/d2e/d36/d66/d90/dfb/f110 to d2/d15/d1a/d45/f136 0 2024-04-02T04:42:51.310 INFO:tasks.workunit.client.1.smithi027.stdout:0/345: mknod d4/d38/d66/d2d/d4c/c6e 0 2024-04-02T04:42:51.311 INFO:tasks.workunit.client.0.smithi016.stdout:2/866: mknod d12/d35/d3f/d2b/da3/db1/c127 0 2024-04-02T04:42:51.312 INFO:tasks.workunit.client.1.smithi027.stdout:0/346: write d4/d21/d28/d4e/f62 [785578,54268] 0 2024-04-02T04:42:51.312 INFO:tasks.workunit.client.0.smithi016.stdout:9/906: dread d0/d4a/f9c [0,4194304] 0 2024-04-02T04:42:51.313 INFO:tasks.workunit.client.1.smithi027.stdout:4/241: creat d24/d38/d44/d41/f4b x:0 0 0 2024-04-02T04:42:51.314 INFO:tasks.workunit.client.1.smithi027.stdout:0/347: dread d4/d38/d66/f24 [0,4194304] 0 2024-04-02T04:42:51.317 INFO:tasks.workunit.client.1.smithi027.stdout:2/315: creat d1/db/d3b/d17/f6a x:0 0 0 2024-04-02T04:42:51.317 INFO:tasks.workunit.client.0.smithi016.stdout:7/811: dwrite d0/d46/d37/dbe/fde [0,4194304] 0 2024-04-02T04:42:51.318 INFO:tasks.workunit.client.1.smithi027.stdout:2/316: read - d1/db/d3b/d17/d12/f41 zero size 2024-04-02T04:42:51.318 INFO:tasks.workunit.client.0.smithi016.stdout:5/777: dread - d3/d40/d4f/d6b/d45/da6/d102/dc4/d108/d6d/fc0 zero size 2024-04-02T04:42:51.319 INFO:tasks.workunit.client.1.smithi027.stdout:7/302: rename d2/d7/da/f59 to d2/d7/da/dc/d16/d3a/d53/f5a 0 2024-04-02T04:42:51.321 INFO:tasks.workunit.client.1.smithi027.stdout:7/303: write d2/d7/da/dc/d16/d3f/f43 [2982757,85906] 0 2024-04-02T04:42:51.322 INFO:tasks.workunit.client.0.smithi016.stdout:3/918: mkdir d2/d15/d1a/d45/da5/db6/d137 0 2024-04-02T04:42:51.322 INFO:tasks.workunit.client.0.smithi016.stdout:3/919: dread - d2/d2e/dee/ffc zero size 2024-04-02T04:42:51.326 INFO:tasks.workunit.client.0.smithi016.stdout:2/867: creat d12/d35/d101/d4f/dfa/f128 x:0 0 0 2024-04-02T04:42:51.327 INFO:tasks.workunit.client.0.smithi016.stdout:2/868: write d12/d35/d3f/f4b [2544919,57527] 0 2024-04-02T04:42:51.329 INFO:tasks.workunit.client.1.smithi027.stdout:8/314: mknod d5/c65 0 2024-04-02T04:42:51.330 INFO:tasks.workunit.client.0.smithi016.stdout:9/907: fdatasync d0/d4e/d50/d63/d6e/fc8 0 2024-04-02T04:42:51.331 INFO:tasks.workunit.client.0.smithi016.stdout:9/908: dread - d0/d9/db/dc9/de1/f118 zero size 2024-04-02T04:42:51.337 INFO:tasks.workunit.client.0.smithi016.stdout:5/778: mkdir d3/d40/d4f/d67/dbf/d111/d116 0 2024-04-02T04:42:51.346 INFO:tasks.workunit.client.1.smithi027.stdout:2/317: symlink d1/db/d3b/d17/d68/l6b 0 2024-04-02T04:42:51.347 INFO:tasks.workunit.client.1.smithi027.stdout:3/239: dwrite d1/de/df/d13/f43 [0,4194304] 0 2024-04-02T04:42:51.354 INFO:tasks.workunit.client.0.smithi016.stdout:9/909: truncate d0/f5d 2497010 0 2024-04-02T04:42:51.355 INFO:tasks.workunit.client.0.smithi016.stdout:3/920: rmdir d2/d15/d1a/d45/da5 39 2024-04-02T04:42:51.355 INFO:tasks.workunit.client.1.smithi027.stdout:5/408: dread d0/df/d11/d13/f28 [0,4194304] 0 2024-04-02T04:42:51.356 INFO:tasks.workunit.client.1.smithi027.stdout:1/266: write d0/f16 [836163,70875] 0 2024-04-02T04:42:51.357 INFO:tasks.workunit.client.1.smithi027.stdout:1/267: read d0/f37 [869807,10411] 0 2024-04-02T04:42:51.358 INFO:tasks.workunit.client.1.smithi027.stdout:8/315: symlink d5/d8/db/df/l66 0 2024-04-02T04:42:51.358 INFO:tasks.workunit.client.1.smithi027.stdout:1/268: truncate d0/de/f38 1121285 0 2024-04-02T04:42:51.359 INFO:tasks.workunit.client.1.smithi027.stdout:1/269: chown d0/de/f23 0 1 2024-04-02T04:42:51.360 INFO:tasks.workunit.client.0.smithi016.stdout:5/779: mkdir d3/d40/d4f/d6b/d45/da6/d102/dc4/dd7/d117 0 2024-04-02T04:42:51.362 INFO:tasks.workunit.client.0.smithi016.stdout:7/812: write d0/d21/f86 [618176,15330] 0 2024-04-02T04:42:51.364 INFO:tasks.workunit.client.0.smithi016.stdout:9/910: mkdir d0/d4e/d50/d63/d6e/d4b/db2/dba/d119 0 2024-04-02T04:42:51.365 INFO:tasks.workunit.client.0.smithi016.stdout:3/921: rename d2/d2e/d36/d66 to d2/d2e/d36/d66/d138 22 2024-04-02T04:42:51.367 INFO:tasks.workunit.client.1.smithi027.stdout:4/242: dwrite f16 [0,4194304] 0 2024-04-02T04:42:51.368 INFO:tasks.workunit.client.0.smithi016.stdout:5/780: mknod d3/d40/d4f/d67/dbf/c118 0 2024-04-02T04:42:51.369 INFO:tasks.workunit.client.1.smithi027.stdout:2/318: mkdir d1/db/d3b/d61/d6c 0 2024-04-02T04:42:51.371 INFO:tasks.workunit.client.0.smithi016.stdout:3/922: dwrite d2/d15/d1a/d45/f8f [0,4194304] 0 2024-04-02T04:42:51.372 INFO:tasks.workunit.client.1.smithi027.stdout:3/240: creat d1/de/df/f4b x:0 0 0 2024-04-02T04:42:51.377 INFO:tasks.workunit.client.0.smithi016.stdout:9/911: stat d0/d3/d39/l41 0 2024-04-02T04:42:51.378 INFO:tasks.workunit.client.0.smithi016.stdout:5/781: mknod d3/d40/d4f/d6b/d45/da6/d102/dc4/d108/c119 0 2024-04-02T04:42:51.381 INFO:tasks.workunit.client.0.smithi016.stdout:2/869: link d12/d35/f40 d12/d35/d3b/d42/d59/f129 0 2024-04-02T04:42:51.382 INFO:tasks.workunit.client.1.smithi027.stdout:1/270: truncate d0/d1e/f2f 913391 0 2024-04-02T04:42:51.383 INFO:tasks.workunit.client.0.smithi016.stdout:3/923: chown d2/d15/d1a/d45/da5/db6 51276 1 2024-04-02T04:42:51.384 INFO:tasks.workunit.client.1.smithi027.stdout:5/409: dwrite d0/d1a/d20/f93 [0,4194304] 0 2024-04-02T04:42:51.384 INFO:tasks.workunit.client.1.smithi027.stdout:1/271: chown d0/l26 455701314 1 2024-04-02T04:42:51.386 INFO:tasks.workunit.client.0.smithi016.stdout:7/813: mknod d0/d46/d16/df6/c111 0 2024-04-02T04:42:51.386 INFO:tasks.workunit.client.1.smithi027.stdout:9/302: sync 2024-04-02T04:42:51.387 INFO:tasks.workunit.client.1.smithi027.stdout:9/303: chown d1a/d1d/d29/d37/d3a/f54 3093 1 2024-04-02T04:42:51.389 INFO:tasks.workunit.client.0.smithi016.stdout:3/924: dwrite d2/d15/d16/d8a/f98 [4194304,4194304] 0 2024-04-02T04:42:51.390 INFO:tasks.workunit.client.0.smithi016.stdout:5/782: chown d3/df/d18/c50 7498 1 2024-04-02T04:42:51.391 INFO:tasks.workunit.client.1.smithi027.stdout:3/241: creat d1/de/d3a/f4c x:0 0 0 2024-04-02T04:42:51.393 INFO:tasks.workunit.client.1.smithi027.stdout:7/304: creat d2/d7/da/f5b x:0 0 0 2024-04-02T04:42:51.395 INFO:tasks.workunit.client.1.smithi027.stdout:7/305: write d2/d7/da/f5b [268540,74525] 0 2024-04-02T04:42:51.395 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:51 smithi027 ceph-mon[37550]: pgmap v24: 65 pgs: 65 active+clean; 2.2 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 70 MiB/s rd, 283 MiB/s wr, 534 op/s 2024-04-02T04:42:51.399 INFO:tasks.workunit.client.1.smithi027.stdout:0/348: getdents d4/d38/d66/d2d 0 2024-04-02T04:42:51.400 INFO:tasks.workunit.client.1.smithi027.stdout:0/349: readlink d4/d21/l64 0 2024-04-02T04:42:51.401 INFO:tasks.workunit.client.0.smithi016.stdout:2/870: creat d12/d35/d101/d76/da0/de7/dfc/d106/f12a x:0 0 0 2024-04-02T04:42:51.401 INFO:tasks.workunit.client.0.smithi016.stdout:2/871: write d12/f43 [3075441,70619] 0 2024-04-02T04:42:51.402 INFO:tasks.workunit.client.0.smithi016.stdout:2/872: chown d12/d35/d3f/d2b/de9/d102 1 1 2024-04-02T04:42:51.405 INFO:tasks.workunit.client.1.smithi027.stdout:5/410: creat d0/d1d/d61/f96 x:0 0 0 2024-04-02T04:42:51.407 INFO:tasks.workunit.client.1.smithi027.stdout:4/243: link f21 d24/d38/d44/d41/d49/f4c 0 2024-04-02T04:42:51.408 INFO:tasks.workunit.client.1.smithi027.stdout:4/244: stat d24/d38 0 2024-04-02T04:42:51.409 INFO:tasks.workunit.client.1.smithi027.stdout:8/316: dwrite d5/d8/db/dd/d5f/d27/f50 [0,4194304] 0 2024-04-02T04:42:51.409 INFO:tasks.workunit.client.0.smithi016.stdout:9/912: rename d0/d4e/d50/d63/d100/c10e to d0/c11a 0 2024-04-02T04:42:51.411 INFO:tasks.workunit.client.1.smithi027.stdout:2/319: mkdir d1/db/d3b/d17/d6d 0 2024-04-02T04:42:51.411 INFO:tasks.workunit.client.1.smithi027.stdout:1/272: dwrite d0/f30 [4194304,4194304] 0 2024-04-02T04:42:51.414 INFO:tasks.workunit.client.1.smithi027.stdout:8/317: write d5/d8/db/dd/d5f/d27/f3a [2224368,88035] 0 2024-04-02T04:42:51.416 INFO:tasks.workunit.client.1.smithi027.stdout:3/242: symlink d1/de/df/d13/l4d 0 2024-04-02T04:42:51.419 INFO:tasks.workunit.client.0.smithi016.stdout:5/783: link d3/d5d/lb2 d3/df/d18/dfd/l11a 0 2024-04-02T04:42:51.419 INFO:tasks.workunit.client.0.smithi016.stdout:5/784: read - d3/df/d18/ff2 zero size 2024-04-02T04:42:51.420 INFO:tasks.workunit.client.1.smithi027.stdout:5/411: creat d0/d1d/f97 x:0 0 0 2024-04-02T04:42:51.422 INFO:tasks.workunit.client.0.smithi016.stdout:2/873: mkdir d12/d35/d3b/d12b 0 2024-04-02T04:42:51.423 INFO:tasks.workunit.client.0.smithi016.stdout:2/874: chown d12/d35/d101/d76/da0/de7/dd1 1 1 2024-04-02T04:42:51.426 INFO:tasks.workunit.client.0.smithi016.stdout:9/913: rename d0/d9/d4d/d78/da6/cc7 to d0/d4a/d99/d46/dd6/c11b 0 2024-04-02T04:42:51.427 INFO:tasks.workunit.client.1.smithi027.stdout:5/412: dread d0/df/d11/f1f [0,4194304] 0 2024-04-02T04:42:51.428 INFO:tasks.workunit.client.1.smithi027.stdout:8/318: dwrite d5/d8/db/dd/d5f/d27/d31/f5e [0,4194304] 0 2024-04-02T04:42:51.429 INFO:tasks.workunit.client.1.smithi027.stdout:8/319: dread - d5/d8/db/dd/f3b zero size 2024-04-02T04:42:51.430 INFO:tasks.workunit.client.0.smithi016.stdout:5/785: creat d3/df/d18/d26/d70/f11b x:0 0 0 2024-04-02T04:42:51.430 INFO:tasks.workunit.client.0.smithi016.stdout:5/786: dread - d3/df/d18/f112 zero size 2024-04-02T04:42:51.431 INFO:tasks.workunit.client.1.smithi027.stdout:8/320: write d5/d8/db/dd/f51 [225069,18674] 0 2024-04-02T04:42:51.432 INFO:tasks.workunit.client.0.smithi016.stdout:5/787: chown d3/df/d93/cea 118 1 2024-04-02T04:42:51.432 INFO:tasks.workunit.client.0.smithi016.stdout:5/788: stat d3/df/d18/d26/f106 0 2024-04-02T04:42:51.432 INFO:tasks.workunit.client.0.smithi016.stdout:5/789: chown d3/df/c7b 3724 1 2024-04-02T04:42:51.433 INFO:tasks.workunit.client.0.smithi016.stdout:7/814: write d0/d46/d16/dc2/dea/f7c [394270,126381] 0 2024-04-02T04:42:51.434 INFO:tasks.workunit.client.0.smithi016.stdout:5/790: chown d3/d40/l44 955291859 1 2024-04-02T04:42:51.435 INFO:tasks.workunit.client.0.smithi016.stdout:5/791: dread d3/df/f14 [0,4194304] 0 2024-04-02T04:42:51.440 INFO:tasks.workunit.client.0.smithi016.stdout:2/875: mknod d12/d35/d101/d76/da0/de7/dd1/c12c 0 2024-04-02T04:42:51.442 INFO:tasks.workunit.client.1.smithi027.stdout:0/350: fdatasync d4/f23 0 2024-04-02T04:42:51.445 INFO:tasks.workunit.client.1.smithi027.stdout:2/320: dread d1/db/d3b/d61/f66 [0,4194304] 0 2024-04-02T04:42:51.446 INFO:tasks.workunit.client.0.smithi016.stdout:7/815: write d0/d46/d52/d82/fbb [1032413,47155] 0 2024-04-02T04:42:51.446 INFO:tasks.workunit.client.1.smithi027.stdout:4/245: rename d24/f35 to d24/d48/f4d 0 2024-04-02T04:42:51.448 INFO:tasks.workunit.client.0.smithi016.stdout:5/792: fsync d3/df/d18/d26/d70/fdd 0 2024-04-02T04:42:51.452 INFO:tasks.workunit.client.1.smithi027.stdout:2/321: dwrite d1/db/d3b/d17/f4a [0,4194304] 0 2024-04-02T04:42:51.452 INFO:tasks.workunit.client.1.smithi027.stdout:5/413: rmdir d0/df/d11/d41 39 2024-04-02T04:42:51.456 INFO:tasks.workunit.client.0.smithi016.stdout:2/876: readlink d12/d70/lfe 0 2024-04-02T04:42:51.458 INFO:tasks.workunit.client.1.smithi027.stdout:1/273: creat d0/d1e/d25/d2b/d59/f5e x:0 0 0 2024-04-02T04:42:51.460 INFO:tasks.workunit.client.1.smithi027.stdout:7/306: write d2/d7/da/dc/d1f/d33/f55 [1175746,127015] 0 2024-04-02T04:42:51.462 INFO:tasks.workunit.client.1.smithi027.stdout:7/307: write d2/d7/da/dc/d16/d3a/d53/f5a [2067385,48789] 0 2024-04-02T04:42:51.465 INFO:tasks.workunit.client.0.smithi016.stdout:9/914: truncate d0/d9/db/dc9/f57 280442 0 2024-04-02T04:42:51.468 INFO:tasks.workunit.client.1.smithi027.stdout:3/243: dwrite d1/f8 [0,4194304] 0 2024-04-02T04:42:51.469 INFO:tasks.workunit.client.1.smithi027.stdout:3/244: fsync d1/de/df/f19 0 2024-04-02T04:42:51.472 INFO:tasks.workunit.client.1.smithi027.stdout:8/321: rename d5/d8/db/df/f4c to d5/d8/db/dd/d5f/d27/d31/d5b/f67 0 2024-04-02T04:42:51.473 INFO:tasks.workunit.client.1.smithi027.stdout:4/246: creat d24/d48/f4e x:0 0 0 2024-04-02T04:42:51.473 INFO:tasks.workunit.client.1.smithi027.stdout:8/322: write d5/d8/db/dd/d5f/f2b [866628,85029] 0 2024-04-02T04:42:51.477 INFO:tasks.workunit.client.1.smithi027.stdout:2/322: symlink d1/l6e 0 2024-04-02T04:42:51.479 INFO:tasks.workunit.client.1.smithi027.stdout:2/323: write d1/db/d3b/d17/d2b/f53 [293811,32805] 0 2024-04-02T04:42:51.481 INFO:tasks.workunit.client.1.smithi027.stdout:1/274: mkdir d0/d1e/d2e/d3e/d5f 0 2024-04-02T04:42:51.481 INFO:tasks.workunit.client.1.smithi027.stdout:4/247: dwrite f15 [0,4194304] 0 2024-04-02T04:42:51.482 INFO:tasks.workunit.client.1.smithi027.stdout:1/275: dread - d0/d1e/d2e/f4a zero size 2024-04-02T04:42:51.484 INFO:tasks.workunit.client.1.smithi027.stdout:2/324: truncate d1/db/d3b/d17/d12/f65 803282 0 2024-04-02T04:42:51.500 INFO:tasks.workunit.client.1.smithi027.stdout:8/323: mkdir d5/d8/db/d68 0 2024-04-02T04:42:51.504 INFO:tasks.workunit.client.1.smithi027.stdout:4/248: mknod d24/d38/d44/d41/c4f 0 2024-04-02T04:42:51.506 INFO:tasks.workunit.client.1.smithi027.stdout:8/324: dwrite d5/d8/db/d57/f62 [0,4194304] 0 2024-04-02T04:42:51.507 INFO:tasks.workunit.client.1.smithi027.stdout:8/325: readlink d5/d8/db/dd/l2f 0 2024-04-02T04:42:51.508 INFO:tasks.workunit.client.1.smithi027.stdout:2/325: symlink d1/db/l6f 0 2024-04-02T04:42:51.508 INFO:tasks.workunit.client.1.smithi027.stdout:8/326: chown d5/d8/db/dd/l2f 211814439 1 2024-04-02T04:42:51.510 INFO:tasks.workunit.client.1.smithi027.stdout:2/326: chown d1/db/d3b/d17/d2b/f52 203139 1 2024-04-02T04:42:51.510 INFO:tasks.workunit.client.1.smithi027.stdout:7/308: creat d2/d1d/d57/f5c x:0 0 0 2024-04-02T04:42:51.514 INFO:tasks.workunit.client.1.smithi027.stdout:1/276: mkdir d0/d1e/d5d/d60 0 2024-04-02T04:42:51.516 INFO:tasks.workunit.client.1.smithi027.stdout:4/249: symlink d24/d48/l50 0 2024-04-02T04:42:51.517 INFO:tasks.workunit.client.1.smithi027.stdout:4/250: write d24/d38/f3e [263720,71306] 0 2024-04-02T04:42:51.519 INFO:tasks.workunit.client.1.smithi027.stdout:8/327: symlink d5/d8/db/l69 0 2024-04-02T04:42:51.519 INFO:tasks.workunit.client.1.smithi027.stdout:4/251: write d24/d48/f4e [437185,101589] 0 2024-04-02T04:42:51.521 INFO:tasks.workunit.client.1.smithi027.stdout:0/351: getdents d4/d3d 0 2024-04-02T04:42:51.526 INFO:tasks.workunit.client.1.smithi027.stdout:7/309: creat d2/d7/da/dc/d16/d3a/d53/f5d x:0 0 0 2024-04-02T04:42:51.529 INFO:tasks.workunit.client.1.smithi027.stdout:5/414: getdents d0/df/d11/d13 0 2024-04-02T04:42:51.531 INFO:tasks.workunit.client.1.smithi027.stdout:1/277: rmdir d0/d1e/d2e/d3e 39 2024-04-02T04:42:51.532 INFO:tasks.workunit.client.1.smithi027.stdout:4/252: dread f23 [0,4194304] 0 2024-04-02T04:42:51.538 INFO:tasks.workunit.client.1.smithi027.stdout:8/328: rename d5/d8/db/dd/d5f/d27/l2c to d5/d8/db/dd/l6a 0 2024-04-02T04:42:51.540 INFO:tasks.workunit.client.1.smithi027.stdout:0/352: rmdir d4 39 2024-04-02T04:42:51.542 INFO:tasks.workunit.client.1.smithi027.stdout:8/329: dread d5/d8/d48/f5d [0,4194304] 0 2024-04-02T04:42:51.544 INFO:tasks.workunit.client.1.smithi027.stdout:7/310: creat d2/d1d/d57/f5e x:0 0 0 2024-04-02T04:42:51.546 INFO:tasks.workunit.client.1.smithi027.stdout:5/415: creat d0/df/d11/d13/f98 x:0 0 0 2024-04-02T04:42:51.547 INFO:tasks.workunit.client.1.smithi027.stdout:5/416: readlink d0/df/d29/l4a 0 2024-04-02T04:42:51.547 INFO:tasks.workunit.client.0.smithi016.stdout:4/815: sync 2024-04-02T04:42:51.547 INFO:tasks.workunit.client.0.smithi016.stdout:6/858: sync 2024-04-02T04:42:51.547 INFO:tasks.workunit.client.0.smithi016.stdout:0/784: sync 2024-04-02T04:42:51.548 INFO:tasks.workunit.client.0.smithi016.stdout:0/785: dread - d0/fc8 zero size 2024-04-02T04:42:51.548 INFO:tasks.workunit.client.0.smithi016.stdout:4/816: stat d5/d1d/d1e/da5/faa 0 2024-04-02T04:42:51.548 INFO:tasks.workunit.client.1.smithi027.stdout:1/278: fdatasync d0/f37 0 2024-04-02T04:42:51.550 INFO:tasks.workunit.client.1.smithi027.stdout:5/417: truncate d0/d1d/f97 482040 0 2024-04-02T04:42:51.551 INFO:tasks.workunit.client.1.smithi027.stdout:4/253: readlink d24/l40 0 2024-04-02T04:42:51.552 INFO:tasks.workunit.client.1.smithi027.stdout:4/254: write d24/f3c [449362,72887] 0 2024-04-02T04:42:51.552 INFO:tasks.workunit.client.1.smithi027.stdout:7/311: dwrite d2/d7/da/dc/d16/d3a/d53/f5a [0,4194304] 0 2024-04-02T04:42:51.553 INFO:tasks.workunit.client.1.smithi027.stdout:6/297: sync 2024-04-02T04:42:51.553 INFO:tasks.workunit.client.1.smithi027.stdout:4/255: fsync d24/f28 0 2024-04-02T04:42:51.554 INFO:tasks.workunit.client.1.smithi027.stdout:6/298: chown d13/d1b/d24/f3a 10586 1 2024-04-02T04:42:51.556 INFO:tasks.workunit.client.1.smithi027.stdout:6/299: dread - d13/d1b/d24/d28/d46/f59 zero size 2024-04-02T04:42:51.557 INFO:tasks.workunit.client.1.smithi027.stdout:6/300: fdatasync d13/d1b/d24/f3a 0 2024-04-02T04:42:51.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:51 smithi016 ceph-mon[28969]: pgmap v24: 65 pgs: 65 active+clean; 2.2 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 70 MiB/s rd, 283 MiB/s wr, 534 op/s 2024-04-02T04:42:51.561 INFO:tasks.workunit.client.1.smithi027.stdout:2/327: link d1/db/d3b/d17/d12/f41 d1/db/d3b/f70 0 2024-04-02T04:42:51.563 INFO:tasks.workunit.client.0.smithi016.stdout:7/816: creat d0/d46/d37/dbe/dae/df8/da9/f112 x:0 0 0 2024-04-02T04:42:51.569 INFO:tasks.workunit.client.1.smithi027.stdout:1/279: creat d0/d1e/d25/f61 x:0 0 0 2024-04-02T04:42:51.570 INFO:tasks.workunit.client.0.smithi016.stdout:6/859: dread d0/d9/d16/d56/d59/f96 [0,4194304] 0 2024-04-02T04:42:51.570 INFO:tasks.workunit.client.0.smithi016.stdout:0/786: mknod d0/d16/d2e/d36/d9f/c10b 0 2024-04-02T04:42:51.572 INFO:tasks.workunit.client.1.smithi027.stdout:5/418: chown d0/df/d11/d41/c54 174254 1 2024-04-02T04:42:51.573 INFO:tasks.workunit.client.0.smithi016.stdout:8/806: sync 2024-04-02T04:42:51.574 INFO:tasks.workunit.client.0.smithi016.stdout:2/877: symlink d12/d9f/d107/dee/l12d 0 2024-04-02T04:42:51.577 INFO:tasks.workunit.client.0.smithi016.stdout:9/915: mknod d0/d4a/d75/d8d/c11c 0 2024-04-02T04:42:51.578 INFO:tasks.workunit.client.0.smithi016.stdout:5/793: dwrite d3/d40/d4f/d6b/d45/da6/d102/dc4/d108/d6d/fa1 [0,4194304] 0 2024-04-02T04:42:51.580 INFO:tasks.workunit.client.1.smithi027.stdout:1/280: dwrite d0/de/f58 [0,4194304] 0 2024-04-02T04:42:51.582 INFO:tasks.workunit.client.1.smithi027.stdout:6/301: write d13/d1b/d24/d25/d27/f51 [1917429,70860] 0 2024-04-02T04:42:51.586 INFO:tasks.workunit.client.0.smithi016.stdout:2/878: unlink d12/d35/d101/f44 0 2024-04-02T04:42:51.588 INFO:tasks.workunit.client.1.smithi027.stdout:2/328: creat d1/db/d3b/d17/d2b/d2f/f71 x:0 0 0 2024-04-02T04:42:51.590 INFO:tasks.workunit.client.0.smithi016.stdout:5/794: dread d3/df/d18/d26/d70/ffe [0,4194304] 0 2024-04-02T04:42:51.592 INFO:tasks.workunit.client.1.smithi027.stdout:7/312: unlink d2/d1c/l49 0 2024-04-02T04:42:51.594 INFO:tasks.workunit.client.0.smithi016.stdout:6/860: mkdir d0/d9/d16/d56/db0/df2/d10d 0 2024-04-02T04:42:51.598 INFO:tasks.workunit.client.0.smithi016.stdout:8/807: dread d1/d6/d36/d3f/d40/d49/f44 [0,4194304] 0 2024-04-02T04:42:51.598 INFO:tasks.workunit.client.0.smithi016.stdout:0/787: mkdir d0/d16/d23/d3e/db3/d10c 0 2024-04-02T04:42:51.599 INFO:tasks.workunit.client.0.smithi016.stdout:5/795: truncate d3/df/d18/d26/f94 565691 0 2024-04-02T04:42:51.600 INFO:tasks.workunit.client.1.smithi027.stdout:5/419: dread d0/d1a/d20/f59 [0,4194304] 0 2024-04-02T04:42:51.600 INFO:tasks.workunit.client.1.smithi027.stdout:4/256: symlink d24/d2d/l51 0 2024-04-02T04:42:51.604 INFO:tasks.workunit.client.1.smithi027.stdout:0/353: mkdir d4/d21/d6f 0 2024-04-02T04:42:51.607 INFO:tasks.workunit.client.1.smithi027.stdout:6/302: symlink d13/d1b/d24/d25/d42/l61 0 2024-04-02T04:42:51.607 INFO:tasks.workunit.client.1.smithi027.stdout:8/330: link d5/d8/d15/c20 d5/d8/db/dd/d5f/d27/d31/c6b 0 2024-04-02T04:42:51.609 INFO:tasks.workunit.client.1.smithi027.stdout:8/331: read d5/d8/db/dd/d5f/f14 [4047006,22174] 0 2024-04-02T04:42:51.610 INFO:tasks.workunit.client.1.smithi027.stdout:1/281: creat d0/d1e/d2e/d3b/d4f/f62 x:0 0 0 2024-04-02T04:42:51.610 INFO:tasks.workunit.client.1.smithi027.stdout:0/354: dread d4/d38/d66/d2d/d4c/f5c [0,4194304] 0 2024-04-02T04:42:51.612 INFO:tasks.workunit.client.1.smithi027.stdout:4/257: creat d24/d38/d44/f52 x:0 0 0 2024-04-02T04:42:51.613 INFO:tasks.workunit.client.1.smithi027.stdout:5/420: creat d0/df/d11/d65/f99 x:0 0 0 2024-04-02T04:42:51.616 INFO:tasks.workunit.client.1.smithi027.stdout:2/329: mknod d1/db/d3b/d17/d6d/c72 0 2024-04-02T04:42:51.617 INFO:tasks.workunit.client.1.smithi027.stdout:2/330: write d1/db/d3b/d17/f4a [31493,126166] 0 2024-04-02T04:42:51.618 INFO:tasks.workunit.client.1.smithi027.stdout:2/331: fdatasync d1/db/d3b/d17/d29/f5a 0 2024-04-02T04:42:51.620 INFO:tasks.workunit.client.1.smithi027.stdout:7/313: link d2/d7/f48 d2/d1d/f5f 0 2024-04-02T04:42:51.621 INFO:tasks.workunit.client.1.smithi027.stdout:8/332: mkdir d5/d8/db/dd/d5f/d27/d31/d5b/d6c 0 2024-04-02T04:42:51.622 INFO:tasks.workunit.client.1.smithi027.stdout:1/282: mknod d0/d1e/c63 0 2024-04-02T04:42:51.623 INFO:tasks.workunit.client.1.smithi027.stdout:0/355: truncate d4/d38/d66/f24 897137 0 2024-04-02T04:42:51.624 INFO:tasks.workunit.client.1.smithi027.stdout:4/258: creat d24/d48/f53 x:0 0 0 2024-04-02T04:42:51.630 INFO:tasks.workunit.client.1.smithi027.stdout:8/333: mknod d5/d8/d48/c6d 0 2024-04-02T04:42:51.632 INFO:tasks.workunit.client.1.smithi027.stdout:1/283: mkdir d0/d1e/d25/d3c/d64 0 2024-04-02T04:42:51.633 INFO:tasks.workunit.client.1.smithi027.stdout:1/284: truncate d0/de/f4d 1402032 0 2024-04-02T04:42:51.634 INFO:tasks.workunit.client.1.smithi027.stdout:6/303: creat d13/f62 x:0 0 0 2024-04-02T04:42:51.636 INFO:tasks.workunit.client.1.smithi027.stdout:7/314: creat d2/d7/da/d35/f60 x:0 0 0 2024-04-02T04:42:51.636 INFO:tasks.workunit.client.1.smithi027.stdout:6/304: truncate d13/d1b/f23 192001 0 2024-04-02T04:42:51.637 INFO:tasks.workunit.client.0.smithi016.stdout:5/796: dread d3/f77 [0,4194304] 0 2024-04-02T04:42:51.639 INFO:tasks.workunit.client.1.smithi027.stdout:6/305: fdatasync d13/d1b/d24/d25/d42/f60 0 2024-04-02T04:42:51.640 INFO:tasks.workunit.client.0.smithi016.stdout:5/797: dread d3/df/d18/f98 [0,4194304] 0 2024-04-02T04:42:51.646 INFO:tasks.workunit.client.0.smithi016.stdout:8/808: symlink d1/d6/d36/d3f/d40/d49/l10a 0 2024-04-02T04:42:51.649 INFO:tasks.workunit.client.1.smithi027.stdout:5/421: truncate d0/df/d11/d41/f90 3004848 0 2024-04-02T04:42:51.650 INFO:tasks.workunit.client.1.smithi027.stdout:4/259: mknod d24/c54 0 2024-04-02T04:42:51.650 INFO:tasks.workunit.client.1.smithi027.stdout:0/356: dread d4/d38/d66/f2f [0,4194304] 0 2024-04-02T04:42:51.651 INFO:tasks.workunit.client.1.smithi027.stdout:4/260: chown d24 55722 1 2024-04-02T04:42:51.654 INFO:tasks.workunit.client.1.smithi027.stdout:1/285: dwrite d0/d1e/d25/f61 [0,4194304] 0 2024-04-02T04:42:51.655 INFO:tasks.workunit.client.1.smithi027.stdout:7/315: truncate d2/d7/da/dc/d16/d3a/d4d/f58 673558 0 2024-04-02T04:42:51.655 INFO:tasks.workunit.client.1.smithi027.stdout:8/334: link d5/d8/db/dd/l2f d5/d8/db/dd/d5f/d27/d31/l6e 0 2024-04-02T04:42:51.656 INFO:tasks.workunit.client.1.smithi027.stdout:1/286: chown d0/f54 1371478 1 2024-04-02T04:42:51.658 INFO:tasks.workunit.client.1.smithi027.stdout:7/316: write d2/d1c/f40 [156311,106113] 0 2024-04-02T04:42:51.663 INFO:tasks.workunit.client.1.smithi027.stdout:5/422: mkdir d0/df/d11/d13/d9a 0 2024-04-02T04:42:51.663 INFO:tasks.workunit.client.0.smithi016.stdout:3/925: sync 2024-04-02T04:42:51.663 INFO:tasks.workunit.client.0.smithi016.stdout:1/796: sync 2024-04-02T04:42:51.663 INFO:tasks.workunit.client.0.smithi016.stdout:6/861: rename d0/d6e/f8b to d0/d9/d16/d56/f10e 0 2024-04-02T04:42:51.664 INFO:tasks.workunit.client.0.smithi016.stdout:5/798: symlink d3/d40/d4f/d6b/d63/l11c 0 2024-04-02T04:42:51.664 INFO:tasks.workunit.client.0.smithi016.stdout:2/879: getdents d12/d35/d3f/d2b/de9/d102/d103 0 2024-04-02T04:42:51.665 INFO:tasks.workunit.client.1.smithi027.stdout:6/306: symlink d13/d1b/d48/l63 0 2024-04-02T04:42:51.665 INFO:tasks.workunit.client.1.smithi027.stdout:7/317: fsync d2/d7/da/f1e 0 2024-04-02T04:42:51.665 INFO:tasks.workunit.client.0.smithi016.stdout:3/926: write d2/d15/fb7 [780647,101491] 0 2024-04-02T04:42:51.665 INFO:tasks.workunit.client.1.smithi027.stdout:4/261: dread f15 [0,4194304] 0 2024-04-02T04:42:51.666 INFO:tasks.workunit.client.1.smithi027.stdout:0/357: dwrite d4/d38/d66/f6c [0,4194304] 0 2024-04-02T04:42:51.666 INFO:tasks.workunit.client.0.smithi016.stdout:2/880: fsync d12/d35/d101/d76/da0/de7/dfc/d106/f12a 0 2024-04-02T04:42:51.670 INFO:tasks.workunit.client.1.smithi027.stdout:8/335: dread d5/d8/db/dd/d5f/d27/d31/f5e [0,4194304] 0 2024-04-02T04:42:51.675 INFO:tasks.workunit.client.1.smithi027.stdout:1/287: creat d0/de/d52/f65 x:0 0 0 2024-04-02T04:42:51.679 INFO:tasks.workunit.client.0.smithi016.stdout:1/797: symlink d14/d19/d1d/d29/d39/d4d/l108 0 2024-04-02T04:42:51.684 INFO:tasks.workunit.client.0.smithi016.stdout:2/881: mknod d12/d35/d101/d76/da0/de7/df8/c12e 0 2024-04-02T04:42:51.685 INFO:tasks.workunit.client.1.smithi027.stdout:6/307: write d13/d1b/f2b [2004935,80701] 0 2024-04-02T04:42:51.686 INFO:tasks.workunit.client.0.smithi016.stdout:0/788: truncate d0/d16/d24/d62/d9c/f59 2838248 0 2024-04-02T04:42:51.692 INFO:tasks.workunit.client.1.smithi027.stdout:7/318: rmdir d2/d7/da/dc 39 2024-04-02T04:42:51.693 INFO:tasks.workunit.client.0.smithi016.stdout:3/927: symlink d2/d15/d70/dcf/d102/l139 0 2024-04-02T04:42:51.694 INFO:tasks.workunit.client.0.smithi016.stdout:8/809: write d1/d6/d36/d3f/d77/fa7 [4585047,22922] 0 2024-04-02T04:42:51.695 INFO:tasks.workunit.client.0.smithi016.stdout:2/882: mkdir d12/d35/d3b/d42/d59/d104/d12f 0 2024-04-02T04:42:51.696 INFO:tasks.workunit.client.0.smithi016.stdout:8/810: dread - d1/d6/d36/d3f/d77/f106 zero size 2024-04-02T04:42:51.696 INFO:tasks.workunit.client.0.smithi016.stdout:8/811: readlink d1/d31/lb1 0 2024-04-02T04:42:51.698 INFO:tasks.workunit.client.0.smithi016.stdout:5/799: write d3/d40/d4f/d6b/f9e [968012,104634] 0 2024-04-02T04:42:51.700 INFO:tasks.workunit.client.0.smithi016.stdout:1/798: dread d14/d19/f82 [8388608,4194304] 0 2024-04-02T04:42:51.701 INFO:tasks.workunit.client.1.smithi027.stdout:0/358: dwrite d4/d3d/f47 [0,4194304] 0 2024-04-02T04:42:51.703 INFO:tasks.workunit.client.0.smithi016.stdout:2/883: creat d12/d35/d101/d76/da0/de7/f130 x:0 0 0 2024-04-02T04:42:51.704 INFO:tasks.workunit.client.0.smithi016.stdout:2/884: write d12/d35/d3f/f39 [909480,35263] 0 2024-04-02T04:42:51.706 INFO:tasks.workunit.client.0.smithi016.stdout:2/885: truncate d12/d35/d3f/d56/dab/f1e 2185032 0 2024-04-02T04:42:51.707 INFO:tasks.workunit.client.0.smithi016.stdout:2/886: chown d12/d35/d3b/d42/d67/c6b 2318226 1 2024-04-02T04:42:51.707 INFO:tasks.workunit.client.1.smithi027.stdout:4/262: symlink d24/l55 0 2024-04-02T04:42:51.711 INFO:tasks.workunit.client.0.smithi016.stdout:5/800: unlink d3/df/c21 0 2024-04-02T04:42:51.711 INFO:tasks.workunit.client.0.smithi016.stdout:5/801: stat d3/df/d18/d26/c109 0 2024-04-02T04:42:51.712 INFO:tasks.workunit.client.0.smithi016.stdout:5/802: write d3/f36 [2093578,123254] 0 2024-04-02T04:42:51.713 INFO:tasks.workunit.client.1.smithi027.stdout:6/308: unlink d13/d1b/d24/d28/d3c/f53 0 2024-04-02T04:42:51.715 INFO:tasks.workunit.client.1.smithi027.stdout:5/423: creat d0/d1d/d61/d7e/f9b x:0 0 0 2024-04-02T04:42:51.717 INFO:tasks.workunit.client.0.smithi016.stdout:3/928: write d2/f1f [4530278,28616] 0 2024-04-02T04:42:51.718 INFO:tasks.workunit.client.0.smithi016.stdout:3/929: dread - d2/d15/d60/f115 zero size 2024-04-02T04:42:51.719 INFO:tasks.workunit.client.1.smithi027.stdout:7/319: fdatasync d2/d7/da/dc/d16/d2f/f27 0 2024-04-02T04:42:51.721 INFO:tasks.workunit.client.0.smithi016.stdout:8/812: symlink d1/d6/d36/d3f/d40/d63/l10b 0 2024-04-02T04:42:51.724 INFO:tasks.workunit.client.1.smithi027.stdout:1/288: symlink d0/d1e/d2e/l66 0 2024-04-02T04:42:51.726 INFO:tasks.workunit.client.0.smithi016.stdout:8/813: dwrite d1/d31/d67/fd1 [0,4194304] 0 2024-04-02T04:42:51.728 INFO:tasks.workunit.client.0.smithi016.stdout:1/799: write d14/d19/d1e/d71/f104 [487679,21482] 0 2024-04-02T04:42:51.728 INFO:tasks.workunit.client.1.smithi027.stdout:0/359: creat d4/d38/d66/d4a/f70 x:0 0 0 2024-04-02T04:42:51.729 INFO:tasks.workunit.client.0.smithi016.stdout:1/800: readlink l1 0 2024-04-02T04:42:51.734 INFO:tasks.workunit.client.1.smithi027.stdout:6/309: rename d13/d1b/f20 to d13/d1b/d48/d58/f64 0 2024-04-02T04:42:51.735 INFO:tasks.workunit.client.0.smithi016.stdout:2/887: dwrite d12/d35/fd9 [0,4194304] 0 2024-04-02T04:42:51.738 INFO:tasks.workunit.client.1.smithi027.stdout:7/320: mkdir d2/d7/da/dc/d1f/d61 0 2024-04-02T04:42:51.740 INFO:tasks.workunit.client.0.smithi016.stdout:2/888: dread d12/d35/d3b/d42/d67/fb3 [0,4194304] 0 2024-04-02T04:42:51.740 INFO:tasks.workunit.client.0.smithi016.stdout:1/801: unlink d14/d19/d1e/d5e/fc3 0 2024-04-02T04:42:51.741 INFO:tasks.workunit.client.1.smithi027.stdout:4/263: symlink d24/d38/d44/d41/d49/l56 0 2024-04-02T04:42:51.742 INFO:tasks.workunit.client.0.smithi016.stdout:3/930: symlink d2/l13a 0 2024-04-02T04:42:51.742 INFO:tasks.workunit.client.1.smithi027.stdout:0/360: mkdir d4/d16/d71 0 2024-04-02T04:42:51.748 INFO:tasks.workunit.client.1.smithi027.stdout:1/289: rename d0/f14 to d0/de/d52/f67 0 2024-04-02T04:42:51.749 INFO:tasks.workunit.client.1.smithi027.stdout:5/424: mkdir d0/df/d11/d13/d9a/d9c 0 2024-04-02T04:42:51.751 INFO:tasks.workunit.client.1.smithi027.stdout:6/310: creat d13/d1b/d24/d28/f65 x:0 0 0 2024-04-02T04:42:51.753 INFO:tasks.workunit.client.1.smithi027.stdout:8/336: getdents d5/d8/db/dd 0 2024-04-02T04:42:51.756 INFO:tasks.workunit.client.1.smithi027.stdout:4/264: fdatasync f1d 0 2024-04-02T04:42:51.757 INFO:tasks.workunit.client.0.smithi016.stdout:3/931: dread - d2/d2e/ff1 zero size 2024-04-02T04:42:51.758 INFO:tasks.workunit.client.0.smithi016.stdout:3/932: chown d2/d2e/d36/d48/fe1 0 1 2024-04-02T04:42:51.758 INFO:tasks.workunit.client.0.smithi016.stdout:3/933: chown d2/d2e/fbf 5 1 2024-04-02T04:42:51.759 INFO:tasks.workunit.client.1.smithi027.stdout:1/290: symlink d0/d1e/d2e/d3b/l68 0 2024-04-02T04:42:51.760 INFO:tasks.workunit.client.1.smithi027.stdout:6/311: dwrite d13/f62 [0,4194304] 0 2024-04-02T04:42:51.762 INFO:tasks.workunit.client.0.smithi016.stdout:2/889: write d12/d9f/fa7 [678446,24362] 0 2024-04-02T04:42:51.767 INFO:tasks.workunit.client.0.smithi016.stdout:4/817: sync 2024-04-02T04:42:51.767 INFO:tasks.workunit.client.0.smithi016.stdout:3/934: read d2/d2e/fd7 [3738256,112495] 0 2024-04-02T04:42:51.768 INFO:tasks.workunit.client.1.smithi027.stdout:4/265: mknod d24/d38/d44/d41/c57 0 2024-04-02T04:42:51.770 INFO:tasks.workunit.client.0.smithi016.stdout:8/814: link d1/d6/d36/d3f/d40/d63/dc5/fe0 d1/d70/f10c 0 2024-04-02T04:42:51.771 INFO:tasks.workunit.client.0.smithi016.stdout:5/803: getdents d3/d40/d4f 0 2024-04-02T04:42:51.774 INFO:tasks.workunit.client.0.smithi016.stdout:1/802: rename d14/d19/d1d/d45/f92 to d14/d19/f109 0 2024-04-02T04:42:51.776 INFO:tasks.workunit.client.1.smithi027.stdout:6/312: dread d13/d1b/d24/d25/d27/f51 [0,4194304] 0 2024-04-02T04:42:51.778 INFO:tasks.workunit.client.1.smithi027.stdout:8/337: symlink d5/d8/db/dd/d4a/l6f 0 2024-04-02T04:42:51.781 INFO:tasks.workunit.client.0.smithi016.stdout:8/815: chown d1/d70/ff5 3068 1 2024-04-02T04:42:51.785 INFO:tasks.workunit.client.0.smithi016.stdout:2/890: rename d12/d35/d3f/d2b/da3/cdb to d12/d35/d101/d76/c131 0 2024-04-02T04:42:51.786 INFO:tasks.workunit.client.0.smithi016.stdout:1/803: mknod d14/d19/d1d/d29/d39/d46/dcb/c10a 0 2024-04-02T04:42:51.788 INFO:tasks.workunit.client.0.smithi016.stdout:4/818: link d5/d8/fb d5/de/dd5/ff7 0 2024-04-02T04:42:51.788 INFO:tasks.workunit.client.1.smithi027.stdout:7/321: getdents d2/d1d 0 2024-04-02T04:42:51.788 INFO:tasks.workunit.client.1.smithi027.stdout:7/322: chown d2/d7/f18 9947832 1 2024-04-02T04:42:51.789 INFO:tasks.workunit.client.1.smithi027.stdout:7/323: write d2/d7/da/dc/d16/d3a/d53/f5d [174197,51253] 0 2024-04-02T04:42:51.794 INFO:tasks.workunit.client.1.smithi027.stdout:1/291: getdents d0/d1e/d2e/d3e/d5f 0 2024-04-02T04:42:51.794 INFO:tasks.workunit.client.1.smithi027.stdout:1/292: stat d0/d1e/d2e/d3e/c56 0 2024-04-02T04:42:51.795 INFO:tasks.workunit.client.1.smithi027.stdout:1/293: stat d0/d1e/d25/d2b 0 2024-04-02T04:42:51.796 INFO:tasks.workunit.client.1.smithi027.stdout:5/425: getdents d0/df/d11/d41 0 2024-04-02T04:42:51.797 INFO:tasks.workunit.client.0.smithi016.stdout:8/816: fsync d1/d6/d36/d3f/d40/d45/fe2 0 2024-04-02T04:42:51.800 INFO:tasks.workunit.client.0.smithi016.stdout:3/935: truncate d2/d2e/d36/d66/d90/fef 2156439 0 2024-04-02T04:42:51.803 INFO:tasks.workunit.client.0.smithi016.stdout:5/804: truncate d3/f1a 2083343 0 2024-04-02T04:42:51.804 INFO:tasks.workunit.client.1.smithi027.stdout:6/313: dwrite d13/d1b/d24/d28/f2a [0,4194304] 0 2024-04-02T04:42:51.806 INFO:tasks.workunit.client.0.smithi016.stdout:4/819: rename d5/d17/d76 to d5/d16/df8 0 2024-04-02T04:42:51.806 INFO:tasks.workunit.client.1.smithi027.stdout:6/314: readlink d13/d1b/d24/d28/d3c/l5e 0 2024-04-02T04:42:51.808 INFO:tasks.workunit.client.1.smithi027.stdout:8/338: dwrite d5/d8/db/dd/d5f/f14 [0,4194304] 0 2024-04-02T04:42:51.808 INFO:tasks.workunit.client.1.smithi027.stdout:6/315: write d13/f16 [2076364,52096] 0 2024-04-02T04:42:51.811 INFO:tasks.workunit.client.1.smithi027.stdout:4/266: link d24/d38/d44/d41/d49/l56 d24/l58 0 2024-04-02T04:42:51.812 INFO:tasks.workunit.client.1.smithi027.stdout:4/267: dread - d24/f3f zero size 2024-04-02T04:42:51.817 INFO:tasks.workunit.client.0.smithi016.stdout:3/936: mkdir d2/d2e/d36/d66/d90/d13b 0 2024-04-02T04:42:51.817 INFO:tasks.workunit.client.1.smithi027.stdout:9/304: sync 2024-04-02T04:42:51.819 INFO:tasks.workunit.client.1.smithi027.stdout:6/316: dwrite d13/d1b/d24/d28/d46/f59 [0,4194304] 0 2024-04-02T04:42:51.820 INFO:tasks.workunit.client.0.smithi016.stdout:1/804: dwrite d14/d19/d1d/d29/f6a [0,4194304] 0 2024-04-02T04:42:51.823 INFO:tasks.workunit.client.1.smithi027.stdout:9/305: read d1a/d1d/d29/d37/d3a/f54 [1025616,97510] 0 2024-04-02T04:42:51.823 INFO:tasks.workunit.client.0.smithi016.stdout:9/916: sync 2024-04-02T04:42:51.823 INFO:tasks.workunit.client.0.smithi016.stdout:7/817: sync 2024-04-02T04:42:51.824 INFO:tasks.workunit.client.1.smithi027.stdout:1/294: rename d0/de/f58 to d0/f69 0 2024-04-02T04:42:51.825 INFO:tasks.workunit.client.0.smithi016.stdout:4/820: mkdir d5/d1d/d1e/da5/df9 0 2024-04-02T04:42:51.831 INFO:tasks.workunit.client.0.smithi016.stdout:5/805: dread d3/f5b [0,4194304] 0 2024-04-02T04:42:51.834 INFO:tasks.workunit.client.0.smithi016.stdout:2/891: rename d12/d35/d3b/f83 to d12/d35/d3b/f132 0 2024-04-02T04:42:51.838 INFO:tasks.workunit.client.1.smithi027.stdout:9/306: rmdir d1a/d1d/d29/d50 39 2024-04-02T04:42:51.840 INFO:tasks.workunit.client.0.smithi016.stdout:8/817: rename d1/d6/d36/d3f/d40/d49/d38/f90 to d1/d6/d36/d3f/d40/d49/d4a/f10d 0 2024-04-02T04:42:51.841 INFO:tasks.workunit.client.1.smithi027.stdout:5/426: rename d0/d5/c6e to d0/d1a/c9d 0 2024-04-02T04:42:51.841 INFO:tasks.workunit.client.0.smithi016.stdout:8/818: read d1/d6/d36/d3f/d40/d45/f5b [3988944,48358] 0 2024-04-02T04:42:51.847 INFO:tasks.workunit.client.1.smithi027.stdout:6/317: link d13/d1b/d24/d28/f2a d13/d1b/d48/f66 0 2024-04-02T04:42:51.849 INFO:tasks.workunit.client.1.smithi027.stdout:9/307: stat c3 0 2024-04-02T04:42:51.854 INFO:tasks.workunit.client.1.smithi027.stdout:8/339: rename d5/d8/db/dd/d4a/l6f to d5/d8/db/d68/l70 0 2024-04-02T04:42:51.855 INFO:tasks.workunit.client.0.smithi016.stdout:3/937: dread - d2/d2e/d36/d48/da2/d88/f103 zero size 2024-04-02T04:42:51.856 INFO:tasks.workunit.client.1.smithi027.stdout:5/427: rmdir d0/df/d11/d41 39 2024-04-02T04:42:51.858 INFO:tasks.workunit.client.1.smithi027.stdout:9/308: dwrite d1a/d1d/f53 [0,4194304] 0 2024-04-02T04:42:51.861 INFO:tasks.workunit.client.0.smithi016.stdout:8/819: chown d1/d70/d71/d74/cb2 19 1 2024-04-02T04:42:51.861 INFO:tasks.workunit.client.1.smithi027.stdout:4/268: write d24/f46 [1003461,78740] 0 2024-04-02T04:42:51.863 INFO:tasks.workunit.client.1.smithi027.stdout:6/318: dread d13/f16 [0,4194304] 0 2024-04-02T04:42:51.864 INFO:tasks.workunit.client.1.smithi027.stdout:1/295: dread d0/de/d52/f67 [0,4194304] 0 2024-04-02T04:42:51.868 INFO:tasks.workunit.client.0.smithi016.stdout:4/821: link d5/d8/d43/dd7/fe4 d5/d1d/d1e/da5/ffa 0 2024-04-02T04:42:51.869 INFO:tasks.workunit.client.0.smithi016.stdout:4/822: chown d5/d17/d18/d6d/d8e/d9c/dcc/d6f/fc5 29590341 1 2024-04-02T04:42:51.869 INFO:tasks.workunit.client.0.smithi016.stdout:7/818: write d0/d46/d52/daf/fdd [685441,46022] 0 2024-04-02T04:42:51.871 INFO:tasks.workunit.client.0.smithi016.stdout:5/806: symlink d3/l11d 0 2024-04-02T04:42:51.872 INFO:tasks.workunit.client.1.smithi027.stdout:5/428: stat d0/d5/l94 0 2024-04-02T04:42:51.877 INFO:tasks.workunit.client.0.smithi016.stdout:3/938: write d2/d15/d16/d8a/dcd/f116 [6246278,29375] 0 2024-04-02T04:42:51.878 INFO:tasks.workunit.client.0.smithi016.stdout:3/939: chown d2/d2e/d36/lcc 71 1 2024-04-02T04:42:51.879 INFO:tasks.workunit.client.1.smithi027.stdout:6/319: mknod d13/d1b/d48/c67 0 2024-04-02T04:42:51.880 INFO:tasks.workunit.client.0.smithi016.stdout:2/892: dwrite d12/d35/d3f/d56/fdd [0,4194304] 0 2024-04-02T04:42:51.881 INFO:tasks.workunit.client.1.smithi027.stdout:1/296: mknod d0/de/c6a 0 2024-04-02T04:42:51.882 INFO:tasks.workunit.client.1.smithi027.stdout:1/297: write d0/de/f38 [67109,85775] 0 2024-04-02T04:42:51.883 INFO:tasks.workunit.client.1.smithi027.stdout:8/340: write d5/d8/f17 [877160,35066] 0 2024-04-02T04:42:51.885 INFO:tasks.workunit.client.0.smithi016.stdout:5/807: mknod d3/d40/d4f/d6b/d63/c11e 0 2024-04-02T04:42:51.885 INFO:tasks.workunit.client.0.smithi016.stdout:7/819: dread d0/d46/f30 [4194304,4194304] 0 2024-04-02T04:42:51.887 INFO:tasks.workunit.client.1.smithi027.stdout:9/309: symlink d1a/d1d/d29/d37/l6d 0 2024-04-02T04:42:51.888 INFO:tasks.workunit.client.1.smithi027.stdout:4/269: mknod d24/d38/d47/c59 0 2024-04-02T04:42:51.891 INFO:tasks.workunit.client.0.smithi016.stdout:8/820: dread d1/d6/f2e [0,4194304] 0 2024-04-02T04:42:51.892 INFO:tasks.workunit.client.1.smithi027.stdout:4/270: dread d24/f46 [0,4194304] 0 2024-04-02T04:42:51.893 INFO:tasks.workunit.client.0.smithi016.stdout:4/823: mknod d5/d17/d18/d6d/d91/dd6/cfb 0 2024-04-02T04:42:51.894 INFO:tasks.workunit.client.1.smithi027.stdout:4/271: truncate d24/d38/d44/d41/f4b 751476 0 2024-04-02T04:42:51.894 INFO:tasks.workunit.client.1.smithi027.stdout:5/429: symlink d0/df/d11/d8c/l9e 0 2024-04-02T04:42:51.895 INFO:tasks.workunit.client.1.smithi027.stdout:4/272: write d24/f2c [16446,113475] 0 2024-04-02T04:42:51.895 INFO:tasks.workunit.client.1.smithi027.stdout:5/430: chown d0/d1d/d61/d7e 780066793 1 2024-04-02T04:42:51.896 INFO:tasks.workunit.client.1.smithi027.stdout:5/431: chown d0/d5/l94 10 1 2024-04-02T04:42:51.898 INFO:tasks.workunit.client.1.smithi027.stdout:5/432: write d0/d1d/f35 [4627975,79916] 0 2024-04-02T04:42:51.904 INFO:tasks.workunit.client.1.smithi027.stdout:9/310: dread d1a/d1d/d29/d37/d4c/f62 [0,4194304] 0 2024-04-02T04:42:51.905 INFO:tasks.workunit.client.1.smithi027.stdout:8/341: dwrite d5/d8/db/dd/d5f/d27/f50 [0,4194304] 0 2024-04-02T04:42:51.905 INFO:tasks.workunit.client.0.smithi016.stdout:2/893: truncate d12/d35/d101/f25 326019 0 2024-04-02T04:42:51.908 INFO:tasks.workunit.client.0.smithi016.stdout:4/824: creat d5/d17/d18/d6d/d8e/ffc x:0 0 0 2024-04-02T04:42:51.908 INFO:tasks.workunit.client.1.smithi027.stdout:8/342: truncate d5/d8/db/dd/d5f/f24 5824928 0 2024-04-02T04:42:51.909 INFO:tasks.workunit.client.0.smithi016.stdout:4/825: chown d5/de/fbc 25825724 1 2024-04-02T04:42:51.912 INFO:tasks.workunit.client.1.smithi027.stdout:8/343: write d5/d8/db/f29 [822818,51651] 0 2024-04-02T04:42:51.913 INFO:tasks.workunit.client.0.smithi016.stdout:5/808: symlink d3/l11f 0 2024-04-02T04:42:51.913 INFO:tasks.workunit.client.0.smithi016.stdout:3/940: write d2/d15/d16/d8a/dd1/d8b/f9c [756727,102521] 0 2024-04-02T04:42:51.913 INFO:tasks.workunit.client.0.smithi016.stdout:6/862: sync 2024-04-02T04:42:51.913 INFO:tasks.workunit.client.0.smithi016.stdout:0/789: sync 2024-04-02T04:42:51.913 INFO:tasks.workunit.client.0.smithi016.stdout:5/809: chown d3/df 12409641 1 2024-04-02T04:42:51.914 INFO:tasks.workunit.client.1.smithi027.stdout:9/311: dwrite d1a/d1d/f24 [0,4194304] 0 2024-04-02T04:42:51.914 INFO:tasks.workunit.client.1.smithi027.stdout:4/273: mkdir d24/d38/d44/d41/d5a 0 2024-04-02T04:42:51.914 INFO:tasks.workunit.client.0.smithi016.stdout:0/790: read - d0/fc8 zero size 2024-04-02T04:42:51.915 INFO:tasks.workunit.client.0.smithi016.stdout:0/791: dread - d0/d4f/f100 zero size 2024-04-02T04:42:51.916 INFO:tasks.workunit.client.0.smithi016.stdout:0/792: chown d0/d4f/f100 6 1 2024-04-02T04:42:51.918 INFO:tasks.workunit.client.1.smithi027.stdout:5/433: rmdir d0/d5/d55 39 2024-04-02T04:42:51.919 INFO:tasks.workunit.client.1.smithi027.stdout:8/344: dread d5/d8/d48/f5d [0,4194304] 0 2024-04-02T04:42:51.921 INFO:tasks.workunit.client.0.smithi016.stdout:7/820: dwrite d0/d46/d16/f8e [0,4194304] 0 2024-04-02T04:42:51.921 INFO:tasks.workunit.client.0.smithi016.stdout:2/894: unlink d12/d35/d3b/c123 0 2024-04-02T04:42:51.922 INFO:tasks.workunit.client.0.smithi016.stdout:2/895: stat d12/f4c 0 2024-04-02T04:42:51.923 INFO:tasks.workunit.client.0.smithi016.stdout:2/896: chown d12/d35/d3f/d56/d66 0 1 2024-04-02T04:42:51.923 INFO:tasks.workunit.client.0.smithi016.stdout:8/821: write d1/d6/d36/d4f/d5d/f5e [574794,63759] 0 2024-04-02T04:42:51.924 INFO:tasks.workunit.client.0.smithi016.stdout:8/822: fdatasync d1/d31/fe3 0 2024-04-02T04:42:51.925 INFO:tasks.workunit.client.1.smithi027.stdout:6/320: link d13/d1b/l2f d13/d1b/d24/d25/d56/l68 0 2024-04-02T04:42:51.926 INFO:tasks.workunit.client.1.smithi027.stdout:6/321: chown d13/l49 754180493 1 2024-04-02T04:42:51.931 INFO:tasks.workunit.client.0.smithi016.stdout:8/823: dwrite d1/d70/ff5 [0,4194304] 0 2024-04-02T04:42:51.931 INFO:tasks.workunit.client.0.smithi016.stdout:7/821: dread d0/d46/dbf/fce [0,4194304] 0 2024-04-02T04:42:51.933 INFO:tasks.workunit.client.0.smithi016.stdout:7/822: rename d0/d46/d37/dbe/dae to d0/d46/d37/dbe/dae/df8/d43/d96/d113 22 2024-04-02T04:42:51.936 INFO:tasks.workunit.client.0.smithi016.stdout:7/823: write d0/d46/d37/dbe/dae/df8/d84/fd0 [171291,109168] 0 2024-04-02T04:42:51.938 INFO:tasks.workunit.client.0.smithi016.stdout:0/793: rmdir d0 39 2024-04-02T04:42:51.938 INFO:tasks.workunit.client.0.smithi016.stdout:8/824: dwrite d1/d6/d36/d4f/f101 [0,4194304] 0 2024-04-02T04:42:51.944 INFO:tasks.workunit.client.0.smithi016.stdout:7/824: dwrite d0/d46/f2c [0,4194304] 0 2024-04-02T04:42:51.944 INFO:tasks.workunit.client.0.smithi016.stdout:6/863: write d0/d9/d16/f23 [2014041,97723] 0 2024-04-02T04:42:51.945 INFO:tasks.workunit.client.1.smithi027.stdout:4/274: mknod d24/d2d/c5b 0 2024-04-02T04:42:51.945 INFO:tasks.workunit.client.0.smithi016.stdout:7/825: chown d0/d46/d37/dbe/dae/df8/d84/d8a 2 1 2024-04-02T04:42:51.948 INFO:tasks.workunit.client.0.smithi016.stdout:3/941: fsync d2/d2e/d36/d66/d90/fef 0 2024-04-02T04:42:51.951 INFO:tasks.workunit.client.1.smithi027.stdout:8/345: mkdir d5/d8/d15/d71 0 2024-04-02T04:42:51.953 INFO:tasks.workunit.client.0.smithi016.stdout:5/810: symlink d3/d40/db6/dfc/l120 0 2024-04-02T04:42:51.956 INFO:tasks.workunit.client.1.smithi027.stdout:6/322: symlink d13/d1b/d24/d25/d42/l69 0 2024-04-02T04:42:51.957 INFO:tasks.workunit.client.1.smithi027.stdout:1/298: getdents d0/d1e/d5d 0 2024-04-02T04:42:51.960 INFO:tasks.workunit.client.1.smithi027.stdout:8/346: dwrite d5/d8/f17 [0,4194304] 0 2024-04-02T04:42:51.963 INFO:tasks.workunit.client.1.smithi027.stdout:9/312: write d1a/d1d/d29/d37/d4c/f62 [370704,47123] 0 2024-04-02T04:42:51.963 INFO:tasks.workunit.client.1.smithi027.stdout:8/347: chown d5/d8/db/dd/d5f/d27/l3d 1 1 2024-04-02T04:42:51.964 INFO:tasks.workunit.client.1.smithi027.stdout:8/348: chown d5/d8 12893259 1 2024-04-02T04:42:51.967 INFO:tasks.workunit.client.1.smithi027.stdout:5/434: write d0/f66 [187916,105116] 0 2024-04-02T04:42:51.967 INFO:tasks.workunit.client.1.smithi027.stdout:8/349: dread - d5/d8/db/dd/f4f zero size 2024-04-02T04:42:51.969 INFO:tasks.workunit.client.1.smithi027.stdout:1/299: dwrite d0/f30 [4194304,4194304] 0 2024-04-02T04:42:51.970 INFO:tasks.workunit.client.0.smithi016.stdout:8/825: creat d1/d70/d71/d74/d82/f10e x:0 0 0 2024-04-02T04:42:51.970 INFO:tasks.workunit.client.1.smithi027.stdout:5/435: write d0/d1d/d61/f96 [901126,8866] 0 2024-04-02T04:42:51.972 INFO:tasks.workunit.client.1.smithi027.stdout:5/436: chown d0/df/d29 14980 1 2024-04-02T04:42:51.972 INFO:tasks.workunit.client.1.smithi027.stdout:8/350: write d5/d8/db/dd/d5f/f24 [5323693,120865] 0 2024-04-02T04:42:51.973 INFO:tasks.workunit.client.1.smithi027.stdout:9/313: dread d1a/d1d/f53 [0,4194304] 0 2024-04-02T04:42:51.974 INFO:tasks.workunit.client.0.smithi016.stdout:9/917: sync 2024-04-02T04:42:51.974 INFO:tasks.workunit.client.0.smithi016.stdout:1/805: sync 2024-04-02T04:42:51.975 INFO:tasks.workunit.client.1.smithi027.stdout:6/323: creat d13/d1b/d48/d58/f6a x:0 0 0 2024-04-02T04:42:51.975 INFO:tasks.workunit.client.1.smithi027.stdout:8/351: chown d5/d8/db/l2a 516974146 1 2024-04-02T04:42:51.975 INFO:tasks.workunit.client.1.smithi027.stdout:9/314: chown d1a/f63 2753856 1 2024-04-02T04:42:51.978 INFO:tasks.workunit.client.0.smithi016.stdout:7/826: fsync d0/f22 0 2024-04-02T04:42:51.981 INFO:tasks.workunit.client.0.smithi016.stdout:1/806: dread d14/d19/d1d/d45/d4f/d86/db4/fd3 [0,4194304] 0 2024-04-02T04:42:51.982 INFO:tasks.workunit.client.1.smithi027.stdout:5/437: mkdir d0/df/d11/d13/d9a/d9f 0 2024-04-02T04:42:51.983 INFO:tasks.workunit.client.0.smithi016.stdout:6/864: mkdir d0/d9/d16/d56/d95/d10f 0 2024-04-02T04:42:51.983 INFO:tasks.workunit.client.1.smithi027.stdout:3/245: sync 2024-04-02T04:42:51.984 INFO:tasks.workunit.client.1.smithi027.stdout:8/352: unlink d5/c65 0 2024-04-02T04:42:51.985 INFO:tasks.workunit.client.0.smithi016.stdout:1/807: write d14/d19/d1d/d29/d39/d46/dcb/d43/d68/dc1/fc4 [2003876,97873] 0 2024-04-02T04:42:51.986 INFO:tasks.workunit.client.0.smithi016.stdout:3/942: dread d2/f3f [0,4194304] 0 2024-04-02T04:42:51.988 INFO:tasks.workunit.client.0.smithi016.stdout:5/811: rmdir d3/d40/d4f/d67 39 2024-04-02T04:42:51.988 INFO:tasks.workunit.client.0.smithi016.stdout:5/812: readlink d3/l11f 0 2024-04-02T04:42:51.988 INFO:tasks.workunit.client.0.smithi016.stdout:5/813: readlink d3/d40/db6/dfc/l120 0 2024-04-02T04:42:51.989 INFO:tasks.workunit.client.1.smithi027.stdout:6/324: rmdir d13/d1b/d24/d28/d46 39 2024-04-02T04:42:51.990 INFO:tasks.workunit.client.1.smithi027.stdout:6/325: write f12 [4387015,23171] 0 2024-04-02T04:42:51.992 INFO:tasks.workunit.client.1.smithi027.stdout:9/315: mkdir d1a/d1d/d3c/d6e 0 2024-04-02T04:42:51.995 INFO:tasks.workunit.client.0.smithi016.stdout:9/918: read d0/d4a/fc2 [3386444,38174] 0 2024-04-02T04:42:51.995 INFO:tasks.workunit.client.0.smithi016.stdout:0/794: dread d0/d16/d24/f2a [0,4194304] 0 2024-04-02T04:42:51.996 INFO:tasks.workunit.client.1.smithi027.stdout:3/246: creat d1/de/df/d13/f4e x:0 0 0 2024-04-02T04:42:51.997 INFO:tasks.workunit.client.0.smithi016.stdout:2/897: dwrite d12/d35/d3b/d42/f65 [0,4194304] 0 2024-04-02T04:42:51.999 INFO:tasks.workunit.client.0.smithi016.stdout:7/827: symlink d0/d46/da2/dc8/l114 0 2024-04-02T04:42:51.999 INFO:tasks.workunit.client.1.smithi027.stdout:6/326: rmdir d13/d1b/d24/d25/d56 39 2024-04-02T04:42:52.003 INFO:tasks.workunit.client.0.smithi016.stdout:6/865: mknod d0/d9/d16/d56/d59/da1/c110 0 2024-04-02T04:42:52.004 INFO:tasks.workunit.client.1.smithi027.stdout:3/247: dwrite d1/de/f40 [0,4194304] 0 2024-04-02T04:42:52.004 INFO:tasks.workunit.client.1.smithi027.stdout:1/300: creat d0/d1e/d2e/f6b x:0 0 0 2024-04-02T04:42:52.010 INFO:tasks.workunit.client.1.smithi027.stdout:8/353: creat d5/d8/d15/d71/f72 x:0 0 0 2024-04-02T04:42:52.011 INFO:tasks.workunit.client.0.smithi016.stdout:4/826: sync 2024-04-02T04:42:52.011 INFO:tasks.workunit.client.0.smithi016.stdout:4/827: chown d5/d1d/f37 6 1 2024-04-02T04:42:52.012 INFO:tasks.workunit.client.1.smithi027.stdout:6/327: rmdir d13/d1b/d48/d58 39 2024-04-02T04:42:52.014 INFO:tasks.workunit.client.1.smithi027.stdout:9/316: symlink d1a/d1d/d29/d37/l6f 0 2024-04-02T04:42:52.016 INFO:tasks.workunit.client.1.smithi027.stdout:9/317: truncate d1a/d1d/f4b 1244635 0 2024-04-02T04:42:52.016 INFO:tasks.workunit.client.1.smithi027.stdout:3/248: symlink d1/de/d3a/l4f 0 2024-04-02T04:42:52.017 INFO:tasks.workunit.client.1.smithi027.stdout:8/354: dread d5/d8/db/dd/d5f/d27/d31/f5e [0,4194304] 0 2024-04-02T04:42:52.018 INFO:tasks.workunit.client.1.smithi027.stdout:1/301: symlink d0/d1e/d2e/d3b/l6c 0 2024-04-02T04:42:52.019 INFO:tasks.workunit.client.1.smithi027.stdout:6/328: dread f7 [0,4194304] 0 2024-04-02T04:42:52.019 INFO:tasks.workunit.client.1.smithi027.stdout:5/438: link d0/d1d/c5a d0/df/d11/d37/ca0 0 2024-04-02T04:42:52.020 INFO:tasks.workunit.client.0.smithi016.stdout:9/919: readlink d0/d4e/d50/d63/d6e/d4b/d58/dd5/lec 0 2024-04-02T04:42:52.024 INFO:tasks.workunit.client.1.smithi027.stdout:1/302: dread d0/de/d52/f67 [0,4194304] 0 2024-04-02T04:42:52.024 INFO:tasks.workunit.client.1.smithi027.stdout:1/303: readlink d0/l26 0 2024-04-02T04:42:52.025 INFO:tasks.workunit.client.0.smithi016.stdout:6/866: creat d0/d9/d16/d56/d59/f111 x:0 0 0 2024-04-02T04:42:52.031 INFO:tasks.workunit.client.0.smithi016.stdout:5/814: rename d3/d40/db6/ccd to d3/df/d93/c121 0 2024-04-02T04:42:52.032 INFO:tasks.workunit.client.1.smithi027.stdout:5/439: creat d0/d1d/d61/d6d/fa1 x:0 0 0 2024-04-02T04:42:52.033 INFO:tasks.workunit.client.0.smithi016.stdout:3/943: write d2/fd [1396210,102145] 0 2024-04-02T04:42:52.034 INFO:tasks.workunit.client.0.smithi016.stdout:8/826: link d1/d6/d36/d3f/d40/d63/c7f d1/d6/d36/d3f/d40/d49/d4a/d108/c10f 0 2024-04-02T04:42:52.034 INFO:tasks.workunit.client.1.smithi027.stdout:3/249: symlink d1/de/l50 0 2024-04-02T04:42:52.035 INFO:tasks.workunit.client.1.smithi027.stdout:3/250: dread - d1/de/df/f4b zero size 2024-04-02T04:42:52.036 INFO:tasks.workunit.client.0.smithi016.stdout:3/944: write d2/d2e/d36/d48/da2/d88/f125 [253278,76518] 0 2024-04-02T04:42:52.036 INFO:tasks.workunit.client.0.smithi016.stdout:1/808: dwrite d14/d19/d1e/d71/ddd/f103 [4194304,4194304] 0 2024-04-02T04:42:52.039 INFO:tasks.workunit.client.0.smithi016.stdout:9/920: mknod d0/d4a/d75/d95/d9b/db4/df9/c11d 0 2024-04-02T04:42:52.040 INFO:tasks.workunit.client.0.smithi016.stdout:7/828: write d0/d46/f30 [5739809,56521] 0 2024-04-02T04:42:52.042 INFO:tasks.workunit.client.1.smithi027.stdout:8/355: mknod d5/d8/db/dd/d5f/d27/d31/d5b/d6c/c73 0 2024-04-02T04:42:52.047 INFO:tasks.workunit.client.0.smithi016.stdout:4/828: mkdir d5/d17/dcd/def/dfd 0 2024-04-02T04:42:52.048 INFO:tasks.workunit.client.1.smithi027.stdout:6/329: mknod d13/d1b/d24/d25/c6b 0 2024-04-02T04:42:52.048 INFO:tasks.workunit.client.1.smithi027.stdout:9/318: dwrite d1a/d1d/d29/d37/d3a/f54 [4194304,4194304] 0 2024-04-02T04:42:52.050 INFO:tasks.workunit.client.1.smithi027.stdout:6/330: write d13/d1b/f2b [47275,64454] 0 2024-04-02T04:42:52.051 INFO:tasks.workunit.client.0.smithi016.stdout:8/827: mkdir d1/d31/d67/d110 0 2024-04-02T04:42:52.051 INFO:tasks.workunit.client.0.smithi016.stdout:8/828: readlink d1/d6/d36/d4f/l8b 0 2024-04-02T04:42:52.054 INFO:tasks.workunit.client.0.smithi016.stdout:0/795: mkdir d0/d16/d23/d3e/db3/d10c/d10d 0 2024-04-02T04:42:52.058 INFO:tasks.workunit.client.1.smithi027.stdout:9/319: dwrite f17 [0,4194304] 0 2024-04-02T04:42:52.059 INFO:tasks.workunit.client.1.smithi027.stdout:1/304: creat d0/d1e/d2e/f6d x:0 0 0 2024-04-02T04:42:52.061 INFO:tasks.workunit.client.0.smithi016.stdout:1/809: creat d14/d19/d1d/d29/d39/d4d/f10b x:0 0 0 2024-04-02T04:42:52.062 INFO:tasks.workunit.client.1.smithi027.stdout:1/305: read d0/fb [1154836,130122] 0 2024-04-02T04:42:52.064 INFO:tasks.workunit.client.0.smithi016.stdout:5/815: dwrite d3/df/d18/d2f/fd6 [0,4194304] 0 2024-04-02T04:42:52.065 INFO:tasks.workunit.client.0.smithi016.stdout:5/816: chown d3/df/d18/fc3 6130 1 2024-04-02T04:42:52.066 INFO:tasks.workunit.client.0.smithi016.stdout:5/817: chown d3/d40/d4f/d6b/d45/da6/d102 98 1 2024-04-02T04:42:52.066 INFO:tasks.workunit.client.0.smithi016.stdout:6/867: mkdir d0/d9/d16/d56/d89/d67/d6f/d112 0 2024-04-02T04:42:52.068 INFO:tasks.workunit.client.0.smithi016.stdout:5/818: write d3/df/d18/d26/d70/ffe [1275628,40701] 0 2024-04-02T04:42:52.074 INFO:tasks.workunit.client.0.smithi016.stdout:4/829: unlink d5/d16/f55 0 2024-04-02T04:42:52.075 INFO:tasks.workunit.client.1.smithi027.stdout:8/356: truncate d5/d8/f2e 2213283 0 2024-04-02T04:42:52.075 INFO:tasks.workunit.client.1.smithi027.stdout:8/357: read d5/d8/f17 [546558,18170] 0 2024-04-02T04:42:52.076 INFO:tasks.workunit.client.1.smithi027.stdout:8/358: readlink d5/d8/db/dd/d5f/d27/l3d 0 2024-04-02T04:42:52.076 INFO:tasks.workunit.client.0.smithi016.stdout:8/829: unlink d1/d6/d36/d3f/d40/d49/lbf 0 2024-04-02T04:42:52.077 INFO:tasks.workunit.client.1.smithi027.stdout:9/320: mkdir d1a/d1d/d29/d51/d70 0 2024-04-02T04:42:52.079 INFO:tasks.workunit.client.1.smithi027.stdout:1/306: unlink d0/d1e/d25/d2b/f57 0 2024-04-02T04:42:52.081 INFO:tasks.workunit.client.0.smithi016.stdout:1/810: chown d14/d19/d50/lc5 12 1 2024-04-02T04:42:52.082 INFO:tasks.workunit.client.1.smithi027.stdout:5/440: getdents d0/df 0 2024-04-02T04:42:52.082 INFO:tasks.workunit.client.0.smithi016.stdout:1/811: dread - d14/d19/d1e/d6d/fe8 zero size 2024-04-02T04:42:52.083 INFO:tasks.workunit.client.0.smithi016.stdout:0/796: dwrite d0/d16/d23/f5a [0,4194304] 0 2024-04-02T04:42:52.083 INFO:tasks.workunit.client.0.smithi016.stdout:0/797: chown d0/d16/f54 266 1 2024-04-02T04:42:52.087 INFO:tasks.workunit.client.1.smithi027.stdout:8/359: creat d5/d8/db/d57/f74 x:0 0 0 2024-04-02T04:42:52.088 INFO:tasks.workunit.client.1.smithi027.stdout:8/360: readlink d5/d8/l3c 0 2024-04-02T04:42:52.089 INFO:tasks.workunit.client.1.smithi027.stdout:8/361: chown d5/d8/d15/d71 283043811 1 2024-04-02T04:42:52.090 INFO:tasks.workunit.client.1.smithi027.stdout:6/331: dwrite d13/d1b/d24/d28/f2a [0,4194304] 0 2024-04-02T04:42:52.090 INFO:tasks.workunit.client.0.smithi016.stdout:2/898: rename c11 to d12/d35/d3b/d42/d67/c133 0 2024-04-02T04:42:52.091 INFO:tasks.workunit.client.1.smithi027.stdout:8/362: chown d5/d8/f17 311 1 2024-04-02T04:42:52.092 INFO:tasks.workunit.client.0.smithi016.stdout:2/899: chown d12/d35/d3f/d2b/de9/d102/d103/f11d 9869510 1 2024-04-02T04:42:52.092 INFO:tasks.workunit.client.1.smithi027.stdout:9/321: truncate d1a/f38 1267690 0 2024-04-02T04:42:52.092 INFO:tasks.workunit.client.0.smithi016.stdout:2/900: fsync d12/d35/d3b/d42/d67/f10b 0 2024-04-02T04:42:52.094 INFO:tasks.workunit.client.1.smithi027.stdout:9/322: write d1a/d1d/d29/d37/d4c/f66 [598570,95417] 0 2024-04-02T04:42:52.096 INFO:tasks.workunit.client.0.smithi016.stdout:5/819: mkdir d3/df/d18/dfd/d122 0 2024-04-02T04:42:52.097 INFO:tasks.workunit.client.1.smithi027.stdout:9/323: read f12 [4244218,70919] 0 2024-04-02T04:42:52.102 INFO:tasks.workunit.client.1.smithi027.stdout:1/307: creat d0/d1e/d2e/d3e/f6e x:0 0 0 2024-04-02T04:42:52.103 INFO:tasks.workunit.client.0.smithi016.stdout:4/830: creat d5/d8/ffe x:0 0 0 2024-04-02T04:42:52.104 INFO:tasks.workunit.client.1.smithi027.stdout:1/308: truncate d0/f5c 1045395 0 2024-04-02T04:42:52.104 INFO:tasks.workunit.client.1.smithi027.stdout:1/309: readlink d0/l27 0 2024-04-02T04:42:52.104 INFO:tasks.workunit.client.1.smithi027.stdout:5/441: chown d0/df/d11/d41/l70 419362792 1 2024-04-02T04:42:52.104 INFO:tasks.workunit.client.0.smithi016.stdout:8/830: creat d1/d70/d71/d74/f111 x:0 0 0 2024-04-02T04:42:52.108 INFO:tasks.workunit.client.0.smithi016.stdout:9/921: creat d0/d4e/d50/d63/d6e/f11e x:0 0 0 2024-04-02T04:42:52.109 INFO:tasks.workunit.client.1.smithi027.stdout:8/363: readlink d5/d8/d5c/l64 0 2024-04-02T04:42:52.111 INFO:tasks.workunit.client.0.smithi016.stdout:7/829: rmdir d0/d46/d16/d49/df9/d106 0 2024-04-02T04:42:52.113 INFO:tasks.workunit.client.1.smithi027.stdout:6/332: dwrite d13/d1b/d48/d58/f6a [0,4194304] 0 2024-04-02T04:42:52.114 INFO:tasks.workunit.client.0.smithi016.stdout:0/798: truncate d0/f32 1430767 0 2024-04-02T04:42:52.114 INFO:tasks.workunit.client.1.smithi027.stdout:6/333: chown d13/d1b/d24/d28 2 1 2024-04-02T04:42:52.117 INFO:tasks.workunit.client.1.smithi027.stdout:1/310: dread d0/f16 [0,4194304] 0 2024-04-02T04:42:52.119 INFO:tasks.workunit.client.1.smithi027.stdout:5/442: creat d0/df/d11/d37/fa2 x:0 0 0 2024-04-02T04:42:52.120 INFO:tasks.workunit.client.1.smithi027.stdout:1/311: chown d0/d1e/d2e/d3e/f42 111981925 1 2024-04-02T04:42:52.121 INFO:tasks.workunit.client.1.smithi027.stdout:5/443: read d0/d1a/d20/f73 [2522755,128726] 0 2024-04-02T04:42:52.122 INFO:tasks.workunit.client.0.smithi016.stdout:5/820: creat d3/df/d18/d26/d48/f123 x:0 0 0 2024-04-02T04:42:52.125 INFO:tasks.workunit.client.0.smithi016.stdout:1/812: symlink d14/d19/d1d/l10c 0 2024-04-02T04:42:52.127 INFO:tasks.workunit.client.0.smithi016.stdout:0/799: mkdir d0/d16/d24/d4e/d8d/d71/df1/d10e 0 2024-04-02T04:42:52.127 INFO:tasks.workunit.client.0.smithi016.stdout:0/800: truncate d0/fc 2179150 0 2024-04-02T04:42:52.128 INFO:tasks.workunit.client.0.smithi016.stdout:6/868: creat d0/d9/d16/d56/d89/d67/f113 x:0 0 0 2024-04-02T04:42:52.128 INFO:tasks.workunit.client.1.smithi027.stdout:6/334: symlink d13/d1b/d48/d58/l6c 0 2024-04-02T04:42:52.129 INFO:tasks.workunit.client.0.smithi016.stdout:7/830: read d0/d46/d16/d49/f3e [3969606,79484] 0 2024-04-02T04:42:52.130 INFO:tasks.workunit.client.0.smithi016.stdout:3/945: rename d2/d15/d70/dcf/d11c/l12f to d2/d15/d1a/d45/df2/l13c 0 2024-04-02T04:42:52.132 INFO:tasks.workunit.client.1.smithi027.stdout:1/312: mknod d0/d1e/d5d/c6f 0 2024-04-02T04:42:52.135 INFO:tasks.workunit.client.1.smithi027.stdout:5/444: rename d0/d1d/d61/l88 to d0/d1d/d61/d7e/la3 0 2024-04-02T04:42:52.144 INFO:tasks.workunit.client.1.smithi027.stdout:0/361: sync 2024-04-02T04:42:52.144 INFO:tasks.workunit.client.1.smithi027.stdout:7/324: sync 2024-04-02T04:42:52.144 INFO:tasks.workunit.client.1.smithi027.stdout:2/332: sync 2024-04-02T04:42:52.144 INFO:tasks.workunit.client.0.smithi016.stdout:6/869: creat d0/d9/d16/d56/d89/d67/d6f/d8a/d91/f114 x:0 0 0 2024-04-02T04:42:52.146 INFO:tasks.workunit.client.1.smithi027.stdout:0/362: write d4/d3d/f47 [2977656,17571] 0 2024-04-02T04:42:52.146 INFO:tasks.workunit.client.1.smithi027.stdout:2/333: chown d1/db/d3b/d17/d29/f4f 3 1 2024-04-02T04:42:52.147 INFO:tasks.workunit.client.1.smithi027.stdout:1/313: dread d0/de/f17 [0,4194304] 0 2024-04-02T04:42:52.148 INFO:tasks.workunit.client.0.smithi016.stdout:9/922: dread d0/d4a/d99/f1f [0,4194304] 0 2024-04-02T04:42:52.182 INFO:tasks.workunit.client.0.smithi016.stdout:5/821: creat d3/d40/d4f/d67/f124 x:0 0 0 2024-04-02T04:42:52.184 INFO:tasks.workunit.client.0.smithi016.stdout:1/813: creat d14/d19/d1d/d29/d39/d46/dcb/d43/d68/dc1/dfa/f10d x:0 0 0 2024-04-02T04:42:52.186 INFO:tasks.workunit.client.0.smithi016.stdout:6/870: symlink d0/d9/d16/d56/d95/l115 0 2024-04-02T04:42:52.187 INFO:tasks.workunit.client.0.smithi016.stdout:6/871: read - d0/d9/d16/d56/d59/f111 zero size 2024-04-02T04:42:52.188 INFO:tasks.workunit.client.0.smithi016.stdout:6/872: readlink d0/d9/d16/l9f 0 2024-04-02T04:42:52.188 INFO:tasks.workunit.client.1.smithi027.stdout:8/364: getdents d5/d8/db/d68 0 2024-04-02T04:42:52.191 INFO:tasks.workunit.client.0.smithi016.stdout:9/923: stat d0/d9/db/dc9/f45 0 2024-04-02T04:42:52.191 INFO:tasks.workunit.client.0.smithi016.stdout:9/924: fdatasync d0/d3/f81 0 2024-04-02T04:42:52.191 INFO:tasks.workunit.client.0.smithi016.stdout:3/946: mknod d2/d2e/d36/d66/d90/d12c/c13d 0 2024-04-02T04:42:52.192 INFO:tasks.workunit.client.0.smithi016.stdout:3/947: readlink d2/d2e/d36/d48/da2/d88/l11a 0 2024-04-02T04:42:52.193 INFO:tasks.workunit.client.1.smithi027.stdout:6/335: mkdir d13/d1b/d24/d25/d56/d6d 0 2024-04-02T04:42:52.194 INFO:tasks.workunit.client.0.smithi016.stdout:5/822: read d3/d40/d4f/d6b/d45/fab [60240,99099] 0 2024-04-02T04:42:52.195 INFO:tasks.workunit.client.0.smithi016.stdout:0/801: creat d0/d16/d2e/d36/f10f x:0 0 0 2024-04-02T04:42:52.195 INFO:tasks.workunit.client.1.smithi027.stdout:2/334: symlink d1/db/d3b/l73 0 2024-04-02T04:42:52.197 INFO:tasks.workunit.client.0.smithi016.stdout:1/814: mkdir d14/d19/d1d/d45/d4f/d86/db4/d10e 0 2024-04-02T04:42:52.198 INFO:tasks.workunit.client.0.smithi016.stdout:7/831: creat d0/d46/f115 x:0 0 0 2024-04-02T04:42:52.199 INFO:tasks.workunit.client.0.smithi016.stdout:7/832: write d0/d46/d37/ff0 [2445942,38629] 0 2024-04-02T04:42:52.200 INFO:tasks.workunit.client.1.smithi027.stdout:0/363: link d4/d38/d66/f3a d4/d21/d28/d4e/d5f/f72 0 2024-04-02T04:42:52.201 INFO:tasks.workunit.client.0.smithi016.stdout:6/873: mknod d0/d9/d16/d46/d5e/c116 0 2024-04-02T04:42:52.201 INFO:tasks.workunit.client.1.smithi027.stdout:1/314: symlink d0/d1e/d25/d3c/d64/l70 0 2024-04-02T04:42:52.204 INFO:tasks.workunit.client.0.smithi016.stdout:8/831: rename d1/d31/fde to d1/d6/d36/d3f/f112 0 2024-04-02T04:42:52.205 INFO:tasks.workunit.client.0.smithi016.stdout:0/802: dread d0/d16/d24/f7b [4194304,4194304] 0 2024-04-02T04:42:52.207 INFO:tasks.workunit.client.0.smithi016.stdout:9/925: symlink d0/d3/d97/dac/l11f 0 2024-04-02T04:42:52.208 INFO:tasks.workunit.client.1.smithi027.stdout:1/315: dwrite d0/f5c [0,4194304] 0 2024-04-02T04:42:52.209 INFO:tasks.workunit.client.0.smithi016.stdout:3/948: fdatasync d2/d15/d1a/f39 0 2024-04-02T04:42:52.210 INFO:tasks.workunit.client.0.smithi016.stdout:3/949: write d2/d15/d1a/f108 [87731,66870] 0 2024-04-02T04:42:52.215 INFO:tasks.workunit.client.1.smithi027.stdout:2/335: creat d1/db/d3b/d17/d2b/f74 x:0 0 0 2024-04-02T04:42:52.217 INFO:tasks.workunit.client.0.smithi016.stdout:4/831: sync 2024-04-02T04:42:52.217 INFO:tasks.workunit.client.0.smithi016.stdout:2/901: sync 2024-04-02T04:42:52.226 INFO:tasks.workunit.client.0.smithi016.stdout:7/833: creat d0/d46/d16/df6/f116 x:0 0 0 2024-04-02T04:42:52.226 INFO:tasks.workunit.client.1.smithi027.stdout:7/325: truncate d2/d7/da/dc/d1f/d33/f55 2099872 0 2024-04-02T04:42:52.228 INFO:tasks.workunit.client.1.smithi027.stdout:1/316: write d0/f69 [5059864,78185] 0 2024-04-02T04:42:52.231 INFO:tasks.workunit.client.1.smithi027.stdout:5/445: link d0/d1a/d20/f59 d0/d68/fa4 0 2024-04-02T04:42:52.232 INFO:tasks.workunit.client.1.smithi027.stdout:5/446: write d0/d1d/d61/d72/f81 [596972,21977] 0 2024-04-02T04:42:52.232 INFO:tasks.workunit.client.0.smithi016.stdout:5/823: write d3/d40/d4f/d6b/d45/da6/d102/dc4/d108/d6d/fc0 [575045,36805] 0 2024-04-02T04:42:52.233 INFO:tasks.workunit.client.1.smithi027.stdout:2/336: mkdir d1/db/d3b/d61/d75 0 2024-04-02T04:42:52.234 INFO:tasks.workunit.client.1.smithi027.stdout:2/337: read - d1/f4d zero size 2024-04-02T04:42:52.236 INFO:tasks.workunit.client.1.smithi027.stdout:8/365: link d5/d8/d15/l1c d5/d8/d15/l75 0 2024-04-02T04:42:52.245 INFO:tasks.workunit.client.0.smithi016.stdout:0/803: write d0/d16/d2e/d36/d9f/fa0 [1719562,60054] 0 2024-04-02T04:42:52.246 INFO:tasks.workunit.client.1.smithi027.stdout:1/317: creat d0/d1e/d5d/f71 x:0 0 0 2024-04-02T04:42:52.247 INFO:tasks.workunit.client.1.smithi027.stdout:0/364: truncate d4/d38/d66/f18 2092904 0 2024-04-02T04:42:52.249 INFO:tasks.workunit.client.0.smithi016.stdout:2/902: rename d12/c15 to d12/d35/d101/d4f/c134 0 2024-04-02T04:42:52.250 INFO:tasks.workunit.client.0.smithi016.stdout:9/926: dwrite d0/d4e/d50/d67/fb6 [0,4194304] 0 2024-04-02T04:42:52.250 INFO:tasks.workunit.client.1.smithi027.stdout:6/336: truncate d13/d1b/d24/d25/d27/f44 2729417 0 2024-04-02T04:42:52.252 INFO:tasks.workunit.client.1.smithi027.stdout:5/447: mkdir d0/df/d29/da5 0 2024-04-02T04:42:52.253 INFO:tasks.workunit.client.0.smithi016.stdout:1/815: symlink d14/d19/d1d/d29/d39/l10f 0 2024-04-02T04:42:52.256 INFO:tasks.workunit.client.0.smithi016.stdout:7/834: symlink d0/d46/d16/d49/df9/l117 0 2024-04-02T04:42:52.262 INFO:tasks.workunit.client.1.smithi027.stdout:1/318: unlink d0/d1e/d25/f29 0 2024-04-02T04:42:52.264 INFO:tasks.workunit.client.0.smithi016.stdout:8/832: mknod d1/d6/d36/d3f/d40/d63/c113 0 2024-04-02T04:42:52.266 INFO:tasks.workunit.client.1.smithi027.stdout:6/337: dread fc [0,4194304] 0 2024-04-02T04:42:52.268 INFO:tasks.workunit.client.0.smithi016.stdout:8/833: dread d1/d70/d71/fb3 [0,4194304] 0 2024-04-02T04:42:52.277 INFO:tasks.workunit.client.0.smithi016.stdout:3/950: creat d2/d15/d10f/f13e x:0 0 0 2024-04-02T04:42:52.278 INFO:tasks.workunit.client.0.smithi016.stdout:4/832: symlink d5/de/df4/lff 0 2024-04-02T04:42:52.280 INFO:tasks.workunit.client.0.smithi016.stdout:0/804: creat d0/d48/f110 x:0 0 0 2024-04-02T04:42:52.283 INFO:tasks.workunit.client.1.smithi027.stdout:1/319: rename d0/l26 to d0/d1e/d25/l72 0 2024-04-02T04:42:52.283 INFO:tasks.workunit.client.1.smithi027.stdout:2/338: dwrite d1/db/d3b/d17/f3f [0,4194304] 0 2024-04-02T04:42:52.284 INFO:tasks.workunit.client.1.smithi027.stdout:8/366: write d5/d8/db/dd/d5f/d27/f50 [2586737,3238] 0 2024-04-02T04:42:52.285 INFO:tasks.workunit.client.0.smithi016.stdout:2/903: dread - d12/d35/d3f/f93 zero size 2024-04-02T04:42:52.287 INFO:tasks.workunit.client.1.smithi027.stdout:6/338: symlink d13/d1b/d48/d58/l6e 0 2024-04-02T04:42:52.289 INFO:tasks.workunit.client.0.smithi016.stdout:9/927: unlink d0/d4a/d75/d95/d9a/ced 0 2024-04-02T04:42:52.290 INFO:tasks.workunit.client.0.smithi016.stdout:2/904: dwrite d12/d35/d101/f114 [0,4194304] 0 2024-04-02T04:42:52.291 INFO:tasks.workunit.client.1.smithi027.stdout:5/448: mkdir d0/df/d29/da5/da6 0 2024-04-02T04:42:52.292 INFO:tasks.workunit.client.0.smithi016.stdout:1/816: mkdir d14/d19/d7a/d110 0 2024-04-02T04:42:52.297 INFO:tasks.workunit.client.0.smithi016.stdout:2/905: dread d12/d35/d101/f114 [0,4194304] 0 2024-04-02T04:42:52.298 INFO:tasks.workunit.client.0.smithi016.stdout:6/874: creat d0/d9/d16/d56/f117 x:0 0 0 2024-04-02T04:42:52.298 INFO:tasks.workunit.client.0.smithi016.stdout:9/928: dwrite d0/d4e/d50/f85 [0,4194304] 0 2024-04-02T04:42:52.299 INFO:tasks.workunit.client.1.smithi027.stdout:1/320: rename d0/d1e/d25/d3c/l43 to d0/d1e/d2e/d3b/d4f/l73 0 2024-04-02T04:42:52.300 INFO:tasks.workunit.client.0.smithi016.stdout:5/824: mknod d3/df/d18/d2f/c125 0 2024-04-02T04:42:52.302 INFO:tasks.workunit.client.1.smithi027.stdout:8/367: unlink d5/d8/d15/c55 0 2024-04-02T04:42:52.304 INFO:tasks.workunit.client.1.smithi027.stdout:8/368: fsync d5/d8/db/dd/d5f/f14 0 2024-04-02T04:42:52.304 INFO:tasks.workunit.client.0.smithi016.stdout:8/834: dread - d1/d70/f88 zero size 2024-04-02T04:42:52.306 INFO:tasks.workunit.client.1.smithi027.stdout:7/326: truncate d2/d7/da/dc/d16/f2c 677708 0 2024-04-02T04:42:52.307 INFO:tasks.workunit.client.0.smithi016.stdout:3/951: mknod d2/d15/d70/dcf/d102/c13f 0 2024-04-02T04:42:52.309 INFO:tasks.workunit.client.0.smithi016.stdout:8/835: dwrite d1/d6/d36/d3f/d40/d49/f2a [0,4194304] 0 2024-04-02T04:42:52.311 INFO:tasks.workunit.client.0.smithi016.stdout:8/836: chown d1/d6/d36/d3f/d40/d49/f2a 85 1 2024-04-02T04:42:52.313 INFO:tasks.workunit.client.0.smithi016.stdout:8/837: write d1/d6/d86/fec [1830613,50227] 0 2024-04-02T04:42:52.315 INFO:tasks.workunit.client.1.smithi027.stdout:4/275: sync 2024-04-02T04:42:52.317 INFO:tasks.workunit.client.0.smithi016.stdout:8/838: chown d1/d6/d36/d3f/d40/d63/d105/f107 2715 1 2024-04-02T04:42:52.317 INFO:tasks.workunit.client.0.smithi016.stdout:3/952: dwrite d2/d15/d70/ff5 [0,4194304] 0 2024-04-02T04:42:52.322 INFO:tasks.workunit.client.0.smithi016.stdout:1/817: chown d14/d19/d1d/d29/d39/f65 14610 1 2024-04-02T04:42:52.327 INFO:tasks.workunit.client.0.smithi016.stdout:2/906: mkdir d12/d9f/d135 0 2024-04-02T04:42:52.329 INFO:tasks.workunit.client.1.smithi027.stdout:8/369: write d5/d8/db/dd/d5f/d27/d31/d5b/f67 [604474,107921] 0 2024-04-02T04:42:52.329 INFO:tasks.workunit.client.1.smithi027.stdout:8/370: stat d5/d8/f33 0 2024-04-02T04:42:52.332 INFO:tasks.workunit.client.1.smithi027.stdout:0/365: chown d4/d38/d66/f2a 61918920 1 2024-04-02T04:42:52.333 INFO:tasks.workunit.client.0.smithi016.stdout:6/875: creat d0/d9/d16/d56/d59/da1/f118 x:0 0 0 2024-04-02T04:42:52.334 INFO:tasks.workunit.client.0.smithi016.stdout:0/805: write d0/d16/d23/d3e/f98 [1072886,98434] 0 2024-04-02T04:42:52.335 INFO:tasks.workunit.client.0.smithi016.stdout:5/825: mknod d3/d40/d4f/d6b/d45/da6/d102/dc4/d108/d6d/c126 0 2024-04-02T04:42:52.337 INFO:tasks.workunit.client.1.smithi027.stdout:7/327: unlink d2/f4e 0 2024-04-02T04:42:52.340 INFO:tasks.workunit.client.1.smithi027.stdout:4/276: mkdir d24/d38/d47/d5c 0 2024-04-02T04:42:52.341 INFO:tasks.workunit.client.1.smithi027.stdout:4/277: write d24/d2d/f31 [623937,47532] 0 2024-04-02T04:42:52.343 INFO:tasks.workunit.client.1.smithi027.stdout:4/278: write d24/f2b [2423947,80106] 0 2024-04-02T04:42:52.343 INFO:tasks.workunit.client.1.smithi027.stdout:6/339: truncate d13/d1b/d24/d28/f2a 3881307 0 2024-04-02T04:42:52.347 INFO:tasks.workunit.client.1.smithi027.stdout:8/371: write d5/d8/db/dd/f10 [5701899,51317] 0 2024-04-02T04:42:52.351 INFO:tasks.workunit.client.0.smithi016.stdout:8/839: symlink d1/d70/d71/d74/l114 0 2024-04-02T04:42:52.352 INFO:tasks.workunit.client.1.smithi027.stdout:8/372: write d5/d8/db/dd/d5f/d27/f3a [374310,39557] 0 2024-04-02T04:42:52.353 INFO:tasks.workunit.client.0.smithi016.stdout:3/953: mkdir d2/d15/d16/d8a/dd1/d6e/d140 0 2024-04-02T04:42:52.355 INFO:tasks.workunit.client.1.smithi027.stdout:4/279: dwrite d24/f3f [0,4194304] 0 2024-04-02T04:42:52.355 INFO:tasks.workunit.client.0.smithi016.stdout:3/954: chown d2/d15/d1a/f1e 781612 1 2024-04-02T04:42:52.355 INFO:tasks.workunit.client.1.smithi027.stdout:6/340: dwrite d13/d1b/d24/d28/f35 [0,4194304] 0 2024-04-02T04:42:52.357 INFO:tasks.workunit.client.0.smithi016.stdout:8/840: dwrite d1/d6/d36/d3f/d40/d45/fb9 [0,4194304] 0 2024-04-02T04:42:52.357 INFO:tasks.workunit.client.1.smithi027.stdout:1/321: write d0/d1e/d25/f31 [5164056,128682] 0 2024-04-02T04:42:52.359 INFO:tasks.workunit.client.1.smithi027.stdout:7/328: creat d2/d1c/f62 x:0 0 0 2024-04-02T04:42:52.361 INFO:tasks.workunit.client.1.smithi027.stdout:3/251: sync 2024-04-02T04:42:52.361 INFO:tasks.workunit.client.1.smithi027.stdout:4/280: chown d24/d38/d44/d41/d49/l56 2545 1 2024-04-02T04:42:52.364 INFO:tasks.workunit.client.1.smithi027.stdout:5/449: symlink d0/d68/d6b/d74/la7 0 2024-04-02T04:42:52.364 INFO:tasks.workunit.client.1.smithi027.stdout:4/281: dread d24/d38/d44/d41/f4b [0,4194304] 0 2024-04-02T04:42:52.370 INFO:tasks.workunit.client.0.smithi016.stdout:7/835: link d0/d21/f10f d0/d46/d52/f118 0 2024-04-02T04:42:52.373 INFO:tasks.workunit.client.1.smithi027.stdout:4/282: dwrite d24/f3a [0,4194304] 0 2024-04-02T04:42:52.374 INFO:tasks.workunit.client.0.smithi016.stdout:2/907: mkdir d12/d35/d3f/d4d/d136 0 2024-04-02T04:42:52.375 INFO:tasks.workunit.client.1.smithi027.stdout:4/283: chown d24/f2b 2946 1 2024-04-02T04:42:52.376 INFO:tasks.workunit.client.1.smithi027.stdout:2/339: getdents d1/db 0 2024-04-02T04:42:52.379 INFO:tasks.workunit.client.0.smithi016.stdout:6/876: fdatasync d0/fc 0 2024-04-02T04:42:52.383 INFO:tasks.workunit.client.0.smithi016.stdout:6/877: stat d0/d9/df/f1b 0 2024-04-02T04:42:52.384 INFO:tasks.workunit.client.0.smithi016.stdout:2/908: dwrite d12/d35/d3f/f61 [0,4194304] 0 2024-04-02T04:42:52.385 INFO:tasks.workunit.client.0.smithi016.stdout:1/818: dwrite d14/d19/d1d/f2c [0,4194304] 0 2024-04-02T04:42:52.386 INFO:tasks.workunit.client.0.smithi016.stdout:3/955: creat d2/d15/d10f/f141 x:0 0 0 2024-04-02T04:42:52.387 INFO:tasks.workunit.client.0.smithi016.stdout:8/841: mknod d1/d6/d36/d3f/d40/d63/d105/c115 0 2024-04-02T04:42:52.387 INFO:tasks.workunit.client.1.smithi027.stdout:8/373: dread d5/d8/db/dd/d5f/d27/f3a [4194304,4194304] 0 2024-04-02T04:42:52.388 INFO:tasks.workunit.client.1.smithi027.stdout:8/374: chown d5/d8/l3c 7 1 2024-04-02T04:42:52.390 INFO:tasks.workunit.client.1.smithi027.stdout:7/329: mknod d2/d7/da/dc/d1f/c63 0 2024-04-02T04:42:52.395 INFO:tasks.workunit.client.1.smithi027.stdout:5/450: write d0/df/d2f/d3b/f64 [828521,15542] 0 2024-04-02T04:42:52.396 INFO:tasks.workunit.client.0.smithi016.stdout:0/806: mkdir d0/d16/d24/d62/d9c/dfb/d111 0 2024-04-02T04:42:52.398 INFO:tasks.workunit.client.1.smithi027.stdout:1/322: dwrite d0/de/f4d [0,4194304] 0 2024-04-02T04:42:52.402 INFO:tasks.workunit.client.0.smithi016.stdout:1/819: dread d14/d19/d20/f24 [0,4194304] 0 2024-04-02T04:42:52.406 INFO:tasks.workunit.client.0.smithi016.stdout:4/833: link d5/d17/d18/d6d/d8e/ca3 d5/d17/c100 0 2024-04-02T04:42:52.406 INFO:tasks.workunit.client.1.smithi027.stdout:4/284: creat d24/d48/f5d x:0 0 0 2024-04-02T04:42:52.407 INFO:tasks.workunit.client.1.smithi027.stdout:4/285: chown ld 24 1 2024-04-02T04:42:52.408 INFO:tasks.workunit.client.1.smithi027.stdout:0/366: creat d4/f73 x:0 0 0 2024-04-02T04:42:52.409 INFO:tasks.workunit.client.0.smithi016.stdout:6/878: rmdir d0/d9/d16/d56/d89/d100 39 2024-04-02T04:42:52.410 INFO:tasks.workunit.client.1.smithi027.stdout:9/324: sync 2024-04-02T04:42:52.413 INFO:tasks.workunit.client.0.smithi016.stdout:3/956: fdatasync d2/d2e/ff1 0 2024-04-02T04:42:52.414 INFO:tasks.workunit.client.0.smithi016.stdout:3/957: readlink d2/d2e/d36/d66/d7a/lc6 0 2024-04-02T04:42:52.414 INFO:tasks.workunit.client.0.smithi016.stdout:3/958: chown d2/d15/d16/d35 175565199 1 2024-04-02T04:42:52.415 INFO:tasks.workunit.client.1.smithi027.stdout:2/340: dread d1/db/d3b/d17/d29/f5a [0,4194304] 0 2024-04-02T04:42:52.415 INFO:tasks.workunit.client.1.smithi027.stdout:6/341: mknod d13/d1b/d24/d28/c6f 0 2024-04-02T04:42:52.417 INFO:tasks.workunit.client.1.smithi027.stdout:9/325: dwrite d1a/d1d/f24 [0,4194304] 0 2024-04-02T04:42:52.421 INFO:tasks.workunit.client.0.smithi016.stdout:7/836: mkdir d0/d46/da2/dc8/dda/de3/d119 0 2024-04-02T04:42:52.424 INFO:tasks.workunit.client.0.smithi016.stdout:8/842: truncate d1/d6/d36/d3f/d77/f8a 421997 0 2024-04-02T04:42:52.427 INFO:tasks.workunit.client.0.smithi016.stdout:9/929: getdents d0/d4e 0 2024-04-02T04:42:52.428 INFO:tasks.workunit.client.1.smithi027.stdout:7/330: mkdir d2/df/d64 0 2024-04-02T04:42:52.429 INFO:tasks.workunit.client.1.smithi027.stdout:8/375: dread d5/d8/db/dd/d5f/d27/d31/f32 [0,4194304] 0 2024-04-02T04:42:52.434 INFO:tasks.workunit.client.0.smithi016.stdout:5/826: truncate d3/df/d18/d2f/fd6 1371098 0 2024-04-02T04:42:52.435 INFO:tasks.workunit.client.1.smithi027.stdout:4/286: fdatasync d24/f33 0 2024-04-02T04:42:52.438 INFO:tasks.workunit.client.0.smithi016.stdout:2/909: mkdir d12/d35/d3b/d42/d59/d91/d137 0 2024-04-02T04:42:52.441 INFO:tasks.workunit.client.0.smithi016.stdout:6/879: dwrite d0/d9/d16/d56/d89/d67/d6f/d8a/fc8 [0,4194304] 0 2024-04-02T04:42:52.441 INFO:tasks.workunit.client.0.smithi016.stdout:3/959: truncate d2/f26 1124683 0 2024-04-02T04:42:52.441 INFO:tasks.workunit.client.0.smithi016.stdout:3/960: readlink d2/l10a 0 2024-04-02T04:42:52.443 INFO:tasks.workunit.client.1.smithi027.stdout:2/341: mkdir d1/db/d3b/d17/d68/d76 0 2024-04-02T04:42:52.447 INFO:tasks.workunit.client.1.smithi027.stdout:9/326: unlink d1a/d1d/d29/d37/d20/l2d 0 2024-04-02T04:42:52.448 INFO:tasks.workunit.client.1.smithi027.stdout:3/252: dwrite d1/f45 [0,4194304] 0 2024-04-02T04:42:52.450 INFO:tasks.workunit.client.1.smithi027.stdout:3/253: truncate d1/de/df/f17 5023405 0 2024-04-02T04:42:52.455 INFO:tasks.workunit.client.1.smithi027.stdout:7/331: rmdir d2/d7/da 39 2024-04-02T04:42:52.461 INFO:tasks.workunit.client.0.smithi016.stdout:9/930: truncate d0/d4a/f69 2123662 0 2024-04-02T04:42:52.462 INFO:tasks.workunit.client.1.smithi027.stdout:8/376: fsync d5/d8/db/dd/d5f/d27/d31/f32 0 2024-04-02T04:42:52.462 INFO:tasks.workunit.client.0.smithi016.stdout:1/820: dwrite f2 [4194304,4194304] 0 2024-04-02T04:42:52.464 INFO:tasks.workunit.client.1.smithi027.stdout:8/377: stat d5/d8/db/df/c34 0 2024-04-02T04:42:52.465 INFO:tasks.workunit.client.1.smithi027.stdout:5/451: creat d0/df/d11/d13/d9a/d9c/fa8 x:0 0 0 2024-04-02T04:42:52.465 INFO:tasks.workunit.client.1.smithi027.stdout:1/323: dwrite d0/f16 [0,4194304] 0 2024-04-02T04:42:52.470 INFO:tasks.workunit.client.0.smithi016.stdout:8/843: write d1/d70/d71/d74/fbd [544547,101281] 0 2024-04-02T04:42:52.470 INFO:tasks.workunit.client.0.smithi016.stdout:5/827: creat d3/df/d18/d26/d48/f127 x:0 0 0 2024-04-02T04:42:52.475 INFO:tasks.workunit.client.1.smithi027.stdout:1/324: dwrite d0/de/f4d [0,4194304] 0 2024-04-02T04:42:52.479 INFO:tasks.workunit.client.1.smithi027.stdout:2/342: creat d1/db/d3b/d17/d2b/f77 x:0 0 0 2024-04-02T04:42:52.482 INFO:tasks.workunit.client.0.smithi016.stdout:2/910: rename d12/d35/l125 to d12/d35/d3b/d42/d59/d91/d137/l138 0 2024-04-02T04:42:52.484 INFO:tasks.workunit.client.1.smithi027.stdout:9/327: mkdir d1a/d1d/d3c/d71 0 2024-04-02T04:42:52.485 INFO:tasks.workunit.client.1.smithi027.stdout:9/328: dread - d1a/d1d/d3c/f5e zero size 2024-04-02T04:42:52.487 INFO:tasks.workunit.client.0.smithi016.stdout:6/880: unlink d0/d6e/fcc 0 2024-04-02T04:42:52.488 INFO:tasks.workunit.client.1.smithi027.stdout:3/254: creat d1/de/df/d13/f51 x:0 0 0 2024-04-02T04:42:52.489 INFO:tasks.workunit.client.0.smithi016.stdout:3/961: stat d2/d15/d16/d8a/lb1 0 2024-04-02T04:42:52.491 INFO:tasks.workunit.client.0.smithi016.stdout:3/962: stat d2/d2e/d36/d48/c6b 0 2024-04-02T04:42:52.491 INFO:tasks.workunit.client.1.smithi027.stdout:7/332: fdatasync d2/d7/da/dc/d1f/d33/d36/f45 0 2024-04-02T04:42:52.492 INFO:tasks.workunit.client.0.smithi016.stdout:4/834: dwrite d5/d1d/d5f/d95/fdf [0,4194304] 0 2024-04-02T04:42:52.492 INFO:tasks.workunit.client.1.smithi027.stdout:7/333: write d2/d1c/f4b [2029162,86163] 0 2024-04-02T04:42:52.492 INFO:tasks.workunit.client.0.smithi016.stdout:3/963: fdatasync d2/d15/d16/d8a/dd1/d6e/d6f/d78/f134 0 2024-04-02T04:42:52.495 INFO:tasks.workunit.client.1.smithi027.stdout:8/378: creat d5/d8/db/f76 x:0 0 0 2024-04-02T04:42:52.497 INFO:tasks.workunit.client.0.smithi016.stdout:4/835: dwrite d5/d17/d18/d1f/f59 [0,4194304] 0 2024-04-02T04:42:52.499 INFO:tasks.workunit.client.0.smithi016.stdout:4/836: chown d5/d17/d18/d6d/d8e/d9c/dcc/d6f/dcf/ff6 127882985 1 2024-04-02T04:42:52.499 INFO:tasks.workunit.client.0.smithi016.stdout:4/837: stat d5/d17/dcd/fce 0 2024-04-02T04:42:52.500 INFO:tasks.workunit.client.1.smithi027.stdout:0/367: mkdir d4/d38/d74 0 2024-04-02T04:42:52.500 INFO:tasks.workunit.client.1.smithi027.stdout:5/452: dread d0/df/d2f/d3b/f3d [0,4194304] 0 2024-04-02T04:42:52.502 INFO:tasks.workunit.client.1.smithi027.stdout:1/325: write d0/f7 [3649712,37642] 0 2024-04-02T04:42:52.504 INFO:tasks.workunit.client.1.smithi027.stdout:6/342: creat d13/d1b/d24/f70 x:0 0 0 2024-04-02T04:42:52.504 INFO:tasks.workunit.client.1.smithi027.stdout:1/326: chown d0/d1e/d2e/l66 31788 1 2024-04-02T04:42:52.506 INFO:tasks.workunit.client.1.smithi027.stdout:2/343: mkdir d1/db/d3b/d17/d68/d78 0 2024-04-02T04:42:52.518 INFO:tasks.workunit.client.1.smithi027.stdout:8/379: symlink d5/d8/db/d68/l77 0 2024-04-02T04:42:52.530 INFO:tasks.workunit.client.1.smithi027.stdout:2/344: mknod d1/db/d3b/d17/d29/c79 0 2024-04-02T04:42:52.530 INFO:tasks.workunit.client.1.smithi027.stdout:3/255: dread d1/de/d3a/f3d [0,4194304] 0 2024-04-02T04:42:52.530 INFO:tasks.workunit.client.1.smithi027.stdout:1/327: dwrite d0/f5c [0,4194304] 0 2024-04-02T04:42:52.534 INFO:tasks.workunit.client.1.smithi027.stdout:7/334: mkdir d2/d7/da/d65 0 2024-04-02T04:42:52.536 INFO:tasks.workunit.client.1.smithi027.stdout:8/380: mknod d5/d8/db/dd/d5f/d27/d31/d5b/c78 0 2024-04-02T04:42:52.538 INFO:tasks.workunit.client.1.smithi027.stdout:0/368: chown d4/d16/d1b/c46 6825 1 2024-04-02T04:42:52.538 INFO:tasks.workunit.client.1.smithi027.stdout:0/369: readlink d4/d38/d66/d4a/l6d 0 2024-04-02T04:42:52.541 INFO:tasks.workunit.client.1.smithi027.stdout:6/343: rmdir d13/d1b/d24/d28/d46 39 2024-04-02T04:42:52.544 INFO:tasks.workunit.client.1.smithi027.stdout:9/329: creat d1a/d1d/f72 x:0 0 0 2024-04-02T04:42:52.546 INFO:tasks.workunit.client.1.smithi027.stdout:9/330: stat d1a/d1d/d29/d37/d4c/c6a 0 2024-04-02T04:42:52.548 INFO:tasks.workunit.client.1.smithi027.stdout:1/328: mkdir d0/de/d52/d74 0 2024-04-02T04:42:52.548 INFO:tasks.workunit.client.1.smithi027.stdout:6/344: dwrite d13/f17 [0,4194304] 0 2024-04-02T04:42:52.552 INFO:tasks.workunit.client.1.smithi027.stdout:7/335: rename d2/d7/da/f5b to d2/d7/da/dc/d16/d3f/d50/f66 0 2024-04-02T04:42:52.552 INFO:tasks.workunit.client.1.smithi027.stdout:6/345: dread d13/f16 [0,4194304] 0 2024-04-02T04:42:52.553 INFO:tasks.workunit.client.1.smithi027.stdout:7/336: read - d2/f4c zero size 2024-04-02T04:42:52.555 INFO:tasks.workunit.client.1.smithi027.stdout:8/381: creat d5/d8/db/df/f79 x:0 0 0 2024-04-02T04:42:52.557 INFO:tasks.workunit.client.1.smithi027.stdout:8/382: truncate d5/d8/db/dd/f3b 187644 0 2024-04-02T04:42:52.559 INFO:tasks.workunit.client.1.smithi027.stdout:0/370: readlink d4/d21/d28/l59 0 2024-04-02T04:42:52.562 INFO:tasks.workunit.client.1.smithi027.stdout:2/345: symlink d1/db/d3b/d17/d68/d78/l7a 0 2024-04-02T04:42:52.564 INFO:tasks.workunit.client.1.smithi027.stdout:2/346: truncate d1/db/d3b/d17/d12/f62 770077 0 2024-04-02T04:42:52.567 INFO:tasks.workunit.client.1.smithi027.stdout:1/329: creat d0/d1e/d2e/d3b/d4f/f75 x:0 0 0 2024-04-02T04:42:52.568 INFO:tasks.workunit.client.1.smithi027.stdout:1/330: dread - d0/d1e/d2e/d3b/d4f/f62 zero size 2024-04-02T04:42:52.573 INFO:tasks.workunit.client.1.smithi027.stdout:7/337: creat d2/d7/da/d35/f67 x:0 0 0 2024-04-02T04:42:52.583 INFO:tasks.workunit.client.1.smithi027.stdout:3/256: creat d1/f52 x:0 0 0 2024-04-02T04:42:52.583 INFO:tasks.workunit.client.1.smithi027.stdout:3/257: dread - d1/f52 zero size 2024-04-02T04:42:52.584 INFO:tasks.workunit.client.1.smithi027.stdout:9/331: mkdir d1a/d1d/d29/d50/d73 0 2024-04-02T04:42:52.586 INFO:tasks.workunit.client.1.smithi027.stdout:1/331: mkdir d0/de/d52/d76 0 2024-04-02T04:42:52.587 INFO:tasks.workunit.client.1.smithi027.stdout:1/332: write d0/d1e/d2e/d3b/d4f/f75 [79384,5350] 0 2024-04-02T04:42:52.587 INFO:tasks.workunit.client.1.smithi027.stdout:6/346: symlink d13/d1b/d24/d25/l71 0 2024-04-02T04:42:52.589 INFO:tasks.workunit.client.1.smithi027.stdout:7/338: symlink d2/d7/da/dc/d1f/d33/l68 0 2024-04-02T04:42:52.590 INFO:tasks.workunit.client.1.smithi027.stdout:7/339: fsync d2/d1c/f3b 0 2024-04-02T04:42:52.591 INFO:tasks.workunit.client.1.smithi027.stdout:7/340: fsync d2/d7/da/f1e 0 2024-04-02T04:42:52.598 INFO:tasks.workunit.client.0.smithi016.stdout:0/807: link d0/d16/d2e/d53/lc3 d0/d16/d24/d4e/d8d/d71/df1/d10e/l112 0 2024-04-02T04:42:52.601 INFO:tasks.workunit.client.0.smithi016.stdout:1/821: symlink d14/d19/d1d/d29/d39/d46/l111 0 2024-04-02T04:42:52.602 INFO:tasks.workunit.client.0.smithi016.stdout:1/822: read f2 [794797,27614] 0 2024-04-02T04:42:52.604 INFO:tasks.workunit.client.1.smithi027.stdout:2/347: dwrite d1/db/d3b/d17/d12/f41 [0,4194304] 0 2024-04-02T04:42:52.605 INFO:tasks.workunit.client.0.smithi016.stdout:8/844: fdatasync d1/d6/f25 0 2024-04-02T04:42:52.606 INFO:tasks.workunit.client.1.smithi027.stdout:2/348: chown d1/db/d3b/d17/f14 8 1 2024-04-02T04:42:52.606 INFO:tasks.workunit.client.0.smithi016.stdout:8/845: stat d1/d6/d36/d3f/d40/d63/d105/c115 0 2024-04-02T04:42:52.607 INFO:tasks.workunit.client.0.smithi016.stdout:8/846: write d1/d70/d71/d74/fa6 [64419,69094] 0 2024-04-02T04:42:52.610 INFO:tasks.workunit.client.1.smithi027.stdout:0/371: write d4/f11 [1699879,80821] 0 2024-04-02T04:42:52.618 INFO:tasks.workunit.client.1.smithi027.stdout:0/372: read d4/f11 [1969633,88855] 0 2024-04-02T04:42:52.619 INFO:tasks.workunit.client.0.smithi016.stdout:2/911: creat d12/d35/d3b/d42/d59/d91/f139 x:0 0 0 2024-04-02T04:42:52.621 INFO:tasks.workunit.client.0.smithi016.stdout:5/828: dread d3/df/d18/d26/f8c [0,4194304] 0 2024-04-02T04:42:52.629 INFO:tasks.workunit.client.0.smithi016.stdout:7/837: link d0/d21/f3c d0/d46/d37/dbe/dae/df8/dec/f11a 0 2024-04-02T04:42:52.629 INFO:tasks.workunit.client.0.smithi016.stdout:7/838: stat d0/d46/d37/dbe/dae/lab 0 2024-04-02T04:42:52.629 INFO:tasks.workunit.client.1.smithi027.stdout:0/373: dwrite d4/d16/d1b/f65 [0,4194304] 0 2024-04-02T04:42:52.629 INFO:tasks.workunit.client.1.smithi027.stdout:9/332: dread d1a/f38 [0,4194304] 0 2024-04-02T04:42:52.629 INFO:tasks.workunit.client.1.smithi027.stdout:8/383: creat d5/f7a x:0 0 0 2024-04-02T04:42:52.631 INFO:tasks.workunit.client.1.smithi027.stdout:7/341: creat d2/d7/da/dc/d16/d3f/d50/f69 x:0 0 0 2024-04-02T04:42:52.631 INFO:tasks.workunit.client.0.smithi016.stdout:0/808: mknod d0/d4f/d76/c113 0 2024-04-02T04:42:52.635 INFO:tasks.workunit.client.1.smithi027.stdout:0/374: dwrite f1 [0,4194304] 0 2024-04-02T04:42:52.639 INFO:tasks.workunit.client.1.smithi027.stdout:2/349: symlink d1/db/d3b/d17/d2b/d2f/l7b 0 2024-04-02T04:42:52.641 INFO:tasks.workunit.client.1.smithi027.stdout:2/350: read d1/db/d3b/d17/d29/f5a [1192940,50796] 0 2024-04-02T04:42:52.643 INFO:tasks.workunit.client.1.smithi027.stdout:3/258: creat d1/de/df/d13/f53 x:0 0 0 2024-04-02T04:42:52.644 INFO:tasks.workunit.client.1.smithi027.stdout:0/375: dwrite d4/d21/d28/d4e/f62 [0,4194304] 0 2024-04-02T04:42:52.648 INFO:tasks.workunit.client.0.smithi016.stdout:2/912: creat d12/d35/d101/d4f/dfa/f13a x:0 0 0 2024-04-02T04:42:52.649 INFO:tasks.workunit.client.0.smithi016.stdout:2/913: chown d12/d26/f29 173 1 2024-04-02T04:42:52.652 INFO:tasks.workunit.client.0.smithi016.stdout:5/829: creat d3/df/dd5/f128 x:0 0 0 2024-04-02T04:42:52.652 INFO:tasks.workunit.client.0.smithi016.stdout:1/823: dread d14/d19/d1e/f23 [0,4194304] 0 2024-04-02T04:42:52.653 INFO:tasks.workunit.client.0.smithi016.stdout:5/830: dread - d3/df/d18/ff2 zero size 2024-04-02T04:42:52.654 INFO:tasks.workunit.client.1.smithi027.stdout:6/347: dread d13/d1b/d24/d25/d42/f60 [0,4194304] 0 2024-04-02T04:42:52.654 INFO:tasks.workunit.client.0.smithi016.stdout:1/824: readlink d14/d19/d1d/d29/d39/d46/dcb/l7e 0 2024-04-02T04:42:52.656 INFO:tasks.workunit.client.0.smithi016.stdout:1/825: write d14/d19/d1e/d6d/fe8 [561665,47470] 0 2024-04-02T04:42:52.657 INFO:tasks.workunit.client.0.smithi016.stdout:0/809: unlink d0/d16/d2e/d53/lc3 0 2024-04-02T04:42:52.659 INFO:tasks.workunit.client.1.smithi027.stdout:7/342: read d2/d7/da/dc/d16/d3a/d53/f5a [438893,71428] 0 2024-04-02T04:42:52.659 INFO:tasks.workunit.client.1.smithi027.stdout:1/333: write d0/f1d [3329816,43886] 0 2024-04-02T04:42:52.660 INFO:tasks.workunit.client.0.smithi016.stdout:9/931: creat d0/f120 x:0 0 0 2024-04-02T04:42:52.660 INFO:tasks.workunit.client.0.smithi016.stdout:3/964: truncate d2/f26 308741 0 2024-04-02T04:42:52.661 INFO:tasks.workunit.client.0.smithi016.stdout:9/932: stat d0/d4a/d75/c103 0 2024-04-02T04:42:52.661 INFO:tasks.workunit.client.0.smithi016.stdout:9/933: readlink d0/d3/d97/dac/l110 0 2024-04-02T04:42:52.662 INFO:tasks.workunit.client.1.smithi027.stdout:2/351: readlink d1/db/d3b/d17/l21 0 2024-04-02T04:42:52.664 INFO:tasks.workunit.client.1.smithi027.stdout:0/376: rename f1 to d4/d21/d28/f75 0 2024-04-02T04:42:52.666 INFO:tasks.workunit.client.0.smithi016.stdout:2/914: mkdir d12/d35/d101/d4f/dfa/d13b 0 2024-04-02T04:42:52.679 INFO:tasks.workunit.client.0.smithi016.stdout:0/810: symlink d0/d16/d24/d62/d9c/d8c/l114 0 2024-04-02T04:42:52.679 INFO:tasks.workunit.client.1.smithi027.stdout:1/334: dwrite d0/fd [0,4194304] 0 2024-04-02T04:42:52.681 INFO:tasks.workunit.client.1.smithi027.stdout:8/384: truncate d5/d8/d48/f5d 2782847 0 2024-04-02T04:42:52.685 INFO:tasks.workunit.client.0.smithi016.stdout:6/881: getdents d0/d9/d16/d46/dbe 0 2024-04-02T04:42:52.687 INFO:tasks.workunit.client.1.smithi027.stdout:6/348: dwrite d13/d1b/d24/d28/f2a [0,4194304] 0 2024-04-02T04:42:52.687 INFO:tasks.workunit.client.1.smithi027.stdout:2/352: rename d1/db/d3b/d61/f66 to d1/db/d3b/d17/d12/f7c 0 2024-04-02T04:42:52.689 INFO:tasks.workunit.client.1.smithi027.stdout:2/353: write d1/db/d3b/d17/d2b/f77 [523439,59302] 0 2024-04-02T04:42:52.690 INFO:tasks.workunit.client.0.smithi016.stdout:5/831: mkdir d3/d129 0 2024-04-02T04:42:52.691 INFO:tasks.workunit.client.0.smithi016.stdout:5/832: chown d3/df/dd5/ce5 2470 1 2024-04-02T04:42:52.695 INFO:tasks.workunit.client.0.smithi016.stdout:4/838: getdents d5/d8 0 2024-04-02T04:42:52.696 INFO:tasks.workunit.client.1.smithi027.stdout:3/259: creat d1/de/f54 x:0 0 0 2024-04-02T04:42:52.697 INFO:tasks.workunit.client.1.smithi027.stdout:1/335: readlink d0/d1e/d25/l72 0 2024-04-02T04:42:52.698 INFO:tasks.workunit.client.1.smithi027.stdout:0/377: mkdir d4/d76 0 2024-04-02T04:42:52.700 INFO:tasks.workunit.client.1.smithi027.stdout:3/260: dread d1/de/d3a/f3d [0,4194304] 0 2024-04-02T04:42:52.701 INFO:tasks.workunit.client.1.smithi027.stdout:8/385: rename d5/d8/db/dd/d5f/f24 to d5/d8/db/dd/d5f/d27/d31/d5b/d6c/f7b 0 2024-04-02T04:42:52.702 INFO:tasks.workunit.client.0.smithi016.stdout:9/934: fdatasync d0/d9/d4d/d78/da6/dc1/f104 0 2024-04-02T04:42:52.703 INFO:tasks.workunit.client.1.smithi027.stdout:8/386: write d5/d8/db/dd/f3b [58438,113255] 0 2024-04-02T04:42:52.706 INFO:tasks.workunit.client.0.smithi016.stdout:8/847: getdents d1/d6/d36/d4f/dc9 0 2024-04-02T04:42:52.708 INFO:tasks.workunit.client.0.smithi016.stdout:8/848: read d1/d6/d36/d3f/d40/d63/d105/f107 [120835,70840] 0 2024-04-02T04:42:52.709 INFO:tasks.workunit.client.0.smithi016.stdout:8/849: write d1/ff [3690136,96116] 0 2024-04-02T04:42:52.709 INFO:tasks.workunit.client.1.smithi027.stdout:7/343: rmdir d2/d7/da/d65 0 2024-04-02T04:42:52.716 INFO:tasks.workunit.client.0.smithi016.stdout:7/839: getdents d0/d46/d37/dbe/dae/df8 0 2024-04-02T04:42:52.717 INFO:tasks.workunit.client.0.smithi016.stdout:1/826: truncate d14/d19/d1e/f2b 2273742 0 2024-04-02T04:42:52.718 INFO:tasks.workunit.client.0.smithi016.stdout:0/811: write d0/d16/d2e/d37/d66/fa3 [3940623,40982] 0 2024-04-02T04:42:52.719 INFO:tasks.workunit.client.0.smithi016.stdout:0/812: dread - d0/fc8 zero size 2024-04-02T04:42:52.720 INFO:tasks.workunit.client.1.smithi027.stdout:0/378: dwrite d4/d38/d66/d2d/d4c/f4d [0,4194304] 0 2024-04-02T04:42:52.720 INFO:tasks.workunit.client.0.smithi016.stdout:0/813: read - d0/d16/d23/d3e/fc6 zero size 2024-04-02T04:42:52.722 INFO:tasks.workunit.client.1.smithi027.stdout:6/349: write d13/d1b/f1f [3927537,57966] 0 2024-04-02T04:42:52.724 INFO:tasks.workunit.client.1.smithi027.stdout:6/350: truncate d13/d1b/d24/f70 37983 0 2024-04-02T04:42:52.725 INFO:tasks.workunit.client.0.smithi016.stdout:6/882: write d0/d9/d16/d56/d89/d67/d6f/d7e/fca [3909925,27018] 0 2024-04-02T04:42:52.726 INFO:tasks.workunit.client.1.smithi027.stdout:2/354: symlink d1/db/d3b/d17/d68/d76/l7d 0 2024-04-02T04:42:52.728 INFO:tasks.workunit.client.1.smithi027.stdout:2/355: truncate d1/db/d3b/d17/d12/f62 1522937 0 2024-04-02T04:42:52.730 INFO:tasks.workunit.client.0.smithi016.stdout:8/850: mkdir d1/d6/d36/d4f/d5d/d116 0 2024-04-02T04:42:52.732 INFO:tasks.workunit.client.1.smithi027.stdout:3/261: mkdir d1/de/d55 0 2024-04-02T04:42:52.734 INFO:tasks.workunit.client.0.smithi016.stdout:7/840: mkdir d0/d46/d16/df6/d11b 0 2024-04-02T04:42:52.736 INFO:tasks.workunit.client.1.smithi027.stdout:2/356: dwrite d1/db/d3b/d17/f5d [0,4194304] 0 2024-04-02T04:42:52.741 INFO:tasks.workunit.client.1.smithi027.stdout:6/351: unlink d13/d1b/d24/d25/d56/l5c 0 2024-04-02T04:42:52.744 INFO:tasks.workunit.client.1.smithi027.stdout:7/344: link d2/d1c/f3b d2/d7/da/dc/d16/d3f/d44/f6a 0 2024-04-02T04:42:52.746 INFO:tasks.workunit.client.0.smithi016.stdout:0/814: rmdir d0 39 2024-04-02T04:42:52.748 INFO:tasks.workunit.client.1.smithi027.stdout:0/379: symlink d4/d69/l77 0 2024-04-02T04:42:52.748 INFO:tasks.workunit.client.1.smithi027.stdout:1/336: dwrite d0/de/ff [0,4194304] 0 2024-04-02T04:42:52.752 INFO:tasks.workunit.client.0.smithi016.stdout:9/935: mknod d0/d4e/d50/d63/d6e/d4b/db2/dba/d119/c121 0 2024-04-02T04:42:52.754 INFO:tasks.workunit.client.1.smithi027.stdout:2/357: creat d1/db/d3b/d17/d12/f7e x:0 0 0 2024-04-02T04:42:52.754 INFO:tasks.workunit.client.0.smithi016.stdout:4/839: truncate d5/d1d/d6c/f72 3013473 0 2024-04-02T04:42:52.755 INFO:tasks.workunit.client.1.smithi027.stdout:2/358: write d1/db/d3b/d17/f6a [667239,8446] 0 2024-04-02T04:42:52.760 INFO:tasks.workunit.client.0.smithi016.stdout:2/915: getdents d12/d35/d101/d76 0 2024-04-02T04:42:52.761 INFO:tasks.workunit.client.0.smithi016.stdout:4/840: dwrite d5/d1d/d5f/d95/fdf [0,4194304] 0 2024-04-02T04:42:52.761 INFO:tasks.workunit.client.0.smithi016.stdout:7/841: mknod d0/d46/dbf/c11c 0 2024-04-02T04:42:52.762 INFO:tasks.workunit.client.1.smithi027.stdout:8/387: link d5/d8/db/dd/d5f/d27/d31/l45 d5/d8/db/df/l7c 0 2024-04-02T04:42:52.764 INFO:tasks.workunit.client.1.smithi027.stdout:7/345: dread d2/d1c/f4b [0,4194304] 0 2024-04-02T04:42:52.764 INFO:tasks.workunit.client.0.smithi016.stdout:1/827: mknod d14/d19/d1e/d71/dcf/d102/c112 0 2024-04-02T04:42:52.765 INFO:tasks.workunit.client.0.smithi016.stdout:1/828: chown d14/d19/d1d/d29/f6a 16667986 1 2024-04-02T04:42:52.765 INFO:tasks.workunit.client.1.smithi027.stdout:2/359: creat d1/db/d3b/d17/d6d/f7f x:0 0 0 2024-04-02T04:42:52.766 INFO:tasks.workunit.client.0.smithi016.stdout:1/829: chown d14/d19/d1d/d29/d39/d46/dcb/d43/f5f 31 1 2024-04-02T04:42:52.769 INFO:tasks.workunit.client.0.smithi016.stdout:9/936: symlink d0/d4e/l122 0 2024-04-02T04:42:52.770 INFO:tasks.workunit.client.0.smithi016.stdout:9/937: chown d0/d3/l4f 198294700 1 2024-04-02T04:42:52.771 INFO:tasks.workunit.client.0.smithi016.stdout:2/916: creat d12/d35/d101/d4f/f13c x:0 0 0 2024-04-02T04:42:52.771 INFO:tasks.workunit.client.0.smithi016.stdout:4/841: creat d5/de/f101 x:0 0 0 2024-04-02T04:42:52.772 INFO:tasks.workunit.client.0.smithi016.stdout:5/833: getdents d3/d40/d4f/d6b/d63 0 2024-04-02T04:42:52.774 INFO:tasks.workunit.client.0.smithi016.stdout:1/830: truncate d14/d19/d1e/d71/f84 438741 0 2024-04-02T04:42:52.774 INFO:tasks.workunit.client.0.smithi016.stdout:9/938: creat d0/d9/d4d/da8/f123 x:0 0 0 2024-04-02T04:42:52.775 INFO:tasks.workunit.client.0.smithi016.stdout:1/831: read - d14/d19/d50/fc8 zero size 2024-04-02T04:42:52.775 INFO:tasks.workunit.client.0.smithi016.stdout:4/842: creat d5/f102 x:0 0 0 2024-04-02T04:42:52.776 INFO:tasks.workunit.client.0.smithi016.stdout:2/917: mknod d12/d35/d3b/d42/d59/d91/c13d 0 2024-04-02T04:42:52.776 INFO:tasks.workunit.client.0.smithi016.stdout:0/815: mknod d0/d16/d2e/d53/da4/c115 0 2024-04-02T04:42:52.777 INFO:tasks.workunit.client.0.smithi016.stdout:4/843: mknod d5/d16/c103 0 2024-04-02T04:42:52.778 INFO:tasks.workunit.client.0.smithi016.stdout:1/832: unlink d14/d19/d1e/d5e/cdc 0 2024-04-02T04:42:52.778 INFO:tasks.workunit.client.0.smithi016.stdout:7/842: dread d0/d46/d16/fc5 [0,4194304] 0 2024-04-02T04:42:52.779 INFO:tasks.workunit.client.0.smithi016.stdout:9/939: rename d0/d9/d4d/d78/da6/dc1/lcc to d0/d4a/d75/d113/l124 0 2024-04-02T04:42:52.780 INFO:tasks.workunit.client.0.smithi016.stdout:1/833: chown d14/d19/d1e/d71/ddd 39367205 1 2024-04-02T04:42:52.780 INFO:tasks.workunit.client.0.smithi016.stdout:5/834: creat d3/df/d18/dfd/d122/f12a x:0 0 0 2024-04-02T04:42:52.781 INFO:tasks.workunit.client.0.smithi016.stdout:2/918: mkdir d12/d35/d3b/d42/d59/d91/d10a/d13e 0 2024-04-02T04:42:52.781 INFO:tasks.workunit.client.0.smithi016.stdout:4/844: dread d5/d1d/d1e/da5/ff3 [0,4194304] 0 2024-04-02T04:42:52.781 INFO:tasks.workunit.client.0.smithi016.stdout:9/940: fdatasync d0/d3/f40 0 2024-04-02T04:42:52.783 INFO:tasks.workunit.client.0.smithi016.stdout:1/834: fsync d14/d19/d20/f24 0 2024-04-02T04:42:52.784 INFO:tasks.workunit.client.0.smithi016.stdout:7/843: rename d0/d46/d37/dbe/de1/cfc to d0/d21/d104/c11d 0 2024-04-02T04:42:52.785 INFO:tasks.workunit.client.0.smithi016.stdout:0/816: dread d0/d16/d24/d4e/d8d/d71/f72 [4194304,4194304] 0 2024-04-02T04:42:52.786 INFO:tasks.workunit.client.0.smithi016.stdout:2/919: mknod d12/d35/d3f/d2b/da3/db1/c13f 0 2024-04-02T04:42:52.787 INFO:tasks.workunit.client.0.smithi016.stdout:4/845: mkdir d5/d17/d18/d6d/da4/db7/dee/d104 0 2024-04-02T04:42:52.788 INFO:tasks.workunit.client.0.smithi016.stdout:9/941: creat d0/d4e/d50/d63/d6e/d4b/d58/d6a/f125 x:0 0 0 2024-04-02T04:42:52.789 INFO:tasks.workunit.client.0.smithi016.stdout:5/835: rename d3/d40/db6/de0/f10c to d3/d40/d4f/d6b/d45/f12b 0 2024-04-02T04:42:52.790 INFO:tasks.workunit.client.0.smithi016.stdout:7/844: fsync d0/d46/fc 0 2024-04-02T04:42:52.793 INFO:tasks.workunit.client.0.smithi016.stdout:9/942: truncate d0/d4e/d50/d63/d6e/d4b/d58/d6a/fbc 1496978 0 2024-04-02T04:42:52.795 INFO:tasks.workunit.client.0.smithi016.stdout:7/845: creat d0/d21/f11e x:0 0 0 2024-04-02T04:42:52.796 INFO:tasks.workunit.client.0.smithi016.stdout:1/835: rename d14/d19/d1d/d29/d39/d46/dcb/d43/c59 to d14/d19/d1e/d71/dcf/d102/c113 0 2024-04-02T04:42:52.797 INFO:tasks.workunit.client.0.smithi016.stdout:4/846: rmdir d5/d16/d3f/df1 0 2024-04-02T04:42:52.798 INFO:tasks.workunit.client.0.smithi016.stdout:7/846: symlink d0/d46/dbf/l11f 0 2024-04-02T04:42:52.800 INFO:tasks.workunit.client.0.smithi016.stdout:9/943: creat d0/d9/d4d/f126 x:0 0 0 2024-04-02T04:42:52.802 INFO:tasks.workunit.client.0.smithi016.stdout:2/920: dread d12/d35/d3f/d56/f94 [0,4194304] 0 2024-04-02T04:42:52.803 INFO:tasks.workunit.client.1.smithi027.stdout:4/287: sync 2024-04-02T04:42:52.803 INFO:tasks.workunit.client.1.smithi027.stdout:0/380: symlink d4/d16/l78 0 2024-04-02T04:42:52.805 INFO:tasks.workunit.client.0.smithi016.stdout:0/817: dread d0/f67 [0,4194304] 0 2024-04-02T04:42:52.805 INFO:tasks.workunit.client.0.smithi016.stdout:4/847: rename d5/d16/df8/fe3 to d5/d17/d18/d6d/d8e/d9c/dcc/d6f/dcf/f105 0 2024-04-02T04:42:52.807 INFO:tasks.workunit.client.0.smithi016.stdout:0/818: write d0/d4f/daf/fd5 [1358697,3479] 0 2024-04-02T04:42:52.809 INFO:tasks.workunit.client.1.smithi027.stdout:4/288: fdatasync d24/f33 0 2024-04-02T04:42:52.811 INFO:tasks.workunit.client.0.smithi016.stdout:2/921: creat d12/d35/d101/d76/da0/de7/dd1/f140 x:0 0 0 2024-04-02T04:42:52.818 INFO:tasks.workunit.client.0.smithi016.stdout:8/851: write d1/d70/d71/d74/f96 [448942,127156] 0 2024-04-02T04:42:52.820 INFO:tasks.workunit.client.1.smithi027.stdout:7/346: link d2/f4c d2/df/f6b 0 2024-04-02T04:42:52.824 INFO:tasks.workunit.client.0.smithi016.stdout:2/922: dread - d12/d9f/fb8 zero size 2024-04-02T04:42:52.828 INFO:tasks.workunit.client.1.smithi027.stdout:6/352: rename d13/d1b/d24/d28/d3c/c43 to d13/d1b/c72 0 2024-04-02T04:42:52.829 INFO:tasks.workunit.client.0.smithi016.stdout:7/847: dread d0/d46/d16/d83/f93 [0,4194304] 0 2024-04-02T04:42:52.830 INFO:tasks.workunit.client.0.smithi016.stdout:4/848: creat d5/d1d/d5f/d95/df2/f106 x:0 0 0 2024-04-02T04:42:52.831 INFO:tasks.workunit.client.1.smithi027.stdout:0/381: creat d4/d21/d28/d4e/d5f/f79 x:0 0 0 2024-04-02T04:42:52.833 INFO:tasks.workunit.client.1.smithi027.stdout:8/388: creat d5/f7d x:0 0 0 2024-04-02T04:42:52.833 INFO:tasks.workunit.client.0.smithi016.stdout:1/836: dread d14/f28 [0,4194304] 0 2024-04-02T04:42:52.837 INFO:tasks.workunit.client.0.smithi016.stdout:2/923: symlink d12/d35/d3f/d2b/da3/db1/l141 0 2024-04-02T04:42:52.839 INFO:tasks.workunit.client.1.smithi027.stdout:2/360: rename d1/db/d3b/d17/d2b/f52 to d1/db/d3b/d17/d68/f80 0 2024-04-02T04:42:52.840 INFO:tasks.workunit.client.0.smithi016.stdout:7/848: mkdir d0/d46/d16/dc2/d120 0 2024-04-02T04:42:52.844 INFO:tasks.workunit.client.1.smithi027.stdout:0/382: mknod d4/d38/d66/d2d/c7a 0 2024-04-02T04:42:52.845 INFO:tasks.workunit.client.0.smithi016.stdout:8/852: symlink d1/d6/d36/d4f/dc9/dfa/l117 0 2024-04-02T04:42:52.845 INFO:tasks.workunit.client.1.smithi027.stdout:8/389: rmdir d5/d8/db/dd/d5f 39 2024-04-02T04:42:52.848 INFO:tasks.workunit.client.0.smithi016.stdout:1/837: creat d14/d19/d1d/d29/d39/d46/dcb/f114 x:0 0 0 2024-04-02T04:42:52.849 INFO:tasks.workunit.client.1.smithi027.stdout:7/347: symlink d2/l6c 0 2024-04-02T04:42:52.849 INFO:tasks.workunit.client.1.smithi027.stdout:7/348: chown d2/d7/da/dc/d16/d3a/d4d 18747728 1 2024-04-02T04:42:52.850 INFO:tasks.workunit.client.0.smithi016.stdout:0/819: rmdir d0/d16/d23/d3e/db3/d10c/d10d 0 2024-04-02T04:42:52.850 INFO:tasks.workunit.client.1.smithi027.stdout:7/349: chown d2/d1c/f40 5 1 2024-04-02T04:42:52.855 INFO:tasks.workunit.client.0.smithi016.stdout:7/849: unlink d0/d46/d37/dbe/dae/df8/c10e 0 2024-04-02T04:42:52.859 INFO:tasks.workunit.client.0.smithi016.stdout:4/849: symlink d5/d16/df8/d7e/l107 0 2024-04-02T04:42:52.867 INFO:tasks.workunit.client.1.smithi027.stdout:7/350: creat d2/d1d/d57/f6d x:0 0 0 2024-04-02T04:42:52.868 INFO:tasks.workunit.client.1.smithi027.stdout:0/383: dwrite d4/d3d/f47 [0,4194304] 0 2024-04-02T04:42:52.868 INFO:tasks.workunit.client.0.smithi016.stdout:5/836: dwrite d3/d40/d4f/d9c/fc7 [0,4194304] 0 2024-04-02T04:42:52.869 INFO:tasks.workunit.client.0.smithi016.stdout:5/837: chown d3/d40/d4f/d67/dbf/d111 126349689 1 2024-04-02T04:42:52.870 INFO:tasks.workunit.client.0.smithi016.stdout:0/820: creat d0/d4f/daf/f116 x:0 0 0 2024-04-02T04:42:52.872 INFO:tasks.workunit.client.1.smithi027.stdout:2/361: mkdir d1/db/d3b/d61/d75/d81 0 2024-04-02T04:42:52.879 INFO:tasks.workunit.client.0.smithi016.stdout:4/850: symlink d5/d17/dcd/l108 0 2024-04-02T04:42:52.879 INFO:tasks.workunit.client.1.smithi027.stdout:4/289: write f13 [205682,92644] 0 2024-04-02T04:42:52.880 INFO:tasks.workunit.client.1.smithi027.stdout:8/390: fsync d5/d8/d48/f5d 0 2024-04-02T04:42:52.887 INFO:tasks.workunit.client.1.smithi027.stdout:7/351: read d2/d7/f18 [1247032,54723] 0 2024-04-02T04:42:52.888 INFO:tasks.workunit.client.0.smithi016.stdout:9/944: dwrite d0/d9/db/dc9/f57 [0,4194304] 0 2024-04-02T04:42:52.888 INFO:tasks.workunit.client.0.smithi016.stdout:5/838: rename d3/d40/d10d to d3/d40/d4f/d67/dbf/d104/d12c 0 2024-04-02T04:42:52.890 INFO:tasks.workunit.client.1.smithi027.stdout:4/290: creat d24/d38/f5e x:0 0 0 2024-04-02T04:42:52.891 INFO:tasks.workunit.client.0.smithi016.stdout:2/924: creat d12/d35/f142 x:0 0 0 2024-04-02T04:42:52.893 INFO:tasks.workunit.client.0.smithi016.stdout:7/850: symlink d0/d46/d16/dc2/dea/l121 0 2024-04-02T04:42:52.894 INFO:tasks.workunit.client.1.smithi027.stdout:0/384: symlink d4/d21/l7b 0 2024-04-02T04:42:52.895 INFO:tasks.workunit.client.0.smithi016.stdout:4/851: symlink d5/d17/dcd/l109 0 2024-04-02T04:42:52.896 INFO:tasks.workunit.client.1.smithi027.stdout:7/352: creat d2/d1c/f6e x:0 0 0 2024-04-02T04:42:52.897 INFO:tasks.workunit.client.1.smithi027.stdout:7/353: stat d2/d7/l32 0 2024-04-02T04:42:52.898 INFO:tasks.workunit.client.0.smithi016.stdout:8/853: creat d1/d6/d36/d3f/f118 x:0 0 0 2024-04-02T04:42:52.901 INFO:tasks.workunit.client.1.smithi027.stdout:4/291: chown fc 0 1 2024-04-02T04:42:52.902 INFO:tasks.workunit.client.1.smithi027.stdout:4/292: fdatasync d24/d38/f5e 0 2024-04-02T04:42:52.903 INFO:tasks.workunit.client.0.smithi016.stdout:3/965: sync 2024-04-02T04:42:52.903 INFO:tasks.workunit.client.1.smithi027.stdout:4/293: write d24/d38/f5e [732127,24539] 0 2024-04-02T04:42:52.906 INFO:tasks.workunit.client.1.smithi027.stdout:0/385: creat d4/d3d/f7c x:0 0 0 2024-04-02T04:42:52.907 INFO:tasks.workunit.client.1.smithi027.stdout:6/353: write d13/d1b/d24/d25/d27/f44 [3735735,3996] 0 2024-04-02T04:42:52.908 INFO:tasks.workunit.client.1.smithi027.stdout:7/354: mkdir d2/d7/da/dc/d16/d3a/d4d/d6f 0 2024-04-02T04:42:52.910 INFO:tasks.workunit.client.0.smithi016.stdout:9/945: creat d0/d4a/d75/d95/d9b/f127 x:0 0 0 2024-04-02T04:42:52.911 INFO:tasks.workunit.client.1.smithi027.stdout:2/362: link d1/db/d3b/d17/l21 d1/db/d3b/d17/d68/d69/l82 0 2024-04-02T04:42:52.912 INFO:tasks.workunit.client.1.smithi027.stdout:2/363: chown d1/db/d3b/d17/d2b/f4e 0 1 2024-04-02T04:42:52.913 INFO:tasks.workunit.client.0.smithi016.stdout:0/821: link d0/d16/d24/d4e/d8d/d71/fd8 d0/d16/d24/d62/d9c/dfb/d111/f117 0 2024-04-02T04:42:52.913 INFO:tasks.workunit.client.1.smithi027.stdout:6/354: dwrite f12 [0,4194304] 0 2024-04-02T04:42:52.915 INFO:tasks.workunit.client.0.smithi016.stdout:1/838: dwrite d14/d19/d1d/f30 [0,4194304] 0 2024-04-02T04:42:52.917 INFO:tasks.workunit.client.0.smithi016.stdout:5/839: mknod d3/d40/d4f/d6b/d45/da6/d102/dc4/d108/d6d/c12d 0 2024-04-02T04:42:52.917 INFO:tasks.workunit.client.0.smithi016.stdout:1/839: write d14/d19/d1d/d29/d4c/fd6 [367382,122123] 0 2024-04-02T04:42:52.918 INFO:tasks.workunit.client.1.smithi027.stdout:4/294: symlink d24/d38/l5f 0 2024-04-02T04:42:52.923 INFO:tasks.workunit.client.1.smithi027.stdout:2/364: dwrite d1/db/d3b/d17/f5d [4194304,4194304] 0 2024-04-02T04:42:52.930 INFO:tasks.workunit.client.1.smithi027.stdout:7/355: dread d2/d7/da/dc/d16/d3f/f43 [0,4194304] 0 2024-04-02T04:42:52.934 INFO:tasks.workunit.client.0.smithi016.stdout:8/854: rmdir d1/d6/d36/d4f/ded 39 2024-04-02T04:42:52.936 INFO:tasks.workunit.client.1.smithi027.stdout:8/391: creat d5/d8/db/dd/d5f/f7e x:0 0 0 2024-04-02T04:42:52.937 INFO:tasks.workunit.client.1.smithi027.stdout:8/392: truncate d5/d8/db/dd/d5f/f7e 755238 0 2024-04-02T04:42:52.939 INFO:tasks.workunit.client.1.smithi027.stdout:6/355: mkdir d13/d1b/d24/d25/d56/d73 0 2024-04-02T04:42:52.939 INFO:tasks.workunit.client.1.smithi027.stdout:8/393: write d5/d8/db/dd/d5f/f2b [420796,118966] 0 2024-04-02T04:42:52.942 INFO:tasks.workunit.client.1.smithi027.stdout:4/295: mkdir d24/d38/d44/d60 0 2024-04-02T04:42:52.943 INFO:tasks.workunit.client.0.smithi016.stdout:9/946: mknod d0/d3/dd1/c128 0 2024-04-02T04:42:52.947 INFO:tasks.workunit.client.0.smithi016.stdout:0/822: creat d0/d4f/f118 x:0 0 0 2024-04-02T04:42:52.947 INFO:tasks.workunit.client.0.smithi016.stdout:2/925: dread d12/d35/d101/f9d [0,4194304] 0 2024-04-02T04:42:52.948 INFO:tasks.workunit.client.0.smithi016.stdout:0/823: chown d0/d16/d24/d4e/d8d/d71/f72 9830 1 2024-04-02T04:42:52.949 INFO:tasks.workunit.client.1.smithi027.stdout:2/365: readlink d1/db/d3b/d17/l4b 0 2024-04-02T04:42:52.949 INFO:tasks.workunit.client.1.smithi027.stdout:8/394: dwrite d5/d8/db/dd/d5f/d27/d31/d5b/f67 [0,4194304] 0 2024-04-02T04:42:52.952 INFO:tasks.workunit.client.0.smithi016.stdout:5/840: unlink d3/f9 0 2024-04-02T04:42:52.953 INFO:tasks.workunit.client.1.smithi027.stdout:7/356: mknod d2/d7/da/dc/d1f/c70 0 2024-04-02T04:42:52.954 INFO:tasks.workunit.client.1.smithi027.stdout:7/357: chown d2/df/f10 958 1 2024-04-02T04:42:52.955 INFO:tasks.workunit.client.1.smithi027.stdout:7/358: write d2/d7/da/dc/f30 [3297735,18681] 0 2024-04-02T04:42:52.956 INFO:tasks.workunit.client.1.smithi027.stdout:7/359: readlink d2/d7/da/dc/d1f/d33/d36/l56 0 2024-04-02T04:42:52.957 INFO:tasks.workunit.client.1.smithi027.stdout:5/453: sync 2024-04-02T04:42:52.957 INFO:tasks.workunit.client.1.smithi027.stdout:9/333: sync 2024-04-02T04:42:52.959 INFO:tasks.workunit.client.0.smithi016.stdout:4/852: mkdir d5/d1d/d1e/d10a 0 2024-04-02T04:42:52.960 INFO:tasks.workunit.client.1.smithi027.stdout:6/356: symlink d13/d1b/d24/d25/d42/l74 0 2024-04-02T04:42:52.961 INFO:tasks.workunit.client.1.smithi027.stdout:6/357: readlink d13/d1b/d24/d25/d42/l69 0 2024-04-02T04:42:52.962 INFO:tasks.workunit.client.0.smithi016.stdout:8/855: rename d1/d6/d36/d3f/d40/d49/d4a/d108 to d1/d6/d86/d119 0 2024-04-02T04:42:52.965 INFO:tasks.workunit.client.0.smithi016.stdout:3/966: write d2/d2e/f8d [4816411,112693] 0 2024-04-02T04:42:52.966 INFO:tasks.workunit.client.0.smithi016.stdout:9/947: mkdir d0/d9/db/d54/d129 0 2024-04-02T04:42:52.969 INFO:tasks.workunit.client.1.smithi027.stdout:8/395: creat d5/d8/f7f x:0 0 0 2024-04-02T04:42:52.970 INFO:tasks.workunit.client.1.smithi027.stdout:8/396: truncate d5/d8/db/df/f79 744315 0 2024-04-02T04:42:52.971 INFO:tasks.workunit.client.0.smithi016.stdout:0/824: symlink d0/d16/d2e/d53/d6f/l119 0 2024-04-02T04:42:52.972 INFO:tasks.workunit.client.0.smithi016.stdout:0/825: write d0/d16/d2e/d36/d9f/fa0 [4638324,117630] 0 2024-04-02T04:42:52.973 INFO:tasks.workunit.client.0.smithi016.stdout:5/841: symlink d3/d40/d4f/d6b/d45/da6/d102/dc4/l12e 0 2024-04-02T04:42:52.976 INFO:tasks.workunit.client.0.smithi016.stdout:7/851: write d0/d21/f36 [891392,77147] 0 2024-04-02T04:42:52.977 INFO:tasks.workunit.client.1.smithi027.stdout:2/366: dread d1/db/d3b/d17/f2c [0,4194304] 0 2024-04-02T04:42:52.978 INFO:tasks.workunit.client.1.smithi027.stdout:7/360: creat d2/d7/da/dc/d1f/d33/f71 x:0 0 0 2024-04-02T04:42:52.978 INFO:tasks.workunit.client.0.smithi016.stdout:4/853: rmdir d5/d17/d18/d6d/d8e/d9c/dcc/d6f 39 2024-04-02T04:42:52.979 INFO:tasks.workunit.client.1.smithi027.stdout:8/397: dwrite d5/d8/db/df/f79 [0,4194304] 0 2024-04-02T04:42:52.980 INFO:tasks.workunit.client.1.smithi027.stdout:2/367: readlink d1/db/d3b/d17/d29/l45 0 2024-04-02T04:42:52.980 INFO:tasks.workunit.client.1.smithi027.stdout:9/334: mkdir d1a/d1d/d29/d50/d74 0 2024-04-02T04:42:52.981 INFO:tasks.workunit.client.1.smithi027.stdout:7/361: write d2/d7/da/dc/f30 [2684816,119621] 0 2024-04-02T04:42:52.982 INFO:tasks.workunit.client.0.smithi016.stdout:8/856: mkdir d1/d6/d36/d3f/d77/d11a 0 2024-04-02T04:42:52.983 INFO:tasks.workunit.client.0.smithi016.stdout:3/967: dread - d2/d2e/d36/d66/d90/faf zero size 2024-04-02T04:42:52.983 INFO:tasks.workunit.client.1.smithi027.stdout:4/296: mknod d24/d38/d44/d41/d5a/c61 0 2024-04-02T04:42:52.986 INFO:tasks.workunit.client.1.smithi027.stdout:0/386: link d4/d21/d28/f48 d4/d21/f7d 0 2024-04-02T04:42:52.986 INFO:tasks.workunit.client.0.smithi016.stdout:2/926: symlink d12/d35/d3b/d42/d59/l143 0 2024-04-02T04:42:52.986 INFO:tasks.workunit.client.0.smithi016.stdout:3/968: write d2/d15/d60/fe5 [145499,67771] 0 2024-04-02T04:42:52.987 INFO:tasks.workunit.client.1.smithi027.stdout:8/398: dwrite d5/d8/f47 [0,4194304] 0 2024-04-02T04:42:52.989 INFO:tasks.workunit.client.1.smithi027.stdout:7/362: dread d2/d1c/f4b [0,4194304] 0 2024-04-02T04:42:52.991 INFO:tasks.workunit.client.0.smithi016.stdout:5/842: mknod d3/df/d18/d26/d70/c12f 0 2024-04-02T04:42:52.996 INFO:tasks.workunit.client.1.smithi027.stdout:5/454: symlink d0/df/d29/da5/da6/la9 0 2024-04-02T04:42:52.998 INFO:tasks.workunit.client.1.smithi027.stdout:9/335: mknod d1a/d1d/d29/d56/c75 0 2024-04-02T04:42:53.000 INFO:tasks.workunit.client.1.smithi027.stdout:9/336: write d1a/d1d/d29/d37/d20/f3f [4323771,125228] 0 2024-04-02T04:42:53.001 INFO:tasks.workunit.client.1.smithi027.stdout:9/337: write fa [1776819,130592] 0 2024-04-02T04:42:53.002 INFO:tasks.workunit.client.0.smithi016.stdout:2/927: unlink d12/d35/d3f/f7e 0 2024-04-02T04:42:53.004 INFO:tasks.workunit.client.1.smithi027.stdout:6/358: mkdir d13/d1b/d48/d75 0 2024-04-02T04:42:53.009 INFO:tasks.workunit.client.1.smithi027.stdout:4/297: unlink d24/d38/c3b 0 2024-04-02T04:42:53.009 INFO:tasks.workunit.client.1.smithi027.stdout:6/359: dread d13/d1b/d24/d28/f35 [0,4194304] 0 2024-04-02T04:42:53.010 INFO:tasks.workunit.client.0.smithi016.stdout:5/843: mknod d3/d40/d4f/d67/dbf/d104/c130 0 2024-04-02T04:42:53.011 INFO:tasks.workunit.client.1.smithi027.stdout:0/387: read d4/d38/d66/f24 [747604,13893] 0 2024-04-02T04:42:53.012 INFO:tasks.workunit.client.1.smithi027.stdout:3/262: sync 2024-04-02T04:42:53.013 INFO:tasks.workunit.client.1.smithi027.stdout:7/363: rmdir d2/d7/da/dc/d16/d2f 39 2024-04-02T04:42:53.016 INFO:tasks.workunit.client.1.smithi027.stdout:5/455: mknod d0/df/d11/d37/caa 0 2024-04-02T04:42:53.017 INFO:tasks.workunit.client.1.smithi027.stdout:5/456: write d0/d1d/f97 [816900,64361] 0 2024-04-02T04:42:53.022 INFO:tasks.workunit.client.0.smithi016.stdout:0/826: dwrite d0/d16/d2e/d36/d9f/fc1 [0,4194304] 0 2024-04-02T04:42:53.022 INFO:tasks.workunit.client.0.smithi016.stdout:3/969: creat d2/d15/d1a/dd5/f142 x:0 0 0 2024-04-02T04:42:53.023 INFO:tasks.workunit.client.0.smithi016.stdout:4/854: write d5/d17/d18/d6d/d8e/d9c/dcc/fc7 [236421,59290] 0 2024-04-02T04:42:53.024 INFO:tasks.workunit.client.0.smithi016.stdout:5/844: dread - d3/d40/d4f/d6b/d45/da6/d102/dc4/d108/f92 zero size 2024-04-02T04:42:53.026 INFO:tasks.workunit.client.1.smithi027.stdout:6/360: dread d13/d1b/f1f [0,4194304] 0 2024-04-02T04:42:53.027 INFO:tasks.workunit.client.0.smithi016.stdout:8/857: write d1/d6/d36/d3f/d40/d45/f5b [709389,30754] 0 2024-04-02T04:42:53.029 INFO:tasks.workunit.client.0.smithi016.stdout:4/855: write d5/d8/fd [1347396,32991] 0 2024-04-02T04:42:53.029 INFO:tasks.workunit.client.0.smithi016.stdout:2/928: symlink d12/d35/l144 0 2024-04-02T04:42:53.029 INFO:tasks.workunit.client.1.smithi027.stdout:4/298: mkdir d24/d38/d47/d62 0 2024-04-02T04:42:53.031 INFO:tasks.workunit.client.0.smithi016.stdout:7/852: dwrite d0/d46/d37/d47/f6b [0,4194304] 0 2024-04-02T04:42:53.031 INFO:tasks.workunit.client.1.smithi027.stdout:4/299: write d24/d48/f4e [1121486,46102] 0 2024-04-02T04:42:53.035 INFO:tasks.workunit.client.1.smithi027.stdout:6/361: dwrite d13/d1b/f2b [0,4194304] 0 2024-04-02T04:42:53.035 INFO:tasks.workunit.client.0.smithi016.stdout:6/883: sync 2024-04-02T04:42:53.042 INFO:tasks.workunit.client.0.smithi016.stdout:0/827: rmdir d0/d16/d24/d62/d9c/dfb 39 2024-04-02T04:42:53.044 INFO:tasks.workunit.client.0.smithi016.stdout:8/858: unlink d1/d70/f10c 0 2024-04-02T04:42:53.052 INFO:tasks.workunit.client.1.smithi027.stdout:5/457: mkdir d0/df/d29/dab 0 2024-04-02T04:42:53.055 INFO:tasks.workunit.client.1.smithi027.stdout:9/338: symlink d1a/d1d/d29/d37/l76 0 2024-04-02T04:42:53.055 INFO:tasks.workunit.client.1.smithi027.stdout:0/388: write f2 [3209302,67359] 0 2024-04-02T04:42:53.060 INFO:tasks.workunit.client.0.smithi016.stdout:8/859: dread - d1/d70/d71/d9e/fef zero size 2024-04-02T04:42:53.061 INFO:tasks.workunit.client.0.smithi016.stdout:8/860: stat d1/d6/d36/d3f/d40/d63/l10b 0 2024-04-02T04:42:53.061 INFO:tasks.workunit.client.1.smithi027.stdout:1/337: sync 2024-04-02T04:42:53.064 INFO:tasks.workunit.client.1.smithi027.stdout:4/300: read f23 [109461,20357] 0 2024-04-02T04:42:53.065 INFO:tasks.workunit.client.0.smithi016.stdout:5/845: dread d3/d40/d4f/d6b/d45/da6/d102/dc4/d108/f5f [0,4194304] 0 2024-04-02T04:42:53.066 INFO:tasks.workunit.client.1.smithi027.stdout:3/263: symlink d1/d1a/l56 0 2024-04-02T04:42:53.066 INFO:tasks.workunit.client.1.smithi027.stdout:3/264: readlink d1/d1a/l2b 0 2024-04-02T04:42:53.066 INFO:tasks.workunit.client.0.smithi016.stdout:5/846: stat d3/d40/d4f/d67/dbf/d111 0 2024-04-02T04:42:53.067 INFO:tasks.workunit.client.0.smithi016.stdout:2/929: dread d12/d35/d3f/d2b/fc1 [0,4194304] 0 2024-04-02T04:42:53.067 INFO:tasks.workunit.client.0.smithi016.stdout:3/970: rmdir d2/d2e/d36/d48/da2/de7/dec 0 2024-04-02T04:42:53.068 INFO:tasks.workunit.client.0.smithi016.stdout:2/930: fdatasync d12/d70/ff3 0 2024-04-02T04:42:53.069 INFO:tasks.workunit.client.0.smithi016.stdout:3/971: chown d2/d15/d16/l41 210993 1 2024-04-02T04:42:53.069 INFO:tasks.workunit.client.1.smithi027.stdout:7/364: creat d2/df/d64/f72 x:0 0 0 2024-04-02T04:42:53.069 INFO:tasks.workunit.client.1.smithi027.stdout:4/301: dread d24/d2d/f31 [0,4194304] 0 2024-04-02T04:42:53.070 INFO:tasks.workunit.client.1.smithi027.stdout:7/365: stat d2/d7/da/d35/f60 0 2024-04-02T04:42:53.070 INFO:tasks.workunit.client.1.smithi027.stdout:4/302: truncate f14 4591274 0 2024-04-02T04:42:53.071 INFO:tasks.workunit.client.0.smithi016.stdout:8/861: readlink d1/d6/d36/ld4 0 2024-04-02T04:42:53.071 INFO:tasks.workunit.client.0.smithi016.stdout:6/884: write d0/f73 [975546,97748] 0 2024-04-02T04:42:53.071 INFO:tasks.workunit.client.1.smithi027.stdout:5/458: readlink d0/df/d11/d37/l7d 0 2024-04-02T04:42:53.072 INFO:tasks.workunit.client.1.smithi027.stdout:4/303: dread d24/f33 [0,4194304] 0 2024-04-02T04:42:53.073 INFO:tasks.workunit.client.1.smithi027.stdout:5/459: truncate d0/df/d11/d13/f3e 4541885 0 2024-04-02T04:42:53.074 INFO:tasks.workunit.client.1.smithi027.stdout:0/389: rmdir d4/d38 39 2024-04-02T04:42:53.079 INFO:tasks.workunit.client.0.smithi016.stdout:0/828: creat d0/f11a x:0 0 0 2024-04-02T04:42:53.079 INFO:tasks.workunit.client.0.smithi016.stdout:4/856: dwrite d5/d17/d18/d25/f3b [0,4194304] 0 2024-04-02T04:42:53.080 INFO:tasks.workunit.client.1.smithi027.stdout:2/368: getdents d1/db/d3b/d17/d2b 0 2024-04-02T04:42:53.080 INFO:tasks.workunit.client.1.smithi027.stdout:5/460: dwrite d0/d1d/f97 [0,4194304] 0 2024-04-02T04:42:53.081 INFO:tasks.workunit.client.1.smithi027.stdout:2/369: stat d1/db/d3b/f37 0 2024-04-02T04:42:53.082 INFO:tasks.workunit.client.1.smithi027.stdout:5/461: truncate d0/d1d/f97 4659473 0 2024-04-02T04:42:53.084 INFO:tasks.workunit.client.0.smithi016.stdout:5/847: rename d3/d40/d4f/d67/dbf/d111/d116 to d3/df/d18/dfd/d10f/d131 0 2024-04-02T04:42:53.086 INFO:tasks.workunit.client.1.smithi027.stdout:1/338: unlink d0/d1e/c5a 0 2024-04-02T04:42:53.089 INFO:tasks.workunit.client.0.smithi016.stdout:2/931: mknod d12/d35/d3b/d42/d59/d91/d137/c145 0 2024-04-02T04:42:53.090 INFO:tasks.workunit.client.1.smithi027.stdout:1/339: chown d0/f5c 91362 1 2024-04-02T04:42:53.090 INFO:tasks.workunit.client.0.smithi016.stdout:2/932: chown d12/d35/d3b/d42/d67/c6b 416 1 2024-04-02T04:42:53.090 INFO:tasks.workunit.client.0.smithi016.stdout:3/972: creat d2/d15/d16/d8a/dd1/d6e/f143 x:0 0 0 2024-04-02T04:42:53.093 INFO:tasks.workunit.client.1.smithi027.stdout:8/399: getdents d5/d8/db/dd/d5f/d27/d31/d5b 0 2024-04-02T04:42:53.096 INFO:tasks.workunit.client.0.smithi016.stdout:2/933: creat d12/d9f/f146 x:0 0 0 2024-04-02T04:42:53.099 INFO:tasks.workunit.client.0.smithi016.stdout:3/973: mkdir d2/d15/d60/d144 0 2024-04-02T04:42:53.100 INFO:tasks.workunit.client.1.smithi027.stdout:8/400: dwrite d5/d8/db/dd/f10 [0,4194304] 0 2024-04-02T04:42:53.100 INFO:tasks.workunit.client.1.smithi027.stdout:0/390: chown d4/d38/d66/f2a 70687 1 2024-04-02T04:42:53.101 INFO:tasks.workunit.client.1.smithi027.stdout:0/391: dread - d4/d16/f6a zero size 2024-04-02T04:42:53.102 INFO:tasks.workunit.client.1.smithi027.stdout:8/401: write d5/d8/d15/f37 [2758264,67294] 0 2024-04-02T04:42:53.104 INFO:tasks.workunit.client.1.smithi027.stdout:8/402: write d5/d8/d15/d71/f72 [410845,11779] 0 2024-04-02T04:42:53.105 INFO:tasks.workunit.client.0.smithi016.stdout:4/857: fsync d5/d17/dcd/fd2 0 2024-04-02T04:42:53.106 INFO:tasks.workunit.client.1.smithi027.stdout:6/362: truncate d13/d1b/f1e 1195123 0 2024-04-02T04:42:53.108 INFO:tasks.workunit.client.0.smithi016.stdout:5/848: mkdir d3/d40/d4f/d132 0 2024-04-02T04:42:53.109 INFO:tasks.workunit.client.0.smithi016.stdout:5/849: dread - d3/df/dd5/f128 zero size 2024-04-02T04:42:53.111 INFO:tasks.workunit.client.0.smithi016.stdout:2/934: creat d12/d35/d3f/d2b/da3/db1/f147 x:0 0 0 2024-04-02T04:42:53.116 INFO:tasks.workunit.client.0.smithi016.stdout:3/974: rename d2/d15/d1a/f1e to d2/d15/d16/d8a/dcd/f145 0 2024-04-02T04:42:53.117 INFO:tasks.workunit.client.0.smithi016.stdout:0/829: write d0/fe [2807101,53028] 0 2024-04-02T04:42:53.117 INFO:tasks.workunit.client.0.smithi016.stdout:3/975: chown d2/d2e/d36/d66 63465 1 2024-04-02T04:42:53.118 INFO:tasks.workunit.client.0.smithi016.stdout:8/862: dwrite d1/d6/d36/d100/f9f [0,4194304] 0 2024-04-02T04:42:53.118 INFO:tasks.workunit.client.0.smithi016.stdout:0/830: fdatasync d0/d16/f80 0 2024-04-02T04:42:53.119 INFO:tasks.workunit.client.1.smithi027.stdout:3/265: mknod d1/c57 0 2024-04-02T04:42:53.120 INFO:tasks.workunit.client.1.smithi027.stdout:1/340: dwrite d0/f69 [0,4194304] 0 2024-04-02T04:42:53.122 INFO:tasks.workunit.client.1.smithi027.stdout:1/341: truncate d0/de/d52/f65 258717 0 2024-04-02T04:42:53.123 INFO:tasks.workunit.client.0.smithi016.stdout:8/863: chown d1/d70/d71/d74/d82/f10e 1 1 2024-04-02T04:42:53.123 INFO:tasks.workunit.client.0.smithi016.stdout:0/831: readlink d0/d16/d2e/l86 0 2024-04-02T04:42:53.124 INFO:tasks.workunit.client.0.smithi016.stdout:3/976: truncate d2/d15/d16/d57/d96/f12e 206307 0 2024-04-02T04:42:53.125 INFO:tasks.workunit.client.0.smithi016.stdout:4/858: symlink d5/d17/dcd/def/l10b 0 2024-04-02T04:42:53.127 INFO:tasks.workunit.client.0.smithi016.stdout:3/977: fsync d2/d15/d16/d8a/dd1/d6e/d6f/d78/f134 0 2024-04-02T04:42:53.128 INFO:tasks.workunit.client.0.smithi016.stdout:3/978: read - d2/d15/d1a/d45/f132 zero size 2024-04-02T04:42:53.129 INFO:tasks.workunit.client.0.smithi016.stdout:5/850: mkdir d3/df/d18/d26/d48/d133 0 2024-04-02T04:42:53.130 INFO:tasks.workunit.client.0.smithi016.stdout:7/853: sync 2024-04-02T04:42:53.130 INFO:tasks.workunit.client.0.smithi016.stdout:1/840: sync 2024-04-02T04:42:53.130 INFO:tasks.workunit.client.0.smithi016.stdout:9/948: sync 2024-04-02T04:42:53.131 INFO:tasks.workunit.client.1.smithi027.stdout:7/366: creat d2/d7/da/dc/d16/d2f/f73 x:0 0 0 2024-04-02T04:42:53.131 INFO:tasks.workunit.client.0.smithi016.stdout:8/864: dwrite d1/d6/d36/d3f/d77/fa7 [0,4194304] 0 2024-04-02T04:42:53.131 INFO:tasks.workunit.client.0.smithi016.stdout:5/851: fsync d3/df/dd5/f128 0 2024-04-02T04:42:53.132 INFO:tasks.workunit.client.1.smithi027.stdout:5/462: dwrite d0/df/d11/d13/f26 [0,4194304] 0 2024-04-02T04:42:53.133 INFO:tasks.workunit.client.1.smithi027.stdout:0/392: symlink d4/d38/l7e 0 2024-04-02T04:42:53.133 INFO:tasks.workunit.client.0.smithi016.stdout:1/841: dread - d14/d19/d1d/d29/d39/d4d/f10b zero size 2024-04-02T04:42:53.137 INFO:tasks.workunit.client.1.smithi027.stdout:9/339: rmdir d1a/d1d/d29/d51/d70 0 2024-04-02T04:42:53.138 INFO:tasks.workunit.client.1.smithi027.stdout:9/340: chown d1a/d1d/d29/d37/d20 5238546 1 2024-04-02T04:42:53.140 INFO:tasks.workunit.client.0.smithi016.stdout:8/865: dwrite d1/d6/d36/d3f/d40/d49/d4a/f5a [0,4194304] 0 2024-04-02T04:42:53.141 INFO:tasks.workunit.client.1.smithi027.stdout:8/403: mknod d5/d8/db/dd/d5f/d27/c80 0 2024-04-02T04:42:53.144 INFO:tasks.workunit.client.0.smithi016.stdout:8/866: chown d1/d6/d36/d3f/d40/d49/d4a/d4c/fba 1590852 1 2024-04-02T04:42:53.145 INFO:tasks.workunit.client.0.smithi016.stdout:8/867: truncate d1/d70/d71/d74/f111 140371 0 2024-04-02T04:42:53.146 INFO:tasks.workunit.client.1.smithi027.stdout:6/363: unlink le 0 2024-04-02T04:42:53.147 INFO:tasks.workunit.client.0.smithi016.stdout:4/859: creat d5/d17/dcd/def/f10c x:0 0 0 2024-04-02T04:42:53.148 INFO:tasks.workunit.client.0.smithi016.stdout:5/852: dread d3/d40/d72/f7d [0,4194304] 0 2024-04-02T04:42:53.151 INFO:tasks.workunit.client.1.smithi027.stdout:2/370: mkdir d1/db/d3b/d83 0 2024-04-02T04:42:53.151 INFO:tasks.workunit.client.1.smithi027.stdout:9/341: dwrite d1a/d1d/f72 [0,4194304] 0 2024-04-02T04:42:53.152 INFO:tasks.workunit.client.0.smithi016.stdout:3/979: unlink d2/d15/d70/dcf/d102/l139 0 2024-04-02T04:42:53.152 INFO:tasks.workunit.client.0.smithi016.stdout:3/980: chown d2/d15/d16/d8a/f93 230045394 1 2024-04-02T04:42:53.153 INFO:tasks.workunit.client.0.smithi016.stdout:9/949: creat d0/d4a/d99/dd3/f12a x:0 0 0 2024-04-02T04:42:53.155 INFO:tasks.workunit.client.0.smithi016.stdout:9/950: write d0/d9/f3d [4818443,16995] 0 2024-04-02T04:42:53.156 INFO:tasks.workunit.client.0.smithi016.stdout:9/951: truncate d0/d3/d97/dac/ff0 677959 0 2024-04-02T04:42:53.158 INFO:tasks.workunit.client.1.smithi027.stdout:4/304: creat d24/f63 x:0 0 0 2024-04-02T04:42:53.160 INFO:tasks.workunit.client.0.smithi016.stdout:1/842: fdatasync d14/d19/d1d/f7d 0 2024-04-02T04:42:53.161 INFO:tasks.workunit.client.1.smithi027.stdout:0/393: dread - d4/d16/d1b/f3e zero size 2024-04-02T04:42:53.162 INFO:tasks.workunit.client.1.smithi027.stdout:0/394: readlink d4/d21/d28/l59 0 2024-04-02T04:42:53.163 INFO:tasks.workunit.client.0.smithi016.stdout:6/885: sync 2024-04-02T04:42:53.164 INFO:tasks.workunit.client.0.smithi016.stdout:4/860: creat d5/d1d/d5f/d95/f10d x:0 0 0 2024-04-02T04:42:53.165 INFO:tasks.workunit.client.0.smithi016.stdout:4/861: fsync d5/d16/df8/d7e/fcb 0 2024-04-02T04:42:53.165 INFO:tasks.workunit.client.1.smithi027.stdout:8/404: mknod d5/d8/db/dd/d5f/d27/d31/d5b/c81 0 2024-04-02T04:42:53.166 INFO:tasks.workunit.client.0.smithi016.stdout:5/853: truncate d3/df/d18/d26/f8c 3724066 0 2024-04-02T04:42:53.168 INFO:tasks.workunit.client.0.smithi016.stdout:3/981: mkdir d2/d15/d16/d8a/dd1/d6e/d6f/d78/d146 0 2024-04-02T04:42:53.169 INFO:tasks.workunit.client.0.smithi016.stdout:8/868: dread d1/d6/d36/d3f/d40/fb6 [4194304,4194304] 0 2024-04-02T04:42:53.171 INFO:tasks.workunit.client.0.smithi016.stdout:7/854: readlink d0/d46/d37/dbe/dae/df8/ld8 0 2024-04-02T04:42:53.171 INFO:tasks.workunit.client.0.smithi016.stdout:7/855: chown d0/d46/d16/d49/c109 5 1 2024-04-02T04:42:53.179 INFO:tasks.workunit.client.0.smithi016.stdout:1/843: unlink d14/d19/d50/c8e 0 2024-04-02T04:42:53.181 INFO:tasks.workunit.client.0.smithi016.stdout:2/935: getdents d12/d35/d3f/d2b/da3/db1 0 2024-04-02T04:42:53.182 INFO:tasks.workunit.client.0.smithi016.stdout:4/862: rmdir d5/d1d/d1e/da5 39 2024-04-02T04:42:53.184 INFO:tasks.workunit.client.0.smithi016.stdout:3/982: dread - d2/d15/d1a/ff0 zero size 2024-04-02T04:42:53.185 INFO:tasks.workunit.client.0.smithi016.stdout:5/854: dread d3/d40/d4f/d6b/d45/da6/d102/ff6 [0,4194304] 0 2024-04-02T04:42:53.185 INFO:tasks.workunit.client.0.smithi016.stdout:8/869: creat d1/d6/d36/d4f/dc9/f11b x:0 0 0 2024-04-02T04:42:53.188 INFO:tasks.workunit.client.0.smithi016.stdout:0/832: getdents d0/d16/d2e/d53 0 2024-04-02T04:42:53.193 INFO:tasks.workunit.client.0.smithi016.stdout:3/983: symlink d2/d2e/d36/d48/da2/dcb/l147 0 2024-04-02T04:42:53.194 INFO:tasks.workunit.client.0.smithi016.stdout:0/833: dwrite d0/d4f/fa2 [0,4194304] 0 2024-04-02T04:42:53.194 INFO:tasks.workunit.client.0.smithi016.stdout:5/855: mknod d3/d40/d4f/d67/dbf/d104/c134 0 2024-04-02T04:42:53.195 INFO:tasks.workunit.client.0.smithi016.stdout:5/856: chown d3/d40/db6/de0 12 1 2024-04-02T04:42:53.196 INFO:tasks.workunit.client.0.smithi016.stdout:7/856: mknod d0/d46/d37/dbe/dae/df8/dec/c122 0 2024-04-02T04:42:53.197 INFO:tasks.workunit.client.0.smithi016.stdout:8/870: rmdir d1/d6/d36/d3f/d40/d63/dc5 39 2024-04-02T04:42:53.201 INFO:tasks.workunit.client.0.smithi016.stdout:8/871: dread d1/d6/d36/f37 [0,4194304] 0 2024-04-02T04:42:53.202 INFO:tasks.workunit.client.0.smithi016.stdout:0/834: chown d0/d16/d2e/d53/da4/de3/cef 1201 1 2024-04-02T04:42:53.203 INFO:tasks.workunit.client.0.smithi016.stdout:8/872: write d1/d6/d86/ff9 [289767,110124] 0 2024-04-02T04:42:53.203 INFO:tasks.workunit.client.0.smithi016.stdout:5/857: rmdir d3/dbd 39 2024-04-02T04:42:53.209 INFO:tasks.workunit.client.0.smithi016.stdout:9/952: write d0/d9/db/f43 [5114196,108838] 0 2024-04-02T04:42:53.209 INFO:tasks.workunit.client.0.smithi016.stdout:8/873: dwrite d1/d6/d36/d3f/d77/f106 [0,4194304] 0 2024-04-02T04:42:53.210 INFO:tasks.workunit.client.0.smithi016.stdout:7/857: creat d0/d46/d37/dbe/dae/df8/d4c/f123 x:0 0 0 2024-04-02T04:42:53.215 INFO:tasks.workunit.client.0.smithi016.stdout:0/835: dwrite d0/d4f/f95 [4194304,4194304] 0 2024-04-02T04:42:53.219 INFO:tasks.workunit.client.0.smithi016.stdout:9/953: mkdir d0/d4e/d50/d63/d6e/d12b 0 2024-04-02T04:42:53.221 INFO:tasks.workunit.client.0.smithi016.stdout:8/874: rename d1/l2 to d1/d31/d67/d110/l11c 0 2024-04-02T04:42:53.226 INFO:tasks.workunit.client.0.smithi016.stdout:2/936: write d12/d35/d3b/d42/d59/f88 [750413,115805] 0 2024-04-02T04:42:53.226 INFO:tasks.workunit.client.0.smithi016.stdout:7/858: mknod d0/d46/d37/dbe/dae/df8/da9/c124 0 2024-04-02T04:42:53.229 INFO:tasks.workunit.client.0.smithi016.stdout:6/886: dwrite d0/d9/d16/de3/ff7 [0,4194304] 0 2024-04-02T04:42:53.230 INFO:tasks.workunit.client.0.smithi016.stdout:9/954: rename d0/d9/db/dc9/d53/l6b to d0/d3/d97/dac/l12c 0 2024-04-02T04:42:53.232 INFO:tasks.workunit.client.0.smithi016.stdout:7/859: creat d0/d21/d104/f125 x:0 0 0 2024-04-02T04:42:53.234 INFO:tasks.workunit.client.0.smithi016.stdout:2/937: mkdir d12/d35/d101/d76/dcf/d148 0 2024-04-02T04:42:53.235 INFO:tasks.workunit.client.1.smithi027.stdout:2/371: mkdir d1/db/d3b/d17/d6d/d84 0 2024-04-02T04:42:53.235 INFO:tasks.workunit.client.0.smithi016.stdout:7/860: readlink d0/d46/d16/l17 0 2024-04-02T04:42:53.236 INFO:tasks.workunit.client.1.smithi027.stdout:4/305: symlink d24/d48/l64 0 2024-04-02T04:42:53.236 INFO:tasks.workunit.client.0.smithi016.stdout:7/861: stat d0/d46/f9a 0 2024-04-02T04:42:53.238 INFO:tasks.workunit.client.0.smithi016.stdout:2/938: chown d12/d35/d101/d4f/f53 15 1 2024-04-02T04:42:53.239 INFO:tasks.workunit.client.0.smithi016.stdout:4/863: dwrite d5/d1d/d1e/f9d [0,4194304] 0 2024-04-02T04:42:53.240 INFO:tasks.workunit.client.1.smithi027.stdout:5/463: write d0/d1d/f97 [4416911,38669] 0 2024-04-02T04:42:53.241 INFO:tasks.workunit.client.0.smithi016.stdout:9/955: creat d0/d9/d4d/d78/da6/f12d x:0 0 0 2024-04-02T04:42:53.241 INFO:tasks.workunit.client.1.smithi027.stdout:9/342: write d1a/f38 [723276,17330] 0 2024-04-02T04:42:53.243 INFO:tasks.workunit.client.1.smithi027.stdout:1/342: truncate d0/de/f17 2764544 0 2024-04-02T04:42:53.245 INFO:tasks.workunit.client.1.smithi027.stdout:4/306: dread d24/f3c [0,4194304] 0 2024-04-02T04:42:53.247 INFO:tasks.workunit.client.1.smithi027.stdout:8/405: dwrite d5/d8/db/dd/f4f [0,4194304] 0 2024-04-02T04:42:53.248 INFO:tasks.workunit.client.1.smithi027.stdout:3/266: rename d1/de/df/f4b to d1/de/f58 0 2024-04-02T04:42:53.252 INFO:tasks.workunit.client.1.smithi027.stdout:4/307: chown d24/c54 1150540 1 2024-04-02T04:42:53.257 INFO:tasks.workunit.client.1.smithi027.stdout:7/367: dwrite d2/d7/da/dc/d16/f2c [0,4194304] 0 2024-04-02T04:42:53.258 INFO:tasks.workunit.client.1.smithi027.stdout:1/343: dread d0/de/d52/f67 [4194304,4194304] 0 2024-04-02T04:42:53.260 INFO:tasks.workunit.client.0.smithi016.stdout:2/939: dread - d12/d35/d3f/d56/d66/ff7 zero size 2024-04-02T04:42:53.262 INFO:tasks.workunit.client.0.smithi016.stdout:0/836: dwrite d0/d16/d2e/f96 [0,4194304] 0 2024-04-02T04:42:53.265 INFO:tasks.workunit.client.0.smithi016.stdout:0/837: stat d0/d16/d2e/d53/l2f 0 2024-04-02T04:42:53.278 INFO:tasks.workunit.client.0.smithi016.stdout:6/887: write d0/d9/d16/d56/d95/fab [829799,44268] 0 2024-04-02T04:42:53.280 INFO:tasks.workunit.client.1.smithi027.stdout:3/267: dwrite d1/de/f58 [0,4194304] 0 2024-04-02T04:42:53.282 INFO:tasks.workunit.client.1.smithi027.stdout:5/464: rename d0/d1d/d61/d6d to d0/df/d11/d65/dac 0 2024-04-02T04:42:53.287 INFO:tasks.workunit.client.1.smithi027.stdout:8/406: fdatasync d5/d8/db/dd/d5f/d27/d31/f5e 0 2024-04-02T04:42:53.289 INFO:tasks.workunit.client.0.smithi016.stdout:8/875: mkdir d1/d6/d36/d3f/d40/de9/d11d 0 2024-04-02T04:42:53.290 INFO:tasks.workunit.client.1.smithi027.stdout:0/395: dwrite d4/f11 [4194304,4194304] 0 2024-04-02T04:42:53.295 INFO:tasks.workunit.client.1.smithi027.stdout:0/396: truncate d4/d21/d28/d4e/d5f/f79 150199 0 2024-04-02T04:42:53.297 INFO:tasks.workunit.client.0.smithi016.stdout:7/862: link d0/d46/fc d0/d21/d104/f126 0 2024-04-02T04:42:53.297 INFO:tasks.workunit.client.0.smithi016.stdout:8/876: write d1/d6/d36/d4f/d5d/f95 [1727507,75698] 0 2024-04-02T04:42:53.297 INFO:tasks.workunit.client.1.smithi027.stdout:7/368: dread d2/d7/da/dc/d16/f2d [0,4194304] 0 2024-04-02T04:42:53.297 INFO:tasks.workunit.client.0.smithi016.stdout:9/956: rename d0/d3/d97/dac/ff0 to d0/d9/db/dc9/d53/f12e 0 2024-04-02T04:42:53.298 INFO:tasks.workunit.client.1.smithi027.stdout:2/372: mknod d1/db/d3b/d17/d6d/d84/c85 0 2024-04-02T04:42:53.299 INFO:tasks.workunit.client.1.smithi027.stdout:2/373: dread - d1/db/d3b/d17/d2b/d2f/f64 zero size 2024-04-02T04:42:53.302 INFO:tasks.workunit.client.0.smithi016.stdout:0/838: dread d0/d16/d2e/d37/f3c [0,4194304] 0 2024-04-02T04:42:53.303 INFO:tasks.workunit.client.0.smithi016.stdout:0/839: stat d0/d16/d2e/d53/d9d 0 2024-04-02T04:42:53.304 INFO:tasks.workunit.client.0.smithi016.stdout:0/840: dread - d0/d16/d23/d3e/fc6 zero size 2024-04-02T04:42:53.304 INFO:tasks.workunit.client.0.smithi016.stdout:6/888: getdents d0/d9/d16/d46/dbe 0 2024-04-02T04:42:53.304 INFO:tasks.workunit.client.0.smithi016.stdout:0/841: chown d0/fc8 485 1 2024-04-02T04:42:53.306 INFO:tasks.workunit.client.1.smithi027.stdout:3/268: symlink d1/d1a/l59 0 2024-04-02T04:42:53.307 INFO:tasks.workunit.client.1.smithi027.stdout:5/465: truncate d0/d1a/d20/f73 627255 0 2024-04-02T04:42:53.309 INFO:tasks.workunit.client.0.smithi016.stdout:1/844: sync 2024-04-02T04:42:53.310 INFO:tasks.workunit.client.0.smithi016.stdout:9/957: unlink d0/d9/db/dc9/d53/f9d 0 2024-04-02T04:42:53.311 INFO:tasks.workunit.client.0.smithi016.stdout:1/845: write d14/d19/d1d/d29/d39/d46/dcb/f114 [62211,88528] 0 2024-04-02T04:42:53.313 INFO:tasks.workunit.client.0.smithi016.stdout:2/940: write d12/d35/d3f/d56/dab/f1f [2417818,95454] 0 2024-04-02T04:42:53.313 INFO:tasks.workunit.client.1.smithi027.stdout:4/308: dwrite fc [0,4194304] 0 2024-04-02T04:42:53.316 INFO:tasks.workunit.client.0.smithi016.stdout:6/889: mkdir d0/d9/d16/d56/db0/df2/d119 0 2024-04-02T04:42:53.317 INFO:tasks.workunit.client.1.smithi027.stdout:1/344: link d0/f7 d0/d1e/d2e/d3b/f77 0 2024-04-02T04:42:53.320 INFO:tasks.workunit.client.0.smithi016.stdout:2/941: dwrite d12/d9f/f146 [0,4194304] 0 2024-04-02T04:42:53.331 INFO:tasks.workunit.client.0.smithi016.stdout:6/890: dread d0/d5c/ffb [4194304,4194304] 0 2024-04-02T04:42:53.331 INFO:tasks.workunit.client.0.smithi016.stdout:6/891: stat d0/d9 0 2024-04-02T04:42:53.359 INFO:tasks.workunit.client.0.smithi016.stdout:0/842: chown d0/d16/d24/d4e/d8d/d71/df1/d10e 2099 1 2024-04-02T04:42:53.380 INFO:tasks.workunit.client.1.smithi027.stdout:6/364: sync 2024-04-02T04:42:53.380 INFO:tasks.workunit.client.1.smithi027.stdout:9/343: sync 2024-04-02T04:42:53.419 INFO:tasks.workunit.client.1.smithi027.stdout:7/369: unlink d2/df/f6b 0 2024-04-02T04:42:53.419 INFO:tasks.workunit.client.0.smithi016.stdout:7/863: dwrite d0/d46/d16/d49/fa [0,4194304] 0 2024-04-02T04:42:53.424 INFO:tasks.workunit.client.1.smithi027.stdout:3/269: write d1/f46 [3832370,12920] 0 2024-04-02T04:42:53.427 INFO:tasks.workunit.client.1.smithi027.stdout:5/466: rmdir d0/df/d29/da5/da6 39 2024-04-02T04:42:53.428 INFO:tasks.workunit.client.1.smithi027.stdout:5/467: truncate d0/d1d/f91 943151 0 2024-04-02T04:42:53.436 INFO:tasks.workunit.client.1.smithi027.stdout:8/407: link d5/d8/d15/d71/f72 d5/d8/db/df/f82 0 2024-04-02T04:42:53.437 INFO:tasks.workunit.client.1.smithi027.stdout:8/408: write d5/f7d [617929,32226] 0 2024-04-02T04:42:53.440 INFO:tasks.workunit.client.1.smithi027.stdout:4/309: mknod d24/d38/d44/c65 0 2024-04-02T04:42:53.443 INFO:tasks.workunit.client.1.smithi027.stdout:1/345: creat d0/d1e/d2e/d3b/f78 x:0 0 0 2024-04-02T04:42:53.450 INFO:tasks.workunit.client.1.smithi027.stdout:6/365: creat d13/d1b/d24/d25/d27/f76 x:0 0 0 2024-04-02T04:42:53.450 INFO:tasks.workunit.client.1.smithi027.stdout:1/346: dwrite d0/de/f51 [0,4194304] 0 2024-04-02T04:42:53.454 INFO:tasks.workunit.client.1.smithi027.stdout:0/397: link d4/d16/d1b/f3e d4/d38/f7f 0 2024-04-02T04:42:53.455 INFO:tasks.workunit.client.1.smithi027.stdout:0/398: write d4/d3d/d49/f50 [2180620,10619] 0 2024-04-02T04:42:53.457 INFO:tasks.workunit.client.1.smithi027.stdout:2/374: mknod d1/db/d3b/d61/d75/d81/c86 0 2024-04-02T04:42:53.459 INFO:tasks.workunit.client.1.smithi027.stdout:2/375: dread - d1/db/d3b/d17/d29/f4f zero size 2024-04-02T04:42:53.463 INFO:tasks.workunit.client.1.smithi027.stdout:0/399: dwrite d4/d3d/f3f [0,4194304] 0 2024-04-02T04:42:53.467 INFO:tasks.workunit.client.1.smithi027.stdout:8/409: mknod d5/d8/db/dd/d4a/c83 0 2024-04-02T04:42:53.470 INFO:tasks.workunit.client.1.smithi027.stdout:1/347: fdatasync d0/de/f51 0 2024-04-02T04:42:53.476 INFO:tasks.workunit.client.1.smithi027.stdout:6/366: creat d13/d1b/d24/d28/d3c/f77 x:0 0 0 2024-04-02T04:42:53.482 INFO:tasks.workunit.client.1.smithi027.stdout:2/376: fdatasync d1/db/d3b/d17/d68/f80 0 2024-04-02T04:42:53.486 INFO:tasks.workunit.client.1.smithi027.stdout:5/468: symlink d0/d5/lad 0 2024-04-02T04:42:53.489 INFO:tasks.workunit.client.1.smithi027.stdout:0/400: creat d4/d38/d66/d4a/f80 x:0 0 0 2024-04-02T04:42:53.490 INFO:tasks.workunit.client.1.smithi027.stdout:2/377: dwrite d1/db/d3b/d17/d12/f62 [0,4194304] 0 2024-04-02T04:42:53.492 INFO:tasks.workunit.client.0.smithi016.stdout:3/984: sync 2024-04-02T04:42:53.492 INFO:tasks.workunit.client.0.smithi016.stdout:5/858: sync 2024-04-02T04:42:53.493 INFO:tasks.workunit.client.0.smithi016.stdout:5/859: chown d3/df/d18/d26/d48/lb7 2093863 1 2024-04-02T04:42:53.493 INFO:tasks.workunit.client.0.smithi016.stdout:5/860: stat d3/df/f14 0 2024-04-02T04:42:53.494 INFO:tasks.workunit.client.0.smithi016.stdout:5/861: stat d3/d40/d4f/d67/f78 0 2024-04-02T04:42:53.522 INFO:tasks.workunit.client.1.smithi027.stdout:5/469: getdents d0/df/d11 0 2024-04-02T04:42:53.548 INFO:tasks.workunit.client.0.smithi016.stdout:1/846: rename d14/d19/d50/cfe to d14/d19/d1d/d29/d39/d46/dcb/d43/d68/c115 0 2024-04-02T04:42:53.551 INFO:tasks.workunit.client.0.smithi016.stdout:0/843: getdents d0/d16/d24/d4e/d8d/d71/df1 0 2024-04-02T04:42:53.551 INFO:tasks.workunit.client.0.smithi016.stdout:0/844: dread - d0/d16/d23/d3e/fc6 zero size 2024-04-02T04:42:53.557 INFO:tasks.workunit.client.0.smithi016.stdout:0/845: mkdir d0/d4f/d11b 0 2024-04-02T04:42:53.560 INFO:tasks.workunit.client.0.smithi016.stdout:5/862: rmdir d3/d40/d4f/d67/dbf/d111/d114 0 2024-04-02T04:42:53.565 INFO:tasks.workunit.client.0.smithi016.stdout:5/863: getdents d3/df/d18/d26/d48 0 2024-04-02T04:42:53.567 INFO:tasks.workunit.client.0.smithi016.stdout:5/864: creat d3/d40/db6/f135 x:0 0 0 2024-04-02T04:42:53.568 INFO:tasks.workunit.client.0.smithi016.stdout:5/865: chown d3/d40/d4f/d6b/d45/da6/d102/dd1/f8d 362 1 2024-04-02T04:42:53.582 INFO:tasks.workunit.client.0.smithi016.stdout:8/877: sync 2024-04-02T04:42:53.582 INFO:tasks.workunit.client.0.smithi016.stdout:6/892: sync 2024-04-02T04:42:53.582 INFO:tasks.workunit.client.0.smithi016.stdout:4/864: sync 2024-04-02T04:42:53.583 INFO:tasks.workunit.client.0.smithi016.stdout:9/958: write d0/d4a/d75/f96 [487216,53519] 0 2024-04-02T04:42:53.585 INFO:tasks.workunit.client.0.smithi016.stdout:8/878: write d1/d6/d36/d3f/d40/d45/dcc/ff4 [1834416,88358] 0 2024-04-02T04:42:53.586 INFO:tasks.workunit.client.0.smithi016.stdout:5/866: dread d3/df/d18/d26/f106 [0,4194304] 0 2024-04-02T04:42:53.588 INFO:tasks.workunit.client.0.smithi016.stdout:5/867: chown d3/d40/d4f/d67/dbf/d104/c134 68623 1 2024-04-02T04:42:53.588 INFO:tasks.workunit.client.0.smithi016.stdout:4/865: rmdir d5/d16/df8/d7e 39 2024-04-02T04:42:53.591 INFO:tasks.workunit.client.0.smithi016.stdout:7/864: dwrite d0/f22 [0,4194304] 0 2024-04-02T04:42:53.594 INFO:tasks.workunit.client.0.smithi016.stdout:5/868: mknod d3/d40/d4f/d6b/d45/c136 0 2024-04-02T04:42:53.597 INFO:tasks.workunit.client.0.smithi016.stdout:9/959: dread d0/d4a/fc2 [0,4194304] 0 2024-04-02T04:42:53.609 INFO:tasks.workunit.client.0.smithi016.stdout:0/846: truncate d0/d16/fe8 915546 0 2024-04-02T04:42:53.614 INFO:tasks.workunit.client.1.smithi027.stdout:6/367: write d13/d1b/f1e [1665980,24417] 0 2024-04-02T04:42:53.615 INFO:tasks.workunit.client.0.smithi016.stdout:2/942: mknod d12/d35/d3b/c149 0 2024-04-02T04:42:53.615 INFO:tasks.workunit.client.1.smithi027.stdout:6/368: readlink d13/d1b/d24/d25/l71 0 2024-04-02T04:42:53.615 INFO:tasks.workunit.client.0.smithi016.stdout:0/847: dwrite d0/d16/d24/f7b [4194304,4194304] 0 2024-04-02T04:42:53.618 INFO:tasks.workunit.client.1.smithi027.stdout:1/348: dwrite d0/f37 [0,4194304] 0 2024-04-02T04:42:53.619 INFO:tasks.workunit.client.0.smithi016.stdout:0/848: readlink d0/d16/d24/d62/d9c/l7e 0 2024-04-02T04:42:53.622 INFO:tasks.workunit.client.0.smithi016.stdout:2/943: creat d12/d35/d3f/d2b/de9/d102/f14a x:0 0 0 2024-04-02T04:42:53.625 INFO:tasks.workunit.client.0.smithi016.stdout:0/849: unlink d0/d16/d24/d4e/d8d/d71/df1/fd2 0 2024-04-02T04:42:53.631 INFO:tasks.workunit.client.1.smithi027.stdout:1/349: creat d0/d1e/d2e/d3e/d5f/f79 x:0 0 0 2024-04-02T04:42:53.632 INFO:tasks.workunit.client.0.smithi016.stdout:0/850: mknod d0/d16/d2e/d53/c11c 0 2024-04-02T04:42:53.633 INFO:tasks.workunit.client.1.smithi027.stdout:8/410: rmdir d5/d8/d15 39 2024-04-02T04:42:53.633 INFO:tasks.workunit.client.0.smithi016.stdout:0/851: readlink d0/d16/d24/ld0 0 2024-04-02T04:42:53.634 INFO:tasks.workunit.client.0.smithi016.stdout:0/852: chown d0/d16/d2e 33091673 1 2024-04-02T04:42:53.637 INFO:tasks.workunit.client.1.smithi027.stdout:8/411: creat d5/d8/db/dd/d5f/d27/d31/f84 x:0 0 0 2024-04-02T04:42:53.637 INFO:tasks.workunit.client.1.smithi027.stdout:5/470: write d0/df/d11/d13/f28 [3293251,85020] 0 2024-04-02T04:42:53.637 INFO:tasks.workunit.client.1.smithi027.stdout:1/350: truncate d0/f5 2080828 0 2024-04-02T04:42:53.641 INFO:tasks.workunit.client.1.smithi027.stdout:8/412: readlink d5/d8/d15/l75 0 2024-04-02T04:42:53.645 INFO:tasks.workunit.client.1.smithi027.stdout:5/471: dwrite d0/d1d/f79 [0,4194304] 0 2024-04-02T04:42:53.645 INFO:tasks.workunit.client.1.smithi027.stdout:8/413: dread d5/d8/db/dd/d5f/f7e [0,4194304] 0 2024-04-02T04:42:53.646 INFO:tasks.workunit.client.0.smithi016.stdout:0/853: rename d0/f105 to d0/d16/d24/d62/d9c/f11d 0 2024-04-02T04:42:53.647 INFO:tasks.workunit.client.1.smithi027.stdout:8/414: stat d5/d8/db/dd/d5f/d27/d31/d5b/d6c 0 2024-04-02T04:42:53.647 INFO:tasks.workunit.client.0.smithi016.stdout:0/854: readlink d0/l5 0 2024-04-02T04:42:53.647 INFO:tasks.workunit.client.1.smithi027.stdout:8/415: chown d5/d8/db/dd/d5f/d27/d31/d61 30524308 1 2024-04-02T04:42:53.650 INFO:tasks.workunit.client.1.smithi027.stdout:5/472: mknod d0/df/d11/d13/d9a/cae 0 2024-04-02T04:42:53.650 INFO:tasks.workunit.client.1.smithi027.stdout:6/369: dread d13/f62 [0,4194304] 0 2024-04-02T04:42:53.653 INFO:tasks.workunit.client.1.smithi027.stdout:6/370: creat d13/d1b/d24/d28/d3c/f78 x:0 0 0 2024-04-02T04:42:53.654 INFO:tasks.workunit.client.1.smithi027.stdout:2/378: mknod d1/db/d3b/c87 0 2024-04-02T04:42:53.655 INFO:tasks.workunit.client.1.smithi027.stdout:6/371: write d13/f17 [927365,2675] 0 2024-04-02T04:42:53.656 INFO:tasks.workunit.client.1.smithi027.stdout:8/416: dread d5/d8/db/dd/d5f/d27/d31/f32 [0,4194304] 0 2024-04-02T04:42:53.658 INFO:tasks.workunit.client.1.smithi027.stdout:5/473: dwrite d0/f33 [0,4194304] 0 2024-04-02T04:42:53.658 INFO:tasks.workunit.client.1.smithi027.stdout:8/417: chown d5/d8/c3f 49021704 1 2024-04-02T04:42:53.659 INFO:tasks.workunit.client.1.smithi027.stdout:8/418: chown d5/d8/db/df 60 1 2024-04-02T04:42:53.660 INFO:tasks.workunit.client.0.smithi016.stdout:3/985: mknod d2/d2e/c148 0 2024-04-02T04:42:53.660 INFO:tasks.workunit.client.0.smithi016.stdout:3/986: chown d2/d15/d16/f54 248659 1 2024-04-02T04:42:53.665 INFO:tasks.workunit.client.1.smithi027.stdout:9/344: rename f17 to d1a/d1d/d29/d50/f77 0 2024-04-02T04:42:53.666 INFO:tasks.workunit.client.0.smithi016.stdout:3/987: creat d2/d15/d60/f149 x:0 0 0 2024-04-02T04:42:53.667 INFO:tasks.workunit.client.0.smithi016.stdout:3/988: stat d2/d2e/dee/f12a 0 2024-04-02T04:42:53.669 INFO:tasks.workunit.client.1.smithi027.stdout:5/474: unlink d0/df/d11/d13/f26 0 2024-04-02T04:42:53.671 INFO:tasks.workunit.client.0.smithi016.stdout:3/989: fdatasync d2/d15/d1a/f128 0 2024-04-02T04:42:53.675 INFO:tasks.workunit.client.1.smithi027.stdout:7/370: rename d2/d7/da/dc/d16/d2f/f27 to d2/d1d/f74 0 2024-04-02T04:42:53.676 INFO:tasks.workunit.client.1.smithi027.stdout:7/371: stat d2/d7/da/dc/d16/d3a 0 2024-04-02T04:42:53.678 INFO:tasks.workunit.client.1.smithi027.stdout:7/372: read d2/d7/da/dc/d16/d3a/d53/f5a [1702741,7332] 0 2024-04-02T04:42:53.680 INFO:tasks.workunit.client.0.smithi016.stdout:3/990: read d2/f26 [293589,102314] 0 2024-04-02T04:42:53.682 INFO:tasks.workunit.client.0.smithi016.stdout:3/991: write d2/d15/d70/dcf/f131 [944276,34323] 0 2024-04-02T04:42:53.683 INFO:tasks.workunit.client.0.smithi016.stdout:0/855: dread d0/d16/d2e/d37/d66/faa [0,4194304] 0 2024-04-02T04:42:53.683 INFO:tasks.workunit.client.1.smithi027.stdout:8/419: mknod d5/d8/db/dd/c85 0 2024-04-02T04:42:53.686 INFO:tasks.workunit.client.1.smithi027.stdout:3/270: rename d1/de/d3a/f3c to d1/d1a/f5a 0 2024-04-02T04:42:53.687 INFO:tasks.workunit.client.1.smithi027.stdout:7/373: fsync d2/d7/da/dc/d16/d3f/d44/f6a 0 2024-04-02T04:42:53.689 INFO:tasks.workunit.client.1.smithi027.stdout:7/374: write d2/d7/da/dc/d16/d3a/d4d/f58 [1183817,87028] 0 2024-04-02T04:42:53.690 INFO:tasks.workunit.client.0.smithi016.stdout:1/847: dwrite d14/d19/d1e/d71/f78 [4194304,4194304] 0 2024-04-02T04:42:53.690 INFO:tasks.workunit.client.1.smithi027.stdout:7/375: stat d2/df 0 2024-04-02T04:42:53.692 INFO:tasks.workunit.client.1.smithi027.stdout:7/376: truncate d2/d7/f48 946936 0 2024-04-02T04:42:53.697 INFO:tasks.workunit.client.0.smithi016.stdout:1/848: creat d14/d19/d1e/d5e/d75/f116 x:0 0 0 2024-04-02T04:42:53.701 INFO:tasks.workunit.client.1.smithi027.stdout:7/377: chown d2/d7/da/dc/d1f/l26 911091 1 2024-04-02T04:42:53.703 INFO:tasks.workunit.client.1.smithi027.stdout:4/310: rename d24/d48/f4d to d24/d38/d44/d41/f66 0 2024-04-02T04:42:53.705 INFO:tasks.workunit.client.0.smithi016.stdout:3/992: dread d2/f3e [0,4194304] 0 2024-04-02T04:42:53.707 INFO:tasks.workunit.client.0.smithi016.stdout:1/849: link d14/d19/daf/c105 d14/d19/d1e/d71/de0/c117 0 2024-04-02T04:42:53.710 INFO:tasks.workunit.client.1.smithi027.stdout:7/378: dwrite d2/d1c/f40 [0,4194304] 0 2024-04-02T04:42:53.711 INFO:tasks.workunit.client.1.smithi027.stdout:0/401: rename d4/d3d/d49 to d4/d38/d66/d2d/d81 0 2024-04-02T04:42:53.713 INFO:tasks.workunit.client.1.smithi027.stdout:3/271: creat d1/de/f5b x:0 0 0 2024-04-02T04:42:53.715 INFO:tasks.workunit.client.1.smithi027.stdout:3/272: chown d1/de/d3a/f4c 1 1 2024-04-02T04:42:53.716 INFO:tasks.workunit.client.1.smithi027.stdout:9/345: rename d1a/d1d/d29/l2e to d1a/d1d/d29/d37/d4c/l78 0 2024-04-02T04:42:53.718 INFO:tasks.workunit.client.0.smithi016.stdout:3/993: mknod d2/d2e/d36/d48/c14a 0 2024-04-02T04:42:53.720 INFO:tasks.workunit.client.1.smithi027.stdout:0/402: unlink d4/d38/d66/f3a 0 2024-04-02T04:42:53.721 INFO:tasks.workunit.client.0.smithi016.stdout:1/850: mknod d14/d19/d1e/d71/ddd/c118 0 2024-04-02T04:42:53.721 INFO:tasks.workunit.client.1.smithi027.stdout:0/403: read - d4/d3d/f7c zero size 2024-04-02T04:42:53.723 INFO:tasks.workunit.client.1.smithi027.stdout:3/273: unlink d1/de/df/d13/f1b 0 2024-04-02T04:42:53.724 INFO:tasks.workunit.client.1.smithi027.stdout:9/346: dwrite d1a/d1d/d29/d37/f35 [0,4194304] 0 2024-04-02T04:42:53.726 INFO:tasks.workunit.client.1.smithi027.stdout:8/420: rename d5/d8/db/dd/d5f/d27/d31/d5b/c81 to d5/d8/db/dd/d4a/c86 0 2024-04-02T04:42:53.727 INFO:tasks.workunit.client.0.smithi016.stdout:6/893: dwrite d0/d6e/fa2 [0,4194304] 0 2024-04-02T04:42:53.727 INFO:tasks.workunit.client.1.smithi027.stdout:7/379: getdents d2/d7/da/dc/d1f/d61 0 2024-04-02T04:42:53.727 INFO:tasks.workunit.client.1.smithi027.stdout:8/421: write d5/d8/db/f76 [379532,75118] 0 2024-04-02T04:42:53.731 INFO:tasks.workunit.client.1.smithi027.stdout:0/404: creat d4/d16/f82 x:0 0 0 2024-04-02T04:42:53.732 INFO:tasks.workunit.client.1.smithi027.stdout:3/274: creat d1/de/df/d13/f5c x:0 0 0 2024-04-02T04:42:53.734 INFO:tasks.workunit.client.0.smithi016.stdout:6/894: unlink d0/d9/d16/d56/d89/d67/d6f/d7e/fca 0 2024-04-02T04:42:53.736 INFO:tasks.workunit.client.0.smithi016.stdout:4/866: dwrite d5/f7 [0,4194304] 0 2024-04-02T04:42:53.737 INFO:tasks.workunit.client.0.smithi016.stdout:7/865: write d0/d46/d16/d49/fa7 [209987,72749] 0 2024-04-02T04:42:53.739 INFO:tasks.workunit.client.1.smithi027.stdout:8/422: creat d5/d8/db/dd/d4a/f87 x:0 0 0 2024-04-02T04:42:53.742 INFO:tasks.workunit.client.1.smithi027.stdout:3/275: rename d1/de/df/f1f to d1/de/df/f5d 0 2024-04-02T04:42:53.745 INFO:tasks.workunit.client.0.smithi016.stdout:5/869: dwrite d3/f10 [0,4194304] 0 2024-04-02T04:42:53.747 INFO:tasks.workunit.client.0.smithi016.stdout:1/851: getdents d14/d19/d1d/d45/d4f 0 2024-04-02T04:42:53.749 INFO:tasks.workunit.client.0.smithi016.stdout:1/852: chown d14/d19/d1d/d29/d39/d46/dcb/d43/f96 16514373 1 2024-04-02T04:42:53.750 INFO:tasks.workunit.client.0.smithi016.stdout:1/853: stat d14/d19/d1d/d45/d4f/d86 0 2024-04-02T04:42:53.750 INFO:tasks.workunit.client.0.smithi016.stdout:4/867: rmdir d5/d1d/d5f/d95/df2 39 2024-04-02T04:42:53.754 INFO:tasks.workunit.client.0.smithi016.stdout:7/866: readlink d0/d46/da2/lee 0 2024-04-02T04:42:53.755 INFO:tasks.workunit.client.0.smithi016.stdout:7/867: fdatasync d0/d87/fb1 0 2024-04-02T04:42:53.755 INFO:tasks.workunit.client.1.smithi027.stdout:3/276: mknod d1/de/df/c5e 0 2024-04-02T04:42:53.756 INFO:tasks.workunit.client.0.smithi016.stdout:4/868: dread d5/d17/d18/d1f/f53 [0,4194304] 0 2024-04-02T04:42:53.756 INFO:tasks.workunit.client.1.smithi027.stdout:8/423: dwrite d5/d8/db/dd/d5f/f14 [0,4194304] 0 2024-04-02T04:42:53.758 INFO:tasks.workunit.client.1.smithi027.stdout:8/424: write d5/d8/db/d57/f74 [1030268,24179] 0 2024-04-02T04:42:53.759 INFO:tasks.workunit.client.0.smithi016.stdout:2/944: dwrite d12/d35/d101/d76/da0/de7/f10f [0,4194304] 0 2024-04-02T04:42:53.763 INFO:tasks.workunit.client.1.smithi027.stdout:3/277: rename d1/d1a/f22 to d1/de/d55/f5f 0 2024-04-02T04:42:53.766 INFO:tasks.workunit.client.1.smithi027.stdout:8/425: write d5/d8/db/dd/d5f/d27/f3a [7073100,68006] 0 2024-04-02T04:42:53.771 INFO:tasks.workunit.client.0.smithi016.stdout:1/854: read d14/d19/d1e/d6d/f8b [1826829,44888] 0 2024-04-02T04:42:53.774 INFO:tasks.workunit.client.1.smithi027.stdout:1/351: dwrite d0/d1e/d25/d3c/f48 [0,4194304] 0 2024-04-02T04:42:53.774 INFO:tasks.workunit.client.1.smithi027.stdout:3/278: symlink d1/de/d55/l60 0 2024-04-02T04:42:53.784 INFO:tasks.workunit.client.0.smithi016.stdout:0/856: dwrite d0/d16/d2e/d53/d9d/f109 [0,4194304] 0 2024-04-02T04:42:53.784 INFO:tasks.workunit.client.0.smithi016.stdout:0/857: write d0/f104 [163734,78360] 0 2024-04-02T04:42:53.789 INFO:tasks.workunit.client.1.smithi027.stdout:2/379: dwrite d1/db/d3b/d17/d12/f38 [4194304,4194304] 0 2024-04-02T04:42:53.791 INFO:tasks.workunit.client.1.smithi027.stdout:6/372: dwrite d13/f14 [0,4194304] 0 2024-04-02T04:42:53.792 INFO:tasks.workunit.client.0.smithi016.stdout:7/868: rename d0/d46/da2/dc8/dda/ffa to d0/d46/da2/dc8/dda/de3/f127 0 2024-04-02T04:42:53.794 INFO:tasks.workunit.client.1.smithi027.stdout:1/352: dwrite d0/f16 [0,4194304] 0 2024-04-02T04:42:53.794 INFO:tasks.workunit.client.0.smithi016.stdout:8/879: sync 2024-04-02T04:42:53.803 INFO:tasks.workunit.client.1.smithi027.stdout:8/426: mkdir d5/d8/db/dd/d5f/d88 0 2024-04-02T04:42:53.803 INFO:tasks.workunit.client.1.smithi027.stdout:5/475: write d0/df/d11/d41/f90 [623960,26723] 0 2024-04-02T04:42:53.803 INFO:tasks.workunit.client.0.smithi016.stdout:5/870: symlink d3/d40/db6/de0/d115/l137 0 2024-04-02T04:42:53.803 INFO:tasks.workunit.client.1.smithi027.stdout:6/373: chown d13/d1b/d24/d25/d42/c4e 19 1 2024-04-02T04:42:53.806 INFO:tasks.workunit.client.1.smithi027.stdout:3/279: truncate d1/f7 3492576 0 2024-04-02T04:42:53.807 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:53 smithi016 ceph-mon[28969]: pgmap v25: 65 pgs: 65 active+clean; 2.2 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 49 MiB/s rd, 210 MiB/s wr, 356 op/s 2024-04-02T04:42:53.807 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:53 smithi016 ceph-mon[28969]: Upgrade: Updating mgr.smithi016.eqnebx 2024-04-02T04:42:53.807 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:53 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:53.807 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:53 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi016.eqnebx", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T04:42:53.807 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:53 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T04:42:53.807 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:53 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:42:53.807 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:53 smithi016 ceph-mon[28969]: Deploying daemon mgr.smithi016.eqnebx on smithi016 2024-04-02T04:42:53.807 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:53 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:42:53.808 INFO:tasks.workunit.client.1.smithi027.stdout:2/380: truncate d1/db/d3b/d17/f2c 1055300 0 2024-04-02T04:42:53.808 INFO:tasks.workunit.client.1.smithi027.stdout:1/353: dwrite d0/de/f51 [0,4194304] 0 2024-04-02T04:42:53.808 INFO:tasks.workunit.client.1.smithi027.stdout:8/427: truncate d5/d8/f33 5055245 0 2024-04-02T04:42:53.809 INFO:tasks.workunit.client.0.smithi016.stdout:1/855: truncate d14/d19/d1d/d29/f51 7561106 0 2024-04-02T04:42:53.810 INFO:tasks.workunit.client.1.smithi027.stdout:3/280: truncate d1/de/f58 4902769 0 2024-04-02T04:42:53.813 INFO:tasks.workunit.client.1.smithi027.stdout:5/476: creat d0/d1d/d61/d7e/faf x:0 0 0 2024-04-02T04:42:53.814 INFO:tasks.workunit.client.1.smithi027.stdout:5/477: fdatasync d0/d1d/d61/f96 0 2024-04-02T04:42:53.817 INFO:tasks.workunit.client.0.smithi016.stdout:6/895: getdents d0 0 2024-04-02T04:42:53.822 INFO:tasks.workunit.client.1.smithi027.stdout:3/281: unlink d1/d1a/f4a 0 2024-04-02T04:42:53.825 INFO:tasks.workunit.client.0.smithi016.stdout:7/869: truncate d0/d46/d16/d49/f3e 4440072 0 2024-04-02T04:42:53.831 INFO:tasks.workunit.client.0.smithi016.stdout:2/945: creat d12/d9f/d107/d126/f14b x:0 0 0 2024-04-02T04:42:53.832 INFO:tasks.workunit.client.0.smithi016.stdout:2/946: chown d12/d35/d3f/cfb 1292765 1 2024-04-02T04:42:53.836 INFO:tasks.workunit.client.0.smithi016.stdout:5/871: dread d3/df/d18/f83 [0,4194304] 0 2024-04-02T04:42:53.836 INFO:tasks.workunit.client.0.smithi016.stdout:6/896: rename d0/d9/d16/d46/dbe/cee to d0/d9/d16/d46/dbe/c11a 0 2024-04-02T04:42:53.840 INFO:tasks.workunit.client.0.smithi016.stdout:1/856: dread d14/d19/d1d/d29/f3c [0,4194304] 0 2024-04-02T04:42:53.840 INFO:tasks.workunit.client.1.smithi027.stdout:9/347: truncate d1a/d1d/d29/d37/d4c/f62 413157 0 2024-04-02T04:42:53.841 INFO:tasks.workunit.client.0.smithi016.stdout:1/857: chown d14/d19/d1e/d71/dc0/fc2 1 1 2024-04-02T04:42:53.841 INFO:tasks.workunit.client.1.smithi027.stdout:9/348: fsync d1a/d1d/d29/d37/f35 0 2024-04-02T04:42:53.842 INFO:tasks.workunit.client.0.smithi016.stdout:8/880: creat d1/d6/d36/d8e/f11e x:0 0 0 2024-04-02T04:42:53.844 INFO:tasks.workunit.client.1.smithi027.stdout:3/282: rename d1/de/d3a to d1/de/df/d61 0 2024-04-02T04:42:53.846 INFO:tasks.workunit.client.1.smithi027.stdout:8/428: getdents d5/d8/db/dd/d5f/d27/d31/d61 0 2024-04-02T04:42:53.847 INFO:tasks.workunit.client.0.smithi016.stdout:1/858: dwrite d14/d19/d1e/d71/de0/fea [0,4194304] 0 2024-04-02T04:42:53.849 INFO:tasks.workunit.client.0.smithi016.stdout:1/859: write d14/d19/d1e/d71/f104 [800080,72175] 0 2024-04-02T04:42:53.849 INFO:tasks.workunit.client.1.smithi027.stdout:8/429: dread d5/d8/d15/d71/f72 [0,4194304] 0 2024-04-02T04:42:53.853 INFO:tasks.workunit.client.1.smithi027.stdout:1/354: creat d0/d1e/d2e/f7a x:0 0 0 2024-04-02T04:42:53.854 INFO:tasks.workunit.client.1.smithi027.stdout:1/355: chown d0/d1e/d25/d2b/c32 832213947 1 2024-04-02T04:42:53.855 INFO:tasks.workunit.client.0.smithi016.stdout:6/897: mkdir d0/d6e/d11b 0 2024-04-02T04:42:53.858 INFO:tasks.workunit.client.1.smithi027.stdout:6/374: link d13/d1b/d24/d25/d56/l68 d13/d1b/d24/d25/d27/l79 0 2024-04-02T04:42:53.861 INFO:tasks.workunit.client.0.smithi016.stdout:4/869: dwrite d5/d8/d43/f4b [0,4194304] 0 2024-04-02T04:42:53.863 INFO:tasks.workunit.client.1.smithi027.stdout:6/375: dwrite fc [0,4194304] 0 2024-04-02T04:42:53.869 INFO:tasks.workunit.client.0.smithi016.stdout:2/947: creat d12/d35/d3b/d42/d59/d104/d12f/f14c x:0 0 0 2024-04-02T04:42:53.874 INFO:tasks.workunit.client.1.smithi027.stdout:3/283: dwrite d1/de/df/f19 [0,4194304] 0 2024-04-02T04:42:53.878 INFO:tasks.workunit.client.0.smithi016.stdout:5/872: mkdir d3/d40/d4f/d6b/d45/da6/d102/dc4/dd7/d117/d138 0 2024-04-02T04:42:53.879 INFO:tasks.workunit.client.1.smithi027.stdout:8/430: creat d5/d8/db/dd/d5f/d27/d31/f89 x:0 0 0 2024-04-02T04:42:53.881 INFO:tasks.workunit.client.1.smithi027.stdout:5/478: creat d0/fb0 x:0 0 0 2024-04-02T04:42:53.888 INFO:tasks.workunit.client.1.smithi027.stdout:3/284: fdatasync d1/de/df/d61/f3d 0 2024-04-02T04:42:53.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:53 smithi027 ceph-mon[37550]: pgmap v25: 65 pgs: 65 active+clean; 2.2 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 49 MiB/s rd, 210 MiB/s wr, 356 op/s 2024-04-02T04:42:53.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:53 smithi027 ceph-mon[37550]: Upgrade: Updating mgr.smithi016.eqnebx 2024-04-02T04:42:53.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:53 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:42:53.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:53 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi016.eqnebx", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T04:42:53.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:53 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T04:42:53.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:53 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:42:53.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:53 smithi027 ceph-mon[37550]: Deploying daemon mgr.smithi016.eqnebx on smithi016 2024-04-02T04:42:53.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:53 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:42:53.896 INFO:tasks.workunit.client.1.smithi027.stdout:9/349: rename d1a/d1d/d29/d37/f35 to d1a/d1d/d29/f79 0 2024-04-02T04:42:53.897 INFO:tasks.workunit.client.1.smithi027.stdout:5/479: dwrite d0/df/d29/f7a [0,4194304] 0 2024-04-02T04:42:53.900 INFO:tasks.workunit.client.1.smithi027.stdout:3/285: symlink d1/d1a/l62 0 2024-04-02T04:42:53.900 INFO:tasks.workunit.client.1.smithi027.stdout:3/286: chown d1/de/l47 0 1 2024-04-02T04:42:53.909 INFO:tasks.workunit.client.1.smithi027.stdout:1/356: rename d0/de/f4d to d0/d1e/d25/d2b/f7b 0 2024-04-02T04:42:53.910 INFO:tasks.workunit.client.1.smithi027.stdout:5/480: creat d0/d68/d6b/d74/fb1 x:0 0 0 2024-04-02T04:42:53.916 INFO:tasks.workunit.client.1.smithi027.stdout:1/357: rmdir d0/d1e/d25 39 2024-04-02T04:42:53.917 INFO:tasks.workunit.client.1.smithi027.stdout:5/481: dwrite d0/df/d11/d13/f1e [0,4194304] 0 2024-04-02T04:42:53.919 INFO:tasks.workunit.client.1.smithi027.stdout:3/287: dwrite d1/de/f38 [0,4194304] 0 2024-04-02T04:42:53.922 INFO:tasks.workunit.client.0.smithi016.stdout:9/960: sync 2024-04-02T04:42:53.923 INFO:tasks.workunit.client.0.smithi016.stdout:6/898: creat d0/d9/d16/d56/d89/d67/d6f/f11c x:0 0 0 2024-04-02T04:42:53.924 INFO:tasks.workunit.client.1.smithi027.stdout:5/482: rmdir d0/d1d/d61/d7e 39 2024-04-02T04:42:53.925 INFO:tasks.workunit.client.0.smithi016.stdout:4/870: creat d5/d1d/d5f/f10e x:0 0 0 2024-04-02T04:42:53.927 INFO:tasks.workunit.client.1.smithi027.stdout:5/483: chown d0/df/d11/d13/d9a/d9c/fa8 18865 1 2024-04-02T04:42:53.930 INFO:tasks.workunit.client.0.smithi016.stdout:7/870: dwrite d0/d46/d37/dbe/dae/df8/d84/d8a/f103 [0,4194304] 0 2024-04-02T04:42:53.931 INFO:tasks.workunit.client.1.smithi027.stdout:1/358: dwrite d0/d1e/d25/f31 [4194304,4194304] 0 2024-04-02T04:42:53.931 INFO:tasks.workunit.client.0.smithi016.stdout:5/873: link d3/df/d18/d26/cc9 d3/df/d18/dfd/d122/c139 0 2024-04-02T04:42:53.932 INFO:tasks.workunit.client.1.smithi027.stdout:3/288: dread d1/de/df/f19 [0,4194304] 0 2024-04-02T04:42:53.934 INFO:tasks.workunit.client.1.smithi027.stdout:1/359: write d0/d1e/d25/d2b/d59/f5e [737542,7482] 0 2024-04-02T04:42:53.934 INFO:tasks.workunit.client.1.smithi027.stdout:5/484: chown d0/df/d11/c50 139239244 1 2024-04-02T04:42:53.936 INFO:tasks.workunit.client.0.smithi016.stdout:8/881: dwrite d1/d6/d36/d8e/ff6 [0,4194304] 0 2024-04-02T04:42:53.938 INFO:tasks.workunit.client.0.smithi016.stdout:2/948: dwrite d12/d35/d101/d76/da0/dbe/fc5 [0,4194304] 0 2024-04-02T04:42:53.940 INFO:tasks.workunit.client.0.smithi016.stdout:8/882: truncate d1/d6/d36/d3f/f112 471864 0 2024-04-02T04:42:53.943 INFO:tasks.workunit.client.1.smithi027.stdout:5/485: dwrite d0/df/d11/d13/f1e [4194304,4194304] 0 2024-04-02T04:42:53.966 INFO:tasks.workunit.client.1.smithi027.stdout:3/289: mkdir d1/de/d55/d63 0 2024-04-02T04:42:53.967 INFO:tasks.workunit.client.1.smithi027.stdout:9/350: write d1a/f57 [846015,60298] 0 2024-04-02T04:42:53.967 INFO:tasks.workunit.client.0.smithi016.stdout:9/961: fdatasync d0/d9/db/dc9/d53/fdb 0 2024-04-02T04:42:53.968 INFO:tasks.workunit.client.1.smithi027.stdout:9/351: write d1a/d1d/d29/f46 [1901963,26865] 0 2024-04-02T04:42:53.975 INFO:tasks.workunit.client.1.smithi027.stdout:5/486: rename d0/df/d11/c46 to d0/df/d2f/d3b/cb2 0 2024-04-02T04:42:53.978 INFO:tasks.workunit.client.0.smithi016.stdout:4/871: truncate d5/d16/d3f/f64 4635881 0 2024-04-02T04:42:53.983 INFO:tasks.workunit.client.1.smithi027.stdout:5/487: dread - d0/d5/d55/f8a zero size 2024-04-02T04:42:53.984 INFO:tasks.workunit.client.1.smithi027.stdout:1/360: write d0/de/f1b [206796,91070] 0 2024-04-02T04:42:53.988 INFO:tasks.workunit.client.0.smithi016.stdout:6/899: mknod d0/d9/d16/d56/d89/c11d 0 2024-04-02T04:42:53.990 INFO:tasks.workunit.client.1.smithi027.stdout:9/352: write d1a/d1d/d29/d50/f77 [166951,31990] 0 2024-04-02T04:42:53.991 INFO:tasks.workunit.client.1.smithi027.stdout:1/361: dwrite d0/de/ff [4194304,4194304] 0 2024-04-02T04:42:53.991 INFO:tasks.workunit.client.1.smithi027.stdout:9/353: stat d1a/f38 0 2024-04-02T04:42:53.992 INFO:tasks.workunit.client.0.smithi016.stdout:7/871: dread d0/d46/d37/dbe/dae/df8/d43/d96/fa5 [0,4194304] 0 2024-04-02T04:42:53.992 INFO:tasks.workunit.client.1.smithi027.stdout:9/354: chown d1a/d1d/d29/d37/d3a 119 1 2024-04-02T04:42:53.993 INFO:tasks.workunit.client.1.smithi027.stdout:3/290: creat d1/f64 x:0 0 0 2024-04-02T04:42:53.996 INFO:tasks.workunit.client.1.smithi027.stdout:5/488: symlink d0/lb3 0 2024-04-02T04:42:53.997 INFO:tasks.workunit.client.1.smithi027.stdout:1/362: dread d0/de/f1b [0,4194304] 0 2024-04-02T04:42:53.997 INFO:tasks.workunit.client.1.smithi027.stdout:5/489: stat d0/df/d11/d37/l7d 0 2024-04-02T04:42:53.999 INFO:tasks.workunit.client.1.smithi027.stdout:1/363: read d0/fd [2952484,31661] 0 2024-04-02T04:42:53.999 INFO:tasks.workunit.client.1.smithi027.stdout:5/490: write d0/df/d2f/d3b/f5b [2562707,129949] 0 2024-04-02T04:42:54.000 INFO:tasks.workunit.client.1.smithi027.stdout:1/364: stat d0/de/c53 0 2024-04-02T04:42:54.001 INFO:tasks.workunit.client.0.smithi016.stdout:5/874: write d3/df/d18/f31 [1812497,20291] 0 2024-04-02T04:42:54.002 INFO:tasks.workunit.client.0.smithi016.stdout:8/883: dread d1/d31/d67/fd1 [0,4194304] 0 2024-04-02T04:42:54.002 INFO:tasks.workunit.client.1.smithi027.stdout:1/365: write d0/f37 [2183148,14409] 0 2024-04-02T04:42:54.003 INFO:tasks.workunit.client.0.smithi016.stdout:5/875: chown d3/d40/d4f/d6b/fa2 77 1 2024-04-02T04:42:54.004 INFO:tasks.workunit.client.0.smithi016.stdout:8/884: write d1/d6/d36/d3f/d40/d45/fee [1272779,99677] 0 2024-04-02T04:42:54.006 INFO:tasks.workunit.client.0.smithi016.stdout:6/900: creat d0/d9/d16/d56/db0/df2/f11e x:0 0 0 2024-04-02T04:42:54.007 INFO:tasks.workunit.client.1.smithi027.stdout:9/355: write f15 [4918977,19874] 0 2024-04-02T04:42:54.008 INFO:tasks.workunit.client.0.smithi016.stdout:6/901: readlink d0/d9/d16/d56/d89/dc5/dce/lf1 0 2024-04-02T04:42:54.009 INFO:tasks.workunit.client.0.smithi016.stdout:2/949: dwrite d12/d70/f75 [0,4194304] 0 2024-04-02T04:42:54.009 INFO:tasks.workunit.client.0.smithi016.stdout:7/872: creat d0/d46/da2/f128 x:0 0 0 2024-04-02T04:42:54.010 INFO:tasks.workunit.client.1.smithi027.stdout:5/491: creat d0/d1a/d20/fb4 x:0 0 0 2024-04-02T04:42:54.010 INFO:tasks.workunit.client.0.smithi016.stdout:6/902: write d0/d78/fa0 [680972,19326] 0 2024-04-02T04:42:54.013 INFO:tasks.workunit.client.1.smithi027.stdout:3/291: rename d1/ca to d1/d1a/c65 0 2024-04-02T04:42:54.016 INFO:tasks.workunit.client.0.smithi016.stdout:2/950: dwrite d12/d35/d3b/d42/d59/f88 [0,4194304] 0 2024-04-02T04:42:54.016 INFO:tasks.workunit.client.0.smithi016.stdout:8/885: mknod d1/d6/d36/d3f/d77/c11f 0 2024-04-02T04:42:54.016 INFO:tasks.workunit.client.0.smithi016.stdout:8/886: stat d1/d6/f2e 0 2024-04-02T04:42:54.020 INFO:tasks.workunit.client.0.smithi016.stdout:4/872: rename d5/d16/d88/f8c to d5/d17/d18/f10f 0 2024-04-02T04:42:54.023 INFO:tasks.workunit.client.1.smithi027.stdout:3/292: chown d1/d1a/c29 4913 1 2024-04-02T04:42:54.027 INFO:tasks.workunit.client.1.smithi027.stdout:5/492: rename d0/d1d/d61/d72/f86 to d0/d1a/d20/fb5 0 2024-04-02T04:42:54.030 INFO:tasks.workunit.client.1.smithi027.stdout:9/356: creat d1a/d1d/d29/d50/d73/f7a x:0 0 0 2024-04-02T04:42:54.033 INFO:tasks.workunit.client.1.smithi027.stdout:3/293: mknod d1/d1a/c66 0 2024-04-02T04:42:54.035 INFO:tasks.workunit.client.1.smithi027.stdout:3/294: dread d1/de/df/f17 [4194304,4194304] 0 2024-04-02T04:42:54.038 INFO:tasks.workunit.client.0.smithi016.stdout:7/873: rename d0/d46/f30 to d0/d46/d52/dd5/f129 0 2024-04-02T04:42:54.039 INFO:tasks.workunit.client.1.smithi027.stdout:5/493: rename d0/df/d2f/d3b/f64 to d0/d68/d6b/d74/fb6 0 2024-04-02T04:42:54.042 INFO:tasks.workunit.client.1.smithi027.stdout:9/357: creat d1a/d1d/d29/d50/d73/f7b x:0 0 0 2024-04-02T04:42:54.044 INFO:tasks.workunit.client.0.smithi016.stdout:7/874: dwrite d0/d46/d16/dc2/ff2 [0,4194304] 0 2024-04-02T04:42:54.045 INFO:tasks.workunit.client.0.smithi016.stdout:5/876: link d3/df/d18/ce8 d3/d40/db6/c13a 0 2024-04-02T04:42:54.045 INFO:tasks.workunit.client.1.smithi027.stdout:1/366: getdents d0/d1e/d25/d2b 0 2024-04-02T04:42:54.046 INFO:tasks.workunit.client.0.smithi016.stdout:5/877: stat d3/df/f33 0 2024-04-02T04:42:54.048 INFO:tasks.workunit.client.1.smithi027.stdout:3/295: read - d1/f2e zero size 2024-04-02T04:42:54.050 INFO:tasks.workunit.client.0.smithi016.stdout:8/887: dwrite d1/d6/d36/d3f/d40/d49/fab [4194304,4194304] 0 2024-04-02T04:42:54.050 INFO:tasks.workunit.client.0.smithi016.stdout:4/873: write d5/d16/d88/f96 [1853355,107495] 0 2024-04-02T04:42:54.052 INFO:tasks.workunit.client.0.smithi016.stdout:4/874: write d5/d1d/d5f/d95/fdf [4828900,61375] 0 2024-04-02T04:42:54.054 INFO:tasks.workunit.client.0.smithi016.stdout:6/903: creat d0/d9/d16/d56/d89/d67/f11f x:0 0 0 2024-04-02T04:42:54.055 INFO:tasks.workunit.client.1.smithi027.stdout:9/358: write d1a/f1f [864421,101256] 0 2024-04-02T04:42:54.055 INFO:tasks.workunit.client.0.smithi016.stdout:6/904: chown d0/d9/d16/c22 0 1 2024-04-02T04:42:54.056 INFO:tasks.workunit.client.0.smithi016.stdout:6/905: chown d0/d9/d16/d56/d89/c2f 29409006 1 2024-04-02T04:42:54.057 INFO:tasks.workunit.client.1.smithi027.stdout:9/359: write d1a/d1d/d29/d37/d3a/f54 [8321308,63499] 0 2024-04-02T04:42:54.058 INFO:tasks.workunit.client.1.smithi027.stdout:1/367: mknod d0/de/c7c 0 2024-04-02T04:42:54.062 INFO:tasks.workunit.client.0.smithi016.stdout:4/875: fsync d5/d17/d18/d1f/f40 0 2024-04-02T04:42:54.065 INFO:tasks.workunit.client.1.smithi027.stdout:9/360: symlink d1a/d1d/d29/d37/d4c/l7c 0 2024-04-02T04:42:54.067 INFO:tasks.workunit.client.1.smithi027.stdout:3/296: dwrite d1/d1a/f5a [0,4194304] 0 2024-04-02T04:42:54.068 INFO:tasks.workunit.client.1.smithi027.stdout:1/368: rename d0/d1e/c63 to d0/d1e/d2e/d3e/d5f/c7d 0 2024-04-02T04:42:54.070 INFO:tasks.workunit.client.1.smithi027.stdout:1/369: write d0/d1e/d2e/f4a [308859,81555] 0 2024-04-02T04:42:54.071 INFO:tasks.workunit.client.0.smithi016.stdout:8/888: mkdir d1/d6/d36/d3f/d120 0 2024-04-02T04:42:54.072 INFO:tasks.workunit.client.1.smithi027.stdout:5/494: dread d0/d5/f2c [0,4194304] 0 2024-04-02T04:42:54.072 INFO:tasks.workunit.client.0.smithi016.stdout:3/994: sync 2024-04-02T04:42:54.073 INFO:tasks.workunit.client.1.smithi027.stdout:5/495: fsync d0/d1d/f97 0 2024-04-02T04:42:54.074 INFO:tasks.workunit.client.1.smithi027.stdout:5/496: fsync d0/d1d/d61/d72/f81 0 2024-04-02T04:42:54.075 INFO:tasks.workunit.client.0.smithi016.stdout:4/876: symlink d5/d1d/d2d/l110 0 2024-04-02T04:42:54.075 INFO:tasks.workunit.client.0.smithi016.stdout:5/878: dwrite d3/df/d18/fc3 [0,4194304] 0 2024-04-02T04:42:54.077 INFO:tasks.workunit.client.0.smithi016.stdout:6/906: mknod d0/d5c/df5/c120 0 2024-04-02T04:42:54.078 INFO:tasks.workunit.client.1.smithi027.stdout:1/370: mknod d0/d1e/d25/d2b/c7e 0 2024-04-02T04:42:54.081 INFO:tasks.workunit.client.0.smithi016.stdout:8/889: creat d1/d6/d36/d4f/dc9/dfa/f121 x:0 0 0 2024-04-02T04:42:54.082 INFO:tasks.workunit.client.1.smithi027.stdout:9/361: symlink d1a/d1d/d29/d37/l7d 0 2024-04-02T04:42:54.085 INFO:tasks.workunit.client.1.smithi027.stdout:5/497: dwrite d0/df/d11/d13/f98 [0,4194304] 0 2024-04-02T04:42:54.087 INFO:tasks.workunit.client.1.smithi027.stdout:5/498: chown d0/df/d2f/d3b 17 1 2024-04-02T04:42:54.088 INFO:tasks.workunit.client.1.smithi027.stdout:5/499: chown d0/df/d2f/d3b/d51/f53 523136874 1 2024-04-02T04:42:54.090 INFO:tasks.workunit.client.0.smithi016.stdout:5/879: fdatasync d3/df/d18/fba 0 2024-04-02T04:42:54.093 INFO:tasks.workunit.client.1.smithi027.stdout:1/371: symlink d0/d1e/d25/d3c/d64/l7f 0 2024-04-02T04:42:54.094 INFO:tasks.workunit.client.1.smithi027.stdout:1/372: chown d0/f1d 707916 1 2024-04-02T04:42:54.101 INFO:tasks.workunit.client.1.smithi027.stdout:9/362: dread d1a/d1d/d29/d37/f1c [4194304,4194304] 0 2024-04-02T04:42:54.102 INFO:tasks.workunit.client.1.smithi027.stdout:1/373: dwrite d0/d1e/d2e/d3b/d4f/f62 [0,4194304] 0 2024-04-02T04:42:54.103 INFO:tasks.workunit.client.1.smithi027.stdout:9/363: chown d1a/d1d/d29/d37/d20 10 1 2024-04-02T04:42:54.103 INFO:tasks.workunit.client.1.smithi027.stdout:5/500: mkdir d0/df/d11/d13/d9a/db7 0 2024-04-02T04:42:54.105 INFO:tasks.workunit.client.1.smithi027.stdout:5/501: fdatasync d0/d68/d6b/d74/fb1 0 2024-04-02T04:42:54.106 INFO:tasks.workunit.client.1.smithi027.stdout:9/364: dread d1a/d1d/f42 [0,4194304] 0 2024-04-02T04:42:54.107 INFO:tasks.workunit.client.1.smithi027.stdout:1/374: creat d0/de/f80 x:0 0 0 2024-04-02T04:42:54.109 INFO:tasks.workunit.client.0.smithi016.stdout:3/995: getdents d2/d15/d1a/d45/da5 0 2024-04-02T04:42:54.114 INFO:tasks.workunit.client.1.smithi027.stdout:3/297: write d1/f32 [1901519,63656] 0 2024-04-02T04:42:54.114 INFO:tasks.workunit.client.0.smithi016.stdout:8/890: creat d1/d6/d36/d3f/d40/d63/f122 x:0 0 0 2024-04-02T04:42:54.119 INFO:tasks.workunit.client.0.smithi016.stdout:5/880: creat d3/df/d18/f13b x:0 0 0 2024-04-02T04:42:54.119 INFO:tasks.workunit.client.1.smithi027.stdout:9/365: dwrite d1a/d1d/d29/d37/f5a [0,4194304] 0 2024-04-02T04:42:54.119 INFO:tasks.workunit.client.1.smithi027.stdout:1/375: dwrite d0/de/f51 [4194304,4194304] 0 2024-04-02T04:42:54.121 INFO:tasks.workunit.client.0.smithi016.stdout:3/996: mknod d2/d15/d70/dcf/c14b 0 2024-04-02T04:42:54.124 INFO:tasks.workunit.client.1.smithi027.stdout:3/298: stat d1/cb 0 2024-04-02T04:42:54.127 INFO:tasks.workunit.client.1.smithi027.stdout:9/366: mkdir d1a/d1d/d29/d37/d20/d7e 0 2024-04-02T04:42:54.128 INFO:tasks.workunit.client.0.smithi016.stdout:4/877: link d5/d17/d18/d1f/f98 d5/d1d/d1e/da5/df9/f111 0 2024-04-02T04:42:54.129 INFO:tasks.workunit.client.1.smithi027.stdout:9/367: readlink d1a/d1d/d29/d37/d20/l28 0 2024-04-02T04:42:54.129 INFO:tasks.workunit.client.1.smithi027.stdout:3/299: chown d1/f7 3654 1 2024-04-02T04:42:54.131 INFO:tasks.workunit.client.1.smithi027.stdout:3/300: creat d1/de/df/f67 x:0 0 0 2024-04-02T04:42:54.132 INFO:tasks.workunit.client.0.smithi016.stdout:4/878: symlink d5/d8/l112 0 2024-04-02T04:42:54.133 INFO:tasks.workunit.client.1.smithi027.stdout:9/368: symlink d1a/l7f 0 2024-04-02T04:42:54.136 INFO:tasks.workunit.client.0.smithi016.stdout:1/860: sync 2024-04-02T04:42:54.136 INFO:tasks.workunit.client.0.smithi016.stdout:0/858: sync 2024-04-02T04:42:54.137 INFO:tasks.workunit.client.0.smithi016.stdout:1/861: fsync d14/d19/d1d/f30 0 2024-04-02T04:42:54.139 INFO:tasks.workunit.client.0.smithi016.stdout:4/879: unlink d5/d17/d18/d1f/l3e 0 2024-04-02T04:42:54.140 INFO:tasks.workunit.client.0.smithi016.stdout:3/997: link d2/d15/d1a/c11e d2/d2e/d36/d66/d90/dfb/c14c 0 2024-04-02T04:42:54.142 INFO:tasks.workunit.client.1.smithi027.stdout:9/369: mknod d1a/d1d/d3c/d71/c80 0 2024-04-02T04:42:54.142 INFO:tasks.workunit.client.0.smithi016.stdout:8/891: write d1/d6/d36/d3f/d40/d49/f43 [4768555,47306] 0 2024-04-02T04:42:54.142 INFO:tasks.workunit.client.0.smithi016.stdout:0/859: mkdir d0/d16/d11e 0 2024-04-02T04:42:54.144 INFO:tasks.workunit.client.1.smithi027.stdout:9/370: truncate d1a/d1d/f24 4468314 0 2024-04-02T04:42:54.144 INFO:tasks.workunit.client.0.smithi016.stdout:0/860: read d0/d16/d2e/d53/d9d/f109 [760347,83347] 0 2024-04-02T04:42:54.149 INFO:tasks.workunit.client.1.smithi027.stdout:3/301: getdents d1 0 2024-04-02T04:42:54.151 INFO:tasks.workunit.client.1.smithi027.stdout:9/371: symlink d1a/l81 0 2024-04-02T04:42:54.152 INFO:tasks.workunit.client.1.smithi027.stdout:9/372: stat d1a/d1d/d29/d37/f1c 0 2024-04-02T04:42:54.156 INFO:tasks.workunit.client.1.smithi027.stdout:3/302: dwrite d1/de/df/d13/f34 [4194304,4194304] 0 2024-04-02T04:42:54.157 INFO:tasks.workunit.client.1.smithi027.stdout:9/373: creat d1a/d1d/d29/d51/f82 x:0 0 0 2024-04-02T04:42:54.157 INFO:tasks.workunit.client.1.smithi027.stdout:3/303: readlink d1/de/df/l39 0 2024-04-02T04:42:54.158 INFO:tasks.workunit.client.1.smithi027.stdout:3/304: chown d1/de/df/d13/f35 202360 1 2024-04-02T04:42:54.163 INFO:tasks.workunit.client.1.smithi027.stdout:3/305: rename d1/de/f5b to d1/de/df/d61/f68 0 2024-04-02T04:42:54.169 INFO:tasks.workunit.client.1.smithi027.stdout:3/306: dwrite d1/d1a/f1d [0,4194304] 0 2024-04-02T04:42:54.171 INFO:tasks.workunit.client.1.smithi027.stdout:3/307: chown d1/de/df/d13/f21 489307410 1 2024-04-02T04:42:54.174 INFO:tasks.workunit.client.1.smithi027.stdout:9/374: dwrite d1a/d1d/d29/f4f [0,4194304] 0 2024-04-02T04:42:54.174 INFO:tasks.workunit.client.1.smithi027.stdout:3/308: symlink d1/de/d55/l69 0 2024-04-02T04:42:54.177 INFO:tasks.workunit.client.1.smithi027.stdout:3/309: dread d1/de/f42 [0,4194304] 0 2024-04-02T04:42:54.178 INFO:tasks.workunit.client.1.smithi027.stdout:9/375: symlink d1a/d1d/d29/d37/d3a/l83 0 2024-04-02T04:42:54.182 INFO:tasks.workunit.client.1.smithi027.stdout:9/376: creat d1a/d1d/d29/d37/d3a/f84 x:0 0 0 2024-04-02T04:42:54.183 INFO:tasks.workunit.client.1.smithi027.stdout:3/310: dread d1/f45 [0,4194304] 0 2024-04-02T04:42:54.184 INFO:tasks.workunit.client.1.smithi027.stdout:9/377: write d1a/d1d/d29/f4f [5089954,27240] 0 2024-04-02T04:42:54.185 INFO:tasks.workunit.client.1.smithi027.stdout:3/311: symlink d1/de/df/l6a 0 2024-04-02T04:42:54.188 INFO:tasks.workunit.client.1.smithi027.stdout:3/312: creat d1/de/df/f6b x:0 0 0 2024-04-02T04:42:54.188 INFO:tasks.workunit.client.1.smithi027.stdout:9/378: read d1a/f1f [2485129,110480] 0 2024-04-02T04:42:54.189 INFO:tasks.workunit.client.1.smithi027.stdout:9/379: dread - d1a/d1d/d29/d51/f82 zero size 2024-04-02T04:42:54.190 INFO:tasks.workunit.client.1.smithi027.stdout:3/313: creat d1/de/df/d61/f6c x:0 0 0 2024-04-02T04:42:54.193 INFO:tasks.workunit.client.0.smithi016.stdout:3/998: mknod d2/d2e/d36/d48/da2/de7/c14d 0 2024-04-02T04:42:54.196 INFO:tasks.workunit.client.0.smithi016.stdout:4/880: creat d5/d1d/d1e/da5/df9/f113 x:0 0 0 2024-04-02T04:42:54.196 INFO:tasks.workunit.client.1.smithi027.stdout:3/314: getdents d1/de 0 2024-04-02T04:42:54.197 INFO:tasks.workunit.client.0.smithi016.stdout:5/881: dwrite d3/df/d18/d26/f91 [0,4194304] 0 2024-04-02T04:42:54.198 INFO:tasks.workunit.client.0.smithi016.stdout:4/881: chown d5/d17/dcd/fd2 166696477 1 2024-04-02T04:42:54.198 INFO:tasks.workunit.client.0.smithi016.stdout:5/882: readlink d3/d40/d4f/d9c/lb8 0 2024-04-02T04:42:54.199 INFO:tasks.workunit.client.1.smithi027.stdout:9/380: dwrite d1a/d1d/d29/d37/d20/f61 [0,4194304] 0 2024-04-02T04:42:54.201 INFO:tasks.workunit.client.1.smithi027.stdout:3/315: unlink d1/d1a/f1d 0 2024-04-02T04:42:54.201 INFO:tasks.workunit.client.0.smithi016.stdout:1/862: dwrite d14/d19/d1d/d29/d39/d46/dcb/d43/f5f [0,4194304] 0 2024-04-02T04:42:54.202 INFO:tasks.workunit.client.0.smithi016.stdout:1/863: chown d14/d19/d1e/d5e 26 1 2024-04-02T04:42:54.202 INFO:tasks.workunit.client.1.smithi027.stdout:3/316: chown d1/de/df/f6b 359 1 2024-04-02T04:42:54.206 INFO:tasks.workunit.client.0.smithi016.stdout:0/861: dread d0/d4f/f9e [0,4194304] 0 2024-04-02T04:42:54.211 INFO:tasks.workunit.client.1.smithi027.stdout:3/317: rename d1/d1a/f49 to d1/f6d 0 2024-04-02T04:42:54.212 INFO:tasks.workunit.client.0.smithi016.stdout:1/864: mknod d14/d19/d1e/d5e/c119 0 2024-04-02T04:42:54.215 INFO:tasks.workunit.client.1.smithi027.stdout:3/318: creat d1/de/df/d61/f6e x:0 0 0 2024-04-02T04:42:54.216 INFO:tasks.workunit.client.1.smithi027.stdout:3/319: dread - d1/f6d zero size 2024-04-02T04:42:54.217 INFO:tasks.workunit.client.1.smithi027.stdout:3/320: dread - d1/de/df/f67 zero size 2024-04-02T04:42:54.218 INFO:tasks.workunit.client.0.smithi016.stdout:4/882: unlink d5/d16/df8/d7e/fe1 0 2024-04-02T04:42:54.218 INFO:tasks.workunit.client.1.smithi027.stdout:9/381: dread d1a/f63 [0,4194304] 0 2024-04-02T04:42:54.218 INFO:tasks.workunit.client.1.smithi027.stdout:9/382: stat d1a/d1d/d29 0 2024-04-02T04:42:54.219 INFO:tasks.workunit.client.0.smithi016.stdout:0/862: symlink d0/d16/dd6/l11f 0 2024-04-02T04:42:54.221 INFO:tasks.workunit.client.0.smithi016.stdout:1/865: mkdir d14/d19/d11a 0 2024-04-02T04:42:54.225 INFO:tasks.workunit.client.0.smithi016.stdout:8/892: link d1/d6/d36/d4f/d5d/cac d1/d70/c123 0 2024-04-02T04:42:54.226 INFO:tasks.workunit.client.1.smithi027.stdout:9/383: dwrite d1a/d1d/d29/d37/f5a [0,4194304] 0 2024-04-02T04:42:54.227 INFO:tasks.workunit.client.0.smithi016.stdout:0/863: rename d0/d16/d2e/d36/d9f/fc1 to d0/d16/d2e/d36/d9f/f120 0 2024-04-02T04:42:54.227 INFO:tasks.workunit.client.0.smithi016.stdout:3/999: dread d2/d2e/d36/d66/f123 [0,4194304] 0 2024-04-02T04:42:54.230 INFO:tasks.workunit.client.1.smithi027.stdout:3/321: getdents d1 0 2024-04-02T04:42:54.234 INFO:tasks.workunit.client.1.smithi027.stdout:9/384: rename d1a/d1d/d29/d51/f82 to d1a/d1d/d29/d37/d20/d7e/f85 0 2024-04-02T04:42:54.235 INFO:tasks.workunit.client.0.smithi016.stdout:9/962: sync 2024-04-02T04:42:54.236 INFO:tasks.workunit.client.0.smithi016.stdout:9/963: readlink d0/d3/l2f 0 2024-04-02T04:42:54.237 INFO:tasks.workunit.client.1.smithi027.stdout:3/322: fsync d1/de/f42 0 2024-04-02T04:42:54.240 INFO:tasks.workunit.client.0.smithi016.stdout:0/864: unlink d0/d16/d2e/l5f 0 2024-04-02T04:42:54.241 INFO:tasks.workunit.client.0.smithi016.stdout:0/865: write d0/d48/fec [795298,214] 0 2024-04-02T04:42:54.242 INFO:tasks.workunit.client.0.smithi016.stdout:1/866: write d14/d19/d1e/f2b [1227016,4903] 0 2024-04-02T04:42:54.244 INFO:tasks.workunit.client.0.smithi016.stdout:9/964: unlink d0/d4e/d50/d63/d6e/d4b/fc0 0 2024-04-02T04:42:54.246 INFO:tasks.workunit.client.1.smithi027.stdout:3/323: creat d1/de/df/f6f x:0 0 0 2024-04-02T04:42:54.247 INFO:tasks.workunit.client.1.smithi027.stdout:9/385: symlink d1a/l86 0 2024-04-02T04:42:54.247 INFO:tasks.workunit.client.1.smithi027.stdout:9/386: stat d1a/l48 0 2024-04-02T04:42:54.248 INFO:tasks.workunit.client.1.smithi027.stdout:9/387: write d1a/d1d/d29/f46 [1683187,124719] 0 2024-04-02T04:42:54.249 INFO:tasks.workunit.client.1.smithi027.stdout:9/388: write d1a/d1d/d29/d37/f5a [3693575,37138] 0 2024-04-02T04:42:54.252 INFO:tasks.workunit.client.1.smithi027.stdout:3/324: rmdir d1/d1a 39 2024-04-02T04:42:54.253 INFO:tasks.workunit.client.1.smithi027.stdout:3/325: dread - d1/de/df/d13/f5c zero size 2024-04-02T04:42:54.256 INFO:tasks.workunit.client.1.smithi027.stdout:9/389: creat d1a/d1d/d29/d37/d20/d7e/f87 x:0 0 0 2024-04-02T04:42:54.259 INFO:tasks.workunit.client.1.smithi027.stdout:3/326: link d1/de/d55/l69 d1/de/d55/d63/l70 0 2024-04-02T04:42:54.264 INFO:tasks.workunit.client.1.smithi027.stdout:9/390: dwrite d1a/f1f [0,4194304] 0 2024-04-02T04:42:54.265 INFO:tasks.workunit.client.1.smithi027.stdout:3/327: rmdir d1/de/df/d61 39 2024-04-02T04:42:54.269 INFO:tasks.workunit.client.1.smithi027.stdout:3/328: creat d1/de/f71 x:0 0 0 2024-04-02T04:42:54.288 INFO:tasks.workunit.client.1.smithi027.stdout:9/391: dread d1a/d1d/d29/d37/d20/f3f [0,4194304] 0 2024-04-02T04:42:54.289 INFO:tasks.workunit.client.0.smithi016.stdout:8/893: truncate d1/d6/d36/d3f/d40/ff3 1073058 0 2024-04-02T04:42:54.291 INFO:tasks.workunit.client.0.smithi016.stdout:4/883: dwrite d5/d17/d18/d1f/f40 [0,4194304] 0 2024-04-02T04:42:54.291 INFO:tasks.workunit.client.0.smithi016.stdout:0/866: truncate d0/d16/d24/fa1 59598 0 2024-04-02T04:42:54.292 INFO:tasks.workunit.client.1.smithi027.stdout:3/329: dread d1/de/df/f17 [0,4194304] 0 2024-04-02T04:42:54.293 INFO:tasks.workunit.client.0.smithi016.stdout:9/965: creat d0/f12f x:0 0 0 2024-04-02T04:42:54.295 INFO:tasks.workunit.client.0.smithi016.stdout:1/867: dwrite d14/f28 [0,4194304] 0 2024-04-02T04:42:54.295 INFO:tasks.workunit.client.1.smithi027.stdout:3/330: mknod d1/de/df/c72 0 2024-04-02T04:42:54.296 INFO:tasks.workunit.client.1.smithi027.stdout:9/392: dwrite d1a/d1d/d29/d37/d3a/f54 [0,4194304] 0 2024-04-02T04:42:54.296 INFO:tasks.workunit.client.0.smithi016.stdout:4/884: symlink d5/d1d/d5f/d95/l114 0 2024-04-02T04:42:54.301 INFO:tasks.workunit.client.1.smithi027.stdout:3/331: mkdir d1/de/df/d13/d73 0 2024-04-02T04:42:54.303 INFO:tasks.workunit.client.1.smithi027.stdout:9/393: creat d1a/d1d/d29/d56/f88 x:0 0 0 2024-04-02T04:42:54.305 INFO:tasks.workunit.client.1.smithi027.stdout:9/394: write d1a/d1d/d29/d37/d3a/f54 [413534,72202] 0 2024-04-02T04:42:54.305 INFO:tasks.workunit.client.0.smithi016.stdout:4/885: dwrite d5/d17/d18/d6d/f74 [0,4194304] 0 2024-04-02T04:42:54.306 INFO:tasks.workunit.client.0.smithi016.stdout:1/868: dread d14/d19/d1d/d29/d39/f65 [0,4194304] 0 2024-04-02T04:42:54.308 INFO:tasks.workunit.client.0.smithi016.stdout:4/886: dread d5/d17/dcd/fd8 [0,4194304] 0 2024-04-02T04:42:54.311 INFO:tasks.workunit.client.1.smithi027.stdout:3/332: dwrite d1/f45 [0,4194304] 0 2024-04-02T04:42:54.312 INFO:tasks.workunit.client.0.smithi016.stdout:8/894: link d1/d6/d36/d3f/d40/d45/f6e d1/d6/d36/d3f/d40/d49/d4a/d4c/f124 0 2024-04-02T04:42:54.313 INFO:tasks.workunit.client.1.smithi027.stdout:3/333: dread - d1/de/df/d13/f51 zero size 2024-04-02T04:42:54.313 INFO:tasks.workunit.client.0.smithi016.stdout:8/895: write d1/d31/f7e [3077299,40283] 0 2024-04-02T04:42:54.314 INFO:tasks.workunit.client.1.smithi027.stdout:3/334: chown d1/de/df/d13/f35 1 1 2024-04-02T04:42:54.318 INFO:tasks.workunit.client.1.smithi027.stdout:9/395: dread f15 [4194304,4194304] 0 2024-04-02T04:42:54.319 INFO:tasks.workunit.client.0.smithi016.stdout:0/867: write d0/d16/d24/d62/d9c/d8c/f92 [1654410,4776] 0 2024-04-02T04:42:54.319 INFO:tasks.workunit.client.0.smithi016.stdout:9/966: write d0/d4a/fc2 [1650658,58548] 0 2024-04-02T04:42:54.319 INFO:tasks.workunit.client.0.smithi016.stdout:0/868: fsync d0/d4f/daf/fd5 0 2024-04-02T04:42:54.319 INFO:tasks.workunit.client.0.smithi016.stdout:0/869: chown d0/fc8 15933987 1 2024-04-02T04:42:54.320 INFO:tasks.workunit.client.1.smithi027.stdout:3/335: mkdir d1/de/d74 0 2024-04-02T04:42:54.321 INFO:tasks.workunit.client.0.smithi016.stdout:0/870: chown d0/d16/d2e/d37/d66/ff4 2122492 1 2024-04-02T04:42:54.321 INFO:tasks.workunit.client.1.smithi027.stdout:3/336: write d1/de/df/f24 [239276,81047] 0 2024-04-02T04:42:54.323 INFO:tasks.workunit.client.1.smithi027.stdout:9/396: rename d1a/d1d/d29/d37/d4c/l7c to d1a/d1d/d29/d50/d73/l89 0 2024-04-02T04:42:54.325 INFO:tasks.workunit.client.0.smithi016.stdout:2/951: sync 2024-04-02T04:42:54.327 INFO:tasks.workunit.client.0.smithi016.stdout:1/869: creat d14/d19/d1d/d29/f11b x:0 0 0 2024-04-02T04:42:54.328 INFO:tasks.workunit.client.0.smithi016.stdout:2/952: write d12/d35/d101/d4f/dfa/f128 [388417,97285] 0 2024-04-02T04:42:54.328 INFO:tasks.workunit.client.0.smithi016.stdout:8/896: dwrite d1/d6/d86/fa9 [0,4194304] 0 2024-04-02T04:42:54.330 INFO:tasks.workunit.client.0.smithi016.stdout:4/887: dread d5/d17/d18/d1f/f70 [0,4194304] 0 2024-04-02T04:42:54.330 INFO:tasks.workunit.client.0.smithi016.stdout:2/953: readlink d12/d35/d3b/lf5 0 2024-04-02T04:42:54.331 INFO:tasks.workunit.client.1.smithi027.stdout:9/397: rename d1a/l48 to d1a/d1d/d29/d37/l8a 0 2024-04-02T04:42:54.332 INFO:tasks.workunit.client.0.smithi016.stdout:0/871: mkdir d0/d16/dd6/ddd/d121 0 2024-04-02T04:42:54.332 INFO:tasks.workunit.client.1.smithi027.stdout:9/398: write d1a/d1d/d29/f4f [5905539,51728] 0 2024-04-02T04:42:54.333 INFO:tasks.workunit.client.1.smithi027.stdout:3/337: getdents d1/de/df/d61 0 2024-04-02T04:42:54.344 INFO:tasks.workunit.client.0.smithi016.stdout:0/872: dread d0/d4f/f95 [0,4194304] 0 2024-04-02T04:42:54.348 INFO:tasks.workunit.client.1.smithi027.stdout:9/399: rmdir d1a/d1d/d29/d50/d74 0 2024-04-02T04:42:54.349 INFO:tasks.workunit.client.0.smithi016.stdout:0/873: truncate d0/d16/d24/f69 3012500 0 2024-04-02T04:42:54.350 INFO:tasks.workunit.client.1.smithi027.stdout:9/400: mkdir d1a/d1d/d29/d56/d8b 0 2024-04-02T04:42:54.352 INFO:tasks.workunit.client.0.smithi016.stdout:0/874: stat d0/d16/d2e/d53/da4/de3/cef 0 2024-04-02T04:42:54.353 INFO:tasks.workunit.client.0.smithi016.stdout:4/888: write d5/d1d/f37 [2762193,9639] 0 2024-04-02T04:42:54.356 INFO:tasks.workunit.client.1.smithi027.stdout:7/380: sync 2024-04-02T04:42:54.357 INFO:tasks.workunit.client.1.smithi027.stdout:4/311: sync 2024-04-02T04:42:54.357 INFO:tasks.workunit.client.1.smithi027.stdout:3/338: write d1/de/df/f17 [1537306,32863] 0 2024-04-02T04:42:54.358 INFO:tasks.workunit.client.1.smithi027.stdout:9/401: getdents d1a 0 2024-04-02T04:42:54.358 INFO:tasks.workunit.client.0.smithi016.stdout:9/967: dread d0/d4e/fad [4194304,4194304] 0 2024-04-02T04:42:54.359 INFO:tasks.workunit.client.1.smithi027.stdout:9/402: fsync d1a/f1f 0 2024-04-02T04:42:54.360 INFO:tasks.workunit.client.1.smithi027.stdout:9/403: chown d1a/d1d/d29/d50 51 1 2024-04-02T04:42:54.363 INFO:tasks.workunit.client.1.smithi027.stdout:4/312: rename l10 to d24/d38/d44/l67 0 2024-04-02T04:42:54.363 INFO:tasks.workunit.client.0.smithi016.stdout:0/875: dwrite d0/d16/d24/d4e/d8d/d71/f72 [0,4194304] 0 2024-04-02T04:42:54.366 INFO:tasks.workunit.client.0.smithi016.stdout:8/897: dwrite d1/d6/d36/d3f/d40/d49/d38/fc4 [0,4194304] 0 2024-04-02T04:42:54.369 INFO:tasks.workunit.client.1.smithi027.stdout:7/381: fsync d2/f4c 0 2024-04-02T04:42:54.372 INFO:tasks.workunit.client.0.smithi016.stdout:4/889: dwrite d5/d17/d18/d25/f41 [0,4194304] 0 2024-04-02T04:42:54.372 INFO:tasks.workunit.client.0.smithi016.stdout:2/954: dwrite d12/d35/d3f/d56/fb7 [0,4194304] 0 2024-04-02T04:42:54.372 INFO:tasks.workunit.client.1.smithi027.stdout:3/339: link d1/f2a d1/de/d55/f75 0 2024-04-02T04:42:54.379 INFO:tasks.workunit.client.0.smithi016.stdout:9/968: rename d0/d9/db/dc9/d53/c91 to d0/d4e/c130 0 2024-04-02T04:42:54.385 INFO:tasks.workunit.client.0.smithi016.stdout:9/969: dwrite d0/d4a/d75/d95/fca [0,4194304] 0 2024-04-02T04:42:54.385 INFO:tasks.workunit.client.1.smithi027.stdout:9/404: dread f15 [0,4194304] 0 2024-04-02T04:42:54.386 INFO:tasks.workunit.client.1.smithi027.stdout:7/382: rmdir d2/d7 39 2024-04-02T04:42:54.388 INFO:tasks.workunit.client.1.smithi027.stdout:0/405: sync 2024-04-02T04:42:54.390 INFO:tasks.workunit.client.1.smithi027.stdout:3/340: creat d1/de/d55/f76 x:0 0 0 2024-04-02T04:42:54.391 INFO:tasks.workunit.client.1.smithi027.stdout:0/406: dread d4/d38/d66/f24 [0,4194304] 0 2024-04-02T04:42:54.393 INFO:tasks.workunit.client.1.smithi027.stdout:4/313: symlink d24/d38/d47/d5c/l68 0 2024-04-02T04:42:54.394 INFO:tasks.workunit.client.1.smithi027.stdout:4/314: dread - d24/d48/f5d zero size 2024-04-02T04:42:54.396 INFO:tasks.workunit.client.1.smithi027.stdout:9/405: mkdir d1a/d1d/d3c/d71/d8c 0 2024-04-02T04:42:54.397 INFO:tasks.workunit.client.0.smithi016.stdout:0/876: symlink d0/d16/dd6/ddd/d121/l122 0 2024-04-02T04:42:54.397 INFO:tasks.workunit.client.1.smithi027.stdout:9/406: dread f15 [4194304,4194304] 0 2024-04-02T04:42:54.400 INFO:tasks.workunit.client.1.smithi027.stdout:3/341: read - d1/d1a/f3f zero size 2024-04-02T04:42:54.401 INFO:tasks.workunit.client.0.smithi016.stdout:9/970: mkdir d0/d4e/d50/d63/d6e/d4b/db2/d131 0 2024-04-02T04:42:54.402 INFO:tasks.workunit.client.0.smithi016.stdout:9/971: write d0/d4a/d75/f117 [398163,53] 0 2024-04-02T04:42:54.403 INFO:tasks.workunit.client.0.smithi016.stdout:9/972: write d0/d9/db/dc9/fdf [4932087,68883] 0 2024-04-02T04:42:54.404 INFO:tasks.workunit.client.0.smithi016.stdout:9/973: chown d0/d9/db/f43 6863 1 2024-04-02T04:42:54.405 INFO:tasks.workunit.client.1.smithi027.stdout:9/407: dwrite d1a/d1d/d29/d37/d20/f61 [0,4194304] 0 2024-04-02T04:42:54.407 INFO:tasks.workunit.client.1.smithi027.stdout:9/408: fdatasync d1a/d1d/d29/d56/f88 0 2024-04-02T04:42:54.408 INFO:tasks.workunit.client.1.smithi027.stdout:9/409: write d1a/f1f [4750107,119675] 0 2024-04-02T04:42:54.409 INFO:tasks.workunit.client.0.smithi016.stdout:7/875: sync 2024-04-02T04:42:54.409 INFO:tasks.workunit.client.0.smithi016.stdout:6/907: sync 2024-04-02T04:42:54.411 INFO:tasks.workunit.client.0.smithi016.stdout:4/890: write d5/d16/f49 [10354935,7723] 0 2024-04-02T04:42:54.411 INFO:tasks.workunit.client.0.smithi016.stdout:6/908: dread - d0/d9/d16/d56/d89/d67/d6f/f11c zero size 2024-04-02T04:42:54.415 INFO:tasks.workunit.client.0.smithi016.stdout:2/955: rename d12/d35/d101/d76/dcf/c108 to d12/d35/d3f/d2b/de9/c14d 0 2024-04-02T04:42:54.416 INFO:tasks.workunit.client.1.smithi027.stdout:9/410: dwrite d1a/d1d/d29/d37/f5a [0,4194304] 0 2024-04-02T04:42:54.420 INFO:tasks.workunit.client.0.smithi016.stdout:0/877: creat d0/d4f/daf/f123 x:0 0 0 2024-04-02T04:42:54.422 INFO:tasks.workunit.client.1.smithi027.stdout:0/407: symlink d4/d38/d66/l83 0 2024-04-02T04:42:54.423 INFO:tasks.workunit.client.1.smithi027.stdout:9/411: dread d1a/f38 [0,4194304] 0 2024-04-02T04:42:54.425 INFO:tasks.workunit.client.0.smithi016.stdout:5/883: sync 2024-04-02T04:42:54.425 INFO:tasks.workunit.client.0.smithi016.stdout:1/870: sync 2024-04-02T04:42:54.428 INFO:tasks.workunit.client.0.smithi016.stdout:8/898: truncate d1/d70/d71/d74/d82/fc2 2314808 0 2024-04-02T04:42:54.429 INFO:tasks.workunit.client.1.smithi027.stdout:4/315: dwrite d24/f25 [0,4194304] 0 2024-04-02T04:42:54.429 INFO:tasks.workunit.client.1.smithi027.stdout:7/383: symlink d2/d7/da/dc/d16/d3a/d4d/d6f/l75 0 2024-04-02T04:42:54.431 INFO:tasks.workunit.client.0.smithi016.stdout:7/876: read d0/d46/d37/dbe/f100 [4168636,87277] 0 2024-04-02T04:42:54.431 INFO:tasks.workunit.client.1.smithi027.stdout:4/316: fdatasync d24/d38/f3d 0 2024-04-02T04:42:54.431 INFO:tasks.workunit.client.0.smithi016.stdout:6/909: symlink d0/d9/d16/l121 0 2024-04-02T04:42:54.434 INFO:tasks.workunit.client.1.smithi027.stdout:0/408: symlink d4/d3d/l84 0 2024-04-02T04:42:54.434 INFO:tasks.workunit.client.0.smithi016.stdout:0/878: truncate d0/d16/d24/fc5 4796 0 2024-04-02T04:42:54.439 INFO:tasks.workunit.client.0.smithi016.stdout:1/871: dread - d14/d19/d1d/d29/d39/d46/dcb/f93 zero size 2024-04-02T04:42:54.439 INFO:tasks.workunit.client.1.smithi027.stdout:1/376: sync 2024-04-02T04:42:54.439 INFO:tasks.workunit.client.1.smithi027.stdout:2/381: sync 2024-04-02T04:42:54.439 INFO:tasks.workunit.client.1.smithi027.stdout:6/376: sync 2024-04-02T04:42:54.439 INFO:tasks.workunit.client.1.smithi027.stdout:8/431: sync 2024-04-02T04:42:54.440 INFO:tasks.workunit.client.1.smithi027.stdout:5/502: sync 2024-04-02T04:42:54.440 INFO:tasks.workunit.client.1.smithi027.stdout:4/317: dread d24/d48/f4e [0,4194304] 0 2024-04-02T04:42:54.440 INFO:tasks.workunit.client.0.smithi016.stdout:7/877: dwrite d0/d46/d37/faa [0,4194304] 0 2024-04-02T04:42:54.441 INFO:tasks.workunit.client.1.smithi027.stdout:7/384: rename d2/d7/l32 to d2/d7/da/dc/d16/d3a/l76 0 2024-04-02T04:42:54.441 INFO:tasks.workunit.client.1.smithi027.stdout:6/377: chown d13/d1b/d24/d25/d56/d73 1295420732 1 2024-04-02T04:42:54.442 INFO:tasks.workunit.client.0.smithi016.stdout:8/899: stat d1/d70/d71/d74/d82/cc6 0 2024-04-02T04:42:54.449 INFO:tasks.workunit.client.0.smithi016.stdout:9/974: dwrite d0/d4e/d50/d63/d6e/f44 [0,4194304] 0 2024-04-02T04:42:54.454 INFO:tasks.workunit.client.1.smithi027.stdout:3/342: getdents d1/de/df/d13 0 2024-04-02T04:42:54.457 INFO:tasks.workunit.client.0.smithi016.stdout:4/891: dwrite d5/d8/d43/f45 [0,4194304] 0 2024-04-02T04:42:54.457 INFO:tasks.workunit.client.0.smithi016.stdout:5/884: mkdir d3/d129/d13c 0 2024-04-02T04:42:54.457 INFO:tasks.workunit.client.1.smithi027.stdout:2/382: creat d1/f88 x:0 0 0 2024-04-02T04:42:54.460 INFO:tasks.workunit.client.1.smithi027.stdout:5/503: mkdir d0/df/d11/d13/d9a/d9c/db8 0 2024-04-02T04:42:54.462 INFO:tasks.workunit.client.1.smithi027.stdout:5/504: read - d0/df/d2f/d3b/d51/f8e zero size 2024-04-02T04:42:54.462 INFO:tasks.workunit.client.1.smithi027.stdout:9/412: dwrite f12 [4194304,4194304] 0 2024-04-02T04:42:54.464 INFO:tasks.workunit.client.0.smithi016.stdout:8/900: dread - d1/d6/d36/d3f/d40/d45/fbc zero size 2024-04-02T04:42:54.465 INFO:tasks.workunit.client.1.smithi027.stdout:8/432: rmdir d5/d8/db/dd/d5f 39 2024-04-02T04:42:54.465 INFO:tasks.workunit.client.0.smithi016.stdout:7/878: dread d0/d46/d16/dc2/dea/d64/f10b [0,4194304] 0 2024-04-02T04:42:54.466 INFO:tasks.workunit.client.1.smithi027.stdout:8/433: write d5/f7d [625763,73231] 0 2024-04-02T04:42:54.466 INFO:tasks.workunit.client.0.smithi016.stdout:9/975: mkdir d0/d4e/d50/d63/dd2/d132 0 2024-04-02T04:42:54.466 INFO:tasks.workunit.client.0.smithi016.stdout:8/901: write d1/d70/d71/d74/f96 [46954,49499] 0 2024-04-02T04:42:54.469 INFO:tasks.workunit.client.1.smithi027.stdout:8/434: write d5/d8/db/df/f22 [2143893,118849] 0 2024-04-02T04:42:54.470 INFO:tasks.workunit.client.1.smithi027.stdout:7/385: chown d2/c3d 866237141 1 2024-04-02T04:42:54.474 INFO:tasks.workunit.client.1.smithi027.stdout:0/409: mkdir d4/d21/d85 0 2024-04-02T04:42:54.478 INFO:tasks.workunit.client.0.smithi016.stdout:5/885: symlink d3/d129/l13d 0 2024-04-02T04:42:54.479 INFO:tasks.workunit.client.1.smithi027.stdout:9/413: symlink d1a/l8d 0 2024-04-02T04:42:54.481 INFO:tasks.workunit.client.0.smithi016.stdout:2/956: dwrite d12/d35/f40 [0,4194304] 0 2024-04-02T04:42:54.481 INFO:tasks.workunit.client.1.smithi027.stdout:5/505: symlink d0/d1a/d6f/lb9 0 2024-04-02T04:42:54.482 INFO:tasks.workunit.client.0.smithi016.stdout:1/872: dread d14/d19/d20/f3b [0,4194304] 0 2024-04-02T04:42:54.482 INFO:tasks.workunit.client.1.smithi027.stdout:5/506: chown d0/df/d11/d41/l71 1 1 2024-04-02T04:42:54.483 INFO:tasks.workunit.client.0.smithi016.stdout:2/957: write d12/d35/d101/d4f/dfa/f13a [583053,59719] 0 2024-04-02T04:42:54.483 INFO:tasks.workunit.client.1.smithi027.stdout:5/507: write d0/d1a/d20/f59 [2139049,51842] 0 2024-04-02T04:42:54.485 INFO:tasks.workunit.client.0.smithi016.stdout:1/873: truncate d14/d19/d1e/d71/f104 1000069 0 2024-04-02T04:42:54.489 INFO:tasks.workunit.client.1.smithi027.stdout:4/318: dwrite f12 [0,4194304] 0 2024-04-02T04:42:54.490 INFO:tasks.workunit.client.1.smithi027.stdout:5/508: dread d0/d1a/f49 [0,4194304] 0 2024-04-02T04:42:54.491 INFO:tasks.workunit.client.1.smithi027.stdout:8/435: chown d5/d8/db/dd/d5f/f7e 7417922 1 2024-04-02T04:42:54.492 INFO:tasks.workunit.client.1.smithi027.stdout:4/319: write d24/f27 [669920,3423] 0 2024-04-02T04:42:54.495 INFO:tasks.workunit.client.1.smithi027.stdout:3/343: mkdir d1/de/df/d13/d73/d77 0 2024-04-02T04:42:54.496 INFO:tasks.workunit.client.1.smithi027.stdout:3/344: chown d1/de/df/l39 16 1 2024-04-02T04:42:54.498 INFO:tasks.workunit.client.1.smithi027.stdout:3/345: dread d1/de/df/f17 [4194304,4194304] 0 2024-04-02T04:42:54.503 INFO:tasks.workunit.client.1.smithi027.stdout:7/386: creat d2/d7/da/dc/d1f/d61/f77 x:0 0 0 2024-04-02T04:42:54.504 INFO:tasks.workunit.client.1.smithi027.stdout:7/387: write d2/d7/da/dc/d16/d3a/d53/f5d [1048557,122791] 0 2024-04-02T04:42:54.505 INFO:tasks.workunit.client.1.smithi027.stdout:8/436: creat d5/d8/d15/f8a x:0 0 0 2024-04-02T04:42:54.508 INFO:tasks.workunit.client.1.smithi027.stdout:4/320: read d24/f2c [615967,9135] 0 2024-04-02T04:42:54.510 INFO:tasks.workunit.client.1.smithi027.stdout:5/509: symlink d0/df/d11/d13/lba 0 2024-04-02T04:42:54.510 INFO:tasks.workunit.client.1.smithi027.stdout:5/510: write d0/df/d29/f7a [5614259,55671] 0 2024-04-02T04:42:54.512 INFO:tasks.workunit.client.1.smithi027.stdout:5/511: write d0/d1a/d20/f93 [831182,22384] 0 2024-04-02T04:42:54.512 INFO:tasks.workunit.client.1.smithi027.stdout:0/410: creat d4/d21/d85/f86 x:0 0 0 2024-04-02T04:42:54.518 INFO:tasks.workunit.client.1.smithi027.stdout:3/346: creat d1/de/df/d13/d73/f78 x:0 0 0 2024-04-02T04:42:54.521 INFO:tasks.workunit.client.1.smithi027.stdout:5/512: dwrite d0/df/d76/f8d [0,4194304] 0 2024-04-02T04:42:54.523 INFO:tasks.workunit.client.1.smithi027.stdout:7/388: fdatasync d2/d1c/f4b 0 2024-04-02T04:42:54.527 INFO:tasks.workunit.client.1.smithi027.stdout:8/437: creat d5/d8/d48/f8b x:0 0 0 2024-04-02T04:42:54.528 INFO:tasks.workunit.client.0.smithi016.stdout:0/879: sync 2024-04-02T04:42:54.532 INFO:tasks.workunit.client.1.smithi027.stdout:8/438: dread d5/d8/db/dd/d5f/f2b [0,4194304] 0 2024-04-02T04:42:54.535 INFO:tasks.workunit.client.1.smithi027.stdout:2/383: getdents d1/db/d3b 0 2024-04-02T04:42:54.537 INFO:tasks.workunit.client.1.smithi027.stdout:0/411: rename d4/d38/f58 to d4/d21/d28/d4e/f87 0 2024-04-02T04:42:54.542 INFO:tasks.workunit.client.1.smithi027.stdout:5/513: symlink d0/df/d29/lbb 0 2024-04-02T04:42:54.546 INFO:tasks.workunit.client.1.smithi027.stdout:8/439: dread d5/d8/db/dd/f1d [0,4194304] 0 2024-04-02T04:42:54.547 INFO:tasks.workunit.client.1.smithi027.stdout:8/440: readlink d5/d8/db/l54 0 2024-04-02T04:42:54.548 INFO:tasks.workunit.client.1.smithi027.stdout:4/321: symlink d24/d38/d44/d60/l69 0 2024-04-02T04:42:54.549 INFO:tasks.workunit.client.1.smithi027.stdout:4/322: stat d24/f32 0 2024-04-02T04:42:54.553 INFO:tasks.workunit.client.1.smithi027.stdout:2/384: mknod d1/db/d3b/d17/d68/d76/c89 0 2024-04-02T04:42:54.559 INFO:tasks.workunit.client.1.smithi027.stdout:8/441: mknod d5/d8/db/d57/c8c 0 2024-04-02T04:42:54.561 INFO:tasks.workunit.client.1.smithi027.stdout:4/323: mknod d24/d38/d44/d41/d49/c6a 0 2024-04-02T04:42:54.565 INFO:tasks.workunit.client.1.smithi027.stdout:0/412: symlink d4/d38/d74/l88 0 2024-04-02T04:42:54.567 INFO:tasks.workunit.client.1.smithi027.stdout:2/385: rename d1/db/d3b/d17/d12/d5c to d1/db/d3b/d61/d6c/d8a 0 2024-04-02T04:42:54.572 INFO:tasks.workunit.client.0.smithi016.stdout:9/976: rename d0/d4e/d50 to d0/d9/db/d133 0 2024-04-02T04:42:54.573 INFO:tasks.workunit.client.1.smithi027.stdout:8/442: mkdir d5/d8/db/dd/d5f/d27/d31/d5b/d8d 0 2024-04-02T04:42:54.574 INFO:tasks.workunit.client.1.smithi027.stdout:8/443: write d5/d8/d15/f8a [993399,33030] 0 2024-04-02T04:42:54.574 INFO:tasks.workunit.client.0.smithi016.stdout:8/902: creat d1/d6/d86/d119/f125 x:0 0 0 2024-04-02T04:42:54.575 INFO:tasks.workunit.client.1.smithi027.stdout:4/324: fdatasync f23 0 2024-04-02T04:42:54.576 INFO:tasks.workunit.client.1.smithi027.stdout:4/325: chown d24/d48/f4e 77636323 1 2024-04-02T04:42:54.576 INFO:tasks.workunit.client.0.smithi016.stdout:4/892: symlink d5/d17/dcd/def/dfd/l115 0 2024-04-02T04:42:54.580 INFO:tasks.workunit.client.0.smithi016.stdout:4/893: dread d5/d8/d43/f45 [0,4194304] 0 2024-04-02T04:42:54.582 INFO:tasks.workunit.client.0.smithi016.stdout:5/886: mknod d3/df/d18/d26/d48/c13e 0 2024-04-02T04:42:54.582 INFO:tasks.workunit.client.1.smithi027.stdout:9/414: dwrite d1a/f38 [0,4194304] 0 2024-04-02T04:42:54.583 INFO:tasks.workunit.client.1.smithi027.stdout:0/413: dread d4/d16/d1b/f20 [0,4194304] 0 2024-04-02T04:42:54.584 INFO:tasks.workunit.client.1.smithi027.stdout:0/414: chown d4/d38/d66/d2d/d81/f50 981278141 1 2024-04-02T04:42:54.585 INFO:tasks.workunit.client.1.smithi027.stdout:2/386: unlink d1/db/d3b/d17/d12/l28 0 2024-04-02T04:42:54.590 INFO:tasks.workunit.client.1.smithi027.stdout:2/387: write d1/db/d3b/f70 [1094607,69805] 0 2024-04-02T04:42:54.591 INFO:tasks.workunit.client.1.smithi027.stdout:5/514: link d0/d1d/d61/f62 d0/d5/fbc 0 2024-04-02T04:42:54.592 INFO:tasks.workunit.client.1.smithi027.stdout:8/444: creat d5/d8/db/f8e x:0 0 0 2024-04-02T04:42:54.593 INFO:tasks.workunit.client.0.smithi016.stdout:0/880: chown d0/d16/d23/d3e/db3/ldf 210 1 2024-04-02T04:42:54.595 INFO:tasks.workunit.client.0.smithi016.stdout:6/910: link d0/d9/d16/d56/d89/l4c d0/d9/d16/d56/da4/l122 0 2024-04-02T04:42:54.596 INFO:tasks.workunit.client.1.smithi027.stdout:4/326: truncate d24/f46 1663770 0 2024-04-02T04:42:54.598 INFO:tasks.workunit.client.1.smithi027.stdout:2/388: read d1/db/d3b/d17/d12/f41 [702010,128883] 0 2024-04-02T04:42:54.599 INFO:tasks.workunit.client.0.smithi016.stdout:9/977: rmdir d0/d4a/d99/dd3 39 2024-04-02T04:42:54.600 INFO:tasks.workunit.client.0.smithi016.stdout:6/911: dwrite d0/d9/d16/d46/dbe/fd1 [0,4194304] 0 2024-04-02T04:42:54.607 INFO:tasks.workunit.client.1.smithi027.stdout:0/415: unlink d4/d16/f6a 0 2024-04-02T04:42:54.608 INFO:tasks.workunit.client.0.smithi016.stdout:4/894: truncate d5/d16/df8/d7e/f81 4475373 0 2024-04-02T04:42:54.610 INFO:tasks.workunit.client.0.smithi016.stdout:5/887: rmdir d3/d129 39 2024-04-02T04:42:54.611 INFO:tasks.workunit.client.0.smithi016.stdout:7/879: write d0/d46/d52/f61 [3944811,16086] 0 2024-04-02T04:42:54.614 INFO:tasks.workunit.client.0.smithi016.stdout:1/874: mkdir d14/d11c 0 2024-04-02T04:42:54.616 INFO:tasks.workunit.client.0.smithi016.stdout:7/880: dwrite d0/d46/d37/faa [4194304,4194304] 0 2024-04-02T04:42:54.623 INFO:tasks.workunit.client.0.smithi016.stdout:2/958: write d12/d35/d3f/d2b/fed [920509,14150] 0 2024-04-02T04:42:54.624 INFO:tasks.workunit.client.0.smithi016.stdout:2/959: fdatasync d12/d35/fca 0 2024-04-02T04:42:54.627 INFO:tasks.workunit.client.0.smithi016.stdout:8/903: mkdir d1/d6/d36/d3f/d120/d126 0 2024-04-02T04:42:54.629 INFO:tasks.workunit.client.0.smithi016.stdout:4/895: dread d5/d8/fb [0,4194304] 0 2024-04-02T04:42:54.630 INFO:tasks.workunit.client.0.smithi016.stdout:0/881: dread d0/f32 [0,4194304] 0 2024-04-02T04:42:54.631 INFO:tasks.workunit.client.0.smithi016.stdout:0/882: readlink d0/d16/d24/d62/d9c/l7e 0 2024-04-02T04:42:54.633 INFO:tasks.workunit.client.0.smithi016.stdout:5/888: symlink d3/d40/d4f/d67/dbf/d104/d12c/l13f 0 2024-04-02T04:42:54.634 INFO:tasks.workunit.client.0.smithi016.stdout:5/889: chown d3/df/d18/d2f/la8 0 1 2024-04-02T04:42:54.637 INFO:tasks.workunit.client.0.smithi016.stdout:7/881: fsync d0/d46/d37/d47/f5c 0 2024-04-02T04:42:54.638 INFO:tasks.workunit.client.1.smithi027.stdout:5/515: creat d0/d1d/fbd x:0 0 0 2024-04-02T04:42:54.639 INFO:tasks.workunit.client.0.smithi016.stdout:7/882: stat d0/d46/d37/dbe/dae/df8/da9/ccf 0 2024-04-02T04:42:54.639 INFO:tasks.workunit.client.0.smithi016.stdout:9/978: write d0/d4a/f9c [299229,123718] 0 2024-04-02T04:42:54.642 INFO:tasks.workunit.client.1.smithi027.stdout:8/445: creat d5/d8/db/dd/d5f/d27/d31/d5b/f8f x:0 0 0 2024-04-02T04:42:54.643 INFO:tasks.workunit.client.0.smithi016.stdout:2/960: mkdir d12/d35/d3f/d2b/d14e 0 2024-04-02T04:42:54.645 INFO:tasks.workunit.client.0.smithi016.stdout:8/904: creat d1/d6/d36/d100/f127 x:0 0 0 2024-04-02T04:42:54.647 INFO:tasks.workunit.client.1.smithi027.stdout:2/389: symlink d1/db/d3b/d17/d68/d76/l8b 0 2024-04-02T04:42:54.650 INFO:tasks.workunit.client.0.smithi016.stdout:8/905: dwrite d1/d70/d71/d74/fc3 [0,4194304] 0 2024-04-02T04:42:54.651 INFO:tasks.workunit.client.0.smithi016.stdout:4/896: dread d5/de/f8d [0,4194304] 0 2024-04-02T04:42:54.655 INFO:tasks.workunit.client.1.smithi027.stdout:2/390: dwrite d1/db/d3b/d17/d2b/f53 [0,4194304] 0 2024-04-02T04:42:54.655 INFO:tasks.workunit.client.0.smithi016.stdout:7/883: symlink d0/d46/da2/dc8/dda/l12a 0 2024-04-02T04:42:54.657 INFO:tasks.workunit.client.0.smithi016.stdout:7/884: chown d0/d46/d37/dbe/dae/df8/ld8 645004 1 2024-04-02T04:42:54.657 INFO:tasks.workunit.client.0.smithi016.stdout:6/912: dwrite d0/d5c/f6b [0,4194304] 0 2024-04-02T04:42:54.658 INFO:tasks.workunit.client.0.smithi016.stdout:9/979: unlink d0/d9/d4d/fb5 0 2024-04-02T04:42:54.659 INFO:tasks.workunit.client.0.smithi016.stdout:7/885: fsync d0/d46/d52/d82/fd3 0 2024-04-02T04:42:54.660 INFO:tasks.workunit.client.0.smithi016.stdout:7/886: readlink d0/d46/d52/d82/lbd 0 2024-04-02T04:42:54.662 INFO:tasks.workunit.client.0.smithi016.stdout:7/887: chown d0/d46/d52/daf/cfb 17221 1 2024-04-02T04:42:54.663 INFO:tasks.workunit.client.1.smithi027.stdout:9/415: mknod d1a/d1d/c8e 0 2024-04-02T04:42:54.666 INFO:tasks.workunit.client.0.smithi016.stdout:5/890: fsync d3/df/d18/d26/f94 0 2024-04-02T04:42:54.667 INFO:tasks.workunit.client.1.smithi027.stdout:8/446: mknod d5/d8/db/dd/d5f/d27/d31/d61/c90 0 2024-04-02T04:42:54.668 INFO:tasks.workunit.client.1.smithi027.stdout:8/447: dread - d5/f7a zero size 2024-04-02T04:42:54.668 INFO:tasks.workunit.client.1.smithi027.stdout:2/391: creat d1/db/d3b/d17/d29/f8c x:0 0 0 2024-04-02T04:42:54.669 INFO:tasks.workunit.client.1.smithi027.stdout:0/416: dread d4/fb [0,4194304] 0 2024-04-02T04:42:54.670 INFO:tasks.workunit.client.1.smithi027.stdout:4/327: link d24/d2d/f31 d24/d48/f6b 0 2024-04-02T04:42:54.672 INFO:tasks.workunit.client.1.smithi027.stdout:9/416: creat d1a/d1d/d3c/d71/f8f x:0 0 0 2024-04-02T04:42:54.674 INFO:tasks.workunit.client.0.smithi016.stdout:4/897: rename d5/de/df4/lff to d5/d16/d3f/l116 0 2024-04-02T04:42:54.677 INFO:tasks.workunit.client.1.smithi027.stdout:2/392: dread d1/db/d3b/d17/d2b/f4e [0,4194304] 0 2024-04-02T04:42:54.679 INFO:tasks.workunit.client.1.smithi027.stdout:9/417: dwrite d1a/d1d/f4b [0,4194304] 0 2024-04-02T04:42:54.680 INFO:tasks.workunit.client.0.smithi016.stdout:4/898: dwrite d5/de/f101 [0,4194304] 0 2024-04-02T04:42:54.681 INFO:tasks.workunit.client.1.smithi027.stdout:0/417: creat d4/d16/f89 x:0 0 0 2024-04-02T04:42:54.681 INFO:tasks.workunit.client.1.smithi027.stdout:2/393: truncate d1/db/d3b/d17/d29/f8c 537735 0 2024-04-02T04:42:54.682 INFO:tasks.workunit.client.0.smithi016.stdout:4/899: readlink d5/d16/df8/d7e/l107 0 2024-04-02T04:42:54.682 INFO:tasks.workunit.client.1.smithi027.stdout:9/418: chown d1a/d1d/d29/d37/d20/d7e/f85 88 1 2024-04-02T04:42:54.683 INFO:tasks.workunit.client.1.smithi027.stdout:4/328: fsync d24/f3c 0 2024-04-02T04:42:54.684 INFO:tasks.workunit.client.0.smithi016.stdout:2/961: symlink d12/d35/d3f/d4d/d136/l14f 0 2024-04-02T04:42:54.684 INFO:tasks.workunit.client.1.smithi027.stdout:2/394: write d1/db/f32 [2185786,111609] 0 2024-04-02T04:42:54.684 INFO:tasks.workunit.client.0.smithi016.stdout:2/962: chown d12/d35/d101/d4f/f68 167197436 1 2024-04-02T04:42:54.686 INFO:tasks.workunit.client.1.smithi027.stdout:2/395: dread - d1/db/d3b/d17/d29/f4f zero size 2024-04-02T04:42:54.686 INFO:tasks.workunit.client.0.smithi016.stdout:7/888: stat d0/d46/d37/dbe/dae/df8/la6 0 2024-04-02T04:42:54.689 INFO:tasks.workunit.client.0.smithi016.stdout:7/889: read d0/d46/d16/d49/f11 [3710459,90143] 0 2024-04-02T04:42:54.695 INFO:tasks.workunit.client.1.smithi027.stdout:9/419: readlink lf 0 2024-04-02T04:42:54.695 INFO:tasks.workunit.client.1.smithi027.stdout:9/420: chown d1a/d1d/d29/d37/f1c 1563 1 2024-04-02T04:42:54.700 INFO:tasks.workunit.client.0.smithi016.stdout:6/913: symlink d0/d9/d16/d56/d89/dfa/l123 0 2024-04-02T04:42:54.702 INFO:tasks.workunit.client.0.smithi016.stdout:6/914: write d0/d9/d16/d46/d5e/ff8 [201953,79775] 0 2024-04-02T04:42:54.703 INFO:tasks.workunit.client.1.smithi027.stdout:5/516: getdents d0/df/d29/da5/da6 0 2024-04-02T04:42:54.704 INFO:tasks.workunit.client.1.smithi027.stdout:0/418: dread d4/d21/d28/f48 [0,4194304] 0 2024-04-02T04:42:54.705 INFO:tasks.workunit.client.0.smithi016.stdout:6/915: chown d0/d9/d16/d56/da4/cbc 6222 1 2024-04-02T04:42:54.705 INFO:tasks.workunit.client.0.smithi016.stdout:8/906: dwrite d1/d31/d67/f79 [0,4194304] 0 2024-04-02T04:42:54.705 INFO:tasks.workunit.client.1.smithi027.stdout:0/419: truncate d4/d16/f27 482632 0 2024-04-02T04:42:54.706 INFO:tasks.workunit.client.0.smithi016.stdout:4/900: dread d5/d17/d18/d25/f6b [0,4194304] 0 2024-04-02T04:42:54.708 INFO:tasks.workunit.client.0.smithi016.stdout:7/890: unlink d0/d46/f115 0 2024-04-02T04:42:54.708 INFO:tasks.workunit.client.0.smithi016.stdout:7/891: chown d0/d46/d37/ff0 1225 1 2024-04-02T04:42:54.709 INFO:tasks.workunit.client.0.smithi016.stdout:7/892: chown d0/l88 9290230 1 2024-04-02T04:42:54.709 INFO:tasks.workunit.client.0.smithi016.stdout:7/893: write d0/d46/d37/faa [8798876,120148] 0 2024-04-02T04:42:54.710 INFO:tasks.workunit.client.1.smithi027.stdout:8/448: rmdir d5/d8/db/dd/d5f/d88 0 2024-04-02T04:42:54.712 INFO:tasks.workunit.client.1.smithi027.stdout:5/517: dwrite d0/d1a/d20/fb4 [0,4194304] 0 2024-04-02T04:42:54.713 INFO:tasks.workunit.client.0.smithi016.stdout:5/891: write d3/d40/d4f/d6b/f74 [77293,107222] 0 2024-04-02T04:42:54.713 INFO:tasks.workunit.client.0.smithi016.stdout:1/875: getdents d14/d19/daf 0 2024-04-02T04:42:54.715 INFO:tasks.workunit.client.1.smithi027.stdout:1/377: sync 2024-04-02T04:42:54.716 INFO:tasks.workunit.client.1.smithi027.stdout:5/518: read d0/d1d/f7c [409451,104948] 0 2024-04-02T04:42:54.719 INFO:tasks.workunit.client.1.smithi027.stdout:2/396: creat d1/db/d3b/d17/d68/d69/f8d x:0 0 0 2024-04-02T04:42:54.721 INFO:tasks.workunit.client.1.smithi027.stdout:0/420: creat d4/d69/f8a x:0 0 0 2024-04-02T04:42:54.722 INFO:tasks.workunit.client.0.smithi016.stdout:2/963: getdents d12/d35/d3f/d2b/de9/d102/d103 0 2024-04-02T04:42:54.723 INFO:tasks.workunit.client.1.smithi027.stdout:9/421: creat d1a/d1d/d29/d56/d8b/f90 x:0 0 0 2024-04-02T04:42:54.724 INFO:tasks.workunit.client.1.smithi027.stdout:8/449: fdatasync d5/d8/f17 0 2024-04-02T04:42:54.724 INFO:tasks.workunit.client.0.smithi016.stdout:2/964: write d12/d35/d3f/d2b/fed [1051945,102527] 0 2024-04-02T04:42:54.727 INFO:tasks.workunit.client.0.smithi016.stdout:6/916: mkdir d0/d9/d16/d56/d89/db4/d124 0 2024-04-02T04:42:54.728 INFO:tasks.workunit.client.0.smithi016.stdout:6/917: write d0/d9/d16/ffd [2715093,27027] 0 2024-04-02T04:42:54.728 INFO:tasks.workunit.client.1.smithi027.stdout:5/519: dread d0/df/d11/d41/f90 [0,4194304] 0 2024-04-02T04:42:54.731 INFO:tasks.workunit.client.1.smithi027.stdout:1/378: rename d0/de/f51 to d0/d1e/d5d/f81 0 2024-04-02T04:42:54.733 INFO:tasks.workunit.client.1.smithi027.stdout:4/329: truncate f15 5042466 0 2024-04-02T04:42:54.734 INFO:tasks.workunit.client.1.smithi027.stdout:2/397: read - d1/db/d3b/f37 zero size 2024-04-02T04:42:54.736 INFO:tasks.workunit.client.1.smithi027.stdout:2/398: write d1/db/d3b/d17/d29/f5a [2386712,26747] 0 2024-04-02T04:42:54.736 INFO:tasks.workunit.client.0.smithi016.stdout:9/980: write d0/d9/db/ff2 [2628541,89931] 0 2024-04-02T04:42:54.736 INFO:tasks.workunit.client.0.smithi016.stdout:6/918: dwrite d0/d9/d16/d56/d89/d67/d6f/f82 [0,4194304] 0 2024-04-02T04:42:54.736 INFO:tasks.workunit.client.0.smithi016.stdout:5/892: creat d3/d40/d4f/d6b/d45/f140 x:0 0 0 2024-04-02T04:42:54.737 INFO:tasks.workunit.client.1.smithi027.stdout:2/399: write d1/f4d [466293,102513] 0 2024-04-02T04:42:54.743 INFO:tasks.workunit.client.0.smithi016.stdout:2/965: chown d12/d35/d3f/d56/dab/l5d 80368 1 2024-04-02T04:42:54.744 INFO:tasks.workunit.client.1.smithi027.stdout:5/520: mkdir d0/df/d11/d13/d9a/d9c/dbe 0 2024-04-02T04:42:54.745 INFO:tasks.workunit.client.1.smithi027.stdout:5/521: readlink d0/df/d76/l83 0 2024-04-02T04:42:54.745 INFO:tasks.workunit.client.0.smithi016.stdout:7/894: dread d0/d21/f4e [0,4194304] 0 2024-04-02T04:42:54.745 INFO:tasks.workunit.client.0.smithi016.stdout:8/907: truncate d1/d6/d36/d3f/d77/f8a 1273216 0 2024-04-02T04:42:54.746 INFO:tasks.workunit.client.1.smithi027.stdout:5/522: fdatasync d0/df/d2f/d3b/d51/f8e 0 2024-04-02T04:42:54.747 INFO:tasks.workunit.client.1.smithi027.stdout:5/523: readlink d0/df/d29/da5/da6/la9 0 2024-04-02T04:42:54.747 INFO:tasks.workunit.client.0.smithi016.stdout:8/908: readlink d1/d6/d36/d3f/d40/d63/l6a 0 2024-04-02T04:42:54.748 INFO:tasks.workunit.client.0.smithi016.stdout:8/909: stat d1/d6/d36/d3f 0 2024-04-02T04:42:54.750 INFO:tasks.workunit.client.0.smithi016.stdout:9/981: fsync d0/d9/db/dc9/f57 0 2024-04-02T04:42:54.751 INFO:tasks.workunit.client.1.smithi027.stdout:0/421: truncate d4/f6 7295904 0 2024-04-02T04:42:54.752 INFO:tasks.workunit.client.1.smithi027.stdout:2/400: dread d1/db/d3b/d17/f2c [0,4194304] 0 2024-04-02T04:42:54.754 INFO:tasks.workunit.client.0.smithi016.stdout:1/876: symlink d14/d19/d1d/d45/d4f/d86/db4/d10e/l11d 0 2024-04-02T04:42:54.756 INFO:tasks.workunit.client.0.smithi016.stdout:6/919: creat d0/d9/d16/d56/d89/db4/f125 x:0 0 0 2024-04-02T04:42:54.757 INFO:tasks.workunit.client.1.smithi027.stdout:5/524: truncate d0/d5/f2c 930551 0 2024-04-02T04:42:54.758 INFO:tasks.workunit.client.0.smithi016.stdout:5/893: truncate d3/df/d18/d26/d48/f61 1240580 0 2024-04-02T04:42:54.760 INFO:tasks.workunit.client.0.smithi016.stdout:7/895: truncate d0/d46/d37/dbe/f100 5069982 0 2024-04-02T04:42:54.762 INFO:tasks.workunit.client.0.smithi016.stdout:2/966: dread d12/d35/d3b/d42/fdf [0,4194304] 0 2024-04-02T04:42:54.763 INFO:tasks.workunit.client.1.smithi027.stdout:1/379: dread d0/f6 [0,4194304] 0 2024-04-02T04:42:54.765 INFO:tasks.workunit.client.1.smithi027.stdout:2/401: creat d1/db/d3b/d17/d6d/d84/f8e x:0 0 0 2024-04-02T04:42:54.766 INFO:tasks.workunit.client.1.smithi027.stdout:2/402: chown d1/db/d3b/d61 555456 1 2024-04-02T04:42:54.770 INFO:tasks.workunit.client.1.smithi027.stdout:5/525: write d0/df/d11/d41/f90 [478909,95288] 0 2024-04-02T04:42:54.772 INFO:tasks.workunit.client.1.smithi027.stdout:0/422: symlink d4/d38/d66/l8b 0 2024-04-02T04:42:54.773 INFO:tasks.workunit.client.0.smithi016.stdout:9/982: dwrite d0/d9/db/d133/d67/fb6 [0,4194304] 0 2024-04-02T04:42:54.773 INFO:tasks.workunit.client.0.smithi016.stdout:6/920: fdatasync d0/d9/d16/d56/d59/f4b 0 2024-04-02T04:42:54.775 INFO:tasks.workunit.client.0.smithi016.stdout:8/910: write d1/d6/d36/d100/fbe [2080178,111963] 0 2024-04-02T04:42:54.777 INFO:tasks.workunit.client.1.smithi027.stdout:1/380: creat d0/d1e/d2e/d3e/f82 x:0 0 0 2024-04-02T04:42:54.777 INFO:tasks.workunit.client.0.smithi016.stdout:9/983: dread d0/d4a/d75/f117 [0,4194304] 0 2024-04-02T04:42:54.781 INFO:tasks.workunit.client.1.smithi027.stdout:2/403: symlink d1/db/d3b/d61/d6c/d8a/l8f 0 2024-04-02T04:42:54.783 INFO:tasks.workunit.client.1.smithi027.stdout:2/404: dread d1/db/d3b/d17/f2c [0,4194304] 0 2024-04-02T04:42:54.784 INFO:tasks.workunit.client.0.smithi016.stdout:9/984: dwrite d0/d4a/d75/d95/d9b/db4/df9/f116 [0,4194304] 0 2024-04-02T04:42:54.784 INFO:tasks.workunit.client.1.smithi027.stdout:8/450: getdents d5/d8 0 2024-04-02T04:42:54.786 INFO:tasks.workunit.client.0.smithi016.stdout:7/896: unlink d0/d46/f1a 0 2024-04-02T04:42:54.788 INFO:tasks.workunit.client.1.smithi027.stdout:5/526: mkdir d0/df/d11/d8c/dbf 0 2024-04-02T04:42:54.793 INFO:tasks.workunit.client.1.smithi027.stdout:5/527: write d0/df/d11/d41/f90 [121464,98175] 0 2024-04-02T04:42:54.794 INFO:tasks.workunit.client.0.smithi016.stdout:9/985: dread d0/d4a/d75/f96 [0,4194304] 0 2024-04-02T04:42:54.795 INFO:tasks.workunit.client.1.smithi027.stdout:2/405: dwrite d1/db/d3b/d17/d2b/f53 [0,4194304] 0 2024-04-02T04:42:54.795 INFO:tasks.workunit.client.1.smithi027.stdout:1/381: symlink d0/d1e/d2e/d3b/d4f/l83 0 2024-04-02T04:42:54.797 INFO:tasks.workunit.client.1.smithi027.stdout:8/451: dwrite d5/d8/db/dd/f4f [0,4194304] 0 2024-04-02T04:42:54.799 INFO:tasks.workunit.client.0.smithi016.stdout:1/877: mkdir d14/d19/d1e/d11e 0 2024-04-02T04:42:54.801 INFO:tasks.workunit.client.1.smithi027.stdout:1/382: readlink d0/d1e/d25/d3c/d64/l7f 0 2024-04-02T04:42:54.802 INFO:tasks.workunit.client.1.smithi027.stdout:1/383: chown d0/d1e/d2e/d3b/d4f/f62 1198 1 2024-04-02T04:42:54.806 INFO:tasks.workunit.client.0.smithi016.stdout:8/911: chown d1/d6/d36/d3f/d40/d45/fb7 30351197 1 2024-04-02T04:42:54.807 INFO:tasks.workunit.client.0.smithi016.stdout:8/912: fdatasync d1/d6/d36/d4f/d5d/f95 0 2024-04-02T04:42:54.809 INFO:tasks.workunit.client.1.smithi027.stdout:1/384: creat d0/d1e/d25/d2b/f84 x:0 0 0 2024-04-02T04:42:54.812 INFO:tasks.workunit.client.1.smithi027.stdout:5/528: getdents d0/df/d29/dab 0 2024-04-02T04:42:54.814 INFO:tasks.workunit.client.1.smithi027.stdout:2/406: symlink d1/db/d3b/d83/l90 0 2024-04-02T04:42:54.815 INFO:tasks.workunit.client.0.smithi016.stdout:7/897: truncate d0/d46/d16/d83/fa8 768236 0 2024-04-02T04:42:54.816 INFO:tasks.workunit.client.0.smithi016.stdout:7/898: readlink d0/d46/d37/dbe/dae/df8/d4c/l67 0 2024-04-02T04:42:54.816 INFO:tasks.workunit.client.1.smithi027.stdout:8/452: mkdir d5/d8/db/dd/d5f/d91 0 2024-04-02T04:42:54.817 INFO:tasks.workunit.client.0.smithi016.stdout:2/967: write d12/d35/f105 [129472,102873] 0 2024-04-02T04:42:54.818 INFO:tasks.workunit.client.1.smithi027.stdout:1/385: symlink d0/d1e/d2e/d3e/l85 0 2024-04-02T04:42:54.819 INFO:tasks.workunit.client.1.smithi027.stdout:1/386: chown d0/d1e/d2e/d3e/f82 6033370 1 2024-04-02T04:42:54.820 INFO:tasks.workunit.client.0.smithi016.stdout:9/986: mkdir d0/d9/db/d133/d63/d6e/d4b/db2/dba/d134 0 2024-04-02T04:42:54.822 INFO:tasks.workunit.client.1.smithi027.stdout:2/407: mkdir d1/d91 0 2024-04-02T04:42:54.823 INFO:tasks.workunit.client.0.smithi016.stdout:0/883: sync 2024-04-02T04:42:54.825 INFO:tasks.workunit.client.1.smithi027.stdout:8/453: unlink d5/d8/f7f 0 2024-04-02T04:42:54.826 INFO:tasks.workunit.client.1.smithi027.stdout:1/387: symlink d0/d1e/d2e/d3e/l86 0 2024-04-02T04:42:54.827 INFO:tasks.workunit.client.0.smithi016.stdout:6/921: write d0/d9/fe2 [2591258,104245] 0 2024-04-02T04:42:54.827 INFO:tasks.workunit.client.1.smithi027.stdout:1/388: fdatasync d0/d1e/d2e/f4a 0 2024-04-02T04:42:54.828 INFO:tasks.workunit.client.0.smithi016.stdout:6/922: chown d0/d9/d16/f3c 276828133 1 2024-04-02T04:42:54.828 INFO:tasks.workunit.client.1.smithi027.stdout:5/529: readlink d0/d1d/d61/d7e/la3 0 2024-04-02T04:42:54.829 INFO:tasks.workunit.client.1.smithi027.stdout:1/389: write d0/d1e/d2e/d3b/f78 [1012656,116656] 0 2024-04-02T04:42:54.833 INFO:tasks.workunit.client.0.smithi016.stdout:5/894: link d3/d40/f60 d3/d40/d4f/d6b/d45/da6/d102/dc4/dd7/f141 0 2024-04-02T04:42:54.836 INFO:tasks.workunit.client.1.smithi027.stdout:5/530: dwrite d0/fb0 [0,4194304] 0 2024-04-02T04:42:54.840 INFO:tasks.workunit.client.1.smithi027.stdout:1/390: mknod d0/d1e/d5d/c87 0 2024-04-02T04:42:54.841 INFO:tasks.workunit.client.1.smithi027.stdout:1/391: chown d0/de/d52/l5b 3 1 2024-04-02T04:42:54.842 INFO:tasks.workunit.client.0.smithi016.stdout:9/987: creat d0/d9/db/d133/d63/d6e/d4b/f135 x:0 0 0 2024-04-02T04:42:54.844 INFO:tasks.workunit.client.0.smithi016.stdout:1/878: symlink d14/d19/d1d/d29/d39/df0/d106/l11f 0 2024-04-02T04:42:54.849 INFO:tasks.workunit.client.1.smithi027.stdout:8/454: symlink d5/l92 0 2024-04-02T04:42:54.851 INFO:tasks.workunit.client.1.smithi027.stdout:5/531: mkdir d0/df/d2f/d3b/d51/dc0 0 2024-04-02T04:42:54.852 INFO:tasks.workunit.client.0.smithi016.stdout:8/913: symlink d1/d6/d36/d3f/d77/d11a/l128 0 2024-04-02T04:42:54.853 INFO:tasks.workunit.client.1.smithi027.stdout:1/392: rmdir d0 39 2024-04-02T04:42:54.855 INFO:tasks.workunit.client.1.smithi027.stdout:5/532: dread d0/d1d/d61/d72/f81 [0,4194304] 0 2024-04-02T04:42:54.857 INFO:tasks.workunit.client.1.smithi027.stdout:5/533: chown d0/df/d11/d37/f40 1110915041 1 2024-04-02T04:42:54.858 INFO:tasks.workunit.client.1.smithi027.stdout:2/408: dwrite d1/db/d3b/f70 [0,4194304] 0 2024-04-02T04:42:54.858 INFO:tasks.workunit.client.1.smithi027.stdout:3/347: sync 2024-04-02T04:42:54.858 INFO:tasks.workunit.client.1.smithi027.stdout:6/378: sync 2024-04-02T04:42:54.858 INFO:tasks.workunit.client.1.smithi027.stdout:7/389: sync 2024-04-02T04:42:54.859 INFO:tasks.workunit.client.0.smithi016.stdout:2/968: write d12/d35/d101/d76/f92 [1942282,5184] 0 2024-04-02T04:42:54.859 INFO:tasks.workunit.client.0.smithi016.stdout:5/895: dread d3/df/d18/d2f/fd6 [0,4194304] 0 2024-04-02T04:42:54.860 INFO:tasks.workunit.client.1.smithi027.stdout:7/390: fsync d2/d1c/f4b 0 2024-04-02T04:42:54.862 INFO:tasks.workunit.client.1.smithi027.stdout:1/393: chown d0/d1e/d25/d2b/f7b 2 1 2024-04-02T04:42:54.862 INFO:tasks.workunit.client.1.smithi027.stdout:6/379: dread - d13/d1b/d24/d28/d3c/f77 zero size 2024-04-02T04:42:54.862 INFO:tasks.workunit.client.1.smithi027.stdout:5/534: rmdir d0/d68 39 2024-04-02T04:42:54.866 INFO:tasks.workunit.client.1.smithi027.stdout:3/348: rename d1/de/df/d13/f4e to d1/d1a/f79 0 2024-04-02T04:42:54.870 INFO:tasks.workunit.client.0.smithi016.stdout:0/884: write d0/d16/d23/f3f [130539,53722] 0 2024-04-02T04:42:54.871 INFO:tasks.workunit.client.1.smithi027.stdout:1/394: stat d0/f5 0 2024-04-02T04:42:54.871 INFO:tasks.workunit.client.0.smithi016.stdout:0/885: readlink d0/d16/l42 0 2024-04-02T04:42:54.872 INFO:tasks.workunit.client.1.smithi027.stdout:5/535: dread d0/f6 [0,4194304] 0 2024-04-02T04:42:54.873 INFO:tasks.workunit.client.1.smithi027.stdout:8/455: rmdir d5/d8/db/dd/d5f 39 2024-04-02T04:42:54.873 INFO:tasks.workunit.client.1.smithi027.stdout:2/409: dwrite d1/db/d3b/d17/d6d/d84/f8e [0,4194304] 0 2024-04-02T04:42:54.877 INFO:tasks.workunit.client.1.smithi027.stdout:7/391: fsync d2/d7/da/dc/d1f/d33/f55 0 2024-04-02T04:42:54.878 INFO:tasks.workunit.client.1.smithi027.stdout:7/392: fsync d2/d7/da/dc/d1f/d33/f71 0 2024-04-02T04:42:54.880 INFO:tasks.workunit.client.1.smithi027.stdout:6/380: rmdir d13/d1b/d24/d25/d27 39 2024-04-02T04:42:54.882 INFO:tasks.workunit.client.0.smithi016.stdout:1/879: dwrite d14/d19/d1e/ff4 [0,4194304] 0 2024-04-02T04:42:54.883 INFO:tasks.workunit.client.0.smithi016.stdout:2/969: creat d12/d35/d3f/d2b/f150 x:0 0 0 2024-04-02T04:42:54.884 INFO:tasks.workunit.client.0.smithi016.stdout:5/896: unlink d3/df/d18/ff2 0 2024-04-02T04:42:54.887 INFO:tasks.workunit.client.1.smithi027.stdout:1/395: creat d0/d1e/d2e/d3e/f88 x:0 0 0 2024-04-02T04:42:54.888 INFO:tasks.workunit.client.0.smithi016.stdout:6/923: dwrite d0/d9/d16/d56/da4/fea [4194304,4194304] 0 2024-04-02T04:42:54.891 INFO:tasks.workunit.client.1.smithi027.stdout:5/536: read d0/d1a/d20/f73 [611733,88085] 0 2024-04-02T04:42:54.895 INFO:tasks.workunit.client.1.smithi027.stdout:2/410: creat d1/db/d3b/d61/d75/f92 x:0 0 0 2024-04-02T04:42:54.897 INFO:tasks.workunit.client.1.smithi027.stdout:7/393: write d2/d7/da/dc/d16/d3f/d50/f66 [538959,84176] 0 2024-04-02T04:42:54.898 INFO:tasks.workunit.client.0.smithi016.stdout:2/970: fdatasync d12/d35/d101/f114 0 2024-04-02T04:42:54.899 INFO:tasks.workunit.client.1.smithi027.stdout:7/394: chown d2/d7/da/dc/d16/d2f/f73 92 1 2024-04-02T04:42:54.899 INFO:tasks.workunit.client.0.smithi016.stdout:0/886: dread d0/d16/d24/d4e/d8d/d71/f7d [0,4194304] 0 2024-04-02T04:42:54.901 INFO:tasks.workunit.client.1.smithi027.stdout:6/381: rename d13/d1b/d24/c45 to d13/d1b/d48/d75/c7a 0 2024-04-02T04:42:54.902 INFO:tasks.workunit.client.1.smithi027.stdout:3/349: creat d1/de/df/d13/d73/d77/f7a x:0 0 0 2024-04-02T04:42:54.904 INFO:tasks.workunit.client.0.smithi016.stdout:5/897: creat d3/d40/d4f/d67/f142 x:0 0 0 2024-04-02T04:42:54.905 INFO:tasks.workunit.client.0.smithi016.stdout:0/887: dwrite d0/d16/d2e/d36/f10f [0,4194304] 0 2024-04-02T04:42:54.907 INFO:tasks.workunit.client.0.smithi016.stdout:8/914: creat d1/d70/f129 x:0 0 0 2024-04-02T04:42:54.909 INFO:tasks.workunit.client.0.smithi016.stdout:6/924: creat d0/d9/d16/f126 x:0 0 0 2024-04-02T04:42:54.911 INFO:tasks.workunit.client.0.smithi016.stdout:9/988: getdents d0/d3/d39 0 2024-04-02T04:42:54.911 INFO:tasks.workunit.client.0.smithi016.stdout:6/925: dread - d0/d9/d16/d56/d89/db4/ffc zero size 2024-04-02T04:42:54.911 INFO:tasks.workunit.client.0.smithi016.stdout:8/915: dread d1/d6/d36/d3f/d40/d45/fe2 [0,4194304] 0 2024-04-02T04:42:54.918 INFO:tasks.workunit.client.1.smithi027.stdout:7/395: unlink d2/df/f10 0 2024-04-02T04:42:54.919 INFO:tasks.workunit.client.1.smithi027.stdout:5/537: write d0/d1d/f7c [1086991,74876] 0 2024-04-02T04:42:54.921 INFO:tasks.workunit.client.1.smithi027.stdout:2/411: rename d1/db/d3b/d17/d29/f8c to d1/db/d3b/d61/f93 0 2024-04-02T04:42:54.923 INFO:tasks.workunit.client.1.smithi027.stdout:6/382: symlink d13/d1b/d24/d25/d42/l7b 0 2024-04-02T04:42:54.925 INFO:tasks.workunit.client.0.smithi016.stdout:2/971: write d12/d35/d3f/d2b/f3d [528028,60958] 0 2024-04-02T04:42:54.926 INFO:tasks.workunit.client.1.smithi027.stdout:7/396: dwrite d2/d7/da/dc/d1f/d61/f77 [0,4194304] 0 2024-04-02T04:42:54.926 INFO:tasks.workunit.client.1.smithi027.stdout:3/350: symlink d1/de/d55/l7b 0 2024-04-02T04:42:54.926 INFO:tasks.workunit.client.0.smithi016.stdout:2/972: write d12/d35/d101/f114 [3552086,43308] 0 2024-04-02T04:42:54.928 INFO:tasks.workunit.client.0.smithi016.stdout:2/973: write d12/d35/d3f/d2b/f150 [716062,12427] 0 2024-04-02T04:42:54.928 INFO:tasks.workunit.client.1.smithi027.stdout:7/397: truncate d2/d7/da/d35/f67 765486 0 2024-04-02T04:42:54.929 INFO:tasks.workunit.client.0.smithi016.stdout:5/898: rmdir d3/d40/db6/de0 39 2024-04-02T04:42:54.936 INFO:tasks.workunit.client.1.smithi027.stdout:8/456: creat d5/d8/db/dd/d5f/d27/d31/d5b/d8d/f93 x:0 0 0 2024-04-02T04:42:54.937 INFO:tasks.workunit.client.0.smithi016.stdout:6/926: truncate d0/d9/f7a 2995720 0 2024-04-02T04:42:54.939 INFO:tasks.workunit.client.0.smithi016.stdout:6/927: read d0/d78/fa0 [3871765,85018] 0 2024-04-02T04:42:54.939 INFO:tasks.workunit.client.0.smithi016.stdout:6/928: fsync d0/d6e/f107 0 2024-04-02T04:42:54.940 INFO:tasks.workunit.client.0.smithi016.stdout:1/880: rename d14/d19/d11a to d14/d19/d1d/d29/d39/d120 0 2024-04-02T04:42:54.940 INFO:tasks.workunit.client.0.smithi016.stdout:9/989: rename d0/d9 to d0/d9/db/d133/d136 22 2024-04-02T04:42:54.941 INFO:tasks.workunit.client.0.smithi016.stdout:6/929: chown d0/d9/d16/d56/d89/db4/ffc 1934 1 2024-04-02T04:42:54.943 INFO:tasks.workunit.client.1.smithi027.stdout:2/412: creat d1/db/d3b/d17/d68/f94 x:0 0 0 2024-04-02T04:42:54.944 INFO:tasks.workunit.client.0.smithi016.stdout:8/916: write d1/d6/d36/d3f/f6c [4879825,39410] 0 2024-04-02T04:42:54.945 INFO:tasks.workunit.client.0.smithi016.stdout:8/917: chown d1/d6/d36/d3f/d40/d45/fee 4909 1 2024-04-02T04:42:54.947 INFO:tasks.workunit.client.1.smithi027.stdout:7/398: dread d2/d7/da/f1e [0,4194304] 0 2024-04-02T04:42:54.950 INFO:tasks.workunit.client.1.smithi027.stdout:3/351: rename d1/cd to d1/de/df/d13/c7c 0 2024-04-02T04:42:54.950 INFO:tasks.workunit.client.1.smithi027.stdout:7/399: rename d2 to d2/d7/da/dc/d16/d3f/d78 22 2024-04-02T04:42:54.952 INFO:tasks.workunit.client.0.smithi016.stdout:2/974: write d12/d35/d101/d4f/dfa/f11b [847557,124177] 0 2024-04-02T04:42:54.954 INFO:tasks.workunit.client.1.smithi027.stdout:8/457: creat d5/d8/db/dd/d5f/d27/d31/d61/f94 x:0 0 0 2024-04-02T04:42:54.958 INFO:tasks.workunit.client.0.smithi016.stdout:5/899: dwrite d3/d40/d4f/d6b/d45/da6/d102/dc4/d108/fe2 [0,4194304] 0 2024-04-02T04:42:54.958 INFO:tasks.workunit.client.1.smithi027.stdout:6/383: symlink d13/d1b/d24/d28/d46/l7c 0 2024-04-02T04:42:54.962 INFO:tasks.workunit.client.0.smithi016.stdout:8/918: chown d1/d70/c123 2078532566 1 2024-04-02T04:42:54.971 INFO:tasks.workunit.client.1.smithi027.stdout:2/413: mknod d1/db/d3b/d17/c95 0 2024-04-02T04:42:54.972 INFO:tasks.workunit.client.1.smithi027.stdout:7/400: write d2/d7/da/dc/d16/d3f/d44/f6a [1830807,90849] 0 2024-04-02T04:42:54.973 INFO:tasks.workunit.client.1.smithi027.stdout:2/414: chown d1/db/d3b/d17/d2b/c43 7316322 1 2024-04-02T04:42:54.974 INFO:tasks.workunit.client.0.smithi016.stdout:1/881: truncate d14/f56 1509939 0 2024-04-02T04:42:54.974 INFO:tasks.workunit.client.0.smithi016.stdout:1/882: readlink l1 0 2024-04-02T04:42:54.975 INFO:tasks.workunit.client.0.smithi016.stdout:1/883: chown d14/d19/d1d/d29/d4c/c5a 712 1 2024-04-02T04:42:54.975 INFO:tasks.workunit.client.1.smithi027.stdout:2/415: fsync d1/db/f32 0 2024-04-02T04:42:54.977 INFO:tasks.workunit.client.0.smithi016.stdout:5/900: mkdir d3/d40/d4f/d67/dbf/d104/d143 0 2024-04-02T04:42:54.978 INFO:tasks.workunit.client.1.smithi027.stdout:2/416: fsync d1/f13 0 2024-04-02T04:42:54.978 INFO:tasks.workunit.client.1.smithi027.stdout:8/458: write d5/d8/db/dd/d5f/f7e [425075,72662] 0 2024-04-02T04:42:54.978 INFO:tasks.workunit.client.1.smithi027.stdout:1/396: link d0/d1e/d2e/l36 d0/l89 0 2024-04-02T04:42:54.978 INFO:tasks.workunit.client.0.smithi016.stdout:5/901: stat d3/d40/db6/fd9 0 2024-04-02T04:42:54.978 INFO:tasks.workunit.client.0.smithi016.stdout:5/902: dread - d3/df/d18/f13b zero size 2024-04-02T04:42:54.979 INFO:tasks.workunit.client.1.smithi027.stdout:3/352: dwrite d1/de/df/d13/f2d [0,4194304] 0 2024-04-02T04:42:54.983 INFO:tasks.workunit.client.1.smithi027.stdout:5/538: link d0/d5/l5f d0/df/d11/d13/d9a/db7/lc1 0 2024-04-02T04:42:54.988 INFO:tasks.workunit.client.1.smithi027.stdout:2/417: dwrite d1/db/d3b/f70 [0,4194304] 0 2024-04-02T04:42:54.990 INFO:tasks.workunit.client.0.smithi016.stdout:9/990: creat d0/d4a/d75/f137 x:0 0 0 2024-04-02T04:42:54.991 INFO:tasks.workunit.client.0.smithi016.stdout:6/930: truncate d0/d9/d16/d56/d89/d67/d6f/f82 640453 0 2024-04-02T04:42:54.994 INFO:tasks.workunit.client.0.smithi016.stdout:5/903: mknod d3/d40/d4f/d6b/d63/c144 0 2024-04-02T04:42:54.995 INFO:tasks.workunit.client.0.smithi016.stdout:2/975: link d12/d70/lfe d12/d35/d101/d4f/l151 0 2024-04-02T04:42:54.998 INFO:tasks.workunit.client.0.smithi016.stdout:9/991: rename d0/d9/db/d133/d63/d6e/d4b/d58 to d0/d4a/d99/d46/dd6/d138 0 2024-04-02T04:42:54.999 INFO:tasks.workunit.client.0.smithi016.stdout:6/931: dwrite d0/d9/d16/d56/da4/f101 [0,4194304] 0 2024-04-02T04:42:55.001 INFO:tasks.workunit.client.1.smithi027.stdout:8/459: rename d5/d8/db/dd/d5f/d27/d31/d61 to d5/d8/db/d68/d95 0 2024-04-02T04:42:55.003 INFO:tasks.workunit.client.1.smithi027.stdout:6/384: truncate d13/f14 1533492 0 2024-04-02T04:42:55.004 INFO:tasks.workunit.client.0.smithi016.stdout:1/884: mknod d14/d19/d1e/d71/c121 0 2024-04-02T04:42:55.007 INFO:tasks.workunit.client.1.smithi027.stdout:2/418: unlink d1/db/l59 0 2024-04-02T04:42:55.008 INFO:tasks.workunit.client.1.smithi027.stdout:2/419: fdatasync d1/db/d3b/d17/d12/f62 0 2024-04-02T04:42:55.009 INFO:tasks.workunit.client.0.smithi016.stdout:8/919: creat d1/d70/d71/f12a x:0 0 0 2024-04-02T04:42:55.014 INFO:tasks.workunit.client.1.smithi027.stdout:3/353: symlink d1/de/d55/d63/l7d 0 2024-04-02T04:42:55.015 INFO:tasks.workunit.client.0.smithi016.stdout:2/976: symlink d12/d35/d3b/d42/d59/d91/l152 0 2024-04-02T04:42:55.016 INFO:tasks.workunit.client.0.smithi016.stdout:2/977: stat d12/d9f/d107/d126/f14b 0 2024-04-02T04:42:55.016 INFO:tasks.workunit.client.0.smithi016.stdout:2/978: stat d12/d35/d3b/d42/d67/fb3 0 2024-04-02T04:42:55.017 INFO:tasks.workunit.client.1.smithi027.stdout:5/539: rename d0/f4f to d0/df/d11/d65/fc2 0 2024-04-02T04:42:55.021 INFO:tasks.workunit.client.0.smithi016.stdout:9/992: creat d0/d9/db/d133/d63/dd2/f139 x:0 0 0 2024-04-02T04:42:55.021 INFO:tasks.workunit.client.0.smithi016.stdout:4/901: sync 2024-04-02T04:42:55.021 INFO:tasks.workunit.client.0.smithi016.stdout:9/993: chown d0/d9/c5a 14 1 2024-04-02T04:42:55.022 INFO:tasks.workunit.client.1.smithi027.stdout:6/385: creat d13/d1b/d48/d75/f7d x:0 0 0 2024-04-02T04:42:55.022 INFO:tasks.workunit.client.1.smithi027.stdout:6/386: write f12 [248406,17630] 0 2024-04-02T04:42:55.029 INFO:tasks.workunit.client.0.smithi016.stdout:6/932: dwrite d0/faa [0,4194304] 0 2024-04-02T04:42:55.031 INFO:tasks.workunit.client.1.smithi027.stdout:6/387: dwrite d13/d1b/d48/d75/f7d [0,4194304] 0 2024-04-02T04:42:55.031 INFO:tasks.workunit.client.0.smithi016.stdout:8/920: mknod d1/d6/d36/d4f/d5d/c12b 0 2024-04-02T04:42:55.032 INFO:tasks.workunit.client.1.smithi027.stdout:7/401: truncate d2/d7/f48 469763 0 2024-04-02T04:42:55.035 INFO:tasks.workunit.client.1.smithi027.stdout:3/354: chown d1/de/df/d13/c7c 129 1 2024-04-02T04:42:55.038 INFO:tasks.workunit.client.1.smithi027.stdout:5/540: mkdir d0/d5/d55/dc3 0 2024-04-02T04:42:55.040 INFO:tasks.workunit.client.0.smithi016.stdout:2/979: truncate d12/d35/d3b/d42/d59/f116 58165 0 2024-04-02T04:42:55.044 INFO:tasks.workunit.client.1.smithi027.stdout:8/460: truncate d5/d8/db/dd/f1f 2031972 0 2024-04-02T04:42:55.045 INFO:tasks.workunit.client.0.smithi016.stdout:7/899: sync 2024-04-02T04:42:55.049 INFO:tasks.workunit.client.0.smithi016.stdout:4/902: dwrite d5/d1d/d5f/d95/df2/f106 [0,4194304] 0 2024-04-02T04:42:55.050 INFO:tasks.workunit.client.1.smithi027.stdout:7/402: mkdir d2/d7/da/dc/d16/d3f/d79 0 2024-04-02T04:42:55.051 INFO:tasks.workunit.client.0.smithi016.stdout:4/903: dread - d5/d8/ffe zero size 2024-04-02T04:42:55.051 INFO:tasks.workunit.client.1.smithi027.stdout:9/422: sync 2024-04-02T04:42:55.052 INFO:tasks.workunit.client.0.smithi016.stdout:8/921: write d1/d70/f88 [51811,122758] 0 2024-04-02T04:42:55.052 INFO:tasks.workunit.client.1.smithi027.stdout:3/355: creat d1/de/df/d13/d73/d77/f7e x:0 0 0 2024-04-02T04:42:55.052 INFO:tasks.workunit.client.0.smithi016.stdout:5/904: mknod d3/d129/d13c/c145 0 2024-04-02T04:42:55.055 INFO:tasks.workunit.client.0.smithi016.stdout:1/885: dwrite d14/d19/d1d/d29/d39/d46/dcb/d43/f6f [0,4194304] 0 2024-04-02T04:42:55.055 INFO:tasks.workunit.client.1.smithi027.stdout:2/420: dwrite d1/db/d3b/f4c [0,4194304] 0 2024-04-02T04:42:55.055 INFO:tasks.workunit.client.0.smithi016.stdout:8/922: chown d1/d31/d67/fd1 15 1 2024-04-02T04:42:55.057 INFO:tasks.workunit.client.0.smithi016.stdout:1/886: chown d14/d19/d1d/d45/d4f/d86/db4/lde 61 1 2024-04-02T04:42:55.057 INFO:tasks.workunit.client.1.smithi027.stdout:2/421: chown d1/db/d3b/d17/d6d/d84 12455948 1 2024-04-02T04:42:55.059 INFO:tasks.workunit.client.0.smithi016.stdout:2/980: write d12/d35/d3b/f132 [1664354,770] 0 2024-04-02T04:42:55.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:54 smithi016 ceph-mon[28969]: pgmap v26: 65 pgs: 65 active+clean; 2.4 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 61 MiB/s rd, 238 MiB/s wr, 405 op/s 2024-04-02T04:42:55.059 INFO:tasks.workunit.client.1.smithi027.stdout:9/423: write d1a/d1d/d29/d37/d20/f61 [3404013,44986] 0 2024-04-02T04:42:55.064 INFO:tasks.workunit.client.1.smithi027.stdout:7/403: dwrite d2/d7/da/f2a [0,4194304] 0 2024-04-02T04:42:55.070 INFO:tasks.workunit.client.0.smithi016.stdout:9/994: symlink d0/d9/db/d133/d63/d6e/d4b/db2/dba/d134/l13a 0 2024-04-02T04:42:55.070 INFO:tasks.workunit.client.1.smithi027.stdout:5/541: dread d0/d5/f63 [0,4194304] 0 2024-04-02T04:42:55.077 INFO:tasks.workunit.client.0.smithi016.stdout:6/933: write d0/d9/d16/d56/d89/d67/d6d/fc3 [454665,100961] 0 2024-04-02T04:42:55.078 INFO:tasks.workunit.client.1.smithi027.stdout:3/356: rmdir d1/de/d55/d63 39 2024-04-02T04:42:55.081 INFO:tasks.workunit.client.0.smithi016.stdout:4/904: rmdir d5/de/dd5 39 2024-04-02T04:42:55.085 INFO:tasks.workunit.client.1.smithi027.stdout:9/424: dread - d1a/d1d/d29/d37/d20/f30 zero size 2024-04-02T04:42:55.086 INFO:tasks.workunit.client.0.smithi016.stdout:0/888: sync 2024-04-02T04:42:55.087 INFO:tasks.workunit.client.1.smithi027.stdout:9/425: truncate d1a/d1d/d29/d50/d73/f7b 150425 0 2024-04-02T04:42:55.089 INFO:tasks.workunit.client.1.smithi027.stdout:8/461: mkdir d5/d8/d15/d71/d96 0 2024-04-02T04:42:55.094 INFO:tasks.workunit.client.0.smithi016.stdout:9/995: mknod d0/d9/db/d133/d63/d6e/d4b/db2/c13b 0 2024-04-02T04:42:55.095 INFO:tasks.workunit.client.0.smithi016.stdout:9/996: chown d0/d4a/d75/d95/d9b/db4/f114 471850 1 2024-04-02T04:42:55.095 INFO:tasks.workunit.client.0.smithi016.stdout:1/887: dread d14/d19/f6b [0,4194304] 0 2024-04-02T04:42:55.096 INFO:tasks.workunit.client.1.smithi027.stdout:3/357: mkdir d1/de/df/d13/d73/d77/d7f 0 2024-04-02T04:42:55.096 INFO:tasks.workunit.client.0.smithi016.stdout:2/981: dread d12/d35/d3f/d2b/de9/d102/f11e [0,4194304] 0 2024-04-02T04:42:55.098 INFO:tasks.workunit.client.1.smithi027.stdout:7/404: dread d2/d7/da/dc/d16/d3f/d44/f6a [0,4194304] 0 2024-04-02T04:42:55.099 INFO:tasks.workunit.client.1.smithi027.stdout:7/405: read - d2/d1d/d57/f6d zero size 2024-04-02T04:42:55.100 INFO:tasks.workunit.client.0.smithi016.stdout:4/905: stat d5/d17/d18/d6d/d8e/d9c/dcc/d6f/fc5 0 2024-04-02T04:42:55.100 INFO:tasks.workunit.client.1.smithi027.stdout:7/406: readlink d2/d7/da/dc/d16/d2f/l3c 0 2024-04-02T04:42:55.102 INFO:tasks.workunit.client.0.smithi016.stdout:0/889: dread - d0/d16/d24/d4e/fcc zero size 2024-04-02T04:42:55.104 INFO:tasks.workunit.client.0.smithi016.stdout:7/900: rename d0/d46/d16/dc2/dea/d64/c68 to d0/c12b 0 2024-04-02T04:42:55.106 INFO:tasks.workunit.client.0.smithi016.stdout:1/888: mknod d14/d19/daf/c122 0 2024-04-02T04:42:55.107 INFO:tasks.workunit.client.0.smithi016.stdout:9/997: mkdir d0/d9/d4d/da8/d13c 0 2024-04-02T04:42:55.108 INFO:tasks.workunit.client.1.smithi027.stdout:9/426: write d1a/d1d/d29/d37/d20/f2b [238971,27413] 0 2024-04-02T04:42:55.108 INFO:tasks.workunit.client.0.smithi016.stdout:8/923: fsync d1/d70/d71/fb3 0 2024-04-02T04:42:55.110 INFO:tasks.workunit.client.0.smithi016.stdout:6/934: dwrite d0/d9/d16/d46/f68 [0,4194304] 0 2024-04-02T04:42:55.112 INFO:tasks.workunit.client.1.smithi027.stdout:5/542: rename d0/d1d/d61/d72/f81 to d0/d1d/d61/fc4 0 2024-04-02T04:42:55.114 INFO:tasks.workunit.client.0.smithi016.stdout:4/906: unlink d5/de/dd5/ff7 0 2024-04-02T04:42:55.114 INFO:tasks.workunit.client.1.smithi027.stdout:7/407: dwrite d2/d7/da/dc/d16/d3f/d44/f6a [0,4194304] 0 2024-04-02T04:42:55.115 INFO:tasks.workunit.client.1.smithi027.stdout:3/358: creat d1/de/d74/f80 x:0 0 0 2024-04-02T04:42:55.116 INFO:tasks.workunit.client.0.smithi016.stdout:0/890: mkdir d0/d16/d24/d4e/d8d/d71/df1/d124 0 2024-04-02T04:42:55.119 INFO:tasks.workunit.client.0.smithi016.stdout:5/905: rename d3/df/d18/d2f/fd6 to d3/df/d18/d26/d70/f146 0 2024-04-02T04:42:55.120 INFO:tasks.workunit.client.0.smithi016.stdout:7/901: symlink d0/d46/d37/dbe/dae/df8/d84/l12c 0 2024-04-02T04:42:55.121 INFO:tasks.workunit.client.0.smithi016.stdout:7/902: chown d0/d46/f9a 2 1 2024-04-02T04:42:55.122 INFO:tasks.workunit.client.0.smithi016.stdout:1/889: dread - d14/da9/f101 zero size 2024-04-02T04:42:55.125 INFO:tasks.workunit.client.1.smithi027.stdout:3/359: dwrite d1/de/df/d13/d73/d77/f7e [0,4194304] 0 2024-04-02T04:42:55.125 INFO:tasks.workunit.client.0.smithi016.stdout:9/998: mknod d0/d9/db/d133/d63/d6e/d4b/c13d 0 2024-04-02T04:42:55.132 INFO:tasks.workunit.client.1.smithi027.stdout:8/462: getdents d5/d8/db 0 2024-04-02T04:42:55.134 INFO:tasks.workunit.client.0.smithi016.stdout:8/924: mknod d1/d6/d86/c12c 0 2024-04-02T04:42:55.134 INFO:tasks.workunit.client.0.smithi016.stdout:8/925: readlink d1/d6/l9 0 2024-04-02T04:42:55.135 INFO:tasks.workunit.client.0.smithi016.stdout:8/926: fsync d1/d6/d36/d3f/d40/d45/fb9 0 2024-04-02T04:42:55.136 INFO:tasks.workunit.client.0.smithi016.stdout:6/935: symlink d0/d9/df/l127 0 2024-04-02T04:42:55.137 INFO:tasks.workunit.client.0.smithi016.stdout:8/927: chown d1/d6/d36/d3f/d40/d49/f43 0 1 2024-04-02T04:42:55.143 INFO:tasks.workunit.client.0.smithi016.stdout:7/903: creat d0/d87/f12d x:0 0 0 2024-04-02T04:42:55.143 INFO:tasks.workunit.client.1.smithi027.stdout:5/543: write d0/d5/fbc [4178544,100147] 0 2024-04-02T04:42:55.145 INFO:tasks.workunit.client.1.smithi027.stdout:5/544: chown d0/df/d11/d37/c48 495 1 2024-04-02T04:42:55.145 INFO:tasks.workunit.client.0.smithi016.stdout:1/890: dread - d14/da9/fd1 zero size 2024-04-02T04:42:55.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:54 smithi027 ceph-mon[37550]: pgmap v26: 65 pgs: 65 active+clean; 2.4 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 61 MiB/s rd, 238 MiB/s wr, 405 op/s 2024-04-02T04:42:55.146 INFO:tasks.workunit.client.1.smithi027.stdout:7/408: write d2/d1d/f25 [834712,113832] 0 2024-04-02T04:42:55.146 INFO:tasks.workunit.client.1.smithi027.stdout:5/545: chown d0/df/d11/d41/l70 8383 1 2024-04-02T04:42:55.147 INFO:tasks.workunit.client.1.smithi027.stdout:3/360: mknod d1/de/d55/d63/c81 0 2024-04-02T04:42:55.149 INFO:tasks.workunit.client.0.smithi016.stdout:9/999: creat d0/d4a/d75/d95/d9b/f13e x:0 0 0 2024-04-02T04:42:55.149 INFO:tasks.workunit.client.1.smithi027.stdout:9/427: link d1a/d1d/d29/d37/d4c/c6a d1a/d1d/d29/d37/c91 0 2024-04-02T04:42:55.151 INFO:tasks.workunit.client.0.smithi016.stdout:2/982: link d12/d35/fea d12/d9f/d135/f153 0 2024-04-02T04:42:55.153 INFO:tasks.workunit.client.0.smithi016.stdout:0/891: write d0/d16/f7a [2059741,4018] 0 2024-04-02T04:42:55.153 INFO:tasks.workunit.client.0.smithi016.stdout:0/892: dread - d0/d16/f8a zero size 2024-04-02T04:42:55.154 INFO:tasks.workunit.client.0.smithi016.stdout:5/906: write d3/d40/d4f/d6b/d45/da6/d102/ff6 [140895,82955] 0 2024-04-02T04:42:55.156 INFO:tasks.workunit.client.1.smithi027.stdout:3/361: unlink d1/f10 0 2024-04-02T04:42:55.157 INFO:tasks.workunit.client.1.smithi027.stdout:3/362: dread - d1/de/df/d61/f68 zero size 2024-04-02T04:42:55.160 INFO:tasks.workunit.client.0.smithi016.stdout:6/936: mkdir d0/d6e/d128 0 2024-04-02T04:42:55.160 INFO:tasks.workunit.client.1.smithi027.stdout:7/409: unlink d2/d7/da/dc/d1f/c4f 0 2024-04-02T04:42:55.161 INFO:tasks.workunit.client.0.smithi016.stdout:4/907: write d5/d1d/d2d/f29 [3115547,92676] 0 2024-04-02T04:42:55.166 INFO:tasks.workunit.client.1.smithi027.stdout:3/363: dwrite d1/de/d55/f5f [4194304,4194304] 0 2024-04-02T04:42:55.167 INFO:tasks.workunit.client.0.smithi016.stdout:7/904: mknod d0/d46/d16/d49/c12e 0 2024-04-02T04:42:55.170 INFO:tasks.workunit.client.1.smithi027.stdout:5/546: dread d0/df/d11/d65/fc2 [0,4194304] 0 2024-04-02T04:42:55.170 INFO:tasks.workunit.client.0.smithi016.stdout:1/891: dread d14/f83 [0,4194304] 0 2024-04-02T04:42:55.170 INFO:tasks.workunit.client.1.smithi027.stdout:9/428: symlink d1a/l92 0 2024-04-02T04:42:55.171 INFO:tasks.workunit.client.0.smithi016.stdout:1/892: readlink d14/d19/d1d/d29/d39/d46/dcb/d43/lf8 0 2024-04-02T04:42:55.172 INFO:tasks.workunit.client.1.smithi027.stdout:7/410: dwrite d2/d1c/f37 [0,4194304] 0 2024-04-02T04:42:55.173 INFO:tasks.workunit.client.0.smithi016.stdout:5/907: rename d3/d40/d4f/d6b/d63/c66 to d3/dbd/c147 0 2024-04-02T04:42:55.178 INFO:tasks.workunit.client.1.smithi027.stdout:9/429: fsync d1a/d1d/d29/d51/f60 0 2024-04-02T04:42:55.181 INFO:tasks.workunit.client.0.smithi016.stdout:7/905: creat d0/d46/da2/dc8/dda/f12f x:0 0 0 2024-04-02T04:42:55.181 INFO:tasks.workunit.client.0.smithi016.stdout:7/906: chown d0/d21/f10f 3 1 2024-04-02T04:42:55.182 INFO:tasks.workunit.client.1.smithi027.stdout:5/547: write d0/df/d2f/d3b/f3d [4994933,114179] 0 2024-04-02T04:42:55.183 INFO:tasks.workunit.client.1.smithi027.stdout:7/411: creat d2/d1c/f7a x:0 0 0 2024-04-02T04:42:55.184 INFO:tasks.workunit.client.1.smithi027.stdout:7/412: chown d2/d7/da/dc/d1f/d33/f71 117835 1 2024-04-02T04:42:55.185 INFO:tasks.workunit.client.1.smithi027.stdout:3/364: symlink d1/de/df/d13/d73/d77/d7f/l82 0 2024-04-02T04:42:55.186 INFO:tasks.workunit.client.0.smithi016.stdout:2/983: write d12/d35/d3b/d42/d59/d91/ff6 [389363,110014] 0 2024-04-02T04:42:55.187 INFO:tasks.workunit.client.1.smithi027.stdout:7/413: write d2/d7/da/dc/d1f/d33/f71 [851499,65075] 0 2024-04-02T04:42:55.188 INFO:tasks.workunit.client.1.smithi027.stdout:3/365: truncate d1/de/df/d61/f68 62973 0 2024-04-02T04:42:55.190 INFO:tasks.workunit.client.1.smithi027.stdout:4/330: sync 2024-04-02T04:42:55.192 INFO:tasks.workunit.client.0.smithi016.stdout:2/984: dwrite d12/d35/d3b/d42/d59/f88 [0,4194304] 0 2024-04-02T04:42:55.193 INFO:tasks.workunit.client.1.smithi027.stdout:7/414: creat d2/d7/da/dc/f7b x:0 0 0 2024-04-02T04:42:55.194 INFO:tasks.workunit.client.0.smithi016.stdout:2/985: fsync d12/d35/d3f/f4b 0 2024-04-02T04:42:55.195 INFO:tasks.workunit.client.0.smithi016.stdout:6/937: dread d0/d9/d16/f8d [0,4194304] 0 2024-04-02T04:42:55.196 INFO:tasks.workunit.client.0.smithi016.stdout:6/938: stat d0/d6e/d11b 0 2024-04-02T04:42:55.197 INFO:tasks.workunit.client.1.smithi027.stdout:9/430: dread d1a/d1d/d29/f46 [0,4194304] 0 2024-04-02T04:42:55.198 INFO:tasks.workunit.client.1.smithi027.stdout:7/415: dread d2/d1c/f37 [0,4194304] 0 2024-04-02T04:42:55.199 INFO:tasks.workunit.client.0.smithi016.stdout:4/908: creat d5/d1d/d1e/f117 x:0 0 0 2024-04-02T04:42:55.199 INFO:tasks.workunit.client.0.smithi016.stdout:0/893: write d0/d16/d23/d3e/ffa [689085,73573] 0 2024-04-02T04:42:55.199 INFO:tasks.workunit.client.0.smithi016.stdout:4/909: fsync d5/d8/fd 0 2024-04-02T04:42:55.203 INFO:tasks.workunit.client.1.smithi027.stdout:4/331: dread d24/f33 [0,4194304] 0 2024-04-02T04:42:55.203 INFO:tasks.workunit.client.0.smithi016.stdout:2/986: creat d12/d35/d3b/d42/d59/d91/d10a/f154 x:0 0 0 2024-04-02T04:42:55.204 INFO:tasks.workunit.client.1.smithi027.stdout:4/332: stat f12 0 2024-04-02T04:42:55.206 INFO:tasks.workunit.client.1.smithi027.stdout:4/333: read f21 [1276941,7116] 0 2024-04-02T04:42:55.206 INFO:tasks.workunit.client.0.smithi016.stdout:6/939: truncate d0/d9/d16/d56/faf 809761 0 2024-04-02T04:42:55.207 INFO:tasks.workunit.client.0.smithi016.stdout:1/893: dwrite d14/d19/d20/f38 [4194304,4194304] 0 2024-04-02T04:42:55.208 INFO:tasks.workunit.client.1.smithi027.stdout:4/334: write d24/f27 [123195,42281] 0 2024-04-02T04:42:55.208 INFO:tasks.workunit.client.1.smithi027.stdout:4/335: chown d24/f28 29190 1 2024-04-02T04:42:55.211 INFO:tasks.workunit.client.0.smithi016.stdout:4/910: read - d5/d16/d88/fde zero size 2024-04-02T04:42:55.212 INFO:tasks.workunit.client.1.smithi027.stdout:5/548: dread d0/d68/d6b/d74/fb6 [0,4194304] 0 2024-04-02T04:42:55.212 INFO:tasks.workunit.client.1.smithi027.stdout:5/549: stat d0/f66 0 2024-04-02T04:42:55.212 INFO:tasks.workunit.client.0.smithi016.stdout:2/987: write d12/d9f/fc9 [714142,30917] 0 2024-04-02T04:42:55.214 INFO:tasks.workunit.client.0.smithi016.stdout:0/894: dwrite d0/d16/d23/d3e/fc6 [0,4194304] 0 2024-04-02T04:42:55.215 INFO:tasks.workunit.client.1.smithi027.stdout:7/416: write d2/f13 [4651790,122138] 0 2024-04-02T04:42:55.216 INFO:tasks.workunit.client.0.smithi016.stdout:4/911: read d5/d8/d43/f4b [7816605,25852] 0 2024-04-02T04:42:55.217 INFO:tasks.workunit.client.0.smithi016.stdout:7/907: dread d0/d46/fe6 [0,4194304] 0 2024-04-02T04:42:55.217 INFO:tasks.workunit.client.1.smithi027.stdout:4/336: truncate f1e 2749157 0 2024-04-02T04:42:55.219 INFO:tasks.workunit.client.1.smithi027.stdout:7/417: dread d2/d7/da/dc/d1f/f47 [0,4194304] 0 2024-04-02T04:42:55.222 INFO:tasks.workunit.client.0.smithi016.stdout:4/912: truncate d5/d17/d18/d6d/d8e/ffc 483816 0 2024-04-02T04:42:55.224 INFO:tasks.workunit.client.0.smithi016.stdout:4/913: dread - d5/d8/ffe zero size 2024-04-02T04:42:55.224 INFO:tasks.workunit.client.0.smithi016.stdout:0/895: creat d0/d16/dd6/ddd/d121/f125 x:0 0 0 2024-04-02T04:42:55.224 INFO:tasks.workunit.client.0.smithi016.stdout:1/894: dwrite f2 [0,4194304] 0 2024-04-02T04:42:55.225 INFO:tasks.workunit.client.1.smithi027.stdout:7/418: creat d2/d7/da/d35/f7c x:0 0 0 2024-04-02T04:42:55.226 INFO:tasks.workunit.client.0.smithi016.stdout:6/940: mknod d0/d9/d16/d56/d89/c129 0 2024-04-02T04:42:55.226 INFO:tasks.workunit.client.1.smithi027.stdout:4/337: creat d24/d38/d47/d62/f6c x:0 0 0 2024-04-02T04:42:55.227 INFO:tasks.workunit.client.0.smithi016.stdout:4/914: write d5/de/f101 [4640278,85844] 0 2024-04-02T04:42:55.228 INFO:tasks.workunit.client.1.smithi027.stdout:4/338: write d24/d38/d47/d62/f6c [286713,34277] 0 2024-04-02T04:42:55.230 INFO:tasks.workunit.client.0.smithi016.stdout:7/908: dread d0/d46/d37/dbe/dae/d9d/fe2 [0,4194304] 0 2024-04-02T04:42:55.231 INFO:tasks.workunit.client.1.smithi027.stdout:5/550: link d0/df/d11/d37/f4e d0/fc5 0 2024-04-02T04:42:55.232 INFO:tasks.workunit.client.0.smithi016.stdout:2/988: mknod d12/d35/d3f/d2b/da3/db1/d11f/c155 0 2024-04-02T04:42:55.232 INFO:tasks.workunit.client.0.smithi016.stdout:6/941: write d0/d9/d16/d56/d89/d67/d6f/f11c [561751,99466] 0 2024-04-02T04:42:55.232 INFO:tasks.workunit.client.1.smithi027.stdout:5/551: readlink d0/d5/l94 0 2024-04-02T04:42:55.233 INFO:tasks.workunit.client.1.smithi027.stdout:7/419: readlink d2/d7/da/dc/d16/d3a/l76 0 2024-04-02T04:42:55.233 INFO:tasks.workunit.client.1.smithi027.stdout:5/552: chown d0/d5/l95 25466402 1 2024-04-02T04:42:55.234 INFO:tasks.workunit.client.0.smithi016.stdout:2/989: stat d12/d35/d101/d4f/dfa/f128 0 2024-04-02T04:42:55.236 INFO:tasks.workunit.client.1.smithi027.stdout:4/339: rename d24/d38/f3e to d24/d2d/f6d 0 2024-04-02T04:42:55.236 INFO:tasks.workunit.client.1.smithi027.stdout:7/420: rename d2/d7 to d2/d7/da/dc/d16/d3f/d44/d7d 22 2024-04-02T04:42:55.237 INFO:tasks.workunit.client.0.smithi016.stdout:0/896: dwrite d0/d16/d24/d4e/d8d/d71/f72 [0,4194304] 0 2024-04-02T04:42:55.239 INFO:tasks.workunit.client.0.smithi016.stdout:4/915: dread - d5/d17/dcd/fce zero size 2024-04-02T04:42:55.242 INFO:tasks.workunit.client.0.smithi016.stdout:6/942: symlink d0/d5c/l12a 0 2024-04-02T04:42:55.242 INFO:tasks.workunit.client.0.smithi016.stdout:1/895: dread d14/d19/d20/f62 [0,4194304] 0 2024-04-02T04:42:55.245 INFO:tasks.workunit.client.1.smithi027.stdout:5/553: creat d0/df/d11/d13/d9a/db7/fc6 x:0 0 0 2024-04-02T04:42:55.247 INFO:tasks.workunit.client.0.smithi016.stdout:2/990: unlink d12/d35/d3f/c4a 0 2024-04-02T04:42:55.247 INFO:tasks.workunit.client.0.smithi016.stdout:2/991: stat d12/d35/d3b/d42/d8d/lec 0 2024-04-02T04:42:55.249 INFO:tasks.workunit.client.0.smithi016.stdout:4/916: unlink d5/de/l6a 0 2024-04-02T04:42:55.251 INFO:tasks.workunit.client.1.smithi027.stdout:5/554: dread d0/d1d/f7c [0,4194304] 0 2024-04-02T04:42:55.253 INFO:tasks.workunit.client.0.smithi016.stdout:0/897: rename d0/d16/d24/d62/d9c/f59 to d0/d16/d11e/f126 0 2024-04-02T04:42:55.256 INFO:tasks.workunit.client.1.smithi027.stdout:7/421: rename d2/d7/da/dc/d1f/c70 to d2/c7e 0 2024-04-02T04:42:55.261 INFO:tasks.workunit.client.1.smithi027.stdout:5/555: creat d0/df/d76/fc7 x:0 0 0 2024-04-02T04:42:55.263 INFO:tasks.workunit.client.1.smithi027.stdout:7/422: creat d2/d7/da/dc/d1f/d33/d36/f7f x:0 0 0 2024-04-02T04:42:55.264 INFO:tasks.workunit.client.1.smithi027.stdout:7/423: truncate d2/f52 1027423 0 2024-04-02T04:42:55.264 INFO:tasks.workunit.client.0.smithi016.stdout:2/992: mknod d12/d35/d101/d4f/dfa/c156 0 2024-04-02T04:42:55.265 INFO:tasks.workunit.client.0.smithi016.stdout:7/909: write d0/d46/d37/dbe/dae/df8/d43/d96/fa5 [295599,118798] 0 2024-04-02T04:42:55.266 INFO:tasks.workunit.client.0.smithi016.stdout:4/917: mkdir d5/de/dd5/d118 0 2024-04-02T04:42:55.271 INFO:tasks.workunit.client.1.smithi027.stdout:7/424: creat d2/d1d/f80 x:0 0 0 2024-04-02T04:42:55.273 INFO:tasks.workunit.client.1.smithi027.stdout:5/556: dread d0/df/d11/d13/f1e [0,4194304] 0 2024-04-02T04:42:55.273 INFO:tasks.workunit.client.1.smithi027.stdout:7/425: write d2/d7/da/dc/d1f/d33/d36/f7f [975694,57894] 0 2024-04-02T04:42:55.274 INFO:tasks.workunit.client.1.smithi027.stdout:5/557: chown d0/f6 476 1 2024-04-02T04:42:55.280 INFO:tasks.workunit.client.1.smithi027.stdout:5/558: mkdir d0/df/d11/d65/dac/dc8 0 2024-04-02T04:42:55.282 INFO:tasks.workunit.client.0.smithi016.stdout:6/943: truncate d0/d9/d16/d46/f7b 1735163 0 2024-04-02T04:42:55.284 INFO:tasks.workunit.client.1.smithi027.stdout:5/559: mknod d0/df/d11/d8c/cc9 0 2024-04-02T04:42:55.284 INFO:tasks.workunit.client.0.smithi016.stdout:1/896: dwrite d14/d19/d1d/d29/f3d [0,4194304] 0 2024-04-02T04:42:55.287 INFO:tasks.workunit.client.0.smithi016.stdout:4/918: rename d5/d17/d18/d6d/d8e/d9c/dcc/c82 to d5/d1d/d5f/d95/df2/c119 0 2024-04-02T04:42:55.288 INFO:tasks.workunit.client.0.smithi016.stdout:0/898: read d0/d16/d2e/d53/d6f/f74 [1021428,4417] 0 2024-04-02T04:42:55.293 INFO:tasks.workunit.client.1.smithi027.stdout:7/426: dread d2/d7/f18 [0,4194304] 0 2024-04-02T04:42:55.298 INFO:tasks.workunit.client.1.smithi027.stdout:7/427: mkdir d2/d7/da/dc/d16/d81 0 2024-04-02T04:42:55.299 INFO:tasks.workunit.client.0.smithi016.stdout:4/919: symlink d5/d16/df8/dca/l11a 0 2024-04-02T04:42:55.299 INFO:tasks.workunit.client.1.smithi027.stdout:7/428: write d2/d7/da/dc/d1f/d33/f71 [794673,31961] 0 2024-04-02T04:42:55.302 INFO:tasks.workunit.client.0.smithi016.stdout:4/920: creat d5/d1d/d1e/da5/df9/f11b x:0 0 0 2024-04-02T04:42:55.304 INFO:tasks.workunit.client.0.smithi016.stdout:0/899: getdents d0/d16 0 2024-04-02T04:42:55.305 INFO:tasks.workunit.client.0.smithi016.stdout:0/900: write d0/d16/dd6/ddd/fff [199613,111028] 0 2024-04-02T04:42:55.305 INFO:tasks.workunit.client.0.smithi016.stdout:0/901: chown d0/d16/d23 11416944 1 2024-04-02T04:42:55.308 INFO:tasks.workunit.client.0.smithi016.stdout:0/902: dread - d0/d16/d24/d62/fe5 zero size 2024-04-02T04:42:55.308 INFO:tasks.workunit.client.0.smithi016.stdout:2/993: write d12/d35/d3b/f57 [367387,4769] 0 2024-04-02T04:42:55.311 INFO:tasks.workunit.client.0.smithi016.stdout:8/928: sync 2024-04-02T04:42:55.311 INFO:tasks.workunit.client.0.smithi016.stdout:5/908: sync 2024-04-02T04:42:55.311 INFO:tasks.workunit.client.1.smithi027.stdout:5/560: truncate d0/d1a/d20/f87 206669 0 2024-04-02T04:42:55.312 INFO:tasks.workunit.client.1.smithi027.stdout:5/561: chown d0/df/d11/d8c 97143 1 2024-04-02T04:42:55.312 INFO:tasks.workunit.client.0.smithi016.stdout:0/903: mknod d0/d16/d2e/d53/d9d/c127 0 2024-04-02T04:42:55.314 INFO:tasks.workunit.client.1.smithi027.stdout:5/562: write d0/df/d11/d65/f99 [545706,10653] 0 2024-04-02T04:42:55.315 INFO:tasks.workunit.client.0.smithi016.stdout:8/929: read - d1/d6/d36/d4f/d5d/fe7 zero size 2024-04-02T04:42:55.317 INFO:tasks.workunit.client.1.smithi027.stdout:5/563: mkdir d0/df/d2f/d3b/dca 0 2024-04-02T04:42:55.318 INFO:tasks.workunit.client.0.smithi016.stdout:0/904: readlink d0/d16/d2e/d53/d6f/l119 0 2024-04-02T04:42:55.318 INFO:tasks.workunit.client.0.smithi016.stdout:7/910: dwrite d0/d46/d37/dbe/dae/df8/dec/f11a [0,4194304] 0 2024-04-02T04:42:55.319 INFO:tasks.workunit.client.1.smithi027.stdout:5/564: readlink d0/df/d76/l83 0 2024-04-02T04:42:55.319 INFO:tasks.workunit.client.1.smithi027.stdout:7/429: truncate d2/d7/da/dc/d16/d3f/f43 1662735 0 2024-04-02T04:42:55.320 INFO:tasks.workunit.client.0.smithi016.stdout:2/994: write d12/d26/f5a [241419,76888] 0 2024-04-02T04:42:55.324 INFO:tasks.workunit.client.0.smithi016.stdout:8/930: unlink d1/d31/d67/ca2 0 2024-04-02T04:42:55.329 INFO:tasks.workunit.client.0.smithi016.stdout:6/944: dwrite d0/d9/d16/d56/fcb [0,4194304] 0 2024-04-02T04:42:55.330 INFO:tasks.workunit.client.0.smithi016.stdout:1/897: dwrite d14/d19/d1d/f7d [0,4194304] 0 2024-04-02T04:42:55.334 INFO:tasks.workunit.client.0.smithi016.stdout:2/995: write d12/d35/d3f/f34 [9406366,127643] 0 2024-04-02T04:42:55.336 INFO:tasks.workunit.client.0.smithi016.stdout:7/911: rename d0/l10 to d0/d21/l130 0 2024-04-02T04:42:55.336 INFO:tasks.workunit.client.0.smithi016.stdout:4/921: dwrite d5/d1d/d1e/da5/df9/f111 [0,4194304] 0 2024-04-02T04:42:55.339 INFO:tasks.workunit.client.1.smithi027.stdout:7/430: rename d2/d7/da/dc/d16/d2f/f41 to d2/f82 0 2024-04-02T04:42:55.344 INFO:tasks.workunit.client.0.smithi016.stdout:1/898: truncate d14/d19/d1e/d71/f104 2019708 0 2024-04-02T04:42:55.345 INFO:tasks.workunit.client.1.smithi027.stdout:5/565: truncate d0/df/d11/d13/f28 1208303 0 2024-04-02T04:42:55.345 INFO:tasks.workunit.client.0.smithi016.stdout:5/909: dwrite d3/d40/d4f/d6b/d45/f51 [0,4194304] 0 2024-04-02T04:42:55.346 INFO:tasks.workunit.client.0.smithi016.stdout:0/905: dwrite d0/d16/d2e/d36/d9f/f120 [0,4194304] 0 2024-04-02T04:42:55.346 INFO:tasks.workunit.client.1.smithi027.stdout:5/566: truncate d0/d68/fa4 3208934 0 2024-04-02T04:42:55.347 INFO:tasks.workunit.client.1.smithi027.stdout:5/567: chown d0/d1d/d61/fc4 10 1 2024-04-02T04:42:55.347 INFO:tasks.workunit.client.0.smithi016.stdout:6/945: read d0/f37 [4228410,36842] 0 2024-04-02T04:42:55.352 INFO:tasks.workunit.client.0.smithi016.stdout:8/931: creat d1/d6/d36/d3f/d40/d45/dcc/f12d x:0 0 0 2024-04-02T04:42:55.352 INFO:tasks.workunit.client.1.smithi027.stdout:7/431: truncate d2/d1d/f5f 418595 0 2024-04-02T04:42:55.355 INFO:tasks.workunit.client.1.smithi027.stdout:5/568: mknod d0/d68/ccb 0 2024-04-02T04:42:55.356 INFO:tasks.workunit.client.0.smithi016.stdout:4/922: unlink d5/d17/d18/da0/leb 0 2024-04-02T04:42:55.357 INFO:tasks.workunit.client.1.smithi027.stdout:5/569: chown d0/df/d11/d65/dac/dc8 0 1 2024-04-02T04:42:55.359 INFO:tasks.workunit.client.0.smithi016.stdout:5/910: read - d3/d40/d4f/fcc zero size 2024-04-02T04:42:55.359 INFO:tasks.workunit.client.0.smithi016.stdout:0/906: write d0/d16/d2e/d36/f10f [2769642,11327] 0 2024-04-02T04:42:55.360 INFO:tasks.workunit.client.1.smithi027.stdout:7/432: creat d2/d7/da/dc/f83 x:0 0 0 2024-04-02T04:42:55.360 INFO:tasks.workunit.client.1.smithi027.stdout:7/433: chown d2/d7/da 35242 1 2024-04-02T04:42:55.362 INFO:tasks.workunit.client.0.smithi016.stdout:8/932: chown d1/c75 24 1 2024-04-02T04:42:55.362 INFO:tasks.workunit.client.0.smithi016.stdout:2/996: dwrite d12/d35/d3b/d42/f65 [0,4194304] 0 2024-04-02T04:42:55.366 INFO:tasks.workunit.client.1.smithi027.stdout:7/434: rename d2/f52 to d2/d7/da/dc/d1f/f84 0 2024-04-02T04:42:55.366 INFO:tasks.workunit.client.0.smithi016.stdout:6/946: dwrite d0/d9/d16/f23 [0,4194304] 0 2024-04-02T04:42:55.367 INFO:tasks.workunit.client.0.smithi016.stdout:0/907: read d0/f104 [121438,43532] 0 2024-04-02T04:42:55.367 INFO:tasks.workunit.client.1.smithi027.stdout:7/435: chown d2/d7/da/dc/d1f/c63 176637571 1 2024-04-02T04:42:55.370 INFO:tasks.workunit.client.1.smithi027.stdout:7/436: chown d2/d7/da/dc/d1f/d61 4 1 2024-04-02T04:42:55.370 INFO:tasks.workunit.client.1.smithi027.stdout:5/570: dread d0/d5/f2c [0,4194304] 0 2024-04-02T04:42:55.372 INFO:tasks.workunit.client.1.smithi027.stdout:7/437: stat d2/d7/da/dc/d16/d3a/d4d/d6f/l75 0 2024-04-02T04:42:55.375 INFO:tasks.workunit.client.0.smithi016.stdout:1/899: creat d14/d19/d1d/d29/d39/f123 x:0 0 0 2024-04-02T04:42:55.379 INFO:tasks.workunit.client.0.smithi016.stdout:2/997: truncate d12/d35/fea 304711 0 2024-04-02T04:42:55.383 INFO:tasks.workunit.client.0.smithi016.stdout:6/947: creat d0/d6e/f12b x:0 0 0 2024-04-02T04:42:55.384 INFO:tasks.workunit.client.0.smithi016.stdout:6/948: chown d0/d9/d16/d56/da4/c106 18282033 1 2024-04-02T04:42:55.385 INFO:tasks.workunit.client.0.smithi016.stdout:5/911: dread d3/f17 [0,4194304] 0 2024-04-02T04:42:55.386 INFO:tasks.workunit.client.0.smithi016.stdout:5/912: chown d3/d40/d4f/d6b/d45 106993712 1 2024-04-02T04:42:55.386 INFO:tasks.workunit.client.0.smithi016.stdout:7/912: write d0/d46/d16/dc2/dea/f42 [2704484,121327] 0 2024-04-02T04:42:55.387 INFO:tasks.workunit.client.0.smithi016.stdout:0/908: symlink d0/d4f/d11b/l128 0 2024-04-02T04:42:55.388 INFO:tasks.workunit.client.1.smithi027.stdout:0/423: sync 2024-04-02T04:42:55.389 INFO:tasks.workunit.client.0.smithi016.stdout:1/900: mknod d14/d19/d1e/d11e/c124 0 2024-04-02T04:42:55.390 INFO:tasks.workunit.client.0.smithi016.stdout:2/998: mkdir d12/d35/d101/d76/dcf/d148/d157 0 2024-04-02T04:42:55.391 INFO:tasks.workunit.client.0.smithi016.stdout:2/999: readlink d12/d35/d3b/d42/d59/d91/l152 0 2024-04-02T04:42:55.393 INFO:tasks.workunit.client.1.smithi027.stdout:0/424: mkdir d4/d21/d28/d4e/d8c 0 2024-04-02T04:42:55.393 INFO:tasks.workunit.client.1.smithi027.stdout:0/425: stat d4/d21/c25 0 2024-04-02T04:42:55.393 INFO:tasks.workunit.client.1.smithi027.stdout:0/426: readlink d4/d38/d74/l88 0 2024-04-02T04:42:55.395 INFO:tasks.workunit.client.0.smithi016.stdout:4/923: write d5/d17/d18/d6d/d8e/d9c/dcc/fa8 [5079039,2705] 0 2024-04-02T04:42:55.396 INFO:tasks.workunit.client.0.smithi016.stdout:0/909: mknod d0/d16/d2e/d53/da4/d102/c129 0 2024-04-02T04:42:55.398 INFO:tasks.workunit.client.0.smithi016.stdout:1/901: symlink d14/d19/d7a/l125 0 2024-04-02T04:42:55.400 INFO:tasks.workunit.client.1.smithi027.stdout:0/427: mknod d4/d21/c8d 0 2024-04-02T04:42:55.401 INFO:tasks.workunit.client.1.smithi027.stdout:5/571: truncate d0/d1a/d20/fb4 3109984 0 2024-04-02T04:42:55.401 INFO:tasks.workunit.client.1.smithi027.stdout:0/428: chown d4/d38/d66/d2d/f56 15 1 2024-04-02T04:42:55.401 INFO:tasks.workunit.client.0.smithi016.stdout:8/933: write d1/d6/d36/d3f/d40/d49/fd5 [266824,70338] 0 2024-04-02T04:42:55.402 INFO:tasks.workunit.client.0.smithi016.stdout:8/934: chown d1/d31/d67/c72 732943 1 2024-04-02T04:42:55.403 INFO:tasks.workunit.client.0.smithi016.stdout:8/935: chown d1/d70/d71/d74/fc3 13576 1 2024-04-02T04:42:55.403 INFO:tasks.workunit.client.0.smithi016.stdout:8/936: readlink d1/d6/d36/d4f/dc9/dfa/l117 0 2024-04-02T04:42:55.405 INFO:tasks.workunit.client.0.smithi016.stdout:4/924: mknod d5/d17/d18/d1f/c11c 0 2024-04-02T04:42:55.406 INFO:tasks.workunit.client.1.smithi027.stdout:0/429: stat d4/l33 0 2024-04-02T04:42:55.407 INFO:tasks.workunit.client.0.smithi016.stdout:8/937: dread d1/d6/d36/d3f/d40/d45/fe2 [0,4194304] 0 2024-04-02T04:42:55.409 INFO:tasks.workunit.client.1.smithi027.stdout:5/572: unlink d0/d1a/d20/f87 0 2024-04-02T04:42:55.410 INFO:tasks.workunit.client.0.smithi016.stdout:5/913: write d3/d40/d4f/fc6 [615819,91663] 0 2024-04-02T04:42:55.410 INFO:tasks.workunit.client.0.smithi016.stdout:6/949: link d0/d9/d16/c49 d0/d9/d16/de3/c12c 0 2024-04-02T04:42:55.411 INFO:tasks.workunit.client.1.smithi027.stdout:5/573: chown d0/fc5 136346521 1 2024-04-02T04:42:55.411 INFO:tasks.workunit.client.1.smithi027.stdout:0/430: readlink d4/d16/d1b/l43 0 2024-04-02T04:42:55.412 INFO:tasks.workunit.client.1.smithi027.stdout:5/574: chown d0/l1 714417126 1 2024-04-02T04:42:55.412 INFO:tasks.workunit.client.0.smithi016.stdout:5/914: write d3/d40/d4f/d6b/d63/f10b [708076,89268] 0 2024-04-02T04:42:55.413 INFO:tasks.workunit.client.1.smithi027.stdout:0/431: chown d4/f6 2451116 1 2024-04-02T04:42:55.413 INFO:tasks.workunit.client.0.smithi016.stdout:8/938: dwrite d1/d6/d36/d3f/d40/d49/f2a [0,4194304] 0 2024-04-02T04:42:55.415 INFO:tasks.workunit.client.0.smithi016.stdout:8/939: write d1/d31/f7e [4247372,115681] 0 2024-04-02T04:42:55.415 INFO:tasks.workunit.client.0.smithi016.stdout:0/910: mkdir d0/d16/d24/d12a 0 2024-04-02T04:42:55.416 INFO:tasks.workunit.client.1.smithi027.stdout:5/575: symlink d0/df/d2f/d3b/d51/lcc 0 2024-04-02T04:42:55.416 INFO:tasks.workunit.client.1.smithi027.stdout:5/576: chown d0/df/d11/d8c 30498 1 2024-04-02T04:42:55.417 INFO:tasks.workunit.client.1.smithi027.stdout:5/577: read d0/d5/f63 [1254001,63863] 0 2024-04-02T04:42:55.418 INFO:tasks.workunit.client.0.smithi016.stdout:8/940: write d1/d70/d71/d74/fbd [1024352,49479] 0 2024-04-02T04:42:55.425 INFO:tasks.workunit.client.0.smithi016.stdout:0/911: dwrite d0/d4f/f118 [0,4194304] 0 2024-04-02T04:42:55.427 INFO:tasks.workunit.client.0.smithi016.stdout:7/913: getdents d0/d46/dbf 0 2024-04-02T04:42:55.427 INFO:tasks.workunit.client.0.smithi016.stdout:7/914: chown d0/d46/d16/d49 88945 1 2024-04-02T04:42:55.430 INFO:tasks.workunit.client.0.smithi016.stdout:5/915: dread - d3/d40/d72/f95 zero size 2024-04-02T04:42:55.432 INFO:tasks.workunit.client.0.smithi016.stdout:4/925: dwrite d5/f7 [0,4194304] 0 2024-04-02T04:42:55.433 INFO:tasks.workunit.client.1.smithi027.stdout:0/432: getdents d4/d38/d66/d2d 0 2024-04-02T04:42:55.435 INFO:tasks.workunit.client.0.smithi016.stdout:4/926: read d5/d16/df8/d7e/fcb [1132144,109354] 0 2024-04-02T04:42:55.435 INFO:tasks.workunit.client.0.smithi016.stdout:6/950: write d0/d9/d16/d56/d59/f96 [724504,16499] 0 2024-04-02T04:42:55.435 INFO:tasks.workunit.client.1.smithi027.stdout:5/578: link d0/c9 d0/df/d76/ccd 0 2024-04-02T04:42:55.436 INFO:tasks.workunit.client.0.smithi016.stdout:4/927: chown d5/f102 1 1 2024-04-02T04:42:55.437 INFO:tasks.workunit.client.0.smithi016.stdout:1/902: creat d14/d19/d1d/d29/f126 x:0 0 0 2024-04-02T04:42:55.438 INFO:tasks.workunit.client.0.smithi016.stdout:8/941: mkdir d1/d6/d36/d3f/d120/d12e 0 2024-04-02T04:42:55.439 INFO:tasks.workunit.client.1.smithi027.stdout:0/433: read - d4/d21/d28/f54 zero size 2024-04-02T04:42:55.440 INFO:tasks.workunit.client.0.smithi016.stdout:8/942: truncate d1/d70/d71/fb3 1038726 0 2024-04-02T04:42:55.441 INFO:tasks.workunit.client.1.smithi027.stdout:0/434: read d4/d3d/f47 [2208999,114144] 0 2024-04-02T04:42:55.443 INFO:tasks.workunit.client.0.smithi016.stdout:7/915: creat d0/d46/d16/dc2/f131 x:0 0 0 2024-04-02T04:42:55.444 INFO:tasks.workunit.client.0.smithi016.stdout:5/916: write d3/d40/d4f/d6b/d45/da6/d102/dc4/dd7/fd2 [473806,48395] 0 2024-04-02T04:42:55.445 INFO:tasks.workunit.client.1.smithi027.stdout:0/435: symlink d4/d38/l8e 0 2024-04-02T04:42:55.445 INFO:tasks.workunit.client.0.smithi016.stdout:6/951: unlink d0/d6e/f81 0 2024-04-02T04:42:55.447 INFO:tasks.workunit.client.0.smithi016.stdout:1/903: fdatasync d14/d19/f4b 0 2024-04-02T04:42:55.448 INFO:tasks.workunit.client.0.smithi016.stdout:1/904: truncate d14/d19/d1e/d6d/fe8 1651276 0 2024-04-02T04:42:55.448 INFO:tasks.workunit.client.0.smithi016.stdout:4/928: write d5/d16/df8/fc0 [2776950,12957] 0 2024-04-02T04:42:55.450 INFO:tasks.workunit.client.0.smithi016.stdout:8/943: mkdir d1/d6/d36/d100/d12f 0 2024-04-02T04:42:55.451 INFO:tasks.workunit.client.0.smithi016.stdout:1/905: truncate d14/d19/d1d/d29/d39/f123 207754 0 2024-04-02T04:42:55.452 INFO:tasks.workunit.client.0.smithi016.stdout:8/944: truncate d1/d6/d36/d3f/d40/d49/fd5 1067540 0 2024-04-02T04:42:55.452 INFO:tasks.workunit.client.1.smithi027.stdout:0/436: creat d4/d21/f8f x:0 0 0 2024-04-02T04:42:55.454 INFO:tasks.workunit.client.1.smithi027.stdout:5/579: write d0/f69 [1358445,122805] 0 2024-04-02T04:42:55.455 INFO:tasks.workunit.client.1.smithi027.stdout:5/580: chown d0/d5/f2c 130850188 1 2024-04-02T04:42:55.455 INFO:tasks.workunit.client.0.smithi016.stdout:5/917: creat d3/df/d18/dfd/d122/f148 x:0 0 0 2024-04-02T04:42:55.455 INFO:tasks.workunit.client.0.smithi016.stdout:6/952: rmdir d0/d9/d16/d56/d89/dc5/dce 39 2024-04-02T04:42:55.456 INFO:tasks.workunit.client.1.smithi027.stdout:5/581: read d0/df/d11/d65/fc2 [523161,1199] 0 2024-04-02T04:42:55.457 INFO:tasks.workunit.client.0.smithi016.stdout:7/916: dread d0/d46/d37/dbe/dae/df8/f7d [0,4194304] 0 2024-04-02T04:42:55.458 INFO:tasks.workunit.client.0.smithi016.stdout:1/906: dwrite f2 [0,4194304] 0 2024-04-02T04:42:55.459 INFO:tasks.workunit.client.0.smithi016.stdout:4/929: mknod d5/d1d/d6c/c11d 0 2024-04-02T04:42:55.459 INFO:tasks.workunit.client.0.smithi016.stdout:6/953: read d0/d9/d16/d56/d89/f45 [1947929,61620] 0 2024-04-02T04:42:55.462 INFO:tasks.workunit.client.0.smithi016.stdout:6/954: write d0/d9/d16/d56/da4/f101 [1078368,29117] 0 2024-04-02T04:42:55.462 INFO:tasks.workunit.client.1.smithi027.stdout:0/437: rename d4/d38/d66/f18 to d4/f90 0 2024-04-02T04:42:55.464 INFO:tasks.workunit.client.0.smithi016.stdout:1/907: truncate d14/d19/f6b 233548 0 2024-04-02T04:42:55.466 INFO:tasks.workunit.client.1.smithi027.stdout:5/582: rename d0/d5 to d0/df/d11/d13/dce 0 2024-04-02T04:42:55.469 INFO:tasks.workunit.client.0.smithi016.stdout:0/912: link d0/d16/fe8 d0/d16/d23/d3e/db3/f12b 0 2024-04-02T04:42:55.473 INFO:tasks.workunit.client.1.smithi027.stdout:5/583: dread d0/d1a/d20/f73 [0,4194304] 0 2024-04-02T04:42:55.477 INFO:tasks.workunit.client.0.smithi016.stdout:1/908: rename d14/d19/d1d/d45/d4f/d86/fa6 to d14/d19/d50/f127 0 2024-04-02T04:42:55.479 INFO:tasks.workunit.client.0.smithi016.stdout:1/909: stat d14/d19/d1d/d29/d39/d46/dcb/d43/d68/dc1/dfa/f10d 0 2024-04-02T04:42:55.480 INFO:tasks.workunit.client.0.smithi016.stdout:7/917: mkdir d0/d46/d37/dbe/dae/df8/d4c/d9e/d132 0 2024-04-02T04:42:55.482 INFO:tasks.workunit.client.0.smithi016.stdout:0/913: unlink d0/d16/d2e/d53/d6f/f74 0 2024-04-02T04:42:55.484 INFO:tasks.workunit.client.1.smithi027.stdout:0/438: write d4/d38/d66/f2a [2044896,116253] 0 2024-04-02T04:42:55.488 INFO:tasks.workunit.client.1.smithi027.stdout:0/439: creat d4/d38/f91 x:0 0 0 2024-04-02T04:42:55.491 INFO:tasks.workunit.client.1.smithi027.stdout:5/584: dwrite d0/f47 [0,4194304] 0 2024-04-02T04:42:55.494 INFO:tasks.workunit.client.1.smithi027.stdout:0/440: symlink d4/d38/d66/d4a/l92 0 2024-04-02T04:42:55.496 INFO:tasks.workunit.client.1.smithi027.stdout:5/585: chown d0/d1d/c5a 1720 1 2024-04-02T04:42:55.505 INFO:tasks.workunit.client.1.smithi027.stdout:5/586: link d0/d1d/fbd d0/df/d2f/d3b/fcf 0 2024-04-02T04:42:55.507 INFO:tasks.workunit.client.1.smithi027.stdout:0/441: truncate d4/d21/d28/d4e/d5f/f72 4113058 0 2024-04-02T04:42:55.509 INFO:tasks.workunit.client.1.smithi027.stdout:5/587: mkdir d0/df/d11/d13/d9a/d9c/dd0 0 2024-04-02T04:42:55.511 INFO:tasks.workunit.client.1.smithi027.stdout:5/588: chown d0/d1a/d20/c3f 16775 1 2024-04-02T04:42:55.512 INFO:tasks.workunit.client.1.smithi027.stdout:0/442: mkdir d4/d21/d93 0 2024-04-02T04:42:55.513 INFO:tasks.workunit.client.1.smithi027.stdout:0/443: write d4/f73 [26962,73688] 0 2024-04-02T04:42:55.514 INFO:tasks.workunit.client.1.smithi027.stdout:5/589: mkdir d0/df/d11/d8c/dd1 0 2024-04-02T04:42:55.515 INFO:tasks.workunit.client.1.smithi027.stdout:5/590: chown d0/df/d11/d13/d9a/d9c/db8 28 1 2024-04-02T04:42:55.515 INFO:tasks.workunit.client.1.smithi027.stdout:0/444: chown d4/c8 7496 1 2024-04-02T04:42:55.516 INFO:tasks.workunit.client.1.smithi027.stdout:5/591: write d0/df/d11/d37/fa2 [54937,53526] 0 2024-04-02T04:42:55.519 INFO:tasks.workunit.client.1.smithi027.stdout:0/445: unlink d4/d21/d28/c39 0 2024-04-02T04:42:55.523 INFO:tasks.workunit.client.1.smithi027.stdout:0/446: rename d4/d3d/f7c to d4/d21/d85/f94 0 2024-04-02T04:42:55.525 INFO:tasks.workunit.client.1.smithi027.stdout:5/592: rmdir d0/df/d11/d65/dac/dc8 0 2024-04-02T04:42:55.530 INFO:tasks.workunit.client.1.smithi027.stdout:5/593: dwrite d0/df/d2f/d3b/f3d [0,4194304] 0 2024-04-02T04:42:55.532 INFO:tasks.workunit.client.1.smithi027.stdout:0/447: mknod d4/d21/d28/d4e/d8c/c95 0 2024-04-02T04:42:55.537 INFO:tasks.workunit.client.1.smithi027.stdout:5/594: rename d0/df/d11/d37/f40 to d0/df/d11/d13/d9a/d9c/dbe/fd2 0 2024-04-02T04:42:55.539 INFO:tasks.workunit.client.1.smithi027.stdout:0/448: creat d4/d3d/f96 x:0 0 0 2024-04-02T04:42:55.545 INFO:tasks.workunit.client.0.smithi016.stdout:6/955: mkdir d0/d9/d16/d56/d89/d67/d6f/d112/d12d 0 2024-04-02T04:42:55.546 INFO:tasks.workunit.client.1.smithi027.stdout:0/449: mkdir d4/d16/d71/d97 0 2024-04-02T04:42:55.546 INFO:tasks.workunit.client.1.smithi027.stdout:0/450: readlink d4/d21/l7b 0 2024-04-02T04:42:55.546 INFO:tasks.workunit.client.1.smithi027.stdout:5/595: creat d0/df/d29/da5/fd3 x:0 0 0 2024-04-02T04:42:55.548 INFO:tasks.workunit.client.1.smithi027.stdout:5/596: read d0/df/d29/f7a [3378777,15591] 0 2024-04-02T04:42:55.550 INFO:tasks.workunit.client.1.smithi027.stdout:0/451: dread d4/d16/d1b/f20 [0,4194304] 0 2024-04-02T04:42:55.551 INFO:tasks.workunit.client.0.smithi016.stdout:8/945: getdents d1/d6/d36/d3f/d40/d49/d38 0 2024-04-02T04:42:55.552 INFO:tasks.workunit.client.0.smithi016.stdout:8/946: read d1/d6/d36/d3f/d40/fd3 [526843,18051] 0 2024-04-02T04:42:55.552 INFO:tasks.workunit.client.1.smithi027.stdout:5/597: write d0/df/d11/f1f [5111272,49227] 0 2024-04-02T04:42:55.553 INFO:tasks.workunit.client.0.smithi016.stdout:6/956: creat d0/d9/d16/d56/d89/d67/d6d/f12e x:0 0 0 2024-04-02T04:42:55.553 INFO:tasks.workunit.client.1.smithi027.stdout:0/452: dread - d4/d21/d28/d4e/f55 zero size 2024-04-02T04:42:55.554 INFO:tasks.workunit.client.0.smithi016.stdout:8/947: write d1/d6/d36/d3f/d40/d45/dcc/ff4 [1233706,39993] 0 2024-04-02T04:42:55.556 INFO:tasks.workunit.client.0.smithi016.stdout:1/910: dread d14/d19/d1d/d29/d39/d46/dcb/d43/f96 [0,4194304] 0 2024-04-02T04:42:55.557 INFO:tasks.workunit.client.1.smithi027.stdout:5/598: unlink d0/d1d/f97 0 2024-04-02T04:42:55.560 INFO:tasks.workunit.client.1.smithi027.stdout:1/397: sync 2024-04-02T04:42:55.560 INFO:tasks.workunit.client.1.smithi027.stdout:0/453: dwrite d4/d38/d66/f2a [0,4194304] 0 2024-04-02T04:42:55.565 INFO:tasks.workunit.client.0.smithi016.stdout:6/957: rmdir d0/d9/d16/d56/d89/d67/d6f/d7e 39 2024-04-02T04:42:55.573 INFO:tasks.workunit.client.1.smithi027.stdout:0/454: truncate d4/f9 1386969 0 2024-04-02T04:42:55.575 INFO:tasks.workunit.client.0.smithi016.stdout:1/911: creat d14/d19/d1d/d29/d39/df0/d106/f128 x:0 0 0 2024-04-02T04:42:55.576 INFO:tasks.workunit.client.0.smithi016.stdout:6/958: fdatasync d0/d9/df/fb9 0 2024-04-02T04:42:55.579 INFO:tasks.workunit.client.1.smithi027.stdout:5/599: dread d0/df/d29/f7a [0,4194304] 0 2024-04-02T04:42:55.581 INFO:tasks.workunit.client.1.smithi027.stdout:0/455: symlink d4/d5d/l98 0 2024-04-02T04:42:55.583 INFO:tasks.workunit.client.1.smithi027.stdout:5/600: chown d0/df/c2d 9 1 2024-04-02T04:42:55.585 INFO:tasks.workunit.client.0.smithi016.stdout:8/948: link d1/d6/d36/d3f/d40/d49/l26 d1/d70/d71/d74/d82/l130 0 2024-04-02T04:42:55.587 INFO:tasks.workunit.client.0.smithi016.stdout:1/912: mkdir d14/d19/d1d/d29/d129 0 2024-04-02T04:42:55.587 INFO:tasks.workunit.client.0.smithi016.stdout:1/913: stat d14/d19/d1d 0 2024-04-02T04:42:55.588 INFO:tasks.workunit.client.0.smithi016.stdout:8/949: mknod d1/d70/d71/d9e/c131 0 2024-04-02T04:42:55.589 INFO:tasks.workunit.client.0.smithi016.stdout:1/914: dread - d14/d19/d7a/da2/fe4 zero size 2024-04-02T04:42:55.590 INFO:tasks.workunit.client.0.smithi016.stdout:8/950: mkdir d1/d6/d36/d3f/d40/de9/d132 0 2024-04-02T04:42:55.591 INFO:tasks.workunit.client.0.smithi016.stdout:1/915: fsync d14/d19/d1d/d29/ff7 0 2024-04-02T04:42:55.598 INFO:tasks.workunit.client.0.smithi016.stdout:8/951: dwrite d1/d31/d67/fd1 [0,4194304] 0 2024-04-02T04:42:55.603 INFO:tasks.workunit.client.0.smithi016.stdout:1/916: creat d14/d19/d1d/d29/d39/d46/dcb/d43/d68/f12a x:0 0 0 2024-04-02T04:42:55.609 INFO:tasks.workunit.client.0.smithi016.stdout:1/917: creat d14/d19/d1e/d71/dcf/f12b x:0 0 0 2024-04-02T04:42:55.611 INFO:tasks.workunit.client.0.smithi016.stdout:8/952: creat d1/d70/f133 x:0 0 0 2024-04-02T04:42:55.614 INFO:tasks.workunit.client.0.smithi016.stdout:1/918: dwrite d14/da9/f101 [0,4194304] 0 2024-04-02T04:42:55.615 INFO:tasks.workunit.client.0.smithi016.stdout:1/919: chown d14/d19/d1d/d29 21514 1 2024-04-02T04:42:55.619 INFO:tasks.workunit.client.0.smithi016.stdout:0/914: write d0/d16/d2e/d37/f4b [4247200,8519] 0 2024-04-02T04:42:55.625 INFO:tasks.workunit.client.0.smithi016.stdout:7/918: truncate d0/d21/f3c 654607 0 2024-04-02T04:42:55.626 INFO:tasks.workunit.client.0.smithi016.stdout:7/919: fsync d0/d46/d37/ff0 0 2024-04-02T04:42:55.629 INFO:tasks.workunit.client.0.smithi016.stdout:0/915: rename d0/d16/d2e/d53/d6f to d0/d16/d24/d4e/d8d/d71/d12c 0 2024-04-02T04:42:55.630 INFO:tasks.workunit.client.0.smithi016.stdout:8/953: link d1/d6/d36/d100/ff8 d1/d6/d36/d3f/d40/de9/d132/f134 0 2024-04-02T04:42:55.630 INFO:tasks.workunit.client.0.smithi016.stdout:8/954: readlink d1/d6/d36/d3f/d40/d45/l61 0 2024-04-02T04:42:55.633 INFO:tasks.workunit.client.0.smithi016.stdout:8/955: creat d1/d6/d36/d100/f135 x:0 0 0 2024-04-02T04:42:55.634 INFO:tasks.workunit.client.0.smithi016.stdout:4/930: sync 2024-04-02T04:42:55.634 INFO:tasks.workunit.client.0.smithi016.stdout:5/918: sync 2024-04-02T04:42:55.637 INFO:tasks.workunit.client.1.smithi027.stdout:1/398: dwrite d0/de/d52/f67 [4194304,4194304] 0 2024-04-02T04:42:55.639 INFO:tasks.workunit.client.0.smithi016.stdout:7/920: rename d0/d46/d16/df6 to d0/d133 0 2024-04-02T04:42:55.641 INFO:tasks.workunit.client.1.smithi027.stdout:0/456: rename d4/d21/d28/d4e/d5f/f72 to d4/d76/f99 0 2024-04-02T04:42:55.642 INFO:tasks.workunit.client.0.smithi016.stdout:8/956: chown d1/f1c 3 1 2024-04-02T04:42:55.647 INFO:tasks.workunit.client.0.smithi016.stdout:4/931: rmdir d5/d17/d18/d6d/d91 39 2024-04-02T04:42:55.648 INFO:tasks.workunit.client.0.smithi016.stdout:8/957: dread d1/d6/d36/d8e/fdb [0,4194304] 0 2024-04-02T04:42:55.649 INFO:tasks.workunit.client.1.smithi027.stdout:5/601: rename d0/df/d11/d13/dce/l94 to d0/df/d11/d8c/ld4 0 2024-04-02T04:42:55.650 INFO:tasks.workunit.client.1.smithi027.stdout:5/602: chown d0/df/d11/d13/d9a 53430686 1 2024-04-02T04:42:55.650 INFO:tasks.workunit.client.0.smithi016.stdout:6/959: dwrite d0/d9/d16/f29 [0,4194304] 0 2024-04-02T04:42:55.651 INFO:tasks.workunit.client.1.smithi027.stdout:5/603: read d0/d1a/d20/f73 [3277515,542] 0 2024-04-02T04:42:55.651 INFO:tasks.workunit.client.0.smithi016.stdout:6/960: chown d0/d9/d16/cda 214856306 1 2024-04-02T04:42:55.652 INFO:tasks.workunit.client.1.smithi027.stdout:0/457: symlink d4/d69/l9a 0 2024-04-02T04:42:55.652 INFO:tasks.workunit.client.0.smithi016.stdout:6/961: stat d0/d9/d16/d56/da4 0 2024-04-02T04:42:55.655 INFO:tasks.workunit.client.0.smithi016.stdout:5/919: dread d3/df/d18/f35 [0,4194304] 0 2024-04-02T04:42:55.660 INFO:tasks.workunit.client.1.smithi027.stdout:5/604: symlink d0/d68/d6b/ld5 0 2024-04-02T04:42:55.666 INFO:tasks.workunit.client.0.smithi016.stdout:6/962: creat d0/d9/d16/d56/db0/f12f x:0 0 0 2024-04-02T04:42:55.667 INFO:tasks.workunit.client.1.smithi027.stdout:5/605: symlink d0/df/d11/d65/ld6 0 2024-04-02T04:42:55.669 INFO:tasks.workunit.client.1.smithi027.stdout:1/399: link d0/l1c d0/d1e/d2e/d3e/d5f/l8a 0 2024-04-02T04:42:55.670 INFO:tasks.workunit.client.0.smithi016.stdout:8/958: mknod d1/d6/d36/d3f/c136 0 2024-04-02T04:42:55.671 INFO:tasks.workunit.client.0.smithi016.stdout:1/920: write d14/d19/d1e/d5e/fe7 [854230,94701] 0 2024-04-02T04:42:55.672 INFO:tasks.workunit.client.1.smithi027.stdout:5/606: dread d0/df/d11/d65/fc2 [0,4194304] 0 2024-04-02T04:42:55.673 INFO:tasks.workunit.client.1.smithi027.stdout:0/458: rename d4/d16/d1b/f3e to d4/f9b 0 2024-04-02T04:42:55.675 INFO:tasks.workunit.client.0.smithi016.stdout:6/963: fsync d0/d9/df/fa6 0 2024-04-02T04:42:55.676 INFO:tasks.workunit.client.1.smithi027.stdout:1/400: mkdir d0/d1e/d2e/d3e/d5f/d8b 0 2024-04-02T04:42:55.677 INFO:tasks.workunit.client.1.smithi027.stdout:1/401: write d0/f1d [4269166,88467] 0 2024-04-02T04:42:55.677 INFO:tasks.workunit.client.0.smithi016.stdout:8/959: dwrite d1/d6/d36/d3f/f118 [0,4194304] 0 2024-04-02T04:42:55.678 INFO:tasks.workunit.client.0.smithi016.stdout:5/920: symlink d3/df/d18/d26/l149 0 2024-04-02T04:42:55.681 INFO:tasks.workunit.client.0.smithi016.stdout:1/921: creat d14/da9/f12c x:0 0 0 2024-04-02T04:42:55.684 INFO:tasks.workunit.client.1.smithi027.stdout:0/459: creat d4/d38/d66/d2d/d4c/f9c x:0 0 0 2024-04-02T04:42:55.688 INFO:tasks.workunit.client.1.smithi027.stdout:5/607: mkdir d0/d68/dd7 0 2024-04-02T04:42:55.690 INFO:tasks.workunit.client.0.smithi016.stdout:6/964: dread - d0/d9/d16/dd8/fe9 zero size 2024-04-02T04:42:55.690 INFO:tasks.workunit.client.1.smithi027.stdout:5/608: truncate d0/df/d11/d65/dac/fa1 318516 0 2024-04-02T04:42:55.692 INFO:tasks.workunit.client.1.smithi027.stdout:1/402: rename d0/d1e/d25/f61 to d0/d1e/d25/d3c/d64/f8c 0 2024-04-02T04:42:55.695 INFO:tasks.workunit.client.1.smithi027.stdout:0/460: mkdir d4/d16/d9d 0 2024-04-02T04:42:55.696 INFO:tasks.workunit.client.1.smithi027.stdout:0/461: fsync d4/d38/d66/d2d/d4c/f4d 0 2024-04-02T04:42:55.697 INFO:tasks.workunit.client.1.smithi027.stdout:0/462: chown d4/d21/d28/d4e/f87 517378 1 2024-04-02T04:42:55.700 INFO:tasks.workunit.client.0.smithi016.stdout:8/960: rename d1/d6/d36/d3f/d40 to d1/d6/d137 0 2024-04-02T04:42:55.701 INFO:tasks.workunit.client.0.smithi016.stdout:8/961: write d1/d6/d137/d45/fee [1843322,112695] 0 2024-04-02T04:42:55.705 INFO:tasks.workunit.client.0.smithi016.stdout:6/965: unlink d0/fa3 0 2024-04-02T04:42:55.705 INFO:tasks.workunit.client.0.smithi016.stdout:7/921: dwrite d0/d21/f57 [0,4194304] 0 2024-04-02T04:42:55.706 INFO:tasks.workunit.client.0.smithi016.stdout:4/932: write d5/d17/d18/d6d/fc3 [433072,69545] 0 2024-04-02T04:42:55.706 INFO:tasks.workunit.client.0.smithi016.stdout:1/922: dread d14/d19/d1e/f5d [4194304,4194304] 0 2024-04-02T04:42:55.707 INFO:tasks.workunit.client.0.smithi016.stdout:1/923: chown d14/d19/d1d/d45 0 1 2024-04-02T04:42:55.707 INFO:tasks.workunit.client.1.smithi027.stdout:5/609: write d0/df/d29/f7a [5043276,112739] 0 2024-04-02T04:42:55.709 INFO:tasks.workunit.client.1.smithi027.stdout:5/610: read d0/d1d/d61/f96 [274041,86972] 0 2024-04-02T04:42:55.710 INFO:tasks.workunit.client.1.smithi027.stdout:5/611: readlink d0/d1a/d20/l5e 0 2024-04-02T04:42:55.715 INFO:tasks.workunit.client.0.smithi016.stdout:8/962: creat d1/d6/d36/d3f/d77/d11a/f138 x:0 0 0 2024-04-02T04:42:55.719 INFO:tasks.workunit.client.1.smithi027.stdout:0/463: rename d4/d16/f82 to d4/d38/d74/f9e 0 2024-04-02T04:42:55.719 INFO:tasks.workunit.client.1.smithi027.stdout:0/464: stat d4/d21/d85 0 2024-04-02T04:42:55.722 INFO:tasks.workunit.client.0.smithi016.stdout:7/922: unlink d0/d46/d52/cc6 0 2024-04-02T04:42:55.723 INFO:tasks.workunit.client.0.smithi016.stdout:4/933: rmdir d5/d1d/d1e/da5 39 2024-04-02T04:42:55.729 INFO:tasks.workunit.client.0.smithi016.stdout:7/923: unlink d0/d46/d37/dbe/fbc 0 2024-04-02T04:42:55.729 INFO:tasks.workunit.client.1.smithi027.stdout:5/612: mknod d0/df/d29/dab/cd8 0 2024-04-02T04:42:55.732 INFO:tasks.workunit.client.0.smithi016.stdout:4/934: mkdir d5/d17/d18/d6d/da4/db7/d11e 0 2024-04-02T04:42:55.735 INFO:tasks.workunit.client.1.smithi027.stdout:1/403: getdents d0/d1e/d25/d3c 0 2024-04-02T04:42:55.735 INFO:tasks.workunit.client.0.smithi016.stdout:5/921: dwrite d3/fad [0,4194304] 0 2024-04-02T04:42:55.736 INFO:tasks.workunit.client.1.smithi027.stdout:1/404: chown d0/de/d52/l5b 63320 1 2024-04-02T04:42:55.740 INFO:tasks.workunit.client.1.smithi027.stdout:6/388: sync 2024-04-02T04:42:55.740 INFO:tasks.workunit.client.1.smithi027.stdout:0/465: rmdir d4/d16/d9d 0 2024-04-02T04:42:55.742 INFO:tasks.workunit.client.0.smithi016.stdout:7/924: rmdir d0/d46/d37/dbe/dae/df8/d84/d8a 39 2024-04-02T04:42:55.742 INFO:tasks.workunit.client.1.smithi027.stdout:6/389: chown d13/d1b/d24/d25/l71 5552 1 2024-04-02T04:42:55.743 INFO:tasks.workunit.client.1.smithi027.stdout:6/390: write d13/f17 [1130281,112368] 0 2024-04-02T04:42:55.747 INFO:tasks.workunit.client.1.smithi027.stdout:5/613: creat d0/df/d11/d13/d9a/d9c/db8/fd9 x:0 0 0 2024-04-02T04:42:55.748 INFO:tasks.workunit.client.1.smithi027.stdout:5/614: dread - d0/df/d11/f58 zero size 2024-04-02T04:42:55.748 INFO:tasks.workunit.client.0.smithi016.stdout:4/935: creat d5/d17/dcd/f11f x:0 0 0 2024-04-02T04:42:55.749 INFO:tasks.workunit.client.0.smithi016.stdout:4/936: stat d5/d16/df8/d7e/cc1 0 2024-04-02T04:42:55.750 INFO:tasks.workunit.client.0.smithi016.stdout:5/922: symlink d3/df/dd5/l14a 0 2024-04-02T04:42:55.754 INFO:tasks.workunit.client.1.smithi027.stdout:6/391: creat d13/d1b/d24/d25/d56/f7e x:0 0 0 2024-04-02T04:42:55.754 INFO:tasks.workunit.client.0.smithi016.stdout:8/963: dwrite d1/d6/d36/d3f/fb4 [0,4194304] 0 2024-04-02T04:42:55.754 INFO:tasks.workunit.client.0.smithi016.stdout:8/964: stat d1/d6/d36/d4f/dc9/dfa 0 2024-04-02T04:42:55.755 INFO:tasks.workunit.client.1.smithi027.stdout:0/466: symlink d4/d21/d28/d4e/d5f/l9f 0 2024-04-02T04:42:55.755 INFO:tasks.workunit.client.1.smithi027.stdout:0/467: read d4/f90 [3714493,49683] 0 2024-04-02T04:42:55.758 INFO:tasks.workunit.client.1.smithi027.stdout:1/405: write d0/de/f23 [887089,7099] 0 2024-04-02T04:42:55.758 INFO:tasks.workunit.client.0.smithi016.stdout:7/925: dread d0/d46/f1d [0,4194304] 0 2024-04-02T04:42:55.759 INFO:tasks.workunit.client.1.smithi027.stdout:1/406: chown d0/de/f38 1398795733 1 2024-04-02T04:42:55.759 INFO:tasks.workunit.client.0.smithi016.stdout:4/937: creat d5/d16/f120 x:0 0 0 2024-04-02T04:42:55.759 INFO:tasks.workunit.client.1.smithi027.stdout:1/407: chown d0/d1e/d25 3664972 1 2024-04-02T04:42:55.760 INFO:tasks.workunit.client.0.smithi016.stdout:7/926: fdatasync d0/d46/d52/d82/fd3 0 2024-04-02T04:42:55.760 INFO:tasks.workunit.client.1.smithi027.stdout:6/392: creat d13/d1b/d24/d28/d3c/f7f x:0 0 0 2024-04-02T04:42:55.761 INFO:tasks.workunit.client.1.smithi027.stdout:6/393: fsync d13/d1b/f2b 0 2024-04-02T04:42:55.761 INFO:tasks.workunit.client.0.smithi016.stdout:5/923: chown d3/f1a 1235990 1 2024-04-02T04:42:55.762 INFO:tasks.workunit.client.0.smithi016.stdout:5/924: chown d3/df/d18/d26/d48/d133 96 1 2024-04-02T04:42:55.763 INFO:tasks.workunit.client.0.smithi016.stdout:8/965: dread - d1/d6/d137/de9/ff2 zero size 2024-04-02T04:42:55.765 INFO:tasks.workunit.client.1.smithi027.stdout:5/615: mknod d0/df/d11/d8c/dbf/cda 0 2024-04-02T04:42:55.766 INFO:tasks.workunit.client.0.smithi016.stdout:1/924: link d14/d19/d1d/d29/d39/d46/dcb/d43/d68/c115 d14/d89/c12d 0 2024-04-02T04:42:55.766 INFO:tasks.workunit.client.1.smithi027.stdout:2/422: sync 2024-04-02T04:42:55.766 INFO:tasks.workunit.client.1.smithi027.stdout:4/340: sync 2024-04-02T04:42:55.767 INFO:tasks.workunit.client.1.smithi027.stdout:9/431: sync 2024-04-02T04:42:55.767 INFO:tasks.workunit.client.1.smithi027.stdout:3/366: sync 2024-04-02T04:42:55.767 INFO:tasks.workunit.client.1.smithi027.stdout:8/463: sync 2024-04-02T04:42:55.767 INFO:tasks.workunit.client.1.smithi027.stdout:7/438: sync 2024-04-02T04:42:55.768 INFO:tasks.workunit.client.0.smithi016.stdout:4/938: fdatasync d5/d17/d18/da0/fa7 0 2024-04-02T04:42:55.768 INFO:tasks.workunit.client.1.smithi027.stdout:9/432: write fa [4772777,60140] 0 2024-04-02T04:42:55.770 INFO:tasks.workunit.client.1.smithi027.stdout:9/433: fsync d1a/d1d/d29/d37/f5a 0 2024-04-02T04:42:55.771 INFO:tasks.workunit.client.1.smithi027.stdout:9/434: write fa [2257505,111491] 0 2024-04-02T04:42:55.773 INFO:tasks.workunit.client.0.smithi016.stdout:5/925: creat d3/d40/d4f/d6b/d45/da6/d102/dc4/d108/f14b x:0 0 0 2024-04-02T04:42:55.774 INFO:tasks.workunit.client.0.smithi016.stdout:8/966: symlink d1/d6/d36/d100/l139 0 2024-04-02T04:42:55.776 INFO:tasks.workunit.client.1.smithi027.stdout:8/464: readlink d5/d8/l42 0 2024-04-02T04:42:55.777 INFO:tasks.workunit.client.1.smithi027.stdout:9/435: dwrite d1a/d1d/d29/d56/f88 [0,4194304] 0 2024-04-02T04:42:55.780 INFO:tasks.workunit.client.0.smithi016.stdout:7/927: dread d0/d46/d37/dbe/dae/df8/d84/d8a/f103 [0,4194304] 0 2024-04-02T04:42:55.782 INFO:tasks.workunit.client.0.smithi016.stdout:0/916: sync 2024-04-02T04:42:55.785 INFO:tasks.workunit.client.1.smithi027.stdout:7/439: dread d2/d7/da/dc/d1f/f47 [0,4194304] 0 2024-04-02T04:42:55.787 INFO:tasks.workunit.client.0.smithi016.stdout:1/925: creat d14/d11c/f12e x:0 0 0 2024-04-02T04:42:55.789 INFO:tasks.workunit.client.0.smithi016.stdout:1/926: chown d14/d19/d1d/d45/d4f/d86/db4/fc7 163 1 2024-04-02T04:42:55.790 INFO:tasks.workunit.client.1.smithi027.stdout:6/394: symlink d13/d1b/d24/d28/l80 0 2024-04-02T04:42:55.791 INFO:tasks.workunit.client.0.smithi016.stdout:4/939: dwrite d5/d1d/f21 [4194304,4194304] 0 2024-04-02T04:42:55.792 INFO:tasks.workunit.client.0.smithi016.stdout:5/926: symlink d3/df/d18/d26/d48/d133/l14c 0 2024-04-02T04:42:55.793 INFO:tasks.workunit.client.1.smithi027.stdout:2/423: dwrite d1/db/d3b/d17/f2c [0,4194304] 0 2024-04-02T04:42:55.795 INFO:tasks.workunit.client.0.smithi016.stdout:7/928: mkdir d0/d46/d37/dbe/dae/df8/d43/d96/d134 0 2024-04-02T04:42:55.798 INFO:tasks.workunit.client.0.smithi016.stdout:6/966: getdents d0/d9/d16/d56/d89/d67/d6d 0 2024-04-02T04:42:55.799 INFO:tasks.workunit.client.1.smithi027.stdout:9/436: fdatasync d1a/d1d/d29/d37/f1c 0 2024-04-02T04:42:55.800 INFO:tasks.workunit.client.0.smithi016.stdout:8/967: fdatasync d1/d6/d137/d49/ffe 0 2024-04-02T04:42:55.805 INFO:tasks.workunit.client.1.smithi027.stdout:2/424: dwrite d1/db/d3b/d17/d2b/f53 [0,4194304] 0 2024-04-02T04:42:55.805 INFO:tasks.workunit.client.1.smithi027.stdout:2/425: fsync d1/f13 0 2024-04-02T04:42:55.806 INFO:tasks.workunit.client.1.smithi027.stdout:2/426: stat d1/db/d3b/d17/d2b/c3e 0 2024-04-02T04:42:55.809 INFO:tasks.workunit.client.0.smithi016.stdout:8/968: dwrite d1/d70/d71/d74/f111 [0,4194304] 0 2024-04-02T04:42:55.809 INFO:tasks.workunit.client.0.smithi016.stdout:5/927: mknod d3/d40/d4f/d6b/d45/da6/c14d 0 2024-04-02T04:42:55.812 INFO:tasks.workunit.client.1.smithi027.stdout:7/440: creat d2/df/f85 x:0 0 0 2024-04-02T04:42:55.812 INFO:tasks.workunit.client.0.smithi016.stdout:7/929: dread d0/f22 [0,4194304] 0 2024-04-02T04:42:55.813 INFO:tasks.workunit.client.1.smithi027.stdout:7/441: write d2/df/f85 [832914,57260] 0 2024-04-02T04:42:55.815 INFO:tasks.workunit.client.1.smithi027.stdout:0/468: getdents d4/d21/d85 0 2024-04-02T04:42:55.816 INFO:tasks.workunit.client.0.smithi016.stdout:0/917: dread d0/d16/d2e/f61 [0,4194304] 0 2024-04-02T04:42:55.817 INFO:tasks.workunit.client.1.smithi027.stdout:4/341: write d24/d2d/f31 [524294,60066] 0 2024-04-02T04:42:55.818 INFO:tasks.workunit.client.1.smithi027.stdout:1/408: getdents d0/de/d52 0 2024-04-02T04:42:55.819 INFO:tasks.workunit.client.0.smithi016.stdout:1/927: write d14/d19/d1d/d29/d39/d46/dcb/d43/d68/f7c [413511,16492] 0 2024-04-02T04:42:55.820 INFO:tasks.workunit.client.0.smithi016.stdout:1/928: fdatasync d14/d19/d1d/d29/d4c/fd6 0 2024-04-02T04:42:55.821 INFO:tasks.workunit.client.1.smithi027.stdout:6/395: write d13/d1b/d24/d25/d27/f76 [98026,62070] 0 2024-04-02T04:42:55.828 INFO:tasks.workunit.client.0.smithi016.stdout:8/969: fsync d1/d6/d137/d63/d105/f107 0 2024-04-02T04:42:55.830 INFO:tasks.workunit.client.1.smithi027.stdout:6/396: dwrite d13/d1b/d24/d28/d3c/f77 [0,4194304] 0 2024-04-02T04:42:55.832 INFO:tasks.workunit.client.1.smithi027.stdout:8/465: rename d5/d8/db/dd/l2f to d5/d8/db/dd/d5f/d27/l97 0 2024-04-02T04:42:55.834 INFO:tasks.workunit.client.1.smithi027.stdout:8/466: stat d5/d8/db/d68/d95/c90 0 2024-04-02T04:42:55.835 INFO:tasks.workunit.client.1.smithi027.stdout:8/467: dread - d5/d8/db/dd/d5f/d27/d31/f89 zero size 2024-04-02T04:42:55.837 INFO:tasks.workunit.client.1.smithi027.stdout:2/427: mkdir d1/db/d3b/d96 0 2024-04-02T04:42:55.840 INFO:tasks.workunit.client.0.smithi016.stdout:7/930: dread d0/d46/d37/dbe/dae/df8/d43/d96/fba [0,4194304] 0 2024-04-02T04:42:55.841 INFO:tasks.workunit.client.1.smithi027.stdout:0/469: rename d4/d38/d74 to d4/d38/da0 0 2024-04-02T04:42:55.844 INFO:tasks.workunit.client.0.smithi016.stdout:4/940: dwrite d5/d1d/d1e/da5/ffa [0,4194304] 0 2024-04-02T04:42:55.845 INFO:tasks.workunit.client.1.smithi027.stdout:7/442: dwrite d2/f82 [0,4194304] 0 2024-04-02T04:42:55.849 INFO:tasks.workunit.client.1.smithi027.stdout:1/409: rmdir d0/d1e/d2e/d3b/d4f 39 2024-04-02T04:42:55.850 INFO:tasks.workunit.client.1.smithi027.stdout:7/443: write d2/d7/da/dc/d16/d2f/f73 [393267,69854] 0 2024-04-02T04:42:55.854 INFO:tasks.workunit.client.1.smithi027.stdout:3/367: link d1/de/c33 d1/c83 0 2024-04-02T04:42:55.857 INFO:tasks.workunit.client.1.smithi027.stdout:4/342: write d24/d48/f4e [2190084,56764] 0 2024-04-02T04:42:55.861 INFO:tasks.workunit.client.0.smithi016.stdout:5/928: symlink d3/df/d18/d2f/l14e 0 2024-04-02T04:42:55.863 INFO:tasks.workunit.client.0.smithi016.stdout:8/970: mknod d1/d6/d137/d45/dcc/c13a 0 2024-04-02T04:42:55.865 INFO:tasks.workunit.client.0.smithi016.stdout:0/918: readlink d0/d16/d2e/d53/da4/lbd 0 2024-04-02T04:42:55.866 INFO:tasks.workunit.client.0.smithi016.stdout:7/931: creat d0/d133/f135 x:0 0 0 2024-04-02T04:42:55.868 INFO:tasks.workunit.client.1.smithi027.stdout:1/410: readlink d0/la 0 2024-04-02T04:42:55.871 INFO:tasks.workunit.client.1.smithi027.stdout:3/368: creat d1/de/df/d13/d73/d77/f84 x:0 0 0 2024-04-02T04:42:55.871 INFO:tasks.workunit.client.1.smithi027.stdout:9/437: link d1a/d1d/d29/d37/d3a/l41 d1a/d1d/d29/d37/d20/d7e/l93 0 2024-04-02T04:42:55.872 INFO:tasks.workunit.client.1.smithi027.stdout:3/369: chown d1/de/df/c5e 91 1 2024-04-02T04:42:55.873 INFO:tasks.workunit.client.1.smithi027.stdout:4/343: creat d24/d38/d44/f6e x:0 0 0 2024-04-02T04:42:55.876 INFO:tasks.workunit.client.0.smithi016.stdout:8/971: chown d1/d6/d36/c64 19740307 1 2024-04-02T04:42:55.878 INFO:tasks.workunit.client.1.smithi027.stdout:8/468: dread d5/d8/d15/f39 [0,4194304] 0 2024-04-02T04:42:55.878 INFO:tasks.workunit.client.1.smithi027.stdout:3/370: dread d1/de/f38 [4194304,4194304] 0 2024-04-02T04:42:55.878 INFO:tasks.workunit.client.0.smithi016.stdout:8/972: write d1/d6/d36/d100/f9f [1495662,83804] 0 2024-04-02T04:42:55.878 INFO:tasks.workunit.client.0.smithi016.stdout:1/929: dwrite d14/d19/d1d/d29/d39/d46/dcb/d43/f96 [0,4194304] 0 2024-04-02T04:42:55.879 INFO:tasks.workunit.client.0.smithi016.stdout:0/919: mkdir d0/d48/d12d 0 2024-04-02T04:42:55.880 INFO:tasks.workunit.client.0.smithi016.stdout:1/930: dread - d14/d19/d1d/d29/d39/d46/dcb/d43/d68/dc1/dfa/f10d zero size 2024-04-02T04:42:55.882 INFO:tasks.workunit.client.1.smithi027.stdout:0/470: write d4/d21/f2e [8619777,10312] 0 2024-04-02T04:42:55.883 INFO:tasks.workunit.client.1.smithi027.stdout:1/411: rmdir d0 39 2024-04-02T04:42:55.884 INFO:tasks.workunit.client.0.smithi016.stdout:6/967: link d0/c30 d0/d9/d16/d56/d89/d67/d6f/d112/c130 0 2024-04-02T04:42:55.884 INFO:tasks.workunit.client.1.smithi027.stdout:0/471: chown d4/d21/d28/d4e/d8c 505560897 1 2024-04-02T04:42:55.885 INFO:tasks.workunit.client.1.smithi027.stdout:7/444: mknod d2/d7/da/dc/d16/d3f/d79/c86 0 2024-04-02T04:42:55.888 INFO:tasks.workunit.client.1.smithi027.stdout:6/397: dread d13/d1b/f1f [0,4194304] 0 2024-04-02T04:42:55.888 INFO:tasks.workunit.client.1.smithi027.stdout:9/438: readlink d1a/l81 0 2024-04-02T04:42:55.888 INFO:tasks.workunit.client.1.smithi027.stdout:0/472: read - d4/d38/d66/d2d/d4c/f9c zero size 2024-04-02T04:42:55.889 INFO:tasks.workunit.client.0.smithi016.stdout:5/929: symlink d3/d40/d4f/d6b/d45/da6/d102/l14f 0 2024-04-02T04:42:55.889 INFO:tasks.workunit.client.1.smithi027.stdout:3/371: dwrite d1/de/df/d13/d73/d77/f7a [0,4194304] 0 2024-04-02T04:42:55.891 INFO:tasks.workunit.client.1.smithi027.stdout:6/398: chown d13/d1b/l26 3 1 2024-04-02T04:42:55.899 INFO:tasks.workunit.client.0.smithi016.stdout:7/932: truncate d0/d46/d16/d49/f3e 1800555 0 2024-04-02T04:42:55.899 INFO:tasks.workunit.client.0.smithi016.stdout:4/941: write d5/d1d/d1e/f9a [1235390,61559] 0 2024-04-02T04:42:55.899 INFO:tasks.workunit.client.1.smithi027.stdout:1/412: write d0/de/f38 [255729,89963] 0 2024-04-02T04:42:55.901 INFO:tasks.workunit.client.0.smithi016.stdout:6/968: symlink d0/d9/d16/d56/db0/df2/l131 0 2024-04-02T04:42:55.901 INFO:tasks.workunit.client.0.smithi016.stdout:7/933: write d0/d46/d37/d47/f6b [2151079,27792] 0 2024-04-02T04:42:55.905 INFO:tasks.workunit.client.1.smithi027.stdout:9/439: creat d1a/d1d/d3c/f94 x:0 0 0 2024-04-02T04:42:55.908 INFO:tasks.workunit.client.0.smithi016.stdout:0/920: dwrite d0/d4f/ffd [0,4194304] 0 2024-04-02T04:42:55.910 INFO:tasks.workunit.client.0.smithi016.stdout:1/931: link d14/d11c/f12e d14/d19/d1e/d11e/f12f 0 2024-04-02T04:42:55.911 INFO:tasks.workunit.client.1.smithi027.stdout:3/372: rename d1/f52 to d1/de/df/d13/f85 0 2024-04-02T04:42:55.914 INFO:tasks.workunit.client.1.smithi027.stdout:6/399: creat d13/d1b/d24/d25/d27/f81 x:0 0 0 2024-04-02T04:42:55.916 INFO:tasks.workunit.client.1.smithi027.stdout:0/473: write d4/d38/d66/d4a/f42 [236192,15133] 0 2024-04-02T04:42:55.917 INFO:tasks.workunit.client.1.smithi027.stdout:0/474: read - d4/d21/d85/f86 zero size 2024-04-02T04:42:55.918 INFO:tasks.workunit.client.0.smithi016.stdout:7/934: symlink d0/d46/d52/d82/l136 0 2024-04-02T04:42:55.925 INFO:tasks.workunit.client.1.smithi027.stdout:1/413: mknod d0/d1e/d25/d2b/c8d 0 2024-04-02T04:42:55.926 INFO:tasks.workunit.client.0.smithi016.stdout:0/921: dread d0/d16/d2e/d37/fae [0,4194304] 0 2024-04-02T04:42:55.928 INFO:tasks.workunit.client.0.smithi016.stdout:4/942: mkdir d5/d1d/d5f/d121 0 2024-04-02T04:42:55.930 INFO:tasks.workunit.client.0.smithi016.stdout:7/935: stat d0/d46/d37/dbe/dae/df8/l7a 0 2024-04-02T04:42:55.933 INFO:tasks.workunit.client.0.smithi016.stdout:6/969: creat d0/d9/d16/d56/d95/d10f/f132 x:0 0 0 2024-04-02T04:42:55.933 INFO:tasks.workunit.client.1.smithi027.stdout:6/400: stat d13/d1b/c31 0 2024-04-02T04:42:55.935 INFO:tasks.workunit.client.0.smithi016.stdout:1/932: dwrite d14/d19/d20/f3b [0,4194304] 0 2024-04-02T04:42:55.936 INFO:tasks.workunit.client.0.smithi016.stdout:0/922: creat d0/d16/d2e/d53/da4/d102/f12e x:0 0 0 2024-04-02T04:42:55.936 INFO:tasks.workunit.client.1.smithi027.stdout:4/344: link f2 d24/d38/d44/d60/f6f 0 2024-04-02T04:42:55.939 INFO:tasks.workunit.client.1.smithi027.stdout:7/445: truncate d2/d7/da/dc/d16/d3a/d53/f5d 338209 0 2024-04-02T04:42:55.941 INFO:tasks.workunit.client.1.smithi027.stdout:5/616: sync 2024-04-02T04:42:55.941 INFO:tasks.workunit.client.0.smithi016.stdout:7/936: chown d0/d46/d52/c8c 0 1 2024-04-02T04:42:55.942 INFO:tasks.workunit.client.1.smithi027.stdout:5/617: chown d0/df/d11/d13/dce/lad 655606555 1 2024-04-02T04:42:55.943 INFO:tasks.workunit.client.1.smithi027.stdout:5/618: stat d0/df/d2f/d3b/f5b 0 2024-04-02T04:42:55.943 INFO:tasks.workunit.client.1.smithi027.stdout:1/414: readlink d0/l27 0 2024-04-02T04:42:55.944 INFO:tasks.workunit.client.0.smithi016.stdout:6/970: dread - d0/d9/d16/de3/f104 zero size 2024-04-02T04:42:55.945 INFO:tasks.workunit.client.1.smithi027.stdout:9/440: symlink d1a/d1d/d29/l95 0 2024-04-02T04:42:55.946 INFO:tasks.workunit.client.0.smithi016.stdout:1/933: creat d14/d19/d7a/f130 x:0 0 0 2024-04-02T04:42:55.947 INFO:tasks.workunit.client.0.smithi016.stdout:4/943: dwrite d5/d16/df8/f7d [0,4194304] 0 2024-04-02T04:42:55.949 INFO:tasks.workunit.client.0.smithi016.stdout:7/937: dwrite d0/d21/f86 [0,4194304] 0 2024-04-02T04:42:55.949 INFO:tasks.workunit.client.0.smithi016.stdout:0/923: rename d0/d16/d24/d4e/de6 to d0/d16/d24/deb/df5/d12f 0 2024-04-02T04:42:55.950 INFO:tasks.workunit.client.1.smithi027.stdout:0/475: truncate d4/f9 12318 0 2024-04-02T04:42:55.951 INFO:tasks.workunit.client.1.smithi027.stdout:1/415: dwrite d0/f69 [4194304,4194304] 0 2024-04-02T04:42:55.952 INFO:tasks.workunit.client.0.smithi016.stdout:4/944: write d5/d17/d18/d25/f41 [6043364,13075] 0 2024-04-02T04:42:55.952 INFO:tasks.workunit.client.0.smithi016.stdout:5/930: getdents d3/d40/db6/de0/d115 0 2024-04-02T04:42:55.953 INFO:tasks.workunit.client.1.smithi027.stdout:2/428: getdents d1/db/d3b/d17/d6d 0 2024-04-02T04:42:55.957 INFO:tasks.workunit.client.0.smithi016.stdout:5/931: chown d3/df/d18/d26/f91 2 1 2024-04-02T04:42:55.958 INFO:tasks.workunit.client.1.smithi027.stdout:9/441: dwrite d1a/d1d/d29/d37/d4c/f66 [0,4194304] 0 2024-04-02T04:42:55.958 INFO:tasks.workunit.client.0.smithi016.stdout:6/971: dwrite d0/d9/d16/d56/d59/f96 [0,4194304] 0 2024-04-02T04:42:55.959 INFO:tasks.workunit.client.0.smithi016.stdout:5/932: chown d3/d40/d4f/d6b/d45/da6/d102/dc4/dd7/fd2 62 1 2024-04-02T04:42:55.965 INFO:tasks.workunit.client.1.smithi027.stdout:0/476: dwrite d4/d38/d66/d4a/f70 [0,4194304] 0 2024-04-02T04:42:55.970 INFO:tasks.workunit.client.0.smithi016.stdout:1/934: dwrite d14/d19/d1d/d45/d4f/d86/db4/fc7 [0,4194304] 0 2024-04-02T04:42:55.972 INFO:tasks.workunit.client.0.smithi016.stdout:0/924: mknod d0/d16/d23/d3e/db3/d10c/c130 0 2024-04-02T04:42:55.974 INFO:tasks.workunit.client.0.smithi016.stdout:1/935: write d14/d19/d1d/d29/d39/d46/dcb/d43/f5f [3112395,38127] 0 2024-04-02T04:42:55.975 INFO:tasks.workunit.client.1.smithi027.stdout:4/345: dwrite d24/d38/d44/d60/f6f [0,4194304] 0 2024-04-02T04:42:55.975 INFO:tasks.workunit.client.1.smithi027.stdout:6/401: symlink d13/d1b/d24/l82 0 2024-04-02T04:42:55.977 INFO:tasks.workunit.client.1.smithi027.stdout:1/416: truncate d0/fd 4144484 0 2024-04-02T04:42:55.980 INFO:tasks.workunit.client.1.smithi027.stdout:1/417: write d0/d1e/d2e/d3e/f88 [305800,41881] 0 2024-04-02T04:42:55.981 INFO:tasks.workunit.client.1.smithi027.stdout:8/469: link d5/d8/l60 d5/d8/l98 0 2024-04-02T04:42:55.981 INFO:tasks.workunit.client.0.smithi016.stdout:5/933: dread - d3/d40/d4f/d6b/d45/da6/d102/dd1/f107 zero size 2024-04-02T04:42:55.982 INFO:tasks.workunit.client.0.smithi016.stdout:1/936: read d14/d19/d1e/ff4 [2635492,91344] 0 2024-04-02T04:42:55.985 INFO:tasks.workunit.client.1.smithi027.stdout:2/429: symlink d1/db/d3b/d17/d68/l97 0 2024-04-02T04:42:55.987 INFO:tasks.workunit.client.0.smithi016.stdout:4/945: unlink d5/d16/d3f/l116 0 2024-04-02T04:42:55.988 INFO:tasks.workunit.client.0.smithi016.stdout:4/946: stat d5/f7 0 2024-04-02T04:42:55.990 INFO:tasks.workunit.client.0.smithi016.stdout:1/937: dwrite d14/d19/d1d/d29/d39/d46/dcb/d43/d68/dc1/dfa/f10d [0,4194304] 0 2024-04-02T04:42:55.992 INFO:tasks.workunit.client.0.smithi016.stdout:7/938: creat d0/f137 x:0 0 0 2024-04-02T04:42:55.994 INFO:tasks.workunit.client.0.smithi016.stdout:6/972: creat d0/d9/d16/d56/d89/dc5/dce/f133 x:0 0 0 2024-04-02T04:42:55.997 INFO:tasks.workunit.client.1.smithi027.stdout:3/373: getdents d1/de/df/d13 0 2024-04-02T04:42:56.000 INFO:tasks.workunit.client.1.smithi027.stdout:6/402: creat d13/d1b/d48/d75/f83 x:0 0 0 2024-04-02T04:42:56.002 INFO:tasks.workunit.client.0.smithi016.stdout:4/947: creat d5/d17/d18/d6d/d8e/d9c/dcc/d6f/dcf/f122 x:0 0 0 2024-04-02T04:42:56.006 INFO:tasks.workunit.client.0.smithi016.stdout:7/939: fsync d0/f9c 0 2024-04-02T04:42:56.006 INFO:tasks.workunit.client.1.smithi027.stdout:8/470: write d5/d8/d15/f3e [1753009,42485] 0 2024-04-02T04:42:56.008 INFO:tasks.workunit.client.1.smithi027.stdout:1/418: rename d0/d1e/d2e/d3e/c56 to d0/de/d52/d74/c8e 0 2024-04-02T04:42:56.009 INFO:tasks.workunit.client.1.smithi027.stdout:2/430: chown d1/db/d3b/d83 33 1 2024-04-02T04:42:56.011 INFO:tasks.workunit.client.0.smithi016.stdout:0/925: rmdir d0/d16/d24/deb/df5/d12f 0 2024-04-02T04:42:56.011 INFO:tasks.workunit.client.1.smithi027.stdout:0/477: dread d4/d16/d1b/f65 [0,4194304] 0 2024-04-02T04:42:56.013 INFO:tasks.workunit.client.1.smithi027.stdout:0/478: truncate d4/d21/d28/d4e/f87 1195888 0 2024-04-02T04:42:56.016 INFO:tasks.workunit.client.1.smithi027.stdout:5/619: creat d0/df/d29/da5/fdb x:0 0 0 2024-04-02T04:42:56.022 INFO:tasks.workunit.client.0.smithi016.stdout:1/938: dread d14/d19/d1d/d29/f6a [0,4194304] 0 2024-04-02T04:42:56.028 INFO:tasks.workunit.client.1.smithi027.stdout:7/446: getdents d2/df 0 2024-04-02T04:42:56.031 INFO:tasks.workunit.client.0.smithi016.stdout:4/948: link d5/l7a d5/d8/d43/dd7/l123 0 2024-04-02T04:42:56.033 INFO:tasks.workunit.client.0.smithi016.stdout:6/973: getdents d0/d9/d16/dd8 0 2024-04-02T04:42:56.036 INFO:tasks.workunit.client.0.smithi016.stdout:4/949: mknod d5/d17/d18/d25/c124 0 2024-04-02T04:42:56.037 INFO:tasks.workunit.client.1.smithi027.stdout:9/442: dwrite d1a/f39 [0,4194304] 0 2024-04-02T04:42:56.037 INFO:tasks.workunit.client.1.smithi027.stdout:9/443: stat d1a 0 2024-04-02T04:42:56.038 INFO:tasks.workunit.client.0.smithi016.stdout:6/974: mkdir d0/d9/d16/d56/d89/d67/d6f/d112/d134 0 2024-04-02T04:42:56.039 INFO:tasks.workunit.client.1.smithi027.stdout:0/479: creat d4/d38/fa1 x:0 0 0 2024-04-02T04:42:56.039 INFO:tasks.workunit.client.0.smithi016.stdout:7/940: dwrite d0/d46/d37/d47/f5c [0,4194304] 0 2024-04-02T04:42:56.039 INFO:tasks.workunit.client.0.smithi016.stdout:6/975: chown d0/d9/d16/d56/da4/f101 58 1 2024-04-02T04:42:56.041 INFO:tasks.workunit.client.0.smithi016.stdout:4/950: symlink d5/d17/d18/d6d/d8e/d9c/l125 0 2024-04-02T04:42:56.044 INFO:tasks.workunit.client.0.smithi016.stdout:7/941: read - d0/d46/d16/d49/fca zero size 2024-04-02T04:42:56.045 INFO:tasks.workunit.client.1.smithi027.stdout:9/444: dwrite d1a/d1d/d29/d37/f5a [0,4194304] 0 2024-04-02T04:42:56.046 INFO:tasks.workunit.client.0.smithi016.stdout:6/976: readlink d0/d9/d16/d56/da4/l122 0 2024-04-02T04:42:56.046 INFO:tasks.workunit.client.0.smithi016.stdout:4/951: rmdir d5/de/dd5 39 2024-04-02T04:42:56.049 INFO:tasks.workunit.client.1.smithi027.stdout:0/480: dwrite d4/d38/fa1 [0,4194304] 0 2024-04-02T04:42:56.050 INFO:tasks.workunit.client.0.smithi016.stdout:0/926: dwrite d0/d16/d24/fc5 [0,4194304] 0 2024-04-02T04:42:56.051 INFO:tasks.workunit.client.0.smithi016.stdout:6/977: chown d0/f51 1676059 1 2024-04-02T04:42:56.053 INFO:tasks.workunit.client.1.smithi027.stdout:4/346: link d24/d38/d47/d5c/l68 d24/d38/d47/l70 0 2024-04-02T04:42:56.055 INFO:tasks.workunit.client.0.smithi016.stdout:7/942: creat d0/d46/da2/dc8/f138 x:0 0 0 2024-04-02T04:42:56.055 INFO:tasks.workunit.client.1.smithi027.stdout:4/347: truncate d24/d38/d44/d60/f6f 9278732 0 2024-04-02T04:42:56.056 INFO:tasks.workunit.client.1.smithi027.stdout:4/348: stat d24/f3a 0 2024-04-02T04:42:56.056 INFO:tasks.workunit.client.1.smithi027.stdout:5/620: dread d0/d1a/f49 [0,4194304] 0 2024-04-02T04:42:56.057 INFO:tasks.workunit.client.1.smithi027.stdout:2/431: mknod d1/db/d3b/d96/c98 0 2024-04-02T04:42:56.057 INFO:tasks.workunit.client.0.smithi016.stdout:4/952: dwrite d5/d1d/d1e/ff0 [0,4194304] 0 2024-04-02T04:42:56.057 INFO:tasks.workunit.client.0.smithi016.stdout:7/943: write d0/fe7 [4309447,56716] 0 2024-04-02T04:42:56.058 INFO:tasks.workunit.client.1.smithi027.stdout:4/349: chown d24/d38/l5f 26925 1 2024-04-02T04:42:56.058 INFO:tasks.workunit.client.1.smithi027.stdout:4/350: dread - d24/d2d/f43 zero size 2024-04-02T04:42:56.058 INFO:tasks.workunit.client.1.smithi027.stdout:1/419: getdents d0/d1e/d2e/d3e/d5f/d8b 0 2024-04-02T04:42:56.059 INFO:tasks.workunit.client.1.smithi027.stdout:5/621: chown d0/df/d29/da5/fd3 3 1 2024-04-02T04:42:56.060 INFO:tasks.workunit.client.1.smithi027.stdout:1/420: stat d0/d1e/d2e/d3e/l85 0 2024-04-02T04:42:56.062 INFO:tasks.workunit.client.0.smithi016.stdout:0/927: creat d0/d16/d24/d12a/f131 x:0 0 0 2024-04-02T04:42:56.063 INFO:tasks.workunit.client.0.smithi016.stdout:0/928: readlink d0/d4f/d11b/l128 0 2024-04-02T04:42:56.064 INFO:tasks.workunit.client.1.smithi027.stdout:6/403: creat d13/d1b/d24/f84 x:0 0 0 2024-04-02T04:42:56.065 INFO:tasks.workunit.client.0.smithi016.stdout:6/978: rmdir d0/d5c/df5 39 2024-04-02T04:42:56.065 INFO:tasks.workunit.client.1.smithi027.stdout:6/404: fdatasync fc 0 2024-04-02T04:42:56.073 INFO:tasks.workunit.client.1.smithi027.stdout:7/447: truncate d2/d1c/f4b 3845520 0 2024-04-02T04:42:56.074 INFO:tasks.workunit.client.1.smithi027.stdout:9/445: dwrite f15 [0,4194304] 0 2024-04-02T04:42:56.076 INFO:tasks.workunit.client.1.smithi027.stdout:7/448: dread - d2/d7/da/dc/f7b zero size 2024-04-02T04:42:56.076 INFO:tasks.workunit.client.1.smithi027.stdout:1/421: dwrite d0/d1e/d2e/d3e/d5f/f79 [0,4194304] 0 2024-04-02T04:42:56.078 INFO:tasks.workunit.client.1.smithi027.stdout:7/449: dread - d2/f4c zero size 2024-04-02T04:42:56.078 INFO:tasks.workunit.client.1.smithi027.stdout:9/446: chown d1a/d1d/d29/d51 1 1 2024-04-02T04:42:56.083 INFO:tasks.workunit.client.0.smithi016.stdout:4/953: symlink d5/de/dd5/d118/l126 0 2024-04-02T04:42:56.083 INFO:tasks.workunit.client.1.smithi027.stdout:4/351: rmdir d24 39 2024-04-02T04:42:56.085 INFO:tasks.workunit.client.0.smithi016.stdout:0/929: dread d0/d16/d24/d4e/d8d/f75 [0,4194304] 0 2024-04-02T04:42:56.085 INFO:tasks.workunit.client.1.smithi027.stdout:3/374: dwrite d1/de/df/f14 [0,4194304] 0 2024-04-02T04:42:56.087 INFO:tasks.workunit.client.1.smithi027.stdout:3/375: truncate d1/d1a/f79 634340 0 2024-04-02T04:42:56.087 INFO:tasks.workunit.client.1.smithi027.stdout:5/622: creat d0/df/d11/d13/d9a/db7/fdc x:0 0 0 2024-04-02T04:42:56.089 INFO:tasks.workunit.client.1.smithi027.stdout:0/481: write d4/d38/fa1 [5125795,118199] 0 2024-04-02T04:42:56.089 INFO:tasks.workunit.client.1.smithi027.stdout:3/376: truncate d1/de/df/f14 5045206 0 2024-04-02T04:42:56.090 INFO:tasks.workunit.client.1.smithi027.stdout:5/623: fsync d0/df/d11/d13/d9a/d9c/fa8 0 2024-04-02T04:42:56.091 INFO:tasks.workunit.client.1.smithi027.stdout:3/377: write d1/de/df/d13/f51 [286104,10152] 0 2024-04-02T04:42:56.092 INFO:tasks.workunit.client.1.smithi027.stdout:5/624: dread d0/d1a/d20/fb5 [0,4194304] 0 2024-04-02T04:42:56.093 INFO:tasks.workunit.client.1.smithi027.stdout:5/625: dread - d0/d1d/fbd zero size 2024-04-02T04:42:56.095 INFO:tasks.workunit.client.1.smithi027.stdout:5/626: chown d0/d68/d6b/ld5 1381 1 2024-04-02T04:42:56.096 INFO:tasks.workunit.client.0.smithi016.stdout:4/954: rename d5/d17/d18/d25 to d5/d17/dcd/def/dfd/d127 0 2024-04-02T04:42:56.098 INFO:tasks.workunit.client.0.smithi016.stdout:7/944: getdents d0/d46/d37/dbe/dae/d9d 0 2024-04-02T04:42:56.098 INFO:tasks.workunit.client.1.smithi027.stdout:5/627: truncate d0/df/d11/d13/dce/fbc 4464637 0 2024-04-02T04:42:56.098 INFO:tasks.workunit.client.1.smithi027.stdout:5/628: readlink d0/df/d11/d13/dce/l5f 0 2024-04-02T04:42:56.099 INFO:tasks.workunit.client.0.smithi016.stdout:4/955: rmdir d5/d8 39 2024-04-02T04:42:56.100 INFO:tasks.workunit.client.1.smithi027.stdout:5/629: chown d0/df/d11/d13/dce/d55/dc3 1 1 2024-04-02T04:42:56.101 INFO:tasks.workunit.client.1.smithi027.stdout:1/422: stat d0/de/c33 0 2024-04-02T04:42:56.102 INFO:tasks.workunit.client.0.smithi016.stdout:6/979: truncate d0/d9/d16/de3/ff7 1772999 0 2024-04-02T04:42:56.103 INFO:tasks.workunit.client.1.smithi027.stdout:1/423: write d0/d1e/d25/f31 [1829646,17359] 0 2024-04-02T04:42:56.105 INFO:tasks.workunit.client.0.smithi016.stdout:6/980: read d0/d9/d16/d56/d89/f3b [2255928,59119] 0 2024-04-02T04:42:56.106 INFO:tasks.workunit.client.0.smithi016.stdout:0/930: dread d0/f19 [0,4194304] 0 2024-04-02T04:42:56.110 INFO:tasks.workunit.client.0.smithi016.stdout:4/956: fdatasync d5/d1d/d6c/f72 0 2024-04-02T04:42:56.110 INFO:tasks.workunit.client.1.smithi027.stdout:9/447: dread d1a/d1d/d29/d37/d4c/f62 [0,4194304] 0 2024-04-02T04:42:56.112 INFO:tasks.workunit.client.0.smithi016.stdout:4/957: write d5/d1d/d1e/f9d [4421479,61051] 0 2024-04-02T04:42:56.114 INFO:tasks.workunit.client.1.smithi027.stdout:2/432: creat d1/db/d57/f99 x:0 0 0 2024-04-02T04:42:56.114 INFO:tasks.workunit.client.0.smithi016.stdout:7/945: dread d0/d46/f2f [0,4194304] 0 2024-04-02T04:42:56.121 INFO:tasks.workunit.client.1.smithi027.stdout:7/450: dwrite d2/d7/da/dc/d1f/f47 [0,4194304] 0 2024-04-02T04:42:56.121 INFO:tasks.workunit.client.1.smithi027.stdout:7/451: stat d2/f4c 0 2024-04-02T04:42:56.122 INFO:tasks.workunit.client.0.smithi016.stdout:8/973: sync 2024-04-02T04:42:56.123 INFO:tasks.workunit.client.1.smithi027.stdout:0/482: write d4/d21/d28/f54 [45703,90621] 0 2024-04-02T04:42:56.126 INFO:tasks.workunit.client.0.smithi016.stdout:6/981: mkdir d0/d9/d16/d56/d135 0 2024-04-02T04:42:56.128 INFO:tasks.workunit.client.0.smithi016.stdout:4/958: write d5/d17/d18/d6d/da4/fe6 [1024669,107063] 0 2024-04-02T04:42:56.129 INFO:tasks.workunit.client.0.smithi016.stdout:6/982: truncate d0/d9/d16/d56/d89/d67/d6d/fc3 601591 0 2024-04-02T04:42:56.130 INFO:tasks.workunit.client.0.smithi016.stdout:0/931: mkdir d0/d132 0 2024-04-02T04:42:56.132 INFO:tasks.workunit.client.1.smithi027.stdout:8/471: getdents d5/d8/db/d57 0 2024-04-02T04:42:56.133 INFO:tasks.workunit.client.1.smithi027.stdout:8/472: readlink l1 0 2024-04-02T04:42:56.134 INFO:tasks.workunit.client.1.smithi027.stdout:4/352: dread d24/d38/d44/d60/f6f [4194304,4194304] 0 2024-04-02T04:42:56.137 INFO:tasks.workunit.client.0.smithi016.stdout:8/974: symlink d1/d6/d137/d49/d4a/l13b 0 2024-04-02T04:42:56.137 INFO:tasks.workunit.client.0.smithi016.stdout:8/975: chown d1/d70/d71/d9e 459494225 1 2024-04-02T04:42:56.139 INFO:tasks.workunit.client.1.smithi027.stdout:2/433: symlink d1/db/d3b/d17/d68/d78/l9a 0 2024-04-02T04:42:56.140 INFO:tasks.workunit.client.1.smithi027.stdout:9/448: read d1a/f1f [2571609,37471] 0 2024-04-02T04:42:56.141 INFO:tasks.workunit.client.0.smithi016.stdout:6/983: chown d0/d9/d16/de3/ff7 28403 1 2024-04-02T04:42:56.142 INFO:tasks.workunit.client.1.smithi027.stdout:8/473: dwrite d5/d8/db/dd/d4a/f87 [0,4194304] 0 2024-04-02T04:42:56.144 INFO:tasks.workunit.client.0.smithi016.stdout:4/959: rename d5/d17/d18/d1f/l46 to d5/d1d/d5f/d121/l128 0 2024-04-02T04:42:56.144 INFO:tasks.workunit.client.0.smithi016.stdout:4/960: fsync d5/f102 0 2024-04-02T04:42:56.145 INFO:tasks.workunit.client.0.smithi016.stdout:7/946: write d0/d46/fc [3530888,127320] 0 2024-04-02T04:42:56.148 INFO:tasks.workunit.client.0.smithi016.stdout:7/947: chown d0/d46/d37/dbe/dae/df8/d43/d96/ddf/cd4 918 1 2024-04-02T04:42:56.148 INFO:tasks.workunit.client.0.smithi016.stdout:8/976: symlink d1/d31/l13c 0 2024-04-02T04:42:56.151 INFO:tasks.workunit.client.1.smithi027.stdout:3/378: link d1/de/d55/f5f d1/d1a/f86 0 2024-04-02T04:42:56.152 INFO:tasks.workunit.client.0.smithi016.stdout:0/932: mknod d0/d16/dd6/c133 0 2024-04-02T04:42:56.153 INFO:tasks.workunit.client.1.smithi027.stdout:6/405: link d13/d1b/d24/d25/d42/l61 d13/d1b/d48/d58/l85 0 2024-04-02T04:42:56.153 INFO:tasks.workunit.client.0.smithi016.stdout:0/933: fsync d0/d16/d23/d3e/f98 0 2024-04-02T04:42:56.153 INFO:tasks.workunit.client.1.smithi027.stdout:6/406: fdatasync d13/d1b/d24/d28/f65 0 2024-04-02T04:42:56.154 INFO:tasks.workunit.client.1.smithi027.stdout:0/483: symlink d4/d38/d66/d4a/la2 0 2024-04-02T04:42:56.158 INFO:tasks.workunit.client.0.smithi016.stdout:0/934: dwrite d0/d16/d24/fc5 [0,4194304] 0 2024-04-02T04:42:56.158 INFO:tasks.workunit.client.1.smithi027.stdout:5/630: link d0/df/d11/d13/d9a/d9c/fa8 d0/df/d11/d65/fdd 0 2024-04-02T04:42:56.163 INFO:tasks.workunit.client.0.smithi016.stdout:4/961: mknod d5/d17/d18/d6d/d91/c129 0 2024-04-02T04:42:56.165 INFO:tasks.workunit.client.0.smithi016.stdout:6/984: truncate d0/d9/d16/f23 1373283 0 2024-04-02T04:42:56.166 INFO:tasks.workunit.client.0.smithi016.stdout:7/948: symlink d0/d46/d37/dbe/dae/df8/l139 0 2024-04-02T04:42:56.168 INFO:tasks.workunit.client.0.smithi016.stdout:0/935: creat d0/d16/dd6/ddd/f134 x:0 0 0 2024-04-02T04:42:56.172 INFO:tasks.workunit.client.0.smithi016.stdout:8/977: dread d1/d6/d137/d49/d4a/d4c/f124 [0,4194304] 0 2024-04-02T04:42:56.173 INFO:tasks.workunit.client.1.smithi027.stdout:8/474: fdatasync d5/d8/d15/d71/f72 0 2024-04-02T04:42:56.173 INFO:tasks.workunit.client.0.smithi016.stdout:8/978: read - d1/d6/d137/d45/fbc zero size 2024-04-02T04:42:56.175 INFO:tasks.workunit.client.1.smithi027.stdout:7/452: symlink d2/d7/l87 0 2024-04-02T04:42:56.176 INFO:tasks.workunit.client.1.smithi027.stdout:7/453: stat d2/d1d 0 2024-04-02T04:42:56.177 INFO:tasks.workunit.client.1.smithi027.stdout:9/449: dwrite d1a/d1d/d29/d37/d20/f3f [4194304,4194304] 0 2024-04-02T04:42:56.179 INFO:tasks.workunit.client.1.smithi027.stdout:1/424: truncate d0/de/d52/f65 196924 0 2024-04-02T04:42:56.181 INFO:tasks.workunit.client.1.smithi027.stdout:1/425: fdatasync d0/d1e/d25/d2b/d59/f5e 0 2024-04-02T04:42:56.181 INFO:tasks.workunit.client.1.smithi027.stdout:7/454: dread d2/d7/da/f2a [0,4194304] 0 2024-04-02T04:42:56.183 INFO:tasks.workunit.client.1.smithi027.stdout:1/426: write d0/d1e/d2e/d3e/d5f/f79 [3158985,85894] 0 2024-04-02T04:42:56.183 INFO:tasks.workunit.client.1.smithi027.stdout:3/379: symlink d1/de/d74/l87 0 2024-04-02T04:42:56.187 INFO:tasks.workunit.client.1.smithi027.stdout:6/407: rmdir d13/d1b/d24 39 2024-04-02T04:42:56.191 INFO:tasks.workunit.client.1.smithi027.stdout:0/484: mknod d4/d38/d66/d2d/d4c/ca3 0 2024-04-02T04:42:56.191 INFO:tasks.workunit.client.1.smithi027.stdout:0/485: dread - d4/d3d/f96 zero size 2024-04-02T04:42:56.192 INFO:tasks.workunit.client.0.smithi016.stdout:8/979: truncate d1/d6/d137/d49/d4a/d4c/dad/fda 726914 0 2024-04-02T04:42:56.195 INFO:tasks.workunit.client.1.smithi027.stdout:5/631: rename d0/df/d11/d65/fc2 to d0/d1a/d20/fde 0 2024-04-02T04:42:56.199 INFO:tasks.workunit.client.1.smithi027.stdout:2/434: mknod d1/db/d3b/c9b 0 2024-04-02T04:42:56.199 INFO:tasks.workunit.client.0.smithi016.stdout:5/934: sync 2024-04-02T04:42:56.201 INFO:tasks.workunit.client.0.smithi016.stdout:4/962: dwrite d5/f60 [0,4194304] 0 2024-04-02T04:42:56.201 INFO:tasks.workunit.client.0.smithi016.stdout:8/980: creat d1/d6/f13d x:0 0 0 2024-04-02T04:42:56.202 INFO:tasks.workunit.client.0.smithi016.stdout:7/949: dwrite d0/d46/d37/dbe/dae/df8/fb6 [0,4194304] 0 2024-04-02T04:42:56.204 INFO:tasks.workunit.client.0.smithi016.stdout:6/985: write d0/f3f [7027367,29324] 0 2024-04-02T04:42:56.204 INFO:tasks.workunit.client.1.smithi027.stdout:9/450: creat d1a/d1d/d3c/d71/f96 x:0 0 0 2024-04-02T04:42:56.205 INFO:tasks.workunit.client.0.smithi016.stdout:8/981: chown d1/d31/l13c 3634117 1 2024-04-02T04:42:56.208 INFO:tasks.workunit.client.0.smithi016.stdout:0/936: dwrite d0/f19 [4194304,4194304] 0 2024-04-02T04:42:56.209 INFO:tasks.workunit.client.0.smithi016.stdout:7/950: truncate d0/d46/d16/dc2/f131 208270 0 2024-04-02T04:42:56.210 INFO:tasks.workunit.client.0.smithi016.stdout:0/937: read d0/d16/fb8 [3449317,27551] 0 2024-04-02T04:42:56.211 INFO:tasks.workunit.client.1.smithi027.stdout:8/475: dread d5/d8/db/dd/d5f/f2b [0,4194304] 0 2024-04-02T04:42:56.212 INFO:tasks.workunit.client.0.smithi016.stdout:0/938: chown d0/d16/f94 71489 1 2024-04-02T04:42:56.215 INFO:tasks.workunit.client.1.smithi027.stdout:9/451: dwrite d1a/d1d/f72 [0,4194304] 0 2024-04-02T04:42:56.222 INFO:tasks.workunit.client.0.smithi016.stdout:4/963: mkdir d5/d17/d18/d6d/da4/db7/d12a 0 2024-04-02T04:42:56.223 INFO:tasks.workunit.client.0.smithi016.stdout:4/964: stat d5/d17/d18/d6d/da4/db7 0 2024-04-02T04:42:56.225 INFO:tasks.workunit.client.0.smithi016.stdout:8/982: dread d1/d6/d36/d4f/d5d/f95 [0,4194304] 0 2024-04-02T04:42:56.226 INFO:tasks.workunit.client.1.smithi027.stdout:0/486: symlink d4/d5d/la4 0 2024-04-02T04:42:56.227 INFO:tasks.workunit.client.0.smithi016.stdout:6/986: symlink d0/d9/d16/d56/d89/dc5/dce/l136 0 2024-04-02T04:42:56.228 INFO:tasks.workunit.client.0.smithi016.stdout:5/935: write d3/df/d18/f98 [5552,53316] 0 2024-04-02T04:42:56.228 INFO:tasks.workunit.client.1.smithi027.stdout:5/632: symlink d0/df/ldf 0 2024-04-02T04:42:56.230 INFO:tasks.workunit.client.1.smithi027.stdout:4/353: creat d24/f71 x:0 0 0 2024-04-02T04:42:56.232 INFO:tasks.workunit.client.0.smithi016.stdout:0/939: write d0/d16/d11e/f126 [3025507,63534] 0 2024-04-02T04:42:56.233 INFO:tasks.workunit.client.0.smithi016.stdout:4/965: truncate d5/de/f8d 443198 0 2024-04-02T04:42:56.233 INFO:tasks.workunit.client.0.smithi016.stdout:0/940: chown d0/d16/d2e/d53/da4/de3/f106 26308 1 2024-04-02T04:42:56.234 INFO:tasks.workunit.client.0.smithi016.stdout:0/941: chown d0/d16/d2e/cdb 4074 1 2024-04-02T04:42:56.234 INFO:tasks.workunit.client.0.smithi016.stdout:4/966: readlink d5/d17/d18/d6d/d8e/d9c/dcc/d6f/dcf/lea 0 2024-04-02T04:42:56.239 INFO:tasks.workunit.client.1.smithi027.stdout:8/476: readlink d5/d8/db/dd/d5f/d27/l97 0 2024-04-02T04:42:56.240 INFO:tasks.workunit.client.1.smithi027.stdout:8/477: chown d5/d8/db/dd/f1d 87 1 2024-04-02T04:42:56.240 INFO:tasks.workunit.client.0.smithi016.stdout:7/951: mknod d0/d46/d37/dbe/dae/df8/d43/d96/c13a 0 2024-04-02T04:42:56.240 INFO:tasks.workunit.client.0.smithi016.stdout:7/952: chown d0/d21/f11e 555197 1 2024-04-02T04:42:56.242 INFO:tasks.workunit.client.0.smithi016.stdout:8/983: symlink d1/d6/d36/d4f/l13e 0 2024-04-02T04:42:56.242 INFO:tasks.workunit.client.0.smithi016.stdout:1/939: sync 2024-04-02T04:42:56.247 INFO:tasks.workunit.client.1.smithi027.stdout:1/427: creat d0/d1e/d5d/d60/f8f x:0 0 0 2024-04-02T04:42:56.248 INFO:tasks.workunit.client.0.smithi016.stdout:1/940: dwrite d14/d19/d1d/d29/f11b [0,4194304] 0 2024-04-02T04:42:56.248 INFO:tasks.workunit.client.1.smithi027.stdout:1/428: stat d0/d1e/d2e/l66 0 2024-04-02T04:42:56.248 INFO:tasks.workunit.client.0.smithi016.stdout:4/967: truncate d5/d1d/d1e/f3c 6024158 0 2024-04-02T04:42:56.250 INFO:tasks.workunit.client.1.smithi027.stdout:6/408: rename d13/d1b/d24/d25/l71 to d13/d1b/d48/d75/l86 0 2024-04-02T04:42:56.255 INFO:tasks.workunit.client.1.smithi027.stdout:1/429: dwrite d0/d1e/d5d/d60/f8f [0,4194304] 0 2024-04-02T04:42:56.260 INFO:tasks.workunit.client.0.smithi016.stdout:5/936: write d3/df/f82 [1017690,31843] 0 2024-04-02T04:42:56.261 INFO:tasks.workunit.client.0.smithi016.stdout:7/953: dwrite d0/d46/d37/dbe/dae/df8/da9/f112 [0,4194304] 0 2024-04-02T04:42:56.261 INFO:tasks.workunit.client.1.smithi027.stdout:2/435: creat d1/d91/f9c x:0 0 0 2024-04-02T04:42:56.262 INFO:tasks.workunit.client.0.smithi016.stdout:6/987: dwrite d0/d9/fc2 [0,4194304] 0 2024-04-02T04:42:56.264 INFO:tasks.workunit.client.1.smithi027.stdout:5/633: dwrite d0/d1d/f7c [0,4194304] 0 2024-04-02T04:42:56.265 INFO:tasks.workunit.client.0.smithi016.stdout:0/942: creat d0/d16/d24/d62/d9c/dfb/f135 x:0 0 0 2024-04-02T04:42:56.278 INFO:tasks.workunit.client.1.smithi027.stdout:9/452: link d1a/d1d/d29/d50/d73/f7b d1a/d1d/d3c/d6e/f97 0 2024-04-02T04:42:56.280 INFO:tasks.workunit.client.1.smithi027.stdout:3/380: dwrite d1/de/df/d13/f36 [0,4194304] 0 2024-04-02T04:42:56.288 INFO:tasks.workunit.client.0.smithi016.stdout:5/937: mknod d3/d40/d4f/d67/c150 0 2024-04-02T04:42:56.288 INFO:tasks.workunit.client.1.smithi027.stdout:3/381: dwrite d1/f46 [4194304,4194304] 0 2024-04-02T04:42:56.292 INFO:tasks.workunit.client.1.smithi027.stdout:4/354: rename d24/d38/f3d to d24/d48/f72 0 2024-04-02T04:42:56.296 INFO:tasks.workunit.client.0.smithi016.stdout:0/943: chown d0/d48/fb9 256973943 1 2024-04-02T04:42:56.297 INFO:tasks.workunit.client.0.smithi016.stdout:8/984: dread d1/d6/d137/d49/d4a/f10d [0,4194304] 0 2024-04-02T04:42:56.299 INFO:tasks.workunit.client.0.smithi016.stdout:8/985: fsync d1/d6/d137/d45/fb9 0 2024-04-02T04:42:56.299 INFO:tasks.workunit.client.1.smithi027.stdout:1/430: fsync d0/d1e/d25/f50 0 2024-04-02T04:42:56.303 INFO:tasks.workunit.client.1.smithi027.stdout:2/436: mknod d1/d91/c9d 0 2024-04-02T04:42:56.303 INFO:tasks.workunit.client.0.smithi016.stdout:5/938: truncate d3/df/d18/d2f/f62 110812 0 2024-04-02T04:42:56.306 INFO:tasks.workunit.client.1.smithi027.stdout:7/455: getdents d2/d7/da 0 2024-04-02T04:42:56.308 INFO:tasks.workunit.client.1.smithi027.stdout:2/437: write d1/db/d3b/d17/d2b/f26 [4823958,50668] 0 2024-04-02T04:42:56.308 INFO:tasks.workunit.client.1.smithi027.stdout:5/634: mknod d0/df/d29/ce0 0 2024-04-02T04:42:56.309 INFO:tasks.workunit.client.1.smithi027.stdout:8/478: dwrite d5/d8/db/df/f82 [0,4194304] 0 2024-04-02T04:42:56.309 INFO:tasks.workunit.client.0.smithi016.stdout:5/939: dwrite d3/df/f82 [0,4194304] 0 2024-04-02T04:42:56.311 INFO:tasks.workunit.client.0.smithi016.stdout:1/941: creat d14/d19/d1e/f131 x:0 0 0 2024-04-02T04:42:56.312 INFO:tasks.workunit.client.1.smithi027.stdout:5/635: readlink d0/d68/d6b/ld5 0 2024-04-02T04:42:56.312 INFO:tasks.workunit.client.1.smithi027.stdout:4/355: dread f12 [0,4194304] 0 2024-04-02T04:42:56.312 INFO:tasks.workunit.client.0.smithi016.stdout:1/942: write d14/d19/d20/f3b [2568427,49606] 0 2024-04-02T04:42:56.314 INFO:tasks.workunit.client.1.smithi027.stdout:9/453: symlink d1a/d1d/d29/d37/d3a/l98 0 2024-04-02T04:42:56.316 INFO:tasks.workunit.client.1.smithi027.stdout:5/636: dread d0/d1a/f49 [0,4194304] 0 2024-04-02T04:42:56.320 INFO:tasks.workunit.client.1.smithi027.stdout:6/409: mkdir d13/d1b/d87 0 2024-04-02T04:42:56.323 INFO:tasks.workunit.client.1.smithi027.stdout:3/382: chown d1/de/df/c27 222577936 1 2024-04-02T04:42:56.323 INFO:tasks.workunit.client.0.smithi016.stdout:6/988: symlink d0/d9/d16/d56/d89/d67/d6f/d7e/ded/l137 0 2024-04-02T04:42:56.326 INFO:tasks.workunit.client.1.smithi027.stdout:0/487: dwrite d4/f60 [0,4194304] 0 2024-04-02T04:42:56.330 INFO:tasks.workunit.client.0.smithi016.stdout:0/944: write d0/d16/d24/d4e/d8d/d71/f7d [256603,90384] 0 2024-04-02T04:42:56.330 INFO:tasks.workunit.client.0.smithi016.stdout:0/945: fdatasync d0/d16/d2e/d37/f108 0 2024-04-02T04:42:56.332 INFO:tasks.workunit.client.1.smithi027.stdout:1/431: rename d0/de/l1a to d0/de/d52/d74/l90 0 2024-04-02T04:42:56.336 INFO:tasks.workunit.client.0.smithi016.stdout:4/968: getdents d5/d17/d18/d6d/d8e/d9c/dcc/d6f 0 2024-04-02T04:42:56.340 INFO:tasks.workunit.client.0.smithi016.stdout:5/940: mkdir d3/d40/d4f/d151 0 2024-04-02T04:42:56.343 INFO:tasks.workunit.client.0.smithi016.stdout:0/946: creat d0/d16/d24/d4e/d8d/d71/f136 x:0 0 0 2024-04-02T04:42:56.343 INFO:tasks.workunit.client.1.smithi027.stdout:7/456: read d2/d1d/f74 [1481356,103657] 0 2024-04-02T04:42:56.344 INFO:tasks.workunit.client.1.smithi027.stdout:4/356: chown d24/c2e 91 1 2024-04-02T04:42:56.346 INFO:tasks.workunit.client.0.smithi016.stdout:8/986: creat d1/d6/d36/d3f/f13f x:0 0 0 2024-04-02T04:42:56.347 INFO:tasks.workunit.client.1.smithi027.stdout:7/457: dread d2/d7/da/dc/d16/d2f/f73 [0,4194304] 0 2024-04-02T04:42:56.347 INFO:tasks.workunit.client.0.smithi016.stdout:8/987: write d1/d6/d36/d3f/d77/fae [2665859,63518] 0 2024-04-02T04:42:56.348 INFO:tasks.workunit.client.0.smithi016.stdout:1/943: write d14/d19/d1e/d71/dc0/fc2 [151146,59896] 0 2024-04-02T04:42:56.350 INFO:tasks.workunit.client.0.smithi016.stdout:4/969: creat d5/d1d/d5f/d121/f12b x:0 0 0 2024-04-02T04:42:56.352 INFO:tasks.workunit.client.0.smithi016.stdout:4/970: truncate d5/d1d/d1e/da5/df9/f11b 780410 0 2024-04-02T04:42:56.353 INFO:tasks.workunit.client.1.smithi027.stdout:6/410: readlink d13/d1b/d24/d25/d27/l79 0 2024-04-02T04:42:56.353 INFO:tasks.workunit.client.0.smithi016.stdout:5/941: fsync d3/df/d18/d26/f106 0 2024-04-02T04:42:56.354 INFO:tasks.workunit.client.0.smithi016.stdout:5/942: write d3/d40/d4f/d6b/d45/f51 [855958,118127] 0 2024-04-02T04:42:56.355 INFO:tasks.workunit.client.0.smithi016.stdout:0/947: dread - d0/d48/fb0 zero size 2024-04-02T04:42:56.356 INFO:tasks.workunit.client.1.smithi027.stdout:9/454: write d1a/d1d/d29/d37/d20/f30 [175074,130846] 0 2024-04-02T04:42:56.357 INFO:tasks.workunit.client.0.smithi016.stdout:0/948: chown d0/d16/d24/d62/d9c/d8c/lc4 255 1 2024-04-02T04:42:56.357 INFO:tasks.workunit.client.0.smithi016.stdout:0/949: stat d0/d4f/d76/ce4 0 2024-04-02T04:42:56.358 INFO:tasks.workunit.client.1.smithi027.stdout:9/455: fdatasync d1a/d1d/d29/d37/d20/d7e/f87 0 2024-04-02T04:42:56.359 INFO:tasks.workunit.client.0.smithi016.stdout:8/988: symlink d1/d31/d67/l140 0 2024-04-02T04:42:56.360 INFO:tasks.workunit.client.1.smithi027.stdout:0/488: creat d4/d21/d28/fa5 x:0 0 0 2024-04-02T04:42:56.363 INFO:tasks.workunit.client.0.smithi016.stdout:8/989: truncate d1/d6/d36/d3f/d77/d11a/f138 346960 0 2024-04-02T04:42:56.365 INFO:tasks.workunit.client.0.smithi016.stdout:4/971: unlink d5/d1d/d6c/l68 0 2024-04-02T04:42:56.366 INFO:tasks.workunit.client.1.smithi027.stdout:1/432: symlink d0/de/d52/l91 0 2024-04-02T04:42:56.366 INFO:tasks.workunit.client.1.smithi027.stdout:1/433: fsync d0/d1e/d5d/f71 0 2024-04-02T04:42:56.367 INFO:tasks.workunit.client.1.smithi027.stdout:8/479: rename d5/l92 to d5/d8/d5c/l99 0 2024-04-02T04:42:56.367 INFO:tasks.workunit.client.0.smithi016.stdout:6/989: dwrite d0/f5d [0,4194304] 0 2024-04-02T04:42:56.367 INFO:tasks.workunit.client.0.smithi016.stdout:4/972: write d5/d1d/d5f/d95/df2/f106 [1911210,108573] 0 2024-04-02T04:42:56.367 INFO:tasks.workunit.client.0.smithi016.stdout:4/973: fdatasync d5/d1d/d5f/d95/f10d 0 2024-04-02T04:42:56.368 INFO:tasks.workunit.client.0.smithi016.stdout:4/974: chown d5/de/f79 2098 1 2024-04-02T04:42:56.368 INFO:tasks.workunit.client.1.smithi027.stdout:8/480: dread - d5/d8/db/f8e zero size 2024-04-02T04:42:56.370 INFO:tasks.workunit.client.0.smithi016.stdout:4/975: chown d5/d1d/d1e/da5/ffa 0 1 2024-04-02T04:42:56.370 INFO:tasks.workunit.client.1.smithi027.stdout:4/357: mknod d24/d38/d47/d62/c73 0 2024-04-02T04:42:56.371 INFO:tasks.workunit.client.1.smithi027.stdout:8/481: chown d5/d8/db/dd/d5f/d27/c80 0 1 2024-04-02T04:42:56.372 INFO:tasks.workunit.client.1.smithi027.stdout:1/434: dwrite d0/de/f80 [0,4194304] 0 2024-04-02T04:42:56.372 INFO:tasks.workunit.client.0.smithi016.stdout:1/944: dwrite d14/d19/d1e/d6d/f8b [0,4194304] 0 2024-04-02T04:42:56.374 INFO:tasks.workunit.client.0.smithi016.stdout:1/945: read - d14/d19/d1e/d11e/f12f zero size 2024-04-02T04:42:56.383 INFO:tasks.workunit.client.0.smithi016.stdout:8/990: dread d1/d6/d137/fca [0,4194304] 0 2024-04-02T04:42:56.385 INFO:tasks.workunit.client.1.smithi027.stdout:3/383: symlink d1/de/d55/l88 0 2024-04-02T04:42:56.386 INFO:tasks.workunit.client.0.smithi016.stdout:5/943: mknod d3/d40/d4f/d67/dbf/d104/d143/c152 0 2024-04-02T04:42:56.386 INFO:tasks.workunit.client.0.smithi016.stdout:4/976: rmdir d5/d17 39 2024-04-02T04:42:56.387 INFO:tasks.workunit.client.1.smithi027.stdout:0/489: creat d4/d21/d85/fa6 x:0 0 0 2024-04-02T04:42:56.389 INFO:tasks.workunit.client.0.smithi016.stdout:0/950: creat d0/d16/d24/d4e/d8d/d71/df1/d124/f137 x:0 0 0 2024-04-02T04:42:56.392 INFO:tasks.workunit.client.1.smithi027.stdout:2/438: creat d1/db/d3b/d17/f9e x:0 0 0 2024-04-02T04:42:56.393 INFO:tasks.workunit.client.1.smithi027.stdout:2/439: write d1/f4d [304955,33732] 0 2024-04-02T04:42:56.394 INFO:tasks.workunit.client.1.smithi027.stdout:4/358: symlink d24/d38/d47/d5c/l74 0 2024-04-02T04:42:56.398 INFO:tasks.workunit.client.1.smithi027.stdout:5/637: dwrite d0/d1a/d20/fde [0,4194304] 0 2024-04-02T04:42:56.398 INFO:tasks.workunit.client.0.smithi016.stdout:1/946: dwrite d14/d19/d50/f127 [0,4194304] 0 2024-04-02T04:42:56.399 INFO:tasks.workunit.client.1.smithi027.stdout:7/458: mknod d2/d7/da/dc/d16/c88 0 2024-04-02T04:42:56.402 INFO:tasks.workunit.client.0.smithi016.stdout:7/954: sync 2024-04-02T04:42:56.402 INFO:tasks.workunit.client.0.smithi016.stdout:5/944: fdatasync d3/f4 0 2024-04-02T04:42:56.403 INFO:tasks.workunit.client.0.smithi016.stdout:8/991: dread d1/d6/d137/d49/f43 [0,4194304] 0 2024-04-02T04:42:56.405 INFO:tasks.workunit.client.0.smithi016.stdout:5/945: truncate d3/fad 4335952 0 2024-04-02T04:42:56.405 INFO:tasks.workunit.client.1.smithi027.stdout:8/482: rename d5/d8/db/dd/d4a/f87 to d5/d8/d5c/f9a 0 2024-04-02T04:42:56.406 INFO:tasks.workunit.client.0.smithi016.stdout:1/947: dwrite d14/d19/d1d/d29/d39/d46/dcb/d43/d68/dc1/fc4 [0,4194304] 0 2024-04-02T04:42:56.409 INFO:tasks.workunit.client.0.smithi016.stdout:1/948: chown d14/d19/d1d/d29/d39/d46/dcb/d43/d68/dc1 644796906 1 2024-04-02T04:42:56.412 INFO:tasks.workunit.client.1.smithi027.stdout:8/483: dread d5/d8/d5c/f9a [0,4194304] 0 2024-04-02T04:42:56.414 INFO:tasks.workunit.client.1.smithi027.stdout:6/411: mknod d13/d1b/d24/d25/d56/d6d/c88 0 2024-04-02T04:42:56.414 INFO:tasks.workunit.client.0.smithi016.stdout:6/990: write d0/d78/f8c [5064073,121177] 0 2024-04-02T04:42:56.416 INFO:tasks.workunit.client.1.smithi027.stdout:3/384: mknod d1/d1a/c89 0 2024-04-02T04:42:56.418 INFO:tasks.workunit.client.1.smithi027.stdout:9/456: mkdir d1a/d1d/d29/d37/d99 0 2024-04-02T04:42:56.421 INFO:tasks.workunit.client.1.smithi027.stdout:7/459: dread d2/d7/da/dc/f30 [0,4194304] 0 2024-04-02T04:42:56.421 INFO:tasks.workunit.client.0.smithi016.stdout:7/955: mkdir d0/d46/da2/d13b 0 2024-04-02T04:42:56.421 INFO:tasks.workunit.client.0.smithi016.stdout:8/992: dread d1/d70/d71/d74/fbd [0,4194304] 0 2024-04-02T04:42:56.422 INFO:tasks.workunit.client.1.smithi027.stdout:4/359: creat d24/d48/f75 x:0 0 0 2024-04-02T04:42:56.422 INFO:tasks.workunit.client.1.smithi027.stdout:7/460: write d2/d1c/f3b [4364261,126425] 0 2024-04-02T04:42:56.422 INFO:tasks.workunit.client.0.smithi016.stdout:7/956: write d0/d46/d52/d82/fd3 [1551356,13218] 0 2024-04-02T04:42:56.423 INFO:tasks.workunit.client.0.smithi016.stdout:8/993: readlink d1/d6/d36/d3f/ldd 0 2024-04-02T04:42:56.423 INFO:tasks.workunit.client.0.smithi016.stdout:5/946: creat d3/d40/d4f/d67/dbf/f153 x:0 0 0 2024-04-02T04:42:56.424 INFO:tasks.workunit.client.1.smithi027.stdout:7/461: write d2/d7/da/dc/d1f/d33/d36/f7f [1438383,70933] 0 2024-04-02T04:42:56.427 INFO:tasks.workunit.client.0.smithi016.stdout:4/977: mknod d5/d8/c12c 0 2024-04-02T04:42:56.430 INFO:tasks.workunit.client.1.smithi027.stdout:2/440: rename d1/db/d3b/d17/f3f to d1/db/d3b/d96/f9f 0 2024-04-02T04:42:56.431 INFO:tasks.workunit.client.1.smithi027.stdout:8/484: creat d5/d8/db/d68/f9b x:0 0 0 2024-04-02T04:42:56.434 INFO:tasks.workunit.client.0.smithi016.stdout:6/991: mknod d0/d9/d16/de3/c138 0 2024-04-02T04:42:56.435 INFO:tasks.workunit.client.1.smithi027.stdout:3/385: unlink d1/de/f40 0 2024-04-02T04:42:56.438 INFO:tasks.workunit.client.1.smithi027.stdout:0/490: mknod d4/d16/d71/d97/ca7 0 2024-04-02T04:42:56.441 INFO:tasks.workunit.client.1.smithi027.stdout:5/638: write d0/df/f80 [829822,29375] 0 2024-04-02T04:42:56.442 INFO:tasks.workunit.client.0.smithi016.stdout:8/994: creat d1/d6/d36/d4f/dc9/f141 x:0 0 0 2024-04-02T04:42:56.445 INFO:tasks.workunit.client.0.smithi016.stdout:7/957: write d0/d46/d37/dbe/dae/df8/d43/d96/fba [4869405,82306] 0 2024-04-02T04:42:56.445 INFO:tasks.workunit.client.0.smithi016.stdout:0/951: truncate d0/d16/d2e/d37/d66/fa3 2885848 0 2024-04-02T04:42:56.451 INFO:tasks.workunit.client.1.smithi027.stdout:7/462: mkdir d2/df/d89 0 2024-04-02T04:42:56.453 INFO:tasks.workunit.client.0.smithi016.stdout:8/995: creat d1/d6/d36/d3f/d77/d11a/f142 x:0 0 0 2024-04-02T04:42:56.454 INFO:tasks.workunit.client.0.smithi016.stdout:7/958: rmdir d0/d46/da2/dc8/dda/de3 39 2024-04-02T04:42:56.455 INFO:tasks.workunit.client.1.smithi027.stdout:1/435: truncate d0/f3d 104364 0 2024-04-02T04:42:56.456 INFO:tasks.workunit.client.0.smithi016.stdout:5/947: dwrite d3/f1a [0,4194304] 0 2024-04-02T04:42:56.457 INFO:tasks.workunit.client.1.smithi027.stdout:2/441: creat d1/db/fa0 x:0 0 0 2024-04-02T04:42:56.457 INFO:tasks.workunit.client.1.smithi027.stdout:1/436: write d0/d1e/d2e/d3e/d5f/f79 [2245170,104091] 0 2024-04-02T04:42:56.458 INFO:tasks.workunit.client.0.smithi016.stdout:5/948: write d3/df/d18/d26/d48/f127 [378904,13455] 0 2024-04-02T04:42:56.460 INFO:tasks.workunit.client.0.smithi016.stdout:1/949: rmdir d14/d19/d1d/d29/d129 0 2024-04-02T04:42:56.464 INFO:tasks.workunit.client.0.smithi016.stdout:0/952: unlink d0/d16/d2e/cdb 0 2024-04-02T04:42:56.468 INFO:tasks.workunit.client.0.smithi016.stdout:4/978: mknod d5/d17/d18/d6d/d8e/c12d 0 2024-04-02T04:42:56.469 INFO:tasks.workunit.client.0.smithi016.stdout:6/992: symlink d0/d9/d16/d56/db0/df2/d119/l139 0 2024-04-02T04:42:56.469 INFO:tasks.workunit.client.1.smithi027.stdout:9/457: mknod d1a/d1d/d29/c9a 0 2024-04-02T04:42:56.469 INFO:tasks.workunit.client.0.smithi016.stdout:4/979: write d5/d1d/d1e/f9a [4027026,104481] 0 2024-04-02T04:42:56.470 INFO:tasks.workunit.client.0.smithi016.stdout:8/996: chown d1/d31/l35 134 1 2024-04-02T04:42:56.470 INFO:tasks.workunit.client.1.smithi027.stdout:9/458: chown d1a/d1d/d29/d37/d20/f30 4512964 1 2024-04-02T04:42:56.472 INFO:tasks.workunit.client.0.smithi016.stdout:7/959: dread d0/d46/d52/daf/fc7 [0,4194304] 0 2024-04-02T04:42:56.472 INFO:tasks.workunit.client.1.smithi027.stdout:0/491: rename d4/d16/f17 to d4/d76/fa8 0 2024-04-02T04:42:56.474 INFO:tasks.workunit.client.0.smithi016.stdout:5/949: mknod d3/df/d18/dfd/d10f/c154 0 2024-04-02T04:42:56.476 INFO:tasks.workunit.client.1.smithi027.stdout:5/639: mknod d0/df/d2f/ce1 0 2024-04-02T04:42:56.478 INFO:tasks.workunit.client.1.smithi027.stdout:5/640: dread - d0/df/d29/da5/fd3 zero size 2024-04-02T04:42:56.479 INFO:tasks.workunit.client.1.smithi027.stdout:0/492: dwrite d4/d38/fa1 [0,4194304] 0 2024-04-02T04:42:56.479 INFO:tasks.workunit.client.1.smithi027.stdout:7/463: creat d2/d1c/f8a x:0 0 0 2024-04-02T04:42:56.481 INFO:tasks.workunit.client.1.smithi027.stdout:0/493: stat d4/d38/d66/d2d/d4c/c6e 0 2024-04-02T04:42:56.484 INFO:tasks.workunit.client.0.smithi016.stdout:1/950: creat d14/d19/d1d/d29/d39/d46/dcb/d43/d68/dc1/f132 x:0 0 0 2024-04-02T04:42:56.485 INFO:tasks.workunit.client.0.smithi016.stdout:0/953: dread d0/d16/d2e/d37/d66/ff4 [0,4194304] 0 2024-04-02T04:42:56.487 INFO:tasks.workunit.client.1.smithi027.stdout:1/437: creat d0/de/f92 x:0 0 0 2024-04-02T04:42:56.487 INFO:tasks.workunit.client.1.smithi027.stdout:5/641: dwrite d0/df/d11/d13/d9a/db7/fc6 [0,4194304] 0 2024-04-02T04:42:56.488 INFO:tasks.workunit.client.0.smithi016.stdout:6/993: fdatasync d0/d9/d16/ffd 0 2024-04-02T04:42:56.489 INFO:tasks.workunit.client.0.smithi016.stdout:6/994: fsync d0/d5c/ffb 0 2024-04-02T04:42:56.489 INFO:tasks.workunit.client.1.smithi027.stdout:3/386: write d1/de/df/d13/f21 [1260144,48319] 0 2024-04-02T04:42:56.490 INFO:tasks.workunit.client.1.smithi027.stdout:3/387: chown d1/d1a/c3e 909917 1 2024-04-02T04:42:56.491 INFO:tasks.workunit.client.1.smithi027.stdout:2/442: creat d1/db/d3b/d17/d68/d78/fa1 x:0 0 0 2024-04-02T04:42:56.492 INFO:tasks.workunit.client.0.smithi016.stdout:8/997: readlink d1/d70/d71/l7b 0 2024-04-02T04:42:56.498 INFO:tasks.workunit.client.0.smithi016.stdout:5/950: mknod d3/d40/d4f/d6b/d45/da6/d102/dc4/d108/c155 0 2024-04-02T04:42:56.500 INFO:tasks.workunit.client.1.smithi027.stdout:4/360: creat d24/f76 x:0 0 0 2024-04-02T04:42:56.503 INFO:tasks.workunit.client.1.smithi027.stdout:4/361: write d24/d38/d47/d62/f6c [996112,34722] 0 2024-04-02T04:42:56.506 INFO:tasks.workunit.client.1.smithi027.stdout:9/459: dwrite d1a/d1d/d29/d37/d20/d7e/f85 [0,4194304] 0 2024-04-02T04:42:56.507 INFO:tasks.workunit.client.1.smithi027.stdout:1/438: creat d0/d1e/d25/f93 x:0 0 0 2024-04-02T04:42:56.509 INFO:tasks.workunit.client.1.smithi027.stdout:3/388: write d1/d1a/f79 [682735,118172] 0 2024-04-02T04:42:56.512 INFO:tasks.workunit.client.1.smithi027.stdout:2/443: mknod d1/db/d3b/d17/d68/d76/ca2 0 2024-04-02T04:42:56.519 INFO:tasks.workunit.client.1.smithi027.stdout:0/494: link f2 d4/d38/fa9 0 2024-04-02T04:42:56.519 INFO:tasks.workunit.client.1.smithi027.stdout:0/495: stat d4/d38/da0 0 2024-04-02T04:42:56.523 INFO:tasks.workunit.client.1.smithi027.stdout:5/642: creat d0/d68/dd7/fe2 x:0 0 0 2024-04-02T04:42:56.525 INFO:tasks.workunit.client.1.smithi027.stdout:4/362: creat d24/d38/d44/d41/d49/f77 x:0 0 0 2024-04-02T04:42:56.530 INFO:tasks.workunit.client.1.smithi027.stdout:1/439: rmdir d0/d1e/d2e/d3b 39 2024-04-02T04:42:56.534 INFO:tasks.workunit.client.1.smithi027.stdout:2/444: creat d1/db/d3b/d17/d6d/d84/fa3 x:0 0 0 2024-04-02T04:42:56.535 INFO:tasks.workunit.client.1.smithi027.stdout:8/485: getdents d5/d8/db/d68/d95 0 2024-04-02T04:42:56.537 INFO:tasks.workunit.client.1.smithi027.stdout:0/496: symlink d4/d16/d71/laa 0 2024-04-02T04:42:56.538 INFO:tasks.workunit.client.1.smithi027.stdout:0/497: write d4/d21/d28/f54 [30694,15488] 0 2024-04-02T04:42:56.540 INFO:tasks.workunit.client.1.smithi027.stdout:4/363: mknod d24/d2d/c78 0 2024-04-02T04:42:56.542 INFO:tasks.workunit.client.1.smithi027.stdout:9/460: mkdir d1a/d1d/d29/d37/d9b 0 2024-04-02T04:42:56.547 INFO:tasks.workunit.client.1.smithi027.stdout:3/389: link d1/de/df/d13/d73/d77/f7a d1/de/f8a 0 2024-04-02T04:42:56.548 INFO:tasks.workunit.client.1.smithi027.stdout:9/461: dwrite d1a/d1d/d3c/f94 [0,4194304] 0 2024-04-02T04:42:56.550 INFO:tasks.workunit.client.1.smithi027.stdout:2/445: creat d1/db/d3b/d96/fa4 x:0 0 0 2024-04-02T04:42:56.551 INFO:tasks.workunit.client.1.smithi027.stdout:9/462: truncate d1a/d1d/d29/d37/d3a/f84 407327 0 2024-04-02T04:42:56.553 INFO:tasks.workunit.client.1.smithi027.stdout:8/486: truncate d5/d8/d48/f8b 812262 0 2024-04-02T04:42:56.554 INFO:tasks.workunit.client.1.smithi027.stdout:9/463: truncate d1a/d1d/d29/d51/f60 329741 0 2024-04-02T04:42:56.555 INFO:tasks.workunit.client.1.smithi027.stdout:9/464: chown d1a/d1d/d29/d56/c59 6 1 2024-04-02T04:42:56.559 INFO:tasks.workunit.client.1.smithi027.stdout:0/498: unlink d4/d38/l7e 0 2024-04-02T04:42:56.560 INFO:tasks.workunit.client.1.smithi027.stdout:4/364: symlink d24/d38/d47/d62/l79 0 2024-04-02T04:42:56.561 INFO:tasks.workunit.client.1.smithi027.stdout:4/365: chown d24/d38/d44/f52 13 1 2024-04-02T04:42:56.565 INFO:tasks.workunit.client.1.smithi027.stdout:3/390: dread d1/de/df/d13/d73/d77/f7e [0,4194304] 0 2024-04-02T04:42:56.570 INFO:tasks.workunit.client.1.smithi027.stdout:8/487: creat d5/d8/db/dd/d5f/d27/d31/f9c x:0 0 0 2024-04-02T04:42:56.572 INFO:tasks.workunit.client.1.smithi027.stdout:9/465: unlink d1a/d1d/f42 0 2024-04-02T04:42:56.573 INFO:tasks.workunit.client.1.smithi027.stdout:9/466: chown d1a/d1d/d29/d56/d8b/f90 0 1 2024-04-02T04:42:56.574 INFO:tasks.workunit.client.1.smithi027.stdout:0/499: fdatasync d4/d38/da0/f9e 0 2024-04-02T04:42:56.575 INFO:tasks.workunit.client.1.smithi027.stdout:9/467: write d1a/d1d/d29/d37/d20/f61 [1668673,42479] 0 2024-04-02T04:42:56.576 INFO:tasks.workunit.client.1.smithi027.stdout:4/366: readlink d24/d38/d44/l67 0 2024-04-02T04:42:56.581 INFO:tasks.workunit.client.1.smithi027.stdout:3/391: rmdir d1/de/df 39 2024-04-02T04:42:56.582 INFO:tasks.workunit.client.1.smithi027.stdout:4/367: dwrite d24/d48/f75 [0,4194304] 0 2024-04-02T04:42:56.585 INFO:tasks.workunit.client.1.smithi027.stdout:5/643: truncate d0/df/d11/d13/f3e 1562435 0 2024-04-02T04:42:56.586 INFO:tasks.workunit.client.1.smithi027.stdout:2/446: mknod d1/db/d3b/d17/ca5 0 2024-04-02T04:42:56.588 INFO:tasks.workunit.client.1.smithi027.stdout:8/488: fdatasync d5/d8/db/dd/d5f/f2b 0 2024-04-02T04:42:56.593 INFO:tasks.workunit.client.1.smithi027.stdout:8/489: dread d5/d8/db/df/f82 [0,4194304] 0 2024-04-02T04:42:56.594 INFO:tasks.workunit.client.1.smithi027.stdout:0/500: mkdir d4/d21/d28/d4e/dab 0 2024-04-02T04:42:56.602 INFO:tasks.workunit.client.1.smithi027.stdout:9/468: truncate d1a/f38 4452225 0 2024-04-02T04:42:56.602 INFO:tasks.workunit.client.1.smithi027.stdout:4/368: symlink d24/d38/d47/d62/l7a 0 2024-04-02T04:42:56.604 INFO:tasks.workunit.client.1.smithi027.stdout:3/392: truncate d1/de/df/d61/f6e 1047348 0 2024-04-02T04:42:56.606 INFO:tasks.workunit.client.1.smithi027.stdout:1/440: dwrite d0/f37 [0,4194304] 0 2024-04-02T04:42:56.609 INFO:tasks.workunit.client.1.smithi027.stdout:2/447: write d1/db/d3b/d17/f63 [5232515,71511] 0 2024-04-02T04:42:56.620 INFO:tasks.workunit.client.0.smithi016.stdout:7/960: creat d0/d133/d11b/f13c x:0 0 0 2024-04-02T04:42:56.623 INFO:tasks.workunit.client.0.smithi016.stdout:5/951: symlink d3/d40/d4f/d6b/d45/da6/d102/dc4/d108/d6d/l156 0 2024-04-02T04:42:56.624 INFO:tasks.workunit.client.1.smithi027.stdout:2/448: dread d1/db/f32 [0,4194304] 0 2024-04-02T04:42:56.624 INFO:tasks.workunit.client.1.smithi027.stdout:1/441: dread d0/f1d [0,4194304] 0 2024-04-02T04:42:56.625 INFO:tasks.workunit.client.1.smithi027.stdout:8/490: dwrite d5/d8/f47 [0,4194304] 0 2024-04-02T04:42:56.625 INFO:tasks.workunit.client.1.smithi027.stdout:8/491: readlink l1 0 2024-04-02T04:42:56.631 INFO:tasks.workunit.client.1.smithi027.stdout:5/644: write d0/d1d/fbd [931105,24411] 0 2024-04-02T04:42:56.631 INFO:tasks.workunit.client.1.smithi027.stdout:9/469: dread d1a/d1d/d29/d37/d3a/f54 [4194304,4194304] 0 2024-04-02T04:42:56.639 INFO:tasks.workunit.client.0.smithi016.stdout:8/998: dwrite d1/d70/d71/fb3 [0,4194304] 0 2024-04-02T04:42:56.640 INFO:tasks.workunit.client.0.smithi016.stdout:8/999: chown d1/d6/d137/d49/d4a/dfc/f102 862269681 1 2024-04-02T04:42:56.644 INFO:tasks.workunit.client.0.smithi016.stdout:0/954: truncate d0/d16/d2e/d36/f10f 290669 0 2024-04-02T04:42:56.648 INFO:tasks.workunit.client.0.smithi016.stdout:6/995: creat d0/d9/d16/d46/f13a x:0 0 0 2024-04-02T04:42:56.650 INFO:tasks.workunit.client.0.smithi016.stdout:4/980: getdents d5/d1d/d5f/d121 0 2024-04-02T04:42:56.656 INFO:tasks.workunit.client.0.smithi016.stdout:1/951: getdents d14/d19/d1e/d11e 0 2024-04-02T04:42:56.657 INFO:tasks.workunit.client.0.smithi016.stdout:6/996: creat d0/d9/d16/d56/d89/d67/d6f/d8a/d91/f13b x:0 0 0 2024-04-02T04:42:56.660 INFO:tasks.workunit.client.0.smithi016.stdout:5/952: write d3/df/dd5/fef [1039434,83932] 0 2024-04-02T04:42:56.664 INFO:tasks.workunit.client.0.smithi016.stdout:7/961: dwrite d0/d46/d37/dbe/dae/df8/d43/d96/ddf/f101 [0,4194304] 0 2024-04-02T04:42:56.666 INFO:tasks.workunit.client.1.smithi027.stdout:0/501: dread d4/d38/d66/d2d/d4c/f5c [0,4194304] 0 2024-04-02T04:42:56.670 INFO:tasks.workunit.client.1.smithi027.stdout:4/369: rmdir d24 39 2024-04-02T04:42:56.676 INFO:tasks.workunit.client.1.smithi027.stdout:4/370: dwrite fc [0,4194304] 0 2024-04-02T04:42:56.677 INFO:tasks.workunit.client.1.smithi027.stdout:4/371: chown ld 399116281 1 2024-04-02T04:42:56.679 INFO:tasks.workunit.client.0.smithi016.stdout:0/955: dwrite d0/d16/fb8 [4194304,4194304] 0 2024-04-02T04:42:56.680 INFO:tasks.workunit.client.0.smithi016.stdout:0/956: readlink d0/d4f/lb7 0 2024-04-02T04:42:56.683 INFO:tasks.workunit.client.1.smithi027.stdout:1/442: rmdir d0/d1e/d5d/d60 39 2024-04-02T04:42:56.683 INFO:tasks.workunit.client.0.smithi016.stdout:6/997: dread d0/d9/d16/d56/f10e [0,4194304] 0 2024-04-02T04:42:56.684 INFO:tasks.workunit.client.1.smithi027.stdout:1/443: write d0/de/f92 [19628,29199] 0 2024-04-02T04:42:56.686 INFO:tasks.workunit.client.0.smithi016.stdout:1/952: write d14/d19/d7a/da2/fe4 [553420,9211] 0 2024-04-02T04:42:56.686 INFO:tasks.workunit.client.1.smithi027.stdout:2/449: creat d1/db/d3b/d17/d29/fa6 x:0 0 0 2024-04-02T04:42:56.688 INFO:tasks.workunit.client.0.smithi016.stdout:5/953: creat d3/d40/d4f/d6b/f157 x:0 0 0 2024-04-02T04:42:56.690 INFO:tasks.workunit.client.1.smithi027.stdout:8/492: rename d5/d8/d15/l1c to d5/d8/db/dd/d5f/d27/d31/d5b/d6c/l9d 0 2024-04-02T04:42:56.691 INFO:tasks.workunit.client.0.smithi016.stdout:4/981: write d5/d16/d3f/f64 [1809228,86929] 0 2024-04-02T04:42:56.691 INFO:tasks.workunit.client.0.smithi016.stdout:0/957: fdatasync d0/d16/d24/fa1 0 2024-04-02T04:42:56.693 INFO:tasks.workunit.client.1.smithi027.stdout:5/645: mknod d0/df/d29/ce3 0 2024-04-02T04:42:56.693 INFO:tasks.workunit.client.1.smithi027.stdout:5/646: stat d0/f66 0 2024-04-02T04:42:56.697 INFO:tasks.workunit.client.1.smithi027.stdout:9/470: mknod d1a/d1d/d29/d50/c9c 0 2024-04-02T04:42:56.698 INFO:tasks.workunit.client.1.smithi027.stdout:9/471: chown d1a/d1d/d29/d56 3975902 1 2024-04-02T04:42:56.701 INFO:tasks.workunit.client.0.smithi016.stdout:7/962: truncate d0/d46/d37/dbe/dae/df8/d43/d96/ddf/f101 200898 0 2024-04-02T04:42:56.703 INFO:tasks.workunit.client.1.smithi027.stdout:9/472: dread d1a/f39 [0,4194304] 0 2024-04-02T04:42:56.703 INFO:tasks.workunit.client.1.smithi027.stdout:0/502: truncate d4/d21/d28/f44 994092 0 2024-04-02T04:42:56.710 INFO:tasks.workunit.client.0.smithi016.stdout:1/953: symlink d14/d19/d1d/d29/l133 0 2024-04-02T04:42:56.711 INFO:tasks.workunit.client.1.smithi027.stdout:9/473: dwrite d1a/d1d/d3c/f5e [0,4194304] 0 2024-04-02T04:42:56.711 INFO:tasks.workunit.client.0.smithi016.stdout:1/954: write d14/d19/d1d/f2c [3637777,59778] 0 2024-04-02T04:42:56.714 INFO:tasks.workunit.client.0.smithi016.stdout:1/955: dread d14/d19/d1d/d29/d39/d46/dcb/f114 [0,4194304] 0 2024-04-02T04:42:56.716 INFO:tasks.workunit.client.1.smithi027.stdout:2/450: symlink d1/db/d3b/d17/d68/la7 0 2024-04-02T04:42:56.717 INFO:tasks.workunit.client.0.smithi016.stdout:0/958: symlink d0/d16/dd6/l138 0 2024-04-02T04:42:56.721 INFO:tasks.workunit.client.0.smithi016.stdout:4/982: creat d5/d17/d18/d6d/da4/db7/dee/d104/f12e x:0 0 0 2024-04-02T04:42:56.723 INFO:tasks.workunit.client.1.smithi027.stdout:8/493: dread d5/d8/db/dd/d5f/d27/f3a [0,4194304] 0 2024-04-02T04:42:56.729 INFO:tasks.workunit.client.1.smithi027.stdout:0/503: creat d4/d38/d66/d2d/fac x:0 0 0 2024-04-02T04:42:56.729 INFO:tasks.workunit.client.0.smithi016.stdout:6/998: dread d0/d9/fd [4194304,4194304] 0 2024-04-02T04:42:56.732 INFO:tasks.workunit.client.0.smithi016.stdout:5/954: dwrite d3/df/d18/d26/d70/fca [0,4194304] 0 2024-04-02T04:42:56.732 INFO:tasks.workunit.client.1.smithi027.stdout:9/474: readlink d1a/d1d/d3c/l44 0 2024-04-02T04:42:56.736 INFO:tasks.workunit.client.0.smithi016.stdout:4/983: creat d5/d17/dcd/def/f12f x:0 0 0 2024-04-02T04:42:56.740 INFO:tasks.workunit.client.0.smithi016.stdout:7/963: dwrite d0/d46/d16/dc2/dea/d64/f10b [0,4194304] 0 2024-04-02T04:42:56.741 INFO:tasks.workunit.client.1.smithi027.stdout:4/372: dwrite f16 [0,4194304] 0 2024-04-02T04:42:56.745 INFO:tasks.workunit.client.1.smithi027.stdout:1/444: dwrite d0/f7 [4194304,4194304] 0 2024-04-02T04:42:56.747 INFO:tasks.workunit.client.1.smithi027.stdout:5/647: link d0/df/d11/d13/f1e d0/df/d11/d13/dce/d55/fe4 0 2024-04-02T04:42:56.748 INFO:tasks.workunit.client.0.smithi016.stdout:0/959: dread d0/d16/f7a [0,4194304] 0 2024-04-02T04:42:56.750 INFO:tasks.workunit.client.1.smithi027.stdout:8/494: write d5/d8/d5c/f9a [2876324,45097] 0 2024-04-02T04:42:56.753 INFO:tasks.workunit.client.0.smithi016.stdout:6/999: rename d0/d9/d16/d46/d5e/fdb to d0/d78/f13c 0 2024-04-02T04:42:56.755 INFO:tasks.workunit.client.1.smithi027.stdout:8/495: dread d5/d8/d15/f39 [0,4194304] 0 2024-04-02T04:42:56.756 INFO:tasks.workunit.client.0.smithi016.stdout:5/955: creat d3/d40/d4f/d67/dbf/d104/f158 x:0 0 0 2024-04-02T04:42:56.758 INFO:tasks.workunit.client.0.smithi016.stdout:5/956: write d3/d40/d4f/d6b/d45/da6/d102/ff6 [1377245,44828] 0 2024-04-02T04:42:56.762 INFO:tasks.workunit.client.1.smithi027.stdout:9/475: symlink d1a/d1d/d3c/d71/l9d 0 2024-04-02T04:42:56.763 INFO:tasks.workunit.client.0.smithi016.stdout:1/956: truncate d14/d19/d1e/d6d/fe8 332917 0 2024-04-02T04:42:56.765 INFO:tasks.workunit.client.1.smithi027.stdout:2/451: rename d1/db/d3b/d17/d68/d78/l9a to d1/db/d3b/d61/la8 0 2024-04-02T04:42:56.766 INFO:tasks.workunit.client.1.smithi027.stdout:9/476: dread d1a/d1d/d3c/f5e [0,4194304] 0 2024-04-02T04:42:56.766 INFO:tasks.workunit.client.1.smithi027.stdout:2/452: fdatasync d1/db/d3b/d17/fc 0 2024-04-02T04:42:56.767 INFO:tasks.workunit.client.1.smithi027.stdout:2/453: fsync d1/db/d3b/d17/d29/f4f 0 2024-04-02T04:42:56.768 INFO:tasks.workunit.client.1.smithi027.stdout:9/477: write d1a/d1d/d29/d37/d20/f61 [1614027,27988] 0 2024-04-02T04:42:56.769 INFO:tasks.workunit.client.1.smithi027.stdout:1/445: rmdir d0/d1e/d25/d3c 39 2024-04-02T04:42:56.773 INFO:tasks.workunit.client.1.smithi027.stdout:5/648: mkdir d0/d68/d6b/de5 0 2024-04-02T04:42:56.775 INFO:tasks.workunit.client.1.smithi027.stdout:9/478: dwrite d1a/f38 [0,4194304] 0 2024-04-02T04:42:56.778 INFO:tasks.workunit.client.0.smithi016.stdout:4/984: dwrite d5/d1d/d6c/f72 [0,4194304] 0 2024-04-02T04:42:56.787 INFO:tasks.workunit.client.0.smithi016.stdout:5/957: write d3/f6 [845521,48886] 0 2024-04-02T04:42:56.788 INFO:tasks.workunit.client.0.smithi016.stdout:5/958: read d3/df/d18/d26/f94 [505864,38946] 0 2024-04-02T04:42:56.789 INFO:tasks.workunit.client.0.smithi016.stdout:0/960: rename d0/d16/dd6/c133 to d0/d16/d2e/c139 0 2024-04-02T04:42:56.791 INFO:tasks.workunit.client.1.smithi027.stdout:4/373: symlink d24/d38/l7b 0 2024-04-02T04:42:56.796 INFO:tasks.workunit.client.1.smithi027.stdout:0/504: dwrite d4/d21/d28/f44 [0,4194304] 0 2024-04-02T04:42:56.799 INFO:tasks.workunit.client.1.smithi027.stdout:9/479: unlink d1a/d1d/d29/d37/d20/f61 0 2024-04-02T04:42:56.803 INFO:tasks.workunit.client.1.smithi027.stdout:4/374: dread d24/d38/d44/d60/f6f [8388608,4194304] 0 2024-04-02T04:42:56.806 INFO:tasks.workunit.client.1.smithi027.stdout:1/446: rename d0/d1e/d25/d3c/c4c to d0/d1e/d2e/d3e/c94 0 2024-04-02T04:42:56.808 INFO:tasks.workunit.client.1.smithi027.stdout:1/447: truncate d0/d1e/d2e/d3e/d5f/f79 4790507 0 2024-04-02T04:42:56.810 INFO:tasks.workunit.client.1.smithi027.stdout:2/454: creat d1/db/d3b/fa9 x:0 0 0 2024-04-02T04:42:56.811 INFO:tasks.workunit.client.0.smithi016.stdout:7/964: link d0/d87/la4 d0/d46/d37/dbe/dae/df8/d4c/d9e/l13d 0 2024-04-02T04:42:56.812 INFO:tasks.workunit.client.1.smithi027.stdout:2/455: fdatasync d1/db/d3b/d17/d29/fa6 0 2024-04-02T04:42:56.812 INFO:tasks.workunit.client.1.smithi027.stdout:4/375: creat d24/d38/d44/d60/f7c x:0 0 0 2024-04-02T04:42:56.812 INFO:tasks.workunit.client.1.smithi027.stdout:8/496: getdents d5/d8/db/dd/d5f/d27/d31/d5b/d6c 0 2024-04-02T04:42:56.814 INFO:tasks.workunit.client.0.smithi016.stdout:5/959: dread d3/f77 [0,4194304] 0 2024-04-02T04:42:56.815 INFO:tasks.workunit.client.1.smithi027.stdout:9/480: symlink d1a/d1d/d29/l9e 0 2024-04-02T04:42:56.816 INFO:tasks.workunit.client.1.smithi027.stdout:9/481: fsync d1a/d1d/d29/d50/d73/f7a 0 2024-04-02T04:42:56.816 INFO:tasks.workunit.client.0.smithi016.stdout:0/961: dread d0/d16/fe8 [0,4194304] 0 2024-04-02T04:42:56.816 INFO:tasks.workunit.client.1.smithi027.stdout:1/448: mknod d0/d1e/d25/d3c/d64/c95 0 2024-04-02T04:42:56.817 INFO:tasks.workunit.client.1.smithi027.stdout:8/497: dread d5/f7d [0,4194304] 0 2024-04-02T04:42:56.817 INFO:tasks.workunit.client.1.smithi027.stdout:9/482: write d1a/d1d/d29/d37/d20/f3f [3535783,52293] 0 2024-04-02T04:42:56.818 INFO:tasks.workunit.client.0.smithi016.stdout:7/965: creat d0/d46/d37/dbe/dae/df8/d4c/f13e x:0 0 0 2024-04-02T04:42:56.818 INFO:tasks.workunit.client.1.smithi027.stdout:8/498: chown d5/l6 61051 1 2024-04-02T04:42:56.818 INFO:tasks.workunit.client.1.smithi027.stdout:5/649: getdents d0/d68/d6b 0 2024-04-02T04:42:56.820 INFO:tasks.workunit.client.1.smithi027.stdout:8/499: chown d5/d8/db/dd/f1d 21570 1 2024-04-02T04:42:56.821 INFO:tasks.workunit.client.1.smithi027.stdout:4/376: unlink d24/f2a 0 2024-04-02T04:42:56.824 INFO:tasks.workunit.client.0.smithi016.stdout:5/960: truncate d3/d40/d72/fd0 38358 0 2024-04-02T04:42:56.830 INFO:tasks.workunit.client.1.smithi027.stdout:1/449: chown d0/d1e/d2e/d3b/l3f 3073649 1 2024-04-02T04:42:56.831 INFO:tasks.workunit.client.1.smithi027.stdout:5/650: mkdir d0/df/d11/d41/de6 0 2024-04-02T04:42:56.832 INFO:tasks.workunit.client.1.smithi027.stdout:5/651: read - d0/df/d11/f58 zero size 2024-04-02T04:42:56.832 INFO:tasks.workunit.client.1.smithi027.stdout:9/483: creat d1a/d1d/d29/d56/d8b/f9f x:0 0 0 2024-04-02T04:42:56.833 INFO:tasks.workunit.client.1.smithi027.stdout:5/652: dread - d0/df/d29/da5/fd3 zero size 2024-04-02T04:42:56.834 INFO:tasks.workunit.client.1.smithi027.stdout:8/500: creat d5/d8/db/dd/d5f/d27/f9e x:0 0 0 2024-04-02T04:42:56.835 INFO:tasks.workunit.client.1.smithi027.stdout:4/377: creat d24/d48/f7d x:0 0 0 2024-04-02T04:42:56.836 INFO:tasks.workunit.client.1.smithi027.stdout:0/505: getdents d4/d38/d66/d4a 0 2024-04-02T04:42:56.837 INFO:tasks.workunit.client.1.smithi027.stdout:2/456: rename d1/db/d3b/l73 to d1/db/d3b/d83/laa 0 2024-04-02T04:42:56.838 INFO:tasks.workunit.client.1.smithi027.stdout:1/450: mkdir d0/d1e/d25/d2b/d59/d96 0 2024-04-02T04:42:56.838 INFO:tasks.workunit.client.1.smithi027.stdout:2/457: write d1/db/d3b/d17/d68/d69/f8d [421603,37043] 0 2024-04-02T04:42:56.840 INFO:tasks.workunit.client.1.smithi027.stdout:1/451: fsync d0/d1e/d25/f93 0 2024-04-02T04:42:56.843 INFO:tasks.workunit.client.1.smithi027.stdout:1/452: write d0/d1e/d25/f31 [1690241,54142] 0 2024-04-02T04:42:56.843 INFO:tasks.workunit.client.1.smithi027.stdout:8/501: mkdir d5/d8/db/dd/d5f/d91/d9f 0 2024-04-02T04:42:56.844 INFO:tasks.workunit.client.1.smithi027.stdout:9/484: dread d1a/f57 [0,4194304] 0 2024-04-02T04:42:56.846 INFO:tasks.workunit.client.1.smithi027.stdout:5/653: link d0/df/d29/f7a d0/df/d11/d13/d9a/d9c/dbe/fe7 0 2024-04-02T04:42:56.847 INFO:tasks.workunit.client.1.smithi027.stdout:8/502: dread - d5/d8/db/d68/f9b zero size 2024-04-02T04:42:56.849 INFO:tasks.workunit.client.1.smithi027.stdout:4/378: rmdir d24 39 2024-04-02T04:42:56.849 INFO:tasks.workunit.client.1.smithi027.stdout:5/654: read - d0/df/d29/da5/fdb zero size 2024-04-02T04:42:56.850 INFO:tasks.workunit.client.1.smithi027.stdout:2/458: dwrite d1/db/d3b/fa9 [0,4194304] 0 2024-04-02T04:42:56.853 INFO:tasks.workunit.client.1.smithi027.stdout:8/503: dread - d5/d8/db/dd/d5f/d27/d31/f84 zero size 2024-04-02T04:42:56.857 INFO:tasks.workunit.client.1.smithi027.stdout:9/485: unlink d1a/d1d/d3c/d71/c80 0 2024-04-02T04:42:56.858 INFO:tasks.workunit.client.1.smithi027.stdout:4/379: dwrite fc [0,4194304] 0 2024-04-02T04:42:56.863 INFO:tasks.workunit.client.1.smithi027.stdout:9/486: dread f15 [0,4194304] 0 2024-04-02T04:42:56.866 INFO:tasks.workunit.client.1.smithi027.stdout:8/504: symlink d5/d8/db/d68/d95/la0 0 2024-04-02T04:42:56.870 INFO:tasks.workunit.client.1.smithi027.stdout:0/506: getdents d4/d38/d66/d2d/d4c 0 2024-04-02T04:42:56.873 INFO:tasks.workunit.client.1.smithi027.stdout:1/453: dwrite d0/f5 [0,4194304] 0 2024-04-02T04:42:56.876 INFO:tasks.workunit.client.1.smithi027.stdout:9/487: dwrite d1a/d1d/d29/f4f [0,4194304] 0 2024-04-02T04:42:56.876 INFO:tasks.workunit.client.1.smithi027.stdout:0/507: dread - d4/d38/d66/d4a/f61 zero size 2024-04-02T04:42:56.877 INFO:tasks.workunit.client.1.smithi027.stdout:0/508: stat d4/d21/d85/f86 0 2024-04-02T04:42:56.878 INFO:tasks.workunit.client.1.smithi027.stdout:0/509: write d4/d16/f89 [103285,116192] 0 2024-04-02T04:42:56.879 INFO:tasks.workunit.client.0.smithi016.stdout:4/985: getdents d5/d17/d18/d1f 0 2024-04-02T04:42:56.880 INFO:tasks.workunit.client.0.smithi016.stdout:5/961: chown d3/d40/d4f/d6b/d45/da6/c14d 344436463 1 2024-04-02T04:42:56.881 INFO:tasks.workunit.client.1.smithi027.stdout:8/505: rename d5/d8/db/df/f82 to d5/d8/db/d57/fa1 0 2024-04-02T04:42:56.881 INFO:tasks.workunit.client.1.smithi027.stdout:4/380: rmdir d24 39 2024-04-02T04:42:56.881 INFO:tasks.workunit.client.1.smithi027.stdout:5/655: creat d0/df/d29/da5/fe8 x:0 0 0 2024-04-02T04:42:56.882 INFO:tasks.workunit.client.0.smithi016.stdout:4/986: readlink d5/d17/dcd/def/dfd/l115 0 2024-04-02T04:42:56.882 INFO:tasks.workunit.client.0.smithi016.stdout:5/962: fsync d3/f1a 0 2024-04-02T04:42:56.884 INFO:tasks.workunit.client.1.smithi027.stdout:5/656: write d0/df/d11/f1f [709368,65806] 0 2024-04-02T04:42:56.884 INFO:tasks.workunit.client.1.smithi027.stdout:2/459: getdents d1/db/d3b/d17/d6d 0 2024-04-02T04:42:56.886 INFO:tasks.workunit.client.1.smithi027.stdout:1/454: mkdir d0/de/d52/d74/d97 0 2024-04-02T04:42:56.888 INFO:tasks.workunit.client.0.smithi016.stdout:0/962: dwrite d0/d16/d2e/d37/fae [0,4194304] 0 2024-04-02T04:42:56.889 INFO:tasks.workunit.client.1.smithi027.stdout:0/510: chown d4/d21/d28/d4e/d5f/f5a 594241 1 2024-04-02T04:42:56.892 INFO:tasks.workunit.client.1.smithi027.stdout:4/381: fdatasync d24/f2b 0 2024-04-02T04:42:56.893 INFO:tasks.workunit.client.0.smithi016.stdout:0/963: dread d0/d16/dd6/ddd/fff [0,4194304] 0 2024-04-02T04:42:56.894 INFO:tasks.workunit.client.0.smithi016.stdout:0/964: readlink d0/d16/l42 0 2024-04-02T04:42:56.895 INFO:tasks.workunit.client.0.smithi016.stdout:7/966: rmdir d0/d46/da2/d13b 0 2024-04-02T04:42:56.896 INFO:tasks.workunit.client.0.smithi016.stdout:7/967: stat d0/d46/d16/dc2/dea/f7c 0 2024-04-02T04:42:56.896 INFO:tasks.workunit.client.1.smithi027.stdout:2/460: dwrite d1/db/d3b/d96/fa4 [0,4194304] 0 2024-04-02T04:42:56.897 INFO:tasks.workunit.client.1.smithi027.stdout:2/461: stat d1/db/d3b/d17/l27 0 2024-04-02T04:42:56.898 INFO:tasks.workunit.client.1.smithi027.stdout:2/462: chown d1/db/d3b/d17/d12 53658 1 2024-04-02T04:42:56.898 INFO:tasks.workunit.client.0.smithi016.stdout:4/987: mkdir d5/d17/dcd/def/d130 0 2024-04-02T04:42:56.899 INFO:tasks.workunit.client.0.smithi016.stdout:4/988: fdatasync d5/d8/d43/dd7/fe4 0 2024-04-02T04:42:56.900 INFO:tasks.workunit.client.0.smithi016.stdout:4/989: readlink d5/d1d/d1e/l26 0 2024-04-02T04:42:56.900 INFO:tasks.workunit.client.1.smithi027.stdout:2/463: chown d1/db/d3b/d17/d68/d78/fa1 4191 1 2024-04-02T04:42:56.901 INFO:tasks.workunit.client.0.smithi016.stdout:5/963: mknod d3/df/d18/dfd/d10f/c159 0 2024-04-02T04:42:56.902 INFO:tasks.workunit.client.1.smithi027.stdout:2/464: write d1/db/d3b/d17/d68/d69/f8d [1394122,71642] 0 2024-04-02T04:42:56.904 INFO:tasks.workunit.client.1.smithi027.stdout:9/488: rename d1a/d1d/d29/d50/l65 to d1a/d1d/d29/d37/d9b/la0 0 2024-04-02T04:42:56.905 INFO:tasks.workunit.client.1.smithi027.stdout:8/506: symlink d5/d8/db/d57/la2 0 2024-04-02T04:42:56.906 INFO:tasks.workunit.client.1.smithi027.stdout:9/489: write d1a/d1d/d29/d37/d20/f2b [1291751,104852] 0 2024-04-02T04:42:56.907 INFO:tasks.workunit.client.1.smithi027.stdout:9/490: stat d1a/d1d/d3c/f5e 0 2024-04-02T04:42:56.916 INFO:tasks.workunit.client.1.smithi027.stdout:2/465: mknod d1/db/d3b/d17/d68/cab 0 2024-04-02T04:42:56.917 INFO:tasks.workunit.client.1.smithi027.stdout:7/464: sync 2024-04-02T04:42:56.917 INFO:tasks.workunit.client.1.smithi027.stdout:6/412: sync 2024-04-02T04:42:56.917 INFO:tasks.workunit.client.1.smithi027.stdout:3/393: sync 2024-04-02T04:42:56.920 INFO:tasks.workunit.client.1.smithi027.stdout:4/382: rename d24/f3c to d24/d38/d47/d5c/f7e 0 2024-04-02T04:42:56.923 INFO:tasks.workunit.client.0.smithi016.stdout:4/990: link d5/d17/d18/d6d/da4/db7/dee/cb8 d5/d8/d43/c131 0 2024-04-02T04:42:56.925 INFO:tasks.workunit.client.1.smithi027.stdout:8/507: creat d5/d8/db/fa3 x:0 0 0 2024-04-02T04:42:56.929 INFO:tasks.workunit.client.0.smithi016.stdout:4/991: mknod d5/d17/dcd/def/c132 0 2024-04-02T04:42:56.931 INFO:tasks.workunit.client.0.smithi016.stdout:0/965: write d0/d4f/daf/fb1 [1419497,98163] 0 2024-04-02T04:42:56.931 INFO:tasks.workunit.client.0.smithi016.stdout:5/964: getdents d3/d40/d4f/d67/dbf/d104 0 2024-04-02T04:42:56.934 INFO:tasks.workunit.client.0.smithi016.stdout:4/992: mkdir d5/d17/d18/d6d/d91/dd6/d133 0 2024-04-02T04:42:56.936 INFO:tasks.workunit.client.0.smithi016.stdout:7/968: dwrite d0/d46/da2/dc8/dda/f102 [0,4194304] 0 2024-04-02T04:42:56.936 INFO:tasks.workunit.client.1.smithi027.stdout:8/508: dread d5/d8/d48/f5d [0,4194304] 0 2024-04-02T04:42:56.936 INFO:tasks.workunit.client.1.smithi027.stdout:5/657: write d0/df/d11/d13/f3e [1239412,72432] 0 2024-04-02T04:42:56.938 INFO:tasks.workunit.client.1.smithi027.stdout:8/509: dread - d5/d8/db/dd/d5f/d27/d31/d5b/f8f zero size 2024-04-02T04:42:56.940 INFO:tasks.workunit.client.0.smithi016.stdout:5/965: rename d3/d40/d4f/d9c to d3/df/d93/d15a 0 2024-04-02T04:42:56.942 INFO:tasks.workunit.client.0.smithi016.stdout:1/957: sync 2024-04-02T04:42:56.943 INFO:tasks.workunit.client.0.smithi016.stdout:1/958: truncate d14/d19/d7a/f130 685260 0 2024-04-02T04:42:56.944 INFO:tasks.workunit.client.1.smithi027.stdout:3/394: mknod d1/de/df/d13/d73/c8b 0 2024-04-02T04:42:56.944 INFO:tasks.workunit.client.0.smithi016.stdout:1/959: readlink d14/d19/d1e/d71/d9c/lb3 0 2024-04-02T04:42:56.946 INFO:tasks.workunit.client.0.smithi016.stdout:4/993: mkdir d5/de/dd5/d118/d134 0 2024-04-02T04:42:56.948 INFO:tasks.workunit.client.1.smithi027.stdout:7/465: rename d2/df/d64 to d2/d7/da/dc/d16/d3f/d50/d8b 0 2024-04-02T04:42:56.950 INFO:tasks.workunit.client.1.smithi027.stdout:4/383: mknod d24/d38/c7f 0 2024-04-02T04:42:56.950 INFO:tasks.workunit.client.1.smithi027.stdout:0/511: truncate d4/d21/f2e 1737815 0 2024-04-02T04:42:56.952 INFO:tasks.workunit.client.0.smithi016.stdout:7/969: chown d0/d46/c26 29 1 2024-04-02T04:42:56.954 INFO:tasks.workunit.client.1.smithi027.stdout:1/455: creat d0/d1e/f98 x:0 0 0 2024-04-02T04:42:56.954 INFO:tasks.workunit.client.0.smithi016.stdout:0/966: write d0/d16/d24/d4e/fcc [538036,32486] 0 2024-04-02T04:42:56.954 INFO:tasks.workunit.client.0.smithi016.stdout:7/970: dread d0/d46/d16/dc2/f131 [0,4194304] 0 2024-04-02T04:42:56.955 INFO:tasks.workunit.client.1.smithi027.stdout:4/384: dread d24/d48/f6b [0,4194304] 0 2024-04-02T04:42:56.955 INFO:tasks.workunit.client.0.smithi016.stdout:1/960: symlink d14/d19/d20/de2/l134 0 2024-04-02T04:42:56.955 INFO:tasks.workunit.client.0.smithi016.stdout:0/967: fdatasync d0/d48/fec 0 2024-04-02T04:42:56.956 INFO:tasks.workunit.client.0.smithi016.stdout:0/968: readlink d0/d16/l42 0 2024-04-02T04:42:56.957 INFO:tasks.workunit.client.1.smithi027.stdout:2/466: write d1/f40 [1166066,129923] 0 2024-04-02T04:42:56.957 INFO:tasks.workunit.client.0.smithi016.stdout:4/994: dread d5/d17/dcd/def/dfd/d127/f3b [0,4194304] 0 2024-04-02T04:42:56.958 INFO:tasks.workunit.client.0.smithi016.stdout:0/969: write d0/d16/d23/d3e/f98 [1629313,84159] 0 2024-04-02T04:42:56.964 INFO:tasks.workunit.client.1.smithi027.stdout:2/467: dwrite d1/db/d3b/d17/d12/f7e [0,4194304] 0 2024-04-02T04:42:56.964 INFO:tasks.workunit.client.0.smithi016.stdout:7/971: unlink d0/d46/c90 0 2024-04-02T04:42:56.966 INFO:tasks.workunit.client.1.smithi027.stdout:7/466: truncate d2/d7/f18 780769 0 2024-04-02T04:42:56.966 INFO:tasks.workunit.client.1.smithi027.stdout:2/468: fdatasync d1/db/d3b/d17/f63 0 2024-04-02T04:42:56.972 INFO:tasks.workunit.client.0.smithi016.stdout:1/961: creat d14/d19/d1d/d29/d4c/f135 x:0 0 0 2024-04-02T04:42:56.978 INFO:tasks.workunit.client.1.smithi027.stdout:9/491: creat d1a/d1d/d29/d37/fa1 x:0 0 0 2024-04-02T04:42:56.978 INFO:tasks.workunit.client.1.smithi027.stdout:3/395: write d1/de/df/d13/d73/d77/f7a [1675211,109220] 0 2024-04-02T04:42:56.983 INFO:tasks.workunit.client.0.smithi016.stdout:1/962: creat d14/d19/d1d/d29/d4c/f136 x:0 0 0 2024-04-02T04:42:56.985 INFO:tasks.workunit.client.1.smithi027.stdout:1/456: mkdir d0/d1e/d5d/d60/d99 0 2024-04-02T04:42:56.986 INFO:tasks.workunit.client.0.smithi016.stdout:0/970: mknod d0/d16/d24/c13a 0 2024-04-02T04:42:56.986 INFO:tasks.workunit.client.1.smithi027.stdout:1/457: read - d0/d1e/d2e/f7a zero size 2024-04-02T04:42:56.987 INFO:tasks.workunit.client.1.smithi027.stdout:1/458: truncate d0/de/f38 2124467 0 2024-04-02T04:42:56.987 INFO:tasks.workunit.client.1.smithi027.stdout:9/492: mkdir d1a/d1d/d29/d50/da2 0 2024-04-02T04:42:56.989 INFO:tasks.workunit.client.0.smithi016.stdout:7/972: rename d0/d46/d37/dbe/dae/df8/lcc to d0/d46/d37/dbe/dae/df8/d43/d96/d134/l13f 0 2024-04-02T04:42:56.990 INFO:tasks.workunit.client.0.smithi016.stdout:4/995: write d5/d17/dcd/fce [162790,126700] 0 2024-04-02T04:42:56.991 INFO:tasks.workunit.client.0.smithi016.stdout:1/963: symlink d14/d19/d1e/d5e/d75/l137 0 2024-04-02T04:42:56.992 INFO:tasks.workunit.client.0.smithi016.stdout:4/996: fsync d5/d17/d18/d6d/da4/db7/dee/d104/f12e 0 2024-04-02T04:42:56.994 INFO:tasks.workunit.client.1.smithi027.stdout:7/467: rename d2/d7/f48 to d2/d7/da/d35/f8c 0 2024-04-02T04:42:56.995 INFO:tasks.workunit.client.1.smithi027.stdout:7/468: readlink d2/d7/da/dc/d16/l1a 0 2024-04-02T04:42:56.996 INFO:tasks.workunit.client.1.smithi027.stdout:1/459: unlink d0/d1e/l22 0 2024-04-02T04:42:56.996 INFO:tasks.workunit.client.0.smithi016.stdout:1/964: fsync d14/d19/d1e/d5e/d75/ffb 0 2024-04-02T04:42:56.997 INFO:tasks.workunit.client.1.smithi027.stdout:9/493: creat d1a/d1d/d29/d37/d3a/fa3 x:0 0 0 2024-04-02T04:42:56.999 INFO:tasks.workunit.client.1.smithi027.stdout:9/494: write d1a/d1d/d29/d37/d20/f3f [684274,77531] 0 2024-04-02T04:42:57.000 INFO:tasks.workunit.client.1.smithi027.stdout:3/396: truncate d1/de/df/d61/f3d 2491360 0 2024-04-02T04:42:57.003 INFO:tasks.workunit.client.0.smithi016.stdout:7/973: unlink d0/d46/d16/d83/fa8 0 2024-04-02T04:42:57.006 INFO:tasks.workunit.client.1.smithi027.stdout:9/495: dwrite d1a/d1d/d29/d50/d73/f7a [0,4194304] 0 2024-04-02T04:42:57.011 INFO:tasks.workunit.client.1.smithi027.stdout:7/469: unlink d2/d1d/d57/f5c 0 2024-04-02T04:42:57.013 INFO:tasks.workunit.client.0.smithi016.stdout:1/965: rename d14/d19/d1d/d45/d4f/d86/db4/l100 to d14/d19/d1d/d29/d4c/l138 0 2024-04-02T04:42:57.015 INFO:tasks.workunit.client.0.smithi016.stdout:4/997: symlink d5/d17/dcd/def/d130/l135 0 2024-04-02T04:42:57.016 INFO:tasks.workunit.client.0.smithi016.stdout:1/966: read d14/d19/d1e/d71/dc0/fc2 [286231,113603] 0 2024-04-02T04:42:57.016 INFO:tasks.workunit.client.0.smithi016.stdout:4/998: read - d5/d17/dcd/def/f12f zero size 2024-04-02T04:42:57.019 INFO:tasks.workunit.client.0.smithi016.stdout:0/971: getdents d0/d16/d2e/d53 0 2024-04-02T04:42:57.022 INFO:tasks.workunit.client.1.smithi027.stdout:2/469: link d1/db/d3b/f60 d1/db/d3b/d17/d68/fac 0 2024-04-02T04:42:57.024 INFO:tasks.workunit.client.0.smithi016.stdout:4/999: symlink d5/d1d/d1e/da5/l136 0 2024-04-02T04:42:57.026 INFO:tasks.workunit.client.0.smithi016.stdout:1/967: rmdir d14/d19/d1d/d29/d39/df0 39 2024-04-02T04:42:57.028 INFO:tasks.workunit.client.0.smithi016.stdout:1/968: read d14/d19/d1d/d29/d39/d46/dcb/d43/d68/f7c [207318,94416] 0 2024-04-02T04:42:57.029 INFO:tasks.workunit.client.1.smithi027.stdout:2/470: dwrite d1/db/d57/f99 [0,4194304] 0 2024-04-02T04:42:57.034 INFO:tasks.workunit.client.1.smithi027.stdout:3/397: mknod d1/de/d55/c8c 0 2024-04-02T04:42:57.036 INFO:tasks.workunit.client.1.smithi027.stdout:2/471: dread d1/f4d [0,4194304] 0 2024-04-02T04:42:57.036 INFO:tasks.workunit.client.0.smithi016.stdout:7/974: creat d0/d46/d52/f140 x:0 0 0 2024-04-02T04:42:57.037 INFO:tasks.workunit.client.0.smithi016.stdout:7/975: chown d0/d46/d16/dc2 318 1 2024-04-02T04:42:57.039 INFO:tasks.workunit.client.0.smithi016.stdout:7/976: write d0/d46/d37/dbe/dae/df8/d43/d96/fa5 [474769,51506] 0 2024-04-02T04:42:57.040 INFO:tasks.workunit.client.0.smithi016.stdout:0/972: symlink d0/d16/d24/l13b 0 2024-04-02T04:42:57.040 INFO:tasks.workunit.client.0.smithi016.stdout:7/977: write d0/d21/f36 [1438791,632] 0 2024-04-02T04:42:57.041 INFO:tasks.workunit.client.0.smithi016.stdout:0/973: dread - d0/f11a zero size 2024-04-02T04:42:57.042 INFO:tasks.workunit.client.1.smithi027.stdout:7/470: mkdir d2/df/d89/d8d 0 2024-04-02T04:42:57.043 INFO:tasks.workunit.client.0.smithi016.stdout:0/974: write d0/d16/d24/fc5 [2891456,29123] 0 2024-04-02T04:42:57.044 INFO:tasks.workunit.client.1.smithi027.stdout:1/460: creat d0/d1e/f9a x:0 0 0 2024-04-02T04:42:57.044 INFO:tasks.workunit.client.0.smithi016.stdout:0/975: dread - d0/d4f/daf/f116 zero size 2024-04-02T04:42:57.047 INFO:tasks.workunit.client.0.smithi016.stdout:7/978: truncate d0/d46/d16/dc2/dea/d64/fd6 2621309 0 2024-04-02T04:42:57.050 INFO:tasks.workunit.client.0.smithi016.stdout:0/976: dwrite d0/d16/d2e/d36/d9f/f120 [0,4194304] 0 2024-04-02T04:42:57.050 INFO:tasks.workunit.client.1.smithi027.stdout:1/461: rmdir d0/d1e/d25/d2b/d59 39 2024-04-02T04:42:57.052 INFO:tasks.workunit.client.1.smithi027.stdout:7/471: dwrite d2/d7/da/dc/d16/d3f/d50/d8b/f72 [0,4194304] 0 2024-04-02T04:42:57.052 INFO:tasks.workunit.client.0.smithi016.stdout:0/977: write d0/d16/d2e/d37/f4b [3971684,84526] 0 2024-04-02T04:42:57.054 INFO:tasks.workunit.client.1.smithi027.stdout:7/472: write d2/d7/da/dc/d1f/d33/f71 [1418878,101994] 0 2024-04-02T04:42:57.055 INFO:tasks.workunit.client.0.smithi016.stdout:1/969: dread d14/d19/d1d/f30 [0,4194304] 0 2024-04-02T04:42:57.056 INFO:tasks.workunit.client.0.smithi016.stdout:7/979: mknod d0/d46/d37/dbe/dae/df8/d43/d96/d134/c141 0 2024-04-02T04:42:57.059 INFO:tasks.workunit.client.1.smithi027.stdout:9/496: truncate d1a/d1d/d29/f79 4116558 0 2024-04-02T04:42:57.064 INFO:tasks.workunit.client.1.smithi027.stdout:7/473: mknod d2/d7/da/c8e 0 2024-04-02T04:42:57.065 INFO:tasks.workunit.client.1.smithi027.stdout:2/472: dread d1/db/d3b/d96/f9f [0,4194304] 0 2024-04-02T04:42:57.066 INFO:tasks.workunit.client.0.smithi016.stdout:7/980: rename d0/d46/d16/d49/fca to d0/d46/da2/f142 0 2024-04-02T04:42:57.067 INFO:tasks.workunit.client.1.smithi027.stdout:2/473: write d1/f40 [756897,78545] 0 2024-04-02T04:42:57.068 INFO:tasks.workunit.client.1.smithi027.stdout:1/462: mkdir d0/d1e/d2e/d3b/d4f/d9b 0 2024-04-02T04:42:57.068 INFO:tasks.workunit.client.1.smithi027.stdout:1/463: chown d0/de/d52/d76 54 1 2024-04-02T04:42:57.068 INFO:tasks.workunit.client.0.smithi016.stdout:7/981: write d0/d46/d37/dbe/dae/df8/d43/d96/fa5 [1496431,13202] 0 2024-04-02T04:42:57.069 INFO:tasks.workunit.client.1.smithi027.stdout:1/464: truncate d0/de/f23 1043010 0 2024-04-02T04:42:57.093 INFO:tasks.workunit.client.0.smithi016.stdout:0/978: creat d0/d132/f13c x:0 0 0 2024-04-02T04:42:57.094 INFO:tasks.workunit.client.1.smithi027.stdout:9/497: rename d1a/f57 to d1a/d1d/d29/d56/d8b/fa4 0 2024-04-02T04:42:57.096 INFO:tasks.workunit.client.0.smithi016.stdout:5/966: sync 2024-04-02T04:42:57.102 INFO:tasks.workunit.client.1.smithi027.stdout:3/398: dwrite d1/de/f42 [0,4194304] 0 2024-04-02T04:42:57.103 INFO:tasks.workunit.client.0.smithi016.stdout:0/979: mkdir d0/d16/dd6/ddd/d13d 0 2024-04-02T04:42:57.103 INFO:tasks.workunit.client.0.smithi016.stdout:0/980: chown d0/d16/d24/d4e/c8f 0 1 2024-04-02T04:42:57.105 INFO:tasks.workunit.client.0.smithi016.stdout:1/970: creat d14/d19/d1e/d71/f139 x:0 0 0 2024-04-02T04:42:57.108 INFO:tasks.workunit.client.0.smithi016.stdout:5/967: mkdir d3/dbd/d15b 0 2024-04-02T04:42:57.110 INFO:tasks.workunit.client.1.smithi027.stdout:3/399: read d1/de/d55/f5f [6660941,5356] 0 2024-04-02T04:42:57.113 INFO:tasks.workunit.client.0.smithi016.stdout:0/981: symlink d0/d16/d24/d4e/l13e 0 2024-04-02T04:42:57.116 INFO:tasks.workunit.client.1.smithi027.stdout:3/400: fsync d1/de/f42 0 2024-04-02T04:42:57.119 INFO:tasks.workunit.client.0.smithi016.stdout:5/968: mkdir d3/d40/d4f/d6b/d45/da6/d102/dc4/d108/d6d/d15c 0 2024-04-02T04:42:57.120 INFO:tasks.workunit.client.0.smithi016.stdout:7/982: creat d0/d46/da2/dc8/dda/f143 x:0 0 0 2024-04-02T04:42:57.123 INFO:tasks.workunit.client.0.smithi016.stdout:0/982: creat d0/d16/d24/deb/f13f x:0 0 0 2024-04-02T04:42:57.124 INFO:tasks.workunit.client.1.smithi027.stdout:7/474: write d2/d7/da/dc/d1f/d33/d36/f45 [711015,85843] 0 2024-04-02T04:42:57.124 INFO:tasks.workunit.client.0.smithi016.stdout:0/983: read - d0/d16/d2e/d53/da4/d102/f12e zero size 2024-04-02T04:42:57.124 INFO:tasks.workunit.client.1.smithi027.stdout:1/465: creat d0/d1e/f9c x:0 0 0 2024-04-02T04:42:57.126 INFO:tasks.workunit.client.1.smithi027.stdout:2/474: dwrite d1/db/d3b/d17/d12/f65 [0,4194304] 0 2024-04-02T04:42:57.126 INFO:tasks.workunit.client.1.smithi027.stdout:3/401: creat d1/de/d55/d63/f8d x:0 0 0 2024-04-02T04:42:57.131 INFO:tasks.workunit.client.1.smithi027.stdout:7/475: fdatasync d2/d1c/f4b 0 2024-04-02T04:42:57.133 INFO:tasks.workunit.client.0.smithi016.stdout:1/971: write d14/f83 [1553855,46190] 0 2024-04-02T04:42:57.137 INFO:tasks.workunit.client.1.smithi027.stdout:3/402: creat d1/de/d74/f8e x:0 0 0 2024-04-02T04:42:57.139 INFO:tasks.workunit.client.1.smithi027.stdout:3/403: chown d1/de/d55/c8c 223683577 1 2024-04-02T04:42:57.140 INFO:tasks.workunit.client.1.smithi027.stdout:1/466: dwrite d0/f30 [0,4194304] 0 2024-04-02T04:42:57.140 INFO:tasks.workunit.client.0.smithi016.stdout:0/984: mknod d0/d16/d24/d62/d9c/dfb/d111/c140 0 2024-04-02T04:42:57.143 INFO:tasks.workunit.client.0.smithi016.stdout:1/972: creat d14/d19/d1d/d45/f13a x:0 0 0 2024-04-02T04:42:57.145 INFO:tasks.workunit.client.0.smithi016.stdout:0/985: symlink d0/d16/d24/d62/l141 0 2024-04-02T04:42:57.147 INFO:tasks.workunit.client.0.smithi016.stdout:7/983: truncate d0/d46/d37/d47/f5c 3778631 0 2024-04-02T04:42:57.149 INFO:tasks.workunit.client.0.smithi016.stdout:1/973: rename d14/d19/d1d/d29/d39/d46/ca8 to d14/d11c/c13b 0 2024-04-02T04:42:57.149 INFO:tasks.workunit.client.1.smithi027.stdout:2/475: write d1/db/d3b/d17/d12/f7c [4520636,90924] 0 2024-04-02T04:42:57.150 INFO:tasks.workunit.client.1.smithi027.stdout:3/404: rmdir d1/de/df 39 2024-04-02T04:42:57.151 INFO:tasks.workunit.client.1.smithi027.stdout:2/476: fdatasync d1/db/d3b/d61/d75/f92 0 2024-04-02T04:42:57.152 INFO:tasks.workunit.client.0.smithi016.stdout:0/986: truncate d0/d48/fb0 306609 0 2024-04-02T04:42:57.153 INFO:tasks.workunit.client.1.smithi027.stdout:7/476: fsync d2/d7/da/f2a 0 2024-04-02T04:42:57.155 INFO:tasks.workunit.client.0.smithi016.stdout:1/974: mknod d14/d19/d1d/d29/d4c/c13c 0 2024-04-02T04:42:57.156 INFO:tasks.workunit.client.0.smithi016.stdout:0/987: fdatasync d0/d4f/f9e 0 2024-04-02T04:42:57.157 INFO:tasks.workunit.client.1.smithi027.stdout:3/405: write d1/de/df/f14 [2373935,93908] 0 2024-04-02T04:42:57.157 INFO:tasks.workunit.client.1.smithi027.stdout:1/467: mkdir d0/d1e/d2e/d3e/d5f/d8b/d9d 0 2024-04-02T04:42:57.159 INFO:tasks.workunit.client.0.smithi016.stdout:7/984: rename d0/d46/d37/dbe/dae/df8/dec/c122 to d0/d46/d37/dbe/dae/df8/d43/d96/c144 0 2024-04-02T04:42:57.160 INFO:tasks.workunit.client.1.smithi027.stdout:2/477: dwrite d1/db/f32 [0,4194304] 0 2024-04-02T04:42:57.161 INFO:tasks.workunit.client.1.smithi027.stdout:2/478: readlink d1/db/l6f 0 2024-04-02T04:42:57.164 INFO:tasks.workunit.client.0.smithi016.stdout:1/975: truncate d14/d19/d1d/d45/fbd 345058 0 2024-04-02T04:42:57.166 INFO:tasks.workunit.client.1.smithi027.stdout:7/477: creat d2/d7/da/dc/d16/d3f/d50/d8b/f8f x:0 0 0 2024-04-02T04:42:57.175 INFO:tasks.workunit.client.1.smithi027.stdout:1/468: dread d0/d1e/d2e/d3e/d5f/f79 [0,4194304] 0 2024-04-02T04:42:57.179 INFO:tasks.workunit.client.1.smithi027.stdout:2/479: rename d1/db/d3b/d17/d2b/d2f to d1/db/d3b/d17/d29/dad 0 2024-04-02T04:42:57.184 INFO:tasks.workunit.client.1.smithi027.stdout:1/469: stat d0/l3 0 2024-04-02T04:42:57.184 INFO:tasks.workunit.client.0.smithi016.stdout:0/988: link d0/d16/d24/c6a d0/d16/d24/d4e/d8d/d71/df1/c142 0 2024-04-02T04:42:57.185 INFO:tasks.workunit.client.1.smithi027.stdout:1/470: fsync d0/de/d52/f67 0 2024-04-02T04:42:57.185 INFO:tasks.workunit.client.1.smithi027.stdout:1/471: fdatasync d0/f3d 0 2024-04-02T04:42:57.186 INFO:tasks.workunit.client.1.smithi027.stdout:2/480: creat d1/db/d3b/d61/d6c/fae x:0 0 0 2024-04-02T04:42:57.186 INFO:tasks.workunit.client.1.smithi027.stdout:3/406: write d1/f7 [2352410,33246] 0 2024-04-02T04:42:57.187 INFO:tasks.workunit.client.0.smithi016.stdout:1/976: link d14/d19/d1d/d29/d4c/f88 d14/d89/f13d 0 2024-04-02T04:42:57.187 INFO:tasks.workunit.client.0.smithi016.stdout:1/977: chown d14/da9 651714 1 2024-04-02T04:42:57.188 INFO:tasks.workunit.client.1.smithi027.stdout:2/481: write d1/db/d3b/d17/d2b/f77 [183254,129777] 0 2024-04-02T04:42:57.189 INFO:tasks.workunit.client.0.smithi016.stdout:7/985: write d0/d46/d37/dbe/dae/df8/d43/d96/ffd [554096,94341] 0 2024-04-02T04:42:57.191 INFO:tasks.workunit.client.0.smithi016.stdout:0/989: symlink d0/d16/d2e/d53/da4/d102/l143 0 2024-04-02T04:42:57.194 INFO:tasks.workunit.client.1.smithi027.stdout:3/407: dwrite d1/de/df/d13/f51 [0,4194304] 0 2024-04-02T04:42:57.197 INFO:tasks.workunit.client.0.smithi016.stdout:1/978: rename d14/d19/d1d/d29/d39/l10f to d14/d19/d1e/d6d/l13e 0 2024-04-02T04:42:57.201 INFO:tasks.workunit.client.0.smithi016.stdout:7/986: symlink d0/d133/d11b/l145 0 2024-04-02T04:42:57.203 INFO:tasks.workunit.client.1.smithi027.stdout:7/478: truncate d2/d7/da/dc/d1f/d33/f55 1690675 0 2024-04-02T04:42:57.211 INFO:tasks.workunit.client.1.smithi027.stdout:2/482: rename d1/db/d3b/d17/d68/d76 to d1/daf 0 2024-04-02T04:42:57.214 INFO:tasks.workunit.client.0.smithi016.stdout:7/987: rename d0/d46/d16/fc5 to d0/d46/da2/dc8/dda/de3/d119/f146 0 2024-04-02T04:42:57.216 INFO:tasks.workunit.client.1.smithi027.stdout:1/472: dwrite d0/f6 [0,4194304] 0 2024-04-02T04:42:57.216 INFO:tasks.workunit.client.1.smithi027.stdout:7/479: rename d2/d7/da/dc/d16/d3a/d53 to d2/d1d/d90 0 2024-04-02T04:42:57.216 INFO:tasks.workunit.client.0.smithi016.stdout:0/990: link d0/d16/d24/f20 d0/d16/d24/d4e/d8d/d71/d12c/f144 0 2024-04-02T04:42:57.220 INFO:tasks.workunit.client.1.smithi027.stdout:1/473: stat d0/fb 0 2024-04-02T04:42:57.221 INFO:tasks.workunit.client.1.smithi027.stdout:2/483: fdatasync d1/db/d3b/f60 0 2024-04-02T04:42:57.222 INFO:tasks.workunit.client.0.smithi016.stdout:1/979: link d14/fb6 d14/d19/d1d/d45/d4f/d86/db4/d10e/f13f 0 2024-04-02T04:42:57.222 INFO:tasks.workunit.client.1.smithi027.stdout:2/484: stat d1/db/d3b/d17/l4b 0 2024-04-02T04:42:57.223 INFO:tasks.workunit.client.1.smithi027.stdout:7/480: mkdir d2/d7/da/dc/d16/d3f/d50/d8b/d91 0 2024-04-02T04:42:57.225 INFO:tasks.workunit.client.1.smithi027.stdout:3/408: creat d1/de/df/f8f x:0 0 0 2024-04-02T04:42:57.225 INFO:tasks.workunit.client.1.smithi027.stdout:3/409: dread - d1/de/df/f6b zero size 2024-04-02T04:42:57.226 INFO:tasks.workunit.client.0.smithi016.stdout:7/988: symlink d0/d46/l147 0 2024-04-02T04:42:57.227 INFO:tasks.workunit.client.1.smithi027.stdout:1/474: mkdir d0/de/d9e 0 2024-04-02T04:42:57.228 INFO:tasks.workunit.client.0.smithi016.stdout:7/989: fdatasync d0/d46/da2/dc8/dda/f102 0 2024-04-02T04:42:57.228 INFO:tasks.workunit.client.0.smithi016.stdout:1/980: dwrite d14/f83 [0,4194304] 0 2024-04-02T04:42:57.229 INFO:tasks.workunit.client.1.smithi027.stdout:1/475: write d0/d1e/d5d/d60/f8f [2655028,41954] 0 2024-04-02T04:42:57.229 INFO:tasks.workunit.client.0.smithi016.stdout:0/991: mkdir d0/d16/dd6/ddd/d13d/d145 0 2024-04-02T04:42:57.230 INFO:tasks.workunit.client.1.smithi027.stdout:7/481: mkdir d2/d1d/d92 0 2024-04-02T04:42:57.231 INFO:tasks.workunit.client.0.smithi016.stdout:5/969: sync 2024-04-02T04:42:57.232 INFO:tasks.workunit.client.1.smithi027.stdout:7/482: write d2/d1d/d57/f6d [989105,51165] 0 2024-04-02T04:42:57.233 INFO:tasks.workunit.client.1.smithi027.stdout:2/485: rename d1/db/d3b/d61/f93 to d1/db/d3b/d83/fb0 0 2024-04-02T04:42:57.235 INFO:tasks.workunit.client.1.smithi027.stdout:3/410: truncate d1/de/df/d13/f35 1817423 0 2024-04-02T04:42:57.236 INFO:tasks.workunit.client.0.smithi016.stdout:7/990: dwrite d0/d21/f36 [0,4194304] 0 2024-04-02T04:42:57.237 INFO:tasks.workunit.client.1.smithi027.stdout:3/411: chown d1/de/df/f24 886321 1 2024-04-02T04:42:57.238 INFO:tasks.workunit.client.0.smithi016.stdout:7/991: chown d0/d46/d37/dbe/dae/df8/fb6 1712148 1 2024-04-02T04:42:57.239 INFO:tasks.workunit.client.0.smithi016.stdout:7/992: stat d0/d46/d37/dbe/dae/d9d/leb 0 2024-04-02T04:42:57.240 INFO:tasks.workunit.client.0.smithi016.stdout:1/981: creat d14/d19/d1d/d29/d39/d4d/f140 x:0 0 0 2024-04-02T04:42:57.242 INFO:tasks.workunit.client.1.smithi027.stdout:6/413: sync 2024-04-02T04:42:57.247 INFO:tasks.workunit.client.1.smithi027.stdout:2/486: symlink d1/db/d3b/d96/lb1 0 2024-04-02T04:42:57.249 INFO:tasks.workunit.client.0.smithi016.stdout:1/982: symlink d14/da9/l141 0 2024-04-02T04:42:57.250 INFO:tasks.workunit.client.0.smithi016.stdout:1/983: readlink d14/d19/d1d/d29/d39/d46/dcb/d43/lf8 0 2024-04-02T04:42:57.253 INFO:tasks.workunit.client.0.smithi016.stdout:0/992: dread d0/d16/d11e/f126 [0,4194304] 0 2024-04-02T04:42:57.253 INFO:tasks.workunit.client.0.smithi016.stdout:7/993: rename d0/d46/d37/dbe/dae/df8/d43/d96/ffd to d0/d46/da2/dc8/f148 0 2024-04-02T04:42:57.258 INFO:tasks.workunit.client.0.smithi016.stdout:1/984: rename d14/d19/d1e/d71/dcf/le5 to d14/d19/d1e/d5e/d75/l142 0 2024-04-02T04:42:57.259 INFO:tasks.workunit.client.0.smithi016.stdout:7/994: dwrite d0/d87/fb1 [0,4194304] 0 2024-04-02T04:42:57.261 INFO:tasks.workunit.client.1.smithi027.stdout:2/487: rename d1/db/d3b/d17/d6d/d84/fa3 to d1/db/d57/fb2 0 2024-04-02T04:42:57.263 INFO:tasks.workunit.client.0.smithi016.stdout:1/985: fdatasync d14/d19/d1e/d11e/f12f 0 2024-04-02T04:42:57.265 INFO:tasks.workunit.client.1.smithi027.stdout:1/476: creat d0/d1e/f9f x:0 0 0 2024-04-02T04:42:57.266 INFO:tasks.workunit.client.1.smithi027.stdout:1/477: dread - d0/d1e/d2e/f6d zero size 2024-04-02T04:42:57.269 INFO:tasks.workunit.client.1.smithi027.stdout:6/414: mknod d13/d1b/d24/d25/d56/d73/c89 0 2024-04-02T04:42:57.270 INFO:tasks.workunit.client.1.smithi027.stdout:6/415: fsync d13/d1b/d48/d75/f7d 0 2024-04-02T04:42:57.277 INFO:tasks.workunit.client.0.smithi016.stdout:1/986: rename d14/d19/d1d/d29/f126 to d14/d19/d1e/d71/ddd/f143 0 2024-04-02T04:42:57.281 INFO:tasks.workunit.client.0.smithi016.stdout:5/970: dwrite d3/d40/d4f/f75 [0,4194304] 0 2024-04-02T04:42:57.282 INFO:tasks.workunit.client.0.smithi016.stdout:5/971: write d3/f1a [2015462,73128] 0 2024-04-02T04:42:57.283 INFO:tasks.workunit.client.1.smithi027.stdout:3/412: write d1/de/f38 [7935524,47446] 0 2024-04-02T04:42:57.287 INFO:tasks.workunit.client.0.smithi016.stdout:1/987: creat d14/d19/d1e/d71/dc0/f144 x:0 0 0 2024-04-02T04:42:57.292 INFO:tasks.workunit.client.1.smithi027.stdout:3/413: dwrite d1/de/f54 [0,4194304] 0 2024-04-02T04:42:57.295 INFO:tasks.workunit.client.1.smithi027.stdout:2/488: rmdir d1/db 39 2024-04-02T04:42:57.295 INFO:tasks.workunit.client.1.smithi027.stdout:7/483: getdents d2/d7/da/dc/d16/d3f/d79 0 2024-04-02T04:42:57.297 INFO:tasks.workunit.client.0.smithi016.stdout:5/972: mknod d3/d40/db6/dfc/c15d 0 2024-04-02T04:42:57.299 INFO:tasks.workunit.client.0.smithi016.stdout:5/973: truncate d3/f28 4939136 0 2024-04-02T04:42:57.301 INFO:tasks.workunit.client.0.smithi016.stdout:1/988: unlink d14/d19/d1e/d71/dcf/lee 0 2024-04-02T04:42:57.301 INFO:tasks.workunit.client.0.smithi016.stdout:0/993: dwrite d0/d16/d2e/d37/d66/ff4 [0,4194304] 0 2024-04-02T04:42:57.301 INFO:tasks.workunit.client.1.smithi027.stdout:1/478: unlink d0/l27 0 2024-04-02T04:42:57.306 INFO:tasks.workunit.client.0.smithi016.stdout:7/995: write d0/d46/d16/dc2/dea/f53 [2329154,87733] 0 2024-04-02T04:42:57.306 INFO:tasks.workunit.client.1.smithi027.stdout:1/479: read d0/f69 [5045431,64500] 0 2024-04-02T04:42:57.307 INFO:tasks.workunit.client.0.smithi016.stdout:1/989: mknod d14/d19/d1d/d45/d4f/d86/db4/c145 0 2024-04-02T04:42:57.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:57 smithi016 ceph-mon[28969]: pgmap v27: 65 pgs: 65 active+clean; 2.8 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 78 MiB/s rd, 273 MiB/s wr, 463 op/s 2024-04-02T04:42:57.310 INFO:tasks.workunit.client.0.smithi016.stdout:5/974: truncate d3/df/d18/d26/d70/fdd 958343 0 2024-04-02T04:42:57.312 INFO:tasks.workunit.client.1.smithi027.stdout:7/484: truncate d2/d1d/d57/f5e 879688 0 2024-04-02T04:42:57.313 INFO:tasks.workunit.client.0.smithi016.stdout:7/996: dwrite d0/d46/fc [0,4194304] 0 2024-04-02T04:42:57.314 INFO:tasks.workunit.client.1.smithi027.stdout:1/480: dwrite d0/d1e/d25/d3c/f48 [0,4194304] 0 2024-04-02T04:42:57.315 INFO:tasks.workunit.client.0.smithi016.stdout:7/997: readlink d0/d46/d37/dbe/dae/df8/d4c/l8f 0 2024-04-02T04:42:57.316 INFO:tasks.workunit.client.1.smithi027.stdout:1/481: truncate d0/d1e/d2e/f6b 559819 0 2024-04-02T04:42:57.316 INFO:tasks.workunit.client.1.smithi027.stdout:1/482: chown d0/l3 0 1 2024-04-02T04:42:57.319 INFO:tasks.workunit.client.0.smithi016.stdout:1/990: truncate d14/d19/d1e/ff4 1991955 0 2024-04-02T04:42:57.325 INFO:tasks.workunit.client.1.smithi027.stdout:7/485: dread - d2/d7/da/d35/f7c zero size 2024-04-02T04:42:57.325 INFO:tasks.workunit.client.1.smithi027.stdout:7/486: readlink d2/d1c/l34 0 2024-04-02T04:42:57.326 INFO:tasks.workunit.client.0.smithi016.stdout:7/998: chown d0/d21/l130 2634 1 2024-04-02T04:42:57.330 INFO:tasks.workunit.client.1.smithi027.stdout:3/414: dread d1/de/df/d13/f2d [0,4194304] 0 2024-04-02T04:42:57.330 INFO:tasks.workunit.client.1.smithi027.stdout:7/487: mknod d2/d7/da/dc/d16/d3a/d4d/c93 0 2024-04-02T04:42:57.331 INFO:tasks.workunit.client.1.smithi027.stdout:3/415: chown d1/de/df/d13/l4d 506 1 2024-04-02T04:42:57.333 INFO:tasks.workunit.client.1.smithi027.stdout:1/483: fsync d0/d1e/f2f 0 2024-04-02T04:42:57.334 INFO:tasks.workunit.client.0.smithi016.stdout:1/991: fsync d14/d19/d1d/d29/f51 0 2024-04-02T04:42:57.335 INFO:tasks.workunit.client.1.smithi027.stdout:1/484: write d0/f16 [3222304,78838] 0 2024-04-02T04:42:57.337 INFO:tasks.workunit.client.1.smithi027.stdout:6/416: dwrite d13/d1b/d24/f41 [0,4194304] 0 2024-04-02T04:42:57.342 INFO:tasks.workunit.client.1.smithi027.stdout:1/485: dwrite d0/f6 [0,4194304] 0 2024-04-02T04:42:57.343 INFO:tasks.workunit.client.0.smithi016.stdout:1/992: creat d14/d19/d50/f146 x:0 0 0 2024-04-02T04:42:57.347 INFO:tasks.workunit.client.0.smithi016.stdout:0/994: write d0/f10 [2566385,113771] 0 2024-04-02T04:42:57.350 INFO:tasks.workunit.client.0.smithi016.stdout:7/999: getdents d0/d46/d37/dbe/dae/df8 0 2024-04-02T04:42:57.352 INFO:tasks.workunit.client.0.smithi016.stdout:5/975: write d3/df/d18/f42 [4354618,99177] 0 2024-04-02T04:42:57.357 INFO:tasks.workunit.client.0.smithi016.stdout:5/976: unlink d3/d40/db6/dfc/l120 0 2024-04-02T04:42:57.357 INFO:tasks.workunit.client.0.smithi016.stdout:1/993: dread f2 [0,4194304] 0 2024-04-02T04:42:57.358 INFO:tasks.workunit.client.0.smithi016.stdout:5/977: chown d3/d40/d4f/d6b/d45/da6/d102/dc4/dd7/d117/d138 56040007 1 2024-04-02T04:42:57.359 INFO:tasks.workunit.client.0.smithi016.stdout:1/994: chown d14/d19/d1e/d5e/cff 57 1 2024-04-02T04:42:57.361 INFO:tasks.workunit.client.0.smithi016.stdout:0/995: mkdir d0/d16/d24/d62/d9c/dfb/d111/d146 0 2024-04-02T04:42:57.362 INFO:tasks.workunit.client.1.smithi027.stdout:3/416: rename d1/de/d74/l87 to d1/l90 0 2024-04-02T04:42:57.363 INFO:tasks.workunit.client.1.smithi027.stdout:3/417: chown d1/de/df/d13/l4d 0 1 2024-04-02T04:42:57.364 INFO:tasks.workunit.client.0.smithi016.stdout:0/996: dread d0/d16/d23/d3e/ffa [0,4194304] 0 2024-04-02T04:42:57.365 INFO:tasks.workunit.client.1.smithi027.stdout:1/486: mknod d0/de/d52/d76/ca0 0 2024-04-02T04:42:57.367 INFO:tasks.workunit.client.0.smithi016.stdout:1/995: rename d14/d19/lbc to d14/d19/d1e/d71/dc0/l147 0 2024-04-02T04:42:57.368 INFO:tasks.workunit.client.1.smithi027.stdout:7/488: getdents d2/df 0 2024-04-02T04:42:57.371 INFO:tasks.workunit.client.1.smithi027.stdout:6/417: rmdir d13/d1b/d87 0 2024-04-02T04:42:57.371 INFO:tasks.workunit.client.1.smithi027.stdout:6/418: dread - d13/d1b/d24/d28/d3c/f78 zero size 2024-04-02T04:42:57.372 INFO:tasks.workunit.client.0.smithi016.stdout:5/978: write d3/d40/d4f/d6b/d45/da6/d102/dc4/fce [305575,1369] 0 2024-04-02T04:42:57.373 INFO:tasks.workunit.client.1.smithi027.stdout:3/418: rename d1/de/df/d13/d73/d77/f7e to d1/de/d55/d63/f91 0 2024-04-02T04:42:57.373 INFO:tasks.workunit.client.1.smithi027.stdout:3/419: stat d1/de/d55/l60 0 2024-04-02T04:42:57.374 INFO:tasks.workunit.client.0.smithi016.stdout:0/997: rmdir d0/d16/d23/d3e/db3/d10c 39 2024-04-02T04:42:57.377 INFO:tasks.workunit.client.0.smithi016.stdout:0/998: creat d0/d16/d24/d4e/d8d/d71/df1/f147 x:0 0 0 2024-04-02T04:42:57.378 INFO:tasks.workunit.client.1.smithi027.stdout:7/489: mknod d2/d7/da/dc/d16/d81/c94 0 2024-04-02T04:42:57.380 INFO:tasks.workunit.client.0.smithi016.stdout:5/979: link d3/d40/d4f/d67/dbf/f153 d3/d40/db6/de0/f15e 0 2024-04-02T04:42:57.380 INFO:tasks.workunit.client.0.smithi016.stdout:5/980: write d3/d40/d4f/d67/f124 [272710,72631] 0 2024-04-02T04:42:57.381 INFO:tasks.workunit.client.1.smithi027.stdout:3/420: dwrite d1/de/df/d61/f6c [0,4194304] 0 2024-04-02T04:42:57.382 INFO:tasks.workunit.client.1.smithi027.stdout:1/487: creat d0/d1e/d5d/fa1 x:0 0 0 2024-04-02T04:42:57.386 INFO:tasks.workunit.client.0.smithi016.stdout:0/999: fsync d0/d16/d24/f69 0 2024-04-02T04:42:57.386 INFO:tasks.workunit.client.0.smithi016.stdout:5/981: creat d3/d129/d13c/f15f x:0 0 0 2024-04-02T04:42:57.387 INFO:tasks.workunit.client.1.smithi027.stdout:3/421: readlink d1/de/df/d13/d73/d77/d7f/l82 0 2024-04-02T04:42:57.387 INFO:tasks.workunit.client.1.smithi027.stdout:3/422: stat d1/de/d55/f76 0 2024-04-02T04:42:57.387 INFO:tasks.workunit.client.1.smithi027.stdout:3/423: stat d1/d1a/f1c 0 2024-04-02T04:42:57.387 INFO:tasks.workunit.client.1.smithi027.stdout:3/424: chown d1/de/df/d13/f5c 26010155 1 2024-04-02T04:42:57.388 INFO:tasks.workunit.client.1.smithi027.stdout:6/419: creat d13/d1b/f8a x:0 0 0 2024-04-02T04:42:57.390 INFO:tasks.workunit.client.1.smithi027.stdout:1/488: creat d0/de/fa2 x:0 0 0 2024-04-02T04:42:57.391 INFO:tasks.workunit.client.1.smithi027.stdout:1/489: dread - d0/d1e/f9a zero size 2024-04-02T04:42:57.395 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:57 smithi027 ceph-mon[37550]: pgmap v27: 65 pgs: 65 active+clean; 2.8 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 78 MiB/s rd, 273 MiB/s wr, 463 op/s 2024-04-02T04:42:57.397 INFO:tasks.workunit.client.1.smithi027.stdout:7/490: link d2/d7/da/dc/d1f/c29 d2/d1c/c95 0 2024-04-02T04:42:57.398 INFO:tasks.workunit.client.0.smithi016.stdout:5/982: symlink d3/df/d18/dfd/d10f/d131/l160 0 2024-04-02T04:42:57.399 INFO:tasks.workunit.client.0.smithi016.stdout:5/983: fsync d3/d40/d4f/d6b/d45/f51 0 2024-04-02T04:42:57.402 INFO:tasks.workunit.client.1.smithi027.stdout:3/425: dread d1/de/df/f14 [0,4194304] 0 2024-04-02T04:42:57.404 INFO:tasks.workunit.client.1.smithi027.stdout:3/426: chown d1/d1a/c66 144 1 2024-04-02T04:42:57.404 INFO:tasks.workunit.client.1.smithi027.stdout:3/427: write d1/de/df/f24 [724410,2259] 0 2024-04-02T04:42:57.406 INFO:tasks.workunit.client.0.smithi016.stdout:5/984: mkdir d3/df/dd5/d161 0 2024-04-02T04:42:57.407 INFO:tasks.workunit.client.1.smithi027.stdout:1/490: creat d0/d1e/fa3 x:0 0 0 2024-04-02T04:42:57.408 INFO:tasks.workunit.client.1.smithi027.stdout:1/491: fsync d0/d1e/d2e/f7a 0 2024-04-02T04:42:57.409 INFO:tasks.workunit.client.1.smithi027.stdout:7/491: truncate d2/d1d/d90/f5a 677162 0 2024-04-02T04:42:57.411 INFO:tasks.workunit.client.0.smithi016.stdout:5/985: creat d3/d40/d4f/d6b/d45/da6/d102/dd1/f162 x:0 0 0 2024-04-02T04:42:57.412 INFO:tasks.workunit.client.1.smithi027.stdout:6/420: write d13/f14 [914467,103915] 0 2024-04-02T04:42:57.414 INFO:tasks.workunit.client.1.smithi027.stdout:1/492: dwrite d0/f5c [0,4194304] 0 2024-04-02T04:42:57.417 INFO:tasks.workunit.client.1.smithi027.stdout:6/421: rmdir d13/d1b/d24/d28/d46 39 2024-04-02T04:42:57.418 INFO:tasks.workunit.client.1.smithi027.stdout:3/428: link d1/de/d55/l7b d1/de/df/d13/d73/l92 0 2024-04-02T04:42:57.419 INFO:tasks.workunit.client.1.smithi027.stdout:3/429: fsync d1/de/df/f24 0 2024-04-02T04:42:57.420 INFO:tasks.workunit.client.1.smithi027.stdout:7/492: truncate d2/d1d/f5f 916353 0 2024-04-02T04:42:57.421 INFO:tasks.workunit.client.0.smithi016.stdout:5/986: link d3/df/d18/d26/d48/l5e d3/d40/l163 0 2024-04-02T04:42:57.422 INFO:tasks.workunit.client.1.smithi027.stdout:7/493: dread - d2/d7/da/d35/f60 zero size 2024-04-02T04:42:57.425 INFO:tasks.workunit.client.1.smithi027.stdout:7/494: symlink d2/d7/da/d35/l96 0 2024-04-02T04:42:57.425 INFO:tasks.workunit.client.1.smithi027.stdout:7/495: chown d2/d1d/d57/f6d 29206593 1 2024-04-02T04:42:57.426 INFO:tasks.workunit.client.1.smithi027.stdout:7/496: chown d2/d1c/f40 22 1 2024-04-02T04:42:57.443 INFO:tasks.workunit.client.1.smithi027.stdout:3/430: dwrite d1/de/df/d13/f51 [4194304,4194304] 0 2024-04-02T04:42:57.444 INFO:tasks.workunit.client.1.smithi027.stdout:6/422: dread d13/f62 [0,4194304] 0 2024-04-02T04:42:57.444 INFO:tasks.workunit.client.1.smithi027.stdout:7/497: dwrite d2/f4c [0,4194304] 0 2024-04-02T04:42:57.446 INFO:tasks.workunit.client.1.smithi027.stdout:6/423: write d13/d1b/d24/d25/d42/f60 [2279719,13060] 0 2024-04-02T04:42:57.448 INFO:tasks.workunit.client.0.smithi016.stdout:5/987: dread d3/df/d18/d26/d70/faf [0,4194304] 0 2024-04-02T04:42:57.448 INFO:tasks.workunit.client.1.smithi027.stdout:3/431: rename d1/l26 to d1/de/d74/l93 0 2024-04-02T04:42:57.449 INFO:tasks.workunit.client.0.smithi016.stdout:5/988: stat d3/d40/d4f/d67/dbf/d104/f158 0 2024-04-02T04:42:57.452 INFO:tasks.workunit.client.0.smithi016.stdout:5/989: fsync d3/d5d/f69 0 2024-04-02T04:42:57.453 INFO:tasks.workunit.client.0.smithi016.stdout:5/990: truncate d3/f1a 4676361 0 2024-04-02T04:42:57.454 INFO:tasks.workunit.client.1.smithi027.stdout:7/498: rename d2/d7/da/dc/d16/d2f/c3e to d2/df/c97 0 2024-04-02T04:42:57.454 INFO:tasks.workunit.client.1.smithi027.stdout:5/658: sync 2024-04-02T04:42:57.454 INFO:tasks.workunit.client.1.smithi027.stdout:8/510: sync 2024-04-02T04:42:57.456 INFO:tasks.workunit.client.1.smithi027.stdout:6/424: dwrite d13/d1b/d48/d75/f83 [0,4194304] 0 2024-04-02T04:42:57.456 INFO:tasks.workunit.client.1.smithi027.stdout:5/659: stat d0/d1d/d61/d72 0 2024-04-02T04:42:57.459 INFO:tasks.workunit.client.1.smithi027.stdout:5/660: readlink d0/df/d11/d13/dce/l5f 0 2024-04-02T04:42:57.462 INFO:tasks.workunit.client.1.smithi027.stdout:5/661: dread d0/d1d/f91 [0,4194304] 0 2024-04-02T04:42:57.463 INFO:tasks.workunit.client.1.smithi027.stdout:6/425: rmdir d13/d1b/d24/d28/d3c 39 2024-04-02T04:42:57.463 INFO:tasks.workunit.client.1.smithi027.stdout:5/662: readlink d0/d1d/l5d 0 2024-04-02T04:42:57.465 INFO:tasks.workunit.client.0.smithi016.stdout:5/991: link d3/d40/d4f/d6b/d45/da6/d102/dc4/d108/f92 d3/d40/d4f/d67/dbf/d111/f164 0 2024-04-02T04:42:57.465 INFO:tasks.workunit.client.1.smithi027.stdout:6/426: stat d13/d1b/d24/d25/d42 0 2024-04-02T04:42:57.467 INFO:tasks.workunit.client.1.smithi027.stdout:5/663: dread - d0/df/d2f/d3b/d51/f8e zero size 2024-04-02T04:42:57.468 INFO:tasks.workunit.client.1.smithi027.stdout:6/427: stat d13/d1b/d24/d25/d56/l68 0 2024-04-02T04:42:57.469 INFO:tasks.workunit.client.1.smithi027.stdout:7/499: dwrite d2/d7/da/dc/d1f/f84 [0,4194304] 0 2024-04-02T04:42:57.472 INFO:tasks.workunit.client.1.smithi027.stdout:3/432: getdents d1/de/df/d13/d73 0 2024-04-02T04:42:57.476 INFO:tasks.workunit.client.1.smithi027.stdout:6/428: creat d13/d1b/d48/d75/f8b x:0 0 0 2024-04-02T04:42:57.477 INFO:tasks.workunit.client.1.smithi027.stdout:7/500: mkdir d2/d7/da/dc/d16/d3f/d98 0 2024-04-02T04:42:57.478 INFO:tasks.workunit.client.0.smithi016.stdout:5/992: getdents d3/d40/d4f/d67/dbf/d104 0 2024-04-02T04:42:57.482 INFO:tasks.workunit.client.0.smithi016.stdout:5/993: creat d3/d40/d4f/d67/dbf/d104/f165 x:0 0 0 2024-04-02T04:42:57.483 INFO:tasks.workunit.client.1.smithi027.stdout:8/511: truncate d5/d8/db/d57/fa1 3923900 0 2024-04-02T04:42:57.485 INFO:tasks.workunit.client.1.smithi027.stdout:3/433: creat d1/f94 x:0 0 0 2024-04-02T04:42:57.486 INFO:tasks.workunit.client.1.smithi027.stdout:8/512: truncate d5/d8/db/dd/d5f/d27/d31/f84 353438 0 2024-04-02T04:42:57.490 INFO:tasks.workunit.client.0.smithi016.stdout:1/996: sync 2024-04-02T04:42:57.492 INFO:tasks.workunit.client.0.smithi016.stdout:5/994: dwrite d3/df/d18/d26/d48/f127 [0,4194304] 0 2024-04-02T04:42:57.493 INFO:tasks.workunit.client.1.smithi027.stdout:3/434: symlink d1/d1a/l95 0 2024-04-02T04:42:57.493 INFO:tasks.workunit.client.1.smithi027.stdout:5/664: dwrite d0/df/d76/f85 [0,4194304] 0 2024-04-02T04:42:57.493 INFO:tasks.workunit.client.0.smithi016.stdout:5/995: chown d3/df/d93/fed 35 1 2024-04-02T04:42:57.494 INFO:tasks.workunit.client.0.smithi016.stdout:5/996: write d3/df/d18/d26/d70/fca [2267813,67757] 0 2024-04-02T04:42:57.495 INFO:tasks.workunit.client.1.smithi027.stdout:5/665: fsync d0/d1d/d61/d7e/f9b 0 2024-04-02T04:42:57.500 INFO:tasks.workunit.client.1.smithi027.stdout:6/429: dread d13/d1b/d24/d28/d3c/f77 [0,4194304] 0 2024-04-02T04:42:57.502 INFO:tasks.workunit.client.1.smithi027.stdout:5/666: write d0/df/d29/da5/fe8 [801085,82484] 0 2024-04-02T04:42:57.504 INFO:tasks.workunit.client.1.smithi027.stdout:3/435: dwrite d1/de/d74/f8e [0,4194304] 0 2024-04-02T04:42:57.507 INFO:tasks.workunit.client.1.smithi027.stdout:5/667: write d0/df/d29/da5/fdb [473531,96259] 0 2024-04-02T04:42:57.512 INFO:tasks.workunit.client.1.smithi027.stdout:7/501: truncate d2/d7/da/dc/d16/d3f/d50/d8b/f72 605340 0 2024-04-02T04:42:57.512 INFO:tasks.workunit.client.1.smithi027.stdout:5/668: chown d0/df/d29/da5/fdb 20 1 2024-04-02T04:42:57.512 INFO:tasks.workunit.client.1.smithi027.stdout:3/436: dread d1/de/f42 [0,4194304] 0 2024-04-02T04:42:57.518 INFO:tasks.workunit.client.1.smithi027.stdout:8/513: fsync d5/d8/db/d57/fa1 0 2024-04-02T04:42:57.523 INFO:tasks.workunit.client.1.smithi027.stdout:3/437: mknod d1/de/df/d13/d73/c96 0 2024-04-02T04:42:57.527 INFO:tasks.workunit.client.1.smithi027.stdout:5/669: dread d0/f6 [0,4194304] 0 2024-04-02T04:42:57.531 INFO:tasks.workunit.client.1.smithi027.stdout:7/502: symlink d2/l99 0 2024-04-02T04:42:57.532 INFO:tasks.workunit.client.1.smithi027.stdout:3/438: dwrite d1/de/df/f67 [0,4194304] 0 2024-04-02T04:42:57.534 INFO:tasks.workunit.client.1.smithi027.stdout:6/430: dread d13/d1b/d48/d58/f6a [0,4194304] 0 2024-04-02T04:42:57.534 INFO:tasks.workunit.client.1.smithi027.stdout:3/439: fsync d1/f46 0 2024-04-02T04:42:57.536 INFO:tasks.workunit.client.1.smithi027.stdout:5/670: fdatasync d0/df/d29/f84 0 2024-04-02T04:42:57.541 INFO:tasks.workunit.client.1.smithi027.stdout:7/503: rename d2/f82 to d2/d1d/d92/f9a 0 2024-04-02T04:42:57.550 INFO:tasks.workunit.client.1.smithi027.stdout:6/431: dwrite d13/d1b/d48/d58/f6a [0,4194304] 0 2024-04-02T04:42:57.550 INFO:tasks.workunit.client.1.smithi027.stdout:5/671: mkdir d0/df/d29/da5/da6/de9 0 2024-04-02T04:42:57.552 INFO:tasks.workunit.client.1.smithi027.stdout:6/432: chown d13/d1b/d24/d25/d56/d6d 1541164 1 2024-04-02T04:42:57.561 INFO:tasks.workunit.client.1.smithi027.stdout:3/440: dwrite d1/de/df/d61/f4c [0,4194304] 0 2024-04-02T04:42:57.562 INFO:tasks.workunit.client.1.smithi027.stdout:3/441: dread - d1/f94 zero size 2024-04-02T04:42:57.564 INFO:tasks.workunit.client.1.smithi027.stdout:3/442: write d1/de/df/d13/d73/d77/f84 [837043,93713] 0 2024-04-02T04:42:57.573 INFO:tasks.workunit.client.1.smithi027.stdout:7/504: link d2/d1d/d57/f6d d2/d1d/d92/f9b 0 2024-04-02T04:42:57.575 INFO:tasks.workunit.client.1.smithi027.stdout:5/672: mkdir d0/df/d2f/d3b/dca/dea 0 2024-04-02T04:42:57.577 INFO:tasks.workunit.client.1.smithi027.stdout:8/514: link d5/d8/l49 d5/d8/db/dd/d5f/la4 0 2024-04-02T04:42:57.583 INFO:tasks.workunit.client.1.smithi027.stdout:5/673: creat d0/df/d11/d13/d9a/d9c/dbe/feb x:0 0 0 2024-04-02T04:42:57.583 INFO:tasks.workunit.client.1.smithi027.stdout:5/674: stat d0/df/d11/c45 0 2024-04-02T04:42:57.584 INFO:tasks.workunit.client.1.smithi027.stdout:5/675: chown d0/d68/d6b/d74 83810667 1 2024-04-02T04:42:57.585 INFO:tasks.workunit.client.1.smithi027.stdout:8/515: chown d5/d8/db/dd/f1f 7871 1 2024-04-02T04:42:57.586 INFO:tasks.workunit.client.1.smithi027.stdout:3/443: link d1/d1a/l2b d1/de/df/d13/d73/d77/d7f/l97 0 2024-04-02T04:42:57.587 INFO:tasks.workunit.client.1.smithi027.stdout:3/444: chown d1/de/c2f 198 1 2024-04-02T04:42:57.588 INFO:tasks.workunit.client.1.smithi027.stdout:8/516: chown d5/d8/db/dd/d5f/d27/d31/d5b/d6c/f7b 7173961 1 2024-04-02T04:42:57.593 INFO:tasks.workunit.client.1.smithi027.stdout:3/445: dread d1/f2a [0,4194304] 0 2024-04-02T04:42:57.593 INFO:tasks.workunit.client.1.smithi027.stdout:5/676: creat d0/fec x:0 0 0 2024-04-02T04:42:57.593 INFO:tasks.workunit.client.1.smithi027.stdout:7/505: dread d2/d7/da/f1e [0,4194304] 0 2024-04-02T04:42:57.595 INFO:tasks.workunit.client.1.smithi027.stdout:7/506: chown d2/d7/da/dc/d16/d81/c94 41 1 2024-04-02T04:42:57.596 INFO:tasks.workunit.client.1.smithi027.stdout:8/517: rename d5/d8/db/dd/d5f/d27/d31/d5b/d8d to d5/d8/db/d57/da5 0 2024-04-02T04:42:57.604 INFO:tasks.workunit.client.1.smithi027.stdout:5/677: mknod d0/df/d2f/d3b/d51/ced 0 2024-04-02T04:42:57.604 INFO:tasks.workunit.client.1.smithi027.stdout:3/446: mknod d1/de/df/d13/d73/d77/c98 0 2024-04-02T04:42:57.605 INFO:tasks.workunit.client.0.smithi016.stdout:1/997: creat d14/d19/d1e/d71/de0/f148 x:0 0 0 2024-04-02T04:42:57.606 INFO:tasks.workunit.client.1.smithi027.stdout:8/518: dread d5/d8/db/dd/d5f/f2b [0,4194304] 0 2024-04-02T04:42:57.607 INFO:tasks.workunit.client.1.smithi027.stdout:7/507: rmdir d2/d7/da/dc/d1f 39 2024-04-02T04:42:57.608 INFO:tasks.workunit.client.1.smithi027.stdout:7/508: chown d2/d1c/f7a 112566593 1 2024-04-02T04:42:57.620 INFO:tasks.workunit.client.0.smithi016.stdout:1/998: link d14/d19/d1e/d5e/d75/l142 d14/d19/d1d/d29/d39/d4d/l149 0 2024-04-02T04:42:57.620 INFO:tasks.workunit.client.0.smithi016.stdout:1/999: chown d14/d19/d50/ca0 0 1 2024-04-02T04:42:57.623 INFO:tasks.workunit.client.0.smithi016.stdout:5/997: dread d3/d40/d4f/d6b/f47 [0,4194304] 0 2024-04-02T04:42:57.626 INFO:tasks.workunit.client.1.smithi027.stdout:8/519: creat d5/d8/db/d57/da5/fa6 x:0 0 0 2024-04-02T04:42:57.629 INFO:tasks.workunit.client.0.smithi016.stdout:5/998: dwrite d3/d40/d4f/d67/f142 [0,4194304] 0 2024-04-02T04:42:57.634 INFO:tasks.workunit.client.1.smithi027.stdout:5/678: link d0/df/d11/d8c/l9e d0/d68/lee 0 2024-04-02T04:42:57.634 INFO:tasks.workunit.client.0.smithi016.stdout:5/999: mkdir d3/dbd/d15b/d166 0 2024-04-02T04:42:57.637 INFO:tasks.workunit.client.0.smithi016.stderr:+ rm -rf -- ./tmp.4MHuDQnisZ 2024-04-02T04:42:57.662 INFO:tasks.workunit.client.1.smithi027.stdout:4/385: sync 2024-04-02T04:42:57.662 INFO:tasks.workunit.client.1.smithi027.stdout:2/489: sync 2024-04-02T04:42:57.662 INFO:tasks.workunit.client.1.smithi027.stdout:9/498: sync 2024-04-02T04:42:57.663 INFO:tasks.workunit.client.1.smithi027.stdout:0/512: sync 2024-04-02T04:42:57.664 INFO:tasks.workunit.client.1.smithi027.stdout:5/679: write d0/fb0 [105884,111310] 0 2024-04-02T04:42:57.667 INFO:tasks.workunit.client.1.smithi027.stdout:7/509: link d2/c7e d2/d1d/d90/c9c 0 2024-04-02T04:42:57.669 INFO:tasks.workunit.client.1.smithi027.stdout:8/520: dwrite d5/d8/db/dd/f10 [0,4194304] 0 2024-04-02T04:42:57.674 INFO:tasks.workunit.client.1.smithi027.stdout:0/513: creat d4/d21/d85/fad x:0 0 0 2024-04-02T04:42:57.674 INFO:tasks.workunit.client.1.smithi027.stdout:2/490: read d1/db/d3b/d17/d12/f62 [3674293,118396] 0 2024-04-02T04:42:57.676 INFO:tasks.workunit.client.1.smithi027.stdout:4/386: dwrite d24/f2b [4194304,4194304] 0 2024-04-02T04:42:57.681 INFO:tasks.workunit.client.1.smithi027.stdout:7/510: dwrite d2/d1c/f8a [0,4194304] 0 2024-04-02T04:42:57.682 INFO:tasks.workunit.client.1.smithi027.stdout:4/387: write d24/d38/d44/f6e [50726,34871] 0 2024-04-02T04:42:57.683 INFO:tasks.workunit.client.1.smithi027.stdout:0/514: read d4/f90 [1182239,72054] 0 2024-04-02T04:42:57.684 INFO:tasks.workunit.client.1.smithi027.stdout:4/388: write d24/d48/f7d [56143,19818] 0 2024-04-02T04:42:57.687 INFO:tasks.workunit.client.1.smithi027.stdout:5/680: link d0/f66 d0/df/d11/d13/d9a/db7/fef 0 2024-04-02T04:42:57.692 INFO:tasks.workunit.client.1.smithi027.stdout:2/491: rmdir d1/db/d3b/d61/d75/d81 39 2024-04-02T04:42:57.694 INFO:tasks.workunit.client.1.smithi027.stdout:8/521: write d5/d8/db/dd/d5f/d27/f50 [2942779,54455] 0 2024-04-02T04:42:57.700 INFO:tasks.workunit.client.1.smithi027.stdout:9/499: truncate d1a/d1d/f24 3364776 0 2024-04-02T04:42:57.701 INFO:tasks.workunit.client.1.smithi027.stdout:0/515: creat d4/d21/d85/fae x:0 0 0 2024-04-02T04:42:57.705 INFO:tasks.workunit.client.1.smithi027.stdout:4/389: rmdir d24/d38/d47/d62 39 2024-04-02T04:42:57.708 INFO:tasks.workunit.client.1.smithi027.stdout:5/681: dread d0/fb0 [0,4194304] 0 2024-04-02T04:42:57.710 INFO:tasks.workunit.client.1.smithi027.stdout:0/516: dwrite d4/d3d/f96 [0,4194304] 0 2024-04-02T04:42:57.710 INFO:tasks.workunit.client.1.smithi027.stdout:2/492: chown d1/db/d3b/f70 3100 1 2024-04-02T04:42:57.714 INFO:tasks.workunit.client.1.smithi027.stdout:8/522: unlink d5/d8/db/d68/l77 0 2024-04-02T04:42:57.717 INFO:tasks.workunit.client.1.smithi027.stdout:7/511: truncate d2/d1d/d90/f5a 1040232 0 2024-04-02T04:42:57.717 INFO:tasks.workunit.client.1.smithi027.stdout:4/390: dwrite d24/d48/f7d [0,4194304] 0 2024-04-02T04:42:57.724 INFO:tasks.workunit.client.1.smithi027.stdout:5/682: creat d0/d68/d6b/d74/ff0 x:0 0 0 2024-04-02T04:42:57.727 INFO:tasks.workunit.client.1.smithi027.stdout:8/523: rename d5/d8/db/dd/c2d to d5/d8/db/dd/d5f/d91/d9f/ca7 0 2024-04-02T04:42:57.728 INFO:tasks.workunit.client.1.smithi027.stdout:4/391: creat d24/d38/d44/d41/f80 x:0 0 0 2024-04-02T04:42:57.729 INFO:tasks.workunit.client.1.smithi027.stdout:4/392: chown d24/d2d/f43 4 1 2024-04-02T04:42:57.730 INFO:tasks.workunit.client.1.smithi027.stdout:2/493: dread d1/db/d3b/d17/d12/f41 [0,4194304] 0 2024-04-02T04:42:57.732 INFO:tasks.workunit.client.1.smithi027.stdout:8/524: mknod d5/d8/db/d57/ca8 0 2024-04-02T04:42:57.734 INFO:tasks.workunit.client.1.smithi027.stdout:8/525: dread d5/d8/db/dd/d5f/f2b [0,4194304] 0 2024-04-02T04:42:57.735 INFO:tasks.workunit.client.1.smithi027.stdout:4/393: creat d24/d48/f81 x:0 0 0 2024-04-02T04:42:57.747 INFO:tasks.workunit.client.1.smithi027.stdout:7/512: link d2/d7/da/dc/d16/f4a d2/d7/da/dc/d16/d3a/d4d/f9d 0 2024-04-02T04:42:57.747 INFO:tasks.workunit.client.1.smithi027.stdout:7/513: chown d2/d1d/d92 86 1 2024-04-02T04:42:57.748 INFO:tasks.workunit.client.1.smithi027.stdout:8/526: mknod d5/d8/db/dd/d5f/d27/d31/d5b/ca9 0 2024-04-02T04:42:57.751 INFO:tasks.workunit.client.1.smithi027.stdout:2/494: rmdir d1/db/d3b/d61/d75/d81 39 2024-04-02T04:42:57.755 INFO:tasks.workunit.client.1.smithi027.stdout:5/683: truncate d0/df/d11/d13/d9a/db7/fc6 204947 0 2024-04-02T04:42:57.757 INFO:tasks.workunit.client.1.smithi027.stdout:8/527: symlink d5/d8/db/dd/d5f/d27/d31/d5b/d6c/laa 0 2024-04-02T04:42:57.758 INFO:tasks.workunit.client.1.smithi027.stdout:2/495: creat d1/db/d3b/d17/d68/d69/fb3 x:0 0 0 2024-04-02T04:42:57.760 INFO:tasks.workunit.client.1.smithi027.stdout:7/514: creat d2/d7/da/dc/d1f/f9e x:0 0 0 2024-04-02T04:42:57.760 INFO:tasks.workunit.client.1.smithi027.stdout:7/515: chown d2/d7/da/d35 1 1 2024-04-02T04:42:57.761 INFO:tasks.workunit.client.1.smithi027.stdout:7/516: chown d2/d1d/d57/f6d 158094183 1 2024-04-02T04:42:57.763 INFO:tasks.workunit.client.1.smithi027.stdout:2/496: read - d1/db/d3b/f37 zero size 2024-04-02T04:42:57.765 INFO:tasks.workunit.client.1.smithi027.stdout:4/394: getdents d24 0 2024-04-02T04:42:57.765 INFO:tasks.workunit.client.1.smithi027.stdout:2/497: truncate d1/db/fa0 161134 0 2024-04-02T04:42:57.828 INFO:tasks.workunit.client.1.smithi027.stdout:1/493: sync 2024-04-02T04:42:57.828 INFO:tasks.workunit.client.1.smithi027.stdout:6/433: sync 2024-04-02T04:42:57.838 INFO:tasks.workunit.client.1.smithi027.stdout:2/498: fsync d1/f13 0 2024-04-02T04:42:57.840 INFO:tasks.workunit.client.1.smithi027.stdout:4/395: mkdir d24/d38/d44/d82 0 2024-04-02T04:42:57.842 INFO:tasks.workunit.client.1.smithi027.stdout:1/494: write d0/d1e/d2e/d3b/f78 [202196,8116] 0 2024-04-02T04:42:57.843 INFO:tasks.workunit.client.1.smithi027.stdout:4/396: chown f12 0 1 2024-04-02T04:42:57.850 INFO:tasks.workunit.client.1.smithi027.stdout:5/684: dwrite d0/df/d11/d13/dce/d55/fe4 [0,4194304] 0 2024-04-02T04:42:57.851 INFO:tasks.workunit.client.1.smithi027.stdout:7/517: truncate d2/d7/da/dc/d1f/f47 398282 0 2024-04-02T04:42:57.852 INFO:tasks.workunit.client.1.smithi027.stdout:6/434: dwrite d13/d1b/f2b [4194304,4194304] 0 2024-04-02T04:42:57.853 INFO:tasks.workunit.client.1.smithi027.stdout:8/528: dread d5/d8/db/dd/f1d [0,4194304] 0 2024-04-02T04:42:57.856 INFO:tasks.workunit.client.1.smithi027.stdout:1/495: creat d0/d1e/d25/d2b/d59/fa4 x:0 0 0 2024-04-02T04:42:57.860 INFO:tasks.workunit.client.1.smithi027.stdout:4/397: dwrite d24/f63 [0,4194304] 0 2024-04-02T04:42:57.868 INFO:tasks.workunit.client.1.smithi027.stdout:2/499: dwrite d1/f56 [0,4194304] 0 2024-04-02T04:42:57.870 INFO:tasks.workunit.client.1.smithi027.stdout:1/496: stat d0/f54 0 2024-04-02T04:42:57.870 INFO:tasks.workunit.client.1.smithi027.stdout:8/529: mkdir d5/d8/db/dd/d5f/d27/d31/d5b/dab 0 2024-04-02T04:42:57.874 INFO:tasks.workunit.client.1.smithi027.stdout:2/500: mkdir d1/daf/db4 0 2024-04-02T04:42:57.877 INFO:tasks.workunit.client.1.smithi027.stdout:1/497: creat d0/d1e/d5d/d60/fa5 x:0 0 0 2024-04-02T04:42:57.878 INFO:tasks.workunit.client.1.smithi027.stdout:7/518: link d2/d7/da/d35/l96 d2/d1d/d90/l9f 0 2024-04-02T04:42:57.883 INFO:tasks.workunit.client.1.smithi027.stdout:8/530: link d5/d8/db/dd/d5f/d27/d31/f9c d5/d8/db/dd/d5f/d91/fac 0 2024-04-02T04:42:57.885 INFO:tasks.workunit.client.1.smithi027.stdout:2/501: dwrite d1/db/d3b/d17/d12/f65 [0,4194304] 0 2024-04-02T04:42:57.886 INFO:tasks.workunit.client.1.smithi027.stdout:1/498: mkdir d0/d1e/d25/d3c/d64/da6 0 2024-04-02T04:42:57.887 INFO:tasks.workunit.client.1.smithi027.stdout:8/531: fsync d5/d8/db/dd/d5f/d27/f50 0 2024-04-02T04:42:57.887 INFO:tasks.workunit.client.1.smithi027.stdout:4/398: dread d24/d38/d44/d41/d49/f4c [0,4194304] 0 2024-04-02T04:42:57.889 INFO:tasks.workunit.client.1.smithi027.stdout:7/519: symlink d2/d1d/d57/la0 0 2024-04-02T04:42:57.897 INFO:tasks.workunit.client.1.smithi027.stdout:8/532: creat d5/d8/db/d68/fad x:0 0 0 2024-04-02T04:42:57.897 INFO:tasks.workunit.client.1.smithi027.stdout:6/435: dwrite f7 [0,4194304] 0 2024-04-02T04:42:57.898 INFO:tasks.workunit.client.1.smithi027.stdout:5/685: dread d0/df/d11/d13/d9a/d9c/dbe/fe7 [0,4194304] 0 2024-04-02T04:42:57.899 INFO:tasks.workunit.client.1.smithi027.stdout:4/399: creat d24/d38/d44/d41/d49/f83 x:0 0 0 2024-04-02T04:42:57.903 INFO:tasks.workunit.client.1.smithi027.stdout:7/520: dwrite d2/d7/da/dc/d16/d3a/d4d/f58 [0,4194304] 0 2024-04-02T04:42:57.916 INFO:tasks.workunit.client.1.smithi027.stdout:4/400: dwrite d24/f71 [0,4194304] 0 2024-04-02T04:42:57.917 INFO:tasks.workunit.client.1.smithi027.stdout:5/686: dwrite d0/df/d11/d65/f99 [0,4194304] 0 2024-04-02T04:42:57.918 INFO:tasks.workunit.client.1.smithi027.stdout:4/401: chown f14 1000 0 2024-04-02T04:42:57.919 INFO:tasks.workunit.client.1.smithi027.stdout:2/502: creat d1/db/d3b/d17/fb5 x:0 0 0 2024-04-02T04:42:57.924 INFO:tasks.workunit.client.1.smithi027.stdout:7/521: dwrite d2/d7/da/dc/d1f/f84 [0,4194304] 0 2024-04-02T04:42:57.925 INFO:tasks.workunit.client.1.smithi027.stdout:6/436: mkdir d13/d1b/d8c 0 2024-04-02T04:42:57.927 INFO:tasks.workunit.client.1.smithi027.stdout:5/687: creat d0/df/d29/ff1 x:0 0 0 2024-04-02T04:42:57.928 INFO:tasks.workunit.client.1.smithi027.stdout:6/437: write d13/d1b/f2b [7713849,7081] 0 2024-04-02T04:42:57.932 INFO:tasks.workunit.client.1.smithi027.stdout:8/533: getdents d5/d8/db/dd/d5f/d27/d31/d5b/d6c 0 2024-04-02T04:42:57.935 INFO:tasks.workunit.client.1.smithi027.stdout:4/402: dread d24/d38/f5e [0,4194304] 0 2024-04-02T04:42:57.937 INFO:tasks.workunit.client.1.smithi027.stdout:6/438: mkdir d13/d8d 0 2024-04-02T04:42:57.937 INFO:tasks.workunit.client.1.smithi027.stdout:4/403: readlink d24/d38/d44/l67 0 2024-04-02T04:42:57.937 INFO:tasks.workunit.client.1.smithi027.stdout:5/688: dwrite d0/d1d/d61/d7e/faf [0,4194304] 0 2024-04-02T04:42:57.939 INFO:tasks.workunit.client.1.smithi027.stdout:2/503: link d1/db/d3b/d17/d68/l6b d1/db/d3b/d17/d68/lb6 0 2024-04-02T04:42:57.949 INFO:tasks.workunit.client.1.smithi027.stdout:4/404: dwrite d24/d38/d44/d41/d49/f83 [0,4194304] 0 2024-04-02T04:42:57.953 INFO:tasks.workunit.client.1.smithi027.stdout:5/689: mkdir d0/df/d11/d13/dce/d55/dc3/df2 0 2024-04-02T04:42:57.961 INFO:tasks.workunit.client.1.smithi027.stdout:6/439: creat d13/d1b/f8e x:0 0 0 2024-04-02T04:42:57.965 INFO:tasks.workunit.client.1.smithi027.stdout:2/504: truncate d1/db/d3b/d17/f63 4855894 0 2024-04-02T04:42:57.967 INFO:tasks.workunit.client.1.smithi027.stdout:4/405: rmdir d24/d38/d44 39 2024-04-02T04:42:57.968 INFO:tasks.workunit.client.1.smithi027.stdout:4/406: dread - d24/d48/f72 zero size 2024-04-02T04:42:57.970 INFO:tasks.workunit.client.1.smithi027.stdout:5/690: read d0/df/d11/d13/f28 [86600,60338] 0 2024-04-02T04:42:57.972 INFO:tasks.workunit.client.1.smithi027.stdout:7/522: rename d2/d7/da/dc/d1f/f47 to d2/d7/da/dc/d1f/fa1 0 2024-04-02T04:42:57.976 INFO:tasks.workunit.client.1.smithi027.stdout:8/534: truncate d5/d8/db/dd/d5f/d27/f3a 8168854 0 2024-04-02T04:42:57.979 INFO:tasks.workunit.client.1.smithi027.stdout:2/505: dread d1/db/d3b/d17/d29/f5a [0,4194304] 0 2024-04-02T04:42:57.983 INFO:tasks.workunit.client.1.smithi027.stdout:6/440: creat d13/d1b/d24/d28/d46/f8f x:0 0 0 2024-04-02T04:42:57.992 INFO:tasks.workunit.client.1.smithi027.stdout:3/447: sync 2024-04-02T04:42:58.007 INFO:tasks.workunit.client.1.smithi027.stdout:8/535: mknod d5/d8/db/dd/d5f/d91/d9f/cae 0 2024-04-02T04:42:58.007 INFO:tasks.workunit.client.1.smithi027.stdout:4/407: write d24/d48/f7d [4776307,41562] 0 2024-04-02T04:42:58.007 INFO:tasks.workunit.client.1.smithi027.stdout:3/448: fsync d1/de/f8a 0 2024-04-02T04:42:58.008 INFO:tasks.workunit.client.1.smithi027.stdout:7/523: fsync d2/d7/da/dc/d16/f4a 0 2024-04-02T04:42:58.010 INFO:tasks.workunit.client.1.smithi027.stdout:3/449: fdatasync d1/de/df/d13/f51 0 2024-04-02T04:42:58.010 INFO:tasks.workunit.client.1.smithi027.stdout:6/441: mknod d13/d1b/d24/d25/d56/d6d/c90 0 2024-04-02T04:42:58.012 INFO:tasks.workunit.client.1.smithi027.stdout:6/442: chown d13/d1b/d48/l63 10501634 1 2024-04-02T04:42:58.016 INFO:tasks.workunit.client.1.smithi027.stdout:7/524: dread d2/d7/da/dc/d1f/f84 [0,4194304] 0 2024-04-02T04:42:58.019 INFO:tasks.workunit.client.1.smithi027.stdout:3/450: dwrite d1/d1a/f79 [0,4194304] 0 2024-04-02T04:42:58.022 INFO:tasks.workunit.client.1.smithi027.stdout:6/443: creat d13/d1b/d24/d25/d27/f91 x:0 0 0 2024-04-02T04:42:58.023 INFO:tasks.workunit.client.1.smithi027.stdout:2/506: link d1/l35 d1/db/d3b/d61/lb7 0 2024-04-02T04:42:58.028 INFO:tasks.workunit.client.1.smithi027.stdout:7/525: mknod d2/d7/da/dc/d16/d3f/ca2 0 2024-04-02T04:42:58.031 INFO:tasks.workunit.client.1.smithi027.stdout:2/507: readlink d1/l35 0 2024-04-02T04:42:58.032 INFO:tasks.workunit.client.1.smithi027.stdout:4/408: dread f11 [0,4194304] 0 2024-04-02T04:42:58.032 INFO:tasks.workunit.client.1.smithi027.stdout:2/508: chown d1/db/d3b/d17/d2b 2 1 2024-04-02T04:42:58.033 INFO:tasks.workunit.client.1.smithi027.stdout:6/444: creat d13/d1b/d24/d28/d3c/f92 x:0 0 0 2024-04-02T04:42:58.035 INFO:tasks.workunit.client.1.smithi027.stdout:7/526: chown d2/d7/da/dc/c11 112343884 1 2024-04-02T04:42:58.036 INFO:tasks.workunit.client.1.smithi027.stdout:7/527: chown d2/d7/da/dc/d1f/f84 38 1 2024-04-02T04:42:58.037 INFO:tasks.workunit.client.1.smithi027.stdout:8/536: rename d5/d8/db/dd/d5f/d91/fac to d5/d8/db/dd/d5f/faf 0 2024-04-02T04:42:58.040 INFO:tasks.workunit.client.1.smithi027.stdout:3/451: mkdir d1/de/d55/d99 0 2024-04-02T04:42:58.044 INFO:tasks.workunit.client.1.smithi027.stdout:6/445: getdents d13/d1b/d8c 0 2024-04-02T04:42:58.045 INFO:tasks.workunit.client.1.smithi027.stdout:8/537: dread d5/d8/db/dd/d5f/f14 [4194304,4194304] 0 2024-04-02T04:42:58.046 INFO:tasks.workunit.client.1.smithi027.stdout:8/538: truncate d5/d8/db/fa3 299156 0 2024-04-02T04:42:58.047 INFO:tasks.workunit.client.1.smithi027.stdout:7/528: mknod d2/d7/da/dc/d16/d3f/d44/ca3 0 2024-04-02T04:42:58.049 INFO:tasks.workunit.client.1.smithi027.stdout:3/452: creat d1/de/d55/d99/f9a x:0 0 0 2024-04-02T04:42:58.052 INFO:tasks.workunit.client.1.smithi027.stdout:4/409: rename d24/d2d/f6d to d24/d38/f84 0 2024-04-02T04:42:58.054 INFO:tasks.workunit.client.1.smithi027.stdout:8/539: mkdir d5/d8/d15/d71/db0 0 2024-04-02T04:42:58.055 INFO:tasks.workunit.client.1.smithi027.stdout:7/529: creat d2/d7/da/dc/d16/d2f/fa4 x:0 0 0 2024-04-02T04:42:58.059 INFO:tasks.workunit.client.1.smithi027.stdout:2/509: rename d1/db/d3b/d96 to d1/db/d3b/d17/d2b/db8 0 2024-04-02T04:42:58.060 INFO:tasks.workunit.client.1.smithi027.stdout:2/510: chown d1/db/d3b/d17/c95 368308998 1 2024-04-02T04:42:58.060 INFO:tasks.workunit.client.1.smithi027.stdout:4/410: mknod d24/d2d/c85 0 2024-04-02T04:42:58.061 INFO:tasks.workunit.client.1.smithi027.stdout:4/411: write d24/d48/f7d [3285374,2968] 0 2024-04-02T04:42:58.062 INFO:tasks.workunit.client.1.smithi027.stdout:8/540: creat d5/d8/db/dd/d5f/d27/d31/d5b/fb1 x:0 0 0 2024-04-02T04:42:58.064 INFO:tasks.workunit.client.1.smithi027.stdout:6/446: creat d13/f93 x:0 0 0 2024-04-02T04:42:58.065 INFO:tasks.workunit.client.1.smithi027.stdout:7/530: mkdir d2/d7/da/dc/d16/d3a/da5 0 2024-04-02T04:42:58.068 INFO:tasks.workunit.client.1.smithi027.stdout:4/412: unlink d24/d48/f53 0 2024-04-02T04:42:58.070 INFO:tasks.workunit.client.1.smithi027.stdout:6/447: dread d13/f14 [0,4194304] 0 2024-04-02T04:42:58.076 INFO:tasks.workunit.client.1.smithi027.stdout:8/541: mkdir d5/d8/db/dd/d5f/db2 0 2024-04-02T04:42:58.078 INFO:tasks.workunit.client.1.smithi027.stdout:7/531: getdents d2/d7/da/dc/d16/d3f/d79 0 2024-04-02T04:42:58.079 INFO:tasks.workunit.client.1.smithi027.stdout:7/532: chown d2/d7/da/dc/d16/d81 59313 1 2024-04-02T04:42:58.081 INFO:tasks.workunit.client.1.smithi027.stdout:6/448: rmdir d13/d1b/d48/d75 39 2024-04-02T04:42:58.093 INFO:tasks.workunit.client.1.smithi027.stdout:2/511: truncate d1/db/d3b/d17/d12/f38 6156979 0 2024-04-02T04:42:58.093 INFO:tasks.workunit.client.1.smithi027.stdout:6/449: creat d13/d8d/f94 x:0 0 0 2024-04-02T04:42:58.094 INFO:tasks.workunit.client.1.smithi027.stdout:6/450: chown d13/d1b/d24/c36 1003 1 2024-04-02T04:42:58.096 INFO:tasks.workunit.client.1.smithi027.stdout:8/542: write d5/f7d [1719890,111077] 0 2024-04-02T04:42:58.101 INFO:tasks.workunit.client.1.smithi027.stdout:2/512: unlink d1/db/d3b/d17/f5d 0 2024-04-02T04:42:58.104 INFO:tasks.workunit.client.1.smithi027.stdout:4/413: getdents d24/d38/d47/d62 0 2024-04-02T04:42:58.107 INFO:tasks.workunit.client.1.smithi027.stdout:7/533: dwrite d2/d1d/f74 [0,4194304] 0 2024-04-02T04:42:58.107 INFO:tasks.workunit.client.1.smithi027.stdout:6/451: dwrite d13/d1b/d24/d28/d3c/f78 [0,4194304] 0 2024-04-02T04:42:58.112 INFO:tasks.workunit.client.1.smithi027.stdout:7/534: fdatasync d2/d7/da/d35/f60 0 2024-04-02T04:42:58.113 INFO:tasks.workunit.client.1.smithi027.stdout:8/543: dwrite d5/d8/db/dd/d5f/d27/d31/f5e [0,4194304] 0 2024-04-02T04:42:58.117 INFO:tasks.workunit.client.1.smithi027.stdout:4/414: unlink d24/d2d/l45 0 2024-04-02T04:42:58.124 INFO:tasks.workunit.client.1.smithi027.stdout:2/513: unlink d1/db/d3b/d17/l2e 0 2024-04-02T04:42:58.126 INFO:tasks.workunit.client.1.smithi027.stdout:8/544: creat d5/d8/db/dd/d5f/d91/fb3 x:0 0 0 2024-04-02T04:42:58.129 INFO:tasks.workunit.client.1.smithi027.stdout:6/452: mknod d13/d1b/d24/c95 0 2024-04-02T04:42:58.130 INFO:tasks.workunit.client.1.smithi027.stdout:2/514: chown d1/db/d3b/l3c 0 1 2024-04-02T04:42:58.130 INFO:tasks.workunit.client.1.smithi027.stdout:6/453: chown d13/d1b/d24/d28/d3c/l5e 0 1 2024-04-02T04:42:58.131 INFO:tasks.workunit.client.1.smithi027.stdout:7/535: mkdir d2/d7/da/da6 0 2024-04-02T04:42:58.133 INFO:tasks.workunit.client.1.smithi027.stdout:8/545: creat d5/d8/db/dd/d5f/d91/d9f/fb4 x:0 0 0 2024-04-02T04:42:58.134 INFO:tasks.workunit.client.1.smithi027.stdout:6/454: unlink d13/d1b/f1f 0 2024-04-02T04:42:58.135 INFO:tasks.workunit.client.1.smithi027.stdout:2/515: mknod d1/db/d3b/d61/cb9 0 2024-04-02T04:42:58.136 INFO:tasks.workunit.client.1.smithi027.stdout:6/455: mkdir d13/d1b/d24/d25/d56/d73/d96 0 2024-04-02T04:42:58.141 INFO:tasks.workunit.client.1.smithi027.stdout:2/516: dwrite d1/db/d3b/d17/d2b/f77 [0,4194304] 0 2024-04-02T04:42:58.206 INFO:tasks.workunit.client.1.smithi027.stdout:1/499: sync 2024-04-02T04:42:58.206 INFO:tasks.workunit.client.1.smithi027.stdout:9/500: sync 2024-04-02T04:42:58.206 INFO:tasks.workunit.client.1.smithi027.stdout:0/517: sync 2024-04-02T04:42:58.206 INFO:tasks.workunit.client.1.smithi027.stdout:9/501: fsync d1a/d1d/d29/d37/d20/f3f 0 2024-04-02T04:42:58.207 INFO:tasks.workunit.client.1.smithi027.stdout:8/546: mkdir d5/db5 0 2024-04-02T04:42:58.208 INFO:tasks.workunit.client.1.smithi027.stdout:6/456: creat d13/d1b/d24/d25/d27/f97 x:0 0 0 2024-04-02T04:42:58.209 INFO:tasks.workunit.client.1.smithi027.stdout:4/415: write f13 [812352,30472] 0 2024-04-02T04:42:58.276 INFO:tasks.workunit.client.1.smithi027.stdout:0/518: rename d4/d21/d28/d4e/f87 to d4/d21/d85/faf 0 2024-04-02T04:42:58.280 INFO:tasks.workunit.client.1.smithi027.stdout:8/547: symlink d5/d8/d5c/lb6 0 2024-04-02T04:42:58.281 INFO:tasks.workunit.client.1.smithi027.stdout:6/457: mknod d13/d1b/d24/d25/d42/c98 0 2024-04-02T04:42:58.287 INFO:tasks.workunit.client.1.smithi027.stdout:0/519: mkdir d4/d21/d28/d4e/d8c/db0 0 2024-04-02T04:42:58.290 INFO:tasks.workunit.client.1.smithi027.stdout:1/500: dwrite d0/fb [0,4194304] 0 2024-04-02T04:42:58.291 INFO:tasks.workunit.client.1.smithi027.stdout:8/548: symlink d5/d8/db/d68/lb7 0 2024-04-02T04:42:58.293 INFO:tasks.workunit.client.1.smithi027.stdout:1/501: dread - d0/de/fa2 zero size 2024-04-02T04:42:58.295 INFO:tasks.workunit.client.1.smithi027.stdout:8/549: fdatasync d5/d8/db/d68/d95/f94 0 2024-04-02T04:42:58.297 INFO:tasks.workunit.client.1.smithi027.stdout:4/416: rename d24/d48/l50 to d24/d38/d44/d60/l86 0 2024-04-02T04:42:58.301 INFO:tasks.workunit.client.1.smithi027.stdout:0/520: dwrite d4/d38/d66/d2d/d4c/f9c [0,4194304] 0 2024-04-02T04:42:58.302 INFO:tasks.workunit.client.1.smithi027.stdout:9/502: creat d1a/d1d/fa5 x:0 0 0 2024-04-02T04:42:58.302 INFO:tasks.workunit.client.1.smithi027.stdout:5/691: sync 2024-04-02T04:42:58.303 INFO:tasks.workunit.client.1.smithi027.stdout:0/521: chown d4/d38/d66/d4a 18846 1 2024-04-02T04:42:58.303 INFO:tasks.workunit.client.1.smithi027.stdout:1/502: dread d0/fb [0,4194304] 0 2024-04-02T04:42:58.304 INFO:tasks.workunit.client.1.smithi027.stdout:5/692: dread - d0/fec zero size 2024-04-02T04:42:58.311 INFO:tasks.workunit.client.1.smithi027.stdout:4/417: creat d24/d38/d44/d41/f87 x:0 0 0 2024-04-02T04:42:58.314 INFO:tasks.workunit.client.1.smithi027.stdout:8/550: dread d5/d8/db/f29 [0,4194304] 0 2024-04-02T04:42:58.318 INFO:tasks.workunit.client.1.smithi027.stdout:0/522: read d4/d38/d66/f6c [2711895,6747] 0 2024-04-02T04:42:58.354 INFO:tasks.workunit.client.1.smithi027.stdout:3/453: sync 2024-04-02T04:42:58.354 INFO:tasks.workunit.client.1.smithi027.stdout:7/536: sync 2024-04-02T04:42:58.355 INFO:tasks.workunit.client.1.smithi027.stdout:3/454: truncate d1/f94 959844 0 2024-04-02T04:42:58.410 INFO:tasks.workunit.client.1.smithi027.stdout:2/517: sync 2024-04-02T04:42:58.414 INFO:tasks.workunit.client.1.smithi027.stdout:2/518: write d1/db/fa0 [524370,51035] 0 2024-04-02T04:42:58.418 INFO:tasks.workunit.client.1.smithi027.stdout:4/418: creat d24/d38/d44/d41/d5a/f88 x:0 0 0 2024-04-02T04:42:58.422 INFO:tasks.workunit.client.1.smithi027.stdout:6/458: link d13/d1b/d24/d25/d42/l69 d13/d1b/d48/d75/l99 0 2024-04-02T04:42:58.423 INFO:tasks.workunit.client.1.smithi027.stdout:8/551: truncate d5/d8/db/dd/f1d 2839916 0 2024-04-02T04:42:58.429 INFO:tasks.workunit.client.1.smithi027.stdout:8/552: dwrite d5/d8/db/d57/da5/f93 [0,4194304] 0 2024-04-02T04:42:58.430 INFO:tasks.workunit.client.1.smithi027.stdout:3/455: creat d1/de/d55/d63/f9b x:0 0 0 2024-04-02T04:42:58.440 INFO:tasks.workunit.client.1.smithi027.stdout:3/456: dwrite d1/d1a/f86 [4194304,4194304] 0 2024-04-02T04:42:58.441 INFO:tasks.workunit.client.1.smithi027.stdout:8/553: dwrite d5/d8/db/d57/f74 [0,4194304] 0 2024-04-02T04:42:58.442 INFO:tasks.workunit.client.1.smithi027.stdout:6/459: creat d13/d1b/d48/d75/f9a x:0 0 0 2024-04-02T04:42:58.445 INFO:tasks.workunit.client.1.smithi027.stdout:3/457: write d1/d1a/f86 [861184,37140] 0 2024-04-02T04:42:58.451 INFO:tasks.workunit.client.1.smithi027.stdout:0/523: mknod d4/d21/d93/cb1 0 2024-04-02T04:42:58.451 INFO:tasks.workunit.client.1.smithi027.stdout:0/524: chown d4/f9b 305 1 2024-04-02T04:42:58.453 INFO:tasks.workunit.client.1.smithi027.stdout:8/554: dwrite d5/d8/db/dd/f10 [4194304,4194304] 0 2024-04-02T04:42:58.455 INFO:tasks.workunit.client.1.smithi027.stdout:0/525: write d4/d38/d66/d2d/d4c/f4d [4663952,7985] 0 2024-04-02T04:42:58.456 INFO:tasks.workunit.client.1.smithi027.stdout:0/526: dread - d4/d21/d85/fad zero size 2024-04-02T04:42:58.463 INFO:tasks.workunit.client.1.smithi027.stdout:1/503: write d0/d1e/f3a [2335482,45698] 0 2024-04-02T04:42:58.464 INFO:tasks.workunit.client.1.smithi027.stdout:1/504: chown d0/d1e/d5d/d60/f8f 181 1 2024-04-02T04:42:58.467 INFO:tasks.workunit.client.1.smithi027.stdout:6/460: mkdir d13/d1b/d48/d75/d9b 0 2024-04-02T04:42:58.469 INFO:tasks.workunit.client.1.smithi027.stdout:3/458: rmdir d1/de/df/d13 39 2024-04-02T04:42:58.471 INFO:tasks.workunit.client.1.smithi027.stdout:3/459: dread - d1/de/d55/d99/f9a zero size 2024-04-02T04:42:58.482 INFO:tasks.workunit.client.1.smithi027.stdout:1/505: unlink d0/d1e/d25/f93 0 2024-04-02T04:42:58.487 INFO:tasks.workunit.client.1.smithi027.stdout:3/460: mkdir d1/d1a/d9c 0 2024-04-02T04:42:58.503 INFO:tasks.workunit.client.1.smithi027.stdout:7/537: getdents d2/d7/da/dc/d1f/d33/d36 0 2024-04-02T04:42:58.514 INFO:tasks.workunit.client.1.smithi027.stdout:7/538: symlink d2/d7/da/dc/d1f/la7 0 2024-04-02T04:42:58.514 INFO:tasks.workunit.client.1.smithi027.stdout:7/539: readlink d2/l99 0 2024-04-02T04:42:58.521 INFO:tasks.workunit.client.1.smithi027.stdout:1/506: getdents d0/d1e/d25/d2b 0 2024-04-02T04:42:58.522 INFO:tasks.workunit.client.1.smithi027.stdout:2/519: creat d1/db/d3b/d17/d2b/fba x:0 0 0 2024-04-02T04:42:58.525 INFO:tasks.workunit.client.1.smithi027.stdout:1/507: readlink d0/de/d52/d74/l90 0 2024-04-02T04:42:58.533 INFO:tasks.workunit.client.1.smithi027.stdout:1/508: dwrite d0/de/f92 [0,4194304] 0 2024-04-02T04:42:58.553 INFO:tasks.workunit.client.1.smithi027.stdout:5/693: rename d0/df/d11/d13/dce/l5f to d0/d68/lf3 0 2024-04-02T04:42:58.563 INFO:tasks.workunit.client.1.smithi027.stdout:1/509: creat d0/d1e/d2e/d3b/fa7 x:0 0 0 2024-04-02T04:42:58.564 INFO:tasks.workunit.client.1.smithi027.stdout:1/510: read - d0/d1e/d25/d2b/d59/fa4 zero size 2024-04-02T04:42:58.630 INFO:tasks.workunit.client.1.smithi027.stdout:9/503: truncate d1a/d1d/d29/d50/d73/f7a 1480572 0 2024-04-02T04:42:58.643 INFO:tasks.workunit.client.1.smithi027.stdout:0/527: dwrite d4/d3d/f3f [0,4194304] 0 2024-04-02T04:42:58.667 INFO:tasks.workunit.client.1.smithi027.stdout:7/540: dwrite d2/d7/da/dc/d16/d2f/f73 [0,4194304] 0 2024-04-02T04:42:58.670 INFO:tasks.workunit.client.1.smithi027.stdout:7/541: mknod d2/d1c/ca8 0 2024-04-02T04:42:58.672 INFO:tasks.workunit.client.1.smithi027.stdout:7/542: getdents d2/d7/da/dc/d16/d3f/d79 0 2024-04-02T04:42:58.673 INFO:tasks.workunit.client.1.smithi027.stdout:7/543: write d2/d7/da/dc/d1f/f9e [585092,22098] 0 2024-04-02T04:42:58.675 INFO:tasks.workunit.client.1.smithi027.stdout:7/544: write d2/df/f85 [22134,45835] 0 2024-04-02T04:42:58.678 INFO:tasks.workunit.client.1.smithi027.stdout:7/545: mknod d2/d7/da/dc/d16/d3f/d79/ca9 0 2024-04-02T04:42:58.683 INFO:tasks.workunit.client.1.smithi027.stdout:8/555: unlink d5/d8/db/dd/c36 0 2024-04-02T04:42:58.685 INFO:tasks.workunit.client.1.smithi027.stdout:7/546: dwrite d2/d7/da/dc/f7b [0,4194304] 0 2024-04-02T04:42:58.686 INFO:tasks.workunit.client.1.smithi027.stdout:8/556: dread d5/d8/d48/f8b [0,4194304] 0 2024-04-02T04:42:58.692 INFO:tasks.workunit.client.1.smithi027.stdout:7/547: link d2/d1d/d90/f5a d2/df/d89/faa 0 2024-04-02T04:42:58.695 INFO:tasks.workunit.client.1.smithi027.stdout:7/548: creat d2/d7/da/dc/fab x:0 0 0 2024-04-02T04:42:58.696 INFO:tasks.workunit.client.1.smithi027.stdout:7/549: rmdir d2/d7 39 2024-04-02T04:42:58.698 INFO:tasks.workunit.client.1.smithi027.stdout:2/520: rename d1/daf to d1/db/d3b/d61/d6c/dbb 0 2024-04-02T04:42:58.702 INFO:tasks.workunit.client.1.smithi027.stdout:5/694: rename d0/df/d11/d41/c54 to d0/d1a/d6f/cf4 0 2024-04-02T04:42:58.704 INFO:tasks.workunit.client.1.smithi027.stdout:2/521: mkdir d1/db/d3b/d61/d6c/dbc 0 2024-04-02T04:42:58.705 INFO:tasks.workunit.client.1.smithi027.stdout:2/522: chown d1/db/d57 0 1 2024-04-02T04:42:58.706 INFO:tasks.workunit.client.1.smithi027.stdout:7/550: dwrite d2/d7/da/dc/d1f/d33/d36/f45 [0,4194304] 0 2024-04-02T04:42:58.711 INFO:tasks.workunit.client.1.smithi027.stdout:5/695: dwrite d0/d1d/d61/d7e/f9b [0,4194304] 0 2024-04-02T04:42:58.713 INFO:tasks.workunit.client.1.smithi027.stdout:5/696: readlink d0/df/d29/da5/da6/la9 0 2024-04-02T04:42:58.720 INFO:tasks.workunit.client.1.smithi027.stdout:5/697: unlink d0/df/d11/d65/c77 0 2024-04-02T04:42:58.721 INFO:tasks.workunit.client.1.smithi027.stdout:5/698: write d0/d1d/d61/d7e/faf [804799,73397] 0 2024-04-02T04:42:58.722 INFO:tasks.workunit.client.1.smithi027.stdout:2/523: getdents d1/db/d3b/d61/d6c/dbb/db4 0 2024-04-02T04:42:58.724 INFO:tasks.workunit.client.1.smithi027.stdout:7/551: creat d2/d7/da/dc/d16/fac x:0 0 0 2024-04-02T04:42:58.729 INFO:tasks.workunit.client.1.smithi027.stdout:5/699: dread d0/d1d/f91 [0,4194304] 0 2024-04-02T04:42:58.731 INFO:tasks.workunit.client.1.smithi027.stdout:2/524: fdatasync d1/db/d3b/d17/d29/dad/f71 0 2024-04-02T04:42:58.735 INFO:tasks.workunit.client.1.smithi027.stdout:5/700: creat d0/df/d2f/d3b/d51/ff5 x:0 0 0 2024-04-02T04:42:58.738 INFO:tasks.workunit.client.1.smithi027.stdout:2/525: creat d1/db/d3b/d17/d68/d78/fbd x:0 0 0 2024-04-02T04:42:58.739 INFO:tasks.workunit.client.1.smithi027.stdout:5/701: fsync d0/df/d11/d13/d9a/d9c/dbe/fe7 0 2024-04-02T04:42:58.739 INFO:tasks.workunit.client.1.smithi027.stdout:2/526: dread - d1/db/d3b/d17/f9e zero size 2024-04-02T04:42:58.742 INFO:tasks.workunit.client.1.smithi027.stdout:7/552: creat d2/d7/da/fad x:0 0 0 2024-04-02T04:42:58.744 INFO:tasks.workunit.client.1.smithi027.stdout:5/702: symlink d0/df/d11/d13/d9a/d9c/lf6 0 2024-04-02T04:42:58.746 INFO:tasks.workunit.client.1.smithi027.stdout:7/553: truncate d2/d7/da/dc/d16/f2c 904861 0 2024-04-02T04:42:58.752 INFO:tasks.workunit.client.1.smithi027.stdout:7/554: rmdir d2/d7/da/dc/d16/d3f/d98 0 2024-04-02T04:42:58.754 INFO:tasks.workunit.client.1.smithi027.stdout:2/527: dread d1/db/d57/f99 [0,4194304] 0 2024-04-02T04:42:58.755 INFO:tasks.workunit.client.1.smithi027.stdout:0/528: rename d4/d21/d28/fa5 to d4/d21/fb2 0 2024-04-02T04:42:58.757 INFO:tasks.workunit.client.1.smithi027.stdout:7/555: rmdir d2/d7/da/d35 39 2024-04-02T04:42:58.758 INFO:tasks.workunit.client.1.smithi027.stdout:7/556: readlink d2/d7/da/dc/d1f/la7 0 2024-04-02T04:42:58.759 INFO:tasks.workunit.client.1.smithi027.stdout:7/557: chown d2/d1d/d57/f6d 1 1 2024-04-02T04:42:58.761 INFO:tasks.workunit.client.1.smithi027.stdout:3/461: sync 2024-04-02T04:42:58.762 INFO:tasks.workunit.client.1.smithi027.stdout:8/557: rename d5/d8/db/d68/d95/c90 to d5/d8/d15/d71/d96/cb8 0 2024-04-02T04:42:58.765 INFO:tasks.workunit.client.1.smithi027.stdout:2/528: dread d1/db/d3b/d17/d68/f80 [0,4194304] 0 2024-04-02T04:42:58.765 INFO:tasks.workunit.client.1.smithi027.stdout:8/558: truncate d5/d8/db/fa3 395090 0 2024-04-02T04:42:58.766 INFO:tasks.workunit.client.1.smithi027.stdout:2/529: write d1/f13 [12976782,26702] 0 2024-04-02T04:42:58.769 INFO:tasks.workunit.client.1.smithi027.stdout:4/419: creat d24/d38/d44/d41/f89 x:0 0 0 2024-04-02T04:42:58.771 INFO:tasks.workunit.client.1.smithi027.stdout:4/420: write d24/d38/d44/d41/d49/f77 [416290,47840] 0 2024-04-02T04:42:58.771 INFO:tasks.workunit.client.1.smithi027.stdout:0/529: rename d4/d21/d28/d4e/dab to d4/d21/d85/db3 0 2024-04-02T04:42:58.772 INFO:tasks.workunit.client.1.smithi027.stdout:6/461: mkdir d13/d1b/d48/d9c 0 2024-04-02T04:42:58.773 INFO:tasks.workunit.client.1.smithi027.stdout:8/559: fsync d5/d8/db/dd/f51 0 2024-04-02T04:42:58.776 INFO:tasks.workunit.client.1.smithi027.stdout:9/504: dread d1a/d1d/d29/d50/d73/f7a [0,4194304] 0 2024-04-02T04:42:58.777 INFO:tasks.workunit.client.1.smithi027.stdout:2/530: mknod d1/db/d3b/d17/d29/dad/cbe 0 2024-04-02T04:42:58.779 INFO:tasks.workunit.client.1.smithi027.stdout:3/462: creat d1/de/df/d13/d73/f9d x:0 0 0 2024-04-02T04:42:58.779 INFO:tasks.workunit.client.1.smithi027.stdout:9/505: dread d1a/d1d/d29/d37/d3a/f84 [0,4194304] 0 2024-04-02T04:42:58.781 INFO:tasks.workunit.client.1.smithi027.stdout:4/421: creat d24/d2d/f8a x:0 0 0 2024-04-02T04:42:58.782 INFO:tasks.workunit.client.1.smithi027.stdout:8/560: dwrite d5/d8/db/dd/d5f/d27/f50 [0,4194304] 0 2024-04-02T04:42:58.784 INFO:tasks.workunit.client.1.smithi027.stdout:4/422: write d24/d38/d44/d60/f7c [60346,69161] 0 2024-04-02T04:42:58.786 INFO:tasks.workunit.client.1.smithi027.stdout:8/561: write d5/d8/db/dd/d5f/d27/d31/d5b/fb1 [836003,117631] 0 2024-04-02T04:42:58.787 INFO:tasks.workunit.client.1.smithi027.stdout:4/423: chown d24/d38/d44/d41/d49/f77 20196012 1 2024-04-02T04:42:58.792 INFO:tasks.workunit.client.1.smithi027.stdout:6/462: mknod d13/d1b/d24/d28/c9d 0 2024-04-02T04:42:58.794 INFO:tasks.workunit.client.1.smithi027.stdout:2/531: dwrite d1/db/d3b/f70 [0,4194304] 0 2024-04-02T04:42:58.795 INFO:tasks.workunit.client.1.smithi027.stdout:8/562: mkdir d5/d8/db/dd/d5f/d27/d31/db9 0 2024-04-02T04:42:58.798 INFO:tasks.workunit.client.1.smithi027.stdout:4/424: creat d24/d38/d44/d41/d49/f8b x:0 0 0 2024-04-02T04:42:58.799 INFO:tasks.workunit.client.1.smithi027.stdout:9/506: getdents d1a/d1d/d29/d37/d99 0 2024-04-02T04:42:58.799 INFO:tasks.workunit.client.1.smithi027.stdout:3/463: dread d1/de/f58 [0,4194304] 0 2024-04-02T04:42:58.800 INFO:tasks.workunit.client.1.smithi027.stdout:4/425: write d24/d38/d44/d60/f7c [1056464,95429] 0 2024-04-02T04:42:58.800 INFO:tasks.workunit.client.1.smithi027.stdout:6/463: rmdir d13/d1b/d24/d25/d56/d73 39 2024-04-02T04:42:58.802 INFO:tasks.workunit.client.1.smithi027.stdout:9/507: write d1a/d1d/d29/d51/f60 [347985,127852] 0 2024-04-02T04:42:58.805 INFO:tasks.workunit.client.1.smithi027.stdout:9/508: write d1a/d1d/d29/d37/d20/f3f [584541,24202] 0 2024-04-02T04:42:58.809 INFO:tasks.workunit.client.1.smithi027.stdout:4/426: dwrite d24/d48/f75 [0,4194304] 0 2024-04-02T04:42:58.811 INFO:tasks.workunit.client.1.smithi027.stdout:2/532: mknod d1/db/d3b/d61/d6c/dbb/db4/cbf 0 2024-04-02T04:42:58.812 INFO:tasks.workunit.client.1.smithi027.stdout:2/533: truncate d1/f88 519589 0 2024-04-02T04:42:58.818 INFO:tasks.workunit.client.1.smithi027.stdout:3/464: mkdir d1/de/df/d9e 0 2024-04-02T04:42:58.820 INFO:tasks.workunit.client.1.smithi027.stdout:9/509: creat d1a/d1d/d29/d51/fa6 x:0 0 0 2024-04-02T04:42:58.826 INFO:tasks.workunit.client.1.smithi027.stdout:4/427: creat d24/d38/d47/d5c/f8c x:0 0 0 2024-04-02T04:42:58.826 INFO:tasks.workunit.client.1.smithi027.stdout:6/464: unlink d13/d1b/d48/d75/c7a 0 2024-04-02T04:42:58.827 INFO:tasks.workunit.client.1.smithi027.stdout:3/465: creat d1/de/d55/d63/f9f x:0 0 0 2024-04-02T04:42:58.828 INFO:tasks.workunit.client.1.smithi027.stdout:9/510: dwrite d1a/d1d/d29/d51/fa6 [0,4194304] 0 2024-04-02T04:42:58.829 INFO:tasks.workunit.client.1.smithi027.stdout:2/534: mkdir d1/db/d3b/d61/d6c/dbc/dc0 0 2024-04-02T04:42:58.836 INFO:tasks.workunit.client.1.smithi027.stdout:3/466: chown d1/de/d74/l93 8258 1 2024-04-02T04:42:58.836 INFO:tasks.workunit.client.1.smithi027.stdout:6/465: dwrite d13/d1b/d48/d58/f6a [0,4194304] 0 2024-04-02T04:42:58.838 INFO:tasks.workunit.client.1.smithi027.stdout:2/535: creat d1/db/d3b/d17/d29/fc1 x:0 0 0 2024-04-02T04:42:58.839 INFO:tasks.workunit.client.1.smithi027.stdout:6/466: write d13/d1b/d48/f66 [870462,34713] 0 2024-04-02T04:42:58.841 INFO:tasks.workunit.client.1.smithi027.stdout:2/536: fsync d1/db/d3b/d17/d68/d78/fa1 0 2024-04-02T04:42:58.849 INFO:tasks.workunit.client.1.smithi027.stdout:5/703: dwrite d0/df/d11/d65/fdd [0,4194304] 0 2024-04-02T04:42:58.850 INFO:tasks.workunit.client.1.smithi027.stdout:5/704: chown d0/df/d29/da5/da6 21141 1 2024-04-02T04:42:58.854 INFO:tasks.workunit.client.1.smithi027.stdout:4/428: creat d24/d38/f8d x:0 0 0 2024-04-02T04:42:58.856 INFO:tasks.workunit.client.1.smithi027.stdout:2/537: truncate d1/db/f30 3372509 0 2024-04-02T04:42:58.859 INFO:tasks.workunit.client.1.smithi027.stdout:5/705: dwrite d0/df/d11/d13/d9a/d9c/fa8 [0,4194304] 0 2024-04-02T04:42:58.860 INFO:tasks.workunit.client.1.smithi027.stdout:9/511: rename d1a/d1d/d29/d37/d4c/l5b to d1a/d1d/d3c/la7 0 2024-04-02T04:42:58.861 INFO:tasks.workunit.client.1.smithi027.stdout:9/512: chown d1a/d1d/d29/d56/c67 0 1 2024-04-02T04:42:58.867 INFO:tasks.workunit.client.1.smithi027.stdout:7/558: write d2/d1d/f5f [261836,13717] 0 2024-04-02T04:42:58.871 INFO:tasks.workunit.client.1.smithi027.stdout:9/513: dwrite d1a/d1d/d29/d37/d20/f3f [0,4194304] 0 2024-04-02T04:42:58.872 INFO:tasks.workunit.client.1.smithi027.stdout:2/538: creat d1/d91/fc2 x:0 0 0 2024-04-02T04:42:58.879 INFO:tasks.workunit.client.1.smithi027.stdout:3/467: rmdir d1/d1a/d9c 0 2024-04-02T04:42:58.880 INFO:tasks.workunit.client.1.smithi027.stdout:3/468: fsync d1/f8 0 2024-04-02T04:42:58.883 INFO:tasks.workunit.client.1.smithi027.stdout:3/469: dread d1/de/df/f14 [4194304,4194304] 0 2024-04-02T04:42:58.884 INFO:tasks.workunit.client.1.smithi027.stdout:3/470: write d1/d1a/f5a [4551690,76737] 0 2024-04-02T04:42:58.885 INFO:tasks.workunit.client.1.smithi027.stdout:7/559: fsync d2/d7/da/dc/d1f/f84 0 2024-04-02T04:42:58.887 INFO:tasks.workunit.client.1.smithi027.stdout:3/471: fdatasync d1/de/d55/d99/f9a 0 2024-04-02T04:42:58.888 INFO:tasks.workunit.client.1.smithi027.stdout:3/472: chown d1/de/d55/f75 3 1 2024-04-02T04:42:58.890 INFO:tasks.workunit.client.1.smithi027.stdout:9/514: creat d1a/d1d/d3c/d71/fa8 x:0 0 0 2024-04-02T04:42:58.891 INFO:tasks.workunit.client.1.smithi027.stdout:2/539: unlink d1/db/d3b/d17/d2b/l5b 0 2024-04-02T04:42:58.891 INFO:tasks.workunit.client.1.smithi027.stdout:5/706: dread d0/d68/fa4 [0,4194304] 0 2024-04-02T04:42:58.896 INFO:tasks.workunit.client.1.smithi027.stdout:4/429: link d24/d38/d44/d41/d49/l56 d24/d48/l8e 0 2024-04-02T04:42:58.899 INFO:tasks.workunit.client.1.smithi027.stdout:2/540: dwrite d1/db/d3b/d17/d68/d69/f8d [0,4194304] 0 2024-04-02T04:42:58.902 INFO:tasks.workunit.client.1.smithi027.stdout:7/560: unlink d2/d1d/f5f 0 2024-04-02T04:42:58.904 INFO:tasks.workunit.client.1.smithi027.stdout:6/467: rename d13/d1b/d24/d25/d42/c4e to d13/c9e 0 2024-04-02T04:42:58.905 INFO:tasks.workunit.client.1.smithi027.stdout:6/468: chown d13/f14 693 1 2024-04-02T04:42:58.906 INFO:tasks.workunit.client.1.smithi027.stdout:9/515: readlink d1a/l43 0 2024-04-02T04:42:58.906 INFO:tasks.workunit.client.1.smithi027.stdout:6/469: write d13/d1b/d24/d25/d27/f81 [499281,130771] 0 2024-04-02T04:42:58.907 INFO:tasks.workunit.client.1.smithi027.stdout:6/470: readlink d13/d1b/d24/d25/d27/l57 0 2024-04-02T04:42:58.909 INFO:tasks.workunit.client.1.smithi027.stdout:6/471: write d13/d1b/f8a [811580,54202] 0 2024-04-02T04:42:58.910 INFO:tasks.workunit.client.1.smithi027.stdout:4/430: mknod d24/d38/d44/d60/c8f 0 2024-04-02T04:42:58.920 INFO:tasks.workunit.client.1.smithi027.stdout:5/707: truncate d0/df/d11/d13/d9a/d9c/dbe/fe7 4532968 0 2024-04-02T04:42:58.921 INFO:tasks.workunit.client.1.smithi027.stdout:1/511: sync 2024-04-02T04:42:58.922 INFO:tasks.workunit.client.1.smithi027.stdout:4/431: unlink f16 0 2024-04-02T04:42:58.924 INFO:tasks.workunit.client.1.smithi027.stdout:2/541: mkdir d1/db/d3b/d61/d6c/dbc/dc0/dc3 0 2024-04-02T04:42:58.925 INFO:tasks.workunit.client.1.smithi027.stdout:2/542: dread - d1/db/d3b/d61/d75/f92 zero size 2024-04-02T04:42:58.926 INFO:tasks.workunit.client.1.smithi027.stdout:7/561: fdatasync d2/d1d/d90/f5a 0 2024-04-02T04:42:58.927 INFO:tasks.workunit.client.1.smithi027.stdout:2/543: write d1/db/d3b/d17/f2c [3508329,40304] 0 2024-04-02T04:42:58.928 INFO:tasks.workunit.client.1.smithi027.stdout:9/516: symlink d1a/d1d/d29/la9 0 2024-04-02T04:42:58.930 INFO:tasks.workunit.client.1.smithi027.stdout:6/472: dread d13/d1b/d24/d28/d3c/f77 [0,4194304] 0 2024-04-02T04:42:58.931 INFO:tasks.workunit.client.1.smithi027.stdout:2/544: read d1/db/d3b/d17/d12/f7c [1421607,79215] 0 2024-04-02T04:42:58.931 INFO:tasks.workunit.client.1.smithi027.stdout:6/473: stat d13/d1b/d24/d28 0 2024-04-02T04:42:58.933 INFO:tasks.workunit.client.1.smithi027.stdout:6/474: write d13/f93 [138105,24933] 0 2024-04-02T04:42:58.934 INFO:tasks.workunit.client.1.smithi027.stdout:1/512: creat d0/de/fa8 x:0 0 0 2024-04-02T04:42:58.936 INFO:tasks.workunit.client.1.smithi027.stdout:4/432: rmdir d24/d38/d47/d5c 39 2024-04-02T04:42:58.937 INFO:tasks.workunit.client.1.smithi027.stdout:6/475: read d13/d1b/f1e [688677,5409] 0 2024-04-02T04:42:58.938 INFO:tasks.workunit.client.1.smithi027.stdout:7/562: dwrite d2/d7/da/dc/d1f/d33/f71 [0,4194304] 0 2024-04-02T04:42:58.939 INFO:tasks.workunit.client.1.smithi027.stdout:7/563: fdatasync d2/d1c/f3b 0 2024-04-02T04:42:58.946 INFO:tasks.workunit.client.1.smithi027.stdout:4/433: dwrite d24/d2d/f8a [0,4194304] 0 2024-04-02T04:42:58.948 INFO:tasks.workunit.client.1.smithi027.stdout:4/434: read - d24/d38/d44/d41/d49/f8b zero size 2024-04-02T04:42:58.950 INFO:tasks.workunit.client.1.smithi027.stdout:3/473: truncate d1/de/df/d61/f6c 3008862 0 2024-04-02T04:42:58.953 INFO:tasks.workunit.client.1.smithi027.stdout:9/517: rename d1a/d1d/d29/d56/d8b/f9f to d1a/d1d/d29/d37/d20/faa 0 2024-04-02T04:42:58.955 INFO:tasks.workunit.client.1.smithi027.stdout:9/518: truncate d1a/d1d/d29/d37/fa1 33447 0 2024-04-02T04:42:58.960 INFO:tasks.workunit.client.1.smithi027.stdout:1/513: symlink d0/d1e/d25/d2b/d59/la9 0 2024-04-02T04:42:58.961 INFO:tasks.workunit.client.1.smithi027.stdout:6/476: creat d13/d1b/d48/d75/f9f x:0 0 0 2024-04-02T04:42:58.963 INFO:tasks.workunit.client.1.smithi027.stdout:7/564: mknod d2/d1c/cae 0 2024-04-02T04:42:58.969 INFO:tasks.workunit.client.1.smithi027.stdout:2/545: fsync d1/db/d3b/f1c 0 2024-04-02T04:42:58.975 INFO:tasks.workunit.client.1.smithi027.stdout:1/514: rmdir d0 39 2024-04-02T04:42:58.979 INFO:tasks.workunit.client.1.smithi027.stdout:2/546: dwrite d1/db/fa0 [0,4194304] 0 2024-04-02T04:42:58.979 INFO:tasks.workunit.client.1.smithi027.stdout:9/519: dwrite d1a/f39 [4194304,4194304] 0 2024-04-02T04:42:58.982 INFO:tasks.workunit.client.1.smithi027.stdout:9/520: read d1a/d1d/d29/d37/d4c/f66 [3167163,27128] 0 2024-04-02T04:42:58.984 INFO:tasks.workunit.client.1.smithi027.stdout:9/521: fsync d1a/d1d/d29/d56/d8b/f90 0 2024-04-02T04:42:58.986 INFO:tasks.workunit.client.1.smithi027.stdout:5/708: truncate d0/d1d/d61/d7e/f9b 509303 0 2024-04-02T04:42:58.987 INFO:tasks.workunit.client.1.smithi027.stdout:3/474: mkdir d1/de/df/d9e/da0 0 2024-04-02T04:42:58.989 INFO:tasks.workunit.client.1.smithi027.stdout:5/709: read d0/df/d11/d13/dce/f2c [473005,93480] 0 2024-04-02T04:42:59.001 INFO:tasks.workunit.client.1.smithi027.stdout:5/710: creat d0/df/d11/d41/ff7 x:0 0 0 2024-04-02T04:42:59.003 INFO:tasks.workunit.client.1.smithi027.stdout:1/515: mknod d0/d1e/d2e/d3b/d4f/caa 0 2024-04-02T04:42:59.005 INFO:tasks.workunit.client.1.smithi027.stdout:7/565: rename d2/d7/da/dc/d1f/d33/d36/f45 to d2/d7/da/dc/faf 0 2024-04-02T04:42:59.005 INFO:tasks.workunit.client.1.smithi027.stdout:2/547: mknod d1/db/d3b/d17/cc4 0 2024-04-02T04:42:59.009 INFO:tasks.workunit.client.1.smithi027.stdout:7/566: chown d2/d7/da/dc/d16/d3a/d4d/d6f/l75 4698994 1 2024-04-02T04:42:59.011 INFO:tasks.workunit.client.1.smithi027.stdout:1/516: dwrite d0/d1e/d2e/d3b/fa7 [0,4194304] 0 2024-04-02T04:42:59.019 INFO:tasks.workunit.client.1.smithi027.stdout:2/548: creat d1/db/d3b/d17/d29/fc5 x:0 0 0 2024-04-02T04:42:59.024 INFO:tasks.workunit.client.1.smithi027.stdout:4/435: getdents d24/d38/d47/d5c 0 2024-04-02T04:42:59.026 INFO:tasks.workunit.client.1.smithi027.stdout:9/522: dwrite d1a/d1d/d3c/d6e/f97 [0,4194304] 0 2024-04-02T04:42:59.030 INFO:tasks.workunit.client.1.smithi027.stdout:4/436: truncate d24/d38/d44/d41/f87 744005 0 2024-04-02T04:42:59.030 INFO:tasks.workunit.client.1.smithi027.stdout:4/437: read d24/f3f [2697689,123467] 0 2024-04-02T04:42:59.032 INFO:tasks.workunit.client.1.smithi027.stdout:1/517: creat d0/d1e/d5d/d60/fab x:0 0 0 2024-04-02T04:42:59.035 INFO:tasks.workunit.client.1.smithi027.stdout:3/475: creat d1/de/d55/fa1 x:0 0 0 2024-04-02T04:42:59.036 INFO:tasks.workunit.client.1.smithi027.stdout:1/518: dread d0/d1e/d2e/f4a [0,4194304] 0 2024-04-02T04:42:59.037 INFO:tasks.workunit.client.1.smithi027.stdout:3/476: write d1/d1a/f3f [24398,107088] 0 2024-04-02T04:42:59.040 INFO:tasks.workunit.client.1.smithi027.stdout:2/549: creat d1/db/d3b/d17/d2b/fc6 x:0 0 0 2024-04-02T04:42:59.046 INFO:tasks.workunit.client.1.smithi027.stdout:1/519: creat d0/d1e/d2e/d3b/d4f/fac x:0 0 0 2024-04-02T04:42:59.050 INFO:tasks.workunit.client.1.smithi027.stdout:2/550: rename d1/db/d3b/d17/d2b/fba to d1/db/d3b/d61/fc7 0 2024-04-02T04:42:59.051 INFO:tasks.workunit.client.1.smithi027.stdout:7/567: dwrite d2/d7/da/dc/d16/f4a [0,4194304] 0 2024-04-02T04:42:59.051 INFO:tasks.workunit.client.1.smithi027.stdout:2/551: fdatasync d1/f88 0 2024-04-02T04:42:59.052 INFO:tasks.workunit.client.1.smithi027.stdout:4/438: creat d24/d38/d44/d82/f90 x:0 0 0 2024-04-02T04:42:59.052 INFO:tasks.workunit.client.1.smithi027.stdout:5/711: write d0/df/d29/f7a [4961788,104834] 0 2024-04-02T04:42:59.055 INFO:tasks.workunit.client.1.smithi027.stdout:2/552: dread d1/f88 [0,4194304] 0 2024-04-02T04:42:59.056 INFO:tasks.workunit.client.1.smithi027.stdout:3/477: fsync d1/de/df/d13/f35 0 2024-04-02T04:42:59.057 INFO:tasks.workunit.client.1.smithi027.stdout:5/712: truncate d0/d68/d6b/d74/fb1 363355 0 2024-04-02T04:42:59.067 INFO:tasks.workunit.client.1.smithi027.stdout:3/478: symlink d1/de/d74/la2 0 2024-04-02T04:42:59.068 INFO:tasks.workunit.client.1.smithi027.stdout:7/568: symlink d2/d7/da/dc/d1f/d61/lb0 0 2024-04-02T04:42:59.070 INFO:tasks.workunit.client.1.smithi027.stdout:5/713: creat d0/d1d/d61/d72/ff8 x:0 0 0 2024-04-02T04:42:59.071 INFO:tasks.workunit.client.1.smithi027.stdout:9/523: link d1a/d1d/d29/d50/c6c d1a/d1d/d29/d37/d20/d7e/cab 0 2024-04-02T04:42:59.072 INFO:tasks.workunit.client.1.smithi027.stdout:9/524: chown d1a/f38 602 1 2024-04-02T04:42:59.079 INFO:tasks.workunit.client.1.smithi027.stdout:5/714: dwrite d0/d68/d6b/d74/fb1 [0,4194304] 0 2024-04-02T04:42:59.079 INFO:tasks.workunit.client.1.smithi027.stdout:1/520: rmdir d0/d1e/d25/d2b/d59/d96 0 2024-04-02T04:42:59.079 INFO:tasks.workunit.client.1.smithi027.stdout:5/715: chown d0/df/d2f/f67 356 1 2024-04-02T04:42:59.079 INFO:tasks.workunit.client.1.smithi027.stdout:5/716: chown d0/d1d/c44 197 1 2024-04-02T04:42:59.082 INFO:tasks.workunit.client.1.smithi027.stdout:7/569: rename d2/f13 to d2/d7/da/dc/d16/d3f/d50/d8b/d91/fb1 0 2024-04-02T04:42:59.084 INFO:tasks.workunit.client.1.smithi027.stdout:1/521: dwrite d0/d1e/d2e/d3b/fa7 [0,4194304] 0 2024-04-02T04:42:59.086 INFO:tasks.workunit.client.1.smithi027.stdout:4/439: link d24/c54 d24/d48/c91 0 2024-04-02T04:42:59.087 INFO:tasks.workunit.client.1.smithi027.stdout:4/440: chown d24/d38/d47 504830776 1 2024-04-02T04:42:59.087 INFO:tasks.workunit.client.1.smithi027.stdout:4/441: chown d24/d48/f7d 0 1 2024-04-02T04:42:59.092 INFO:tasks.workunit.client.1.smithi027.stdout:3/479: mknod d1/de/d55/ca3 0 2024-04-02T04:42:59.097 INFO:tasks.workunit.client.1.smithi027.stdout:5/717: fdatasync d0/df/d2f/f67 0 2024-04-02T04:42:59.099 INFO:tasks.workunit.client.1.smithi027.stdout:7/570: mkdir d2/d7/da/dc/d16/d3a/db2 0 2024-04-02T04:42:59.102 INFO:tasks.workunit.client.1.smithi027.stdout:4/442: dwrite d24/d38/f8d [0,4194304] 0 2024-04-02T04:42:59.102 INFO:tasks.workunit.client.1.smithi027.stdout:1/522: fdatasync d0/d1e/d25/d2b/f7b 0 2024-04-02T04:42:59.104 INFO:tasks.workunit.client.1.smithi027.stdout:2/553: getdents d1/db/d57 0 2024-04-02T04:42:59.105 INFO:tasks.workunit.client.1.smithi027.stdout:0/530: sync 2024-04-02T04:42:59.107 INFO:tasks.workunit.client.1.smithi027.stdout:3/480: dwrite d1/de/df/d13/d73/d77/f84 [0,4194304] 0 2024-04-02T04:42:59.108 INFO:tasks.workunit.client.1.smithi027.stdout:9/525: dread d1a/f26 [0,4194304] 0 2024-04-02T04:42:59.109 INFO:tasks.workunit.client.1.smithi027.stdout:0/531: truncate d4/d38/da0/f9e 905809 0 2024-04-02T04:42:59.116 INFO:tasks.workunit.client.1.smithi027.stdout:5/718: write d0/d68/d6b/d74/fb6 [119248,31911] 0 2024-04-02T04:42:59.122 INFO:tasks.workunit.client.1.smithi027.stdout:7/571: unlink d2/d7/da/dc/d16/f2d 0 2024-04-02T04:42:59.125 INFO:tasks.workunit.client.1.smithi027.stdout:5/719: dwrite d0/d68/dd7/fe2 [0,4194304] 0 2024-04-02T04:42:59.125 INFO:tasks.workunit.client.1.smithi027.stdout:1/523: stat d0/d1e/d2e/d3e/d5f/c7d 0 2024-04-02T04:42:59.127 INFO:tasks.workunit.client.1.smithi027.stdout:2/554: symlink d1/db/d3b/d17/d2b/db8/lc8 0 2024-04-02T04:42:59.132 INFO:tasks.workunit.client.1.smithi027.stdout:0/532: creat d4/d38/da0/fb4 x:0 0 0 2024-04-02T04:42:59.134 INFO:tasks.workunit.client.1.smithi027.stdout:4/443: dwrite d24/d2d/f43 [0,4194304] 0 2024-04-02T04:42:59.138 INFO:tasks.workunit.client.1.smithi027.stdout:8/563: sync 2024-04-02T04:42:59.138 INFO:tasks.workunit.client.1.smithi027.stdout:6/477: sync 2024-04-02T04:42:59.139 INFO:tasks.workunit.client.1.smithi027.stdout:3/481: dread d1/de/d74/f8e [0,4194304] 0 2024-04-02T04:42:59.141 INFO:tasks.workunit.client.1.smithi027.stdout:9/526: getdents d1a/d1d/d3c/d71/d8c 0 2024-04-02T04:42:59.142 INFO:tasks.workunit.client.1.smithi027.stdout:9/527: stat d1a/d1d/d29/d51/fa6 0 2024-04-02T04:42:59.145 INFO:tasks.workunit.client.1.smithi027.stdout:1/524: rename d0/f5c to d0/d1e/d2e/d3e/d5f/d8b/d9d/fad 0 2024-04-02T04:42:59.145 INFO:tasks.workunit.client.1.smithi027.stdout:1/525: chown d0/de/c2a 12439 1 2024-04-02T04:42:59.146 INFO:tasks.workunit.client.1.smithi027.stdout:1/526: chown d0/f7 81241199 1 2024-04-02T04:42:59.147 INFO:tasks.workunit.client.1.smithi027.stdout:2/555: creat d1/db/d3b/d61/d6c/dbc/dc0/fc9 x:0 0 0 2024-04-02T04:42:59.148 INFO:tasks.workunit.client.1.smithi027.stdout:0/533: rmdir d4/d21/d28/d4e/d5f 39 2024-04-02T04:42:59.149 INFO:tasks.workunit.client.1.smithi027.stdout:2/556: stat d1/db/d3b/d17/d2b/c3e 0 2024-04-02T04:42:59.150 INFO:tasks.workunit.client.1.smithi027.stdout:4/444: creat d24/d2d/f92 x:0 0 0 2024-04-02T04:42:59.151 INFO:tasks.workunit.client.1.smithi027.stdout:8/564: rmdir d5/d8/db/d68 39 2024-04-02T04:42:59.154 INFO:tasks.workunit.client.1.smithi027.stdout:3/482: mkdir d1/de/d55/d63/da4 0 2024-04-02T04:42:59.154 INFO:tasks.workunit.client.1.smithi027.stdout:1/527: dwrite d0/d1e/d2e/f6d [0,4194304] 0 2024-04-02T04:42:59.158 INFO:tasks.workunit.client.1.smithi027.stdout:5/720: creat d0/df/d11/d13/dce/d55/dc3/df2/ff9 x:0 0 0 2024-04-02T04:42:59.159 INFO:tasks.workunit.client.1.smithi027.stdout:5/721: chown d0/df/d11/d13/dce/d55/f8a 162698 1 2024-04-02T04:42:59.159 INFO:tasks.workunit.client.1.smithi027.stdout:9/528: truncate d1a/f26 2106482 0 2024-04-02T04:42:59.161 INFO:tasks.workunit.client.1.smithi027.stdout:5/722: write d0/f69 [2383508,53383] 0 2024-04-02T04:42:59.162 INFO:tasks.workunit.client.1.smithi027.stdout:3/483: read d1/de/f30 [116178,128760] 0 2024-04-02T04:42:59.164 INFO:tasks.workunit.client.1.smithi027.stdout:3/484: chown d1/de/d55/l88 1998762 1 2024-04-02T04:42:59.164 INFO:tasks.workunit.client.1.smithi027.stdout:7/572: link d2/d7/da/dc/d16/d3a/d4d/f58 d2/d7/da/fb3 0 2024-04-02T04:42:59.173 INFO:tasks.workunit.client.1.smithi027.stdout:3/485: dwrite d1/de/df/d13/f5c [0,4194304] 0 2024-04-02T04:42:59.174 INFO:tasks.workunit.client.1.smithi027.stdout:3/486: readlink d1/d1a/l62 0 2024-04-02T04:42:59.175 INFO:tasks.workunit.client.1.smithi027.stdout:3/487: chown d1/d1a/c65 10695501 1 2024-04-02T04:42:59.178 INFO:tasks.workunit.client.1.smithi027.stdout:9/529: mknod d1a/d1d/d29/d50/d73/cac 0 2024-04-02T04:42:59.180 INFO:tasks.workunit.client.1.smithi027.stdout:5/723: mknod d0/d1d/cfa 0 2024-04-02T04:42:59.186 INFO:tasks.workunit.client.1.smithi027.stdout:6/478: creat d13/d1b/d24/d25/d56/fa0 x:0 0 0 2024-04-02T04:42:59.186 INFO:tasks.workunit.client.1.smithi027.stdout:4/445: dwrite d24/f32 [0,4194304] 0 2024-04-02T04:42:59.191 INFO:tasks.workunit.client.1.smithi027.stdout:9/530: mkdir d1a/d1d/d29/d37/d20/dad 0 2024-04-02T04:42:59.192 INFO:tasks.workunit.client.1.smithi027.stdout:0/534: write d4/f23 [113834,1046] 0 2024-04-02T04:42:59.195 INFO:tasks.workunit.client.1.smithi027.stdout:2/557: link d1/db/d3b/d61/f5f d1/d91/fca 0 2024-04-02T04:42:59.196 INFO:tasks.workunit.client.1.smithi027.stdout:2/558: write d1/f40 [1086102,119960] 0 2024-04-02T04:42:59.197 INFO:tasks.workunit.client.1.smithi027.stdout:8/565: mkdir d5/d8/db/dd/d5f/d91/dba 0 2024-04-02T04:42:59.201 INFO:tasks.workunit.client.1.smithi027.stdout:3/488: mkdir d1/de/df/da5 0 2024-04-02T04:42:59.204 INFO:tasks.workunit.client.1.smithi027.stdout:2/559: dwrite d1/db/d3b/d17/f14 [0,4194304] 0 2024-04-02T04:42:59.210 INFO:tasks.workunit.client.1.smithi027.stdout:6/479: rename d13/d1b/d24/d28/f2a to d13/d1b/d24/d25/d56/d73/fa1 0 2024-04-02T04:42:59.212 INFO:tasks.workunit.client.1.smithi027.stdout:6/480: read d13/d1b/d24/d25/d27/f81 [90365,41421] 0 2024-04-02T04:42:59.213 INFO:tasks.workunit.client.1.smithi027.stdout:9/531: symlink d1a/d1d/d29/lae 0 2024-04-02T04:42:59.213 INFO:tasks.workunit.client.1.smithi027.stdout:3/489: mkdir d1/d1a/da6 0 2024-04-02T04:42:59.215 INFO:tasks.workunit.client.1.smithi027.stdout:0/535: mkdir d4/d21/d28/d4e/db5 0 2024-04-02T04:42:59.216 INFO:tasks.workunit.client.1.smithi027.stdout:2/560: symlink d1/db/lcb 0 2024-04-02T04:42:59.217 INFO:tasks.workunit.client.1.smithi027.stdout:4/446: creat d24/d38/d47/f93 x:0 0 0 2024-04-02T04:42:59.218 INFO:tasks.workunit.client.1.smithi027.stdout:2/561: truncate d1/db/d3b/d17/f9e 377666 0 2024-04-02T04:42:59.220 INFO:tasks.workunit.client.1.smithi027.stdout:8/566: rename d5/d8/db/df/c34 to d5/d8/db/dd/d4a/cbb 0 2024-04-02T04:42:59.222 INFO:tasks.workunit.client.1.smithi027.stdout:6/481: creat d13/d1b/d48/d58/fa2 x:0 0 0 2024-04-02T04:42:59.224 INFO:tasks.workunit.client.1.smithi027.stdout:5/724: link d0/df/d11/d13/lba d0/lfb 0 2024-04-02T04:42:59.225 INFO:tasks.workunit.client.1.smithi027.stdout:5/725: write d0/f69 [1370192,30477] 0 2024-04-02T04:42:59.225 INFO:tasks.workunit.client.1.smithi027.stdout:0/536: creat d4/d38/fb6 x:0 0 0 2024-04-02T04:42:59.234 INFO:tasks.workunit.client.1.smithi027.stdout:9/532: rename d1a/d1d/d29/d37/d20/faa to d1a/d1d/d3c/d71/faf 0 2024-04-02T04:42:59.234 INFO:tasks.workunit.client.1.smithi027.stdout:8/567: readlink d5/d8/d15/l75 0 2024-04-02T04:42:59.234 INFO:tasks.workunit.client.1.smithi027.stdout:9/533: chown d1a/d1d/d3c/l44 268670654 1 2024-04-02T04:42:59.240 INFO:tasks.workunit.client.1.smithi027.stdout:3/490: creat d1/de/fa7 x:0 0 0 2024-04-02T04:42:59.242 INFO:tasks.workunit.client.1.smithi027.stdout:8/568: write d5/d8/db/dd/f4f [1899514,75046] 0 2024-04-02T04:42:59.244 INFO:tasks.workunit.client.1.smithi027.stdout:2/562: dread d1/db/d3b/d17/d2b/db8/f9f [0,4194304] 0 2024-04-02T04:42:59.248 INFO:tasks.workunit.client.1.smithi027.stdout:5/726: link d0/df/d11/d13/dce/d55/f8a d0/df/ffc 0 2024-04-02T04:42:59.251 INFO:tasks.workunit.client.1.smithi027.stdout:6/482: dread d13/d1b/d24/d28/f35 [0,4194304] 0 2024-04-02T04:42:59.254 INFO:tasks.workunit.client.1.smithi027.stdout:6/483: dread d13/f14 [0,4194304] 0 2024-04-02T04:42:59.254 INFO:tasks.workunit.client.1.smithi027.stdout:8/569: unlink d5/l1e 0 2024-04-02T04:42:59.255 INFO:tasks.workunit.client.1.smithi027.stdout:3/491: dread d1/f46 [0,4194304] 0 2024-04-02T04:42:59.255 INFO:tasks.workunit.client.1.smithi027.stdout:6/484: stat d13/d1b/d24/d25/d56/d73 0 2024-04-02T04:42:59.256 INFO:tasks.workunit.client.1.smithi027.stdout:3/492: dread - d1/de/df/f6b zero size 2024-04-02T04:42:59.257 INFO:tasks.workunit.client.1.smithi027.stdout:3/493: write d1/de/d55/f5f [9396381,127577] 0 2024-04-02T04:42:59.261 INFO:tasks.workunit.client.1.smithi027.stdout:4/447: getdents d24/d38/d44/d82 0 2024-04-02T04:42:59.263 INFO:tasks.workunit.client.1.smithi027.stdout:5/727: dread d0/df/d2f/d3b/d51/f53 [0,4194304] 0 2024-04-02T04:42:59.265 INFO:tasks.workunit.client.1.smithi027.stdout:6/485: truncate d13/f16 3630688 0 2024-04-02T04:42:59.266 INFO:tasks.workunit.client.1.smithi027.stdout:0/537: dwrite d4/d3d/f47 [4194304,4194304] 0 2024-04-02T04:42:59.269 INFO:tasks.workunit.client.1.smithi027.stdout:6/486: truncate d13/d1b/d24/d28/d3c/f92 523099 0 2024-04-02T04:42:59.270 INFO:tasks.workunit.client.1.smithi027.stdout:2/563: mknod d1/db/d3b/ccc 0 2024-04-02T04:42:59.271 INFO:tasks.workunit.client.1.smithi027.stdout:4/448: creat d24/d38/d44/d60/f94 x:0 0 0 2024-04-02T04:42:59.272 INFO:tasks.workunit.client.1.smithi027.stdout:8/570: symlink d5/d8/db/dd/lbc 0 2024-04-02T04:42:59.273 INFO:tasks.workunit.client.1.smithi027.stdout:4/449: chown ld 2986 1 2024-04-02T04:42:59.273 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:42:59 smithi016 ceph-mon[28969]: pgmap v28: 65 pgs: 65 active+clean; 2.8 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 56 MiB/s rd, 202 MiB/s wr, 383 op/s 2024-04-02T04:42:59.276 INFO:tasks.workunit.client.1.smithi027.stdout:4/450: write d24/d48/f7d [5633042,69840] 0 2024-04-02T04:42:59.278 INFO:tasks.workunit.client.1.smithi027.stdout:9/534: rename d1a/c33 to d1a/d1d/d29/d37/cb0 0 2024-04-02T04:42:59.281 INFO:tasks.workunit.client.1.smithi027.stdout:9/535: write f12 [4871488,34499] 0 2024-04-02T04:42:59.281 INFO:tasks.workunit.client.1.smithi027.stdout:6/487: fdatasync d13/f62 0 2024-04-02T04:42:59.283 INFO:tasks.workunit.client.1.smithi027.stdout:2/564: dwrite d1/f40 [0,4194304] 0 2024-04-02T04:42:59.285 INFO:tasks.workunit.client.1.smithi027.stdout:3/494: dread d1/de/df/d61/f6c [0,4194304] 0 2024-04-02T04:42:59.287 INFO:tasks.workunit.client.1.smithi027.stdout:5/728: getdents d0/df/d29/da5/da6/de9 0 2024-04-02T04:42:59.287 INFO:tasks.workunit.client.1.smithi027.stdout:3/495: stat d1/d1a/c3e 0 2024-04-02T04:42:59.289 INFO:tasks.workunit.client.1.smithi027.stdout:2/565: read - d1/db/d3b/d17/d29/f4f zero size 2024-04-02T04:42:59.289 INFO:tasks.workunit.client.1.smithi027.stdout:8/571: dwrite d5/d8/db/dd/f4f [0,4194304] 0 2024-04-02T04:42:59.290 INFO:tasks.workunit.client.1.smithi027.stdout:3/496: write d1/de/df/f6b [539109,11823] 0 2024-04-02T04:42:59.292 INFO:tasks.workunit.client.1.smithi027.stdout:8/572: fsync d5/d8/db/dd/d5f/d27/f9e 0 2024-04-02T04:42:59.296 INFO:tasks.workunit.client.1.smithi027.stdout:3/497: write d1/de/df/d61/f4c [3670086,119428] 0 2024-04-02T04:42:59.301 INFO:tasks.workunit.client.1.smithi027.stdout:4/451: mkdir d24/d38/d44/d82/d95 0 2024-04-02T04:42:59.305 INFO:tasks.workunit.client.1.smithi027.stdout:3/498: dwrite d1/de/d55/d63/f9f [0,4194304] 0 2024-04-02T04:42:59.306 INFO:tasks.workunit.client.1.smithi027.stdout:4/452: read d24/f71 [4115435,3290] 0 2024-04-02T04:42:59.310 INFO:tasks.workunit.client.1.smithi027.stdout:9/536: creat d1a/d1d/d29/d50/d73/fb1 x:0 0 0 2024-04-02T04:42:59.312 INFO:tasks.workunit.client.1.smithi027.stdout:6/488: creat d13/d8d/fa3 x:0 0 0 2024-04-02T04:42:59.312 INFO:tasks.workunit.client.1.smithi027.stdout:2/566: mkdir d1/db/d3b/d17/d68/dcd 0 2024-04-02T04:42:59.317 INFO:tasks.workunit.client.1.smithi027.stdout:9/537: dread d1a/d1d/d29/d37/d20/f3f [0,4194304] 0 2024-04-02T04:42:59.318 INFO:tasks.workunit.client.1.smithi027.stdout:5/729: rename d0/df/d2f/d3b/l4c to d0/df/d29/da5/da6/de9/lfd 0 2024-04-02T04:42:59.320 INFO:tasks.workunit.client.1.smithi027.stdout:2/567: dread d1/db/d3b/f70 [0,4194304] 0 2024-04-02T04:42:59.323 INFO:tasks.workunit.client.1.smithi027.stdout:2/568: write d1/db/d3b/d17/d29/fc5 [709635,15032] 0 2024-04-02T04:42:59.324 INFO:tasks.workunit.client.1.smithi027.stdout:3/499: mknod d1/d1a/ca8 0 2024-04-02T04:42:59.328 INFO:tasks.workunit.client.1.smithi027.stdout:6/489: creat d13/d8d/fa4 x:0 0 0 2024-04-02T04:42:59.328 INFO:tasks.workunit.client.1.smithi027.stdout:4/453: dread f21 [0,4194304] 0 2024-04-02T04:42:59.332 INFO:tasks.workunit.client.1.smithi027.stdout:5/730: read d0/d68/fa4 [2567420,29578] 0 2024-04-02T04:42:59.332 INFO:tasks.workunit.client.1.smithi027.stdout:3/500: dwrite d1/de/f71 [0,4194304] 0 2024-04-02T04:42:59.334 INFO:tasks.workunit.client.1.smithi027.stdout:2/569: symlink d1/db/d57/lce 0 2024-04-02T04:42:59.335 INFO:tasks.workunit.client.1.smithi027.stdout:3/501: write d1/de/df/d13/d73/f78 [142672,43259] 0 2024-04-02T04:42:59.341 INFO:tasks.workunit.client.1.smithi027.stdout:9/538: dread d1a/d1d/f4b [0,4194304] 0 2024-04-02T04:42:59.341 INFO:tasks.workunit.client.1.smithi027.stdout:9/539: fsync f12 0 2024-04-02T04:42:59.342 INFO:tasks.workunit.client.1.smithi027.stdout:4/454: creat d24/d38/d44/d82/f96 x:0 0 0 2024-04-02T04:42:59.342 INFO:tasks.workunit.client.1.smithi027.stdout:9/540: stat d1a/d1d/f4b 0 2024-04-02T04:42:59.344 INFO:tasks.workunit.client.1.smithi027.stdout:5/731: mkdir d0/df/d11/d13/dce/d55/dc3/dfe 0 2024-04-02T04:42:59.346 INFO:tasks.workunit.client.1.smithi027.stdout:5/732: read d0/d1a/d20/f93 [1019385,4615] 0 2024-04-02T04:42:59.346 INFO:tasks.workunit.client.1.smithi027.stdout:3/502: creat d1/de/df/d13/d73/fa9 x:0 0 0 2024-04-02T04:42:59.347 INFO:tasks.workunit.client.1.smithi027.stdout:6/490: mknod d13/ca5 0 2024-04-02T04:42:59.350 INFO:tasks.workunit.client.1.smithi027.stdout:9/541: rmdir d1a/d1d/d3c/d71 39 2024-04-02T04:42:59.353 INFO:tasks.workunit.client.1.smithi027.stdout:4/455: mkdir d24/d38/d44/d82/d97 0 2024-04-02T04:42:59.354 INFO:tasks.workunit.client.1.smithi027.stdout:6/491: dwrite d13/d8d/fa3 [0,4194304] 0 2024-04-02T04:42:59.355 INFO:tasks.workunit.client.1.smithi027.stdout:6/492: chown d13/d1b/d24/d25/d56/fa0 395387434 1 2024-04-02T04:42:59.361 INFO:tasks.workunit.client.1.smithi027.stdout:5/733: rmdir d0/df 39 2024-04-02T04:42:59.368 INFO:tasks.workunit.client.1.smithi027.stdout:4/456: mknod d24/d38/d47/d62/c98 0 2024-04-02T04:42:59.369 INFO:tasks.workunit.client.1.smithi027.stdout:4/457: chown d24/f3a 0 1 2024-04-02T04:42:59.371 INFO:tasks.workunit.client.1.smithi027.stdout:9/542: dread d1a/f63 [0,4194304] 0 2024-04-02T04:42:59.378 INFO:tasks.workunit.client.1.smithi027.stdout:9/543: dwrite d1a/d1d/d29/d50/d73/f7b [4194304,4194304] 0 2024-04-02T04:42:59.380 INFO:tasks.workunit.client.1.smithi027.stdout:3/503: mkdir d1/d1a/da6/daa 0 2024-04-02T04:42:59.383 INFO:tasks.workunit.client.1.smithi027.stdout:5/734: mkdir d0/d1d/d61/d7e/dff 0 2024-04-02T04:42:59.386 INFO:tasks.workunit.client.1.smithi027.stdout:4/458: rename l1f to d24/d38/d44/l99 0 2024-04-02T04:42:59.387 INFO:tasks.workunit.client.1.smithi027.stdout:4/459: fdatasync d24/f27 0 2024-04-02T04:42:59.390 INFO:tasks.workunit.client.1.smithi027.stdout:8/573: symlink d5/d8/db/dd/d5f/d27/d31/d5b/lbd 0 2024-04-02T04:42:59.392 INFO:tasks.workunit.client.1.smithi027.stdout:2/570: getdents d1/db/d3b/d61 0 2024-04-02T04:42:59.395 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:42:59 smithi027 ceph-mon[37550]: pgmap v28: 65 pgs: 65 active+clean; 2.8 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 56 MiB/s rd, 202 MiB/s wr, 383 op/s 2024-04-02T04:42:59.401 INFO:tasks.workunit.client.1.smithi027.stdout:6/493: creat d13/d1b/d24/d25/d56/fa6 x:0 0 0 2024-04-02T04:42:59.404 INFO:tasks.workunit.client.1.smithi027.stdout:4/460: creat d24/d38/d47/d62/f9a x:0 0 0 2024-04-02T04:42:59.404 INFO:tasks.workunit.client.1.smithi027.stdout:4/461: chown d24 15360 1 2024-04-02T04:42:59.406 INFO:tasks.workunit.client.1.smithi027.stdout:2/571: dread d1/db/d3b/d17/fc [8388608,4194304] 0 2024-04-02T04:42:59.408 INFO:tasks.workunit.client.1.smithi027.stdout:2/572: chown d1/db/d3b/d17/d6d/d84/c85 2 1 2024-04-02T04:42:59.409 INFO:tasks.workunit.client.1.smithi027.stdout:2/573: write d1/db/d3b/d17/d68/d69/fb3 [1022114,1804] 0 2024-04-02T04:42:59.411 INFO:tasks.workunit.client.1.smithi027.stdout:2/574: read - d1/db/d3b/d61/d6c/dbc/dc0/fc9 zero size 2024-04-02T04:42:59.411 INFO:tasks.workunit.client.1.smithi027.stdout:3/504: unlink d1/d1a/l2b 0 2024-04-02T04:42:59.414 INFO:tasks.workunit.client.1.smithi027.stdout:5/735: unlink d0/df/d11/d37/c48 0 2024-04-02T04:42:59.418 INFO:tasks.workunit.client.1.smithi027.stdout:8/574: mknod d5/d8/cbe 0 2024-04-02T04:42:59.419 INFO:tasks.workunit.client.1.smithi027.stdout:2/575: dwrite d1/db/d3b/d17/d68/d78/fa1 [0,4194304] 0 2024-04-02T04:42:59.420 INFO:tasks.workunit.client.1.smithi027.stdout:2/576: write d1/db/d3b/d61/d75/f92 [990590,90912] 0 2024-04-02T04:42:59.422 INFO:tasks.workunit.client.1.smithi027.stdout:9/544: creat d1a/d1d/d3c/fb2 x:0 0 0 2024-04-02T04:42:59.422 INFO:tasks.workunit.client.1.smithi027.stdout:2/577: write d1/f88 [1016679,126481] 0 2024-04-02T04:42:59.424 INFO:tasks.workunit.client.1.smithi027.stdout:2/578: chown d1/db/d3b/d61/d6c/d8a 25848 1 2024-04-02T04:42:59.425 INFO:tasks.workunit.client.1.smithi027.stdout:2/579: chown d1/db/d3b/d17/d68/d69 1553204121 1 2024-04-02T04:42:59.429 INFO:tasks.workunit.client.1.smithi027.stdout:6/494: mkdir d13/d1b/d8c/da7 0 2024-04-02T04:42:59.431 INFO:tasks.workunit.client.1.smithi027.stdout:4/462: dwrite d24/d48/f6b [0,4194304] 0 2024-04-02T04:42:59.432 INFO:tasks.workunit.client.1.smithi027.stdout:5/736: rmdir d0/df/d11/d13/d9a/d9c/db8 39 2024-04-02T04:42:59.442 INFO:tasks.workunit.client.1.smithi027.stdout:8/575: unlink d5/d8/db/dd/d5f/d27/f9e 0 2024-04-02T04:42:59.442 INFO:tasks.workunit.client.1.smithi027.stdout:8/576: chown d5/l16 438001 1 2024-04-02T04:42:59.442 INFO:tasks.workunit.client.1.smithi027.stdout:4/463: dwrite d24/d38/d44/d41/d49/f83 [4194304,4194304] 0 2024-04-02T04:42:59.452 INFO:tasks.workunit.client.1.smithi027.stdout:9/545: dwrite d1a/d1d/d29/d56/d8b/fa4 [0,4194304] 0 2024-04-02T04:42:59.455 INFO:tasks.workunit.client.1.smithi027.stdout:9/546: write d1a/d1d/d29/d56/d8b/fa4 [3006676,79289] 0 2024-04-02T04:42:59.457 INFO:tasks.workunit.client.1.smithi027.stdout:5/737: creat d0/d1d/f100 x:0 0 0 2024-04-02T04:42:59.461 INFO:tasks.workunit.client.1.smithi027.stdout:2/580: dwrite d1/db/d57/f99 [0,4194304] 0 2024-04-02T04:42:59.462 INFO:tasks.workunit.client.1.smithi027.stdout:2/581: chown d1/db/d3b/d17/f63 58993164 1 2024-04-02T04:42:59.467 INFO:tasks.workunit.client.1.smithi027.stdout:3/505: rename d1/de/d55/d63/f9b to d1/fab 0 2024-04-02T04:42:59.469 INFO:tasks.workunit.client.1.smithi027.stdout:3/506: read d1/de/d55/f75 [473997,93227] 0 2024-04-02T04:42:59.470 INFO:tasks.workunit.client.1.smithi027.stdout:3/507: fsync d1/de/d74/f80 0 2024-04-02T04:42:59.471 INFO:tasks.workunit.client.1.smithi027.stdout:3/508: chown d1/de/df/f6b 80358449 1 2024-04-02T04:42:59.472 INFO:tasks.workunit.client.1.smithi027.stdout:6/495: mknod d13/d1b/d24/ca8 0 2024-04-02T04:42:59.472 INFO:tasks.workunit.client.1.smithi027.stdout:3/509: fsync d1/de/df/d61/f6e 0 2024-04-02T04:42:59.474 INFO:tasks.workunit.client.1.smithi027.stdout:3/510: chown d1/cb 48351 1 2024-04-02T04:42:59.476 INFO:tasks.workunit.client.1.smithi027.stdout:4/464: mkdir d24/d38/d44/d82/d97/d9b 0 2024-04-02T04:42:59.477 INFO:tasks.workunit.client.1.smithi027.stdout:6/496: dread d13/d1b/d48/d58/f6a [0,4194304] 0 2024-04-02T04:42:59.478 INFO:tasks.workunit.client.1.smithi027.stdout:8/577: mknod d5/d8/db/d68/d95/cbf 0 2024-04-02T04:42:59.487 INFO:tasks.workunit.client.1.smithi027.stdout:6/497: dwrite d13/d1b/d24/d25/d27/f76 [0,4194304] 0 2024-04-02T04:42:59.489 INFO:tasks.workunit.client.1.smithi027.stdout:1/528: sync 2024-04-02T04:42:59.489 INFO:tasks.workunit.client.1.smithi027.stdout:7/573: sync 2024-04-02T04:42:59.490 INFO:tasks.workunit.client.1.smithi027.stdout:9/547: dread d1a/d1d/f72 [0,4194304] 0 2024-04-02T04:42:59.494 INFO:tasks.workunit.client.1.smithi027.stdout:5/738: creat d0/df/d11/d13/d9a/d9c/db8/f101 x:0 0 0 2024-04-02T04:42:59.496 INFO:tasks.workunit.client.1.smithi027.stdout:5/739: read - d0/df/d29/ff1 zero size 2024-04-02T04:42:59.496 INFO:tasks.workunit.client.1.smithi027.stdout:4/465: rmdir d24/d2d 39 2024-04-02T04:42:59.498 INFO:tasks.workunit.client.1.smithi027.stdout:5/740: stat d0/df/d11/d13/f28 0 2024-04-02T04:42:59.498 INFO:tasks.workunit.client.1.smithi027.stdout:5/741: chown d0/df/d11/d13 5456 1 2024-04-02T04:42:59.499 INFO:tasks.workunit.client.1.smithi027.stdout:8/578: stat d5/d8/db/dd/d4a/cbb 0 2024-04-02T04:42:59.503 INFO:tasks.workunit.client.1.smithi027.stdout:2/582: mkdir d1/db/d3b/d17/dcf 0 2024-04-02T04:42:59.509 INFO:tasks.workunit.client.1.smithi027.stdout:2/583: dwrite d1/db/d3b/d17/d68/d78/fa1 [4194304,4194304] 0 2024-04-02T04:42:59.517 INFO:tasks.workunit.client.1.smithi027.stdout:1/529: creat d0/d1e/d25/d2b/fae x:0 0 0 2024-04-02T04:42:59.522 INFO:tasks.workunit.client.1.smithi027.stdout:3/511: mknod d1/cac 0 2024-04-02T04:42:59.525 INFO:tasks.workunit.client.1.smithi027.stdout:1/530: dwrite d0/d1e/d5d/d60/fab [0,4194304] 0 2024-04-02T04:42:59.526 INFO:tasks.workunit.client.1.smithi027.stdout:1/531: chown d0/de/d52/l91 6 1 2024-04-02T04:42:59.530 INFO:tasks.workunit.client.1.smithi027.stdout:6/498: write d13/d1b/d24/d25/d27/f81 [1161686,1547] 0 2024-04-02T04:42:59.532 INFO:tasks.workunit.client.1.smithi027.stdout:5/742: rename d0/df/d11/d37/fa2 to d0/df/d11/d41/de6/f102 0 2024-04-02T04:42:59.532 INFO:tasks.workunit.client.1.smithi027.stdout:6/499: readlink d13/d1b/d24/d25/d42/l7b 0 2024-04-02T04:42:59.534 INFO:tasks.workunit.client.1.smithi027.stdout:5/743: fdatasync d0/d1d/d61/f96 0 2024-04-02T04:42:59.536 INFO:tasks.workunit.client.1.smithi027.stdout:6/500: chown d13/d1b/d24/d25/d56/l5b 61279 1 2024-04-02T04:42:59.536 INFO:tasks.workunit.client.1.smithi027.stdout:1/532: dwrite d0/d1e/d2e/f7a [0,4194304] 0 2024-04-02T04:42:59.536 INFO:tasks.workunit.client.1.smithi027.stdout:2/584: creat d1/db/d3b/d17/d2b/db8/fd0 x:0 0 0 2024-04-02T04:42:59.539 INFO:tasks.workunit.client.1.smithi027.stdout:2/585: fsync d1/db/d3b/d17/d12/f7e 0 2024-04-02T04:42:59.542 INFO:tasks.workunit.client.1.smithi027.stdout:9/548: creat d1a/d1d/d3c/d6e/fb3 x:0 0 0 2024-04-02T04:42:59.543 INFO:tasks.workunit.client.1.smithi027.stdout:9/549: chown d1a/d1d/d29/l9e 3 1 2024-04-02T04:42:59.544 INFO:tasks.workunit.client.1.smithi027.stdout:9/550: fdatasync d1a/d1d/d3c/d6e/fb3 0 2024-04-02T04:42:59.545 INFO:tasks.workunit.client.1.smithi027.stdout:3/512: creat d1/de/d55/d99/fad x:0 0 0 2024-04-02T04:42:59.551 INFO:tasks.workunit.client.1.smithi027.stdout:8/579: fsync d5/d8/db/dd/f1d 0 2024-04-02T04:42:59.554 INFO:tasks.workunit.client.1.smithi027.stdout:0/538: sync 2024-04-02T04:42:59.562 INFO:tasks.workunit.client.1.smithi027.stdout:5/744: creat d0/df/d29/da5/da6/f103 x:0 0 0 2024-04-02T04:42:59.563 INFO:tasks.workunit.client.1.smithi027.stdout:1/533: dread d0/d1e/d25/d3c/d64/f8c [0,4194304] 0 2024-04-02T04:42:59.563 INFO:tasks.workunit.client.1.smithi027.stdout:5/745: truncate d0/fec 426924 0 2024-04-02T04:42:59.565 INFO:tasks.workunit.client.1.smithi027.stdout:4/466: dread d24/f3f [0,4194304] 0 2024-04-02T04:42:59.566 INFO:tasks.workunit.client.1.smithi027.stdout:7/574: write d2/d1d/d90/f5d [591967,29204] 0 2024-04-02T04:42:59.569 INFO:tasks.workunit.client.1.smithi027.stdout:6/501: creat d13/d1b/d24/d25/d42/fa9 x:0 0 0 2024-04-02T04:42:59.569 INFO:tasks.workunit.client.1.smithi027.stdout:9/551: fsync d1a/d1d/d3c/d6e/fb3 0 2024-04-02T04:42:59.570 INFO:tasks.workunit.client.1.smithi027.stdout:9/552: fsync d1a/d1d/d29/d37/d20/d7e/f87 0 2024-04-02T04:42:59.570 INFO:tasks.workunit.client.1.smithi027.stdout:2/586: creat d1/db/d3b/d61/d6c/dbc/fd1 x:0 0 0 2024-04-02T04:42:59.573 INFO:tasks.workunit.client.1.smithi027.stdout:3/513: rmdir d1/de/df/d61 39 2024-04-02T04:42:59.579 INFO:tasks.workunit.client.1.smithi027.stdout:5/746: creat d0/df/d2f/d3b/f104 x:0 0 0 2024-04-02T04:42:59.580 INFO:tasks.workunit.client.1.smithi027.stdout:5/747: write d0/df/d29/da5/da6/f103 [824584,9537] 0 2024-04-02T04:42:59.581 INFO:tasks.workunit.client.1.smithi027.stdout:1/534: creat d0/d1e/d2e/d3e/d5f/d8b/d9d/faf x:0 0 0 2024-04-02T04:42:59.585 INFO:tasks.workunit.client.1.smithi027.stdout:7/575: creat d2/d7/da/dc/d16/d3f/d44/fb4 x:0 0 0 2024-04-02T04:42:59.587 INFO:tasks.workunit.client.1.smithi027.stdout:6/502: mkdir d13/d1b/d24/d25/d56/d6d/daa 0 2024-04-02T04:42:59.588 INFO:tasks.workunit.client.1.smithi027.stdout:9/553: creat d1a/d1d/d29/d56/d8b/fb4 x:0 0 0 2024-04-02T04:42:59.590 INFO:tasks.workunit.client.1.smithi027.stdout:2/587: mkdir d1/db/d3b/d61/dd2 0 2024-04-02T04:42:59.598 INFO:tasks.workunit.client.1.smithi027.stdout:5/748: mkdir d0/df/d2f/d3b/d105 0 2024-04-02T04:42:59.602 INFO:tasks.workunit.client.1.smithi027.stdout:4/467: mknod d24/d38/d44/c9c 0 2024-04-02T04:42:59.604 INFO:tasks.workunit.client.1.smithi027.stdout:7/576: truncate d2/d1d/f25 703786 0 2024-04-02T04:42:59.607 INFO:tasks.workunit.client.1.smithi027.stdout:9/554: rename l6 to d1a/d1d/d29/d56/lb5 0 2024-04-02T04:42:59.615 INFO:tasks.workunit.client.1.smithi027.stdout:2/588: creat d1/db/d3b/d17/d29/dad/fd3 x:0 0 0 2024-04-02T04:42:59.615 INFO:tasks.workunit.client.1.smithi027.stdout:8/580: link d5/d8/db/d57/f62 d5/d8/d15/d71/d96/fc0 0 2024-04-02T04:42:59.615 INFO:tasks.workunit.client.1.smithi027.stdout:3/514: write d1/f2a [879685,94019] 0 2024-04-02T04:42:59.616 INFO:tasks.workunit.client.1.smithi027.stdout:0/539: creat d4/d16/d1b/fb7 x:0 0 0 2024-04-02T04:42:59.618 INFO:tasks.workunit.client.1.smithi027.stdout:5/749: fdatasync d0/f6 0 2024-04-02T04:42:59.622 INFO:tasks.workunit.client.1.smithi027.stdout:5/750: dread d0/df/f80 [0,4194304] 0 2024-04-02T04:42:59.623 INFO:tasks.workunit.client.1.smithi027.stdout:7/577: readlink d2/d7/da/d35/l42 0 2024-04-02T04:42:59.624 INFO:tasks.workunit.client.1.smithi027.stdout:5/751: chown d0/df/c30 13 1 2024-04-02T04:42:59.628 INFO:tasks.workunit.client.1.smithi027.stdout:1/535: dwrite d0/d1e/d2e/d3e/f6e [0,4194304] 0 2024-04-02T04:42:59.655 INFO:tasks.workunit.client.1.smithi027.stdout:2/589: fsync d1/f13 0 2024-04-02T04:42:59.655 INFO:tasks.workunit.client.1.smithi027.stdout:2/590: stat d1/db/d57/lce 0 2024-04-02T04:42:59.656 INFO:tasks.workunit.client.1.smithi027.stdout:3/515: rmdir d1/de/d74 39 2024-04-02T04:42:59.657 INFO:tasks.workunit.client.1.smithi027.stdout:2/591: write d1/db/d3b/d17/d29/fc1 [883232,106690] 0 2024-04-02T04:42:59.663 INFO:tasks.workunit.client.1.smithi027.stdout:3/516: dwrite d1/de/f8a [4194304,4194304] 0 2024-04-02T04:42:59.675 INFO:tasks.workunit.client.1.smithi027.stdout:2/592: mkdir d1/db/d3b/d61/d6c/dbc/dd4 0 2024-04-02T04:42:59.678 INFO:tasks.workunit.client.1.smithi027.stdout:6/503: getdents d13/d1b/d24/d25/d42 0 2024-04-02T04:42:59.680 INFO:tasks.workunit.client.1.smithi027.stdout:3/517: fdatasync d1/de/df/d13/f2d 0 2024-04-02T04:42:59.681 INFO:tasks.workunit.client.1.smithi027.stdout:3/518: chown d1/f2e 5311079 1 2024-04-02T04:42:59.683 INFO:tasks.workunit.client.1.smithi027.stdout:0/540: write d4/d38/d66/f24 [734347,87750] 0 2024-04-02T04:42:59.684 INFO:tasks.workunit.client.1.smithi027.stdout:4/468: write d24/f39 [592134,27097] 0 2024-04-02T04:42:59.685 INFO:tasks.workunit.client.1.smithi027.stdout:0/541: write d4/f73 [287350,87489] 0 2024-04-02T04:42:59.688 INFO:tasks.workunit.client.1.smithi027.stdout:8/581: mkdir d5/d8/db/dd/d5f/d27/d31/db9/dc1 0 2024-04-02T04:42:59.691 INFO:tasks.workunit.client.1.smithi027.stdout:7/578: link d2/d1c/cae d2/d7/da/dc/d16/d3f/d44/cb5 0 2024-04-02T04:42:59.693 INFO:tasks.workunit.client.1.smithi027.stdout:7/579: write d2/d7/da/dc/d16/d3f/d50/f66 [735013,9402] 0 2024-04-02T04:42:59.694 INFO:tasks.workunit.client.1.smithi027.stdout:5/752: rmdir d0/df/d11/d13/dce/d55/dc3/dfe 0 2024-04-02T04:42:59.697 INFO:tasks.workunit.client.1.smithi027.stdout:0/542: dread d4/d38/d66/d2d/d4c/f4d [0,4194304] 0 2024-04-02T04:42:59.698 INFO:tasks.workunit.client.1.smithi027.stdout:9/555: rename d1a/d1d/d3c/d71 to d1a/db6 0 2024-04-02T04:42:59.701 INFO:tasks.workunit.client.1.smithi027.stdout:6/504: mknod d13/d1b/d24/cab 0 2024-04-02T04:42:59.703 INFO:tasks.workunit.client.1.smithi027.stdout:7/580: chown d2/df/d89/faa 4 1 2024-04-02T04:42:59.705 INFO:tasks.workunit.client.1.smithi027.stdout:7/581: write d2/d7/da/dc/d16/fac [316311,93057] 0 2024-04-02T04:42:59.705 INFO:tasks.workunit.client.1.smithi027.stdout:4/469: mknod d24/d48/c9d 0 2024-04-02T04:42:59.705 INFO:tasks.workunit.client.1.smithi027.stdout:1/536: link d0/fd d0/d1e/d5d/d60/fb0 0 2024-04-02T04:42:59.706 INFO:tasks.workunit.client.1.smithi027.stdout:7/582: stat d2/d1d/d92 0 2024-04-02T04:42:59.707 INFO:tasks.workunit.client.1.smithi027.stdout:0/543: creat d4/d16/d71/d97/fb8 x:0 0 0 2024-04-02T04:42:59.710 INFO:tasks.workunit.client.1.smithi027.stdout:2/593: rename d1/db/d3b/d17/d29 to d1/db/d3b/d17/d68/dcd/dd5 0 2024-04-02T04:42:59.718 INFO:tasks.workunit.client.1.smithi027.stdout:6/505: unlink d13/d1b/d24/d28/d3c/f77 0 2024-04-02T04:42:59.724 INFO:tasks.workunit.client.1.smithi027.stdout:5/753: write d0/d1a/d20/f93 [4870794,20190] 0 2024-04-02T04:42:59.725 INFO:tasks.workunit.client.1.smithi027.stdout:7/583: mkdir d2/d7/da/dc/d16/d3f/d50/d8b/db6 0 2024-04-02T04:42:59.726 INFO:tasks.workunit.client.1.smithi027.stdout:0/544: mknod d4/d21/d85/cb9 0 2024-04-02T04:42:59.729 INFO:tasks.workunit.client.1.smithi027.stdout:2/594: creat d1/db/d3b/d17/d68/d69/fd6 x:0 0 0 2024-04-02T04:42:59.730 INFO:tasks.workunit.client.1.smithi027.stdout:9/556: mkdir d1a/d1d/db7 0 2024-04-02T04:42:59.734 INFO:tasks.workunit.client.1.smithi027.stdout:4/470: mknod d24/d38/d44/d82/d95/c9e 0 2024-04-02T04:42:59.740 INFO:tasks.workunit.client.1.smithi027.stdout:1/537: dwrite d0/d1e/d2e/d3e/d5f/f79 [0,4194304] 0 2024-04-02T04:42:59.741 INFO:tasks.workunit.client.1.smithi027.stdout:7/584: dread d2/d7/da/dc/d1f/d33/f71 [0,4194304] 0 2024-04-02T04:42:59.742 INFO:tasks.workunit.client.1.smithi027.stdout:0/545: creat d4/d16/d71/d97/fba x:0 0 0 2024-04-02T04:42:59.742 INFO:tasks.workunit.client.1.smithi027.stdout:7/585: chown d2/l6c 723075 1 2024-04-02T04:42:59.743 INFO:tasks.workunit.client.1.smithi027.stdout:8/582: truncate d5/d8/db/dd/f10 7494211 0 2024-04-02T04:42:59.744 INFO:tasks.workunit.client.1.smithi027.stdout:8/583: read - d5/d8/db/d57/da5/fa6 zero size 2024-04-02T04:42:59.745 INFO:tasks.workunit.client.1.smithi027.stdout:2/595: mknod d1/db/d3b/d17/d6d/cd7 0 2024-04-02T04:42:59.747 INFO:tasks.workunit.client.1.smithi027.stdout:6/506: creat d13/d1b/d24/d25/d56/d73/d96/fac x:0 0 0 2024-04-02T04:42:59.748 INFO:tasks.workunit.client.1.smithi027.stdout:2/596: write d1/db/d3b/d17/d68/f94 [833990,40732] 0 2024-04-02T04:42:59.749 INFO:tasks.workunit.client.1.smithi027.stdout:4/471: truncate d24/d38/d44/d41/f4b 945117 0 2024-04-02T04:42:59.751 INFO:tasks.workunit.client.1.smithi027.stdout:6/507: stat d13/d1b/d48/l63 0 2024-04-02T04:42:59.753 INFO:tasks.workunit.client.1.smithi027.stdout:1/538: chown d0/c46 4 1 2024-04-02T04:42:59.753 INFO:tasks.workunit.client.1.smithi027.stdout:8/584: mkdir d5/d8/db/dd/d5f/d91/dc2 0 2024-04-02T04:42:59.758 INFO:tasks.workunit.client.1.smithi027.stdout:5/754: dwrite d0/d1d/f91 [0,4194304] 0 2024-04-02T04:42:59.760 INFO:tasks.workunit.client.1.smithi027.stdout:9/557: dwrite d1a/db6/faf [0,4194304] 0 2024-04-02T04:42:59.761 INFO:tasks.workunit.client.1.smithi027.stdout:5/755: dread - d0/d1d/f100 zero size 2024-04-02T04:42:59.768 INFO:tasks.workunit.client.1.smithi027.stdout:6/508: dwrite d13/d1b/d24/d28/d3c/f78 [4194304,4194304] 0 2024-04-02T04:42:59.774 INFO:tasks.workunit.client.1.smithi027.stdout:4/472: write f12 [3001610,75002] 0 2024-04-02T04:42:59.776 INFO:tasks.workunit.client.1.smithi027.stdout:5/756: dwrite d0/d1a/d20/fde [0,4194304] 0 2024-04-02T04:42:59.782 INFO:tasks.workunit.client.1.smithi027.stdout:0/546: creat d4/d21/fbb x:0 0 0 2024-04-02T04:42:59.784 INFO:tasks.workunit.client.1.smithi027.stdout:7/586: truncate d2/d7/da/dc/d16/d3f/d50/d8b/f72 518031 0 2024-04-02T04:42:59.788 INFO:tasks.workunit.client.1.smithi027.stdout:1/539: symlink d0/d1e/lb1 0 2024-04-02T04:42:59.789 INFO:tasks.workunit.client.1.smithi027.stdout:8/585: symlink d5/d8/db/lc3 0 2024-04-02T04:42:59.790 INFO:tasks.workunit.client.1.smithi027.stdout:4/473: mkdir d24/d38/d44/d82/d97/d9f 0 2024-04-02T04:42:59.791 INFO:tasks.workunit.client.1.smithi027.stdout:5/757: mknod d0/df/d11/d65/dac/c106 0 2024-04-02T04:42:59.792 INFO:tasks.workunit.client.1.smithi027.stdout:5/758: chown d0/df/d11/d41/ff7 1 1 2024-04-02T04:42:59.793 INFO:tasks.workunit.client.1.smithi027.stdout:5/759: chown d0/df/d29/da5/da6/de9 153 1 2024-04-02T04:42:59.794 INFO:tasks.workunit.client.1.smithi027.stdout:0/547: rename d4/d38/d66/d2d/c7a to d4/d69/cbc 0 2024-04-02T04:42:59.796 INFO:tasks.workunit.client.1.smithi027.stdout:4/474: mknod d24/d48/ca0 0 2024-04-02T04:42:59.797 INFO:tasks.workunit.client.1.smithi027.stdout:5/760: rmdir d0/df/d29/da5/da6 39 2024-04-02T04:42:59.799 INFO:tasks.workunit.client.1.smithi027.stdout:1/540: mknod d0/cb2 0 2024-04-02T04:42:59.800 INFO:tasks.workunit.client.1.smithi027.stdout:7/587: rename d2/d7/da/dc/d1f/l26 to d2/d7/da/dc/d16/d3a/d4d/d6f/lb7 0 2024-04-02T04:42:59.802 INFO:tasks.workunit.client.1.smithi027.stdout:2/597: getdents d1/db/d3b/d17/d68 0 2024-04-02T04:42:59.805 INFO:tasks.workunit.client.1.smithi027.stdout:4/475: symlink d24/d38/d44/d41/la1 0 2024-04-02T04:42:59.805 INFO:tasks.workunit.client.1.smithi027.stdout:7/588: dread d2/df/f85 [0,4194304] 0 2024-04-02T04:42:59.808 INFO:tasks.workunit.client.1.smithi027.stdout:7/589: chown d2/d7/da/dc/d16/d3f/d44/ca3 2 1 2024-04-02T04:42:59.812 INFO:tasks.workunit.client.1.smithi027.stdout:8/586: dread d5/d8/db/d57/fa1 [0,4194304] 0 2024-04-02T04:42:59.817 INFO:tasks.workunit.client.1.smithi027.stdout:6/509: dread d13/d1b/d24/f3a [0,4194304] 0 2024-04-02T04:42:59.817 INFO:tasks.workunit.client.1.smithi027.stdout:9/558: dwrite d1a/d1d/d29/d37/d3a/f54 [8388608,4194304] 0 2024-04-02T04:42:59.818 INFO:tasks.workunit.client.1.smithi027.stdout:1/541: fsync d0/de/d52/f65 0 2024-04-02T04:42:59.822 INFO:tasks.workunit.client.1.smithi027.stdout:7/590: mknod d2/d7/da/dc/d16/d81/cb8 0 2024-04-02T04:42:59.832 INFO:tasks.workunit.client.1.smithi027.stdout:4/476: unlink d24/d2d/c85 0 2024-04-02T04:42:59.837 INFO:tasks.workunit.client.1.smithi027.stdout:0/548: rename d4/d38/d66/f2f to d4/d21/d28/d4e/fbd 0 2024-04-02T04:42:59.838 INFO:tasks.workunit.client.1.smithi027.stdout:1/542: dread d0/d1e/d5d/f81 [4194304,4194304] 0 2024-04-02T04:42:59.840 INFO:tasks.workunit.client.1.smithi027.stdout:1/543: dread - d0/d1e/d2e/d3b/d4f/fac zero size 2024-04-02T04:42:59.840 INFO:tasks.workunit.client.1.smithi027.stdout:7/591: mknod d2/d1d/d90/cb9 0 2024-04-02T04:42:59.841 INFO:tasks.workunit.client.1.smithi027.stdout:2/598: link d1/db/d3b/f37 d1/db/d3b/d17/d6d/d84/fd8 0 2024-04-02T04:42:59.843 INFO:tasks.workunit.client.1.smithi027.stdout:1/544: chown d0/d1e/d2e/d3e/d5f/c7d 89548 1 2024-04-02T04:42:59.843 INFO:tasks.workunit.client.1.smithi027.stdout:4/477: rmdir d24 39 2024-04-02T04:42:59.846 INFO:tasks.workunit.client.1.smithi027.stdout:8/587: dread d5/d8/d15/d71/d96/fc0 [0,4194304] 0 2024-04-02T04:42:59.846 INFO:tasks.workunit.client.1.smithi027.stdout:6/510: symlink d13/d1b/lad 0 2024-04-02T04:42:59.847 INFO:tasks.workunit.client.1.smithi027.stdout:9/559: dwrite d1a/d1d/f4b [0,4194304] 0 2024-04-02T04:42:59.849 INFO:tasks.workunit.client.1.smithi027.stdout:0/549: symlink d4/d21/d28/lbe 0 2024-04-02T04:42:59.855 INFO:tasks.workunit.client.1.smithi027.stdout:9/560: mkdir d1a/d1d/d3c/d6e/db8 0 2024-04-02T04:42:59.857 INFO:tasks.workunit.client.1.smithi027.stdout:8/588: rename d5/d8/d48 to d5/d8/db/dd/d5f/d27/d31/d5b/dc4 0 2024-04-02T04:42:59.860 INFO:tasks.workunit.client.1.smithi027.stdout:0/550: rmdir d4/d38/d66/d4a 39 2024-04-02T04:42:59.860 INFO:tasks.workunit.client.1.smithi027.stdout:2/599: dwrite d1/db/d3b/d17/d68/dcd/dd5/f5a [0,4194304] 0 2024-04-02T04:42:59.861 INFO:tasks.workunit.client.1.smithi027.stdout:1/545: dwrite d0/d1e/d5d/f81 [0,4194304] 0 2024-04-02T04:42:59.862 INFO:tasks.workunit.client.1.smithi027.stdout:0/551: fdatasync d4/d38/f91 0 2024-04-02T04:42:59.864 INFO:tasks.workunit.client.1.smithi027.stdout:0/552: chown d4/d38/d66/d2d/f3c 2301 1 2024-04-02T04:42:59.865 INFO:tasks.workunit.client.1.smithi027.stdout:4/478: unlink d24/d38/l5f 0 2024-04-02T04:42:59.868 INFO:tasks.workunit.client.1.smithi027.stdout:6/511: symlink d13/d1b/d48/d75/d9b/lae 0 2024-04-02T04:42:59.872 INFO:tasks.workunit.client.1.smithi027.stdout:8/589: creat d5/d8/db/d68/d95/fc5 x:0 0 0 2024-04-02T04:42:59.873 INFO:tasks.workunit.client.1.smithi027.stdout:8/590: chown d5/d8/db/dd/d5f/f2b 48002275 1 2024-04-02T04:42:59.874 INFO:tasks.workunit.client.1.smithi027.stdout:8/591: chown d5/d8/d15/f37 70824 1 2024-04-02T04:42:59.875 INFO:tasks.workunit.client.1.smithi027.stdout:1/546: write d0/de/d52/f65 [821440,41435] 0 2024-04-02T04:42:59.879 INFO:tasks.workunit.client.1.smithi027.stdout:9/561: creat d1a/d1d/d3c/d6e/db8/fb9 x:0 0 0 2024-04-02T04:42:59.886 INFO:tasks.workunit.client.1.smithi027.stdout:4/479: creat d24/d38/d44/d60/fa2 x:0 0 0 2024-04-02T04:42:59.887 INFO:tasks.workunit.client.1.smithi027.stdout:4/480: dread - d24/d38/d44/d60/fa2 zero size 2024-04-02T04:42:59.888 INFO:tasks.workunit.client.1.smithi027.stdout:4/481: chown d24/d38/d44/d60/f94 48984275 1 2024-04-02T04:42:59.890 INFO:tasks.workunit.client.1.smithi027.stdout:7/592: getdents d2/d7/da/dc 0 2024-04-02T04:42:59.895 INFO:tasks.workunit.client.1.smithi027.stdout:0/553: rename d4/d21/d28/d4e/d5f/l9f to d4/d38/d66/d2d/d81/lbf 0 2024-04-02T04:42:59.896 INFO:tasks.workunit.client.1.smithi027.stdout:2/600: dread d1/db/d3b/d17/d12/f7e [0,4194304] 0 2024-04-02T04:42:59.897 INFO:tasks.workunit.client.1.smithi027.stdout:1/547: mknod d0/d1e/d2e/d3b/d4f/d9b/cb3 0 2024-04-02T04:42:59.897 INFO:tasks.workunit.client.1.smithi027.stdout:0/554: truncate d4/d38/d66/f2a 4957513 0 2024-04-02T04:42:59.898 INFO:tasks.workunit.client.1.smithi027.stdout:2/601: write d1/d91/f9c [107207,120891] 0 2024-04-02T04:42:59.901 INFO:tasks.workunit.client.1.smithi027.stdout:8/592: getdents d5/d8/db/dd/d5f/d27/d31/db9/dc1 0 2024-04-02T04:42:59.902 INFO:tasks.workunit.client.1.smithi027.stdout:9/562: creat d1a/d1d/d29/d50/da2/fba x:0 0 0 2024-04-02T04:42:59.903 INFO:tasks.workunit.client.1.smithi027.stdout:9/563: chown d1a 45 1 2024-04-02T04:42:59.903 INFO:tasks.workunit.client.1.smithi027.stdout:9/564: readlink d1a/l81 0 2024-04-02T04:42:59.905 INFO:tasks.workunit.client.1.smithi027.stdout:7/593: write d2/d7/da/dc/d1f/f84 [136711,85935] 0 2024-04-02T04:42:59.908 INFO:tasks.workunit.client.1.smithi027.stdout:6/512: dwrite d13/d1b/d24/f4a [0,4194304] 0 2024-04-02T04:42:59.908 INFO:tasks.workunit.client.1.smithi027.stdout:2/602: rename d1/db/d3b/d17/d68/dcd to d1/db/d3b/d17/d6d/dd9 0 2024-04-02T04:42:59.912 INFO:tasks.workunit.client.1.smithi027.stdout:9/565: creat d1a/d1d/d29/d50/da2/fbb x:0 0 0 2024-04-02T04:42:59.912 INFO:tasks.workunit.client.1.smithi027.stdout:9/566: dread - d1a/db6/fa8 zero size 2024-04-02T04:42:59.914 INFO:tasks.workunit.client.1.smithi027.stdout:8/593: fsync d5/d8/db/dd/f3b 0 2024-04-02T04:42:59.916 INFO:tasks.workunit.client.1.smithi027.stdout:6/513: unlink d13/d1b/d24/d28/d3c/l52 0 2024-04-02T04:42:59.919 INFO:tasks.workunit.client.1.smithi027.stdout:1/548: link d0/d1e/c24 d0/de/cb4 0 2024-04-02T04:42:59.924 INFO:tasks.workunit.client.1.smithi027.stdout:9/567: chown d1a/d1d/d29/d37/cb0 893891145 1 2024-04-02T04:42:59.924 INFO:tasks.workunit.client.1.smithi027.stdout:6/514: dwrite d13/d1b/d24/d25/d56/fa0 [0,4194304] 0 2024-04-02T04:42:59.926 INFO:tasks.workunit.client.1.smithi027.stdout:9/568: write d1a/d1d/d29/d50/d73/fb1 [972534,74315] 0 2024-04-02T04:42:59.927 INFO:tasks.workunit.client.1.smithi027.stdout:9/569: chown d1a/d1d/fa5 323314668 1 2024-04-02T04:42:59.932 INFO:tasks.workunit.client.1.smithi027.stdout:1/549: creat d0/de/d52/d74/fb5 x:0 0 0 2024-04-02T04:42:59.937 INFO:tasks.workunit.client.1.smithi027.stdout:7/594: rename d2/d7/da to d2/dba 0 2024-04-02T04:42:59.939 INFO:tasks.workunit.client.1.smithi027.stdout:7/595: chown d2/df/f85 3130915 1 2024-04-02T04:42:59.941 INFO:tasks.workunit.client.1.smithi027.stdout:6/515: write d13/d1b/d24/d25/d42/f60 [1051363,12979] 0 2024-04-02T04:42:59.944 INFO:tasks.workunit.client.1.smithi027.stdout:6/516: fdatasync d13/d1b/d24/d28/d3c/f92 0 2024-04-02T04:42:59.944 INFO:tasks.workunit.client.1.smithi027.stdout:8/594: dwrite d5/d8/db/df/f22 [0,4194304] 0 2024-04-02T04:42:59.945 INFO:tasks.workunit.client.1.smithi027.stdout:9/570: unlink d1a/d1d/d29/d37/d20/d7e/cab 0 2024-04-02T04:42:59.946 INFO:tasks.workunit.client.1.smithi027.stdout:9/571: readlink d1a/d1d/d29/l9e 0 2024-04-02T04:42:59.952 INFO:tasks.workunit.client.1.smithi027.stdout:2/603: rename d1/db/d3b/d61/d6c to d1/db/d3b/d17/d6d/dd9/dd5/dda 0 2024-04-02T04:42:59.956 INFO:tasks.workunit.client.1.smithi027.stdout:1/550: dwrite d0/d1e/d25/d2b/fae [0,4194304] 0 2024-04-02T04:42:59.958 INFO:tasks.workunit.client.1.smithi027.stdout:7/596: dread d2/dba/dc/d1f/f84 [0,4194304] 0 2024-04-02T04:42:59.965 INFO:tasks.workunit.client.1.smithi027.stdout:1/551: creat d0/d1e/d2e/d3e/d5f/fb6 x:0 0 0 2024-04-02T04:42:59.967 INFO:tasks.workunit.client.1.smithi027.stdout:1/552: write d0/d1e/d5d/d60/f8f [753363,47242] 0 2024-04-02T04:42:59.971 INFO:tasks.workunit.client.1.smithi027.stdout:8/595: rmdir d5/d8/db/dd/d5f/d91/dba 0 2024-04-02T04:42:59.974 INFO:tasks.workunit.client.1.smithi027.stdout:1/553: dwrite d0/f3d [0,4194304] 0 2024-04-02T04:42:59.976 INFO:tasks.workunit.client.1.smithi027.stdout:1/554: chown d0/d1e/d5d/d60/f8f 82231 1 2024-04-02T04:42:59.977 INFO:tasks.workunit.client.1.smithi027.stdout:6/517: dread d13/d1b/d48/d58/f64 [0,4194304] 0 2024-04-02T04:42:59.977 INFO:tasks.workunit.client.1.smithi027.stdout:9/572: creat d1a/d1d/d3c/fbc x:0 0 0 2024-04-02T04:42:59.979 INFO:tasks.workunit.client.1.smithi027.stdout:7/597: symlink d2/dba/dc/d16/d3f/d50/d8b/db6/lbb 0 2024-04-02T04:42:59.981 INFO:tasks.workunit.client.1.smithi027.stdout:7/598: readlink d2/dba/dc/d1f/d61/lb0 0 2024-04-02T04:42:59.983 INFO:tasks.workunit.client.1.smithi027.stdout:8/596: truncate d5/d8/db/f29 135598 0 2024-04-02T04:42:59.986 INFO:tasks.workunit.client.1.smithi027.stdout:9/573: mkdir d1a/d1d/d29/d50/d73/dbd 0 2024-04-02T04:42:59.987 INFO:tasks.workunit.client.1.smithi027.stdout:6/518: mknod d13/d1b/d24/d25/d42/caf 0 2024-04-02T04:42:59.987 INFO:tasks.workunit.client.1.smithi027.stdout:2/604: dwrite d1/db/d3b/d17/fc [12582912,4194304] 0 2024-04-02T04:42:59.988 INFO:tasks.workunit.client.1.smithi027.stdout:6/519: stat d13/f62 0 2024-04-02T04:42:59.989 INFO:tasks.workunit.client.1.smithi027.stdout:6/520: stat d13/d1b/d48/d75 0 2024-04-02T04:42:59.991 INFO:tasks.workunit.client.1.smithi027.stdout:7/599: mknod d2/dba/dc/d1f/d33/d36/cbc 0 2024-04-02T04:42:59.991 INFO:tasks.workunit.client.1.smithi027.stdout:8/597: truncate d5/d8/db/df/f79 645887 0 2024-04-02T04:42:59.992 INFO:tasks.workunit.client.1.smithi027.stdout:6/521: read d13/d1b/d24/d25/d27/f76 [657740,128185] 0 2024-04-02T04:42:59.994 INFO:tasks.workunit.client.1.smithi027.stdout:6/522: dread - d13/d1b/d24/f84 zero size 2024-04-02T04:43:00.002 INFO:tasks.workunit.client.1.smithi027.stdout:9/574: dwrite d1a/d1d/d29/d37/d20/f3f [4194304,4194304] 0 2024-04-02T04:43:00.003 INFO:tasks.workunit.client.1.smithi027.stdout:1/555: getdents d0/d1e/d25 0 2024-04-02T04:43:00.003 INFO:tasks.workunit.client.1.smithi027.stdout:8/598: dwrite d5/d8/db/dd/d5f/d27/d31/d5b/f8f [0,4194304] 0 2024-04-02T04:43:00.010 INFO:tasks.workunit.client.1.smithi027.stdout:7/600: dread d2/dba/d35/f8c [0,4194304] 0 2024-04-02T04:43:00.011 INFO:tasks.workunit.client.1.smithi027.stdout:1/556: mkdir d0/de/d52/d74/db7 0 2024-04-02T04:43:00.013 INFO:tasks.workunit.client.1.smithi027.stdout:1/557: readlink d0/de/d52/d74/l90 0 2024-04-02T04:43:00.022 INFO:tasks.workunit.client.1.smithi027.stdout:2/605: rmdir d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/dd4 0 2024-04-02T04:43:00.022 INFO:tasks.workunit.client.1.smithi027.stdout:2/606: fdatasync d1/db/f32 0 2024-04-02T04:43:00.024 INFO:tasks.workunit.client.1.smithi027.stdout:2/607: dread - d1/db/d3b/d17/d6d/dd9/dd5/f4f zero size 2024-04-02T04:43:00.025 INFO:tasks.workunit.client.1.smithi027.stdout:8/599: mkdir d5/d8/db/dd/d5f/db2/dc6 0 2024-04-02T04:43:00.035 INFO:tasks.workunit.client.1.smithi027.stdout:9/575: truncate d1a/d1d/d29/f4f 3917804 0 2024-04-02T04:43:00.036 INFO:tasks.workunit.client.1.smithi027.stdout:9/576: fdatasync d1a/d1d/d3c/fbc 0 2024-04-02T04:43:00.036 INFO:tasks.workunit.client.1.smithi027.stdout:2/608: mknod d1/db/d3b/d17/d2b/db8/cdb 0 2024-04-02T04:43:00.038 INFO:tasks.workunit.client.1.smithi027.stdout:9/577: write d1a/d1d/d3c/d6e/fb3 [816946,128898] 0 2024-04-02T04:43:00.039 INFO:tasks.workunit.client.1.smithi027.stdout:9/578: stat d1a/d1d/d3c/d6e/fb3 0 2024-04-02T04:43:00.039 INFO:tasks.workunit.client.1.smithi027.stdout:8/600: rmdir d5/d8 39 2024-04-02T04:43:00.040 INFO:tasks.workunit.client.1.smithi027.stdout:9/579: readlink d1a/d1d/l45 0 2024-04-02T04:43:00.043 INFO:tasks.workunit.client.1.smithi027.stdout:7/601: creat d2/dba/dc/d16/fbd x:0 0 0 2024-04-02T04:43:00.044 INFO:tasks.workunit.client.1.smithi027.stdout:7/602: fsync d2/dba/d35/f67 0 2024-04-02T04:43:00.046 INFO:tasks.workunit.client.1.smithi027.stdout:2/609: unlink d1/db/d3b/d17/d68/cab 0 2024-04-02T04:43:00.047 INFO:tasks.workunit.client.1.smithi027.stdout:2/610: write d1/d91/f9c [866408,27288] 0 2024-04-02T04:43:00.053 INFO:tasks.workunit.client.1.smithi027.stdout:9/580: rename d1a/d1d/fa5 to d1a/d1d/d29/d37/d99/fbe 0 2024-04-02T04:43:00.054 INFO:tasks.workunit.client.1.smithi027.stdout:9/581: dread - d1a/d1d/d3c/fbc zero size 2024-04-02T04:43:00.058 INFO:tasks.workunit.client.1.smithi027.stdout:7/603: mknod d2/dba/dc/d16/d3f/cbe 0 2024-04-02T04:43:00.060 INFO:tasks.workunit.client.1.smithi027.stdout:8/601: write d5/d8/db/dd/d5f/d27/d31/f5e [4869025,23188] 0 2024-04-02T04:43:00.062 INFO:tasks.workunit.client.1.smithi027.stdout:2/611: rename d1/db/d3b/d17/d6d/dd9/dd5/dda/dbb/db4/cbf to d1/db/d3b/d17/d68/d69/cdc 0 2024-04-02T04:43:00.064 INFO:tasks.workunit.client.1.smithi027.stdout:7/604: mknod d2/d1c/cbf 0 2024-04-02T04:43:00.065 INFO:tasks.workunit.client.1.smithi027.stdout:7/605: stat d2/d1c/f4b 0 2024-04-02T04:43:00.065 INFO:tasks.workunit.client.1.smithi027.stdout:7/606: stat d2/dba/dc/f30 0 2024-04-02T04:43:00.066 INFO:tasks.workunit.client.1.smithi027.stdout:8/602: mknod d5/d8/db/dd/d5f/d91/d9f/cc7 0 2024-04-02T04:43:00.067 INFO:tasks.workunit.client.1.smithi027.stdout:9/582: rename d1a/db6/fa8 to d1a/d1d/d29/d50/d73/dbd/fbf 0 2024-04-02T04:43:00.068 INFO:tasks.workunit.client.1.smithi027.stdout:2/612: chown d1/db/d3b/d17/d12/l44 779 1 2024-04-02T04:43:00.069 INFO:tasks.workunit.client.1.smithi027.stdout:7/607: mkdir d2/dba/dc/d16/d3f/dc0 0 2024-04-02T04:43:00.070 INFO:tasks.workunit.client.1.smithi027.stdout:2/613: fdatasync d1/db/d3b/d17/f14 0 2024-04-02T04:43:00.075 INFO:tasks.workunit.client.1.smithi027.stdout:2/614: creat d1/db/d3b/d61/fdd x:0 0 0 2024-04-02T04:43:00.077 INFO:tasks.workunit.client.1.smithi027.stdout:7/608: truncate d2/d1d/f25 737656 0 2024-04-02T04:43:00.079 INFO:tasks.workunit.client.1.smithi027.stdout:8/603: mknod d5/d8/db/cc8 0 2024-04-02T04:43:00.082 INFO:tasks.workunit.client.1.smithi027.stdout:2/615: dwrite d1/db/f32 [4194304,4194304] 0 2024-04-02T04:43:00.083 INFO:tasks.workunit.client.1.smithi027.stdout:2/616: chown d1/c2a 45 1 2024-04-02T04:43:00.085 INFO:tasks.workunit.client.1.smithi027.stdout:2/617: write d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/fd1 [722085,44210] 0 2024-04-02T04:43:00.092 INFO:tasks.workunit.client.1.smithi027.stdout:2/618: symlink d1/db/d3b/d17/d6d/dd9/dd5/lde 0 2024-04-02T04:43:00.092 INFO:tasks.workunit.client.1.smithi027.stdout:8/604: dread d5/d8/db/dd/d5f/d27/d31/d5b/d6c/f7b [4194304,4194304] 0 2024-04-02T04:43:00.092 INFO:tasks.workunit.client.1.smithi027.stdout:9/583: write d1a/d1d/d29/d50/d73/f7a [1251615,54401] 0 2024-04-02T04:43:00.097 INFO:tasks.workunit.client.1.smithi027.stdout:2/619: mkdir d1/db/d3b/d17/d6d/dd9/dd5/dda/dbb/db4/ddf 0 2024-04-02T04:43:00.102 INFO:tasks.workunit.client.1.smithi027.stdout:2/620: rename d1/l6e to d1/db/le0 0 2024-04-02T04:43:00.108 INFO:tasks.workunit.client.1.smithi027.stdout:2/621: write d1/f4d [1058124,38343] 0 2024-04-02T04:43:00.112 INFO:tasks.workunit.client.1.smithi027.stdout:9/584: dread d1a/d1d/d29/d50/d73/f7b [0,4194304] 0 2024-04-02T04:43:00.114 INFO:tasks.workunit.client.1.smithi027.stdout:2/622: rename d1/db/d3b/d17/d68/d78 to d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/de1 0 2024-04-02T04:43:00.115 INFO:tasks.workunit.client.1.smithi027.stdout:2/623: stat d1/db/d3b/d17/d6d/dd9 0 2024-04-02T04:43:00.117 INFO:tasks.workunit.client.1.smithi027.stdout:9/585: mknod d1a/d1d/d29/d37/cc0 0 2024-04-02T04:43:00.118 INFO:tasks.workunit.client.1.smithi027.stdout:9/586: write d1a/db6/f96 [213400,29943] 0 2024-04-02T04:43:00.119 INFO:tasks.workunit.client.1.smithi027.stdout:9/587: write d1a/d1d/d3c/d6e/fb3 [2623,21223] 0 2024-04-02T04:43:00.121 INFO:tasks.workunit.client.1.smithi027.stdout:2/624: link d1/db/d3b/d17/cc4 d1/db/d3b/d17/d68/ce2 0 2024-04-02T04:43:00.123 INFO:tasks.workunit.client.1.smithi027.stdout:9/588: readlink d1a/d1d/d29/d37/l4a 0 2024-04-02T04:43:00.125 INFO:tasks.workunit.client.1.smithi027.stdout:9/589: mknod d1a/d1d/d3c/d6e/cc1 0 2024-04-02T04:43:00.130 INFO:tasks.workunit.client.1.smithi027.stdout:9/590: getdents d1a 0 2024-04-02T04:43:00.132 INFO:tasks.workunit.client.1.smithi027.stdout:9/591: mknod d1a/d1d/d29/d37/d20/dad/cc2 0 2024-04-02T04:43:00.134 INFO:tasks.workunit.client.1.smithi027.stdout:9/592: write d1a/d1d/d29/d37/d20/d7e/f87 [810165,100453] 0 2024-04-02T04:43:00.137 INFO:tasks.workunit.client.1.smithi027.stdout:2/625: truncate d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/fd1 662997 0 2024-04-02T04:43:00.139 INFO:tasks.workunit.client.1.smithi027.stdout:2/626: write d1/db/d3b/d17/d6d/dd9/dd5/fc1 [1728751,89760] 0 2024-04-02T04:43:00.142 INFO:tasks.workunit.client.1.smithi027.stdout:9/593: rename d1a/d1d/d29/d37/d4c/l78 to d1a/d1d/d29/d37/d20/lc3 0 2024-04-02T04:43:00.143 INFO:tasks.workunit.client.1.smithi027.stdout:9/594: write d1a/d1d/d3c/fb2 [64185,24422] 0 2024-04-02T04:43:00.143 INFO:tasks.workunit.client.1.smithi027.stdout:9/595: chown d1a/d1d 591370 1 2024-04-02T04:43:00.144 INFO:tasks.workunit.client.1.smithi027.stdout:2/627: creat d1/db/d57/fe3 x:0 0 0 2024-04-02T04:43:00.147 INFO:tasks.workunit.client.1.smithi027.stdout:9/596: mknod d1a/db6/d8c/cc4 0 2024-04-02T04:43:00.148 INFO:tasks.workunit.client.1.smithi027.stdout:9/597: fdatasync d1a/d1d/d29/d37/d20/f2b 0 2024-04-02T04:43:00.150 INFO:tasks.workunit.client.1.smithi027.stdout:2/628: rename d1/db/d3b/d17/d68/d69/fb3 to d1/db/d3b/d17/d6d/dd9/dd5/dda/dbb/fe4 0 2024-04-02T04:43:00.153 INFO:tasks.workunit.client.1.smithi027.stdout:2/629: fsync d1/db/d3b/d83/fb0 0 2024-04-02T04:43:00.155 INFO:tasks.workunit.client.1.smithi027.stdout:2/630: mkdir d1/db/de5 0 2024-04-02T04:43:00.156 INFO:tasks.workunit.client.1.smithi027.stdout:9/598: getdents d1a/db6 0 2024-04-02T04:43:00.208 INFO:tasks.workunit.client.1.smithi027.stdout:3/519: sync 2024-04-02T04:43:00.219 INFO:tasks.workunit.client.1.smithi027.stdout:9/599: dwrite d1a/d1d/d29/f4f [0,4194304] 0 2024-04-02T04:43:00.223 INFO:tasks.workunit.client.1.smithi027.stdout:3/520: dread d1/d1a/f1c [0,4194304] 0 2024-04-02T04:43:00.229 INFO:tasks.workunit.client.1.smithi027.stdout:3/521: dread d1/de/d55/f75 [0,4194304] 0 2024-04-02T04:43:00.230 INFO:tasks.workunit.client.1.smithi027.stdout:9/600: unlink d1a/d1d/d29/f79 0 2024-04-02T04:43:00.233 INFO:tasks.workunit.client.1.smithi027.stdout:9/601: creat d1a/d1d/d29/d37/d99/fc5 x:0 0 0 2024-04-02T04:43:00.234 INFO:tasks.workunit.client.1.smithi027.stdout:3/522: symlink d1/de/df/lae 0 2024-04-02T04:43:00.238 INFO:tasks.workunit.client.1.smithi027.stdout:3/523: symlink d1/de/laf 0 2024-04-02T04:43:00.241 INFO:tasks.workunit.client.1.smithi027.stdout:9/602: getdents d1a/d1d/d29/d51 0 2024-04-02T04:43:00.243 INFO:tasks.workunit.client.1.smithi027.stdout:9/603: read d1a/d1d/d3c/f94 [2858488,38861] 0 2024-04-02T04:43:00.244 INFO:tasks.workunit.client.1.smithi027.stdout:9/604: write d1a/d1d/d29/d50/d73/f7a [808956,83135] 0 2024-04-02T04:43:00.254 INFO:tasks.workunit.client.1.smithi027.stdout:3/524: link d1/l37 d1/de/d55/lb0 0 2024-04-02T04:43:00.258 INFO:tasks.workunit.client.1.smithi027.stdout:9/605: getdents d1a/d1d/d3c/d6e/db8 0 2024-04-02T04:43:00.263 INFO:tasks.workunit.client.1.smithi027.stdout:9/606: write d1a/d1d/d29/d37/d4c/f62 [427394,34436] 0 2024-04-02T04:43:00.268 INFO:tasks.workunit.client.1.smithi027.stdout:3/525: dread d1/de/df/d13/f34 [4194304,4194304] 0 2024-04-02T04:43:00.271 INFO:tasks.workunit.client.1.smithi027.stdout:3/526: dread - d1/f64 zero size 2024-04-02T04:43:00.273 INFO:tasks.workunit.client.1.smithi027.stdout:3/527: chown d1/de/df/f5d 11166 1 2024-04-02T04:43:00.273 INFO:tasks.workunit.client.1.smithi027.stdout:9/607: dwrite f15 [0,4194304] 0 2024-04-02T04:43:00.275 INFO:tasks.workunit.client.1.smithi027.stdout:9/608: write d1a/d1d/d29/d37/d20/f2b [587321,124312] 0 2024-04-02T04:43:00.275 INFO:tasks.workunit.client.1.smithi027.stdout:9/609: chown cb 1722479804 1 2024-04-02T04:43:00.280 INFO:tasks.workunit.client.1.smithi027.stdout:3/528: mkdir d1/de/df/d13/d73/db1 0 2024-04-02T04:43:00.281 INFO:tasks.workunit.client.1.smithi027.stdout:3/529: fsync d1/de/df/f67 0 2024-04-02T04:43:00.281 INFO:tasks.workunit.client.1.smithi027.stdout:9/610: rename d1a/d1d/d3c/f5e to d1a/d1d/d3c/d6e/fc6 0 2024-04-02T04:43:00.284 INFO:tasks.workunit.client.1.smithi027.stdout:3/530: read d1/de/f58 [4692984,41751] 0 2024-04-02T04:43:00.291 INFO:tasks.workunit.client.1.smithi027.stdout:9/611: dwrite d1a/d1d/d29/d50/d73/f7b [4194304,4194304] 0 2024-04-02T04:43:00.298 INFO:tasks.workunit.client.1.smithi027.stdout:9/612: dread d1a/d1d/d29/d51/f60 [0,4194304] 0 2024-04-02T04:43:00.300 INFO:tasks.workunit.client.1.smithi027.stdout:3/531: dread d1/d1a/f5a [0,4194304] 0 2024-04-02T04:43:00.300 INFO:tasks.workunit.client.1.smithi027.stdout:9/613: chown d1a/d1d/d29/d37/d20/l28 23741 1 2024-04-02T04:43:00.302 INFO:tasks.workunit.client.1.smithi027.stdout:9/614: rmdir d1a/d1d/d3c/d6e/db8 39 2024-04-02T04:43:00.305 INFO:tasks.workunit.client.1.smithi027.stdout:9/615: unlink d1a/d1d/d29/d37/cc0 0 2024-04-02T04:43:00.309 INFO:tasks.workunit.client.1.smithi027.stdout:9/616: mknod d1a/d1d/d29/cc7 0 2024-04-02T04:43:00.313 INFO:tasks.workunit.client.1.smithi027.stdout:9/617: dread f15 [4194304,4194304] 0 2024-04-02T04:43:00.316 INFO:tasks.workunit.client.1.smithi027.stdout:9/618: creat d1a/d1d/d29/d37/d3a/fc8 x:0 0 0 2024-04-02T04:43:00.320 INFO:tasks.workunit.client.1.smithi027.stdout:9/619: mkdir d1a/d1d/d29/d37/d4c/dc9 0 2024-04-02T04:43:00.332 INFO:tasks.workunit.client.1.smithi027.stdout:9/620: dwrite d1a/d1d/d29/f46 [0,4194304] 0 2024-04-02T04:43:00.336 INFO:tasks.workunit.client.1.smithi027.stdout:9/621: creat d1a/db6/fca x:0 0 0 2024-04-02T04:43:00.355 INFO:tasks.workunit.client.1.smithi027.stdout:9/622: dread d1a/d1d/f53 [0,4194304] 0 2024-04-02T04:43:00.356 INFO:tasks.workunit.client.1.smithi027.stdout:9/623: fsync d1a/db6/f8f 0 2024-04-02T04:43:00.358 INFO:tasks.workunit.client.1.smithi027.stdout:9/624: rmdir d1a/d1d/d29/d37/d20 39 2024-04-02T04:43:00.364 INFO:tasks.workunit.client.1.smithi027.stdout:5/761: sync 2024-04-02T04:43:00.364 INFO:tasks.workunit.client.1.smithi027.stdout:4/482: sync 2024-04-02T04:43:00.364 INFO:tasks.workunit.client.1.smithi027.stdout:0/555: sync 2024-04-02T04:43:00.366 INFO:tasks.workunit.client.1.smithi027.stdout:4/483: write d24/d38/d47/d5c/f8c [733952,24834] 0 2024-04-02T04:43:00.369 INFO:tasks.workunit.client.1.smithi027.stdout:5/762: mkdir d0/df/d2f/d107 0 2024-04-02T04:43:00.372 INFO:tasks.workunit.client.1.smithi027.stdout:9/625: dread fa [0,4194304] 0 2024-04-02T04:43:00.376 INFO:tasks.workunit.client.1.smithi027.stdout:4/484: dwrite d24/d38/d47/d62/f9a [0,4194304] 0 2024-04-02T04:43:00.377 INFO:tasks.workunit.client.1.smithi027.stdout:9/626: read d1a/d1d/d3c/fb2 [19178,95831] 0 2024-04-02T04:43:00.379 INFO:tasks.workunit.client.1.smithi027.stdout:0/556: creat d4/d16/d1b/fc0 x:0 0 0 2024-04-02T04:43:00.381 INFO:tasks.workunit.client.1.smithi027.stdout:9/627: write d1a/f39 [7246284,25624] 0 2024-04-02T04:43:00.382 INFO:tasks.workunit.client.1.smithi027.stdout:5/763: dwrite d0/df/d11/d13/f3e [0,4194304] 0 2024-04-02T04:43:00.387 INFO:tasks.workunit.client.1.smithi027.stdout:0/557: mknod d4/d38/d66/d2d/d81/cc1 0 2024-04-02T04:43:00.387 INFO:tasks.workunit.client.1.smithi027.stdout:0/558: write d4/d3d/f47 [5018837,19747] 0 2024-04-02T04:43:00.388 INFO:tasks.workunit.client.1.smithi027.stdout:0/559: dread - d4/d21/d28/d4e/f55 zero size 2024-04-02T04:43:00.390 INFO:tasks.workunit.client.1.smithi027.stdout:5/764: rename d0/d1d/d61/d72/ff8 to d0/d1d/d61/d7e/dff/f108 0 2024-04-02T04:43:00.391 INFO:tasks.workunit.client.1.smithi027.stdout:4/485: creat d24/d38/d47/fa3 x:0 0 0 2024-04-02T04:43:00.392 INFO:tasks.workunit.client.1.smithi027.stdout:5/765: write d0/d1a/d20/fde [4487979,49799] 0 2024-04-02T04:43:00.395 INFO:tasks.workunit.client.1.smithi027.stdout:4/486: fdatasync f21 0 2024-04-02T04:43:00.397 INFO:tasks.workunit.client.1.smithi027.stdout:9/628: dread d1a/d1d/d29/d37/d20/f30 [0,4194304] 0 2024-04-02T04:43:00.401 INFO:tasks.workunit.client.1.smithi027.stdout:5/766: dread d0/df/d11/d13/d9a/d9c/dbe/fd2 [0,4194304] 0 2024-04-02T04:43:00.401 INFO:tasks.workunit.client.1.smithi027.stdout:4/487: mknod d24/d38/d44/d82/d95/ca4 0 2024-04-02T04:43:00.402 INFO:tasks.workunit.client.1.smithi027.stdout:9/629: rmdir d1a/d1d/d29/d37/d20/dad 39 2024-04-02T04:43:00.403 INFO:tasks.workunit.client.1.smithi027.stdout:0/560: dwrite d4/d16/d1b/f20 [4194304,4194304] 0 2024-04-02T04:43:00.405 INFO:tasks.workunit.client.1.smithi027.stdout:4/488: truncate d24/d38/d47/d5c/f8c 1267357 0 2024-04-02T04:43:00.406 INFO:tasks.workunit.client.1.smithi027.stdout:5/767: symlink d0/d68/d6b/d74/l109 0 2024-04-02T04:43:00.409 INFO:tasks.workunit.client.1.smithi027.stdout:9/630: dread d1a/d1d/d29/d50/d73/fb1 [0,4194304] 0 2024-04-02T04:43:00.410 INFO:tasks.workunit.client.1.smithi027.stdout:0/561: read d4/d21/f7d [537733,66155] 0 2024-04-02T04:43:00.411 INFO:tasks.workunit.client.1.smithi027.stdout:9/631: write d1a/d1d/d29/d37/d20/f3f [5820525,103022] 0 2024-04-02T04:43:00.413 INFO:tasks.workunit.client.1.smithi027.stdout:4/489: creat d24/d38/d44/fa5 x:0 0 0 2024-04-02T04:43:00.413 INFO:tasks.workunit.client.1.smithi027.stdout:5/768: truncate d0/d1d/f79 4320198 0 2024-04-02T04:43:00.414 INFO:tasks.workunit.client.1.smithi027.stdout:4/490: chown d24/d38/d44/d41 30 1 2024-04-02T04:43:00.416 INFO:tasks.workunit.client.1.smithi027.stdout:9/632: dread d1a/d1d/d29/d37/d20/d7e/f87 [0,4194304] 0 2024-04-02T04:43:00.418 INFO:tasks.workunit.client.1.smithi027.stdout:0/562: chown d4/d3d/l4b 201 1 2024-04-02T04:43:00.419 INFO:tasks.workunit.client.1.smithi027.stdout:5/769: rmdir d0/df/d11 39 2024-04-02T04:43:00.422 INFO:tasks.workunit.client.1.smithi027.stdout:4/491: write d24/f2b [8204695,127690] 0 2024-04-02T04:43:00.424 INFO:tasks.workunit.client.1.smithi027.stdout:5/770: stat d0/df/d11/d8c/dbf 0 2024-04-02T04:43:00.424 INFO:tasks.workunit.client.1.smithi027.stdout:9/633: dread d1a/d1d/f72 [0,4194304] 0 2024-04-02T04:43:00.426 INFO:tasks.workunit.client.1.smithi027.stdout:9/634: write d1a/db6/faf [2585879,78386] 0 2024-04-02T04:43:00.426 INFO:tasks.workunit.client.1.smithi027.stdout:5/771: rmdir d0/df/d11/d65 39 2024-04-02T04:43:00.429 INFO:tasks.workunit.client.1.smithi027.stdout:0/563: creat d4/d16/d1b/fc2 x:0 0 0 2024-04-02T04:43:00.438 INFO:tasks.workunit.client.1.smithi027.stdout:5/772: creat d0/df/d11/d13/d9a/d9f/f10a x:0 0 0 2024-04-02T04:43:00.441 INFO:tasks.workunit.client.1.smithi027.stdout:5/773: stat d0/df/d11/d65/dac/c106 0 2024-04-02T04:43:00.443 INFO:tasks.workunit.client.1.smithi027.stdout:9/635: link d1a/d1d/d29/d37/d20/dad/cc2 d1a/d1d/d29/d56/d8b/ccb 0 2024-04-02T04:43:00.448 INFO:tasks.workunit.client.1.smithi027.stdout:5/774: dwrite d0/df/d11/d13/d9a/db7/fdc [0,4194304] 0 2024-04-02T04:43:00.451 INFO:tasks.workunit.client.1.smithi027.stdout:9/636: mkdir d1a/d1d/d29/d37/d20/dcc 0 2024-04-02T04:43:00.452 INFO:tasks.workunit.client.1.smithi027.stdout:9/637: chown d1a/db6/d8c/cc4 39269082 1 2024-04-02T04:43:00.454 INFO:tasks.workunit.client.1.smithi027.stdout:0/564: write d4/d21/d28/f40 [1002022,48817] 0 2024-04-02T04:43:00.458 INFO:tasks.workunit.client.1.smithi027.stdout:9/638: dread d1a/d1d/f53 [0,4194304] 0 2024-04-02T04:43:00.458 INFO:tasks.workunit.client.1.smithi027.stdout:0/565: read d4/d38/d66/d2d/d4c/f4d [2636685,65944] 0 2024-04-02T04:43:00.462 INFO:tasks.workunit.client.1.smithi027.stdout:0/566: creat d4/d38/fc3 x:0 0 0 2024-04-02T04:43:00.467 INFO:tasks.workunit.client.1.smithi027.stdout:9/639: dread d1a/d1d/d29/d50/d73/f7b [0,4194304] 0 2024-04-02T04:43:00.467 INFO:tasks.workunit.client.1.smithi027.stdout:0/567: mkdir d4/d76/dc4 0 2024-04-02T04:43:00.467 INFO:tasks.workunit.client.1.smithi027.stdout:5/775: write d0/df/d2f/d3b/f5b [1021234,42621] 0 2024-04-02T04:43:00.467 INFO:tasks.workunit.client.1.smithi027.stdout:5/776: write d0/d68/d6b/d74/fb1 [2285230,92095] 0 2024-04-02T04:43:00.469 INFO:tasks.workunit.client.1.smithi027.stdout:9/640: symlink d1a/d1d/d29/d50/d73/lcd 0 2024-04-02T04:43:00.470 INFO:tasks.workunit.client.1.smithi027.stdout:5/777: read d0/df/d2f/d3b/f3d [3740478,110004] 0 2024-04-02T04:43:00.470 INFO:tasks.workunit.client.1.smithi027.stdout:5/778: readlink d0/l1 0 2024-04-02T04:43:00.471 INFO:tasks.workunit.client.1.smithi027.stdout:5/779: write d0/df/d29/f7a [5920998,56655] 0 2024-04-02T04:43:00.473 INFO:tasks.workunit.client.1.smithi027.stdout:9/641: write d1a/d1d/d29/d50/d73/dbd/fbf [571950,9097] 0 2024-04-02T04:43:00.480 INFO:tasks.workunit.client.1.smithi027.stdout:0/568: creat d4/d21/fc5 x:0 0 0 2024-04-02T04:43:00.485 INFO:tasks.workunit.client.1.smithi027.stdout:5/780: write d0/df/d29/f84 [705346,92963] 0 2024-04-02T04:43:00.486 INFO:tasks.workunit.client.1.smithi027.stdout:0/569: mkdir d4/d21/d28/d4e/db5/dc6 0 2024-04-02T04:43:00.488 INFO:tasks.workunit.client.1.smithi027.stdout:9/642: getdents d1a/d1d/d29/d37/d9b 0 2024-04-02T04:43:00.490 INFO:tasks.workunit.client.1.smithi027.stdout:9/643: write d1a/d1d/d29/d37/d20/f2b [2165647,44656] 0 2024-04-02T04:43:00.494 INFO:tasks.workunit.client.1.smithi027.stdout:9/644: mkdir d1a/dce 0 2024-04-02T04:43:00.497 INFO:tasks.workunit.client.1.smithi027.stdout:0/570: dwrite d4/d38/d66/d4a/f42 [0,4194304] 0 2024-04-02T04:43:00.499 INFO:tasks.workunit.client.1.smithi027.stdout:0/571: chown d4/d21/c25 485 1 2024-04-02T04:43:00.501 INFO:tasks.workunit.client.1.smithi027.stdout:9/645: dread d1a/d1d/d29/d37/d3a/f84 [0,4194304] 0 2024-04-02T04:43:00.501 INFO:tasks.workunit.client.1.smithi027.stdout:5/781: dread d0/df/d11/d41/f90 [0,4194304] 0 2024-04-02T04:43:00.502 INFO:tasks.workunit.client.1.smithi027.stdout:0/572: write d4/d16/f27 [572413,18859] 0 2024-04-02T04:43:00.503 INFO:tasks.workunit.client.1.smithi027.stdout:9/646: chown d1a/d1d/d29/d37/d20/d7e/f87 29278 1 2024-04-02T04:43:00.503 INFO:tasks.workunit.client.1.smithi027.stdout:0/573: chown d4/d38/d66/l26 0 1 2024-04-02T04:43:00.504 INFO:tasks.workunit.client.1.smithi027.stdout:5/782: write d0/df/d11/d13/d9a/db7/fdc [4473986,5987] 0 2024-04-02T04:43:00.506 INFO:tasks.workunit.client.1.smithi027.stdout:9/647: truncate d1a/d1d/d29/d56/f88 4589048 0 2024-04-02T04:43:00.508 INFO:tasks.workunit.client.1.smithi027.stdout:0/574: dread d4/d21/d28/f54 [0,4194304] 0 2024-04-02T04:43:00.510 INFO:tasks.workunit.client.1.smithi027.stdout:5/783: mkdir d0/d68/d6b/d10b 0 2024-04-02T04:43:00.510 INFO:tasks.workunit.client.1.smithi027.stdout:0/575: truncate d4/d21/d28/f54 1173310 0 2024-04-02T04:43:00.512 INFO:tasks.workunit.client.1.smithi027.stdout:9/648: mkdir d1a/db6/d8c/dcf 0 2024-04-02T04:43:00.513 INFO:tasks.workunit.client.1.smithi027.stdout:9/649: write d1a/f38 [1265098,127105] 0 2024-04-02T04:43:00.524 INFO:tasks.workunit.client.1.smithi027.stdout:5/784: creat d0/f10c x:0 0 0 2024-04-02T04:43:00.525 INFO:tasks.workunit.client.1.smithi027.stdout:0/576: creat d4/d16/d1b/fc7 x:0 0 0 2024-04-02T04:43:00.525 INFO:tasks.workunit.client.1.smithi027.stdout:9/650: dwrite d1a/d1d/d3c/d6e/f97 [0,4194304] 0 2024-04-02T04:43:00.528 INFO:tasks.workunit.client.1.smithi027.stdout:0/577: fsync d4/d16/d1b/f65 0 2024-04-02T04:43:00.532 INFO:tasks.workunit.client.1.smithi027.stdout:9/651: symlink d1a/db6/d8c/dcf/ld0 0 2024-04-02T04:43:00.534 INFO:tasks.workunit.client.1.smithi027.stdout:0/578: mknod d4/d3d/cc8 0 2024-04-02T04:43:00.547 INFO:tasks.workunit.client.1.smithi027.stdout:6/523: sync 2024-04-02T04:43:00.551 INFO:tasks.workunit.client.1.smithi027.stdout:6/524: rename d13/d1b/d24/d25/c6b to d13/d1b/d48/d75/cb0 0 2024-04-02T04:43:00.553 INFO:tasks.workunit.client.1.smithi027.stdout:5/785: write d0/d1d/f79 [4426794,42024] 0 2024-04-02T04:43:00.554 INFO:tasks.workunit.client.1.smithi027.stdout:5/786: fsync d0/d1d/f100 0 2024-04-02T04:43:00.556 INFO:tasks.workunit.client.1.smithi027.stdout:5/787: fdatasync d0/df/d11/d65/dac/fa1 0 2024-04-02T04:43:00.559 INFO:tasks.workunit.client.1.smithi027.stdout:9/652: write d1a/d1d/d29/d50/f77 [1078746,63040] 0 2024-04-02T04:43:00.561 INFO:tasks.workunit.client.1.smithi027.stdout:5/788: fdatasync d0/fc5 0 2024-04-02T04:43:00.567 INFO:tasks.workunit.client.1.smithi027.stdout:6/525: dread d13/d8d/fa3 [0,4194304] 0 2024-04-02T04:43:00.568 INFO:tasks.workunit.client.1.smithi027.stdout:5/789: truncate d0/df/d11/d13/dce/d55/dc3/df2/ff9 28589 0 2024-04-02T04:43:00.568 INFO:tasks.workunit.client.1.smithi027.stdout:0/579: dwrite d4/d21/fb2 [0,4194304] 0 2024-04-02T04:43:00.569 INFO:tasks.workunit.client.1.smithi027.stdout:0/580: stat d4/d21/c25 0 2024-04-02T04:43:00.576 INFO:tasks.workunit.client.1.smithi027.stdout:6/526: dwrite d13/d1b/d24/d25/d27/f81 [0,4194304] 0 2024-04-02T04:43:00.584 INFO:tasks.workunit.client.1.smithi027.stdout:6/527: mknod d13/d1b/d24/d28/d46/cb1 0 2024-04-02T04:43:00.587 INFO:tasks.workunit.client.1.smithi027.stdout:0/581: mkdir d4/d21/d28/d4e/db5/dc6/dc9 0 2024-04-02T04:43:00.591 INFO:tasks.workunit.client.1.smithi027.stdout:6/528: truncate d13/f16 4465473 0 2024-04-02T04:43:00.592 INFO:tasks.workunit.client.1.smithi027.stdout:6/529: chown d13/d1b/d48/d58/f64 48863 1 2024-04-02T04:43:00.595 INFO:tasks.workunit.client.1.smithi027.stdout:9/653: truncate d1a/d1d/d29/d37/d3a/f54 11149806 0 2024-04-02T04:43:00.597 INFO:tasks.workunit.client.1.smithi027.stdout:6/530: creat d13/d1b/d24/d25/d56/d73/d96/fb2 x:0 0 0 2024-04-02T04:43:00.598 INFO:tasks.workunit.client.1.smithi027.stdout:6/531: readlink d13/l15 0 2024-04-02T04:43:00.601 INFO:tasks.workunit.client.1.smithi027.stdout:0/582: dread d4/d76/fa8 [0,4194304] 0 2024-04-02T04:43:00.602 INFO:tasks.workunit.client.1.smithi027.stdout:6/532: symlink d13/d1b/d24/d28/d46/lb3 0 2024-04-02T04:43:00.608 INFO:tasks.workunit.client.1.smithi027.stdout:5/790: dwrite d0/df/d2f/d3b/f3d [4194304,4194304] 0 2024-04-02T04:43:00.609 INFO:tasks.workunit.client.1.smithi027.stdout:0/583: mknod d4/d16/d1b/cca 0 2024-04-02T04:43:00.610 INFO:tasks.workunit.client.1.smithi027.stdout:0/584: chown d4/d16/d71/d97 127 1 2024-04-02T04:43:00.613 INFO:tasks.workunit.client.1.smithi027.stdout:6/533: symlink d13/d1b/lb4 0 2024-04-02T04:43:00.621 INFO:tasks.workunit.client.1.smithi027.stdout:0/585: fsync d4/d38/d66/d2d/d4c/f4d 0 2024-04-02T04:43:00.622 INFO:tasks.workunit.client.1.smithi027.stdout:0/586: read - d4/d16/d1b/fc0 zero size 2024-04-02T04:43:00.623 INFO:tasks.workunit.client.1.smithi027.stdout:0/587: fdatasync d4/d16/f89 0 2024-04-02T04:43:00.625 INFO:tasks.workunit.client.1.smithi027.stdout:9/654: write d1a/f26 [2013561,90056] 0 2024-04-02T04:43:00.626 INFO:tasks.workunit.client.1.smithi027.stdout:9/655: readlink d1a/d1d/d29/l36 0 2024-04-02T04:43:00.634 INFO:tasks.workunit.client.1.smithi027.stdout:6/534: dread d13/d1b/d24/d25/d27/f3f [0,4194304] 0 2024-04-02T04:43:00.635 INFO:tasks.workunit.client.1.smithi027.stdout:6/535: write d13/d1b/d24/d28/f65 [335385,87668] 0 2024-04-02T04:43:00.648 INFO:tasks.workunit.client.1.smithi027.stdout:0/588: mknod d4/d21/d85/db3/ccb 0 2024-04-02T04:43:00.654 INFO:tasks.workunit.client.1.smithi027.stdout:6/536: rmdir d13/d8d 39 2024-04-02T04:43:00.655 INFO:tasks.workunit.client.1.smithi027.stdout:6/537: read - d13/d1b/d24/d25/d56/d73/d96/fac zero size 2024-04-02T04:43:00.659 INFO:tasks.workunit.client.1.smithi027.stdout:0/589: rmdir d4/d76 39 2024-04-02T04:43:00.660 INFO:tasks.workunit.client.1.smithi027.stdout:9/656: getdents d1a/d1d/d29/d51 0 2024-04-02T04:43:00.661 INFO:tasks.workunit.client.1.smithi027.stdout:9/657: write d1a/d1d/d29/d37/d99/fc5 [964007,111610] 0 2024-04-02T04:43:00.664 INFO:tasks.workunit.client.1.smithi027.stdout:0/590: unlink d4/f6 0 2024-04-02T04:43:00.664 INFO:tasks.workunit.client.1.smithi027.stdout:0/591: dread - d4/d21/d85/f94 zero size 2024-04-02T04:43:00.665 INFO:tasks.workunit.client.1.smithi027.stdout:5/791: truncate d0/d68/dd7/fe2 4085840 0 2024-04-02T04:43:00.666 INFO:tasks.workunit.client.1.smithi027.stdout:9/658: mkdir d1a/d1d/d29/d50/d73/dbd/dd1 0 2024-04-02T04:43:00.670 INFO:tasks.workunit.client.1.smithi027.stdout:0/592: creat d4/d38/fcc x:0 0 0 2024-04-02T04:43:00.672 INFO:tasks.workunit.client.1.smithi027.stdout:9/659: creat d1a/d1d/d29/d37/d3a/fd2 x:0 0 0 2024-04-02T04:43:00.672 INFO:tasks.workunit.client.1.smithi027.stdout:5/792: dread d0/df/d11/d37/f4e [0,4194304] 0 2024-04-02T04:43:00.675 INFO:tasks.workunit.client.1.smithi027.stdout:9/660: creat d1a/d1d/d29/d50/fd3 x:0 0 0 2024-04-02T04:43:00.683 INFO:tasks.workunit.client.1.smithi027.stdout:0/593: truncate d4/d38/d66/d4a/f70 488271 0 2024-04-02T04:43:00.684 INFO:tasks.workunit.client.1.smithi027.stdout:9/661: link d1a/d1d/d29/d37/l6f d1a/db6/d8c/dcf/ld4 0 2024-04-02T04:43:00.685 INFO:tasks.workunit.client.1.smithi027.stdout:0/594: rmdir d4/d16 39 2024-04-02T04:43:00.690 INFO:tasks.workunit.client.1.smithi027.stdout:9/662: mkdir d1a/d1d/d29/d37/d20/dcc/dd5 0 2024-04-02T04:43:00.692 INFO:tasks.workunit.client.1.smithi027.stdout:9/663: chown d1a/l81 9 1 2024-04-02T04:43:00.693 INFO:tasks.workunit.client.1.smithi027.stdout:0/595: dwrite d4/d3d/f47 [0,4194304] 0 2024-04-02T04:43:00.695 INFO:tasks.workunit.client.1.smithi027.stdout:5/793: dwrite d0/d1a/d20/f73 [0,4194304] 0 2024-04-02T04:43:00.710 INFO:tasks.workunit.client.1.smithi027.stdout:0/596: creat d4/d76/dc4/fcd x:0 0 0 2024-04-02T04:43:00.711 INFO:tasks.workunit.client.1.smithi027.stdout:9/664: truncate d1a/d1d/d29/d37/d99/fc5 741128 0 2024-04-02T04:43:00.712 INFO:tasks.workunit.client.1.smithi027.stdout:0/597: chown d4/d21/d28/d4e/f62 28 1 2024-04-02T04:43:00.713 INFO:tasks.workunit.client.1.smithi027.stdout:5/794: truncate d0/df/d11/d65/f99 3904761 0 2024-04-02T04:43:00.714 INFO:tasks.workunit.client.1.smithi027.stdout:9/665: mkdir d1a/dd6 0 2024-04-02T04:43:00.717 INFO:tasks.workunit.client.1.smithi027.stdout:9/666: chown d1a/d1d/d29/d37/d4c/c5f 2 1 2024-04-02T04:43:00.717 INFO:tasks.workunit.client.1.smithi027.stdout:0/598: chown d4/d16/d71/d97/fb8 0 1 2024-04-02T04:43:00.720 INFO:tasks.workunit.client.1.smithi027.stdout:5/795: rename d0/df/d29/dab/cd8 to d0/df/d11/d13/dce/c10d 0 2024-04-02T04:43:00.724 INFO:tasks.workunit.client.1.smithi027.stdout:0/599: dwrite d4/f23 [0,4194304] 0 2024-04-02T04:43:00.730 INFO:tasks.workunit.client.1.smithi027.stdout:9/667: getdents d1a/d1d/d29/d37/d3a 0 2024-04-02T04:43:00.734 INFO:tasks.workunit.client.1.smithi027.stdout:0/600: rename d4/d38/d66/d2d/d4c/f4d to d4/d16/d71/d97/fce 0 2024-04-02T04:43:00.735 INFO:tasks.workunit.client.1.smithi027.stdout:5/796: dwrite d0/d1a/d20/fb5 [0,4194304] 0 2024-04-02T04:43:00.737 INFO:tasks.workunit.client.1.smithi027.stdout:9/668: mknod d1a/d1d/d29/d37/d20/d7e/cd7 0 2024-04-02T04:43:00.740 INFO:tasks.workunit.client.1.smithi027.stdout:9/669: truncate d1a/d1d/d29/d56/d8b/f90 392447 0 2024-04-02T04:43:00.746 INFO:tasks.workunit.client.1.smithi027.stdout:0/601: link d4/d21/f8f d4/d38/d66/d2d/d4c/fcf 0 2024-04-02T04:43:00.750 INFO:tasks.workunit.client.1.smithi027.stdout:5/797: creat d0/df/d11/d13/dce/f10e x:0 0 0 2024-04-02T04:43:00.752 INFO:tasks.workunit.client.1.smithi027.stdout:5/798: read d0/d1d/d61/f62 [3216333,37939] 0 2024-04-02T04:43:00.753 INFO:tasks.workunit.client.1.smithi027.stdout:9/670: mkdir d1a/d1d/db7/dd8 0 2024-04-02T04:43:00.754 INFO:tasks.workunit.client.1.smithi027.stdout:0/602: write d4/d21/d28/f75 [5599195,63209] 0 2024-04-02T04:43:00.756 INFO:tasks.workunit.client.1.smithi027.stdout:5/799: getdents d0/df/d2f/d3b/dca/dea 0 2024-04-02T04:43:00.760 INFO:tasks.workunit.client.1.smithi027.stdout:0/603: rename d4/d21/d85/faf to d4/d21/d28/d4e/fd0 0 2024-04-02T04:43:00.767 INFO:tasks.workunit.client.1.smithi027.stdout:0/604: dwrite d4/d21/d28/d4e/d5f/f5a [4194304,4194304] 0 2024-04-02T04:43:00.771 INFO:tasks.workunit.client.1.smithi027.stdout:0/605: mkdir d4/d38/da0/dd1 0 2024-04-02T04:43:00.775 INFO:tasks.workunit.client.1.smithi027.stdout:1/558: sync 2024-04-02T04:43:00.775 INFO:tasks.workunit.client.1.smithi027.stdout:5/800: write d0/df/d2f/d3b/d51/f53 [784756,52123] 0 2024-04-02T04:43:00.778 INFO:tasks.workunit.client.1.smithi027.stdout:9/671: dwrite d1a/d1d/f24 [0,4194304] 0 2024-04-02T04:43:00.783 INFO:tasks.workunit.client.1.smithi027.stdout:9/672: dread d1a/d1d/d29/d37/d20/f3f [4194304,4194304] 0 2024-04-02T04:43:00.784 INFO:tasks.workunit.client.1.smithi027.stdout:9/673: dread - d1a/db6/f8f zero size 2024-04-02T04:43:00.786 INFO:tasks.workunit.client.1.smithi027.stdout:1/559: mkdir d0/d1e/d25/d3c/d64/da6/db8 0 2024-04-02T04:43:00.786 INFO:tasks.workunit.client.1.smithi027.stdout:9/674: dread - d1a/d1d/d29/d50/da2/fba zero size 2024-04-02T04:43:00.787 INFO:tasks.workunit.client.1.smithi027.stdout:0/606: rmdir d4/d21/d85 39 2024-04-02T04:43:00.787 INFO:tasks.workunit.client.1.smithi027.stdout:9/675: chown d1a/d1d/d29/d50/l55 67085 1 2024-04-02T04:43:00.788 INFO:tasks.workunit.client.1.smithi027.stdout:0/607: write d4/d38/da0/f9e [271138,69231] 0 2024-04-02T04:43:00.789 INFO:tasks.workunit.client.1.smithi027.stdout:0/608: fsync d4/d38/fb6 0 2024-04-02T04:43:00.790 INFO:tasks.workunit.client.1.smithi027.stdout:0/609: write d4/d38/fb6 [397536,81043] 0 2024-04-02T04:43:00.791 INFO:tasks.workunit.client.1.smithi027.stdout:1/560: symlink d0/d1e/d2e/d3e/lb9 0 2024-04-02T04:43:00.794 INFO:tasks.workunit.client.1.smithi027.stdout:9/676: read d1a/d1d/d29/d37/d99/fc5 [591726,11729] 0 2024-04-02T04:43:00.795 INFO:tasks.workunit.client.1.smithi027.stdout:0/610: fdatasync d4/d38/da0/f9e 0 2024-04-02T04:43:00.795 INFO:tasks.workunit.client.1.smithi027.stdout:5/801: write d0/f6 [1944377,85665] 0 2024-04-02T04:43:00.799 INFO:tasks.workunit.client.1.smithi027.stdout:1/561: creat d0/d1e/d25/d3c/d64/da6/db8/fba x:0 0 0 2024-04-02T04:43:00.805 INFO:tasks.workunit.client.1.smithi027.stdout:0/611: rename d4/d21/d85/f94 to d4/d5d/fd2 0 2024-04-02T04:43:00.806 INFO:tasks.workunit.client.1.smithi027.stdout:0/612: chown d4/d69/l77 2583 1 2024-04-02T04:43:00.806 INFO:tasks.workunit.client.1.smithi027.stdout:5/802: symlink d0/df/d11/l10f 0 2024-04-02T04:43:00.808 INFO:tasks.workunit.client.1.smithi027.stdout:9/677: mknod d1a/d1d/d29/d37/d20/dcc/dd5/cd9 0 2024-04-02T04:43:00.809 INFO:tasks.workunit.client.1.smithi027.stdout:0/613: dread d4/d38/da0/f9e [0,4194304] 0 2024-04-02T04:43:00.809 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:00 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:43:00.809 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:00 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:43:00.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:00 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:43:00.810 INFO:tasks.workunit.client.1.smithi027.stdout:1/562: truncate d0/d1e/d25/d2b/f84 115788 0 2024-04-02T04:43:00.813 INFO:tasks.workunit.client.1.smithi027.stdout:1/563: dread d0/f3d [0,4194304] 0 2024-04-02T04:43:00.813 INFO:tasks.workunit.client.1.smithi027.stdout:0/614: symlink d4/d76/dc4/ld3 0 2024-04-02T04:43:00.822 INFO:tasks.workunit.client.1.smithi027.stdout:5/803: creat d0/f110 x:0 0 0 2024-04-02T04:43:00.823 INFO:tasks.workunit.client.1.smithi027.stdout:9/678: creat d1a/d1d/d29/d37/fda x:0 0 0 2024-04-02T04:43:00.826 INFO:tasks.workunit.client.1.smithi027.stdout:9/679: truncate d1a/d1d/d29/d37/fa1 228070 0 2024-04-02T04:43:00.830 INFO:tasks.workunit.client.1.smithi027.stdout:0/615: dread f2 [8388608,4194304] 0 2024-04-02T04:43:00.831 INFO:tasks.workunit.client.1.smithi027.stdout:5/804: dwrite d0/df/d2f/d3b/d51/ff5 [0,4194304] 0 2024-04-02T04:43:00.832 INFO:tasks.workunit.client.1.smithi027.stdout:1/564: dread d0/d1e/d5d/d60/fb0 [0,4194304] 0 2024-04-02T04:43:00.844 INFO:tasks.workunit.client.1.smithi027.stdout:1/565: creat d0/d1e/d25/d3c/d64/da6/fbb x:0 0 0 2024-04-02T04:43:00.853 INFO:tasks.workunit.client.1.smithi027.stdout:1/566: mkdir d0/de/d52/d74/dbc 0 2024-04-02T04:43:00.854 INFO:tasks.workunit.client.1.smithi027.stdout:9/680: getdents d1a/d1d/d29/d37/d20/dcc 0 2024-04-02T04:43:00.857 INFO:tasks.workunit.client.1.smithi027.stdout:9/681: symlink d1a/d1d/db7/ldb 0 2024-04-02T04:43:00.859 INFO:tasks.workunit.client.1.smithi027.stdout:5/805: truncate d0/df/d11/d13/d9a/db7/fdc 2049644 0 2024-04-02T04:43:00.860 INFO:tasks.workunit.client.1.smithi027.stdout:1/567: creat d0/fbd x:0 0 0 2024-04-02T04:43:00.861 INFO:tasks.workunit.client.1.smithi027.stdout:1/568: chown d0/d1e/d25/d3c/d64/l70 1781716 1 2024-04-02T04:43:00.862 INFO:tasks.workunit.client.1.smithi027.stdout:1/569: chown d0/de/f92 32097 1 2024-04-02T04:43:00.862 INFO:tasks.workunit.client.1.smithi027.stdout:9/682: rename d1a/d1d/d29/d50/f77 to d1a/d1d/d29/d37/d9b/fdc 0 2024-04-02T04:43:00.864 INFO:tasks.workunit.client.1.smithi027.stdout:1/570: unlink d0/d1e/d2e/d3b/d4f/f62 0 2024-04-02T04:43:00.881 INFO:tasks.workunit.client.1.smithi027.stdout:1/571: write d0/f54 [991958,100285] 0 2024-04-02T04:43:00.885 INFO:tasks.workunit.client.1.smithi027.stdout:1/572: symlink d0/d1e/d2e/lbe 0 2024-04-02T04:43:00.886 INFO:tasks.workunit.client.1.smithi027.stdout:1/573: chown d0/de/c7c 146521534 1 2024-04-02T04:43:00.887 INFO:tasks.workunit.client.1.smithi027.stdout:1/574: stat d0/l8 0 2024-04-02T04:43:00.889 INFO:tasks.workunit.client.1.smithi027.stdout:1/575: write d0/d1e/d5d/d60/f8f [3877783,26202] 0 2024-04-02T04:43:00.891 INFO:tasks.workunit.client.1.smithi027.stdout:1/576: symlink d0/d1e/d25/d3c/d64/lbf 0 2024-04-02T04:43:00.892 INFO:tasks.workunit.client.1.smithi027.stdout:1/577: chown d0/de/f80 49316 1 2024-04-02T04:43:00.893 INFO:tasks.workunit.client.1.smithi027.stdout:1/578: mkdir d0/de/dc0 0 2024-04-02T04:43:00.895 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:00 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:43:00.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:00 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:43:00.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:00 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:43:00.899 INFO:tasks.workunit.client.1.smithi027.stdout:1/579: dwrite d0/de/fa2 [0,4194304] 0 2024-04-02T04:43:00.950 INFO:tasks.workunit.client.1.smithi027.stdout:7/609: sync 2024-04-02T04:43:00.950 INFO:tasks.workunit.client.1.smithi027.stdout:8/605: sync 2024-04-02T04:43:00.968 INFO:tasks.workunit.client.1.smithi027.stdout:1/580: creat d0/de/d52/d74/fc1 x:0 0 0 2024-04-02T04:43:00.969 INFO:tasks.workunit.client.1.smithi027.stdout:7/610: mknod d2/d1d/cc1 0 2024-04-02T04:43:00.972 INFO:tasks.workunit.client.1.smithi027.stdout:8/606: rename d5/d8/db/d57/da5 to d5/d8/db/dd/d4a/dc9 0 2024-04-02T04:43:00.978 INFO:tasks.workunit.client.1.smithi027.stdout:8/607: rmdir d5/d8/d15 39 2024-04-02T04:43:00.980 INFO:tasks.workunit.client.1.smithi027.stdout:7/611: write d2/dba/dc/d16/d3f/d50/f69 [972432,80199] 0 2024-04-02T04:43:00.981 INFO:tasks.workunit.client.1.smithi027.stdout:1/581: link d0/d1e/d2e/d3e/d5f/d8b/d9d/fad d0/de/d52/d74/d97/fc2 0 2024-04-02T04:43:00.983 INFO:tasks.workunit.client.1.smithi027.stdout:8/608: dread - d5/d8/db/dd/d5f/d27/d31/f89 zero size 2024-04-02T04:43:00.983 INFO:tasks.workunit.client.1.smithi027.stdout:8/609: chown d5/d8/db/fa3 34413705 1 2024-04-02T04:43:00.984 INFO:tasks.workunit.client.1.smithi027.stdout:7/612: creat d2/dba/dc/d16/d2f/fc2 x:0 0 0 2024-04-02T04:43:00.984 INFO:tasks.workunit.client.1.smithi027.stdout:7/613: readlink d2/dba/dc/d1f/la7 0 2024-04-02T04:43:00.985 INFO:tasks.workunit.client.1.smithi027.stdout:7/614: chown d2/dba/dc/d16/d3a/l76 939 1 2024-04-02T04:43:00.986 INFO:tasks.workunit.client.1.smithi027.stdout:1/582: symlink d0/d1e/d5d/d60/lc3 0 2024-04-02T04:43:00.992 INFO:tasks.workunit.client.1.smithi027.stdout:7/615: truncate d2/dba/f1e 1169876 0 2024-04-02T04:43:00.995 INFO:tasks.workunit.client.1.smithi027.stdout:1/583: rmdir d0/d1e/d5d/d60/d99 0 2024-04-02T04:43:01.009 INFO:tasks.workunit.client.1.smithi027.stdout:7/616: write d2/d1d/f80 [385619,128479] 0 2024-04-02T04:43:01.013 INFO:tasks.workunit.client.1.smithi027.stdout:1/584: dwrite d0/f69 [0,4194304] 0 2024-04-02T04:43:01.022 INFO:tasks.workunit.client.1.smithi027.stdout:1/585: getdents d0/d1e/d25/d2b/d59 0 2024-04-02T04:43:01.023 INFO:tasks.workunit.client.1.smithi027.stdout:1/586: write d0/f69 [3251324,26593] 0 2024-04-02T04:43:01.024 INFO:tasks.workunit.client.1.smithi027.stdout:1/587: chown d0/de/f38 120978 1 2024-04-02T04:43:01.029 INFO:tasks.workunit.client.1.smithi027.stdout:1/588: creat d0/d1e/d5d/fc4 x:0 0 0 2024-04-02T04:43:01.031 INFO:tasks.workunit.client.1.smithi027.stdout:7/617: dwrite d2/dba/f2a [0,4194304] 0 2024-04-02T04:43:01.036 INFO:tasks.workunit.client.1.smithi027.stdout:1/589: rename d0/d1e/d25/d3c/d64/l70 to d0/d1e/d25/lc5 0 2024-04-02T04:43:01.038 INFO:tasks.workunit.client.1.smithi027.stdout:7/618: dwrite d2/f4c [0,4194304] 0 2024-04-02T04:43:01.042 INFO:tasks.workunit.client.1.smithi027.stdout:7/619: creat d2/dba/dc/fc3 x:0 0 0 2024-04-02T04:43:01.046 INFO:tasks.workunit.client.1.smithi027.stdout:7/620: creat d2/d1d/d92/fc4 x:0 0 0 2024-04-02T04:43:01.047 INFO:tasks.workunit.client.1.smithi027.stdout:1/590: getdents d0/d1e/d2e/d3b/d4f 0 2024-04-02T04:43:01.048 INFO:tasks.workunit.client.1.smithi027.stdout:1/591: chown d0/d1e/d2e/d3b/d4f/f75 8291165 1 2024-04-02T04:43:01.056 INFO:tasks.workunit.client.1.smithi027.stdout:1/592: dwrite d0/d1e/f98 [0,4194304] 0 2024-04-02T04:43:01.063 INFO:tasks.workunit.client.1.smithi027.stdout:7/621: symlink d2/dba/dc/d1f/d61/lc5 0 2024-04-02T04:43:01.066 INFO:tasks.workunit.client.1.smithi027.stdout:1/593: truncate d0/d1e/d25/d3c/d64/da6/db8/fba 511990 0 2024-04-02T04:43:01.066 INFO:tasks.workunit.client.1.smithi027.stdout:7/622: fdatasync d2/dba/dc/d16/d3f/d50/f66 0 2024-04-02T04:43:01.069 INFO:tasks.workunit.client.1.smithi027.stdout:1/594: creat d0/de/d52/d74/db7/fc6 x:0 0 0 2024-04-02T04:43:01.070 INFO:tasks.workunit.client.1.smithi027.stdout:7/623: truncate d2/d1d/f25 975466 0 2024-04-02T04:43:01.071 INFO:tasks.workunit.client.1.smithi027.stdout:1/595: creat d0/d1e/d2e/d3e/d5f/fc7 x:0 0 0 2024-04-02T04:43:01.073 INFO:tasks.workunit.client.1.smithi027.stdout:1/596: read d0/de/d52/f67 [5776309,113762] 0 2024-04-02T04:43:01.098 INFO:tasks.workunit.client.1.smithi027.stdout:6/538: sync 2024-04-02T04:43:01.099 INFO:tasks.workunit.client.1.smithi027.stdout:4/492: sync 2024-04-02T04:43:01.099 INFO:tasks.workunit.client.1.smithi027.stdout:2/631: sync 2024-04-02T04:43:01.099 INFO:tasks.workunit.client.1.smithi027.stdout:3/532: sync 2024-04-02T04:43:01.101 INFO:tasks.workunit.client.1.smithi027.stdout:2/632: truncate d1/db/d3b/d17/d6d/dd9/dd5/f4f 413074 0 2024-04-02T04:43:01.103 INFO:tasks.workunit.client.1.smithi027.stdout:3/533: unlink d1/f6d 0 2024-04-02T04:43:01.107 INFO:tasks.workunit.client.1.smithi027.stdout:7/624: dwrite d2/d1c/f6e [0,4194304] 0 2024-04-02T04:43:01.184 INFO:tasks.workunit.client.1.smithi027.stdout:6/539: mknod d13/d1b/d24/d25/d56/d73/cb5 0 2024-04-02T04:43:01.191 INFO:tasks.workunit.client.1.smithi027.stdout:7/625: creat d2/d1d/d57/fc6 x:0 0 0 2024-04-02T04:43:01.194 INFO:tasks.workunit.client.1.smithi027.stdout:2/633: mkdir d1/db/de5/de6 0 2024-04-02T04:43:01.196 INFO:tasks.workunit.client.1.smithi027.stdout:7/626: rmdir d2/dba/dc/d16 39 2024-04-02T04:43:01.201 INFO:tasks.workunit.client.1.smithi027.stdout:7/627: mkdir d2/df/dc7 0 2024-04-02T04:43:01.203 INFO:tasks.workunit.client.1.smithi027.stdout:4/493: dwrite d24/d38/d44/d41/d49/f4c [0,4194304] 0 2024-04-02T04:43:01.204 INFO:tasks.workunit.client.1.smithi027.stdout:1/597: dwrite d0/d1e/d25/f50 [0,4194304] 0 2024-04-02T04:43:01.204 INFO:tasks.workunit.client.1.smithi027.stdout:6/540: dwrite d13/f1a [0,4194304] 0 2024-04-02T04:43:01.208 INFO:tasks.workunit.client.1.smithi027.stdout:2/634: rename d1/db/d3b/d17/d6d/dd9/dd5/dad/l42 to d1/db/le7 0 2024-04-02T04:43:01.209 INFO:tasks.workunit.client.1.smithi027.stdout:7/628: dread - d2/dba/dc/fc3 zero size 2024-04-02T04:43:01.217 INFO:tasks.workunit.client.1.smithi027.stdout:2/635: rename d1/db/d3b/f60 to d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/dc0/dc3/fe8 0 2024-04-02T04:43:01.217 INFO:tasks.workunit.client.1.smithi027.stdout:1/598: read d0/d1e/d25/f31 [4527779,98519] 0 2024-04-02T04:43:01.219 INFO:tasks.workunit.client.1.smithi027.stdout:1/599: write d0/fbd [350146,34535] 0 2024-04-02T04:43:01.220 INFO:tasks.workunit.client.1.smithi027.stdout:1/600: write d0/d1e/d2e/f6b [419495,17088] 0 2024-04-02T04:43:01.222 INFO:tasks.workunit.client.1.smithi027.stdout:2/636: creat d1/db/d3b/d17/d68/d69/fe9 x:0 0 0 2024-04-02T04:43:01.224 INFO:tasks.workunit.client.1.smithi027.stdout:6/541: getdents d13/d1b/d8c 0 2024-04-02T04:43:01.224 INFO:tasks.workunit.client.1.smithi027.stdout:1/601: dread d0/d1e/f2f [0,4194304] 0 2024-04-02T04:43:01.226 INFO:tasks.workunit.client.1.smithi027.stdout:2/637: symlink d1/db/d3b/d61/dd2/lea 0 2024-04-02T04:43:01.226 INFO:tasks.workunit.client.1.smithi027.stdout:4/494: fdatasync f21 0 2024-04-02T04:43:01.228 INFO:tasks.workunit.client.1.smithi027.stdout:5/806: sync 2024-04-02T04:43:01.228 INFO:tasks.workunit.client.1.smithi027.stdout:0/616: sync 2024-04-02T04:43:01.230 INFO:tasks.workunit.client.1.smithi027.stdout:6/542: symlink d13/d1b/d24/d25/d56/d73/lb6 0 2024-04-02T04:43:01.230 INFO:tasks.workunit.client.1.smithi027.stdout:7/629: write d2/d1c/f40 [2727440,29822] 0 2024-04-02T04:43:01.235 INFO:tasks.workunit.client.1.smithi027.stdout:1/602: rename d0/f5 to d0/d1e/d2e/d3b/d4f/fc8 0 2024-04-02T04:43:01.237 INFO:tasks.workunit.client.1.smithi027.stdout:4/495: truncate d24/d38/d47/d5c/f7e 936965 0 2024-04-02T04:43:01.241 INFO:tasks.workunit.client.1.smithi027.stdout:5/807: unlink d0/df/d11/d13/f3e 0 2024-04-02T04:43:01.242 INFO:tasks.workunit.client.1.smithi027.stdout:1/603: dwrite d0/d1e/d25/d3c/d64/da6/db8/fba [0,4194304] 0 2024-04-02T04:43:01.243 INFO:tasks.workunit.client.1.smithi027.stdout:1/604: dread - d0/de/d52/d74/fb5 zero size 2024-04-02T04:43:01.244 INFO:tasks.workunit.client.1.smithi027.stdout:0/617: truncate d4/d38/d66/d2d/f3c 2192491 0 2024-04-02T04:43:01.245 INFO:tasks.workunit.client.1.smithi027.stdout:6/543: mkdir d13/d1b/d8c/db7 0 2024-04-02T04:43:01.247 INFO:tasks.workunit.client.1.smithi027.stdout:7/630: rmdir d2/d1d 39 2024-04-02T04:43:01.248 INFO:tasks.workunit.client.1.smithi027.stdout:6/544: write d13/d1b/d48/f66 [1829368,44237] 0 2024-04-02T04:43:01.253 INFO:tasks.workunit.client.1.smithi027.stdout:4/496: symlink d24/d38/d44/d41/d5a/la6 0 2024-04-02T04:43:01.253 INFO:tasks.workunit.client.1.smithi027.stdout:2/638: dwrite d1/db/d3b/d17/f4a [0,4194304] 0 2024-04-02T04:43:01.255 INFO:tasks.workunit.client.1.smithi027.stdout:2/639: stat d1/db/d3b/d17/d6d/dd9/dd5/f4f 0 2024-04-02T04:43:01.255 INFO:tasks.workunit.client.1.smithi027.stdout:2/640: readlink d1/db/d3b/d83/l90 0 2024-04-02T04:43:01.263 INFO:tasks.workunit.client.1.smithi027.stdout:2/641: dwrite d1/f56 [0,4194304] 0 2024-04-02T04:43:01.265 INFO:tasks.workunit.client.1.smithi027.stdout:1/605: rmdir d0/de/d52 39 2024-04-02T04:43:01.268 INFO:tasks.workunit.client.1.smithi027.stdout:2/642: dread d1/db/f32 [4194304,4194304] 0 2024-04-02T04:43:01.271 INFO:tasks.workunit.client.1.smithi027.stdout:4/497: readlink d24/d38/d44/l99 0 2024-04-02T04:43:01.275 INFO:tasks.workunit.client.1.smithi027.stdout:9/683: sync 2024-04-02T04:43:01.276 INFO:tasks.workunit.client.1.smithi027.stdout:0/618: truncate d4/d76/f99 1546395 0 2024-04-02T04:43:01.282 INFO:tasks.workunit.client.1.smithi027.stdout:1/606: chown d0/d1e/d2e/d3e/l4b 9804009 1 2024-04-02T04:43:01.283 INFO:tasks.workunit.client.1.smithi027.stdout:7/631: fsync d2/d1d/d57/f6d 0 2024-04-02T04:43:01.286 INFO:tasks.workunit.client.1.smithi027.stdout:6/545: mkdir d13/d1b/d48/d9c/db8 0 2024-04-02T04:43:01.287 INFO:tasks.workunit.client.1.smithi027.stdout:6/546: dread - d13/d1b/d24/d28/d46/f8f zero size 2024-04-02T04:43:01.288 INFO:tasks.workunit.client.1.smithi027.stdout:7/632: dread d2/d1d/d57/f6d [0,4194304] 0 2024-04-02T04:43:01.290 INFO:tasks.workunit.client.1.smithi027.stdout:2/643: rmdir d1/db/de5 39 2024-04-02T04:43:01.306 INFO:tasks.workunit.client.1.smithi027.stdout:0/619: creat d4/d21/d93/fd4 x:0 0 0 2024-04-02T04:43:01.309 INFO:tasks.workunit.client.1.smithi027.stdout:1/607: mknod d0/d1e/d2e/d3b/d4f/d9b/cc9 0 2024-04-02T04:43:01.310 INFO:tasks.workunit.client.1.smithi027.stdout:1/608: readlink d0/d1e/l20 0 2024-04-02T04:43:01.310 INFO:tasks.workunit.client.1.smithi027.stdout:1/609: chown d0/d1e/d2e/d3e/l85 1235593 1 2024-04-02T04:43:01.313 INFO:tasks.workunit.client.1.smithi027.stdout:6/547: fdatasync d13/d1b/d24/d25/d27/f3f 0 2024-04-02T04:43:01.316 INFO:tasks.workunit.client.1.smithi027.stdout:3/534: sync 2024-04-02T04:43:01.316 INFO:tasks.workunit.client.1.smithi027.stdout:8/610: sync 2024-04-02T04:43:01.317 INFO:tasks.workunit.client.1.smithi027.stdout:2/644: creat d1/db/d3b/d17/d68/d69/feb x:0 0 0 2024-04-02T04:43:01.321 INFO:tasks.workunit.client.1.smithi027.stdout:5/808: getdents d0/df/d29 0 2024-04-02T04:43:01.324 INFO:tasks.workunit.client.1.smithi027.stdout:9/684: mknod d1a/d1d/d29/d37/cdd 0 2024-04-02T04:43:01.325 INFO:tasks.workunit.client.1.smithi027.stdout:9/685: chown d1a/d1d/d29/c9a 387 1 2024-04-02T04:43:01.328 INFO:tasks.workunit.client.1.smithi027.stdout:0/620: rename d4/d3d to d4/d16/dd5 0 2024-04-02T04:43:01.333 INFO:tasks.workunit.client.1.smithi027.stdout:1/610: creat d0/d1e/d2e/d3e/d5f/fca x:0 0 0 2024-04-02T04:43:01.333 INFO:tasks.workunit.client.1.smithi027.stdout:9/686: dwrite d1a/db6/f96 [0,4194304] 0 2024-04-02T04:43:01.336 INFO:tasks.workunit.client.1.smithi027.stdout:6/548: mkdir d13/d1b/d48/d58/db9 0 2024-04-02T04:43:01.344 INFO:tasks.workunit.client.1.smithi027.stdout:8/611: truncate d5/d8/db/f76 258018 0 2024-04-02T04:43:01.350 INFO:tasks.workunit.client.1.smithi027.stdout:1/611: creat d0/d1e/d2e/d3e/fcb x:0 0 0 2024-04-02T04:43:01.351 INFO:tasks.workunit.client.1.smithi027.stdout:9/687: mkdir d1a/d1d/d29/d50/da2/dde 0 2024-04-02T04:43:01.354 INFO:tasks.workunit.client.1.smithi027.stdout:4/498: dwrite d24/f28 [0,4194304] 0 2024-04-02T04:43:01.356 INFO:tasks.workunit.client.1.smithi027.stdout:3/535: creat d1/de/df/da5/fb2 x:0 0 0 2024-04-02T04:43:01.358 INFO:tasks.workunit.client.1.smithi027.stdout:2/645: mkdir d1/db/d3b/d17/dec 0 2024-04-02T04:43:01.359 INFO:tasks.workunit.client.1.smithi027.stdout:4/499: write d24/d38/d44/fa5 [1009984,36498] 0 2024-04-02T04:43:01.367 INFO:tasks.workunit.client.1.smithi027.stdout:9/688: creat d1a/d1d/d29/d37/d20/dad/fdf x:0 0 0 2024-04-02T04:43:01.370 INFO:tasks.workunit.client.1.smithi027.stdout:9/689: write d1a/d1d/d29/d37/d20/d7e/f85 [4987937,68609] 0 2024-04-02T04:43:01.372 INFO:tasks.workunit.client.1.smithi027.stdout:9/690: dread - d1a/d1d/d29/d37/d99/fbe zero size 2024-04-02T04:43:01.376 INFO:tasks.workunit.client.1.smithi027.stdout:5/809: creat d0/df/d11/d13/d9a/d9c/f111 x:0 0 0 2024-04-02T04:43:01.381 INFO:tasks.workunit.client.1.smithi027.stdout:6/549: rename d13/d1b/d24/c95 to d13/d1b/d24/d25/d56/cba 0 2024-04-02T04:43:01.385 INFO:tasks.workunit.client.1.smithi027.stdout:6/550: write d13/d1b/d24/d25/d27/f97 [878352,25392] 0 2024-04-02T04:43:01.386 INFO:tasks.workunit.client.1.smithi027.stdout:9/691: symlink d1a/le0 0 2024-04-02T04:43:01.388 INFO:tasks.workunit.client.1.smithi027.stdout:8/612: creat d5/d8/db/fca x:0 0 0 2024-04-02T04:43:01.389 INFO:tasks.workunit.client.1.smithi027.stdout:5/810: symlink d0/df/d29/l112 0 2024-04-02T04:43:01.391 INFO:tasks.workunit.client.1.smithi027.stdout:3/536: fdatasync d1/de/df/d61/f3d 0 2024-04-02T04:43:01.393 INFO:tasks.workunit.client.1.smithi027.stdout:6/551: write d13/d8d/fa4 [1006802,37484] 0 2024-04-02T04:43:01.398 INFO:tasks.workunit.client.1.smithi027.stdout:5/811: creat d0/df/d11/d37/f113 x:0 0 0 2024-04-02T04:43:01.400 INFO:tasks.workunit.client.1.smithi027.stdout:3/537: mkdir d1/de/df/d13/db3 0 2024-04-02T04:43:01.400 INFO:tasks.workunit.client.1.smithi027.stdout:9/692: dwrite d1a/d1d/d29/d37/d99/fbe [0,4194304] 0 2024-04-02T04:43:01.403 INFO:tasks.workunit.client.1.smithi027.stdout:3/538: write d1/de/df/f67 [1312925,81562] 0 2024-04-02T04:43:01.403 INFO:tasks.workunit.client.1.smithi027.stdout:6/552: write d13/d1b/d48/d58/f64 [5541315,71145] 0 2024-04-02T04:43:01.403 INFO:tasks.workunit.client.1.smithi027.stdout:3/539: chown d1/de/f71 13107 1 2024-04-02T04:43:01.404 INFO:tasks.workunit.client.1.smithi027.stdout:1/612: rename d0/d1e/d25/d3c/d64/f8c to d0/de/fcc 0 2024-04-02T04:43:01.405 INFO:tasks.workunit.client.1.smithi027.stdout:3/540: write d1/de/df/f67 [578817,83234] 0 2024-04-02T04:43:01.406 INFO:tasks.workunit.client.1.smithi027.stdout:3/541: chown d1/de/df/d13/f51 966 1 2024-04-02T04:43:01.408 INFO:tasks.workunit.client.1.smithi027.stdout:8/613: mkdir d5/d8/db/dd/d5f/d27/d31/db9/dc1/dcb 0 2024-04-02T04:43:01.413 INFO:tasks.workunit.client.1.smithi027.stdout:5/812: symlink d0/df/d29/l114 0 2024-04-02T04:43:01.413 INFO:tasks.workunit.client.1.smithi027.stdout:1/613: dwrite d0/d1e/fa3 [0,4194304] 0 2024-04-02T04:43:01.414 INFO:tasks.workunit.client.1.smithi027.stdout:7/633: truncate d2/dba/dc/d16/d3a/d4d/f9d 3769489 0 2024-04-02T04:43:01.415 INFO:tasks.workunit.client.1.smithi027.stdout:6/553: mknod d13/d1b/d48/d75/d9b/cbb 0 2024-04-02T04:43:01.420 INFO:tasks.workunit.client.1.smithi027.stdout:4/500: rename d24/d38/f84 to d24/d38/d44/d82/fa7 0 2024-04-02T04:43:01.424 INFO:tasks.workunit.client.1.smithi027.stdout:3/542: creat d1/de/df/d9e/fb4 x:0 0 0 2024-04-02T04:43:01.424 INFO:tasks.workunit.client.1.smithi027.stdout:0/621: write d4/d38/d66/d2d/d4c/fcf [172985,61410] 0 2024-04-02T04:43:01.430 INFO:tasks.workunit.client.1.smithi027.stdout:1/614: read - d0/de/d52/d74/db7/fc6 zero size 2024-04-02T04:43:01.435 INFO:tasks.workunit.client.1.smithi027.stdout:5/813: truncate d0/df/d2f/f67 1264650 0 2024-04-02T04:43:01.438 INFO:tasks.workunit.client.1.smithi027.stdout:2/646: dwrite d1/db/d3b/f37 [0,4194304] 0 2024-04-02T04:43:01.440 INFO:tasks.workunit.client.1.smithi027.stdout:9/693: rename d1a/d1d/db7/dd8 to d1a/db6/d8c/de1 0 2024-04-02T04:43:01.444 INFO:tasks.workunit.client.1.smithi027.stdout:0/622: creat d4/d16/dd5/fd6 x:0 0 0 2024-04-02T04:43:01.446 INFO:tasks.workunit.client.1.smithi027.stdout:8/614: creat d5/d8/d15/d71/d96/fcc x:0 0 0 2024-04-02T04:43:01.447 INFO:tasks.workunit.client.1.smithi027.stdout:5/814: dwrite d0/d68/d6b/d74/ff0 [0,4194304] 0 2024-04-02T04:43:01.450 INFO:tasks.workunit.client.1.smithi027.stdout:1/615: mknod d0/d1e/d2e/d3b/d4f/ccd 0 2024-04-02T04:43:01.452 INFO:tasks.workunit.client.1.smithi027.stdout:7/634: creat d2/df/d89/d8d/fc8 x:0 0 0 2024-04-02T04:43:01.455 INFO:tasks.workunit.client.1.smithi027.stdout:7/635: truncate d2/dba/dc/d16/fbd 942209 0 2024-04-02T04:43:01.461 INFO:tasks.workunit.client.1.smithi027.stdout:9/694: rmdir d1a/d1d/d29/d37 39 2024-04-02T04:43:01.464 INFO:tasks.workunit.client.1.smithi027.stdout:2/647: mknod d1/db/d3b/d17/d6d/dd9/dd5/dda/dbb/ced 0 2024-04-02T04:43:01.464 INFO:tasks.workunit.client.1.smithi027.stdout:4/501: dread f14 [0,4194304] 0 2024-04-02T04:43:01.465 INFO:tasks.workunit.client.1.smithi027.stdout:4/502: chown d24/d38/d44/d60/f6f 11908625 1 2024-04-02T04:43:01.469 INFO:tasks.workunit.client.1.smithi027.stdout:5/815: mkdir d0/d1d/d61/d72/d115 0 2024-04-02T04:43:01.471 INFO:tasks.workunit.client.1.smithi027.stdout:1/616: rmdir d0/de 39 2024-04-02T04:43:01.471 INFO:tasks.workunit.client.1.smithi027.stdout:0/623: dwrite d4/d76/fa8 [0,4194304] 0 2024-04-02T04:43:01.476 INFO:tasks.workunit.client.1.smithi027.stdout:2/648: mknod d1/db/d3b/d83/cee 0 2024-04-02T04:43:01.479 INFO:tasks.workunit.client.1.smithi027.stdout:8/615: mknod d5/d8/db/dd/d5f/db2/dc6/ccd 0 2024-04-02T04:43:01.481 INFO:tasks.workunit.client.1.smithi027.stdout:5/816: mknod d0/df/d11/d13/d9a/c116 0 2024-04-02T04:43:01.483 INFO:tasks.workunit.client.1.smithi027.stdout:1/617: stat d0/de/d52/d74/fb5 0 2024-04-02T04:43:01.490 INFO:tasks.workunit.client.1.smithi027.stdout:6/554: rename d13/d1b/d24/d28/l38 to d13/d1b/d48/lbc 0 2024-04-02T04:43:01.491 INFO:tasks.workunit.client.1.smithi027.stdout:9/695: mkdir d1a/d1d/d29/d37/d20/dcc/dd5/de2 0 2024-04-02T04:43:01.491 INFO:tasks.workunit.client.1.smithi027.stdout:3/543: dwrite d1/de/df/f6f [0,4194304] 0 2024-04-02T04:43:01.492 INFO:tasks.workunit.client.1.smithi027.stdout:2/649: creat d1/db/d3b/d17/d6d/fef x:0 0 0 2024-04-02T04:43:01.494 INFO:tasks.workunit.client.1.smithi027.stdout:1/618: dwrite d0/d1e/d5d/d60/f8f [0,4194304] 0 2024-04-02T04:43:01.495 INFO:tasks.workunit.client.1.smithi027.stdout:3/544: chown d1/de/f38 4204 1 2024-04-02T04:43:01.495 INFO:tasks.workunit.client.1.smithi027.stdout:2/650: readlink d1/db/lcb 0 2024-04-02T04:43:01.506 INFO:tasks.workunit.client.1.smithi027.stdout:5/817: rename d0/df/d29/f7a to d0/d68/d6b/f117 0 2024-04-02T04:43:01.507 INFO:tasks.workunit.client.1.smithi027.stdout:5/818: fdatasync d0/df/d29/ff1 0 2024-04-02T04:43:01.511 INFO:tasks.workunit.client.1.smithi027.stdout:6/555: dread d13/d1b/d24/d25/d42/f60 [0,4194304] 0 2024-04-02T04:43:01.511 INFO:tasks.workunit.client.1.smithi027.stdout:9/696: symlink d1a/d1d/d29/d56/d8b/le3 0 2024-04-02T04:43:01.513 INFO:tasks.workunit.client.1.smithi027.stdout:6/556: write d13/d1b/d24/d25/d27/f81 [2628455,83936] 0 2024-04-02T04:43:01.514 INFO:tasks.workunit.client.1.smithi027.stdout:9/697: dread d1a/d1d/d29/d37/d3a/f84 [0,4194304] 0 2024-04-02T04:43:01.514 INFO:tasks.workunit.client.1.smithi027.stdout:3/545: chown d1/de/d74/la2 1 1 2024-04-02T04:43:01.517 INFO:tasks.workunit.client.1.smithi027.stdout:3/546: truncate d1/de/df/f6b 1574214 0 2024-04-02T04:43:01.518 INFO:tasks.workunit.client.1.smithi027.stdout:4/503: creat d24/fa8 x:0 0 0 2024-04-02T04:43:01.519 INFO:tasks.workunit.client.1.smithi027.stdout:4/504: chown d24/d38/d47/d62/c73 23 1 2024-04-02T04:43:01.520 INFO:tasks.workunit.client.1.smithi027.stdout:8/616: symlink d5/lce 0 2024-04-02T04:43:01.524 INFO:tasks.workunit.client.1.smithi027.stdout:4/505: dread f13 [0,4194304] 0 2024-04-02T04:43:01.535 INFO:tasks.workunit.client.1.smithi027.stdout:9/698: mkdir d1a/d1d/d29/d56/de4 0 2024-04-02T04:43:01.537 INFO:tasks.workunit.client.1.smithi027.stdout:9/699: truncate d1a/d1d/d29/d50/d73/dbd/fbf 1030087 0 2024-04-02T04:43:01.543 INFO:tasks.workunit.client.1.smithi027.stdout:8/617: rmdir d5/d8/db/dd/d5f/d27/d31 39 2024-04-02T04:43:01.545 INFO:tasks.workunit.client.1.smithi027.stdout:0/624: getdents d4/d16/d1b 0 2024-04-02T04:43:01.552 INFO:tasks.workunit.client.1.smithi027.stdout:6/557: symlink d13/d1b/d24/d25/lbd 0 2024-04-02T04:43:01.552 INFO:tasks.workunit.client.1.smithi027.stdout:4/506: dwrite d24/f76 [0,4194304] 0 2024-04-02T04:43:01.554 INFO:tasks.workunit.client.1.smithi027.stdout:2/651: mkdir d1/db/de5/de6/df0 0 2024-04-02T04:43:01.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:01 smithi016 ceph-mon[28969]: pgmap v29: 65 pgs: 65 active+clean; 3.3 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 91 MiB/s rd, 294 MiB/s wr, 582 op/s 2024-04-02T04:43:01.560 INFO:tasks.workunit.client.1.smithi027.stdout:9/700: unlink d1a/d1d/f53 0 2024-04-02T04:43:01.561 INFO:tasks.workunit.client.1.smithi027.stdout:3/547: creat d1/de/df/d9e/da0/fb5 x:0 0 0 2024-04-02T04:43:01.564 INFO:tasks.workunit.client.1.smithi027.stdout:8/618: chown d5/d8/l60 53 1 2024-04-02T04:43:01.565 INFO:tasks.workunit.client.1.smithi027.stdout:0/625: creat d4/d21/d28/fd7 x:0 0 0 2024-04-02T04:43:01.570 INFO:tasks.workunit.client.1.smithi027.stdout:7/636: dread d2/dba/dc/d16/d3a/d4d/f9d [0,4194304] 0 2024-04-02T04:43:01.572 INFO:tasks.workunit.client.1.smithi027.stdout:2/652: mkdir d1/db/d3b/d17/d6d/df1 0 2024-04-02T04:43:01.576 INFO:tasks.workunit.client.1.smithi027.stdout:1/619: rename d0/f16 to d0/d1e/d25/d2b/fce 0 2024-04-02T04:43:01.576 INFO:tasks.workunit.client.1.smithi027.stdout:1/620: write d0/f69 [8304343,22467] 0 2024-04-02T04:43:01.579 INFO:tasks.workunit.client.1.smithi027.stdout:3/548: mknod d1/de/df/d9e/cb6 0 2024-04-02T04:43:01.582 INFO:tasks.workunit.client.1.smithi027.stdout:8/619: mkdir d5/d8/db/dd/d5f/db2/dcf 0 2024-04-02T04:43:01.583 INFO:tasks.workunit.client.1.smithi027.stdout:9/701: dwrite d1a/f63 [0,4194304] 0 2024-04-02T04:43:01.584 INFO:tasks.workunit.client.1.smithi027.stdout:0/626: dread d4/d38/d66/d2d/f56 [0,4194304] 0 2024-04-02T04:43:01.585 INFO:tasks.workunit.client.1.smithi027.stdout:0/627: chown d4/d16/d1b/fc0 23134 1 2024-04-02T04:43:01.586 INFO:tasks.workunit.client.1.smithi027.stdout:5/819: getdents d0/d68/d6b/d74 0 2024-04-02T04:43:01.587 INFO:tasks.workunit.client.1.smithi027.stdout:5/820: truncate d0/f66 4585277 0 2024-04-02T04:43:01.589 INFO:tasks.workunit.client.1.smithi027.stdout:6/558: mknod d13/d1b/d24/d25/d56/d6d/daa/cbe 0 2024-04-02T04:43:01.590 INFO:tasks.workunit.client.1.smithi027.stdout:6/559: chown d13/d1b/c2e 230 1 2024-04-02T04:43:01.592 INFO:tasks.workunit.client.1.smithi027.stdout:7/637: symlink d2/dba/dc/d16/d3a/lc9 0 2024-04-02T04:43:01.593 INFO:tasks.workunit.client.1.smithi027.stdout:7/638: chown d2/dba/dc/d16/d3a 1421 1 2024-04-02T04:43:01.594 INFO:tasks.workunit.client.1.smithi027.stdout:6/560: dread d13/d8d/fa3 [0,4194304] 0 2024-04-02T04:43:01.596 INFO:tasks.workunit.client.1.smithi027.stdout:2/653: unlink d1/db/d3b/d17/d6d/f7f 0 2024-04-02T04:43:01.603 INFO:tasks.workunit.client.1.smithi027.stdout:4/507: write d24/d38/d44/f52 [996886,57633] 0 2024-04-02T04:43:01.604 INFO:tasks.workunit.client.1.smithi027.stdout:4/508: stat d24/d38/d47/d5c/l68 0 2024-04-02T04:43:01.611 INFO:tasks.workunit.client.1.smithi027.stdout:9/702: dread d1a/d1d/d29/d37/d20/d7e/f87 [0,4194304] 0 2024-04-02T04:43:01.618 INFO:tasks.workunit.client.1.smithi027.stdout:5/821: dwrite d0/fb0 [4194304,4194304] 0 2024-04-02T04:43:01.619 INFO:tasks.workunit.client.1.smithi027.stdout:5/822: write d0/d68/d6b/d74/fb1 [4876787,110891] 0 2024-04-02T04:43:01.628 INFO:tasks.workunit.client.1.smithi027.stdout:4/509: mkdir d24/d38/d44/d82/da9 0 2024-04-02T04:43:01.638 INFO:tasks.workunit.client.1.smithi027.stdout:7/639: creat d2/dba/dc/d1f/d61/fca x:0 0 0 2024-04-02T04:43:01.640 INFO:tasks.workunit.client.1.smithi027.stdout:2/654: write d1/f40 [5130250,78267] 0 2024-04-02T04:43:01.659 INFO:tasks.workunit.client.1.smithi027.stdout:5/823: write d0/df/d76/fc7 [782235,61104] 0 2024-04-02T04:43:01.669 INFO:tasks.workunit.client.1.smithi027.stdout:3/549: creat d1/de/d55/fb7 x:0 0 0 2024-04-02T04:43:01.671 INFO:tasks.workunit.client.1.smithi027.stdout:1/621: mkdir d0/d1e/d25/d2b/d59/dcf 0 2024-04-02T04:43:01.673 INFO:tasks.workunit.client.1.smithi027.stdout:9/703: unlink c19 0 2024-04-02T04:43:01.674 INFO:tasks.workunit.client.1.smithi027.stdout:7/640: mkdir d2/dba/dc/d16/d3a/d4d/dcb 0 2024-04-02T04:43:01.674 INFO:tasks.workunit.client.1.smithi027.stdout:9/704: readlink d1a/d1d/d29/d37/l1e 0 2024-04-02T04:43:01.676 INFO:tasks.workunit.client.1.smithi027.stdout:7/641: write d2/d1c/f6e [2108699,107052] 0 2024-04-02T04:43:01.677 INFO:tasks.workunit.client.1.smithi027.stdout:6/561: rename fc to d13/fbf 0 2024-04-02T04:43:01.678 INFO:tasks.workunit.client.1.smithi027.stdout:6/562: chown d13/d1b/d24/d28/d3c/l5e 3873 1 2024-04-02T04:43:01.679 INFO:tasks.workunit.client.1.smithi027.stdout:6/563: fsync d13/d1b/d24/d25/d56/fa0 0 2024-04-02T04:43:01.681 INFO:tasks.workunit.client.1.smithi027.stdout:4/510: creat d24/d38/d44/d82/da9/faa x:0 0 0 2024-04-02T04:43:01.683 INFO:tasks.workunit.client.1.smithi027.stdout:8/620: rmdir d5/d8/db/dd/d5f/d27/d31/d5b/dab 0 2024-04-02T04:43:01.683 INFO:tasks.workunit.client.1.smithi027.stdout:8/621: fsync d5/f7d 0 2024-04-02T04:43:01.685 INFO:tasks.workunit.client.1.smithi027.stdout:0/628: getdents d4/d38 0 2024-04-02T04:43:01.691 INFO:tasks.workunit.client.1.smithi027.stdout:0/629: dwrite d4/d21/d28/fd7 [0,4194304] 0 2024-04-02T04:43:01.696 INFO:tasks.workunit.client.1.smithi027.stdout:5/824: creat d0/df/d2f/d3b/d105/f118 x:0 0 0 2024-04-02T04:43:01.698 INFO:tasks.workunit.client.1.smithi027.stdout:5/825: chown d0/df/d11/d13/d9a/d9c/dbe 1677 1 2024-04-02T04:43:01.703 INFO:tasks.workunit.client.1.smithi027.stdout:8/622: chown d5/d8/db/dd/d5f/d27/l97 50 1 2024-04-02T04:43:01.704 INFO:tasks.workunit.client.1.smithi027.stdout:5/826: dwrite d0/df/d2f/d3b/f104 [0,4194304] 0 2024-04-02T04:43:01.711 INFO:tasks.workunit.client.1.smithi027.stdout:0/630: fsync d4/d21/d28/d4e/fd0 0 2024-04-02T04:43:01.712 INFO:tasks.workunit.client.1.smithi027.stdout:9/705: write d1a/d1d/d29/d37/d3a/f54 [6457728,76748] 0 2024-04-02T04:43:01.713 INFO:tasks.workunit.client.1.smithi027.stdout:0/631: chown d4/d16/d71/d97/fb8 46268573 1 2024-04-02T04:43:01.715 INFO:tasks.workunit.client.1.smithi027.stdout:0/632: write d4/d21/fbb [478738,116664] 0 2024-04-02T04:43:01.716 INFO:tasks.workunit.client.1.smithi027.stdout:0/633: fdatasync d4/d21/d28/f44 0 2024-04-02T04:43:01.716 INFO:tasks.workunit.client.1.smithi027.stdout:7/642: creat d2/dba/da6/fcc x:0 0 0 2024-04-02T04:43:01.718 INFO:tasks.workunit.client.1.smithi027.stdout:7/643: write d2/dba/dc/d16/f2c [2827126,8939] 0 2024-04-02T04:43:01.728 INFO:tasks.workunit.client.1.smithi027.stdout:4/511: truncate d24/d2d/f43 2550671 0 2024-04-02T04:43:01.730 INFO:tasks.workunit.client.1.smithi027.stdout:8/623: creat d5/d8/db/df/fd0 x:0 0 0 2024-04-02T04:43:01.731 INFO:tasks.workunit.client.1.smithi027.stdout:1/622: link d0/d1e/d25/d2b/d59/la9 d0/d1e/d2e/d3b/d4f/ld0 0 2024-04-02T04:43:01.732 INFO:tasks.workunit.client.1.smithi027.stdout:8/624: write d5/d8/db/fca [321008,66801] 0 2024-04-02T04:43:01.735 INFO:tasks.workunit.client.1.smithi027.stdout:9/706: creat d1a/d1d/db7/fe5 x:0 0 0 2024-04-02T04:43:01.738 INFO:tasks.workunit.client.1.smithi027.stdout:0/634: dread - d4/d38/d66/d4a/f80 zero size 2024-04-02T04:43:01.739 INFO:tasks.workunit.client.1.smithi027.stdout:0/635: read d4/d38/da0/f9e [584963,8570] 0 2024-04-02T04:43:01.741 INFO:tasks.workunit.client.1.smithi027.stdout:2/655: getdents d1/db/d3b 0 2024-04-02T04:43:01.742 INFO:tasks.workunit.client.1.smithi027.stdout:6/564: creat d13/d1b/d24/d25/d56/fc0 x:0 0 0 2024-04-02T04:43:01.744 INFO:tasks.workunit.client.1.smithi027.stdout:1/623: mkdir d0/de/d52/dd1 0 2024-04-02T04:43:01.745 INFO:tasks.workunit.client.1.smithi027.stdout:5/827: creat d0/df/d2f/d3b/d51/dc0/f119 x:0 0 0 2024-04-02T04:43:01.746 INFO:tasks.workunit.client.1.smithi027.stdout:8/625: mknod d5/d8/db/dd/d5f/d91/cd1 0 2024-04-02T04:43:01.748 INFO:tasks.workunit.client.1.smithi027.stdout:9/707: creat d1a/d1d/d29/d37/d3a/fe6 x:0 0 0 2024-04-02T04:43:01.750 INFO:tasks.workunit.client.1.smithi027.stdout:0/636: fdatasync d4/d38/d66/d2d/d4c/f5c 0 2024-04-02T04:43:01.751 INFO:tasks.workunit.client.1.smithi027.stdout:2/656: creat d1/db/d57/ff2 x:0 0 0 2024-04-02T04:43:01.752 INFO:tasks.workunit.client.1.smithi027.stdout:2/657: dread - d1/d91/fc2 zero size 2024-04-02T04:43:01.752 INFO:tasks.workunit.client.1.smithi027.stdout:6/565: mknod d13/d1b/d48/d9c/cc1 0 2024-04-02T04:43:01.754 INFO:tasks.workunit.client.1.smithi027.stdout:1/624: symlink d0/d1e/d2e/d3e/d5f/ld2 0 2024-04-02T04:43:01.756 INFO:tasks.workunit.client.1.smithi027.stdout:5/828: symlink d0/d1d/d61/d72/l11a 0 2024-04-02T04:43:01.757 INFO:tasks.workunit.client.1.smithi027.stdout:8/626: rmdir d5/d8/db/d57 39 2024-04-02T04:43:01.760 INFO:tasks.workunit.client.1.smithi027.stdout:9/708: chown d1a/d1d/d29/d37/l8a 7979 1 2024-04-02T04:43:01.761 INFO:tasks.workunit.client.1.smithi027.stdout:2/658: dwrite d1/db/d3b/d83/fb0 [0,4194304] 0 2024-04-02T04:43:01.763 INFO:tasks.workunit.client.1.smithi027.stdout:6/566: mknod d13/d1b/d24/d25/d56/d73/cc2 0 2024-04-02T04:43:01.765 INFO:tasks.workunit.client.1.smithi027.stdout:1/625: dwrite d0/d1e/d2e/f6d [4194304,4194304] 0 2024-04-02T04:43:01.768 INFO:tasks.workunit.client.1.smithi027.stdout:1/626: stat d0/d1e/d2e/d3e/d5f/c7d 0 2024-04-02T04:43:01.769 INFO:tasks.workunit.client.1.smithi027.stdout:8/627: fdatasync d5/d8/db/d57/fa1 0 2024-04-02T04:43:01.773 INFO:tasks.workunit.client.1.smithi027.stdout:9/709: rename d1a/db6/d8c/cc4 to d1a/d1d/d29/d37/d20/d7e/ce7 0 2024-04-02T04:43:01.776 INFO:tasks.workunit.client.1.smithi027.stdout:2/659: creat d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/de1/ff3 x:0 0 0 2024-04-02T04:43:01.776 INFO:tasks.workunit.client.1.smithi027.stdout:9/710: write d1a/d1d/d29/d50/fd3 [724104,28215] 0 2024-04-02T04:43:01.777 INFO:tasks.workunit.client.1.smithi027.stdout:6/567: dwrite d13/d1b/d24/d25/d56/fa6 [0,4194304] 0 2024-04-02T04:43:01.779 INFO:tasks.workunit.client.1.smithi027.stdout:9/711: write d1a/d1d/d29/d50/da2/fba [79991,7458] 0 2024-04-02T04:43:01.787 INFO:tasks.workunit.client.1.smithi027.stdout:9/712: dread d1a/d1d/d3c/d6e/fb3 [0,4194304] 0 2024-04-02T04:43:01.788 INFO:tasks.workunit.client.1.smithi027.stdout:2/660: rmdir d1/db/d3b/d17/d6d/dd9/dd5 39 2024-04-02T04:43:01.789 INFO:tasks.workunit.client.1.smithi027.stdout:2/661: chown d1/db/d3b/d61/d75/f92 1 1 2024-04-02T04:43:01.789 INFO:tasks.workunit.client.1.smithi027.stdout:2/662: stat d1/db/d3b/d17/d68/f94 0 2024-04-02T04:43:01.791 INFO:tasks.workunit.client.1.smithi027.stdout:0/637: link d4/d38/l8e d4/d38/d66/ld8 0 2024-04-02T04:43:01.792 INFO:tasks.workunit.client.1.smithi027.stdout:0/638: write d4/d76/fa8 [486292,16004] 0 2024-04-02T04:43:01.794 INFO:tasks.workunit.client.1.smithi027.stdout:9/713: mkdir d1a/d1d/d29/d37/d99/de8 0 2024-04-02T04:43:01.795 INFO:tasks.workunit.client.1.smithi027.stdout:9/714: stat d1a/d1d/d29/d37/c4d 0 2024-04-02T04:43:01.795 INFO:tasks.workunit.client.1.smithi027.stdout:9/715: chown d1a/d1d/d3c/fb2 1 1 2024-04-02T04:43:01.802 INFO:tasks.workunit.client.1.smithi027.stdout:8/628: creat d5/d8/db/dd/fd2 x:0 0 0 2024-04-02T04:43:01.803 INFO:tasks.workunit.client.1.smithi027.stdout:8/629: read - d5/d8/db/dd/d4a/dc9/fa6 zero size 2024-04-02T04:43:01.805 INFO:tasks.workunit.client.1.smithi027.stdout:0/639: unlink d4/d38/d66/d2d/c30 0 2024-04-02T04:43:01.806 INFO:tasks.workunit.client.1.smithi027.stdout:0/640: write d4/d76/dc4/fcd [269530,87554] 0 2024-04-02T04:43:01.807 INFO:tasks.workunit.client.1.smithi027.stdout:2/663: mknod d1/db/d3b/d17/d6d/df1/cf4 0 2024-04-02T04:43:01.810 INFO:tasks.workunit.client.1.smithi027.stdout:8/630: dread d5/f7d [0,4194304] 0 2024-04-02T04:43:01.813 INFO:tasks.workunit.client.1.smithi027.stdout:0/641: symlink d4/d21/d28/ld9 0 2024-04-02T04:43:01.814 INFO:tasks.workunit.client.1.smithi027.stdout:0/642: readlink d4/d38/d66/d2d/l41 0 2024-04-02T04:43:01.815 INFO:tasks.workunit.client.1.smithi027.stdout:0/643: fsync d4/d16/d1b/fc2 0 2024-04-02T04:43:01.817 INFO:tasks.workunit.client.1.smithi027.stdout:2/664: truncate d1/db/d3b/d17/d2b/f4e 1554211 0 2024-04-02T04:43:01.819 INFO:tasks.workunit.client.1.smithi027.stdout:2/665: dread - d1/db/d3b/d61/f5f zero size 2024-04-02T04:43:01.820 INFO:tasks.workunit.client.1.smithi027.stdout:9/716: dwrite d1a/d1d/d29/d37/d20/d7e/f87 [0,4194304] 0 2024-04-02T04:43:01.829 INFO:tasks.workunit.client.1.smithi027.stdout:9/717: creat d1a/db6/d8c/de1/fe9 x:0 0 0 2024-04-02T04:43:01.830 INFO:tasks.workunit.client.1.smithi027.stdout:8/631: link d5/d8/l60 d5/d8/db/dd/d5f/d27/d31/d5b/d6c/ld3 0 2024-04-02T04:43:01.835 INFO:tasks.workunit.client.1.smithi027.stdout:8/632: read d5/d8/db/dd/d4a/dc9/f93 [782065,66298] 0 2024-04-02T04:43:01.836 INFO:tasks.workunit.client.1.smithi027.stdout:9/718: dread d1a/d1d/d29/d37/d4c/f62 [0,4194304] 0 2024-04-02T04:43:01.837 INFO:tasks.workunit.client.1.smithi027.stdout:2/666: dwrite d1/db/d3b/d17/d6d/dd9/dd5/f5a [4194304,4194304] 0 2024-04-02T04:43:01.842 INFO:tasks.workunit.client.1.smithi027.stdout:0/644: dwrite d4/f9 [0,4194304] 0 2024-04-02T04:43:01.845 INFO:tasks.workunit.client.1.smithi027.stdout:0/645: write d4/d16/d1b/fb7 [944110,12764] 0 2024-04-02T04:43:01.845 INFO:tasks.workunit.client.1.smithi027.stdout:9/719: creat d1a/d1d/db7/fea x:0 0 0 2024-04-02T04:43:01.849 INFO:tasks.workunit.client.1.smithi027.stdout:9/720: dread - d1a/d1d/d29/d37/d3a/fe6 zero size 2024-04-02T04:43:01.850 INFO:tasks.workunit.client.1.smithi027.stdout:8/633: mknod d5/d8/db/dd/d5f/d27/d31/db9/cd4 0 2024-04-02T04:43:01.852 INFO:tasks.workunit.client.1.smithi027.stdout:9/721: fdatasync d1a/db6/d8c/de1/fe9 0 2024-04-02T04:43:01.855 INFO:tasks.workunit.client.1.smithi027.stdout:8/634: dread d5/d8/db/fa3 [0,4194304] 0 2024-04-02T04:43:01.855 INFO:tasks.workunit.client.1.smithi027.stdout:9/722: chown d1a/d1d/d3c/d6e/fc6 985 1 2024-04-02T04:43:01.856 INFO:tasks.workunit.client.1.smithi027.stdout:0/646: dwrite d4/d38/d66/f24 [0,4194304] 0 2024-04-02T04:43:01.858 INFO:tasks.workunit.client.1.smithi027.stdout:2/667: mknod d1/db/d3b/d17/d6d/dd9/dd5/dda/dbb/db4/ddf/cf5 0 2024-04-02T04:43:01.865 INFO:tasks.workunit.client.1.smithi027.stdout:9/723: unlink d1a/d1d/d29/d37/d20/d7e/f85 0 2024-04-02T04:43:01.869 INFO:tasks.workunit.client.1.smithi027.stdout:9/724: dread d1a/d1d/d29/d51/f60 [0,4194304] 0 2024-04-02T04:43:01.870 INFO:tasks.workunit.client.1.smithi027.stdout:9/725: write d1a/d1d/d29/d37/d3a/fd2 [110338,105542] 0 2024-04-02T04:43:01.870 INFO:tasks.workunit.client.1.smithi027.stdout:8/635: mkdir d5/d8/db/dd5 0 2024-04-02T04:43:01.871 INFO:tasks.workunit.client.1.smithi027.stdout:8/636: dread - d5/d8/db/dd/d4a/dc9/fa6 zero size 2024-04-02T04:43:01.878 INFO:tasks.workunit.client.1.smithi027.stdout:8/637: mknod d5/d8/db/dd/d5f/d91/cd6 0 2024-04-02T04:43:01.885 INFO:tasks.workunit.client.1.smithi027.stdout:0/647: link d4/d38/f7f d4/d38/d66/fda 0 2024-04-02T04:43:01.889 INFO:tasks.workunit.client.1.smithi027.stdout:8/638: creat d5/d8/db/dd/d5f/d27/d31/fd7 x:0 0 0 2024-04-02T04:43:01.892 INFO:tasks.workunit.client.1.smithi027.stdout:9/726: truncate d1a/d1d/d29/f46 2481545 0 2024-04-02T04:43:01.894 INFO:tasks.workunit.client.1.smithi027.stdout:2/668: dwrite d1/db/f30 [4194304,4194304] 0 2024-04-02T04:43:01.898 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:01 smithi027 ceph-mon[37550]: pgmap v29: 65 pgs: 65 active+clean; 3.3 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 91 MiB/s rd, 294 MiB/s wr, 582 op/s 2024-04-02T04:43:01.903 INFO:tasks.workunit.client.1.smithi027.stdout:2/669: readlink d1/db/d3b/d17/d6d/dd9/dd5/l67 0 2024-04-02T04:43:01.903 INFO:tasks.workunit.client.1.smithi027.stdout:9/727: dwrite d1a/d1d/f24 [0,4194304] 0 2024-04-02T04:43:01.904 INFO:tasks.workunit.client.1.smithi027.stdout:0/648: dwrite d4/d69/f8a [0,4194304] 0 2024-04-02T04:43:01.907 INFO:tasks.workunit.client.1.smithi027.stdout:8/639: truncate d5/d8/db/dd/d5f/d27/d31/f84 353735 0 2024-04-02T04:43:01.916 INFO:tasks.workunit.client.1.smithi027.stdout:2/670: mknod d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/dc0/cf6 0 2024-04-02T04:43:01.919 INFO:tasks.workunit.client.1.smithi027.stdout:2/671: truncate d1/db/d3b/d17/d68/d69/fe9 625875 0 2024-04-02T04:43:01.919 INFO:tasks.workunit.client.1.smithi027.stdout:8/640: creat d5/db5/fd8 x:0 0 0 2024-04-02T04:43:01.921 INFO:tasks.workunit.client.1.smithi027.stdout:0/649: dread d4/d21/d28/f40 [0,4194304] 0 2024-04-02T04:43:01.926 INFO:tasks.workunit.client.1.smithi027.stdout:2/672: rename d1/db/d3b/d17/c25 to d1/db/d3b/d17/d6d/cf7 0 2024-04-02T04:43:01.928 INFO:tasks.workunit.client.1.smithi027.stdout:2/673: dread - d1/db/d3b/d17/d68/d69/feb zero size 2024-04-02T04:43:01.929 INFO:tasks.workunit.client.1.smithi027.stdout:0/650: rmdir d4/d21/d28/d4e/db5/dc6/dc9 0 2024-04-02T04:43:01.930 INFO:tasks.workunit.client.1.smithi027.stdout:0/651: write d4/d16/d1b/fc0 [740345,13096] 0 2024-04-02T04:43:01.931 INFO:tasks.workunit.client.1.smithi027.stdout:2/674: creat d1/db/d3b/d83/ff8 x:0 0 0 2024-04-02T04:43:01.931 INFO:tasks.workunit.client.1.smithi027.stdout:0/652: dread - d4/d5d/fd2 zero size 2024-04-02T04:43:01.933 INFO:tasks.workunit.client.1.smithi027.stdout:2/675: dread - d1/db/d3b/d17/d6d/dd9/dd5/dad/f71 zero size 2024-04-02T04:43:01.934 INFO:tasks.workunit.client.1.smithi027.stdout:8/641: dread d5/d8/f47 [0,4194304] 0 2024-04-02T04:43:01.934 INFO:tasks.workunit.client.1.smithi027.stdout:0/653: mknod d4/d21/d28/cdb 0 2024-04-02T04:43:01.934 INFO:tasks.workunit.client.1.smithi027.stdout:2/676: read - d1/d91/fca zero size 2024-04-02T04:43:01.937 INFO:tasks.workunit.client.1.smithi027.stdout:8/642: creat d5/db5/fd9 x:0 0 0 2024-04-02T04:43:01.938 INFO:tasks.workunit.client.1.smithi027.stdout:2/677: write d1/db/d3b/d61/fc7 [368026,33727] 0 2024-04-02T04:43:01.940 INFO:tasks.workunit.client.1.smithi027.stdout:2/678: fsync d1/db/d3b/f37 0 2024-04-02T04:43:01.940 INFO:tasks.workunit.client.1.smithi027.stdout:0/654: creat d4/d21/d28/d4e/d5f/fdc x:0 0 0 2024-04-02T04:43:01.940 INFO:tasks.workunit.client.1.smithi027.stdout:2/679: dread - d1/db/d3b/d61/fdd zero size 2024-04-02T04:43:01.943 INFO:tasks.workunit.client.1.smithi027.stdout:8/643: creat d5/d8/db/dd/fda x:0 0 0 2024-04-02T04:43:01.945 INFO:tasks.workunit.client.1.smithi027.stdout:0/655: creat d4/d38/d66/d4a/fdd x:0 0 0 2024-04-02T04:43:01.946 INFO:tasks.workunit.client.1.smithi027.stdout:2/680: mkdir d1/db/d3b/d17/dcf/df9 0 2024-04-02T04:43:01.949 INFO:tasks.workunit.client.1.smithi027.stdout:0/656: creat d4/d21/d28/fde x:0 0 0 2024-04-02T04:43:01.951 INFO:tasks.workunit.client.1.smithi027.stdout:8/644: dwrite d5/d8/d15/d71/d96/fcc [0,4194304] 0 2024-04-02T04:43:01.953 INFO:tasks.workunit.client.1.smithi027.stdout:2/681: dread d1/f56 [0,4194304] 0 2024-04-02T04:43:01.958 INFO:tasks.workunit.client.1.smithi027.stdout:0/657: rename d4/d38/da0/f9e to d4/d16/d71/fdf 0 2024-04-02T04:43:02.011 INFO:tasks.workunit.client.1.smithi027.stdout:8/645: chown d5/d8/db/dd/d5f/d27/d31/d5b/dc4 476400630 1 2024-04-02T04:43:02.011 INFO:tasks.workunit.client.1.smithi027.stdout:8/646: chown d5/d8/db/df 5190944 1 2024-04-02T04:43:02.013 INFO:tasks.workunit.client.1.smithi027.stdout:2/682: rename d1/db/d3b/d17/d6d/dd9/dd5/dda/dbb/db4/ddf/cf5 to d1/db/d3b/d61/dd2/cfa 0 2024-04-02T04:43:02.021 INFO:tasks.workunit.client.1.smithi027.stdout:0/658: rename d4/d16/d71/d97 to d4/de0 0 2024-04-02T04:43:02.021 INFO:tasks.workunit.client.1.smithi027.stdout:2/683: link d1/db/d3b/d83/l90 d1/db/d3b/d17/d6d/df1/lfb 0 2024-04-02T04:43:02.024 INFO:tasks.workunit.client.1.smithi027.stdout:0/659: mknod d4/d21/d85/db3/ce1 0 2024-04-02T04:43:02.026 INFO:tasks.workunit.client.1.smithi027.stdout:8/647: link d5/d8/db/dd/c28 d5/cdb 0 2024-04-02T04:43:02.027 INFO:tasks.workunit.client.1.smithi027.stdout:0/660: rmdir d4/d38/da0 39 2024-04-02T04:43:02.027 INFO:tasks.workunit.client.1.smithi027.stdout:0/661: readlink d4/d69/l77 0 2024-04-02T04:43:02.030 INFO:tasks.workunit.client.1.smithi027.stdout:2/684: symlink d1/db/d3b/d17/lfc 0 2024-04-02T04:43:02.031 INFO:tasks.workunit.client.1.smithi027.stdout:2/685: read - d1/db/d3b/d17/d2b/db8/fd0 zero size 2024-04-02T04:43:02.033 INFO:tasks.workunit.client.1.smithi027.stdout:8/648: chown d5/d8/db/dd/d4a/c86 6582 1 2024-04-02T04:43:02.033 INFO:tasks.workunit.client.1.smithi027.stdout:2/686: chown d1/db/d3b/d17/d68/d69/fd6 218197257 1 2024-04-02T04:43:02.033 INFO:tasks.workunit.client.1.smithi027.stdout:8/649: chown d5/d8/d15/f37 1882913 1 2024-04-02T04:43:02.035 INFO:tasks.workunit.client.1.smithi027.stdout:0/662: mkdir d4/de0/de2 0 2024-04-02T04:43:02.037 INFO:tasks.workunit.client.1.smithi027.stdout:2/687: creat d1/db/d3b/d17/d6d/dd9/dd5/dda/dbb/db4/ddf/ffd x:0 0 0 2024-04-02T04:43:02.041 INFO:tasks.workunit.client.1.smithi027.stdout:2/688: mkdir d1/dfe 0 2024-04-02T04:43:02.041 INFO:tasks.workunit.client.1.smithi027.stdout:2/689: read d1/f88 [862144,83104] 0 2024-04-02T04:43:02.042 INFO:tasks.workunit.client.1.smithi027.stdout:0/663: rmdir d4/d21/d28/d4e/db5/dc6 0 2024-04-02T04:43:02.047 INFO:tasks.workunit.client.1.smithi027.stdout:2/690: dread d1/f4d [0,4194304] 0 2024-04-02T04:43:02.048 INFO:tasks.workunit.client.1.smithi027.stdout:2/691: truncate d1/f40 5728897 0 2024-04-02T04:43:02.049 INFO:tasks.workunit.client.1.smithi027.stdout:2/692: chown d1/db/d3b/d17/c2d 10 1 2024-04-02T04:43:02.057 INFO:tasks.workunit.client.1.smithi027.stdout:0/664: dread d4/f60 [0,4194304] 0 2024-04-02T04:43:02.061 INFO:tasks.workunit.client.1.smithi027.stdout:0/665: rename d4/d16/d1b/c31 to d4/de0/ce3 0 2024-04-02T04:43:02.063 INFO:tasks.workunit.client.1.smithi027.stdout:0/666: symlink d4/de0/le4 0 2024-04-02T04:43:02.064 INFO:tasks.workunit.client.1.smithi027.stdout:0/667: readlink d4/d5d/la4 0 2024-04-02T04:43:02.065 INFO:tasks.workunit.client.1.smithi027.stdout:0/668: chown d4/d38/d66/d2d 16544579 1 2024-04-02T04:43:02.065 INFO:tasks.workunit.client.1.smithi027.stdout:2/693: truncate d1/f4d 722474 0 2024-04-02T04:43:02.067 INFO:tasks.workunit.client.1.smithi027.stdout:0/669: fsync d4/d38/d66/d4a/f80 0 2024-04-02T04:43:02.068 INFO:tasks.workunit.client.1.smithi027.stdout:2/694: mknod d1/db/d3b/d17/d68/d69/cff 0 2024-04-02T04:43:02.070 INFO:tasks.workunit.client.1.smithi027.stdout:2/695: read d1/db/d3b/d17/f63 [770823,112860] 0 2024-04-02T04:43:02.070 INFO:tasks.workunit.client.1.smithi027.stdout:0/670: creat d4/d5d/fe5 x:0 0 0 2024-04-02T04:43:02.070 INFO:tasks.workunit.client.1.smithi027.stdout:2/696: stat d1/db/d3b/d17/d6d/dd9 0 2024-04-02T04:43:02.074 INFO:tasks.workunit.client.1.smithi027.stdout:0/671: symlink d4/d21/d85/db3/le6 0 2024-04-02T04:43:02.075 INFO:tasks.workunit.client.1.smithi027.stdout:0/672: write d4/d21/d85/fad [508770,19421] 0 2024-04-02T04:43:02.077 INFO:tasks.workunit.client.1.smithi027.stdout:0/673: readlink d4/d38/d66/l83 0 2024-04-02T04:43:02.077 INFO:tasks.workunit.client.1.smithi027.stdout:2/697: dwrite d1/db/d57/ff2 [0,4194304] 0 2024-04-02T04:43:02.079 INFO:tasks.workunit.client.1.smithi027.stdout:2/698: fdatasync d1/db/d3b/d17/d68/d69/feb 0 2024-04-02T04:43:02.083 INFO:tasks.workunit.client.1.smithi027.stdout:0/674: mknod d4/ce7 0 2024-04-02T04:43:02.085 INFO:tasks.workunit.client.1.smithi027.stdout:0/675: truncate d4/d21/f8f 1020840 0 2024-04-02T04:43:02.089 INFO:tasks.workunit.client.1.smithi027.stdout:0/676: creat d4/d16/dd5/fe8 x:0 0 0 2024-04-02T04:43:02.090 INFO:tasks.workunit.client.1.smithi027.stdout:0/677: write d4/d21/d28/fd7 [5100955,77046] 0 2024-04-02T04:43:02.102 INFO:tasks.workunit.client.1.smithi027.stdout:2/699: truncate d1/db/d3b/d17/d6d/dd9/dd5/f5a 8141281 0 2024-04-02T04:43:02.104 INFO:tasks.workunit.client.1.smithi027.stdout:0/678: truncate d4/d21/fb2 131777 0 2024-04-02T04:43:02.105 INFO:tasks.workunit.client.1.smithi027.stdout:2/700: mkdir d1/db/d3b/d61/d75/d100 0 2024-04-02T04:43:02.107 INFO:tasks.workunit.client.1.smithi027.stdout:0/679: read d4/d21/d28/d4e/fbd [2290097,126411] 0 2024-04-02T04:43:02.108 INFO:tasks.workunit.client.1.smithi027.stdout:2/701: symlink d1/db/d3b/d61/l101 0 2024-04-02T04:43:02.108 INFO:tasks.workunit.client.1.smithi027.stdout:0/680: truncate d4/f73 558741 0 2024-04-02T04:43:02.109 INFO:tasks.workunit.client.1.smithi027.stdout:2/702: chown d1/db/d3b/d17/d2b/f53 3310 1 2024-04-02T04:43:02.112 INFO:tasks.workunit.client.1.smithi027.stdout:0/681: rename d4/d38/c57 to d4/d21/d28/ce9 0 2024-04-02T04:43:02.119 INFO:tasks.workunit.client.1.smithi027.stdout:2/703: link d1/db/d3b/d61/fc7 d1/db/d3b/d17/d6d/dd9/dd5/dda/dbb/db4/ddf/f102 0 2024-04-02T04:43:02.123 INFO:tasks.workunit.client.1.smithi027.stdout:2/704: creat d1/db/d3b/d17/d6d/df1/f103 x:0 0 0 2024-04-02T04:43:02.125 INFO:tasks.workunit.client.1.smithi027.stdout:2/705: dread - d1/db/d3b/d17/fb5 zero size 2024-04-02T04:43:02.130 INFO:tasks.workunit.client.1.smithi027.stdout:2/706: dwrite d1/db/d3b/d17/d12/f65 [0,4194304] 0 2024-04-02T04:43:02.137 INFO:tasks.workunit.client.1.smithi027.stdout:2/707: creat d1/db/d3b/d17/dcf/df9/f104 x:0 0 0 2024-04-02T04:43:02.142 INFO:tasks.workunit.client.1.smithi027.stdout:2/708: creat d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/f105 x:0 0 0 2024-04-02T04:43:02.143 INFO:tasks.workunit.client.1.smithi027.stdout:2/709: chown d1/db/d3b/d17/d2b/db8/cdb 11 1 2024-04-02T04:43:02.144 INFO:tasks.workunit.client.1.smithi027.stdout:2/710: chown d1/db/d3b/d17/d6d/cd7 55947 1 2024-04-02T04:43:02.145 INFO:tasks.workunit.client.1.smithi027.stdout:2/711: chown d1/db/d3b/d17/d2b/f53 14485 1 2024-04-02T04:43:02.146 INFO:tasks.workunit.client.1.smithi027.stdout:0/682: write d4/d21/fb2 [623744,121096] 0 2024-04-02T04:43:02.148 INFO:tasks.workunit.client.1.smithi027.stdout:0/683: dread - d4/d38/d66/d4a/f61 zero size 2024-04-02T04:43:02.154 INFO:tasks.workunit.client.1.smithi027.stdout:2/712: dwrite d1/db/d3b/d17/d6d/dd9/dd5/dda/dbb/fe4 [0,4194304] 0 2024-04-02T04:43:02.157 INFO:tasks.workunit.client.1.smithi027.stdout:0/684: dwrite d4/d21/d93/fd4 [0,4194304] 0 2024-04-02T04:43:02.169 INFO:tasks.workunit.client.1.smithi027.stdout:0/685: creat d4/d21/d28/d4e/d5f/fea x:0 0 0 2024-04-02T04:43:02.169 INFO:tasks.workunit.client.1.smithi027.stdout:2/713: getdents d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/dc0/dc3 0 2024-04-02T04:43:02.194 INFO:tasks.workunit.client.1.smithi027.stdout:3/550: sync 2024-04-02T04:43:02.199 INFO:tasks.workunit.client.1.smithi027.stdout:3/551: dwrite d1/de/d55/fb7 [0,4194304] 0 2024-04-02T04:43:02.249 INFO:tasks.workunit.client.1.smithi027.stdout:2/714: mkdir d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/d106 0 2024-04-02T04:43:02.250 INFO:tasks.workunit.client.1.smithi027.stdout:2/715: chown d1/db 0 1 2024-04-02T04:43:02.251 INFO:tasks.workunit.client.1.smithi027.stdout:3/552: rmdir d1/de/d74 39 2024-04-02T04:43:02.253 INFO:tasks.workunit.client.1.smithi027.stdout:0/686: unlink d4/f14 0 2024-04-02T04:43:02.255 INFO:tasks.workunit.client.1.smithi027.stdout:2/716: rename d1/l22 to d1/db/d3b/d17/dec/l107 0 2024-04-02T04:43:02.257 INFO:tasks.workunit.client.1.smithi027.stdout:2/717: read d1/db/d3b/d17/d6d/d84/fd8 [3660835,42797] 0 2024-04-02T04:43:02.258 INFO:tasks.workunit.client.1.smithi027.stdout:0/687: unlink d4/d38/d66/d4a/l6d 0 2024-04-02T04:43:02.261 INFO:tasks.workunit.client.1.smithi027.stdout:3/553: dread d1/de/df/d61/f3d [0,4194304] 0 2024-04-02T04:43:02.262 INFO:tasks.workunit.client.1.smithi027.stdout:2/718: symlink d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/d106/l108 0 2024-04-02T04:43:02.264 INFO:tasks.workunit.client.1.smithi027.stdout:2/719: chown d1/db/d3b/d17/d2b/f74 28580 1 2024-04-02T04:43:02.266 INFO:tasks.workunit.client.1.smithi027.stdout:0/688: dwrite d4/d5d/fd2 [0,4194304] 0 2024-04-02T04:43:02.270 INFO:tasks.workunit.client.1.smithi027.stdout:3/554: link d1/de/df/f8f d1/de/df/d61/fb8 0 2024-04-02T04:43:02.275 INFO:tasks.workunit.client.1.smithi027.stdout:2/720: creat d1/db/d3b/d17/dec/f109 x:0 0 0 2024-04-02T04:43:02.276 INFO:tasks.workunit.client.1.smithi027.stdout:2/721: write d1/db/d3b/d17/d6d/dd9/dd5/dad/fd3 [743127,76522] 0 2024-04-02T04:43:02.279 INFO:tasks.workunit.client.1.smithi027.stdout:3/555: mkdir d1/de/d55/d63/db9 0 2024-04-02T04:43:02.280 INFO:tasks.workunit.client.1.smithi027.stdout:0/689: dread d4/d38/fa1 [0,4194304] 0 2024-04-02T04:43:02.289 INFO:tasks.workunit.client.1.smithi027.stdout:2/722: mkdir d1/db/d10a 0 2024-04-02T04:43:02.290 INFO:tasks.workunit.client.1.smithi027.stdout:0/690: creat d4/d16/feb x:0 0 0 2024-04-02T04:43:02.292 INFO:tasks.workunit.client.1.smithi027.stdout:3/556: symlink d1/de/d55/d63/db9/lba 0 2024-04-02T04:43:02.295 INFO:tasks.workunit.client.1.smithi027.stdout:3/557: mknod d1/de/d55/d99/cbb 0 2024-04-02T04:43:02.296 INFO:tasks.workunit.client.1.smithi027.stdout:3/558: fdatasync d1/de/d55/fa1 0 2024-04-02T04:43:02.297 INFO:tasks.workunit.client.1.smithi027.stdout:0/691: creat d4/fec x:0 0 0 2024-04-02T04:43:02.299 INFO:tasks.workunit.client.1.smithi027.stdout:0/692: write d4/d16/dd5/fe8 [87874,105940] 0 2024-04-02T04:43:02.303 INFO:tasks.workunit.client.1.smithi027.stdout:2/723: getdents d1/db/d3b/d17/dcf 0 2024-04-02T04:43:02.306 INFO:tasks.workunit.client.1.smithi027.stdout:0/693: write d4/d16/d1b/f20 [7026035,127472] 0 2024-04-02T04:43:02.310 INFO:tasks.workunit.client.1.smithi027.stdout:0/694: dread d4/d21/fbb [0,4194304] 0 2024-04-02T04:43:02.310 INFO:tasks.workunit.client.1.smithi027.stdout:2/724: link d1/db/d3b/d17/d6d/dd9/dd5/dad/cbe d1/db/d3b/d17/d6d/c10b 0 2024-04-02T04:43:02.314 INFO:tasks.workunit.client.1.smithi027.stdout:0/695: mknod d4/d21/d28/d4e/d5f/ced 0 2024-04-02T04:43:02.314 INFO:tasks.workunit.client.1.smithi027.stdout:3/559: rmdir d1/de 39 2024-04-02T04:43:02.315 INFO:tasks.workunit.client.1.smithi027.stdout:0/696: truncate d4/fec 550675 0 2024-04-02T04:43:02.315 INFO:tasks.workunit.client.1.smithi027.stdout:2/725: symlink d1/db/d3b/d61/d75/d81/l10c 0 2024-04-02T04:43:02.316 INFO:tasks.workunit.client.1.smithi027.stdout:2/726: read d1/db/f30 [4941941,112622] 0 2024-04-02T04:43:02.317 INFO:tasks.workunit.client.1.smithi027.stdout:3/560: mknod d1/d1a/cbc 0 2024-04-02T04:43:02.318 INFO:tasks.workunit.client.1.smithi027.stdout:2/727: write d1/db/d3b/d17/d68/d69/fe9 [880386,66132] 0 2024-04-02T04:43:02.320 INFO:tasks.workunit.client.1.smithi027.stdout:2/728: chown d1/db/d3b/d17/d6d/dd9/dd5/lde 243855863 1 2024-04-02T04:43:02.321 INFO:tasks.workunit.client.1.smithi027.stdout:2/729: chown d1/db/d3b/d61/lb7 243414905 1 2024-04-02T04:43:02.326 INFO:tasks.workunit.client.1.smithi027.stdout:2/730: unlink d1/db/d3b/d61/d75/f92 0 2024-04-02T04:43:02.327 INFO:tasks.workunit.client.1.smithi027.stdout:0/697: link d4/d21/d28/d4e/d5f/ced d4/d21/d28/d4e/d8c/cee 0 2024-04-02T04:43:02.327 INFO:tasks.workunit.client.1.smithi027.stdout:2/731: chown d1/db/d3b/d17/d6d/dd9/dd5/f4f 2 1 2024-04-02T04:43:02.332 INFO:tasks.workunit.client.1.smithi027.stdout:2/732: unlink d1/db/d3b/d17/d6d/dd9/dd5/dda/d8a/l8f 0 2024-04-02T04:43:02.340 INFO:tasks.workunit.client.1.smithi027.stdout:7/644: sync 2024-04-02T04:43:02.340 INFO:tasks.workunit.client.1.smithi027.stdout:4/512: sync 2024-04-02T04:43:02.346 INFO:tasks.workunit.client.1.smithi027.stdout:4/513: dread d24/d38/d47/d5c/f8c [0,4194304] 0 2024-04-02T04:43:02.347 INFO:tasks.workunit.client.1.smithi027.stdout:7/645: dwrite d2/dba/d35/f67 [0,4194304] 0 2024-04-02T04:43:02.349 INFO:tasks.workunit.client.1.smithi027.stdout:7/646: write d2/dba/da6/fcc [120176,98606] 0 2024-04-02T04:43:02.349 INFO:tasks.workunit.client.1.smithi027.stdout:4/514: readlink d24/d38/d44/d41/d49/l56 0 2024-04-02T04:43:02.353 INFO:tasks.workunit.client.1.smithi027.stdout:7/647: mkdir d2/dba/dc/d16/d2f/dcd 0 2024-04-02T04:43:02.355 INFO:tasks.workunit.client.1.smithi027.stdout:4/515: creat d24/d38/d44/d41/d5a/fab x:0 0 0 2024-04-02T04:43:02.356 INFO:tasks.workunit.client.1.smithi027.stdout:7/648: chown d2/dba/f1e 5625 1 2024-04-02T04:43:02.357 INFO:tasks.workunit.client.1.smithi027.stdout:7/649: write d2/d1c/f40 [2591895,84450] 0 2024-04-02T04:43:02.361 INFO:tasks.workunit.client.1.smithi027.stdout:3/561: write d1/de/df/d61/f6c [2371123,11044] 0 2024-04-02T04:43:02.361 INFO:tasks.workunit.client.1.smithi027.stdout:4/516: link d24/l58 d24/d38/d44/d41/d49/lac 0 2024-04-02T04:43:02.362 INFO:tasks.workunit.client.1.smithi027.stdout:3/562: dread - d1/de/df/d13/d73/fa9 zero size 2024-04-02T04:43:02.365 INFO:tasks.workunit.client.1.smithi027.stdout:4/517: fdatasync d24/f46 0 2024-04-02T04:43:02.368 INFO:tasks.workunit.client.1.smithi027.stdout:4/518: dread f21 [0,4194304] 0 2024-04-02T04:43:02.370 INFO:tasks.workunit.client.1.smithi027.stdout:0/698: write d4/d21/d28/d4e/d5f/f79 [151202,10825] 0 2024-04-02T04:43:02.374 INFO:tasks.workunit.client.1.smithi027.stdout:0/699: truncate d4/d38/fc3 476061 0 2024-04-02T04:43:02.379 INFO:tasks.workunit.client.1.smithi027.stdout:4/519: dwrite d24/d2d/f92 [0,4194304] 0 2024-04-02T04:43:02.383 INFO:tasks.workunit.client.1.smithi027.stdout:2/733: dwrite d1/db/d3b/d17/d2b/f4e [0,4194304] 0 2024-04-02T04:43:02.383 INFO:tasks.workunit.client.1.smithi027.stdout:0/700: symlink d4/d76/lef 0 2024-04-02T04:43:02.386 INFO:tasks.workunit.client.1.smithi027.stdout:2/734: dread d1/db/d3b/d17/d6d/dd9/dd5/f4f [0,4194304] 0 2024-04-02T04:43:02.387 INFO:tasks.workunit.client.1.smithi027.stdout:2/735: chown d1/db/d3b/d61/d75/d81 685 1 2024-04-02T04:43:02.390 INFO:tasks.workunit.client.1.smithi027.stdout:4/520: symlink d24/d2d/lad 0 2024-04-02T04:43:02.405 INFO:tasks.workunit.client.1.smithi027.stdout:0/701: dwrite d4/d16/d1b/fc2 [0,4194304] 0 2024-04-02T04:43:02.407 INFO:tasks.workunit.client.1.smithi027.stdout:2/736: mkdir d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/d10d 0 2024-04-02T04:43:02.409 INFO:tasks.workunit.client.1.smithi027.stdout:4/521: mknod d24/d38/d44/d82/d97/cae 0 2024-04-02T04:43:02.417 INFO:tasks.workunit.client.1.smithi027.stdout:0/702: chown d4/d16/d1b/cca 0 1 2024-04-02T04:43:02.420 INFO:tasks.workunit.client.1.smithi027.stdout:2/737: creat d1/db/d3b/d17/d6d/dd9/dd5/dda/d8a/f10e x:0 0 0 2024-04-02T04:43:02.421 INFO:tasks.workunit.client.1.smithi027.stdout:2/738: dread - d1/db/d3b/d17/d2b/fc6 zero size 2024-04-02T04:43:02.423 INFO:tasks.workunit.client.1.smithi027.stdout:2/739: fdatasync d1/db/d3b/d17/d2b/f4e 0 2024-04-02T04:43:02.423 INFO:tasks.workunit.client.1.smithi027.stdout:4/522: creat d24/d38/d44/d82/d95/faf x:0 0 0 2024-04-02T04:43:02.425 INFO:tasks.workunit.client.1.smithi027.stdout:4/523: chown d24/d2d/c5b 84763779 1 2024-04-02T04:43:02.425 INFO:tasks.workunit.client.1.smithi027.stdout:2/740: chown d1/db/d3b/d17/d6d/d84/fd8 2017888 1 2024-04-02T04:43:02.428 INFO:tasks.workunit.client.1.smithi027.stdout:0/703: creat d4/d38/d66/ff0 x:0 0 0 2024-04-02T04:43:02.429 INFO:tasks.workunit.client.1.smithi027.stdout:4/524: mkdir d24/d48/db0 0 2024-04-02T04:43:02.432 INFO:tasks.workunit.client.1.smithi027.stdout:2/741: creat d1/db/d3b/d17/d6d/dd9/dd5/dda/d8a/f10f x:0 0 0 2024-04-02T04:43:02.432 INFO:tasks.workunit.client.1.smithi027.stdout:2/742: dread - d1/db/d3b/d61/f5f zero size 2024-04-02T04:43:02.433 INFO:tasks.workunit.client.1.smithi027.stdout:0/704: rename d4/d21/l64 to d4/d21/d28/d4e/d8c/db0/lf1 0 2024-04-02T04:43:02.448 INFO:tasks.workunit.client.1.smithi027.stdout:2/743: write d1/db/d3b/d17/d12/f41 [2088884,58330] 0 2024-04-02T04:43:02.448 INFO:tasks.workunit.client.1.smithi027.stdout:0/705: write d4/d21/d85/f86 [570091,56330] 0 2024-04-02T04:43:02.451 INFO:tasks.workunit.client.1.smithi027.stdout:4/525: truncate d24/d2d/f31 1566649 0 2024-04-02T04:43:02.452 INFO:tasks.workunit.client.1.smithi027.stdout:4/526: readlink d24/d38/d47/d5c/l68 0 2024-04-02T04:43:02.454 INFO:tasks.workunit.client.1.smithi027.stdout:4/527: fsync d24/f76 0 2024-04-02T04:43:02.454 INFO:tasks.workunit.client.1.smithi027.stdout:2/744: mkdir d1/db/d3b/d17/d6d/dd9/dd5/dad/d110 0 2024-04-02T04:43:02.455 INFO:tasks.workunit.client.1.smithi027.stdout:4/528: read d24/f76 [3021923,103191] 0 2024-04-02T04:43:02.456 INFO:tasks.workunit.client.1.smithi027.stdout:4/529: readlink d24/d38/l7b 0 2024-04-02T04:43:02.457 INFO:tasks.workunit.client.1.smithi027.stdout:0/706: dwrite d4/d38/d66/d4a/f42 [0,4194304] 0 2024-04-02T04:43:02.458 INFO:tasks.workunit.client.1.smithi027.stdout:2/745: write d1/db/d3b/d17/d6d/dd9/dd5/fc5 [1446740,65337] 0 2024-04-02T04:43:02.458 INFO:tasks.workunit.client.1.smithi027.stdout:1/627: sync 2024-04-02T04:43:02.458 INFO:tasks.workunit.client.1.smithi027.stdout:5/829: sync 2024-04-02T04:43:02.459 INFO:tasks.workunit.client.1.smithi027.stdout:2/746: chown d1/db/d3b/d61/fc7 420 1 2024-04-02T04:43:02.460 INFO:tasks.workunit.client.1.smithi027.stdout:2/747: chown d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/d10d 241 1 2024-04-02T04:43:02.461 INFO:tasks.workunit.client.1.smithi027.stdout:4/530: creat d24/d38/d44/fb1 x:0 0 0 2024-04-02T04:43:02.462 INFO:tasks.workunit.client.1.smithi027.stdout:2/748: write d1/db/d3b/d17/f14 [16351157,112018] 0 2024-04-02T04:43:02.462 INFO:tasks.workunit.client.1.smithi027.stdout:4/531: stat d24/d38/d44/d82/d95/ca4 0 2024-04-02T04:43:02.463 INFO:tasks.workunit.client.1.smithi027.stdout:0/707: dread - d4/d38/da0/fb4 zero size 2024-04-02T04:43:02.463 INFO:tasks.workunit.client.1.smithi027.stdout:4/532: chown d24/d2d/f92 88 1 2024-04-02T04:43:02.467 INFO:tasks.workunit.client.1.smithi027.stdout:5/830: symlink d0/d68/d6b/d74/l11b 0 2024-04-02T04:43:02.468 INFO:tasks.workunit.client.1.smithi027.stdout:1/628: mkdir d0/d1e/d2e/d3b/d4f/dd3 0 2024-04-02T04:43:02.468 INFO:tasks.workunit.client.1.smithi027.stdout:4/533: fdatasync d24/d38/d44/f52 0 2024-04-02T04:43:02.470 INFO:tasks.workunit.client.1.smithi027.stdout:1/629: fdatasync d0/d1e/d5d/fa1 0 2024-04-02T04:43:02.470 INFO:tasks.workunit.client.1.smithi027.stdout:4/534: dread - d24/d48/f72 zero size 2024-04-02T04:43:02.471 INFO:tasks.workunit.client.1.smithi027.stdout:0/708: truncate d4/d38/d66/d2d/fac 779350 0 2024-04-02T04:43:02.472 INFO:tasks.workunit.client.1.smithi027.stdout:2/749: symlink d1/db/d3b/d17/d6d/dd9/dd5/dda/d8a/l111 0 2024-04-02T04:43:02.473 INFO:tasks.workunit.client.1.smithi027.stdout:2/750: readlink d1/db/d3b/d17/d6d/dd9/dd5/dda/dbb/l8b 0 2024-04-02T04:43:02.474 INFO:tasks.workunit.client.1.smithi027.stdout:2/751: chown d1/db/d3b/d17/d6d/dd9/dd5/dda/dbb/fe4 13449107 1 2024-04-02T04:43:02.475 INFO:tasks.workunit.client.1.smithi027.stdout:5/831: rename d0/df/d11/d65/f6c to d0/d1d/d61/d7e/dff/f11c 0 2024-04-02T04:43:02.478 INFO:tasks.workunit.client.1.smithi027.stdout:4/535: mknod d24/d38/d44/d82/d95/cb2 0 2024-04-02T04:43:02.480 INFO:tasks.workunit.client.1.smithi027.stdout:2/752: creat d1/db/d3b/d17/d6d/dd9/dd5/dda/f112 x:0 0 0 2024-04-02T04:43:02.483 INFO:tasks.workunit.client.1.smithi027.stdout:0/709: rename d4/d21/d28/fd7 to d4/de0/de2/ff2 0 2024-04-02T04:43:02.487 INFO:tasks.workunit.client.1.smithi027.stdout:1/630: dread d0/de/f80 [0,4194304] 0 2024-04-02T04:43:02.491 INFO:tasks.workunit.client.1.smithi027.stdout:0/710: mknod d4/de0/cf3 0 2024-04-02T04:43:02.496 INFO:tasks.workunit.client.1.smithi027.stdout:1/631: mknod d0/d1e/d25/d3c/cd4 0 2024-04-02T04:43:02.499 INFO:tasks.workunit.client.1.smithi027.stdout:0/711: creat d4/d38/d66/d2d/ff4 x:0 0 0 2024-04-02T04:43:02.505 INFO:tasks.workunit.client.1.smithi027.stdout:0/712: rmdir d4/d21/d28/d4e 39 2024-04-02T04:43:02.514 INFO:tasks.workunit.client.1.smithi027.stdout:0/713: dread - d4/d21/d28/d4e/d5f/fdc zero size 2024-04-02T04:43:02.514 INFO:tasks.workunit.client.1.smithi027.stdout:0/714: mkdir d4/d16/d71/df5 0 2024-04-02T04:43:02.515 INFO:tasks.workunit.client.1.smithi027.stdout:4/536: write d24/f71 [4397637,14211] 0 2024-04-02T04:43:02.518 INFO:tasks.workunit.client.1.smithi027.stdout:2/753: write d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/fd1 [1520993,124939] 0 2024-04-02T04:43:02.519 INFO:tasks.workunit.client.1.smithi027.stdout:2/754: write d1/db/d3b/fa9 [2990906,121429] 0 2024-04-02T04:43:02.522 INFO:tasks.workunit.client.1.smithi027.stdout:1/632: truncate d0/f54 1068886 0 2024-04-02T04:43:02.524 INFO:tasks.workunit.client.1.smithi027.stdout:4/537: symlink d24/d38/d44/d60/lb3 0 2024-04-02T04:43:02.524 INFO:tasks.workunit.client.1.smithi027.stdout:4/538: dread - d24/d38/d44/d82/f90 zero size 2024-04-02T04:43:02.530 INFO:tasks.workunit.client.1.smithi027.stdout:2/755: mkdir d1/db/d3b/d17/d6d/dd9/dd5/dda/dbb/db4/ddf/d113 0 2024-04-02T04:43:02.530 INFO:tasks.workunit.client.1.smithi027.stdout:4/539: dwrite fc [0,4194304] 0 2024-04-02T04:43:02.534 INFO:tasks.workunit.client.1.smithi027.stdout:0/715: dread d4/d38/d66/d2d/d4c/f9c [0,4194304] 0 2024-04-02T04:43:02.540 INFO:tasks.workunit.client.1.smithi027.stdout:4/540: truncate f2 4688708 0 2024-04-02T04:43:02.540 INFO:tasks.workunit.client.1.smithi027.stdout:0/716: dwrite d4/f23 [0,4194304] 0 2024-04-02T04:43:02.545 INFO:tasks.workunit.client.1.smithi027.stdout:2/756: rename d1/db/fa0 to d1/db/d3b/d17/d2b/f114 0 2024-04-02T04:43:02.550 INFO:tasks.workunit.client.1.smithi027.stdout:1/633: mknod d0/de/d9e/cd5 0 2024-04-02T04:43:02.552 INFO:tasks.workunit.client.1.smithi027.stdout:4/541: mkdir d24/d38/d47/d62/db4 0 2024-04-02T04:43:02.558 INFO:tasks.workunit.client.1.smithi027.stdout:2/757: mknod d1/db/d3b/d17/d6d/df1/c115 0 2024-04-02T04:43:02.560 INFO:tasks.workunit.client.1.smithi027.stdout:0/717: dwrite d4/d16/d71/fdf [0,4194304] 0 2024-04-02T04:43:02.561 INFO:tasks.workunit.client.1.smithi027.stdout:2/758: write d1/db/d3b/d17/d6d/dd9/dd5/fc1 [1117367,2935] 0 2024-04-02T04:43:02.563 INFO:tasks.workunit.client.1.smithi027.stdout:2/759: chown d1/db/d3b/d17/d6d/dd9/dd5/dda/d8a/f10f 986 1 2024-04-02T04:43:02.564 INFO:tasks.workunit.client.1.smithi027.stdout:4/542: fdatasync d24/d48/f4e 0 2024-04-02T04:43:02.566 INFO:tasks.workunit.client.1.smithi027.stdout:4/543: readlink d24/d38/d47/d62/l79 0 2024-04-02T04:43:02.567 INFO:tasks.workunit.client.1.smithi027.stdout:2/760: read - d1/db/d3b/d17/d6d/dd9/dd5/dda/d8a/f10e zero size 2024-04-02T04:43:02.575 INFO:tasks.workunit.client.1.smithi027.stdout:2/761: dread d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/fd1 [0,4194304] 0 2024-04-02T04:43:02.579 INFO:tasks.workunit.client.1.smithi027.stdout:4/544: readlink d24/d38/d44/d41/d49/lac 0 2024-04-02T04:43:02.583 INFO:tasks.workunit.client.1.smithi027.stdout:2/762: dwrite d1/db/d3b/d17/d6d/dd9/dd5/dda/d8a/f10f [0,4194304] 0 2024-04-02T04:43:02.585 INFO:tasks.workunit.client.1.smithi027.stdout:1/634: dread d0/d1e/d25/d2b/fae [0,4194304] 0 2024-04-02T04:43:02.587 INFO:tasks.workunit.client.1.smithi027.stdout:1/635: truncate d0/fbd 758095 0 2024-04-02T04:43:02.594 INFO:tasks.workunit.client.1.smithi027.stdout:2/763: rmdir d1/db/d3b 39 2024-04-02T04:43:02.597 INFO:tasks.workunit.client.1.smithi027.stdout:4/545: unlink d24/d38/d44/d41/f4b 0 2024-04-02T04:43:02.599 INFO:tasks.workunit.client.1.smithi027.stdout:1/636: mknod d0/d1e/d2e/d3b/d4f/cd6 0 2024-04-02T04:43:02.601 INFO:tasks.workunit.client.1.smithi027.stdout:2/764: write d1/db/d3b/d17/d2b/db8/fa4 [167268,106343] 0 2024-04-02T04:43:02.603 INFO:tasks.workunit.client.1.smithi027.stdout:2/765: dread - d1/db/d3b/d17/d6d/df1/f103 zero size 2024-04-02T04:43:02.609 INFO:tasks.workunit.client.1.smithi027.stdout:2/766: dwrite d1/db/d3b/d17/f4a [0,4194304] 0 2024-04-02T04:43:02.611 INFO:tasks.workunit.client.1.smithi027.stdout:9/728: sync 2024-04-02T04:43:02.611 INFO:tasks.workunit.client.1.smithi027.stdout:8/650: sync 2024-04-02T04:43:02.611 INFO:tasks.workunit.client.1.smithi027.stdout:6/568: sync 2024-04-02T04:43:02.611 INFO:tasks.workunit.client.1.smithi027.stdout:1/637: unlink d0/de/cb4 0 2024-04-02T04:43:02.614 INFO:tasks.workunit.client.1.smithi027.stdout:4/546: creat d24/fb5 x:0 0 0 2024-04-02T04:43:02.621 INFO:tasks.workunit.client.1.smithi027.stdout:0/718: write d4/d38/d66/d2d/f3c [120037,71193] 0 2024-04-02T04:43:02.622 INFO:tasks.workunit.client.1.smithi027.stdout:4/547: dwrite d24/d38/d44/d60/f7c [0,4194304] 0 2024-04-02T04:43:02.623 INFO:tasks.workunit.client.1.smithi027.stdout:8/651: rmdir d5/d8/db/dd/d5f/d27/d31/d5b/d6c 39 2024-04-02T04:43:02.628 INFO:tasks.workunit.client.1.smithi027.stdout:7/650: sync 2024-04-02T04:43:02.628 INFO:tasks.workunit.client.1.smithi027.stdout:3/563: sync 2024-04-02T04:43:02.628 INFO:tasks.workunit.client.1.smithi027.stdout:5/832: sync 2024-04-02T04:43:02.632 INFO:tasks.workunit.client.1.smithi027.stdout:1/638: symlink d0/ld7 0 2024-04-02T04:43:02.638 INFO:tasks.workunit.client.1.smithi027.stdout:0/719: symlink d4/d21/d28/d4e/d8c/lf6 0 2024-04-02T04:43:02.640 INFO:tasks.workunit.client.1.smithi027.stdout:0/720: truncate d4/d21/d28/f54 1363951 0 2024-04-02T04:43:02.642 INFO:tasks.workunit.client.1.smithi027.stdout:8/652: mkdir d5/db5/ddc 0 2024-04-02T04:43:02.647 INFO:tasks.workunit.client.1.smithi027.stdout:2/767: write d1/db/d3b/f51 [110613,83948] 0 2024-04-02T04:43:02.655 INFO:tasks.workunit.client.1.smithi027.stdout:9/729: truncate d1a/f38 650830 0 2024-04-02T04:43:02.681 INFO:tasks.workunit.client.1.smithi027.stdout:7/651: read d2/dba/dc/faf [108774,57786] 0 2024-04-02T04:43:02.682 INFO:tasks.workunit.client.1.smithi027.stdout:5/833: creat d0/df/d2f/f11d x:0 0 0 2024-04-02T04:43:02.683 INFO:tasks.workunit.client.1.smithi027.stdout:3/564: write d1/de/df/d13/f85 [327650,128760] 0 2024-04-02T04:43:02.686 INFO:tasks.workunit.client.1.smithi027.stdout:6/569: rename d13/d1b/d24/d28/c4d to d13/d1b/cc3 0 2024-04-02T04:43:02.688 INFO:tasks.workunit.client.1.smithi027.stdout:1/639: creat d0/d1e/d25/d3c/d64/fd8 x:0 0 0 2024-04-02T04:43:02.697 INFO:tasks.workunit.client.1.smithi027.stdout:2/768: unlink d1/db/d3b/d17/d6d/dd9/dd5/fc1 0 2024-04-02T04:43:02.699 INFO:tasks.workunit.client.1.smithi027.stdout:9/730: fsync d1a/d1d/d3c/d6e/db8/fb9 0 2024-04-02T04:43:02.702 INFO:tasks.workunit.client.1.smithi027.stdout:5/834: mkdir d0/df/d11/d13/d9a/d9c/db8/d11e 0 2024-04-02T04:43:02.705 INFO:tasks.workunit.client.1.smithi027.stdout:3/565: creat d1/d1a/fbd x:0 0 0 2024-04-02T04:43:02.706 INFO:tasks.workunit.client.1.smithi027.stdout:6/570: mknod d13/d1b/d48/d75/d9b/cc4 0 2024-04-02T04:43:02.707 INFO:tasks.workunit.client.1.smithi027.stdout:1/640: mkdir d0/de/d52/d74/dd9 0 2024-04-02T04:43:02.709 INFO:tasks.workunit.client.1.smithi027.stdout:4/548: rmdir d24/d38/d44/d82/d97/d9f 0 2024-04-02T04:43:02.715 INFO:tasks.workunit.client.1.smithi027.stdout:7/652: write d2/dba/dc/d16/d3a/d4d/f58 [2459631,50233] 0 2024-04-02T04:43:02.716 INFO:tasks.workunit.client.1.smithi027.stdout:7/653: chown d2/df/f85 100 1 2024-04-02T04:43:02.718 INFO:tasks.workunit.client.1.smithi027.stdout:9/731: creat d1a/d1d/d29/d37/d20/d7e/feb x:0 0 0 2024-04-02T04:43:02.719 INFO:tasks.workunit.client.1.smithi027.stdout:9/732: readlink d1a/d1d/d29/lae 0 2024-04-02T04:43:02.724 INFO:tasks.workunit.client.1.smithi027.stdout:5/835: creat d0/df/d11/d8c/f11f x:0 0 0 2024-04-02T04:43:02.725 INFO:tasks.workunit.client.1.smithi027.stdout:3/566: creat d1/de/df/d9e/fbe x:0 0 0 2024-04-02T04:43:02.726 INFO:tasks.workunit.client.1.smithi027.stdout:3/567: readlink d1/de/laf 0 2024-04-02T04:43:02.728 INFO:tasks.workunit.client.1.smithi027.stdout:1/641: fdatasync d0/d1e/d2e/d3e/f82 0 2024-04-02T04:43:02.729 INFO:tasks.workunit.client.1.smithi027.stdout:4/549: dread - d24/d48/f5d zero size 2024-04-02T04:43:02.730 INFO:tasks.workunit.client.1.smithi027.stdout:0/721: creat d4/d38/ff7 x:0 0 0 2024-04-02T04:43:02.730 INFO:tasks.workunit.client.1.smithi027.stdout:8/653: dwrite d5/d8/f2e [0,4194304] 0 2024-04-02T04:43:02.732 INFO:tasks.workunit.client.1.smithi027.stdout:2/769: mknod d1/db/d3b/d17/d6d/dd9/dd5/dad/d110/c116 0 2024-04-02T04:43:02.733 INFO:tasks.workunit.client.1.smithi027.stdout:7/654: symlink d2/d1d/d92/lce 0 2024-04-02T04:43:02.735 INFO:tasks.workunit.client.1.smithi027.stdout:9/733: write d1a/d1d/d29/d37/d20/f30 [1158805,30067] 0 2024-04-02T04:43:02.735 INFO:tasks.workunit.client.1.smithi027.stdout:1/642: dread d0/de/f80 [0,4194304] 0 2024-04-02T04:43:02.737 INFO:tasks.workunit.client.1.smithi027.stdout:1/643: dread - d0/d1e/d2e/d3e/d5f/fc7 zero size 2024-04-02T04:43:02.738 INFO:tasks.workunit.client.1.smithi027.stdout:5/836: stat d0/df/d29/da5/da6/f103 0 2024-04-02T04:43:02.740 INFO:tasks.workunit.client.1.smithi027.stdout:6/571: write d13/d1b/d24/f41 [2337216,40268] 0 2024-04-02T04:43:02.748 INFO:tasks.workunit.client.1.smithi027.stdout:9/734: mkdir d1a/db6/dec 0 2024-04-02T04:43:02.750 INFO:tasks.workunit.client.1.smithi027.stdout:0/722: mknod d4/d38/cf8 0 2024-04-02T04:43:02.750 INFO:tasks.workunit.client.1.smithi027.stdout:4/550: dread d24/f2c [0,4194304] 0 2024-04-02T04:43:02.752 INFO:tasks.workunit.client.1.smithi027.stdout:0/723: write d4/d38/ff7 [664261,27264] 0 2024-04-02T04:43:02.755 INFO:tasks.workunit.client.1.smithi027.stdout:3/568: truncate d1/de/df/d61/f6e 677819 0 2024-04-02T04:43:02.756 INFO:tasks.workunit.client.1.smithi027.stdout:7/655: fsync d2/d1d/f25 0 2024-04-02T04:43:02.758 INFO:tasks.workunit.client.1.smithi027.stdout:2/770: symlink d1/db/d3b/d17/d6d/dd9/dd5/dda/dbb/db4/ddf/d113/l117 0 2024-04-02T04:43:02.759 INFO:tasks.workunit.client.1.smithi027.stdout:8/654: dread d5/d8/db/dd/d5f/d27/d31/d5b/f67 [0,4194304] 0 2024-04-02T04:43:02.765 INFO:tasks.workunit.client.1.smithi027.stdout:4/551: creat d24/d38/d44/d82/d95/fb6 x:0 0 0 2024-04-02T04:43:02.765 INFO:tasks.workunit.client.1.smithi027.stdout:0/724: mknod d4/d5d/cf9 0 2024-04-02T04:43:02.765 INFO:tasks.workunit.client.1.smithi027.stdout:4/552: write d24/fb5 [504741,82926] 0 2024-04-02T04:43:02.767 INFO:tasks.workunit.client.1.smithi027.stdout:0/725: truncate d4/d16/d1b/fb7 978176 0 2024-04-02T04:43:02.768 INFO:tasks.workunit.client.1.smithi027.stdout:3/569: symlink d1/de/d55/d63/db9/lbf 0 2024-04-02T04:43:02.772 INFO:tasks.workunit.client.1.smithi027.stdout:1/644: rename d0/d1e/d5d/d60 to d0/d1e/d2e/d3b/dda 0 2024-04-02T04:43:02.772 INFO:tasks.workunit.client.1.smithi027.stdout:9/735: dwrite d1a/d1d/d3c/f94 [0,4194304] 0 2024-04-02T04:43:02.773 INFO:tasks.workunit.client.1.smithi027.stdout:2/771: creat d1/db/d3b/d17/d6d/d84/f118 x:0 0 0 2024-04-02T04:43:02.773 INFO:tasks.workunit.client.1.smithi027.stdout:1/645: chown d0/d1e/d2e/d3e/lb9 0 1 2024-04-02T04:43:02.776 INFO:tasks.workunit.client.1.smithi027.stdout:6/572: link d13/d1b/d24/d25/d56/fa0 d13/d1b/d24/d25/d56/fc5 0 2024-04-02T04:43:02.780 INFO:tasks.workunit.client.1.smithi027.stdout:1/646: dwrite d0/d1e/d2e/d3e/f6e [0,4194304] 0 2024-04-02T04:43:02.782 INFO:tasks.workunit.client.1.smithi027.stdout:7/656: creat d2/dba/dc/d16/d2f/dcd/fcf x:0 0 0 2024-04-02T04:43:02.783 INFO:tasks.workunit.client.1.smithi027.stdout:1/647: truncate d0/d1e/d5d/fa1 189017 0 2024-04-02T04:43:02.787 INFO:tasks.workunit.client.1.smithi027.stdout:8/655: symlink d5/d8/db/dd/d5f/ldd 0 2024-04-02T04:43:02.790 INFO:tasks.workunit.client.1.smithi027.stdout:0/726: write f2 [10322181,13895] 0 2024-04-02T04:43:02.790 INFO:tasks.workunit.client.1.smithi027.stdout:9/736: fsync d1a/d1d/d3c/d6e/fc6 0 2024-04-02T04:43:02.796 INFO:tasks.workunit.client.1.smithi027.stdout:4/553: link d24/d48/f72 d24/d38/d44/d82/fb7 0 2024-04-02T04:43:02.798 INFO:tasks.workunit.client.1.smithi027.stdout:3/570: getdents d1/de/df/d13/db3 0 2024-04-02T04:43:02.799 INFO:tasks.workunit.client.1.smithi027.stdout:8/656: chown d5/d8/db/f76 7454133 1 2024-04-02T04:43:02.803 INFO:tasks.workunit.client.1.smithi027.stdout:9/737: creat d1a/d1d/d29/d37/d9b/fed x:0 0 0 2024-04-02T04:43:02.807 INFO:tasks.workunit.client.1.smithi027.stdout:4/554: creat d24/d38/d44/d60/fb8 x:0 0 0 2024-04-02T04:43:02.810 INFO:tasks.workunit.client.1.smithi027.stdout:8/657: dwrite d5/d8/db/df/fd0 [0,4194304] 0 2024-04-02T04:43:02.810 INFO:tasks.workunit.client.1.smithi027.stdout:2/772: link d1/db/d3b/d17/d6d/dd9/dd5/dda/d8a/l111 d1/db/d3b/d17/d6d/dd9/dd5/dda/dbb/db4/l119 0 2024-04-02T04:43:02.813 INFO:tasks.workunit.client.1.smithi027.stdout:6/573: write d13/f17 [3525657,31795] 0 2024-04-02T04:43:02.814 INFO:tasks.workunit.client.1.smithi027.stdout:9/738: dwrite d1a/d1d/d29/d56/d8b/fa4 [0,4194304] 0 2024-04-02T04:43:02.814 INFO:tasks.workunit.client.1.smithi027.stdout:3/571: creat d1/de/d55/fc0 x:0 0 0 2024-04-02T04:43:02.816 INFO:tasks.workunit.client.1.smithi027.stdout:8/658: fdatasync d5/d8/db/f8e 0 2024-04-02T04:43:02.817 INFO:tasks.workunit.client.1.smithi027.stdout:4/555: mknod d24/d48/db0/cb9 0 2024-04-02T04:43:02.819 INFO:tasks.workunit.client.1.smithi027.stdout:4/556: chown d24/d38/d44/d60/f7c 158449 1 2024-04-02T04:43:02.820 INFO:tasks.workunit.client.1.smithi027.stdout:2/773: unlink d1/db/d3b/d17/l33 0 2024-04-02T04:43:02.820 INFO:tasks.workunit.client.1.smithi027.stdout:0/727: write d4/d38/d66/d4a/f61 [346660,51600] 0 2024-04-02T04:43:02.822 INFO:tasks.workunit.client.1.smithi027.stdout:6/574: symlink d13/d1b/d24/d28/d46/lc6 0 2024-04-02T04:43:02.825 INFO:tasks.workunit.client.1.smithi027.stdout:9/739: mknod d1a/db6/d8c/dcf/cee 0 2024-04-02T04:43:02.828 INFO:tasks.workunit.client.1.smithi027.stdout:3/572: dread d1/de/df/f24 [0,4194304] 0 2024-04-02T04:43:02.832 INFO:tasks.workunit.client.1.smithi027.stdout:0/728: symlink d4/d38/d66/d2d/d81/lfa 0 2024-04-02T04:43:02.832 INFO:tasks.workunit.client.1.smithi027.stdout:6/575: mknod d13/d1b/d24/d25/d42/cc7 0 2024-04-02T04:43:02.833 INFO:tasks.workunit.client.1.smithi027.stdout:0/729: chown d4/d38/ff7 143247705 1 2024-04-02T04:43:02.834 INFO:tasks.workunit.client.1.smithi027.stdout:6/576: dread - d13/d1b/f8e zero size 2024-04-02T04:43:02.835 INFO:tasks.workunit.client.1.smithi027.stdout:4/557: dread d24/d38/d47/d62/f9a [0,4194304] 0 2024-04-02T04:43:02.836 INFO:tasks.workunit.client.1.smithi027.stdout:2/774: truncate d1/db/d3b/d17/d2b/db8/f9f 3526228 0 2024-04-02T04:43:02.838 INFO:tasks.workunit.client.1.smithi027.stdout:4/558: rename d24/d38/d44/d82/da9 to d24/d38/d44/d82/da9/dba 22 2024-04-02T04:43:02.838 INFO:tasks.workunit.client.1.smithi027.stdout:9/740: mkdir d1a/d1d/d29/d50/d73/def 0 2024-04-02T04:43:02.842 INFO:tasks.workunit.client.1.smithi027.stdout:8/659: creat d5/d8/db/dd5/fde x:0 0 0 2024-04-02T04:43:02.843 INFO:tasks.workunit.client.1.smithi027.stdout:3/573: mknod d1/de/df/d9e/da0/cc1 0 2024-04-02T04:43:02.844 INFO:tasks.workunit.client.1.smithi027.stdout:3/574: chown d1/de/d55/f75 11 1 2024-04-02T04:43:02.845 INFO:tasks.workunit.client.1.smithi027.stdout:0/730: mkdir d4/d21/d85/dfb 0 2024-04-02T04:43:02.850 INFO:tasks.workunit.client.1.smithi027.stdout:9/741: read d1a/d1d/d29/d37/f1c [1412122,53387] 0 2024-04-02T04:43:02.854 INFO:tasks.workunit.client.1.smithi027.stdout:6/577: mkdir d13/d1b/d24/d25/d56/dc8 0 2024-04-02T04:43:02.855 INFO:tasks.workunit.client.1.smithi027.stdout:6/578: read - d13/d1b/d24/d25/d42/fa9 zero size 2024-04-02T04:43:02.856 INFO:tasks.workunit.client.1.smithi027.stdout:8/660: dread d5/d8/db/df/f79 [0,4194304] 0 2024-04-02T04:43:02.857 INFO:tasks.workunit.client.1.smithi027.stdout:8/661: write d5/db5/fd8 [146246,23291] 0 2024-04-02T04:43:02.859 INFO:tasks.workunit.client.1.smithi027.stdout:8/662: write d5/d8/db/dd/d5f/d27/d31/d5b/dc4/f5d [3504361,6676] 0 2024-04-02T04:43:02.860 INFO:tasks.workunit.client.1.smithi027.stdout:4/559: truncate d24/d38/d47/d5c/f7e 1829413 0 2024-04-02T04:43:02.862 INFO:tasks.workunit.client.1.smithi027.stdout:9/742: creat d1a/d1d/d29/d37/d3a/ff0 x:0 0 0 2024-04-02T04:43:02.863 INFO:tasks.workunit.client.1.smithi027.stdout:6/579: dread d13/d1b/d24/d25/d27/f97 [0,4194304] 0 2024-04-02T04:43:02.863 INFO:tasks.workunit.client.1.smithi027.stdout:9/743: dread - d1a/d1d/db7/fe5 zero size 2024-04-02T04:43:02.864 INFO:tasks.workunit.client.1.smithi027.stdout:6/580: truncate d13/d1b/d24/d28/d46/f8f 531576 0 2024-04-02T04:43:02.867 INFO:tasks.workunit.client.1.smithi027.stdout:2/775: rename d1/db/d3b/d17/d6d/dd9/dd5/dda/dbb/c89 to d1/c11a 0 2024-04-02T04:43:02.874 INFO:tasks.workunit.client.1.smithi027.stdout:4/560: unlink d24/d38/d44/fb1 0 2024-04-02T04:43:02.875 INFO:tasks.workunit.client.1.smithi027.stdout:0/731: truncate d4/d21/d28/f75 7557204 0 2024-04-02T04:43:02.885 INFO:tasks.workunit.client.1.smithi027.stdout:2/776: rmdir d1/db/d3b/d17/d6d/dd9/dd5/dda 39 2024-04-02T04:43:02.886 INFO:tasks.workunit.client.1.smithi027.stdout:2/777: chown d1/db/d3b/d17/d12/l36 360840 1 2024-04-02T04:43:02.890 INFO:tasks.workunit.client.1.smithi027.stdout:0/732: fdatasync d4/d21/fbb 0 2024-04-02T04:43:02.891 INFO:tasks.workunit.client.1.smithi027.stdout:9/744: dwrite d1a/d1d/d29/d50/d73/fb1 [0,4194304] 0 2024-04-02T04:43:02.891 INFO:tasks.workunit.client.1.smithi027.stdout:0/733: chown d4/de0/fb8 7 1 2024-04-02T04:43:02.893 INFO:tasks.workunit.client.1.smithi027.stdout:8/663: mkdir d5/d8/db/dd/d5f/d27/d31/d5b/d6c/ddf 0 2024-04-02T04:43:02.895 INFO:tasks.workunit.client.1.smithi027.stdout:6/581: mkdir d13/d1b/d24/d25/d56/dc8/dc9 0 2024-04-02T04:43:02.897 INFO:tasks.workunit.client.1.smithi027.stdout:6/582: chown d13/d1b/d48/d58/fa2 5 1 2024-04-02T04:43:02.898 INFO:tasks.workunit.client.1.smithi027.stdout:2/778: dread - d1/db/d3b/d17/d6d/dd9/dd5/dda/d8a/f10e zero size 2024-04-02T04:43:02.898 INFO:tasks.workunit.client.1.smithi027.stdout:0/734: dread d4/d21/d28/f48 [0,4194304] 0 2024-04-02T04:43:02.899 INFO:tasks.workunit.client.1.smithi027.stdout:4/561: fsync d24/f33 0 2024-04-02T04:43:02.901 INFO:tasks.workunit.client.1.smithi027.stdout:0/735: truncate d4/d21/f8f 1153532 0 2024-04-02T04:43:02.901 INFO:tasks.workunit.client.1.smithi027.stdout:4/562: dread - d24/d38/d44/d41/f80 zero size 2024-04-02T04:43:02.902 INFO:tasks.workunit.client.1.smithi027.stdout:0/736: chown d4/d21/d85/db3/ce1 96253307 1 2024-04-02T04:43:02.902 INFO:tasks.workunit.client.1.smithi027.stdout:4/563: chown d24/d2d/lad 1351378 1 2024-04-02T04:43:02.903 INFO:tasks.workunit.client.1.smithi027.stdout:4/564: write d24/f76 [2088949,102825] 0 2024-04-02T04:43:02.904 INFO:tasks.workunit.client.1.smithi027.stdout:8/664: stat d5/d8/l3c 0 2024-04-02T04:43:02.905 INFO:tasks.workunit.client.1.smithi027.stdout:3/575: rename d1/de/d55/d63/f9f to d1/de/df/fc2 0 2024-04-02T04:43:02.906 INFO:tasks.workunit.client.1.smithi027.stdout:6/583: fsync d13/d1b/d24/d28/f35 0 2024-04-02T04:43:02.907 INFO:tasks.workunit.client.1.smithi027.stdout:6/584: write d13/f62 [1576072,19737] 0 2024-04-02T04:43:02.910 INFO:tasks.workunit.client.1.smithi027.stdout:2/779: symlink d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/l11b 0 2024-04-02T04:43:02.912 INFO:tasks.workunit.client.1.smithi027.stdout:9/745: mknod d1a/d1d/d29/cf1 0 2024-04-02T04:43:02.912 INFO:tasks.workunit.client.1.smithi027.stdout:9/746: stat d1a/d1d/c8e 0 2024-04-02T04:43:02.912 INFO:tasks.workunit.client.1.smithi027.stdout:9/747: readlink d1a/l8d 0 2024-04-02T04:43:02.913 INFO:tasks.workunit.client.1.smithi027.stdout:9/748: chown d1a/d1d/d29/d37/d20/dcc/dd5 21 1 2024-04-02T04:43:02.914 INFO:tasks.workunit.client.1.smithi027.stdout:9/749: write d1a/db6/faf [413070,85775] 0 2024-04-02T04:43:02.915 INFO:tasks.workunit.client.1.smithi027.stdout:0/737: symlink d4/d69/lfc 0 2024-04-02T04:43:02.923 INFO:tasks.workunit.client.1.smithi027.stdout:8/665: symlink d5/d8/db/dd/d5f/d91/d9f/le0 0 2024-04-02T04:43:02.923 INFO:tasks.workunit.client.1.smithi027.stdout:3/576: symlink d1/de/d55/d63/db9/lc3 0 2024-04-02T04:43:02.925 INFO:tasks.workunit.client.1.smithi027.stdout:6/585: mknod d13/d1b/d24/d25/d27/cca 0 2024-04-02T04:43:02.927 INFO:tasks.workunit.client.1.smithi027.stdout:9/750: fdatasync d1a/f1f 0 2024-04-02T04:43:02.928 INFO:tasks.workunit.client.1.smithi027.stdout:9/751: readlink d1a/l43 0 2024-04-02T04:43:02.928 INFO:tasks.workunit.client.1.smithi027.stdout:4/565: fdatasync d24/f3a 0 2024-04-02T04:43:02.933 INFO:tasks.workunit.client.1.smithi027.stdout:2/780: symlink d1/db/d3b/d17/l11c 0 2024-04-02T04:43:02.942 INFO:tasks.workunit.client.1.smithi027.stdout:3/577: truncate d1/f8 4151170 0 2024-04-02T04:43:02.943 INFO:tasks.workunit.client.1.smithi027.stdout:4/566: dread d24/d38/d44/d41/d49/f83 [4194304,4194304] 0 2024-04-02T04:43:02.946 INFO:tasks.workunit.client.1.smithi027.stdout:2/781: mknod d1/db/d3b/d17/d6d/dd9/dd5/dda/c11d 0 2024-04-02T04:43:02.948 INFO:tasks.workunit.client.1.smithi027.stdout:6/586: mknod d13/d1b/d48/ccb 0 2024-04-02T04:43:02.950 INFO:tasks.workunit.client.1.smithi027.stdout:6/587: read - d13/d1b/d24/d25/d27/f91 zero size 2024-04-02T04:43:02.955 INFO:tasks.workunit.client.1.smithi027.stdout:9/752: dwrite d1a/d1d/d29/d37/d99/fc5 [0,4194304] 0 2024-04-02T04:43:02.957 INFO:tasks.workunit.client.1.smithi027.stdout:9/753: write d1a/d1d/f4b [3100253,16619] 0 2024-04-02T04:43:02.958 INFO:tasks.workunit.client.1.smithi027.stdout:2/782: mknod d1/db/d3b/d17/d2b/db8/c11e 0 2024-04-02T04:43:02.958 INFO:tasks.workunit.client.1.smithi027.stdout:9/754: chown f12 21 1 2024-04-02T04:43:02.961 INFO:tasks.workunit.client.1.smithi027.stdout:9/755: write d1a/d1d/d29/d56/d8b/f90 [838886,10158] 0 2024-04-02T04:43:02.962 INFO:tasks.workunit.client.1.smithi027.stdout:4/567: dread f14 [0,4194304] 0 2024-04-02T04:43:02.963 INFO:tasks.workunit.client.1.smithi027.stdout:9/756: stat d1a/d1d/d29/d37/fa1 0 2024-04-02T04:43:02.971 INFO:tasks.workunit.client.1.smithi027.stdout:6/588: dread d13/d1b/d24/d25/d27/f51 [0,4194304] 0 2024-04-02T04:43:02.972 INFO:tasks.workunit.client.1.smithi027.stdout:9/757: dwrite d1a/d1d/d3c/d6e/db8/fb9 [0,4194304] 0 2024-04-02T04:43:02.980 INFO:tasks.workunit.client.1.smithi027.stdout:9/758: rename f12 to d1a/d1d/d29/d37/d9b/ff2 0 2024-04-02T04:43:02.980 INFO:tasks.workunit.client.1.smithi027.stdout:9/759: chown d1a/d1d/d3c/l44 18807 1 2024-04-02T04:43:02.982 INFO:tasks.workunit.client.1.smithi027.stdout:9/760: write d1a/db6/f8f [28635,43975] 0 2024-04-02T04:43:02.982 INFO:tasks.workunit.client.1.smithi027.stdout:4/568: creat d24/fbb x:0 0 0 2024-04-02T04:43:02.984 INFO:tasks.workunit.client.1.smithi027.stdout:2/783: write d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/dc0/dc3/fe8 [377233,63712] 0 2024-04-02T04:43:02.984 INFO:tasks.workunit.client.1.smithi027.stdout:6/589: write d13/fbf [3928291,97537] 0 2024-04-02T04:43:02.986 INFO:tasks.workunit.client.1.smithi027.stdout:4/569: mkdir d24/d38/d44/dbc 0 2024-04-02T04:43:02.988 INFO:tasks.workunit.client.1.smithi027.stdout:9/761: rename d1a/d1d/d29/d37/fda to d1a/db6/ff3 0 2024-04-02T04:43:02.990 INFO:tasks.workunit.client.1.smithi027.stdout:2/784: unlink d1/db/d3b/d17/f2c 0 2024-04-02T04:43:02.990 INFO:tasks.workunit.client.1.smithi027.stdout:2/785: fsync d1/db/d3b/d17/d68/d69/fd6 0 2024-04-02T04:43:02.991 INFO:tasks.workunit.client.1.smithi027.stdout:4/570: mknod d24/d48/cbd 0 2024-04-02T04:43:02.994 INFO:tasks.workunit.client.1.smithi027.stdout:6/590: rename d13/d1b/d8c to d13/d1b/d24/d28/d46/dcc 0 2024-04-02T04:43:02.995 INFO:tasks.workunit.client.1.smithi027.stdout:9/762: fdatasync d1a/d1d/d29/d37/d9b/ff2 0 2024-04-02T04:43:02.996 INFO:tasks.workunit.client.1.smithi027.stdout:2/786: readlink d1/l9 0 2024-04-02T04:43:02.996 INFO:tasks.workunit.client.1.smithi027.stdout:4/571: dread fc [0,4194304] 0 2024-04-02T04:43:02.996 INFO:tasks.workunit.client.1.smithi027.stdout:6/591: chown d13/d1b/d48/d75/d9b/lae 808 1 2024-04-02T04:43:02.998 INFO:tasks.workunit.client.1.smithi027.stdout:4/572: dread - d24/d38/d47/fa3 zero size 2024-04-02T04:43:03.000 INFO:tasks.workunit.client.1.smithi027.stdout:2/787: rmdir d1/db/d3b/d61/d75/d81 39 2024-04-02T04:43:03.002 INFO:tasks.workunit.client.1.smithi027.stdout:6/592: rename d13/d1b/d24/d28/d46/f59 to d13/d1b/d24/d25/d56/dc8/dc9/fcd 0 2024-04-02T04:43:03.003 INFO:tasks.workunit.client.1.smithi027.stdout:6/593: readlink d13/d1b/d24/d25/d27/l79 0 2024-04-02T04:43:03.007 INFO:tasks.workunit.client.1.smithi027.stdout:2/788: mkdir d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/d10d/d11f 0 2024-04-02T04:43:03.008 INFO:tasks.workunit.client.1.smithi027.stdout:6/594: write d13/d1b/d24/d25/d27/f51 [1972688,60024] 0 2024-04-02T04:43:03.009 INFO:tasks.workunit.client.1.smithi027.stdout:9/763: link d1a/d1d/d29/d37/d9b/la0 d1a/db6/d8c/lf4 0 2024-04-02T04:43:03.017 INFO:tasks.workunit.client.1.smithi027.stdout:4/573: dread d24/d38/d44/d60/f6f [0,4194304] 0 2024-04-02T04:43:03.018 INFO:tasks.workunit.client.1.smithi027.stdout:6/595: dread d13/f14 [0,4194304] 0 2024-04-02T04:43:03.019 INFO:tasks.workunit.client.1.smithi027.stdout:2/789: link d1/db/d3b/d17/d6d/dd9/dd5/dad/l3d d1/db/d3b/d17/d2b/db8/l120 0 2024-04-02T04:43:03.025 INFO:tasks.workunit.client.1.smithi027.stdout:9/764: dread d1a/d1d/d29/f4f [0,4194304] 0 2024-04-02T04:43:03.026 INFO:tasks.workunit.client.1.smithi027.stdout:2/790: creat d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/de1/f121 x:0 0 0 2024-04-02T04:43:03.027 INFO:tasks.workunit.client.1.smithi027.stdout:2/791: dread - d1/db/d3b/d83/ff8 zero size 2024-04-02T04:43:03.033 INFO:tasks.workunit.client.1.smithi027.stdout:2/792: fdatasync d1/db/d3b/f1c 0 2024-04-02T04:43:03.038 INFO:tasks.workunit.client.1.smithi027.stdout:2/793: fsync d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/de1/f121 0 2024-04-02T04:43:03.038 INFO:tasks.workunit.client.1.smithi027.stdout:9/765: link d1a/d1d/d29/d50/c9c d1a/d1d/d29/d37/d99/de8/cf5 0 2024-04-02T04:43:03.039 INFO:tasks.workunit.client.1.smithi027.stdout:6/596: truncate d13/d1b/d24/d25/d42/f60 847473 0 2024-04-02T04:43:03.042 INFO:tasks.workunit.client.1.smithi027.stdout:4/574: dwrite d24/d48/f4e [0,4194304] 0 2024-04-02T04:43:03.044 INFO:tasks.workunit.client.1.smithi027.stdout:9/766: creat d1a/db6/d8c/dcf/ff6 x:0 0 0 2024-04-02T04:43:03.046 INFO:tasks.workunit.client.1.smithi027.stdout:2/794: rmdir d1/db/d3b/d17/d6d/dd9/dd5/dda/dbb/db4/ddf 39 2024-04-02T04:43:03.049 INFO:tasks.workunit.client.1.smithi027.stdout:2/795: read d1/db/d3b/d17/d2b/f4e [2202273,39902] 0 2024-04-02T04:43:03.051 INFO:tasks.workunit.client.1.smithi027.stdout:2/796: readlink d1/db/d3b/d61/lb7 0 2024-04-02T04:43:03.053 INFO:tasks.workunit.client.1.smithi027.stdout:6/597: dwrite d13/d1b/d24/d25/d56/d73/d96/fac [0,4194304] 0 2024-04-02T04:43:03.055 INFO:tasks.workunit.client.1.smithi027.stdout:6/598: fdatasync d13/f93 0 2024-04-02T04:43:03.055 INFO:tasks.workunit.client.1.smithi027.stdout:2/797: chown d1/db/d3b/d17/dcf 252794 1 2024-04-02T04:43:03.058 INFO:tasks.workunit.client.1.smithi027.stdout:9/767: dwrite d1a/d1d/d29/d37/d20/d7e/feb [0,4194304] 0 2024-04-02T04:43:03.061 INFO:tasks.workunit.client.1.smithi027.stdout:9/768: chown d1a/d1d/db7/fea 27 1 2024-04-02T04:43:03.068 INFO:tasks.workunit.client.1.smithi027.stdout:6/599: rename d13/d1b/d48/d58/l5a to d13/d8d/lce 0 2024-04-02T04:43:03.070 INFO:tasks.workunit.client.1.smithi027.stdout:9/769: symlink d1a/d1d/d29/d50/lf7 0 2024-04-02T04:43:03.071 INFO:tasks.workunit.client.1.smithi027.stdout:9/770: stat d1a/d1d/d3c/d6e 0 2024-04-02T04:43:03.072 INFO:tasks.workunit.client.1.smithi027.stdout:9/771: stat d1a/d1d/f4b 0 2024-04-02T04:43:03.075 INFO:tasks.workunit.client.1.smithi027.stdout:9/772: creat d1a/d1d/d29/d56/ff8 x:0 0 0 2024-04-02T04:43:03.075 INFO:tasks.workunit.client.1.smithi027.stdout:6/600: getdents d13/d1b/d24/d25/d56/dc8 0 2024-04-02T04:43:03.081 INFO:tasks.workunit.client.1.smithi027.stdout:9/773: dwrite f15 [0,4194304] 0 2024-04-02T04:43:03.083 INFO:tasks.workunit.client.1.smithi027.stdout:9/774: write d1a/d1d/d29/f4f [1532516,50523] 0 2024-04-02T04:43:03.085 INFO:tasks.workunit.client.1.smithi027.stdout:9/775: dread d1a/d1d/d29/d37/fa1 [0,4194304] 0 2024-04-02T04:43:03.085 INFO:tasks.workunit.client.1.smithi027.stdout:9/776: stat d1a/l7f 0 2024-04-02T04:43:03.088 INFO:tasks.workunit.client.1.smithi027.stdout:9/777: symlink d1a/d1d/d29/d50/d73/dbd/dd1/lf9 0 2024-04-02T04:43:03.098 INFO:tasks.workunit.client.1.smithi027.stdout:2/798: dread d1/db/d3b/d17/d6d/dd9/dd5/f5a [0,4194304] 0 2024-04-02T04:43:03.101 INFO:tasks.workunit.client.1.smithi027.stdout:2/799: link d1/db/d3b/d17/d12/f65 d1/db/d3b/d17/dec/f122 0 2024-04-02T04:43:03.102 INFO:tasks.workunit.client.1.smithi027.stdout:2/800: fsync d1/db/d3b/d17/d6d/dd9/dd5/dda/f112 0 2024-04-02T04:43:03.104 INFO:tasks.workunit.client.1.smithi027.stdout:2/801: rmdir d1/db/de5/de6 39 2024-04-02T04:43:03.111 INFO:tasks.workunit.client.1.smithi027.stdout:9/778: getdents d1a/d1d/d29/d50 0 2024-04-02T04:43:03.113 INFO:tasks.workunit.client.1.smithi027.stdout:9/779: read d1a/db6/faf [450734,69336] 0 2024-04-02T04:43:03.120 INFO:tasks.workunit.client.1.smithi027.stdout:9/780: dwrite d1a/d1d/d29/d56/f88 [0,4194304] 0 2024-04-02T04:43:03.121 INFO:tasks.workunit.client.1.smithi027.stdout:2/802: write d1/db/d3b/f1c [1118397,18824] 0 2024-04-02T04:43:03.122 INFO:tasks.workunit.client.1.smithi027.stdout:9/781: mkdir d1a/d1d/d29/d51/dfa 0 2024-04-02T04:43:03.127 INFO:tasks.workunit.client.1.smithi027.stdout:2/803: truncate d1/db/d3b/d17/d2b/f74 82085 0 2024-04-02T04:43:03.129 INFO:tasks.workunit.client.1.smithi027.stdout:9/782: mkdir d1a/d1d/d29/d50/da2/dfb 0 2024-04-02T04:43:03.134 INFO:tasks.workunit.client.1.smithi027.stdout:2/804: link d1/db/d57/ff2 d1/db/d3b/d61/dd2/f123 0 2024-04-02T04:43:03.137 INFO:tasks.workunit.client.1.smithi027.stdout:2/805: unlink d1/db/d3b/d17/d68/fac 0 2024-04-02T04:43:03.139 INFO:tasks.workunit.client.1.smithi027.stdout:9/783: creat d1a/d1d/ffc x:0 0 0 2024-04-02T04:43:03.142 INFO:tasks.workunit.client.1.smithi027.stdout:9/784: symlink d1a/d1d/d29/d50/da2/dfb/lfd 0 2024-04-02T04:43:03.143 INFO:tasks.workunit.client.1.smithi027.stdout:2/806: dwrite d1/db/d57/f99 [0,4194304] 0 2024-04-02T04:43:03.143 INFO:tasks.workunit.client.1.smithi027.stdout:9/785: rmdir d1a/d1d/d29/d37 39 2024-04-02T04:43:03.147 INFO:tasks.workunit.client.1.smithi027.stdout:9/786: creat d1a/d1d/d29/d50/d73/ffe x:0 0 0 2024-04-02T04:43:03.151 INFO:tasks.workunit.client.1.smithi027.stdout:9/787: mknod d1a/d1d/d3c/d6e/db8/cff 0 2024-04-02T04:43:03.153 INFO:tasks.workunit.client.1.smithi027.stdout:9/788: chown d1a/d1d/d29/d37/d3a/c49 690888 1 2024-04-02T04:43:03.155 INFO:tasks.workunit.client.1.smithi027.stdout:9/789: mkdir d1a/db6/d8c/de1/d100 0 2024-04-02T04:43:03.156 INFO:tasks.workunit.client.1.smithi027.stdout:9/790: dread - d1a/d1d/d29/d50/da2/fbb zero size 2024-04-02T04:43:03.157 INFO:tasks.workunit.client.1.smithi027.stdout:9/791: chown d1a/d1d/d29/d37/d3a/fa3 287 1 2024-04-02T04:43:03.160 INFO:tasks.workunit.client.1.smithi027.stdout:9/792: dread d1a/d1d/d29/d37/d4c/f62 [0,4194304] 0 2024-04-02T04:43:03.162 INFO:tasks.workunit.client.1.smithi027.stdout:9/793: creat d1a/d1d/d29/d37/d99/de8/f101 x:0 0 0 2024-04-02T04:43:03.165 INFO:tasks.workunit.client.1.smithi027.stdout:7/657: sync 2024-04-02T04:43:03.166 INFO:tasks.workunit.client.1.smithi027.stdout:5/837: sync 2024-04-02T04:43:03.166 INFO:tasks.workunit.client.1.smithi027.stdout:1/648: sync 2024-04-02T04:43:03.166 INFO:tasks.workunit.client.1.smithi027.stdout:0/738: sync 2024-04-02T04:43:03.169 INFO:tasks.workunit.client.1.smithi027.stdout:9/794: dwrite d1a/d1d/d3c/f94 [0,4194304] 0 2024-04-02T04:43:03.175 INFO:tasks.workunit.client.1.smithi027.stdout:0/739: dwrite d4/d21/fc5 [0,4194304] 0 2024-04-02T04:43:03.177 INFO:tasks.workunit.client.1.smithi027.stdout:9/795: mkdir d1a/db6/d102 0 2024-04-02T04:43:03.187 INFO:tasks.workunit.client.1.smithi027.stdout:9/796: dwrite d1a/db6/fca [0,4194304] 0 2024-04-02T04:43:03.187 INFO:tasks.workunit.client.1.smithi027.stdout:2/807: truncate d1/db/d3b/d17/d2b/f114 3518425 0 2024-04-02T04:43:03.196 INFO:tasks.workunit.client.1.smithi027.stdout:7/658: link d2/df/f85 d2/dba/dc/d1f/d33/fd0 0 2024-04-02T04:43:03.198 INFO:tasks.workunit.client.1.smithi027.stdout:2/808: symlink d1/db/d3b/d17/d6d/l124 0 2024-04-02T04:43:03.201 INFO:tasks.workunit.client.1.smithi027.stdout:0/740: rename d4/d21/f7d to d4/d38/ffd 0 2024-04-02T04:43:03.211 INFO:tasks.workunit.client.1.smithi027.stdout:7/659: dwrite d2/dba/dc/d16/d3f/d44/fb4 [0,4194304] 0 2024-04-02T04:43:03.213 INFO:tasks.workunit.client.1.smithi027.stdout:1/649: dread d0/d1e/d25/f31 [4194304,4194304] 0 2024-04-02T04:43:03.213 INFO:tasks.workunit.client.1.smithi027.stdout:2/809: dwrite d1/db/d3b/f1c [0,4194304] 0 2024-04-02T04:43:03.214 INFO:tasks.workunit.client.1.smithi027.stdout:0/741: rename d4/d38/da0 to d4/d76/dc4/dfe 0 2024-04-02T04:43:03.216 INFO:tasks.workunit.client.1.smithi027.stdout:1/650: chown d0/de/c18 5 1 2024-04-02T04:43:03.216 INFO:tasks.workunit.client.1.smithi027.stdout:5/838: dwrite d0/df/d11/d13/d9a/db7/fef [0,4194304] 0 2024-04-02T04:43:03.220 INFO:tasks.workunit.client.1.smithi027.stdout:5/839: write d0/d1d/f100 [121312,20053] 0 2024-04-02T04:43:03.220 INFO:tasks.workunit.client.1.smithi027.stdout:7/660: unlink d2/d1d/d90/f5d 0 2024-04-02T04:43:03.224 INFO:tasks.workunit.client.1.smithi027.stdout:7/661: write d2/d1c/f6e [4378026,10375] 0 2024-04-02T04:43:03.226 INFO:tasks.workunit.client.1.smithi027.stdout:9/797: dread d1a/d1d/d29/d51/fa6 [0,4194304] 0 2024-04-02T04:43:03.228 INFO:tasks.workunit.client.1.smithi027.stdout:1/651: dread d0/d1e/d25/d2b/f84 [0,4194304] 0 2024-04-02T04:43:03.230 INFO:tasks.workunit.client.1.smithi027.stdout:1/652: chown d0/de/d52/l91 1803217 1 2024-04-02T04:43:03.231 INFO:tasks.workunit.client.1.smithi027.stdout:9/798: write d1a/d1d/d29/d37/d20/dad/fdf [757049,95723] 0 2024-04-02T04:43:03.231 INFO:tasks.workunit.client.1.smithi027.stdout:0/742: dread d4/d38/d66/d2d/d4c/fcf [0,4194304] 0 2024-04-02T04:43:03.231 INFO:tasks.workunit.client.1.smithi027.stdout:5/840: truncate d0/df/d11/d13/dce/f63 2574265 0 2024-04-02T04:43:03.233 INFO:tasks.workunit.client.1.smithi027.stdout:2/810: mkdir d1/db/d3b/d17/d6d/dd9/dd5/d125 0 2024-04-02T04:43:03.236 INFO:tasks.workunit.client.1.smithi027.stdout:1/653: write d0/d1e/d5d/f81 [2418287,115394] 0 2024-04-02T04:43:03.238 INFO:tasks.workunit.client.1.smithi027.stdout:7/662: dwrite d2/d1d/d92/f9a [0,4194304] 0 2024-04-02T04:43:03.240 INFO:tasks.workunit.client.1.smithi027.stdout:3/578: sync 2024-04-02T04:43:03.240 INFO:tasks.workunit.client.1.smithi027.stdout:8/666: sync 2024-04-02T04:43:03.240 INFO:tasks.workunit.client.1.smithi027.stdout:5/841: mkdir d0/df/d11/d37/d120 0 2024-04-02T04:43:03.241 INFO:tasks.workunit.client.1.smithi027.stdout:2/811: symlink d1/db/d3b/d17/d6d/df1/l126 0 2024-04-02T04:43:03.243 INFO:tasks.workunit.client.1.smithi027.stdout:3/579: dread - d1/de/d55/fa1 zero size 2024-04-02T04:43:03.245 INFO:tasks.workunit.client.1.smithi027.stdout:1/654: fdatasync d0/de/f80 0 2024-04-02T04:43:03.245 INFO:tasks.workunit.client.1.smithi027.stdout:9/799: mkdir d1a/d1d/d29/d56/de4/d103 0 2024-04-02T04:43:03.245 INFO:tasks.workunit.client.1.smithi027.stdout:7/663: chown d2/dba/dc/f30 424538 1 2024-04-02T04:43:03.251 INFO:tasks.workunit.client.1.smithi027.stdout:3/580: rename d1/de/df/d61/f68 to d1/de/df/d13/d73/d77/fc4 0 2024-04-02T04:43:03.253 INFO:tasks.workunit.client.1.smithi027.stdout:8/667: creat d5/d8/db/dd/d5f/d27/d31/db9/dc1/fe1 x:0 0 0 2024-04-02T04:43:03.253 INFO:tasks.workunit.client.1.smithi027.stdout:2/812: dwrite d1/db/d3b/d17/d12/f41 [0,4194304] 0 2024-04-02T04:43:03.255 INFO:tasks.workunit.client.1.smithi027.stdout:3/581: write d1/fab [636104,98958] 0 2024-04-02T04:43:03.255 INFO:tasks.workunit.client.1.smithi027.stdout:8/668: chown d5/d8/db/d57/c8c 1 1 2024-04-02T04:43:03.256 INFO:tasks.workunit.client.1.smithi027.stdout:1/655: creat d0/d1e/d5d/fdb x:0 0 0 2024-04-02T04:43:03.256 INFO:tasks.workunit.client.1.smithi027.stdout:9/800: creat d1a/d1d/f104 x:0 0 0 2024-04-02T04:43:03.258 INFO:tasks.workunit.client.1.smithi027.stdout:1/656: fsync d0/d1e/d25/d3c/d64/fd8 0 2024-04-02T04:43:03.260 INFO:tasks.workunit.client.1.smithi027.stdout:3/582: symlink d1/de/d55/d63/lc5 0 2024-04-02T04:43:03.261 INFO:tasks.workunit.client.1.smithi027.stdout:3/583: chown d1/c83 12 1 2024-04-02T04:43:03.264 INFO:tasks.workunit.client.1.smithi027.stdout:9/801: getdents d1a 0 2024-04-02T04:43:03.266 INFO:tasks.workunit.client.1.smithi027.stdout:0/743: rmdir d4 39 2024-04-02T04:43:03.268 INFO:tasks.workunit.client.1.smithi027.stdout:1/657: rename d0/de/c33 to d0/d1e/d25/d3c/d64/cdc 0 2024-04-02T04:43:03.270 INFO:tasks.workunit.client.1.smithi027.stdout:8/669: getdents d5/d8/d15 0 2024-04-02T04:43:03.273 INFO:tasks.workunit.client.1.smithi027.stdout:3/584: unlink d1/de/d55/d63/l70 0 2024-04-02T04:43:03.278 INFO:tasks.workunit.client.1.smithi027.stdout:9/802: rename d1a/db6 to d1a/d1d/d29/d51/d105 0 2024-04-02T04:43:03.282 INFO:tasks.workunit.client.1.smithi027.stdout:3/585: mkdir d1/de/df/d9e/dc6 0 2024-04-02T04:43:03.290 INFO:tasks.workunit.client.1.smithi027.stdout:9/803: symlink d1a/d1d/d29/d56/l106 0 2024-04-02T04:43:03.291 INFO:tasks.workunit.client.1.smithi027.stdout:5/842: write d0/df/d11/d37/f4e [850852,4384] 0 2024-04-02T04:43:03.295 INFO:tasks.workunit.client.1.smithi027.stdout:0/744: symlink d4/d38/d66/lff 0 2024-04-02T04:43:03.299 INFO:tasks.workunit.client.1.smithi027.stdout:9/804: dwrite d1a/d1d/d29/d37/d9b/fed [0,4194304] 0 2024-04-02T04:43:03.299 INFO:tasks.workunit.client.1.smithi027.stdout:2/813: write d1/db/d3b/d17/d2b/db8/f9f [2411097,24865] 0 2024-04-02T04:43:03.306 INFO:tasks.workunit.client.1.smithi027.stdout:8/670: getdents d5/d8/db/dd/d5f/d27/d31/db9/dc1 0 2024-04-02T04:43:03.307 INFO:tasks.workunit.client.1.smithi027.stdout:8/671: read - d5/d8/db/dd/d4a/dc9/fa6 zero size 2024-04-02T04:43:03.314 INFO:tasks.workunit.client.1.smithi027.stdout:1/658: dwrite d0/f7 [4194304,4194304] 0 2024-04-02T04:43:03.317 INFO:tasks.workunit.client.1.smithi027.stdout:8/672: dread - d5/d8/db/dd/d5f/d27/d31/f9c zero size 2024-04-02T04:43:03.318 INFO:tasks.workunit.client.1.smithi027.stdout:9/805: mkdir d1a/d1d/d29/d37/d107 0 2024-04-02T04:43:03.321 INFO:tasks.workunit.client.1.smithi027.stdout:8/673: write d5/d8/db/fca [1350476,82097] 0 2024-04-02T04:43:03.334 INFO:tasks.workunit.client.1.smithi027.stdout:5/843: dread d0/d1d/f31 [0,4194304] 0 2024-04-02T04:43:03.416 INFO:tasks.workunit.client.1.smithi027.stdout:4/575: sync 2024-04-02T04:43:03.416 INFO:tasks.workunit.client.1.smithi027.stdout:6/601: sync 2024-04-02T04:43:03.418 INFO:tasks.workunit.client.1.smithi027.stdout:4/576: truncate d24/d38/d47/fa3 788722 0 2024-04-02T04:43:03.425 INFO:tasks.workunit.client.1.smithi027.stdout:6/602: dwrite d13/d1b/d24/d25/d42/fa9 [0,4194304] 0 2024-04-02T04:43:03.427 INFO:tasks.workunit.client.1.smithi027.stdout:4/577: dwrite d24/f71 [4194304,4194304] 0 2024-04-02T04:43:03.431 INFO:tasks.workunit.client.1.smithi027.stdout:1/659: dread - d0/d1e/f9a zero size 2024-04-02T04:43:03.438 INFO:tasks.workunit.client.1.smithi027.stdout:0/745: link d4/d21/d28/f75 d4/d76/dc4/dfe/dd1/f100 0 2024-04-02T04:43:03.443 INFO:tasks.workunit.client.1.smithi027.stdout:6/603: creat d13/d1b/d24/d25/d56/d6d/fcf x:0 0 0 2024-04-02T04:43:03.445 INFO:tasks.workunit.client.1.smithi027.stdout:4/578: symlink d24/d38/d44/d41/d5a/lbe 0 2024-04-02T04:43:03.446 INFO:tasks.workunit.client.1.smithi027.stdout:4/579: stat d24/d2d/f92 0 2024-04-02T04:43:03.450 INFO:tasks.workunit.client.1.smithi027.stdout:9/806: creat d1a/d1d/d29/d37/d4c/dc9/f108 x:0 0 0 2024-04-02T04:43:03.453 INFO:tasks.workunit.client.1.smithi027.stdout:1/660: dread d0/de/f92 [0,4194304] 0 2024-04-02T04:43:03.471 INFO:tasks.workunit.client.1.smithi027.stdout:1/661: fdatasync d0/d1e/f9c 0 2024-04-02T04:43:03.472 INFO:tasks.workunit.client.1.smithi027.stdout:1/662: write d0/f7 [7730843,94476] 0 2024-04-02T04:43:03.505 INFO:tasks.workunit.client.1.smithi027.stdout:9/807: rename d1a/d1d/d29/d50/lf7 to d1a/d1d/d29/d37/d20/d7e/l109 0 2024-04-02T04:43:03.508 INFO:tasks.workunit.client.1.smithi027.stdout:9/808: mkdir d1a/d1d/d29/d50/d73/dbd/d10a 0 2024-04-02T04:43:03.516 INFO:tasks.workunit.client.1.smithi027.stdout:9/809: dwrite d1a/d1d/d29/d51/d105/d8c/dcf/ff6 [0,4194304] 0 2024-04-02T04:43:03.543 INFO:tasks.workunit.client.1.smithi027.stdout:1/663: creat d0/de/d52/fdd x:0 0 0 2024-04-02T04:43:03.545 INFO:tasks.workunit.client.1.smithi027.stdout:1/664: truncate d0/d1e/d2e/d3e/f82 482249 0 2024-04-02T04:43:03.546 INFO:tasks.workunit.client.1.smithi027.stdout:1/665: fdatasync d0/de/d52/d74/fb5 0 2024-04-02T04:43:03.564 INFO:tasks.workunit.client.1.smithi027.stdout:9/810: symlink d1a/d1d/d3c/l10b 0 2024-04-02T04:43:03.580 INFO:tasks.workunit.client.1.smithi027.stdout:4/580: write fc [1943153,13522] 0 2024-04-02T04:43:03.581 INFO:tasks.workunit.client.1.smithi027.stdout:4/581: chown d24/d48/f72 229994 1 2024-04-02T04:43:03.584 INFO:tasks.workunit.client.1.smithi027.stdout:4/582: mknod d24/d38/cbf 0 2024-04-02T04:43:03.587 INFO:tasks.workunit.client.1.smithi027.stdout:4/583: dread f2 [0,4194304] 0 2024-04-02T04:43:03.588 INFO:tasks.workunit.client.1.smithi027.stdout:4/584: write d24/f71 [6078202,3574] 0 2024-04-02T04:43:03.591 INFO:tasks.workunit.client.1.smithi027.stdout:4/585: mkdir d24/d38/d47/dc0 0 2024-04-02T04:43:03.592 INFO:tasks.workunit.client.1.smithi027.stdout:4/586: chown d24/d38 30996 1 2024-04-02T04:43:03.595 INFO:tasks.workunit.client.1.smithi027.stdout:4/587: symlink d24/d38/d44/dbc/lc1 0 2024-04-02T04:43:03.595 INFO:tasks.workunit.client.1.smithi027.stdout:4/588: fdatasync d24/d38/f8d 0 2024-04-02T04:43:03.598 INFO:tasks.workunit.client.1.smithi027.stdout:4/589: getdents d24/d38/d47/d5c 0 2024-04-02T04:43:03.611 INFO:tasks.workunit.client.1.smithi027.stdout:4/590: dread f11 [0,4194304] 0 2024-04-02T04:43:03.613 INFO:tasks.workunit.client.1.smithi027.stdout:4/591: creat d24/d38/d44/d82/d95/fc2 x:0 0 0 2024-04-02T04:43:03.615 INFO:tasks.workunit.client.1.smithi027.stdout:4/592: creat d24/d48/db0/fc3 x:0 0 0 2024-04-02T04:43:03.620 INFO:tasks.workunit.client.1.smithi027.stdout:6/604: mknod d13/d1b/d24/d25/cd0 0 2024-04-02T04:43:03.627 INFO:tasks.workunit.client.1.smithi027.stdout:1/666: truncate d0/d1e/d25/d3c/d64/da6/db8/fba 483567 0 2024-04-02T04:43:03.643 INFO:tasks.workunit.client.1.smithi027.stdout:6/605: dread d13/f1a [0,4194304] 0 2024-04-02T04:43:03.646 INFO:tasks.workunit.client.1.smithi027.stdout:9/811: dwrite d1a/f38 [0,4194304] 0 2024-04-02T04:43:03.646 INFO:tasks.workunit.client.1.smithi027.stdout:9/812: stat d1a/l8d 0 2024-04-02T04:43:03.648 INFO:tasks.workunit.client.1.smithi027.stdout:9/813: read d1a/d1d/d29/d56/d8b/f90 [665243,601] 0 2024-04-02T04:43:03.699 INFO:tasks.workunit.client.1.smithi027.stdout:7/664: sync 2024-04-02T04:43:03.699 INFO:tasks.workunit.client.1.smithi027.stdout:7/665: chown d2/dba/dc/d1f/d33/d36/f7f 6222 1 2024-04-02T04:43:03.718 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:03 smithi016 ceph-mon[28969]: pgmap v30: 65 pgs: 65 active+clean; 3.3 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 72 MiB/s rd, 184 MiB/s wr, 400 op/s 2024-04-02T04:43:03.733 INFO:tasks.workunit.client.1.smithi027.stdout:6/606: rmdir d13/d1b/d24/d25/d27 39 2024-04-02T04:43:03.734 INFO:tasks.workunit.client.1.smithi027.stdout:6/607: write d13/f17 [3843345,65223] 0 2024-04-02T04:43:03.735 INFO:tasks.workunit.client.1.smithi027.stdout:6/608: stat d13/d1b/f8a 0 2024-04-02T04:43:03.739 INFO:tasks.workunit.client.1.smithi027.stdout:8/674: link d5/d8/d15/d71/d96/cb8 d5/d8/db/dd/d5f/d27/ce2 0 2024-04-02T04:43:03.748 INFO:tasks.workunit.client.1.smithi027.stdout:6/609: rename d13/f62 to d13/d1b/d24/fd1 0 2024-04-02T04:43:03.751 INFO:tasks.workunit.client.1.smithi027.stdout:1/667: truncate d0/fb 3538664 0 2024-04-02T04:43:03.759 INFO:tasks.workunit.client.1.smithi027.stdout:6/610: creat d13/d1b/d24/d25/fd2 x:0 0 0 2024-04-02T04:43:03.765 INFO:tasks.workunit.client.1.smithi027.stdout:7/666: dread d2/d1c/f4b [0,4194304] 0 2024-04-02T04:43:03.766 INFO:tasks.workunit.client.1.smithi027.stdout:7/667: write d2/dba/dc/d16/fac [1384757,12531] 0 2024-04-02T04:43:03.769 INFO:tasks.workunit.client.1.smithi027.stdout:7/668: creat d2/df/fd1 x:0 0 0 2024-04-02T04:43:03.774 INFO:tasks.workunit.client.1.smithi027.stdout:7/669: rename d2/dba/dc/d16/d3f/ca2 to d2/dba/dc/d16/d3f/d44/cd2 0 2024-04-02T04:43:03.779 INFO:tasks.workunit.client.1.smithi027.stdout:8/675: rmdir d5/d8/d15/d71/d96 39 2024-04-02T04:43:03.780 INFO:tasks.workunit.client.1.smithi027.stdout:7/670: dread d2/dba/dc/d1f/d33/f55 [0,4194304] 0 2024-04-02T04:43:03.782 INFO:tasks.workunit.client.1.smithi027.stdout:8/676: chown d5/d8/db/dd/d5f/d27/d31/d5b/dc4/f5d 2030 1 2024-04-02T04:43:03.786 INFO:tasks.workunit.client.1.smithi027.stdout:8/677: chown d5/d8/db/dd/d5f/d91/d9f/le0 209879 1 2024-04-02T04:43:03.787 INFO:tasks.workunit.client.1.smithi027.stdout:9/814: dwrite d1a/d1d/d29/d37/fa1 [0,4194304] 0 2024-04-02T04:43:03.789 INFO:tasks.workunit.client.1.smithi027.stdout:9/815: fdatasync d1a/d1d/d29/d56/d8b/fa4 0 2024-04-02T04:43:03.790 INFO:tasks.workunit.client.1.smithi027.stdout:9/816: write d1a/d1d/f4b [1249997,24644] 0 2024-04-02T04:43:03.791 INFO:tasks.workunit.client.1.smithi027.stdout:9/817: readlink d1a/d1d/d29/d50/d73/lcd 0 2024-04-02T04:43:03.791 INFO:tasks.workunit.client.1.smithi027.stdout:8/678: unlink d5/d8/db/df/l7c 0 2024-04-02T04:43:03.791 INFO:tasks.workunit.client.1.smithi027.stdout:1/668: write d0/d1e/f9c [496564,72612] 0 2024-04-02T04:43:03.795 INFO:tasks.workunit.client.1.smithi027.stdout:9/818: creat d1a/d1d/d29/d50/da2/dfb/f10c x:0 0 0 2024-04-02T04:43:03.799 INFO:tasks.workunit.client.1.smithi027.stdout:8/679: mkdir d5/d8/db/dd/d5f/d27/d31/de3 0 2024-04-02T04:43:03.799 INFO:tasks.workunit.client.1.smithi027.stdout:6/611: dwrite d13/d1b/d48/d58/f6a [0,4194304] 0 2024-04-02T04:43:03.803 INFO:tasks.workunit.client.1.smithi027.stdout:9/819: mkdir d1a/d1d/d29/d51/d105/d10d 0 2024-04-02T04:43:03.807 INFO:tasks.workunit.client.1.smithi027.stdout:8/680: chown d5/d8/db/dd/d5f/d27/d31/d5b/dc4/f8b 1379534 1 2024-04-02T04:43:03.809 INFO:tasks.workunit.client.1.smithi027.stdout:9/820: readlink d1a/d1d/d29/d51/d105/d8c/dcf/ld4 0 2024-04-02T04:43:03.813 INFO:tasks.workunit.client.1.smithi027.stdout:8/681: dread d5/d8/d15/f8a [0,4194304] 0 2024-04-02T04:43:03.815 INFO:tasks.workunit.client.1.smithi027.stdout:7/671: dwrite d2/dba/dc/d1f/fa1 [0,4194304] 0 2024-04-02T04:43:03.817 INFO:tasks.workunit.client.1.smithi027.stdout:7/672: dread - d2/dba/dc/fc3 zero size 2024-04-02T04:43:03.817 INFO:tasks.workunit.client.1.smithi027.stdout:1/669: truncate d0/d1e/d5d/f81 972260 0 2024-04-02T04:43:03.819 INFO:tasks.workunit.client.1.smithi027.stdout:8/682: creat d5/d8/db/dd/d5f/d27/d31/d5b/dc4/fe4 x:0 0 0 2024-04-02T04:43:03.826 INFO:tasks.workunit.client.1.smithi027.stdout:8/683: rename d5/d8/d5c/f9a to d5/d8/db/dd/d5f/db2/fe5 0 2024-04-02T04:43:03.827 INFO:tasks.workunit.client.1.smithi027.stdout:7/673: getdents d2/dba/dc/d16/d3a/d4d/dcb 0 2024-04-02T04:43:03.830 INFO:tasks.workunit.client.1.smithi027.stdout:1/670: write d0/f54 [753776,89414] 0 2024-04-02T04:43:03.837 INFO:tasks.workunit.client.1.smithi027.stdout:8/684: truncate d5/d8/f47 2206341 0 2024-04-02T04:43:03.840 INFO:tasks.workunit.client.1.smithi027.stdout:1/671: mkdir d0/d1e/d2e/d3b/d4f/dd3/dde 0 2024-04-02T04:43:03.842 INFO:tasks.workunit.client.1.smithi027.stdout:1/672: chown d0/d1e/d2e/d3e/d5f/d8b/d9d/fad 14 1 2024-04-02T04:43:03.844 INFO:tasks.workunit.client.1.smithi027.stdout:7/674: dwrite d2/d1d/d57/f5e [0,4194304] 0 2024-04-02T04:43:03.844 INFO:tasks.workunit.client.1.smithi027.stdout:1/673: chown d0/de/d52/d74/l90 580619235 1 2024-04-02T04:43:03.846 INFO:tasks.workunit.client.1.smithi027.stdout:7/675: write d2/dba/da6/fcc [28167,24399] 0 2024-04-02T04:43:03.847 INFO:tasks.workunit.client.1.smithi027.stdout:8/685: rename d5/d8/db/dd/d5f/d27/d31/db9/dc1/dcb to d5/d8/d5c/de6 0 2024-04-02T04:43:03.851 INFO:tasks.workunit.client.1.smithi027.stdout:1/674: truncate d0/f30 8471178 0 2024-04-02T04:43:03.855 INFO:tasks.workunit.client.1.smithi027.stdout:1/675: dread d0/d1e/d2e/d3b/f77 [4194304,4194304] 0 2024-04-02T04:43:03.857 INFO:tasks.workunit.client.1.smithi027.stdout:1/676: write d0/de/d52/d74/db7/fc6 [904345,10257] 0 2024-04-02T04:43:03.862 INFO:tasks.workunit.client.1.smithi027.stdout:7/676: link d2/dba/dc/d16/d3a/l76 d2/dba/d35/ld3 0 2024-04-02T04:43:03.863 INFO:tasks.workunit.client.1.smithi027.stdout:3/586: sync 2024-04-02T04:43:03.870 INFO:tasks.workunit.client.1.smithi027.stdout:8/686: dwrite d5/d8/d15/d71/d96/fc0 [0,4194304] 0 2024-04-02T04:43:03.874 INFO:tasks.workunit.client.1.smithi027.stdout:8/687: mknod d5/d8/d15/ce7 0 2024-04-02T04:43:03.876 INFO:tasks.workunit.client.1.smithi027.stdout:7/677: write d2/d1d/f74 [475549,61902] 0 2024-04-02T04:43:03.879 INFO:tasks.workunit.client.1.smithi027.stdout:8/688: symlink d5/d8/db/d68/le8 0 2024-04-02T04:43:03.895 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:03 smithi027 ceph-mon[37550]: pgmap v30: 65 pgs: 65 active+clean; 3.3 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 72 MiB/s rd, 184 MiB/s wr, 400 op/s 2024-04-02T04:43:03.896 INFO:tasks.workunit.client.1.smithi027.stdout:7/678: creat d2/dba/d35/fd4 x:0 0 0 2024-04-02T04:43:03.897 INFO:tasks.workunit.client.1.smithi027.stdout:3/587: link d1/de/d55/l7b d1/de/df/d9e/lc7 0 2024-04-02T04:43:03.899 INFO:tasks.workunit.client.1.smithi027.stdout:8/689: symlink d5/d8/db/dd/d5f/d27/d31/db9/le9 0 2024-04-02T04:43:03.899 INFO:tasks.workunit.client.1.smithi027.stdout:7/679: symlink d2/dba/d35/ld5 0 2024-04-02T04:43:03.901 INFO:tasks.workunit.client.1.smithi027.stdout:3/588: creat d1/d1a/da6/fc8 x:0 0 0 2024-04-02T04:43:03.905 INFO:tasks.workunit.client.1.smithi027.stdout:8/690: rename d5/d8/db/dd/d5f/d27/d31/d5b/d6c/ddf to d5/d8/db/dd/d5f/d27/d31/d5b/dc4/dea 0 2024-04-02T04:43:03.908 INFO:tasks.workunit.client.1.smithi027.stdout:8/691: truncate d5/d8/db/dd/d5f/d27/d31/d5b/dc4/fe4 615651 0 2024-04-02T04:43:03.909 INFO:tasks.workunit.client.1.smithi027.stdout:8/692: write d5/d8/f2e [3169577,62028] 0 2024-04-02T04:43:03.919 INFO:tasks.workunit.client.1.smithi027.stdout:3/589: write d1/de/d55/f76 [289652,55860] 0 2024-04-02T04:43:03.924 INFO:tasks.workunit.client.1.smithi027.stdout:7/680: dwrite d2/d1d/f25 [0,4194304] 0 2024-04-02T04:43:03.924 INFO:tasks.workunit.client.1.smithi027.stdout:3/590: symlink d1/de/df/d13/lc9 0 2024-04-02T04:43:03.925 INFO:tasks.workunit.client.1.smithi027.stdout:3/591: fdatasync d1/de/fa7 0 2024-04-02T04:43:03.926 INFO:tasks.workunit.client.1.smithi027.stdout:8/693: dread d5/d8/db/dd/f1f [0,4194304] 0 2024-04-02T04:43:03.929 INFO:tasks.workunit.client.1.smithi027.stdout:8/694: chown d5/d8/db/dd/d5f/d27/d31/d5b/dc4/fe4 13180512 1 2024-04-02T04:43:03.931 INFO:tasks.workunit.client.1.smithi027.stdout:3/592: dwrite d1/de/f38 [4194304,4194304] 0 2024-04-02T04:43:03.932 INFO:tasks.workunit.client.1.smithi027.stdout:8/695: dread - d5/d8/db/d68/f9b zero size 2024-04-02T04:43:03.933 INFO:tasks.workunit.client.1.smithi027.stdout:3/593: readlink d1/l37 0 2024-04-02T04:43:03.934 INFO:tasks.workunit.client.1.smithi027.stdout:8/696: write d5/d8/d15/d71/d96/fcc [1566028,60560] 0 2024-04-02T04:43:03.938 INFO:tasks.workunit.client.1.smithi027.stdout:3/594: fsync d1/f2a 0 2024-04-02T04:43:03.939 INFO:tasks.workunit.client.1.smithi027.stdout:3/595: chown d1/de/d55/l88 2 1 2024-04-02T04:43:03.940 INFO:tasks.workunit.client.1.smithi027.stdout:8/697: creat d5/d8/db/dd/d4a/dc9/feb x:0 0 0 2024-04-02T04:43:03.950 INFO:tasks.workunit.client.1.smithi027.stdout:8/698: symlink d5/d8/db/dd/d5f/d27/lec 0 2024-04-02T04:43:03.951 INFO:tasks.workunit.client.1.smithi027.stdout:3/596: creat d1/de/df/d13/d73/d77/d7f/fca x:0 0 0 2024-04-02T04:43:03.953 INFO:tasks.workunit.client.1.smithi027.stdout:8/699: dread d5/d8/db/df/fd0 [0,4194304] 0 2024-04-02T04:43:03.954 INFO:tasks.workunit.client.1.smithi027.stdout:8/700: stat d5/d8/db/d68/le8 0 2024-04-02T04:43:03.956 INFO:tasks.workunit.client.1.smithi027.stdout:8/701: write d5/d8/db/d57/fa1 [934911,119121] 0 2024-04-02T04:43:03.958 INFO:tasks.workunit.client.1.smithi027.stdout:8/702: dread d5/d8/db/dd/d5f/d27/d31/d5b/dc4/fe4 [0,4194304] 0 2024-04-02T04:43:03.968 INFO:tasks.workunit.client.1.smithi027.stdout:3/597: dwrite d1/f8 [0,4194304] 0 2024-04-02T04:43:03.971 INFO:tasks.workunit.client.1.smithi027.stdout:3/598: creat d1/de/df/d13/d73/db1/fcb x:0 0 0 2024-04-02T04:43:03.981 INFO:tasks.workunit.client.1.smithi027.stdout:3/599: truncate d1/d1a/f3f 119245 0 2024-04-02T04:43:03.986 INFO:tasks.workunit.client.1.smithi027.stdout:3/600: getdents d1/de/df 0 2024-04-02T04:43:03.987 INFO:tasks.workunit.client.1.smithi027.stdout:3/601: chown d1/de/l12 15079940 1 2024-04-02T04:43:03.987 INFO:tasks.workunit.client.1.smithi027.stdout:3/602: chown d1/d1a/l95 1267599 1 2024-04-02T04:43:03.989 INFO:tasks.workunit.client.1.smithi027.stdout:3/603: write d1/de/d55/fc0 [387260,124745] 0 2024-04-02T04:43:03.989 INFO:tasks.workunit.client.1.smithi027.stdout:3/604: chown d1/de/d55/d63 785835692 1 2024-04-02T04:43:03.991 INFO:tasks.workunit.client.1.smithi027.stdout:3/605: read d1/de/df/f6b [1088756,33488] 0 2024-04-02T04:43:03.995 INFO:tasks.workunit.client.1.smithi027.stdout:3/606: creat d1/de/d74/fcc x:0 0 0 2024-04-02T04:43:04.000 INFO:tasks.workunit.client.1.smithi027.stdout:2/814: sync 2024-04-02T04:43:04.002 INFO:tasks.workunit.client.1.smithi027.stdout:3/607: truncate d1/d1a/f3f 849218 0 2024-04-02T04:43:04.003 INFO:tasks.workunit.client.1.smithi027.stdout:2/815: rmdir d1/db/d3b/d17/d6d/dd9/dd5/dad/d110 39 2024-04-02T04:43:04.006 INFO:tasks.workunit.client.1.smithi027.stdout:3/608: symlink d1/de/df/d9e/da0/lcd 0 2024-04-02T04:43:04.009 INFO:tasks.workunit.client.1.smithi027.stdout:3/609: symlink d1/de/lce 0 2024-04-02T04:43:04.015 INFO:tasks.workunit.client.1.smithi027.stdout:3/610: write d1/de/f58 [3369177,41504] 0 2024-04-02T04:43:04.016 INFO:tasks.workunit.client.1.smithi027.stdout:2/816: dread d1/db/d3b/d17/f4a [0,4194304] 0 2024-04-02T04:43:04.020 INFO:tasks.workunit.client.1.smithi027.stdout:3/611: dread d1/de/df/d13/f36 [0,4194304] 0 2024-04-02T04:43:04.026 INFO:tasks.workunit.client.1.smithi027.stdout:2/817: write d1/db/d3b/d17/d6d/dd9/dd5/dda/fae [5187,129873] 0 2024-04-02T04:43:04.028 INFO:tasks.workunit.client.1.smithi027.stdout:2/818: write d1/db/d3b/d17/f9e [1274118,56021] 0 2024-04-02T04:43:04.030 INFO:tasks.workunit.client.1.smithi027.stdout:2/819: chown d1/db/d3b/d17/f9e 59536 1 2024-04-02T04:43:04.030 INFO:tasks.workunit.client.1.smithi027.stdout:3/612: getdents d1/de/df/d13/d73/db1 0 2024-04-02T04:43:04.034 INFO:tasks.workunit.client.1.smithi027.stdout:2/820: rename d1/db/d3b/d17/d6d/dd9/dd5/dad to d1/db/d3b/d17/d6d/d127 0 2024-04-02T04:43:04.036 INFO:tasks.workunit.client.1.smithi027.stdout:3/613: rmdir d1/d1a/da6/daa 0 2024-04-02T04:43:04.037 INFO:tasks.workunit.client.1.smithi027.stdout:3/614: readlink d1/d1a/l28 0 2024-04-02T04:43:04.038 INFO:tasks.workunit.client.1.smithi027.stdout:3/615: dread - d1/de/d74/f80 zero size 2024-04-02T04:43:04.039 INFO:tasks.workunit.client.1.smithi027.stdout:2/821: dread d1/db/d3b/d17/d68/f80 [0,4194304] 0 2024-04-02T04:43:04.040 INFO:tasks.workunit.client.1.smithi027.stdout:3/616: creat d1/de/df/d13/d73/db1/fcf x:0 0 0 2024-04-02T04:43:04.042 INFO:tasks.workunit.client.1.smithi027.stdout:2/822: creat d1/db/d3b/d17/d68/d69/f128 x:0 0 0 2024-04-02T04:43:04.045 INFO:tasks.workunit.client.1.smithi027.stdout:2/823: symlink d1/db/d3b/d61/l129 0 2024-04-02T04:43:04.050 INFO:tasks.workunit.client.1.smithi027.stdout:2/824: creat d1/db/d3b/d17/d6d/dd9/dd5/f12a x:0 0 0 2024-04-02T04:43:04.056 INFO:tasks.workunit.client.1.smithi027.stdout:2/825: getdents d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/de1 0 2024-04-02T04:43:04.057 INFO:tasks.workunit.client.1.smithi027.stdout:3/617: dread d1/de/df/d13/f51 [4194304,4194304] 0 2024-04-02T04:43:04.060 INFO:tasks.workunit.client.1.smithi027.stdout:3/618: symlink d1/de/df/d13/d73/ld0 0 2024-04-02T04:43:04.060 INFO:tasks.workunit.client.1.smithi027.stdout:3/619: dread - d1/de/df/d9e/fb4 zero size 2024-04-02T04:43:04.065 INFO:tasks.workunit.client.1.smithi027.stdout:2/826: dwrite d1/db/d3b/d17/dec/f122 [0,4194304] 0 2024-04-02T04:43:04.066 INFO:tasks.workunit.client.1.smithi027.stdout:3/620: rename d1/de/df/d13/d73/d77/f84 to d1/de/d74/fd1 0 2024-04-02T04:43:04.082 INFO:tasks.workunit.client.1.smithi027.stdout:3/621: write d1/de/df/d61/fb8 [110211,42785] 0 2024-04-02T04:43:04.083 INFO:tasks.workunit.client.1.smithi027.stdout:3/622: truncate d1/d1a/fbd 600355 0 2024-04-02T04:43:04.089 INFO:tasks.workunit.client.1.smithi027.stdout:3/623: getdents d1/de/d55/d63/db9 0 2024-04-02T04:43:04.091 INFO:tasks.workunit.client.1.smithi027.stdout:3/624: read d1/de/d55/f75 [813387,112322] 0 2024-04-02T04:43:04.094 INFO:tasks.workunit.client.1.smithi027.stdout:3/625: rmdir d1/de/df/d13/d73/d77/d7f 39 2024-04-02T04:43:04.103 INFO:tasks.workunit.client.1.smithi027.stdout:3/626: dwrite d1/de/df/f24 [0,4194304] 0 2024-04-02T04:43:04.104 INFO:tasks.workunit.client.1.smithi027.stdout:3/627: fdatasync d1/de/df/d61/fb8 0 2024-04-02T04:43:04.112 INFO:tasks.workunit.client.1.smithi027.stdout:3/628: dwrite d1/de/df/f8f [0,4194304] 0 2024-04-02T04:43:04.115 INFO:tasks.workunit.client.1.smithi027.stdout:3/629: mknod d1/d1a/da6/cd2 0 2024-04-02T04:43:04.122 INFO:tasks.workunit.client.1.smithi027.stdout:3/630: dwrite d1/de/df/d61/f4c [0,4194304] 0 2024-04-02T04:43:04.134 INFO:tasks.workunit.client.1.smithi027.stdout:3/631: dwrite d1/d1a/f3f [0,4194304] 0 2024-04-02T04:43:04.136 INFO:tasks.workunit.client.1.smithi027.stdout:3/632: truncate d1/de/df/d13/d73/fa9 150578 0 2024-04-02T04:43:04.144 INFO:tasks.workunit.client.1.smithi027.stdout:3/633: dwrite d1/f2a [0,4194304] 0 2024-04-02T04:43:04.146 INFO:tasks.workunit.client.1.smithi027.stdout:3/634: write d1/de/df/d13/d73/d77/d7f/fca [754594,12453] 0 2024-04-02T04:43:04.150 INFO:tasks.workunit.client.1.smithi027.stdout:3/635: mknod d1/de/df/d9e/cd3 0 2024-04-02T04:43:04.158 INFO:tasks.workunit.client.1.smithi027.stdout:3/636: dwrite d1/de/df/d13/f2d [4194304,4194304] 0 2024-04-02T04:43:04.165 INFO:tasks.workunit.client.1.smithi027.stdout:3/637: link d1/de/d74/fcc d1/de/df/d9e/dc6/fd4 0 2024-04-02T04:43:04.169 INFO:tasks.workunit.client.1.smithi027.stdout:3/638: dread d1/de/df/d13/f36 [0,4194304] 0 2024-04-02T04:43:04.170 INFO:tasks.workunit.client.1.smithi027.stdout:3/639: write d1/d1a/f86 [2062606,87430] 0 2024-04-02T04:43:04.170 INFO:tasks.workunit.client.1.smithi027.stdout:3/640: chown d1/de/d74/fd1 408963058 1 2024-04-02T04:43:04.172 INFO:tasks.workunit.client.1.smithi027.stdout:3/641: fsync d1/de/df/d13/f36 0 2024-04-02T04:43:04.173 INFO:tasks.workunit.client.1.smithi027.stdout:4/593: sync 2024-04-02T04:43:04.175 INFO:tasks.workunit.client.1.smithi027.stdout:4/594: creat d24/d38/d44/d60/fc4 x:0 0 0 2024-04-02T04:43:04.176 INFO:tasks.workunit.client.1.smithi027.stdout:4/595: fdatasync d24/d38/d44/f52 0 2024-04-02T04:43:04.177 INFO:tasks.workunit.client.1.smithi027.stdout:3/642: rename d1/de/df/d13/d73/f78 to d1/de/d55/fd5 0 2024-04-02T04:43:04.182 INFO:tasks.workunit.client.1.smithi027.stdout:3/643: rename d1/d1a/f5a to d1/de/df/d13/db3/fd6 0 2024-04-02T04:43:04.186 INFO:tasks.workunit.client.1.smithi027.stdout:4/596: link d24/d48/c9d d24/d38/d47/d62/cc5 0 2024-04-02T04:43:04.187 INFO:tasks.workunit.client.1.smithi027.stdout:4/597: chown d24/d48 17291 1 2024-04-02T04:43:04.190 INFO:tasks.workunit.client.1.smithi027.stdout:3/644: dwrite d1/de/df/fc2 [0,4194304] 0 2024-04-02T04:43:04.192 INFO:tasks.workunit.client.1.smithi027.stdout:3/645: write d1/de/df/d61/f6c [1902196,11239] 0 2024-04-02T04:43:04.198 INFO:tasks.workunit.client.1.smithi027.stdout:3/646: dread d1/d1a/f3f [0,4194304] 0 2024-04-02T04:43:04.199 INFO:tasks.workunit.client.1.smithi027.stdout:3/647: chown d1/de/df/fc2 0 1 2024-04-02T04:43:04.206 INFO:tasks.workunit.client.1.smithi027.stdout:4/598: dwrite f23 [0,4194304] 0 2024-04-02T04:43:04.215 INFO:tasks.workunit.client.1.smithi027.stdout:3/648: getdents d1/de/d74 0 2024-04-02T04:43:04.215 INFO:tasks.workunit.client.1.smithi027.stdout:3/649: fdatasync d1/de/f42 0 2024-04-02T04:43:04.218 INFO:tasks.workunit.client.1.smithi027.stdout:3/650: rmdir d1/d1a 39 2024-04-02T04:43:04.220 INFO:tasks.workunit.client.1.smithi027.stdout:3/651: mknod d1/de/d74/cd7 0 2024-04-02T04:43:04.222 INFO:tasks.workunit.client.1.smithi027.stdout:3/652: mknod d1/cd8 0 2024-04-02T04:43:04.223 INFO:tasks.workunit.client.1.smithi027.stdout:4/599: truncate d24/d48/f75 3567942 0 2024-04-02T04:43:04.225 INFO:tasks.workunit.client.1.smithi027.stdout:3/653: mknod d1/de/df/d61/cd9 0 2024-04-02T04:43:04.226 INFO:tasks.workunit.client.1.smithi027.stdout:4/600: readlink d24/l55 0 2024-04-02T04:43:04.230 INFO:tasks.workunit.client.1.smithi027.stdout:4/601: link d24/f3a d24/d38/d47/d62/fc6 0 2024-04-02T04:43:04.230 INFO:tasks.workunit.client.1.smithi027.stdout:4/602: readlink d24/d2d/lad 0 2024-04-02T04:43:04.233 INFO:tasks.workunit.client.1.smithi027.stdout:4/603: symlink d24/d38/d44/dbc/lc7 0 2024-04-02T04:43:04.234 INFO:tasks.workunit.client.1.smithi027.stdout:4/604: write d24/fbb [579869,49773] 0 2024-04-02T04:43:04.236 INFO:tasks.workunit.client.1.smithi027.stdout:4/605: dread d24/d38/d44/d41/f87 [0,4194304] 0 2024-04-02T04:43:04.238 INFO:tasks.workunit.client.1.smithi027.stdout:4/606: write d24/d48/f4e [3549691,8913] 0 2024-04-02T04:43:04.239 INFO:tasks.workunit.client.1.smithi027.stdout:4/607: truncate d24/fbb 672487 0 2024-04-02T04:43:04.241 INFO:tasks.workunit.client.1.smithi027.stdout:3/654: dread d1/de/d55/d63/f91 [0,4194304] 0 2024-04-02T04:43:04.243 INFO:tasks.workunit.client.1.smithi027.stdout:4/608: creat d24/d48/db0/fc8 x:0 0 0 2024-04-02T04:43:04.246 INFO:tasks.workunit.client.1.smithi027.stdout:4/609: creat d24/d48/fc9 x:0 0 0 2024-04-02T04:43:04.247 INFO:tasks.workunit.client.1.smithi027.stdout:4/610: readlink d24/d38/d44/d41/d49/lac 0 2024-04-02T04:43:04.248 INFO:tasks.workunit.client.1.smithi027.stdout:4/611: chown d24/d38/d44/d41/f87 12 1 2024-04-02T04:43:04.248 INFO:tasks.workunit.client.1.smithi027.stdout:4/612: readlink d24/d2d/l51 0 2024-04-02T04:43:04.254 INFO:tasks.workunit.client.1.smithi027.stdout:4/613: dwrite f12 [4194304,4194304] 0 2024-04-02T04:43:04.256 INFO:tasks.workunit.client.1.smithi027.stdout:3/655: dwrite d1/de/f54 [0,4194304] 0 2024-04-02T04:43:04.259 INFO:tasks.workunit.client.1.smithi027.stdout:4/614: mkdir d24/d38/d47/dc0/dca 0 2024-04-02T04:43:04.261 INFO:tasks.workunit.client.1.smithi027.stdout:4/615: symlink d24/d38/d44/d60/lcb 0 2024-04-02T04:43:04.264 INFO:tasks.workunit.client.1.smithi027.stdout:4/616: rmdir d24 39 2024-04-02T04:43:04.266 INFO:tasks.workunit.client.1.smithi027.stdout:3/656: dread d1/de/df/d13/f85 [0,4194304] 0 2024-04-02T04:43:04.269 INFO:tasks.workunit.client.1.smithi027.stdout:3/657: creat d1/de/df/d13/fda x:0 0 0 2024-04-02T04:43:04.270 INFO:tasks.workunit.client.1.smithi027.stdout:3/658: chown d1/de/df/d13/db3/fd6 89 1 2024-04-02T04:43:04.271 INFO:tasks.workunit.client.1.smithi027.stdout:3/659: chown d1/de/df/d9e/fb4 163923464 1 2024-04-02T04:43:04.275 INFO:tasks.workunit.client.1.smithi027.stdout:3/660: truncate d1/de/df/f5d 3231639 0 2024-04-02T04:43:04.275 INFO:tasks.workunit.client.1.smithi027.stdout:3/661: readlink d1/de/d55/l88 0 2024-04-02T04:43:04.276 INFO:tasks.workunit.client.1.smithi027.stdout:3/662: chown d1/d1a/c65 32 1 2024-04-02T04:43:04.298 INFO:tasks.workunit.client.1.smithi027.stdout:3/663: write d1/de/df/f17 [3283870,37153] 0 2024-04-02T04:43:04.381 INFO:tasks.workunit.client.1.smithi027.stdout:5/844: sync 2024-04-02T04:43:04.381 INFO:tasks.workunit.client.1.smithi027.stdout:5/845: stat d0/d1a 0 2024-04-02T04:43:04.381 INFO:tasks.workunit.client.1.smithi027.stdout:5/846: dread - d0/df/d11/d37/f113 zero size 2024-04-02T04:43:04.388 INFO:tasks.workunit.client.1.smithi027.stdout:5/847: creat d0/df/f121 x:0 0 0 2024-04-02T04:43:04.394 INFO:tasks.workunit.client.1.smithi027.stdout:5/848: dwrite d0/df/d11/d37/f113 [0,4194304] 0 2024-04-02T04:43:04.397 INFO:tasks.workunit.client.1.smithi027.stdout:5/849: creat d0/d1d/f122 x:0 0 0 2024-04-02T04:43:04.403 INFO:tasks.workunit.client.1.smithi027.stdout:5/850: getdents d0/d68/d6b/d74 0 2024-04-02T04:43:04.407 INFO:tasks.workunit.client.1.smithi027.stdout:5/851: rename d0/d1d/cfa to d0/d1d/d61/d72/d115/c123 0 2024-04-02T04:43:04.548 INFO:tasks.workunit.client.1.smithi027.stdout:6/612: sync 2024-04-02T04:43:04.549 INFO:tasks.workunit.client.1.smithi027.stdout:0/746: sync 2024-04-02T04:43:04.549 INFO:tasks.workunit.client.1.smithi027.stdout:6/613: dread - d13/d8d/f94 zero size 2024-04-02T04:43:04.562 INFO:tasks.workunit.client.1.smithi027.stdout:0/747: dread d4/d69/f8a [0,4194304] 0 2024-04-02T04:43:04.566 INFO:tasks.workunit.client.1.smithi027.stdout:6/614: dread d13/d1b/d24/d25/d27/f44 [0,4194304] 0 2024-04-02T04:43:04.570 INFO:tasks.workunit.client.1.smithi027.stdout:0/748: write d4/d76/f99 [986297,84456] 0 2024-04-02T04:43:04.574 INFO:tasks.workunit.client.1.smithi027.stdout:0/749: unlink d4/d21/d85/fad 0 2024-04-02T04:43:04.578 INFO:tasks.workunit.client.1.smithi027.stdout:0/750: mknod d4/d16/d1b/c101 0 2024-04-02T04:43:04.578 INFO:tasks.workunit.client.1.smithi027.stdout:0/751: chown d4 3 1 2024-04-02T04:43:04.580 INFO:tasks.workunit.client.1.smithi027.stdout:6/615: dread d13/d8d/fa3 [0,4194304] 0 2024-04-02T04:43:04.580 INFO:tasks.workunit.client.1.smithi027.stdout:9/821: sync 2024-04-02T04:43:04.581 INFO:tasks.workunit.client.1.smithi027.stdout:1/677: sync 2024-04-02T04:43:04.581 INFO:tasks.workunit.client.1.smithi027.stdout:2/827: sync 2024-04-02T04:43:04.581 INFO:tasks.workunit.client.1.smithi027.stdout:8/703: sync 2024-04-02T04:43:04.581 INFO:tasks.workunit.client.1.smithi027.stdout:7/681: sync 2024-04-02T04:43:04.581 INFO:tasks.workunit.client.1.smithi027.stdout:3/664: sync 2024-04-02T04:43:04.581 INFO:tasks.workunit.client.1.smithi027.stdout:5/852: sync 2024-04-02T04:43:04.581 INFO:tasks.workunit.client.1.smithi027.stdout:4/617: sync 2024-04-02T04:43:04.584 INFO:tasks.workunit.client.1.smithi027.stdout:1/678: read d0/d1e/d2e/d3b/f77 [6721038,12332] 0 2024-04-02T04:43:04.585 INFO:tasks.workunit.client.1.smithi027.stdout:8/704: read - d5/d8/db/dd/fd2 zero size 2024-04-02T04:43:04.587 INFO:tasks.workunit.client.1.smithi027.stdout:2/828: chown d1/db/d3b/d17/d68/l6b 3016991 1 2024-04-02T04:43:04.590 INFO:tasks.workunit.client.1.smithi027.stdout:5/853: stat d0/df/d11/d65/ld6 0 2024-04-02T04:43:04.590 INFO:tasks.workunit.client.1.smithi027.stdout:7/682: creat d2/dba/dc/d16/d3f/d50/d8b/db6/fd6 x:0 0 0 2024-04-02T04:43:04.591 INFO:tasks.workunit.client.1.smithi027.stdout:9/822: mknod d1a/d1d/d29/d37/d20/d7e/c10e 0 2024-04-02T04:43:04.595 INFO:tasks.workunit.client.1.smithi027.stdout:6/616: dwrite d13/d1b/d24/fd1 [4194304,4194304] 0 2024-04-02T04:43:04.596 INFO:tasks.workunit.client.1.smithi027.stdout:4/618: write d24/f28 [2363045,124270] 0 2024-04-02T04:43:04.596 INFO:tasks.workunit.client.1.smithi027.stdout:3/665: symlink d1/de/df/d13/d73/ldb 0 2024-04-02T04:43:04.599 INFO:tasks.workunit.client.1.smithi027.stdout:0/752: dread d4/d16/d1b/f20 [4194304,4194304] 0 2024-04-02T04:43:04.603 INFO:tasks.workunit.client.1.smithi027.stdout:7/683: unlink d2/df/fd1 0 2024-04-02T04:43:04.604 INFO:tasks.workunit.client.1.smithi027.stdout:7/684: stat d2/l6c 0 2024-04-02T04:43:04.609 INFO:tasks.workunit.client.1.smithi027.stdout:2/829: mkdir d1/db/d3b/d17/d6d/dd9/dd5/dda/dbb/db4/d12b 0 2024-04-02T04:43:04.609 INFO:tasks.workunit.client.1.smithi027.stdout:6/617: dwrite d13/d1b/d24/d28/d3c/f78 [8388608,4194304] 0 2024-04-02T04:43:04.610 INFO:tasks.workunit.client.1.smithi027.stdout:1/679: write d0/d1e/f9f [950299,84923] 0 2024-04-02T04:43:04.614 INFO:tasks.workunit.client.1.smithi027.stdout:8/705: creat d5/db5/ddc/fed x:0 0 0 2024-04-02T04:43:04.622 INFO:tasks.workunit.client.1.smithi027.stdout:0/753: mknod d4/d38/d66/d2d/d4c/c102 0 2024-04-02T04:43:04.623 INFO:tasks.workunit.client.1.smithi027.stdout:0/754: chown d4/de0/le4 825077 1 2024-04-02T04:43:04.624 INFO:tasks.workunit.client.1.smithi027.stdout:7/685: mkdir d2/d1d/dd7 0 2024-04-02T04:43:04.626 INFO:tasks.workunit.client.1.smithi027.stdout:2/830: unlink d1/db/l19 0 2024-04-02T04:43:04.628 INFO:tasks.workunit.client.1.smithi027.stdout:2/831: dread - d1/db/d3b/d17/d6d/d84/f118 zero size 2024-04-02T04:43:04.631 INFO:tasks.workunit.client.1.smithi027.stdout:5/854: write d0/df/d11/d13/d9a/d9c/dbe/feb [439498,99942] 0 2024-04-02T04:43:04.632 INFO:tasks.workunit.client.1.smithi027.stdout:0/755: mknod d4/de0/de2/c103 0 2024-04-02T04:43:04.634 INFO:tasks.workunit.client.1.smithi027.stdout:9/823: link d1a/d1d/d29/d37/d99/de8/f101 d1a/d1d/d29/d56/de4/f10f 0 2024-04-02T04:43:04.642 INFO:tasks.workunit.client.1.smithi027.stdout:1/680: symlink d0/de/d52/d74/dd9/ldf 0 2024-04-02T04:43:04.643 INFO:tasks.workunit.client.1.smithi027.stdout:3/666: link d1/d1a/l59 d1/de/df/d13/db3/ldc 0 2024-04-02T04:43:04.645 INFO:tasks.workunit.client.1.smithi027.stdout:3/667: dread - d1/de/fa7 zero size 2024-04-02T04:43:04.648 INFO:tasks.workunit.client.1.smithi027.stdout:4/619: rename d24/d38/d44/c9c to d24/d38/d47/ccc 0 2024-04-02T04:43:04.649 INFO:tasks.workunit.client.1.smithi027.stdout:6/618: dwrite f7 [0,4194304] 0 2024-04-02T04:43:04.650 INFO:tasks.workunit.client.1.smithi027.stdout:6/619: readlink d13/l19 0 2024-04-02T04:43:04.651 INFO:tasks.workunit.client.1.smithi027.stdout:6/620: chown d13/d1b/f2b 21539000 1 2024-04-02T04:43:04.654 INFO:tasks.workunit.client.1.smithi027.stdout:8/706: truncate d5/d8/d15/d71/d96/fcc 3220871 0 2024-04-02T04:43:04.655 INFO:tasks.workunit.client.1.smithi027.stdout:5/855: dread d0/fb0 [4194304,4194304] 0 2024-04-02T04:43:04.656 INFO:tasks.workunit.client.1.smithi027.stdout:7/686: symlink d2/dba/dc/d16/d3f/dc0/ld8 0 2024-04-02T04:43:04.658 INFO:tasks.workunit.client.1.smithi027.stdout:9/824: mknod d1a/d1d/d29/d50/d73/dbd/dd1/c110 0 2024-04-02T04:43:04.714 INFO:tasks.workunit.client.1.smithi027.stdout:2/832: creat d1/db/d3b/d17/d6d/dd9/dd5/dda/dbb/db4/ddf/f12c x:0 0 0 2024-04-02T04:43:04.729 INFO:tasks.workunit.client.1.smithi027.stdout:9/825: mknod d1a/d1d/d29/d37/d9b/c111 0 2024-04-02T04:43:04.730 INFO:tasks.workunit.client.1.smithi027.stdout:9/826: truncate d1a/f38 4371717 0 2024-04-02T04:43:04.731 INFO:tasks.workunit.client.1.smithi027.stdout:9/827: fsync d1a/d1d/d3c/d6e/db8/fb9 0 2024-04-02T04:43:04.731 INFO:tasks.workunit.client.1.smithi027.stdout:1/681: getdents d0/d1e/d2e/d3b/d4f/dd3/dde 0 2024-04-02T04:43:04.732 INFO:tasks.workunit.client.1.smithi027.stdout:1/682: chown d0/de/d52/d74/d97/fc2 8 1 2024-04-02T04:43:04.732 INFO:tasks.workunit.client.1.smithi027.stdout:2/833: rmdir d1/db/d3b/d61/d75 39 2024-04-02T04:43:04.738 INFO:tasks.workunit.client.1.smithi027.stdout:1/683: dwrite d0/d1e/d2e/d3b/d4f/fac [0,4194304] 0 2024-04-02T04:43:04.741 INFO:tasks.workunit.client.1.smithi027.stdout:8/707: fdatasync d5/d8/db/dd/d5f/d27/f3a 0 2024-04-02T04:43:04.745 INFO:tasks.workunit.client.1.smithi027.stdout:5/856: creat d0/df/d11/d37/d120/f124 x:0 0 0 2024-04-02T04:43:04.756 INFO:tasks.workunit.client.1.smithi027.stdout:2/834: dread d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/dc0/dc3/fe8 [0,4194304] 0 2024-04-02T04:43:04.757 INFO:tasks.workunit.client.1.smithi027.stdout:1/684: mknod d0/d1e/d2e/d3b/dda/ce0 0 2024-04-02T04:43:04.758 INFO:tasks.workunit.client.1.smithi027.stdout:2/835: write d1/db/d3b/f1c [1904824,106088] 0 2024-04-02T04:43:04.759 INFO:tasks.workunit.client.1.smithi027.stdout:8/708: mknod d5/d8/db/dd/d5f/d27/d31/d5b/dc4/dea/cee 0 2024-04-02T04:43:04.760 INFO:tasks.workunit.client.1.smithi027.stdout:8/709: dread - d5/d8/db/dd/fd2 zero size 2024-04-02T04:43:04.761 INFO:tasks.workunit.client.1.smithi027.stdout:5/857: mkdir d0/d68/d6b/d74/d125 0 2024-04-02T04:43:04.761 INFO:tasks.workunit.client.1.smithi027.stdout:9/828: symlink d1a/d1d/d29/d37/d107/l112 0 2024-04-02T04:43:04.762 INFO:tasks.workunit.client.1.smithi027.stdout:8/710: readlink d5/d8/db/dd/d5f/d27/l97 0 2024-04-02T04:43:04.769 INFO:tasks.workunit.client.1.smithi027.stdout:5/858: truncate d0/df/d76/f85 398124 0 2024-04-02T04:43:04.770 INFO:tasks.workunit.client.1.smithi027.stdout:9/829: creat d1a/d1d/d29/d50/d73/dbd/d10a/f113 x:0 0 0 2024-04-02T04:43:04.772 INFO:tasks.workunit.client.1.smithi027.stdout:2/836: symlink d1/db/d3b/d61/d75/d100/l12d 0 2024-04-02T04:43:04.772 INFO:tasks.workunit.client.1.smithi027.stdout:5/859: dread d0/fc5 [0,4194304] 0 2024-04-02T04:43:04.773 INFO:tasks.workunit.client.1.smithi027.stdout:1/685: link d0/d1e/d2e/d3b/d4f/ccd d0/d1e/d2e/d3b/d4f/ce1 0 2024-04-02T04:43:04.774 INFO:tasks.workunit.client.1.smithi027.stdout:2/837: chown d1/db/d3b/d17/d6d/dd9/dd5/f4f 13778 1 2024-04-02T04:43:04.775 INFO:tasks.workunit.client.1.smithi027.stdout:9/830: symlink d1a/d1d/d29/d50/d73/dbd/d10a/l114 0 2024-04-02T04:43:04.777 INFO:tasks.workunit.client.1.smithi027.stdout:9/831: readlink d1a/l7f 0 2024-04-02T04:43:04.779 INFO:tasks.workunit.client.1.smithi027.stdout:0/756: truncate d4/d69/f8a 1630377 0 2024-04-02T04:43:04.779 INFO:tasks.workunit.client.1.smithi027.stdout:5/860: truncate d0/df/d29/da5/da6/f103 1376032 0 2024-04-02T04:43:04.780 INFO:tasks.workunit.client.1.smithi027.stdout:7/687: write d2/dba/dc/d1f/d33/d36/f7f [1885490,113712] 0 2024-04-02T04:43:04.789 INFO:tasks.workunit.client.1.smithi027.stdout:0/757: dread - d4/d38/d66/ff0 zero size 2024-04-02T04:43:04.789 INFO:tasks.workunit.client.1.smithi027.stdout:3/668: dwrite d1/de/f30 [0,4194304] 0 2024-04-02T04:43:04.790 INFO:tasks.workunit.client.1.smithi027.stdout:7/688: chown d2/d1d/d92/fc4 24686077 1 2024-04-02T04:43:04.790 INFO:tasks.workunit.client.1.smithi027.stdout:5/861: chown d0/df/d2f/d3b/d51/ced 0 1 2024-04-02T04:43:04.792 INFO:tasks.workunit.client.1.smithi027.stdout:1/686: dwrite d0/d1e/d5d/fa1 [0,4194304] 0 2024-04-02T04:43:04.795 INFO:tasks.workunit.client.1.smithi027.stdout:9/832: symlink d1a/d1d/d29/d37/d20/dcc/dd5/de2/l115 0 2024-04-02T04:43:04.795 INFO:tasks.workunit.client.1.smithi027.stdout:4/620: dwrite d24/d38/d44/d82/fb7 [0,4194304] 0 2024-04-02T04:43:04.799 INFO:tasks.workunit.client.1.smithi027.stdout:0/758: fdatasync d4/d38/d66/f2a 0 2024-04-02T04:43:04.801 INFO:tasks.workunit.client.1.smithi027.stdout:6/621: dwrite d13/d1b/d24/d25/d56/dc8/dc9/fcd [0,4194304] 0 2024-04-02T04:43:04.803 INFO:tasks.workunit.client.1.smithi027.stdout:0/759: fdatasync d4/d16/f27 0 2024-04-02T04:43:04.803 INFO:tasks.workunit.client.1.smithi027.stdout:5/862: rmdir d0/df/d11/d13/d9a 39 2024-04-02T04:43:04.804 INFO:tasks.workunit.client.1.smithi027.stdout:5/863: chown d0/d1a 219093694 1 2024-04-02T04:43:04.805 INFO:tasks.workunit.client.1.smithi027.stdout:1/687: read d0/d1e/d2e/d3b/d4f/f75 [81414,68662] 0 2024-04-02T04:43:04.806 INFO:tasks.workunit.client.1.smithi027.stdout:9/833: read d1a/d1d/d29/d51/d105/faf [1128033,10215] 0 2024-04-02T04:43:04.807 INFO:tasks.workunit.client.1.smithi027.stdout:7/689: creat d2/dba/dc/d16/d3f/d50/d8b/db6/fd9 x:0 0 0 2024-04-02T04:43:04.810 INFO:tasks.workunit.client.1.smithi027.stdout:3/669: mknod d1/de/d55/d63/da4/cdd 0 2024-04-02T04:43:04.811 INFO:tasks.workunit.client.1.smithi027.stdout:7/690: readlink d2/d1d/d92/lce 0 2024-04-02T04:43:04.818 INFO:tasks.workunit.client.1.smithi027.stdout:5/864: rmdir d0/df/d11/d65/dac 39 2024-04-02T04:43:04.820 INFO:tasks.workunit.client.1.smithi027.stdout:1/688: truncate d0/de/fcc 4117677 0 2024-04-02T04:43:04.823 INFO:tasks.workunit.client.1.smithi027.stdout:3/670: mknod d1/de/d55/d63/db9/cde 0 2024-04-02T04:43:04.828 INFO:tasks.workunit.client.1.smithi027.stdout:5/865: mknod d0/d1d/d61/d72/d115/c126 0 2024-04-02T04:43:04.834 INFO:tasks.workunit.client.1.smithi027.stdout:9/834: creat d1a/d1d/d29/d51/d105/d8c/de1/d100/f116 x:0 0 0 2024-04-02T04:43:04.837 INFO:tasks.workunit.client.1.smithi027.stdout:0/760: dread d4/d21/d28/d4e/d5f/f5a [0,4194304] 0 2024-04-02T04:43:04.839 INFO:tasks.workunit.client.1.smithi027.stdout:6/622: write d13/d1b/d24/d25/d27/f76 [1487605,45730] 0 2024-04-02T04:43:04.844 INFO:tasks.workunit.client.1.smithi027.stdout:4/621: dwrite d24/d38/d47/f93 [0,4194304] 0 2024-04-02T04:43:04.852 INFO:tasks.workunit.client.1.smithi027.stdout:7/691: dwrite d2/dba/d35/f8c [0,4194304] 0 2024-04-02T04:43:04.856 INFO:tasks.workunit.client.1.smithi027.stdout:1/689: dread d0/f69 [0,4194304] 0 2024-04-02T04:43:04.859 INFO:tasks.workunit.client.1.smithi027.stdout:0/761: creat d4/d21/d28/d4e/db5/f104 x:0 0 0 2024-04-02T04:43:04.861 INFO:tasks.workunit.client.1.smithi027.stdout:9/835: dread d1a/d1d/d3c/f94 [0,4194304] 0 2024-04-02T04:43:04.863 INFO:tasks.workunit.client.1.smithi027.stdout:9/836: stat d1a/d1d/d29/d50/d73/dbd/dd1/c110 0 2024-04-02T04:43:04.866 INFO:tasks.workunit.client.1.smithi027.stdout:0/762: dread d4/d21/d28/f40 [0,4194304] 0 2024-04-02T04:43:04.868 INFO:tasks.workunit.client.1.smithi027.stdout:0/763: dread - d4/d38/d66/ff0 zero size 2024-04-02T04:43:04.869 INFO:tasks.workunit.client.1.smithi027.stdout:3/671: creat d1/de/df/d13/d73/d77/fdf x:0 0 0 2024-04-02T04:43:04.872 INFO:tasks.workunit.client.1.smithi027.stdout:7/692: creat d2/dba/dc/d16/d3f/fda x:0 0 0 2024-04-02T04:43:04.875 INFO:tasks.workunit.client.1.smithi027.stdout:5/866: mkdir d0/df/d11/d13/d9a/d9c/dd0/d127 0 2024-04-02T04:43:04.879 INFO:tasks.workunit.client.1.smithi027.stdout:7/693: dwrite d2/dba/dc/d16/d3f/d44/fb4 [0,4194304] 0 2024-04-02T04:43:04.882 INFO:tasks.workunit.client.1.smithi027.stdout:9/837: rename d1a/d1d/d29/d37/d99/de8 to d1a/d1d/d29/d50/d117 0 2024-04-02T04:43:04.884 INFO:tasks.workunit.client.1.smithi027.stdout:9/838: read d1a/f1f [2192595,77017] 0 2024-04-02T04:43:04.884 INFO:tasks.workunit.client.1.smithi027.stdout:3/672: creat d1/d1a/fe0 x:0 0 0 2024-04-02T04:43:04.884 INFO:tasks.workunit.client.1.smithi027.stdout:4/622: write d24/d38/d44/d60/f94 [144528,68968] 0 2024-04-02T04:43:04.889 INFO:tasks.workunit.client.1.smithi027.stdout:6/623: dwrite d13/d1b/d24/d25/d27/f97 [0,4194304] 0 2024-04-02T04:43:04.889 INFO:tasks.workunit.client.1.smithi027.stdout:4/623: readlink d24/d38/d44/dbc/lc1 0 2024-04-02T04:43:04.890 INFO:tasks.workunit.client.1.smithi027.stdout:3/673: dread d1/de/df/d13/f85 [0,4194304] 0 2024-04-02T04:43:04.892 INFO:tasks.workunit.client.1.smithi027.stdout:3/674: write d1/d1a/fbd [587742,81276] 0 2024-04-02T04:43:04.892 INFO:tasks.workunit.client.1.smithi027.stdout:1/690: write d0/f6 [629592,28747] 0 2024-04-02T04:43:04.894 INFO:tasks.workunit.client.1.smithi027.stdout:3/675: chown d1/de/f42 1923489 1 2024-04-02T04:43:04.895 INFO:tasks.workunit.client.1.smithi027.stdout:3/676: write d1/f2a [3556949,122460] 0 2024-04-02T04:43:04.896 INFO:tasks.workunit.client.1.smithi027.stdout:3/677: write d1/de/d55/fa1 [640007,45696] 0 2024-04-02T04:43:04.898 INFO:tasks.workunit.client.1.smithi027.stdout:3/678: write d1/de/f30 [4495148,113836] 0 2024-04-02T04:43:04.900 INFO:tasks.workunit.client.1.smithi027.stdout:9/839: mkdir d1a/d1d/d29/d37/d9b/d118 0 2024-04-02T04:43:04.902 INFO:tasks.workunit.client.1.smithi027.stdout:7/694: dread d2/df/f85 [0,4194304] 0 2024-04-02T04:43:04.906 INFO:tasks.workunit.client.1.smithi027.stdout:5/867: write d0/df/d11/d13/f1e [3741049,36887] 0 2024-04-02T04:43:04.910 INFO:tasks.workunit.client.1.smithi027.stdout:6/624: write d13/d1b/d24/f70 [911343,96225] 0 2024-04-02T04:43:04.911 INFO:tasks.workunit.client.1.smithi027.stdout:6/625: chown d13/d1b/c2e 95997013 1 2024-04-02T04:43:04.912 INFO:tasks.workunit.client.1.smithi027.stdout:6/626: write f7 [4312780,96032] 0 2024-04-02T04:43:04.919 INFO:tasks.workunit.client.1.smithi027.stdout:3/679: dread d1/d1a/f86 [0,4194304] 0 2024-04-02T04:43:04.922 INFO:tasks.workunit.client.1.smithi027.stdout:6/627: dwrite d13/d1b/d24/d28/d3c/f78 [8388608,4194304] 0 2024-04-02T04:43:04.924 INFO:tasks.workunit.client.1.smithi027.stdout:9/840: dread d1a/d1d/d29/d51/fa6 [0,4194304] 0 2024-04-02T04:43:04.924 INFO:tasks.workunit.client.1.smithi027.stdout:5/868: dread d0/df/d11/d13/d9a/db7/fc6 [0,4194304] 0 2024-04-02T04:43:04.925 INFO:tasks.workunit.client.1.smithi027.stdout:5/869: stat d0/d1a/f49 0 2024-04-02T04:43:04.926 INFO:tasks.workunit.client.1.smithi027.stdout:0/764: rename d4/d21/d85/cb9 to d4/d21/d28/c105 0 2024-04-02T04:43:04.928 INFO:tasks.workunit.client.1.smithi027.stdout:4/624: write d24/d48/f75 [4046878,55046] 0 2024-04-02T04:43:04.929 INFO:tasks.workunit.client.1.smithi027.stdout:4/625: fsync d24/fbb 0 2024-04-02T04:43:04.931 INFO:tasks.workunit.client.1.smithi027.stdout:3/680: fsync d1/de/d74/f8e 0 2024-04-02T04:43:04.932 INFO:tasks.workunit.client.1.smithi027.stdout:3/681: stat d1/d1a/cbc 0 2024-04-02T04:43:04.933 INFO:tasks.workunit.client.1.smithi027.stdout:6/628: mkdir d13/d1b/d48/d58/dd3 0 2024-04-02T04:43:04.934 INFO:tasks.workunit.client.1.smithi027.stdout:5/870: dread - d0/df/d29/da5/fd3 zero size 2024-04-02T04:43:04.935 INFO:tasks.workunit.client.1.smithi027.stdout:0/765: mknod d4/d16/c106 0 2024-04-02T04:43:04.937 INFO:tasks.workunit.client.1.smithi027.stdout:1/691: rename d0/d1e/d25/d3c/d64/da6/db8 to d0/de/d52/d74/dbc/de2 0 2024-04-02T04:43:04.938 INFO:tasks.workunit.client.1.smithi027.stdout:1/692: fsync d0/d1e/d2e/d3e/d5f/fc7 0 2024-04-02T04:43:04.940 INFO:tasks.workunit.client.1.smithi027.stdout:1/693: write d0/de/d52/d74/fc1 [366906,49617] 0 2024-04-02T04:43:04.942 INFO:tasks.workunit.client.1.smithi027.stdout:4/626: rmdir d24/d38/d44/d60 39 2024-04-02T04:43:04.945 INFO:tasks.workunit.client.1.smithi027.stdout:3/682: creat d1/de/df/d61/fe1 x:0 0 0 2024-04-02T04:43:04.952 INFO:tasks.workunit.client.1.smithi027.stdout:9/841: link d1a/d1d/d29/d50/d73/dbd/fbf d1a/d1d/d29/d37/d9b/d118/f119 0 2024-04-02T04:43:04.953 INFO:tasks.workunit.client.1.smithi027.stdout:1/694: rmdir d0/d1e/d2e/d3e 39 2024-04-02T04:43:04.954 INFO:tasks.workunit.client.1.smithi027.stdout:1/695: write d0/d1e/f9c [1285161,15410] 0 2024-04-02T04:43:04.956 INFO:tasks.workunit.client.1.smithi027.stdout:4/627: creat d24/d38/d44/d41/d5a/fcd x:0 0 0 2024-04-02T04:43:04.957 INFO:tasks.workunit.client.1.smithi027.stdout:6/629: mknod d13/d1b/d24/cd4 0 2024-04-02T04:43:04.958 INFO:tasks.workunit.client.1.smithi027.stdout:3/683: mkdir d1/de/df/d13/d73/de2 0 2024-04-02T04:43:04.959 INFO:tasks.workunit.client.1.smithi027.stdout:6/630: chown d13/d1b/d24/d25/d56/dc8/dc9/fcd 1872763 1 2024-04-02T04:43:04.962 INFO:tasks.workunit.client.1.smithi027.stdout:5/871: dwrite d0/df/d11/f58 [0,4194304] 0 2024-04-02T04:43:04.963 INFO:tasks.workunit.client.1.smithi027.stdout:7/695: rename d2/dba/dc/d16/d3f/f43 to d2/df/fdb 0 2024-04-02T04:43:04.970 INFO:tasks.workunit.client.1.smithi027.stdout:4/628: unlink d24/f71 0 2024-04-02T04:43:04.971 INFO:tasks.workunit.client.1.smithi027.stdout:3/684: fdatasync d1/de/df/d13/f43 0 2024-04-02T04:43:04.973 INFO:tasks.workunit.client.1.smithi027.stdout:6/631: fdatasync d13/d1b/d48/d75/f7d 0 2024-04-02T04:43:04.976 INFO:tasks.workunit.client.1.smithi027.stdout:0/766: rename d4/d16/d1b/f65 to d4/d76/dc4/dfe/dd1/f107 0 2024-04-02T04:43:04.977 INFO:tasks.workunit.client.1.smithi027.stdout:0/767: write d4/f90 [4445251,109488] 0 2024-04-02T04:43:04.978 INFO:tasks.workunit.client.1.smithi027.stdout:7/696: unlink d2/d1d/d92/f9a 0 2024-04-02T04:43:04.979 INFO:tasks.workunit.client.1.smithi027.stdout:7/697: chown d2/d1c/cbf 14 1 2024-04-02T04:43:04.980 INFO:tasks.workunit.client.1.smithi027.stdout:7/698: fdatasync d2/dba/dc/d16/d3f/d50/f69 0 2024-04-02T04:43:04.983 INFO:tasks.workunit.client.1.smithi027.stdout:4/629: write d24/d38/d44/d60/fc4 [476419,35800] 0 2024-04-02T04:43:04.992 INFO:tasks.workunit.client.1.smithi027.stdout:6/632: creat d13/d1b/d48/d75/fd5 x:0 0 0 2024-04-02T04:43:04.992 INFO:tasks.workunit.client.1.smithi027.stdout:4/630: dwrite d24/d38/d44/d82/d95/fc2 [0,4194304] 0 2024-04-02T04:43:04.993 INFO:tasks.workunit.client.1.smithi027.stdout:6/633: read d13/d1b/d24/f41 [1736816,117661] 0 2024-04-02T04:43:04.995 INFO:tasks.workunit.client.1.smithi027.stdout:9/842: rmdir d1a/d1d/d29/d50/d73/def 0 2024-04-02T04:43:04.997 INFO:tasks.workunit.client.1.smithi027.stdout:6/634: write d13/d1b/d24/d25/d56/d6d/fcf [10944,36309] 0 2024-04-02T04:43:04.997 INFO:tasks.workunit.client.1.smithi027.stdout:9/843: write f15 [5339804,90101] 0 2024-04-02T04:43:05.006 INFO:tasks.workunit.client.1.smithi027.stdout:0/768: creat d4/d69/f108 x:0 0 0 2024-04-02T04:43:05.010 INFO:tasks.workunit.client.1.smithi027.stdout:7/699: creat d2/d1c/fdc x:0 0 0 2024-04-02T04:43:05.014 INFO:tasks.workunit.client.1.smithi027.stdout:0/769: dwrite d4/d21/d28/d4e/d5f/f79 [0,4194304] 0 2024-04-02T04:43:05.015 INFO:tasks.workunit.client.1.smithi027.stdout:4/631: fdatasync d24/d38/d44/d60/f6f 0 2024-04-02T04:43:05.016 INFO:tasks.workunit.client.1.smithi027.stdout:0/770: chown d4/d21/d28/d4e/d5f/f79 525363 1 2024-04-02T04:43:05.018 INFO:tasks.workunit.client.1.smithi027.stdout:7/700: dread d2/dba/dc/d1f/d33/fd0 [0,4194304] 0 2024-04-02T04:43:05.018 INFO:tasks.workunit.client.1.smithi027.stdout:6/635: chown d13/d1b/d24/d25/d56/d73/fa1 907328 1 2024-04-02T04:43:05.018 INFO:tasks.workunit.client.1.smithi027.stdout:4/632: write d24/d38/d44/d82/d95/fb6 [409735,45715] 0 2024-04-02T04:43:05.027 INFO:tasks.workunit.client.1.smithi027.stdout:6/636: dwrite d13/d1b/d24/fd1 [4194304,4194304] 0 2024-04-02T04:43:05.030 INFO:tasks.workunit.client.1.smithi027.stdout:3/685: dwrite d1/de/df/d61/f6e [0,4194304] 0 2024-04-02T04:43:05.038 INFO:tasks.workunit.client.1.smithi027.stdout:0/771: fsync d4/f9b 0 2024-04-02T04:43:05.046 INFO:tasks.workunit.client.1.smithi027.stdout:7/701: creat d2/dba/dc/d16/d3a/d4d/dcb/fdd x:0 0 0 2024-04-02T04:43:05.052 INFO:tasks.workunit.client.1.smithi027.stdout:9/844: getdents d1a/d1d/d3c/d6e 0 2024-04-02T04:43:05.056 INFO:tasks.workunit.client.1.smithi027.stdout:7/702: mkdir d2/df/d89/dde 0 2024-04-02T04:43:05.057 INFO:tasks.workunit.client.1.smithi027.stdout:6/637: creat d13/d1b/d48/fd6 x:0 0 0 2024-04-02T04:43:05.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:04 smithi016 ceph-mon[28969]: pgmap v31: 65 pgs: 65 active+clean; 3.3 GiB data, 9.9 GiB used, 526 GiB / 536 GiB avail; 79 MiB/s rd, 200 MiB/s wr, 470 op/s 2024-04-02T04:43:05.059 INFO:tasks.workunit.client.1.smithi027.stdout:3/686: write d1/de/df/d13/d73/d77/fc4 [385700,27618] 0 2024-04-02T04:43:05.060 INFO:tasks.workunit.client.1.smithi027.stdout:4/633: getdents d24/d2d 0 2024-04-02T04:43:05.061 INFO:tasks.workunit.client.1.smithi027.stdout:3/687: stat d1/de/df/d13 0 2024-04-02T04:43:05.062 INFO:tasks.workunit.client.1.smithi027.stdout:0/772: link d4/d21/d85/db3/ce1 d4/d21/d93/c109 0 2024-04-02T04:43:05.066 INFO:tasks.workunit.client.1.smithi027.stdout:3/688: dread d1/de/df/d13/d73/d77/d7f/fca [0,4194304] 0 2024-04-02T04:43:05.071 INFO:tasks.workunit.client.1.smithi027.stdout:9/845: link d1a/d1d/d29/d37/d99/fc5 d1a/d1d/d29/d37/d20/f11a 0 2024-04-02T04:43:05.072 INFO:tasks.workunit.client.1.smithi027.stdout:9/846: write d1a/d1d/d29/d51/d105/d8c/de1/fe9 [77884,29314] 0 2024-04-02T04:43:05.074 INFO:tasks.workunit.client.1.smithi027.stdout:7/703: creat d2/dba/fdf x:0 0 0 2024-04-02T04:43:05.074 INFO:tasks.workunit.client.1.smithi027.stdout:9/847: dread - d1a/d1d/d29/d56/de4/f10f zero size 2024-04-02T04:43:05.075 INFO:tasks.workunit.client.1.smithi027.stdout:4/634: write d24/f33 [332062,8452] 0 2024-04-02T04:43:05.076 INFO:tasks.workunit.client.1.smithi027.stdout:6/638: creat d13/d1b/d48/fd7 x:0 0 0 2024-04-02T04:43:05.078 INFO:tasks.workunit.client.1.smithi027.stdout:3/689: mkdir d1/de/d55/d63/da4/de3 0 2024-04-02T04:43:05.079 INFO:tasks.workunit.client.1.smithi027.stdout:0/773: fsync d4/d21/f2e 0 2024-04-02T04:43:05.091 INFO:tasks.workunit.client.1.smithi027.stdout:9/848: dread d1a/d1d/d3c/d6e/db8/fb9 [0,4194304] 0 2024-04-02T04:43:05.096 INFO:tasks.workunit.client.1.smithi027.stdout:0/774: unlink d4/d38/d66/c12 0 2024-04-02T04:43:05.097 INFO:tasks.workunit.client.1.smithi027.stdout:7/704: unlink d2/d1d/d90/l9f 0 2024-04-02T04:43:05.101 INFO:tasks.workunit.client.1.smithi027.stdout:9/849: symlink d1a/l11b 0 2024-04-02T04:43:05.102 INFO:tasks.workunit.client.1.smithi027.stdout:6/639: dwrite d13/d1b/d48/d75/f9f [0,4194304] 0 2024-04-02T04:43:05.106 INFO:tasks.workunit.client.1.smithi027.stdout:3/690: mkdir d1/de/df/d9e/dc6/de4 0 2024-04-02T04:43:05.114 INFO:tasks.workunit.client.1.smithi027.stdout:9/850: dwrite d1a/d1d/d29/d37/d20/dad/fdf [0,4194304] 0 2024-04-02T04:43:05.114 INFO:tasks.workunit.client.1.smithi027.stdout:7/705: chown d2/dba/dc/d16/d3f/d44/cb5 934 1 2024-04-02T04:43:05.114 INFO:tasks.workunit.client.1.smithi027.stdout:4/635: dwrite d24/d38/d44/d41/f66 [0,4194304] 0 2024-04-02T04:43:05.117 INFO:tasks.workunit.client.1.smithi027.stdout:9/851: dread - d1a/d1d/d29/d50/d73/ffe zero size 2024-04-02T04:43:05.118 INFO:tasks.workunit.client.1.smithi027.stdout:6/640: creat d13/d1b/d24/d25/d56/dc8/dc9/fd8 x:0 0 0 2024-04-02T04:43:05.122 INFO:tasks.workunit.client.1.smithi027.stdout:0/775: rename d4/d21/c8d to d4/d69/c10a 0 2024-04-02T04:43:05.131 INFO:tasks.workunit.client.1.smithi027.stdout:7/706: creat d2/dba/dc/d16/d3f/d44/fe0 x:0 0 0 2024-04-02T04:43:05.132 INFO:tasks.workunit.client.1.smithi027.stdout:6/641: dwrite d13/d1b/d24/d25/d56/d73/d96/fb2 [0,4194304] 0 2024-04-02T04:43:05.132 INFO:tasks.workunit.client.1.smithi027.stdout:4/636: symlink d24/d38/d44/d60/lce 0 2024-04-02T04:43:05.132 INFO:tasks.workunit.client.1.smithi027.stdout:4/637: fdatasync d24/d38/d44/d82/d95/faf 0 2024-04-02T04:43:05.139 INFO:tasks.workunit.client.1.smithi027.stdout:6/642: dwrite d13/d1b/d24/fd1 [0,4194304] 0 2024-04-02T04:43:05.142 INFO:tasks.workunit.client.1.smithi027.stdout:3/691: write d1/de/df/d13/f21 [1569813,31915] 0 2024-04-02T04:43:05.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:04 smithi027 ceph-mon[37550]: pgmap v31: 65 pgs: 65 active+clean; 3.3 GiB data, 9.9 GiB used, 526 GiB / 536 GiB avail; 79 MiB/s rd, 200 MiB/s wr, 470 op/s 2024-04-02T04:43:05.147 INFO:tasks.workunit.client.1.smithi027.stdout:9/852: creat d1a/d1d/d29/d51/d105/d102/f11c x:0 0 0 2024-04-02T04:43:05.151 INFO:tasks.workunit.client.1.smithi027.stdout:0/776: chown d4/d21/d85/fa6 0 1 2024-04-02T04:43:05.152 INFO:tasks.workunit.client.1.smithi027.stdout:0/777: fdatasync d4/f9 0 2024-04-02T04:43:05.152 INFO:tasks.workunit.client.1.smithi027.stdout:6/643: truncate d13/d1b/f1e 631261 0 2024-04-02T04:43:05.153 INFO:tasks.workunit.client.1.smithi027.stdout:0/778: write d4/de0/fba [40184,56215] 0 2024-04-02T04:43:05.155 INFO:tasks.workunit.client.1.smithi027.stdout:3/692: creat d1/de/df/d9e/da0/fe5 x:0 0 0 2024-04-02T04:43:05.161 INFO:tasks.workunit.client.1.smithi027.stdout:9/853: getdents d1a/dd6 0 2024-04-02T04:43:05.162 INFO:tasks.workunit.client.1.smithi027.stdout:6/644: truncate d13/d1b/d48/d75/f7d 1935833 0 2024-04-02T04:43:05.169 INFO:tasks.workunit.client.1.smithi027.stdout:7/707: dwrite d2/dba/dc/f83 [0,4194304] 0 2024-04-02T04:43:05.169 INFO:tasks.workunit.client.1.smithi027.stdout:9/854: fdatasync d1a/d1d/d29/d37/d4c/f66 0 2024-04-02T04:43:05.174 INFO:tasks.workunit.client.1.smithi027.stdout:0/779: read d4/d21/d28/f48 [4003899,75479] 0 2024-04-02T04:43:05.175 INFO:tasks.workunit.client.1.smithi027.stdout:7/708: read d2/dba/dc/d1f/fa1 [2924108,114529] 0 2024-04-02T04:43:05.175 INFO:tasks.workunit.client.1.smithi027.stdout:6/645: rmdir d13/d1b/d48/d58 39 2024-04-02T04:43:05.178 INFO:tasks.workunit.client.1.smithi027.stdout:9/855: dwrite d1a/d1d/d29/d37/d4c/dc9/f108 [0,4194304] 0 2024-04-02T04:43:05.180 INFO:tasks.workunit.client.1.smithi027.stdout:9/856: write f15 [1477791,82562] 0 2024-04-02T04:43:05.181 INFO:tasks.workunit.client.1.smithi027.stdout:4/638: link f1d d24/d38/fcf 0 2024-04-02T04:43:05.183 INFO:tasks.workunit.client.1.smithi027.stdout:4/639: chown d24/d38/d47/d62/f9a 3877671 1 2024-04-02T04:43:05.184 INFO:tasks.workunit.client.1.smithi027.stdout:4/640: write f12 [2887390,14630] 0 2024-04-02T04:43:05.191 INFO:tasks.workunit.client.1.smithi027.stdout:3/693: creat d1/fe6 x:0 0 0 2024-04-02T04:43:05.196 INFO:tasks.workunit.client.1.smithi027.stdout:0/780: write d4/d21/d85/fa6 [892407,85702] 0 2024-04-02T04:43:05.198 INFO:tasks.workunit.client.1.smithi027.stdout:4/641: creat d24/d38/d47/d62/fd0 x:0 0 0 2024-04-02T04:43:05.200 INFO:tasks.workunit.client.1.smithi027.stdout:6/646: mknod d13/d1b/d24/cd9 0 2024-04-02T04:43:05.205 INFO:tasks.workunit.client.1.smithi027.stdout:0/781: creat d4/d69/f10b x:0 0 0 2024-04-02T04:43:05.208 INFO:tasks.workunit.client.1.smithi027.stdout:1/696: sync 2024-04-02T04:43:05.208 INFO:tasks.workunit.client.1.smithi027.stdout:8/711: sync 2024-04-02T04:43:05.208 INFO:tasks.workunit.client.1.smithi027.stdout:2/838: sync 2024-04-02T04:43:05.209 INFO:tasks.workunit.client.1.smithi027.stdout:9/857: truncate d1a/d1d/d29/d37/d3a/f54 6032792 0 2024-04-02T04:43:05.210 INFO:tasks.workunit.client.1.smithi027.stdout:6/647: rmdir d13/d1b/d48/d75/d9b 39 2024-04-02T04:43:05.217 INFO:tasks.workunit.client.1.smithi027.stdout:7/709: link d2/dba/d35/ld3 d2/df/d89/d8d/le1 0 2024-04-02T04:43:05.217 INFO:tasks.workunit.client.1.smithi027.stdout:7/710: stat d2/d1c/f40 0 2024-04-02T04:43:05.222 INFO:tasks.workunit.client.1.smithi027.stdout:9/858: creat d1a/d1d/d29/d50/d73/dbd/d10a/f11d x:0 0 0 2024-04-02T04:43:05.223 INFO:tasks.workunit.client.1.smithi027.stdout:3/694: truncate d1/f45 5373721 0 2024-04-02T04:43:05.224 INFO:tasks.workunit.client.1.smithi027.stdout:9/859: chown d1a/d1d/d3c/d6e/db8/cff 0 1 2024-04-02T04:43:05.224 INFO:tasks.workunit.client.1.smithi027.stdout:3/695: readlink d1/de/d55/d63/db9/lba 0 2024-04-02T04:43:05.227 INFO:tasks.workunit.client.1.smithi027.stdout:8/712: rename d5/d8/db/d57/f62 to d5/d8/db/dd/d5f/d27/d31/d5b/dc4/fef 0 2024-04-02T04:43:05.229 INFO:tasks.workunit.client.1.smithi027.stdout:0/782: unlink d4/d21/d28/c105 0 2024-04-02T04:43:05.230 INFO:tasks.workunit.client.1.smithi027.stdout:4/642: link d24/d38/d47/d5c/f8c d24/d38/d44/d41/d5a/fd1 0 2024-04-02T04:43:05.231 INFO:tasks.workunit.client.1.smithi027.stdout:4/643: chown d24/d38/d44/d41/d49/l56 85713081 1 2024-04-02T04:43:05.231 INFO:tasks.workunit.client.1.smithi027.stdout:7/711: symlink d2/dba/dc/d1f/d61/le2 0 2024-04-02T04:43:05.231 INFO:tasks.workunit.client.1.smithi027.stdout:4/644: chown d24/d2d/c78 234983290 1 2024-04-02T04:43:05.233 INFO:tasks.workunit.client.1.smithi027.stdout:1/697: symlink d0/d1e/d2e/d3e/d5f/le3 0 2024-04-02T04:43:05.238 INFO:tasks.workunit.client.1.smithi027.stdout:6/648: rename d13/d1b/d24/d25/d56/f7e to d13/d1b/d24/d25/d56/dc8/dc9/fda 0 2024-04-02T04:43:05.241 INFO:tasks.workunit.client.1.smithi027.stdout:8/713: fsync d5/d8/db/f8e 0 2024-04-02T04:43:05.245 INFO:tasks.workunit.client.1.smithi027.stdout:0/783: unlink d4/d16/d1b/fb7 0 2024-04-02T04:43:05.245 INFO:tasks.workunit.client.1.smithi027.stdout:0/784: readlink d4/d38/d66/l83 0 2024-04-02T04:43:05.247 INFO:tasks.workunit.client.1.smithi027.stdout:9/860: mknod d1a/d1d/d29/d37/c11e 0 2024-04-02T04:43:05.248 INFO:tasks.workunit.client.1.smithi027.stdout:1/698: creat d0/de/d52/d74/fe4 x:0 0 0 2024-04-02T04:43:05.254 INFO:tasks.workunit.client.1.smithi027.stdout:4/645: rename d24/d48/f72 to d24/d38/d44/d82/fd2 0 2024-04-02T04:43:05.257 INFO:tasks.workunit.client.1.smithi027.stdout:9/861: dwrite d1a/d1d/d29/d51/d105/d8c/de1/fe9 [0,4194304] 0 2024-04-02T04:43:05.258 INFO:tasks.workunit.client.1.smithi027.stdout:1/699: dwrite d0/f6 [4194304,4194304] 0 2024-04-02T04:43:05.260 INFO:tasks.workunit.client.1.smithi027.stdout:1/700: stat d0/l89 0 2024-04-02T04:43:05.265 INFO:tasks.workunit.client.1.smithi027.stdout:4/646: dwrite d24/f76 [0,4194304] 0 2024-04-02T04:43:05.271 INFO:tasks.workunit.client.1.smithi027.stdout:4/647: chown d24/d38/d44/dbc/lc7 704 1 2024-04-02T04:43:05.273 INFO:tasks.workunit.client.1.smithi027.stdout:0/785: unlink d4/d38/d66/d2d/d4c/f5c 0 2024-04-02T04:43:05.277 INFO:tasks.workunit.client.1.smithi027.stdout:9/862: dwrite d1a/d1d/d29/d37/d3a/ff0 [0,4194304] 0 2024-04-02T04:43:05.284 INFO:tasks.workunit.client.1.smithi027.stdout:3/696: read d1/de/d55/f5f [2562211,17647] 0 2024-04-02T04:43:05.286 INFO:tasks.workunit.client.1.smithi027.stdout:1/701: rename d0/d1e/l20 to d0/d1e/d2e/d3b/d4f/dd3/le5 0 2024-04-02T04:43:05.292 INFO:tasks.workunit.client.1.smithi027.stdout:9/863: mknod d1a/d1d/d29/d37/d9b/d118/c11f 0 2024-04-02T04:43:05.292 INFO:tasks.workunit.client.1.smithi027.stdout:3/697: dwrite d1/de/df/d9e/fbe [0,4194304] 0 2024-04-02T04:43:05.295 INFO:tasks.workunit.client.1.smithi027.stdout:9/864: chown d1a/d1d/d29/d37/d3a/l98 4401 1 2024-04-02T04:43:05.295 INFO:tasks.workunit.client.1.smithi027.stdout:8/714: getdents d5/d8/db/dd/d5f 0 2024-04-02T04:43:05.301 INFO:tasks.workunit.client.1.smithi027.stdout:1/702: symlink d0/d1e/d2e/le6 0 2024-04-02T04:43:05.302 INFO:tasks.workunit.client.1.smithi027.stdout:7/712: truncate d2/d1d/d57/f5e 3720996 0 2024-04-02T04:43:05.304 INFO:tasks.workunit.client.1.smithi027.stdout:3/698: fdatasync d1/f2e 0 2024-04-02T04:43:05.310 INFO:tasks.workunit.client.1.smithi027.stdout:1/703: rename d0/de/d52/d74/fe4 to d0/de/d9e/fe7 0 2024-04-02T04:43:05.311 INFO:tasks.workunit.client.1.smithi027.stdout:1/704: chown d0/de/d52/d74/dbc/de2 520330865 1 2024-04-02T04:43:05.313 INFO:tasks.workunit.client.1.smithi027.stdout:7/713: mknod d2/dba/dc/d16/d3f/d79/ce3 0 2024-04-02T04:43:05.314 INFO:tasks.workunit.client.1.smithi027.stdout:7/714: fsync d2/dba/d35/f67 0 2024-04-02T04:43:05.316 INFO:tasks.workunit.client.1.smithi027.stdout:7/715: rename d2/dba to d2/dba/dc/d1f/d33/d36/de4 22 2024-04-02T04:43:05.324 INFO:tasks.workunit.client.1.smithi027.stdout:3/699: dread d1/de/df/f24 [0,4194304] 0 2024-04-02T04:43:05.324 INFO:tasks.workunit.client.1.smithi027.stdout:1/705: symlink d0/de/d52/d74/dd9/le8 0 2024-04-02T04:43:05.326 INFO:tasks.workunit.client.1.smithi027.stdout:3/700: stat d1/d1a/l62 0 2024-04-02T04:43:05.326 INFO:tasks.workunit.client.1.smithi027.stdout:4/648: dwrite d24/d2d/f31 [0,4194304] 0 2024-04-02T04:43:05.328 INFO:tasks.workunit.client.1.smithi027.stdout:4/649: dread - d24/fa8 zero size 2024-04-02T04:43:05.330 INFO:tasks.workunit.client.1.smithi027.stdout:7/716: mkdir d2/dba/dc/d16/d2f/de5 0 2024-04-02T04:43:05.332 INFO:tasks.workunit.client.1.smithi027.stdout:0/786: dwrite d4/d38/d66/d4a/f70 [0,4194304] 0 2024-04-02T04:43:05.342 INFO:tasks.workunit.client.1.smithi027.stdout:4/650: symlink d24/d2d/ld3 0 2024-04-02T04:43:05.344 INFO:tasks.workunit.client.1.smithi027.stdout:7/717: fsync d2/dba/dc/d1f/f9e 0 2024-04-02T04:43:05.348 INFO:tasks.workunit.client.1.smithi027.stdout:8/715: write d5/d8/db/dd/f10 [2278137,98323] 0 2024-04-02T04:43:05.367 INFO:tasks.workunit.client.1.smithi027.stdout:0/787: creat d4/d16/d1b/f10c x:0 0 0 2024-04-02T04:43:05.368 INFO:tasks.workunit.client.1.smithi027.stdout:8/716: symlink d5/d8/db/dd/d5f/d27/d31/lf0 0 2024-04-02T04:43:05.368 INFO:tasks.workunit.client.1.smithi027.stdout:0/788: chown d4/d38/d66/f24 10587 1 2024-04-02T04:43:05.368 INFO:tasks.workunit.client.1.smithi027.stdout:0/789: chown d4/d21/d85/fa6 61848 1 2024-04-02T04:43:05.371 INFO:tasks.workunit.client.1.smithi027.stdout:8/717: readlink d5/d8/l49 0 2024-04-02T04:43:05.371 INFO:tasks.workunit.client.1.smithi027.stdout:8/718: stat d5/f7a 0 2024-04-02T04:43:05.374 INFO:tasks.workunit.client.1.smithi027.stdout:8/719: unlink d5/d8/db/dd/d4a/dc9/f93 0 2024-04-02T04:43:05.390 INFO:tasks.workunit.client.1.smithi027.stdout:5/872: sync 2024-04-02T04:43:05.398 INFO:tasks.workunit.client.1.smithi027.stdout:3/701: write d1/f32 [3056389,122376] 0 2024-04-02T04:43:05.399 INFO:tasks.workunit.client.1.smithi027.stdout:0/790: mkdir d4/d21/d93/d10d 0 2024-04-02T04:43:05.399 INFO:tasks.workunit.client.1.smithi027.stdout:1/706: write d0/d1e/d2e/d3e/f82 [1489899,108977] 0 2024-04-02T04:43:05.400 INFO:tasks.workunit.client.1.smithi027.stdout:7/718: write d2/d1c/f7a [994325,74715] 0 2024-04-02T04:43:05.404 INFO:tasks.workunit.client.1.smithi027.stdout:3/702: chown d1/de/fa7 23 1 2024-04-02T04:43:05.405 INFO:tasks.workunit.client.1.smithi027.stdout:7/719: write d2/dba/f2a [2409623,123312] 0 2024-04-02T04:43:05.406 INFO:tasks.workunit.client.1.smithi027.stdout:5/873: stat d0/df/d11/d13/dce/d55/dc3/df2/ff9 0 2024-04-02T04:43:05.406 INFO:tasks.workunit.client.1.smithi027.stdout:0/791: fsync d4/d38/d66/f6c 0 2024-04-02T04:43:05.408 INFO:tasks.workunit.client.1.smithi027.stdout:3/703: chown d1/d1a/l62 1203259 1 2024-04-02T04:43:05.409 INFO:tasks.workunit.client.1.smithi027.stdout:5/874: write d0/f110 [467620,28709] 0 2024-04-02T04:43:05.412 INFO:tasks.workunit.client.1.smithi027.stdout:7/720: creat d2/dba/dc/d16/d81/fe6 x:0 0 0 2024-04-02T04:43:05.412 INFO:tasks.workunit.client.1.smithi027.stdout:4/651: dwrite d24/f32 [0,4194304] 0 2024-04-02T04:43:05.415 INFO:tasks.workunit.client.1.smithi027.stdout:5/875: symlink d0/d1d/d61/d7e/l128 0 2024-04-02T04:43:05.417 INFO:tasks.workunit.client.1.smithi027.stdout:7/721: rename d2/dba/dc/d16/d3f to d2/dba/dc/d16/d3f/de7 22 2024-04-02T04:43:05.418 INFO:tasks.workunit.client.1.smithi027.stdout:5/876: fsync d0/df/d2f/d3b/d105/f118 0 2024-04-02T04:43:05.419 INFO:tasks.workunit.client.1.smithi027.stdout:1/707: truncate d0/d1e/d2e/d3e/d5f/d8b/d9d/faf 987877 0 2024-04-02T04:43:05.420 INFO:tasks.workunit.client.1.smithi027.stdout:7/722: dread - d2/dba/d35/fd4 zero size 2024-04-02T04:43:05.421 INFO:tasks.workunit.client.1.smithi027.stdout:0/792: creat d4/d21/d93/d10d/f10e x:0 0 0 2024-04-02T04:43:05.422 INFO:tasks.workunit.client.1.smithi027.stdout:3/704: dwrite d1/d1a/fbd [0,4194304] 0 2024-04-02T04:43:05.423 INFO:tasks.workunit.client.1.smithi027.stdout:4/652: symlink d24/d38/d44/d82/ld4 0 2024-04-02T04:43:05.426 INFO:tasks.workunit.client.1.smithi027.stdout:5/877: creat d0/d68/dd7/f129 x:0 0 0 2024-04-02T04:43:05.427 INFO:tasks.workunit.client.1.smithi027.stdout:4/653: fsync d24/d38/d47/f93 0 2024-04-02T04:43:05.433 INFO:tasks.workunit.client.1.smithi027.stdout:1/708: dwrite d0/f54 [0,4194304] 0 2024-04-02T04:43:05.433 INFO:tasks.workunit.client.1.smithi027.stdout:7/723: symlink d2/dba/le8 0 2024-04-02T04:43:05.434 INFO:tasks.workunit.client.1.smithi027.stdout:8/720: dread d5/d8/db/dd/d5f/d27/d31/f5e [0,4194304] 0 2024-04-02T04:43:05.436 INFO:tasks.workunit.client.1.smithi027.stdout:1/709: stat d0/d1e/d2e/d3e/f42 0 2024-04-02T04:43:05.436 INFO:tasks.workunit.client.1.smithi027.stdout:7/724: chown d2/dba/dc/d16/d81 7767301 1 2024-04-02T04:43:05.446 INFO:tasks.workunit.client.1.smithi027.stdout:1/710: creat d0/d1e/d2e/d3e/d5f/fe9 x:0 0 0 2024-04-02T04:43:05.447 INFO:tasks.workunit.client.1.smithi027.stdout:1/711: stat d0/de/c53 0 2024-04-02T04:43:05.447 INFO:tasks.workunit.client.1.smithi027.stdout:8/721: creat d5/d8/db/dd/d5f/db2/dcf/ff1 x:0 0 0 2024-04-02T04:43:05.450 INFO:tasks.workunit.client.1.smithi027.stdout:1/712: truncate d0/d1e/d25/d3c/d64/da6/fbb 726951 0 2024-04-02T04:43:05.451 INFO:tasks.workunit.client.1.smithi027.stdout:7/725: rename d2/dba/dc/d16/d2f/l3c to d2/dba/dc/d16/d3f/d50/d8b/d91/le9 0 2024-04-02T04:43:05.453 INFO:tasks.workunit.client.1.smithi027.stdout:3/705: write d1/de/df/d13/f53 [928963,24489] 0 2024-04-02T04:43:05.454 INFO:tasks.workunit.client.1.smithi027.stdout:0/793: dread d4/d16/d1b/fc2 [0,4194304] 0 2024-04-02T04:43:05.455 INFO:tasks.workunit.client.1.smithi027.stdout:8/722: dread - d5/f7a zero size 2024-04-02T04:43:05.456 INFO:tasks.workunit.client.1.smithi027.stdout:4/654: dread d24/f3a [0,4194304] 0 2024-04-02T04:43:05.456 INFO:tasks.workunit.client.1.smithi027.stdout:0/794: write d4/f73 [1275137,21242] 0 2024-04-02T04:43:05.459 INFO:tasks.workunit.client.1.smithi027.stdout:7/726: creat d2/dba/dc/d16/d3f/d50/fea x:0 0 0 2024-04-02T04:43:05.462 INFO:tasks.workunit.client.1.smithi027.stdout:7/727: truncate d2/dba/dc/fc3 427391 0 2024-04-02T04:43:05.463 INFO:tasks.workunit.client.1.smithi027.stdout:1/713: unlink d0/de/d52/d74/dbc/de2/fba 0 2024-04-02T04:43:05.464 INFO:tasks.workunit.client.1.smithi027.stdout:5/878: dwrite d0/d1d/d61/d7e/f9b [0,4194304] 0 2024-04-02T04:43:05.465 INFO:tasks.workunit.client.1.smithi027.stdout:1/714: write d0/de/d52/d74/fb5 [642285,51300] 0 2024-04-02T04:43:05.469 INFO:tasks.workunit.client.1.smithi027.stdout:4/655: creat d24/d38/d44/d82/fd5 x:0 0 0 2024-04-02T04:43:05.469 INFO:tasks.workunit.client.1.smithi027.stdout:3/706: truncate d1/de/df/d13/f34 3017982 0 2024-04-02T04:43:05.473 INFO:tasks.workunit.client.1.smithi027.stdout:0/795: rename d4/d16/c106 to d4/d38/d66/d4a/c10f 0 2024-04-02T04:43:05.474 INFO:tasks.workunit.client.1.smithi027.stdout:7/728: creat d2/d1c/feb x:0 0 0 2024-04-02T04:43:05.475 INFO:tasks.workunit.client.1.smithi027.stdout:1/715: dwrite d0/d1e/d2e/d3e/f82 [0,4194304] 0 2024-04-02T04:43:05.480 INFO:tasks.workunit.client.1.smithi027.stdout:8/723: mknod d5/d8/db/dd/d5f/d27/cf2 0 2024-04-02T04:43:05.487 INFO:tasks.workunit.client.1.smithi027.stdout:8/724: dwrite d5/d8/db/dd/d5f/d27/d31/d5b/dc4/f5d [0,4194304] 0 2024-04-02T04:43:05.496 INFO:tasks.workunit.client.1.smithi027.stdout:3/707: mkdir d1/de/d55/de7 0 2024-04-02T04:43:05.497 INFO:tasks.workunit.client.1.smithi027.stdout:0/796: symlink d4/d16/d71/df5/l110 0 2024-04-02T04:43:05.498 INFO:tasks.workunit.client.1.smithi027.stdout:3/708: write d1/de/df/d13/f36 [4044701,108768] 0 2024-04-02T04:43:05.499 INFO:tasks.workunit.client.1.smithi027.stdout:5/879: write d0/df/d11/d13/dce/fbc [5464388,79747] 0 2024-04-02T04:43:05.500 INFO:tasks.workunit.client.1.smithi027.stdout:8/725: fdatasync d5/d8/db/d68/fad 0 2024-04-02T04:43:05.503 INFO:tasks.workunit.client.1.smithi027.stdout:7/729: truncate d2/dba/d35/f67 2264212 0 2024-04-02T04:43:05.505 INFO:tasks.workunit.client.1.smithi027.stdout:1/716: link d0/d1e/d25/f50 d0/de/d52/d74/d97/fea 0 2024-04-02T04:43:05.507 INFO:tasks.workunit.client.1.smithi027.stdout:0/797: mknod d4/d21/d93/c111 0 2024-04-02T04:43:05.510 INFO:tasks.workunit.client.1.smithi027.stdout:3/709: mknod d1/d1a/ce8 0 2024-04-02T04:43:05.513 INFO:tasks.workunit.client.1.smithi027.stdout:4/656: dwrite d24/d38/d44/d41/d49/f77 [0,4194304] 0 2024-04-02T04:43:05.513 INFO:tasks.workunit.client.1.smithi027.stdout:1/717: dwrite d0/d1e/f98 [0,4194304] 0 2024-04-02T04:43:05.515 INFO:tasks.workunit.client.1.smithi027.stdout:4/657: readlink d24/d38/d44/dbc/lc1 0 2024-04-02T04:43:05.516 INFO:tasks.workunit.client.1.smithi027.stdout:8/726: mknod d5/db5/cf3 0 2024-04-02T04:43:05.520 INFO:tasks.workunit.client.1.smithi027.stdout:8/727: readlink d5/d8/db/dd/d5f/d27/d31/d5b/lbd 0 2024-04-02T04:43:05.524 INFO:tasks.workunit.client.1.smithi027.stdout:2/839: sync 2024-04-02T04:43:05.524 INFO:tasks.workunit.client.1.smithi027.stdout:7/730: creat d2/dba/dc/d16/d3f/d79/fec x:0 0 0 2024-04-02T04:43:05.525 INFO:tasks.workunit.client.1.smithi027.stdout:4/658: dwrite d24/fbb [0,4194304] 0 2024-04-02T04:43:05.528 INFO:tasks.workunit.client.1.smithi027.stdout:0/798: mknod d4/d21/d85/c112 0 2024-04-02T04:43:05.529 INFO:tasks.workunit.client.1.smithi027.stdout:2/840: truncate d1/db/d3b/f51 426944 0 2024-04-02T04:43:05.534 INFO:tasks.workunit.client.1.smithi027.stdout:5/880: symlink d0/d68/d6b/d10b/l12a 0 2024-04-02T04:43:05.537 INFO:tasks.workunit.client.1.smithi027.stdout:8/728: creat d5/db5/ddc/ff4 x:0 0 0 2024-04-02T04:43:05.539 INFO:tasks.workunit.client.1.smithi027.stdout:1/718: write d0/d1e/d2e/d3b/f77 [9315966,59646] 0 2024-04-02T04:43:05.539 INFO:tasks.workunit.client.1.smithi027.stdout:4/659: dwrite d24/d48/db0/fc3 [0,4194304] 0 2024-04-02T04:43:05.541 INFO:tasks.workunit.client.1.smithi027.stdout:7/731: creat d2/dba/dc/d16/d3f/dc0/fed x:0 0 0 2024-04-02T04:43:05.542 INFO:tasks.workunit.client.1.smithi027.stdout:5/881: chown d0/d1d/d61/d72/l11a 49 1 2024-04-02T04:43:05.542 INFO:tasks.workunit.client.1.smithi027.stdout:7/732: stat d2/dba/dc 0 2024-04-02T04:43:05.548 INFO:tasks.workunit.client.1.smithi027.stdout:4/660: fdatasync d24/d38/d44/d41/d49/f4c 0 2024-04-02T04:43:05.548 INFO:tasks.workunit.client.1.smithi027.stdout:0/799: dwrite d4/d21/d28/d4e/db5/f104 [0,4194304] 0 2024-04-02T04:43:05.550 INFO:tasks.workunit.client.1.smithi027.stdout:7/733: fdatasync d2/dba/dc/d16/d81/fe6 0 2024-04-02T04:43:05.552 INFO:tasks.workunit.client.1.smithi027.stdout:5/882: dread - d0/df/d2f/d3b/d51/f8e zero size 2024-04-02T04:43:05.553 INFO:tasks.workunit.client.1.smithi027.stdout:5/883: fsync d0/f6 0 2024-04-02T04:43:05.561 INFO:tasks.workunit.client.1.smithi027.stdout:7/734: creat d2/d1d/d90/fee x:0 0 0 2024-04-02T04:43:05.563 INFO:tasks.workunit.client.1.smithi027.stdout:3/710: dread d1/f7 [0,4194304] 0 2024-04-02T04:43:05.566 INFO:tasks.workunit.client.1.smithi027.stdout:2/841: dread d1/db/d3b/d17/d6d/dd9/dd5/dda/dbb/fe4 [0,4194304] 0 2024-04-02T04:43:05.571 INFO:tasks.workunit.client.1.smithi027.stdout:4/661: unlink d24/d48/c9d 0 2024-04-02T04:43:05.573 INFO:tasks.workunit.client.1.smithi027.stdout:6/649: sync 2024-04-02T04:43:05.576 INFO:tasks.workunit.client.1.smithi027.stdout:7/735: dread d2/dba/dc/d16/f2c [0,4194304] 0 2024-04-02T04:43:05.579 INFO:tasks.workunit.client.1.smithi027.stdout:4/662: dwrite d24/d38/d44/d60/fb8 [0,4194304] 0 2024-04-02T04:43:05.580 INFO:tasks.workunit.client.1.smithi027.stdout:7/736: read - d2/dba/dc/d16/d3f/d50/fea zero size 2024-04-02T04:43:05.582 INFO:tasks.workunit.client.1.smithi027.stdout:6/650: dread d13/d1b/d24/d25/d56/d73/d96/fb2 [0,4194304] 0 2024-04-02T04:43:05.582 INFO:tasks.workunit.client.1.smithi027.stdout:5/884: truncate d0/d1a/d20/fb4 3106427 0 2024-04-02T04:43:05.585 INFO:tasks.workunit.client.1.smithi027.stdout:5/885: write d0/d1a/d20/fde [533378,62123] 0 2024-04-02T04:43:05.586 INFO:tasks.workunit.client.1.smithi027.stdout:6/651: write d13/d1b/d24/d25/d56/dc8/dc9/fcd [2038904,8375] 0 2024-04-02T04:43:05.590 INFO:tasks.workunit.client.1.smithi027.stdout:8/729: dwrite d5/d8/db/d68/d95/f94 [0,4194304] 0 2024-04-02T04:43:05.590 INFO:tasks.workunit.client.1.smithi027.stdout:6/652: write d13/d1b/d24/d25/d27/f97 [3680084,120790] 0 2024-04-02T04:43:05.594 INFO:tasks.workunit.client.1.smithi027.stdout:2/842: symlink d1/db/d3b/d61/l12e 0 2024-04-02T04:43:05.594 INFO:tasks.workunit.client.1.smithi027.stdout:8/730: write d5/d8/f2e [187592,101493] 0 2024-04-02T04:43:05.597 INFO:tasks.workunit.client.1.smithi027.stdout:1/719: link d0/d1e/d2e/d3e/l85 d0/d1e/d2e/d3b/leb 0 2024-04-02T04:43:05.600 INFO:tasks.workunit.client.1.smithi027.stdout:5/886: dwrite d0/df/d11/d13/d9a/d9c/dbe/feb [0,4194304] 0 2024-04-02T04:43:05.601 INFO:tasks.workunit.client.1.smithi027.stdout:5/887: chown d0/df/d11/d13/dce/lad 1990 1 2024-04-02T04:43:05.606 INFO:tasks.workunit.client.1.smithi027.stdout:7/737: dread d2/dba/dc/d16/d3f/d50/d8b/f72 [0,4194304] 0 2024-04-02T04:43:05.606 INFO:tasks.workunit.client.1.smithi027.stdout:9/865: sync 2024-04-02T04:43:05.608 INFO:tasks.workunit.client.1.smithi027.stdout:9/866: fsync d1a/d1d/d29/d37/d3a/fc8 0 2024-04-02T04:43:05.611 INFO:tasks.workunit.client.1.smithi027.stdout:0/800: rmdir d4/d21/d6f 0 2024-04-02T04:43:05.616 INFO:tasks.workunit.client.1.smithi027.stdout:7/738: dwrite d2/dba/dc/d16/d3f/d50/f69 [0,4194304] 0 2024-04-02T04:43:05.621 INFO:tasks.workunit.client.1.smithi027.stdout:3/711: link d1/d1a/da6/cd2 d1/de/df/d61/ce9 0 2024-04-02T04:43:05.622 INFO:tasks.workunit.client.1.smithi027.stdout:6/653: mkdir d13/d1b/d24/d25/d56/dc8/ddb 0 2024-04-02T04:43:05.625 INFO:tasks.workunit.client.1.smithi027.stdout:2/843: rmdir d1/db/d3b 39 2024-04-02T04:43:05.630 INFO:tasks.workunit.client.1.smithi027.stdout:8/731: symlink d5/d8/db/d57/lf5 0 2024-04-02T04:43:05.631 INFO:tasks.workunit.client.1.smithi027.stdout:8/732: chown d5/d8/db/dd/d5f/d27/d31/d5b/dc4/c6d 39 1 2024-04-02T04:43:05.632 INFO:tasks.workunit.client.1.smithi027.stdout:5/888: fsync d0/df/d11/d41/f90 0 2024-04-02T04:43:05.633 INFO:tasks.workunit.client.1.smithi027.stdout:8/733: write d5/d8/db/dd5/fde [128215,10617] 0 2024-04-02T04:43:05.637 INFO:tasks.workunit.client.1.smithi027.stdout:9/867: symlink d1a/d1d/d29/d50/d117/l120 0 2024-04-02T04:43:05.648 INFO:tasks.workunit.client.1.smithi027.stdout:7/739: symlink d2/dba/dc/d16/d2f/dcd/lef 0 2024-04-02T04:43:05.650 INFO:tasks.workunit.client.1.smithi027.stdout:4/663: dwrite d24/f27 [0,4194304] 0 2024-04-02T04:43:05.652 INFO:tasks.workunit.client.1.smithi027.stdout:6/654: creat d13/d1b/d24/d28/d3c/fdc x:0 0 0 2024-04-02T04:43:05.662 INFO:tasks.workunit.client.1.smithi027.stdout:9/868: mknod d1a/d1d/d29/d37/d107/c121 0 2024-04-02T04:43:05.663 INFO:tasks.workunit.client.1.smithi027.stdout:0/801: dread d4/fb [0,4194304] 0 2024-04-02T04:43:05.663 INFO:tasks.workunit.client.1.smithi027.stdout:9/869: chown d1a/d1d/d3c/d6e/db8 252133 1 2024-04-02T04:43:05.674 INFO:tasks.workunit.client.1.smithi027.stdout:1/720: dwrite d0/f30 [0,4194304] 0 2024-04-02T04:43:05.713 INFO:tasks.workunit.client.1.smithi027.stdout:4/664: unlink d24/d38/d44/d41/c4f 0 2024-04-02T04:43:05.714 INFO:tasks.workunit.client.1.smithi027.stdout:4/665: write d24/d38/d47/fa3 [1227373,100851] 0 2024-04-02T04:43:05.721 INFO:tasks.workunit.client.1.smithi027.stdout:4/666: dwrite d24/f33 [0,4194304] 0 2024-04-02T04:43:05.725 INFO:tasks.workunit.client.1.smithi027.stdout:9/870: creat d1a/d1d/d29/d51/d105/d8c/f122 x:0 0 0 2024-04-02T04:43:05.726 INFO:tasks.workunit.client.1.smithi027.stdout:9/871: fdatasync d1a/d1d/d29/d50/da2/fbb 0 2024-04-02T04:43:05.727 INFO:tasks.workunit.client.1.smithi027.stdout:3/712: link d1/de/df/l6a d1/de/d55/d63/da4/de3/lea 0 2024-04-02T04:43:05.727 INFO:tasks.workunit.client.1.smithi027.stdout:1/721: read d0/fb [1639917,128118] 0 2024-04-02T04:43:05.734 INFO:tasks.workunit.client.1.smithi027.stdout:2/844: symlink d1/db/d3b/d61/d75/d81/l12f 0 2024-04-02T04:43:05.739 INFO:tasks.workunit.client.1.smithi027.stdout:4/667: fsync d24/d48/f81 0 2024-04-02T04:43:05.745 INFO:tasks.workunit.client.1.smithi027.stdout:9/872: read - d1a/d1d/d3c/fbc zero size 2024-04-02T04:43:05.749 INFO:tasks.workunit.client.1.smithi027.stdout:2/845: dread d1/f13 [0,4194304] 0 2024-04-02T04:43:05.749 INFO:tasks.workunit.client.1.smithi027.stdout:6/655: creat d13/d1b/d24/d25/d56/fdd x:0 0 0 2024-04-02T04:43:05.752 INFO:tasks.workunit.client.1.smithi027.stdout:8/734: rename d5/cdb to d5/d8/db/dd/cf6 0 2024-04-02T04:43:05.754 INFO:tasks.workunit.client.1.smithi027.stdout:3/713: dwrite d1/de/df/f24 [4194304,4194304] 0 2024-04-02T04:43:05.755 INFO:tasks.workunit.client.1.smithi027.stdout:3/714: write d1/de/df/d13/d73/d77/fdf [471393,77798] 0 2024-04-02T04:43:05.757 INFO:tasks.workunit.client.1.smithi027.stdout:4/668: symlink d24/d48/db0/ld6 0 2024-04-02T04:43:05.758 INFO:tasks.workunit.client.1.smithi027.stdout:9/873: creat d1a/d1d/d29/d50/da2/dfb/f123 x:0 0 0 2024-04-02T04:43:05.760 INFO:tasks.workunit.client.1.smithi027.stdout:4/669: write d24/d38/d47/f93 [1361152,65805] 0 2024-04-02T04:43:05.763 INFO:tasks.workunit.client.1.smithi027.stdout:6/656: unlink d13/d1b/d24/d25/d42/l74 0 2024-04-02T04:43:05.767 INFO:tasks.workunit.client.1.smithi027.stdout:3/715: dwrite d1/de/d55/fa1 [0,4194304] 0 2024-04-02T04:43:05.767 INFO:tasks.workunit.client.1.smithi027.stdout:8/735: dread d5/d8/db/fa3 [0,4194304] 0 2024-04-02T04:43:05.769 INFO:tasks.workunit.client.1.smithi027.stdout:2/846: symlink d1/l130 0 2024-04-02T04:43:05.771 INFO:tasks.workunit.client.1.smithi027.stdout:7/740: write d2/d1c/f37 [25163,57687] 0 2024-04-02T04:43:05.776 INFO:tasks.workunit.client.1.smithi027.stdout:9/874: dwrite d1a/d1d/d29/d51/d105/f8f [0,4194304] 0 2024-04-02T04:43:05.777 INFO:tasks.workunit.client.1.smithi027.stdout:6/657: unlink d13/d1b/d24/ca8 0 2024-04-02T04:43:05.778 INFO:tasks.workunit.client.1.smithi027.stdout:7/741: chown d2/dba/dc/d1f/d33/c38 8461 1 2024-04-02T04:43:05.778 INFO:tasks.workunit.client.1.smithi027.stdout:5/889: write d0/df/d11/d13/dce/f63 [57847,127855] 0 2024-04-02T04:43:05.778 INFO:tasks.workunit.client.1.smithi027.stdout:9/875: stat d1a/d1d/d29/l9e 0 2024-04-02T04:43:05.785 INFO:tasks.workunit.client.1.smithi027.stdout:8/736: dwrite d5/db5/ddc/fed [0,4194304] 0 2024-04-02T04:43:05.787 INFO:tasks.workunit.client.1.smithi027.stdout:2/847: unlink d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/dc0/fc9 0 2024-04-02T04:43:05.791 INFO:tasks.workunit.client.1.smithi027.stdout:7/742: mknod d2/dba/dc/d16/d3f/d44/cf0 0 2024-04-02T04:43:05.791 INFO:tasks.workunit.client.1.smithi027.stdout:0/802: truncate d4/d5d/fd2 4187761 0 2024-04-02T04:43:05.793 INFO:tasks.workunit.client.1.smithi027.stdout:1/722: write d0/f37 [1448775,48658] 0 2024-04-02T04:43:05.794 INFO:tasks.workunit.client.1.smithi027.stdout:9/876: truncate d1a/d1d/d29/d56/de4/f10f 474456 0 2024-04-02T04:43:05.797 INFO:tasks.workunit.client.1.smithi027.stdout:9/877: dread d1a/d1d/d29/d56/f88 [4194304,4194304] 0 2024-04-02T04:43:05.804 INFO:tasks.workunit.client.1.smithi027.stdout:6/658: dread d13/f16 [0,4194304] 0 2024-04-02T04:43:05.805 INFO:tasks.workunit.client.1.smithi027.stdout:1/723: mkdir d0/d1e/d25/d3c/d64/dec 0 2024-04-02T04:43:05.811 INFO:tasks.workunit.client.1.smithi027.stdout:3/716: truncate d1/f45 3456904 0 2024-04-02T04:43:05.812 INFO:tasks.workunit.client.1.smithi027.stdout:3/717: stat d1/l31 0 2024-04-02T04:43:05.814 INFO:tasks.workunit.client.1.smithi027.stdout:4/670: dwrite d24/d38/f5e [0,4194304] 0 2024-04-02T04:43:05.815 INFO:tasks.workunit.client.1.smithi027.stdout:4/671: stat d24/d2d/f92 0 2024-04-02T04:43:05.816 INFO:tasks.workunit.client.1.smithi027.stdout:6/659: symlink d13/d1b/d24/d25/d56/d73/d96/lde 0 2024-04-02T04:43:05.819 INFO:tasks.workunit.client.1.smithi027.stdout:8/737: write d5/d8/db/fa3 [1113519,128359] 0 2024-04-02T04:43:05.821 INFO:tasks.workunit.client.1.smithi027.stdout:2/848: write d1/db/d3b/d17/d2b/f53 [37007,66755] 0 2024-04-02T04:43:05.822 INFO:tasks.workunit.client.1.smithi027.stdout:8/738: write d5/d8/f2e [3430957,74237] 0 2024-04-02T04:43:05.831 INFO:tasks.workunit.client.1.smithi027.stdout:3/718: unlink d1/de/df/l39 0 2024-04-02T04:43:05.831 INFO:tasks.workunit.client.1.smithi027.stdout:7/743: write d2/dba/d35/f67 [340876,88492] 0 2024-04-02T04:43:05.834 INFO:tasks.workunit.client.1.smithi027.stdout:7/744: dread - d2/dba/dc/d16/d3f/d50/d8b/db6/fd9 zero size 2024-04-02T04:43:05.835 INFO:tasks.workunit.client.1.smithi027.stdout:9/878: write d1a/d1d/d29/d37/d4c/f66 [4839210,93929] 0 2024-04-02T04:43:05.837 INFO:tasks.workunit.client.1.smithi027.stdout:4/672: creat d24/d38/d44/d82/d97/fd7 x:0 0 0 2024-04-02T04:43:05.838 INFO:tasks.workunit.client.1.smithi027.stdout:6/660: symlink d13/d1b/d24/d25/d56/dc8/ldf 0 2024-04-02T04:43:05.841 INFO:tasks.workunit.client.1.smithi027.stdout:3/719: dwrite d1/f2a [0,4194304] 0 2024-04-02T04:43:05.856 INFO:tasks.workunit.client.1.smithi027.stdout:0/803: creat d4/d21/d28/f113 x:0 0 0 2024-04-02T04:43:05.860 INFO:tasks.workunit.client.1.smithi027.stdout:1/724: truncate d0/d1e/d2e/f6d 5360499 0 2024-04-02T04:43:05.861 INFO:tasks.workunit.client.1.smithi027.stdout:7/745: mknod d2/d1c/cf1 0 2024-04-02T04:43:05.863 INFO:tasks.workunit.client.1.smithi027.stdout:8/739: dread d5/d8/db/dd/f1d [0,4194304] 0 2024-04-02T04:43:05.866 INFO:tasks.workunit.client.1.smithi027.stdout:7/746: dread d2/dba/dc/d16/d3f/d50/f66 [0,4194304] 0 2024-04-02T04:43:05.867 INFO:tasks.workunit.client.1.smithi027.stdout:4/673: mknod d24/d38/d47/d5c/cd8 0 2024-04-02T04:43:05.869 INFO:tasks.workunit.client.1.smithi027.stdout:6/661: rmdir d13/d1b/d24/d25/d56/d6d/daa 39 2024-04-02T04:43:05.873 INFO:tasks.workunit.client.1.smithi027.stdout:2/849: dwrite d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/dc0/dc3/fe8 [0,4194304] 0 2024-04-02T04:43:05.873 INFO:tasks.workunit.client.1.smithi027.stdout:0/804: unlink d4/d16/f89 0 2024-04-02T04:43:05.876 INFO:tasks.workunit.client.1.smithi027.stdout:1/725: read d0/d1e/d2e/d3b/dda/fab [1504654,45497] 0 2024-04-02T04:43:05.878 INFO:tasks.workunit.client.1.smithi027.stdout:9/879: mkdir d1a/dce/d124 0 2024-04-02T04:43:05.880 INFO:tasks.workunit.client.1.smithi027.stdout:7/747: symlink d2/dba/dc/d1f/d33/d36/lf2 0 2024-04-02T04:43:05.884 INFO:tasks.workunit.client.1.smithi027.stdout:6/662: truncate d13/d1b/d24/f3a 2817411 0 2024-04-02T04:43:05.887 INFO:tasks.workunit.client.1.smithi027.stdout:2/850: creat d1/db/d3b/d83/f131 x:0 0 0 2024-04-02T04:43:05.888 INFO:tasks.workunit.client.1.smithi027.stdout:2/851: readlink d1/db/d3b/d17/d6d/l124 0 2024-04-02T04:43:05.893 INFO:tasks.workunit.client.1.smithi027.stdout:9/880: write d1a/d1d/d29/d37/d9b/fdc [673158,22538] 0 2024-04-02T04:43:05.898 INFO:tasks.workunit.client.1.smithi027.stdout:0/805: truncate d4/d76/dc4/dfe/dd1/f100 528351 0 2024-04-02T04:43:05.900 INFO:tasks.workunit.client.1.smithi027.stdout:0/806: chown d4/d16 2027730 1 2024-04-02T04:43:05.903 INFO:tasks.workunit.client.1.smithi027.stdout:8/740: rename d5/d8/db/lc3 to d5/d8/db/dd/d5f/d27/d31/d5b/lf7 0 2024-04-02T04:43:05.907 INFO:tasks.workunit.client.1.smithi027.stdout:9/881: creat d1a/f125 x:0 0 0 2024-04-02T04:43:05.907 INFO:tasks.workunit.client.1.smithi027.stdout:7/748: mkdir d2/d7/df3 0 2024-04-02T04:43:05.910 INFO:tasks.workunit.client.1.smithi027.stdout:6/663: symlink d13/d1b/d24/d28/d46/dcc/db7/le0 0 2024-04-02T04:43:05.910 INFO:tasks.workunit.client.1.smithi027.stdout:1/726: link d0/l8 d0/de/d52/d74/led 0 2024-04-02T04:43:05.913 INFO:tasks.workunit.client.1.smithi027.stdout:4/674: rename d24/f2b to d24/d38/d44/d41/d5a/fd9 0 2024-04-02T04:43:05.915 INFO:tasks.workunit.client.1.smithi027.stdout:7/749: dwrite d2/dba/dc/d16/fbd [0,4194304] 0 2024-04-02T04:43:05.916 INFO:tasks.workunit.client.1.smithi027.stdout:9/882: dread - d1a/d1d/d29/d56/d8b/fb4 zero size 2024-04-02T04:43:05.920 INFO:tasks.workunit.client.1.smithi027.stdout:2/852: creat d1/db/d3b/d17/f132 x:0 0 0 2024-04-02T04:43:05.920 INFO:tasks.workunit.client.1.smithi027.stdout:6/664: fsync d13/d1b/d48/d75/f8b 0 2024-04-02T04:43:05.924 INFO:tasks.workunit.client.1.smithi027.stdout:6/665: chown d13/d1b/d48/d75/f9f 7 1 2024-04-02T04:43:05.925 INFO:tasks.workunit.client.1.smithi027.stdout:7/750: dwrite d2/dba/da6/fcc [0,4194304] 0 2024-04-02T04:43:05.926 INFO:tasks.workunit.client.1.smithi027.stdout:1/727: chown d0/d1e/d2e/d3b/d4f/ce1 5963662 1 2024-04-02T04:43:05.930 INFO:tasks.workunit.client.1.smithi027.stdout:4/675: readlink d24/l40 0 2024-04-02T04:43:05.932 INFO:tasks.workunit.client.1.smithi027.stdout:0/807: rename d4/d38/d66/d4a/fdd to d4/d21/d28/d4e/d8c/db0/f114 0 2024-04-02T04:43:05.934 INFO:tasks.workunit.client.1.smithi027.stdout:0/808: write d4/d76/fa8 [4511272,70029] 0 2024-04-02T04:43:05.945 INFO:tasks.workunit.client.1.smithi027.stdout:7/751: mkdir d2/dba/dc/d16/d3a/d4d/df4 0 2024-04-02T04:43:05.947 INFO:tasks.workunit.client.1.smithi027.stdout:8/741: dwrite d5/d8/db/dd/d5f/d27/d31/d5b/dc4/fe4 [0,4194304] 0 2024-04-02T04:43:05.948 INFO:tasks.workunit.client.1.smithi027.stdout:4/676: symlink d24/d38/d44/d82/d97/lda 0 2024-04-02T04:43:05.950 INFO:tasks.workunit.client.1.smithi027.stdout:4/677: chown d24/d38/d44/f6e 84032124 1 2024-04-02T04:43:05.950 INFO:tasks.workunit.client.1.smithi027.stdout:8/742: write d5/d8/db/dd/d5f/d27/d31/fd7 [131231,107487] 0 2024-04-02T04:43:05.952 INFO:tasks.workunit.client.1.smithi027.stdout:0/809: creat d4/d69/f115 x:0 0 0 2024-04-02T04:43:05.953 INFO:tasks.workunit.client.1.smithi027.stdout:8/743: stat d5/d8/db/dd/fd2 0 2024-04-02T04:43:05.953 INFO:tasks.workunit.client.1.smithi027.stdout:9/883: write d1a/d1d/d29/d56/d8b/f90 [650864,66351] 0 2024-04-02T04:43:05.958 INFO:tasks.workunit.client.1.smithi027.stdout:8/744: write d5/d8/db/dd/fd2 [685057,61432] 0 2024-04-02T04:43:05.962 INFO:tasks.workunit.client.1.smithi027.stdout:4/678: creat d24/d38/d44/d41/d5a/fdb x:0 0 0 2024-04-02T04:43:05.963 INFO:tasks.workunit.client.1.smithi027.stdout:2/853: dread d1/db/d3b/d17/d68/d69/f8d [0,4194304] 0 2024-04-02T04:43:05.964 INFO:tasks.workunit.client.1.smithi027.stdout:6/666: creat d13/d1b/d48/fe1 x:0 0 0 2024-04-02T04:43:05.964 INFO:tasks.workunit.client.1.smithi027.stdout:0/810: dread d4/d38/ff7 [0,4194304] 0 2024-04-02T04:43:05.965 INFO:tasks.workunit.client.1.smithi027.stdout:4/679: fdatasync d24/d48/f4e 0 2024-04-02T04:43:05.967 INFO:tasks.workunit.client.1.smithi027.stdout:8/745: mkdir d5/d8/db/dd/d5f/df8 0 2024-04-02T04:43:05.967 INFO:tasks.workunit.client.1.smithi027.stdout:9/884: dwrite d1a/d1d/db7/fea [0,4194304] 0 2024-04-02T04:43:05.972 INFO:tasks.workunit.client.1.smithi027.stdout:9/885: dread - d1a/d1d/d29/d50/da2/dfb/f123 zero size 2024-04-02T04:43:05.972 INFO:tasks.workunit.client.1.smithi027.stdout:2/854: truncate d1/db/d3b/d17/d6d/dd9/dd5/dda/f112 287400 0 2024-04-02T04:43:05.973 INFO:tasks.workunit.client.1.smithi027.stdout:1/728: truncate d0/de/d52/d74/db7/fc6 550917 0 2024-04-02T04:43:05.976 INFO:tasks.workunit.client.1.smithi027.stdout:0/811: dwrite d4/d38/fa9 [8388608,4194304] 0 2024-04-02T04:43:05.979 INFO:tasks.workunit.client.1.smithi027.stdout:7/752: write d2/dba/dc/d1f/d61/f77 [3418599,110532] 0 2024-04-02T04:43:05.985 INFO:tasks.workunit.client.1.smithi027.stdout:6/667: mkdir d13/d1b/d48/d75/d9b/de2 0 2024-04-02T04:43:05.988 INFO:tasks.workunit.client.1.smithi027.stdout:2/855: creat d1/db/d3b/d61/d75/d81/f133 x:0 0 0 2024-04-02T04:43:05.990 INFO:tasks.workunit.client.1.smithi027.stdout:2/856: read - d1/db/d3b/d17/f132 zero size 2024-04-02T04:43:05.990 INFO:tasks.workunit.client.1.smithi027.stdout:9/886: mkdir d1a/d1d/d29/d37/d20/dcc/dd5/de2/d126 0 2024-04-02T04:43:05.994 INFO:tasks.workunit.client.1.smithi027.stdout:2/857: truncate d1/db/d3b/d17/d2b/db8/fd0 242514 0 2024-04-02T04:43:05.995 INFO:tasks.workunit.client.1.smithi027.stdout:2/858: fsync d1/db/d3b/d17/d2b/db8/fd0 0 2024-04-02T04:43:05.996 INFO:tasks.workunit.client.1.smithi027.stdout:1/729: read d0/d1e/d25/d3c/f48 [2916411,118084] 0 2024-04-02T04:43:06.002 INFO:tasks.workunit.client.1.smithi027.stdout:6/668: dread d13/d1b/d24/d25/d42/fa9 [0,4194304] 0 2024-04-02T04:43:06.006 INFO:tasks.workunit.client.1.smithi027.stdout:4/680: mkdir d24/d38/d47/d62/ddc 0 2024-04-02T04:43:06.007 INFO:tasks.workunit.client.1.smithi027.stdout:4/681: write d24/d38/d44/d60/fc4 [1468553,58811] 0 2024-04-02T04:43:06.008 INFO:tasks.workunit.client.1.smithi027.stdout:4/682: fsync d24/f27 0 2024-04-02T04:43:06.009 INFO:tasks.workunit.client.1.smithi027.stdout:1/730: fsync d0/de/f80 0 2024-04-02T04:43:06.010 INFO:tasks.workunit.client.1.smithi027.stdout:4/683: read - d24/d38/d44/d41/d5a/fdb zero size 2024-04-02T04:43:06.014 INFO:tasks.workunit.client.1.smithi027.stdout:8/746: truncate d5/d8/db/dd/d5f/d27/d31/f84 704708 0 2024-04-02T04:43:06.016 INFO:tasks.workunit.client.1.smithi027.stdout:0/812: truncate d4/d76/f99 659212 0 2024-04-02T04:43:06.016 INFO:tasks.workunit.client.1.smithi027.stdout:0/813: chown d4/d38/ff7 45 1 2024-04-02T04:43:06.017 INFO:tasks.workunit.client.1.smithi027.stdout:7/753: write d2/df/fdb [1678242,50363] 0 2024-04-02T04:43:06.020 INFO:tasks.workunit.client.1.smithi027.stdout:9/887: truncate d1a/d1d/d3c/d6e/fc6 3382608 0 2024-04-02T04:43:06.029 INFO:tasks.workunit.client.1.smithi027.stdout:8/747: symlink d5/d8/db/dd5/lf9 0 2024-04-02T04:43:06.034 INFO:tasks.workunit.client.1.smithi027.stdout:0/814: mknod d4/d76/dc4/dfe/c116 0 2024-04-02T04:43:06.042 INFO:tasks.workunit.client.1.smithi027.stdout:6/669: symlink d13/d1b/d48/d58/dd3/le3 0 2024-04-02T04:43:06.042 INFO:tasks.workunit.client.1.smithi027.stdout:4/684: dread d24/d48/f7d [0,4194304] 0 2024-04-02T04:43:06.042 INFO:tasks.workunit.client.1.smithi027.stdout:8/748: rename d5/d8/db/dd/d5f/d27/d31/d5b/fb1 to d5/d8/db/d68/ffa 0 2024-04-02T04:43:06.043 INFO:tasks.workunit.client.1.smithi027.stdout:8/749: dread - d5/d8/db/f8e zero size 2024-04-02T04:43:06.049 INFO:tasks.workunit.client.1.smithi027.stdout:6/670: fdatasync d13/f16 0 2024-04-02T04:43:06.051 INFO:tasks.workunit.client.1.smithi027.stdout:4/685: chown d24/d38/d47/d62 90 1 2024-04-02T04:43:06.057 INFO:tasks.workunit.client.1.smithi027.stdout:6/671: dwrite d13/d1b/f8e [0,4194304] 0 2024-04-02T04:43:06.057 INFO:tasks.workunit.client.1.smithi027.stdout:8/750: dread d5/d8/f2e [0,4194304] 0 2024-04-02T04:43:06.059 INFO:tasks.workunit.client.1.smithi027.stdout:9/888: truncate d1a/d1d/d29/f4f 120765 0 2024-04-02T04:43:06.062 INFO:tasks.workunit.client.1.smithi027.stdout:9/889: stat d1a/l86 0 2024-04-02T04:43:06.064 INFO:tasks.workunit.client.1.smithi027.stdout:4/686: truncate d24/d38/d44/d82/f96 119548 0 2024-04-02T04:43:06.064 INFO:tasks.workunit.client.1.smithi027.stdout:7/754: link d2/dba/d35/l96 d2/d1d/d90/lf5 0 2024-04-02T04:43:06.066 INFO:tasks.workunit.client.1.smithi027.stdout:0/815: dwrite d4/d38/f91 [0,4194304] 0 2024-04-02T04:43:06.068 INFO:tasks.workunit.client.1.smithi027.stdout:7/755: truncate d2/df/fdb 2674552 0 2024-04-02T04:43:06.069 INFO:tasks.workunit.client.1.smithi027.stdout:6/672: mknod d13/d1b/d24/d28/d46/dcc/da7/ce4 0 2024-04-02T04:43:06.072 INFO:tasks.workunit.client.1.smithi027.stdout:9/890: dwrite d1a/d1d/d29/d37/d9b/ff2 [4194304,4194304] 0 2024-04-02T04:43:06.073 INFO:tasks.workunit.client.1.smithi027.stdout:9/891: dread - d1a/f125 zero size 2024-04-02T04:43:06.078 INFO:tasks.workunit.client.1.smithi027.stdout:4/687: mkdir d24/d38/d47/dc0/ddd 0 2024-04-02T04:43:06.078 INFO:tasks.workunit.client.1.smithi027.stdout:9/892: write d1a/d1d/d29/d50/d73/fb1 [2234040,9790] 0 2024-04-02T04:43:06.081 INFO:tasks.workunit.client.1.smithi027.stdout:0/816: stat d4/c5 0 2024-04-02T04:43:06.083 INFO:tasks.workunit.client.1.smithi027.stdout:7/756: rmdir d2/df 39 2024-04-02T04:43:06.085 INFO:tasks.workunit.client.1.smithi027.stdout:6/673: truncate d13/d1b/d24/f41 1172478 0 2024-04-02T04:43:06.088 INFO:tasks.workunit.client.1.smithi027.stdout:7/757: dread d2/dba/dc/d16/d3f/d50/f69 [0,4194304] 0 2024-04-02T04:43:06.089 INFO:tasks.workunit.client.1.smithi027.stdout:8/751: write d5/d8/f33 [6011556,58883] 0 2024-04-02T04:43:06.091 INFO:tasks.workunit.client.1.smithi027.stdout:8/752: rename d5/d8/d15 to d5/d8/d15/d71/db0/dfb 22 2024-04-02T04:43:06.093 INFO:tasks.workunit.client.1.smithi027.stdout:0/817: rmdir d4/d16/d1b 39 2024-04-02T04:43:06.095 INFO:tasks.workunit.client.1.smithi027.stdout:4/688: mknod d24/d38/d44/d60/cde 0 2024-04-02T04:43:06.102 INFO:tasks.workunit.client.1.smithi027.stdout:6/674: creat d13/d1b/d24/d28/d3c/fe5 x:0 0 0 2024-04-02T04:43:06.106 INFO:tasks.workunit.client.1.smithi027.stdout:9/893: creat d1a/d1d/d29/d56/de4/d103/f127 x:0 0 0 2024-04-02T04:43:06.108 INFO:tasks.workunit.client.1.smithi027.stdout:9/894: read d1a/d1d/d29/d37/d20/d7e/f87 [2236081,20222] 0 2024-04-02T04:43:06.110 INFO:tasks.workunit.client.1.smithi027.stdout:8/753: chown d5/d8/db/dd/d5f/d27/d31/c4e 122772 1 2024-04-02T04:43:06.110 INFO:tasks.workunit.client.1.smithi027.stdout:9/895: write d1a/d1d/d29/d37/d4c/dc9/f108 [2680824,57549] 0 2024-04-02T04:43:06.116 INFO:tasks.workunit.client.1.smithi027.stdout:7/758: mknod d2/dba/cf6 0 2024-04-02T04:43:06.118 INFO:tasks.workunit.client.1.smithi027.stdout:0/818: dwrite d4/d21/d85/fa6 [0,4194304] 0 2024-04-02T04:43:06.123 INFO:tasks.workunit.client.1.smithi027.stdout:9/896: symlink d1a/d1d/d29/d51/l128 0 2024-04-02T04:43:06.125 INFO:tasks.workunit.client.1.smithi027.stdout:8/754: chown d5/d8/db/dd/d5f/l52 16995 1 2024-04-02T04:43:06.137 INFO:tasks.workunit.client.1.smithi027.stdout:7/759: rmdir d2/dba 39 2024-04-02T04:43:06.141 INFO:tasks.workunit.client.1.smithi027.stdout:0/819: write d4/d21/fb2 [1088821,50883] 0 2024-04-02T04:43:06.142 INFO:tasks.workunit.client.1.smithi027.stdout:9/897: fdatasync d1a/d1d/d3c/d6e/f97 0 2024-04-02T04:43:06.145 INFO:tasks.workunit.client.1.smithi027.stdout:6/675: write d13/d1b/d24/f3a [2326718,59956] 0 2024-04-02T04:43:06.148 INFO:tasks.workunit.client.1.smithi027.stdout:4/689: creat d24/d38/d47/fdf x:0 0 0 2024-04-02T04:43:06.148 INFO:tasks.workunit.client.1.smithi027.stdout:6/676: dread d13/f93 [0,4194304] 0 2024-04-02T04:43:06.151 INFO:tasks.workunit.client.1.smithi027.stdout:7/760: dread d2/d1d/f25 [0,4194304] 0 2024-04-02T04:43:06.157 INFO:tasks.workunit.client.1.smithi027.stdout:0/820: write d4/f23 [1829627,29565] 0 2024-04-02T04:43:06.158 INFO:tasks.workunit.client.1.smithi027.stdout:6/677: rmdir d13/d1b/d24/d25/d56/d73 39 2024-04-02T04:43:06.159 INFO:tasks.workunit.client.1.smithi027.stdout:6/678: chown d13/d1b/d48/fd7 1098694 1 2024-04-02T04:43:06.166 INFO:tasks.workunit.client.1.smithi027.stdout:6/679: read - d13/d1b/d48/d75/f9a zero size 2024-04-02T04:43:06.167 INFO:tasks.workunit.client.1.smithi027.stdout:9/898: truncate d1a/d1d/d29/d56/d8b/f90 619343 0 2024-04-02T04:43:06.171 INFO:tasks.workunit.client.1.smithi027.stdout:9/899: chown d1a/d1d/d29/d37/d3a/fa3 10360275 1 2024-04-02T04:43:06.174 INFO:tasks.workunit.client.1.smithi027.stdout:4/690: mknod d24/d38/d47/d62/db4/ce0 0 2024-04-02T04:43:06.176 INFO:tasks.workunit.client.1.smithi027.stdout:7/761: dwrite d2/dba/dc/d1f/fa1 [0,4194304] 0 2024-04-02T04:43:06.178 INFO:tasks.workunit.client.1.smithi027.stdout:0/821: dwrite d4/d38/f7f [0,4194304] 0 2024-04-02T04:43:06.178 INFO:tasks.workunit.client.1.smithi027.stdout:8/755: getdents d5/d8/db/dd/d5f/db2/dcf 0 2024-04-02T04:43:06.179 INFO:tasks.workunit.client.1.smithi027.stdout:0/822: readlink d4/d69/l9a 0 2024-04-02T04:43:06.184 INFO:tasks.workunit.client.1.smithi027.stdout:9/900: fsync d1a/d1d/d29/d37/d3a/f84 0 2024-04-02T04:43:06.187 INFO:tasks.workunit.client.1.smithi027.stdout:4/691: fdatasync d24/d38/d44/d41/f89 0 2024-04-02T04:43:06.191 INFO:tasks.workunit.client.1.smithi027.stdout:7/762: symlink d2/dba/d35/lf7 0 2024-04-02T04:43:06.200 INFO:tasks.workunit.client.1.smithi027.stdout:4/692: mknod d24/d48/ce1 0 2024-04-02T04:43:06.202 INFO:tasks.workunit.client.1.smithi027.stdout:6/680: dwrite d13/d1b/d24/f41 [0,4194304] 0 2024-04-02T04:43:06.203 INFO:tasks.workunit.client.1.smithi027.stdout:4/693: stat d24/d38/d44/d60/fc4 0 2024-04-02T04:43:06.203 INFO:tasks.workunit.client.1.smithi027.stdout:7/763: dread - d2/d1c/f62 zero size 2024-04-02T04:43:06.210 INFO:tasks.workunit.client.1.smithi027.stdout:8/756: dwrite d5/d8/db/dd/d5f/d27/f3a [4194304,4194304] 0 2024-04-02T04:43:06.210 INFO:tasks.workunit.client.1.smithi027.stdout:6/681: creat d13/d1b/d48/d58/dd3/fe6 x:0 0 0 2024-04-02T04:43:06.211 INFO:tasks.workunit.client.1.smithi027.stdout:9/901: dread d1a/d1d/d29/d51/d105/d8c/dcf/ff6 [0,4194304] 0 2024-04-02T04:43:06.212 INFO:tasks.workunit.client.1.smithi027.stdout:7/764: mkdir d2/dba/dc/d16/d2f/dcd/df8 0 2024-04-02T04:43:06.217 INFO:tasks.workunit.client.1.smithi027.stdout:8/757: fsync d5/f7a 0 2024-04-02T04:43:06.221 INFO:tasks.workunit.client.1.smithi027.stdout:5/890: sync 2024-04-02T04:43:06.221 INFO:tasks.workunit.client.1.smithi027.stdout:9/902: unlink d1a/d1d/d29/d37/d20/f2b 0 2024-04-02T04:43:06.221 INFO:tasks.workunit.client.1.smithi027.stdout:6/682: dwrite d13/d1b/f8e [0,4194304] 0 2024-04-02T04:43:06.225 INFO:tasks.workunit.client.1.smithi027.stdout:7/765: unlink d2/dba/dc/d1f/f84 0 2024-04-02T04:43:06.225 INFO:tasks.workunit.client.1.smithi027.stdout:9/903: fdatasync d1a/d1d/d29/d50/d73/dbd/d10a/f11d 0 2024-04-02T04:43:06.225 INFO:tasks.workunit.client.1.smithi027.stdout:7/766: fsync d2/d1d/d92/fc4 0 2024-04-02T04:43:06.229 INFO:tasks.workunit.client.1.smithi027.stdout:7/767: chown d2/dba/dc/d16/d2f/dcd/fcf 24683576 1 2024-04-02T04:43:06.234 INFO:tasks.workunit.client.1.smithi027.stdout:5/891: symlink d0/d1a/l12b 0 2024-04-02T04:43:06.238 INFO:tasks.workunit.client.1.smithi027.stdout:4/694: dwrite f1e [0,4194304] 0 2024-04-02T04:43:06.240 INFO:tasks.workunit.client.1.smithi027.stdout:9/904: stat d1a/d1d/d29/f46 0 2024-04-02T04:43:06.240 INFO:tasks.workunit.client.1.smithi027.stdout:9/905: chown d1a/d1d 745489 1 2024-04-02T04:43:06.242 INFO:tasks.workunit.client.1.smithi027.stdout:9/906: chown d1a/d1d/d29/d37/d3a/c52 21240654 1 2024-04-02T04:43:06.245 INFO:tasks.workunit.client.1.smithi027.stdout:8/758: creat d5/d8/db/dd/d5f/df8/ffc x:0 0 0 2024-04-02T04:43:06.247 INFO:tasks.workunit.client.1.smithi027.stdout:6/683: getdents d13/d1b/d48/d58/db9 0 2024-04-02T04:43:06.248 INFO:tasks.workunit.client.1.smithi027.stdout:4/695: rmdir d24/d38/d44/d41/d49 39 2024-04-02T04:43:06.251 INFO:tasks.workunit.client.1.smithi027.stdout:9/907: rename d1a/d1d/d29/d51/c5d to d1a/d1d/d29/d56/c129 0 2024-04-02T04:43:06.252 INFO:tasks.workunit.client.1.smithi027.stdout:9/908: readlink d1a/le0 0 2024-04-02T04:43:06.253 INFO:tasks.workunit.client.1.smithi027.stdout:6/684: dread - d13/d1b/d24/f84 zero size 2024-04-02T04:43:06.255 INFO:tasks.workunit.client.1.smithi027.stdout:8/759: dread - d5/d8/db/dd/d5f/d91/d9f/fb4 zero size 2024-04-02T04:43:06.256 INFO:tasks.workunit.client.1.smithi027.stdout:9/909: chown d1a/d1d/d29/d50/d117/cf5 452820 1 2024-04-02T04:43:06.257 INFO:tasks.workunit.client.1.smithi027.stdout:4/696: creat d24/d38/d47/dc0/dca/fe2 x:0 0 0 2024-04-02T04:43:06.259 INFO:tasks.workunit.client.1.smithi027.stdout:7/768: dread d2/d1c/f40 [0,4194304] 0 2024-04-02T04:43:06.260 INFO:tasks.workunit.client.1.smithi027.stdout:7/769: chown d2/d1c/l34 278 1 2024-04-02T04:43:06.261 INFO:tasks.workunit.client.1.smithi027.stdout:8/760: rmdir d5/d8/db/dd/d5f/db2/dcf 39 2024-04-02T04:43:06.262 INFO:tasks.workunit.client.1.smithi027.stdout:5/892: dread d0/d68/d6b/d74/fb6 [0,4194304] 0 2024-04-02T04:43:06.262 INFO:tasks.workunit.client.1.smithi027.stdout:7/770: write d2/d1c/fdc [449252,97722] 0 2024-04-02T04:43:06.263 INFO:tasks.workunit.client.1.smithi027.stdout:6/685: creat d13/d1b/d24/d25/d56/d73/d96/fe7 x:0 0 0 2024-04-02T04:43:06.263 INFO:tasks.workunit.client.1.smithi027.stdout:8/761: chown d5/d8/d15/d71/d96 4192999 1 2024-04-02T04:43:06.266 INFO:tasks.workunit.client.1.smithi027.stdout:4/697: creat d24/d38/d47/d5c/fe3 x:0 0 0 2024-04-02T04:43:06.268 INFO:tasks.workunit.client.1.smithi027.stdout:9/910: rename d1a/d1d/d29/d37/d9b/fed to d1a/d1d/d29/d37/d20/d7e/f12a 0 2024-04-02T04:43:06.269 INFO:tasks.workunit.client.1.smithi027.stdout:9/911: readlink d1a/d1d/d29/d37/l7d 0 2024-04-02T04:43:06.277 INFO:tasks.workunit.client.1.smithi027.stdout:7/771: dread d2/dba/fb3 [0,4194304] 0 2024-04-02T04:43:06.277 INFO:tasks.workunit.client.1.smithi027.stdout:6/686: mknod d13/d1b/d24/d28/d46/dcc/da7/ce8 0 2024-04-02T04:43:06.278 INFO:tasks.workunit.client.1.smithi027.stdout:4/698: mkdir d24/d48/de4 0 2024-04-02T04:43:06.279 INFO:tasks.workunit.client.1.smithi027.stdout:6/687: dread - d13/d1b/d24/d25/d56/d73/d96/fe7 zero size 2024-04-02T04:43:06.280 INFO:tasks.workunit.client.1.smithi027.stdout:6/688: chown d13/d1b/d24/d28/l80 174842 1 2024-04-02T04:43:06.282 INFO:tasks.workunit.client.1.smithi027.stdout:9/912: symlink d1a/d1d/d29/d37/d4c/dc9/l12b 0 2024-04-02T04:43:06.285 INFO:tasks.workunit.client.1.smithi027.stdout:8/762: mkdir d5/d8/db/dd/d5f/d91/dc2/dfd 0 2024-04-02T04:43:06.286 INFO:tasks.workunit.client.1.smithi027.stdout:5/893: dwrite d0/df/d11/d13/dce/d55/dc3/df2/ff9 [0,4194304] 0 2024-04-02T04:43:06.287 INFO:tasks.workunit.client.1.smithi027.stdout:5/894: fsync d0/df/d29/ff1 0 2024-04-02T04:43:06.289 INFO:tasks.workunit.client.1.smithi027.stdout:4/699: symlink d24/d38/d47/dc0/dca/le5 0 2024-04-02T04:43:06.291 INFO:tasks.workunit.client.1.smithi027.stdout:6/689: creat d13/d1b/d24/d25/d56/dc8/fe9 x:0 0 0 2024-04-02T04:43:06.294 INFO:tasks.workunit.client.1.smithi027.stdout:7/772: mknod d2/dba/dc/d16/d3a/da5/cf9 0 2024-04-02T04:43:06.296 INFO:tasks.workunit.client.1.smithi027.stdout:8/763: creat d5/d8/db/dd/d4a/dc9/ffe x:0 0 0 2024-04-02T04:43:06.298 INFO:tasks.workunit.client.1.smithi027.stdout:4/700: fsync d24/f2c 0 2024-04-02T04:43:06.301 INFO:tasks.workunit.client.1.smithi027.stdout:6/690: creat d13/d1b/d24/d28/d3c/fea x:0 0 0 2024-04-02T04:43:06.302 INFO:tasks.workunit.client.1.smithi027.stdout:6/691: readlink d13/d1b/d24/d25/d27/l30 0 2024-04-02T04:43:06.310 INFO:tasks.workunit.client.1.smithi027.stdout:7/773: truncate d2/dba/dc/d16/d2f/f73 170588 0 2024-04-02T04:43:06.313 INFO:tasks.workunit.client.1.smithi027.stdout:5/895: creat d0/df/d2f/d3b/dca/dea/f12c x:0 0 0 2024-04-02T04:43:06.316 INFO:tasks.workunit.client.1.smithi027.stdout:8/764: truncate d5/d8/db/dd/f4f 4226169 0 2024-04-02T04:43:06.317 INFO:tasks.workunit.client.1.smithi027.stdout:8/765: chown d5/d8/f2e 66371292 1 2024-04-02T04:43:06.320 INFO:tasks.workunit.client.1.smithi027.stdout:6/692: symlink d13/d1b/d24/d25/d27/leb 0 2024-04-02T04:43:06.321 INFO:tasks.workunit.client.1.smithi027.stdout:6/693: dread - d13/d1b/d24/d25/d56/dc8/dc9/fd8 zero size 2024-04-02T04:43:06.322 INFO:tasks.workunit.client.1.smithi027.stdout:7/774: symlink d2/dba/dc/d16/d3f/d50/d8b/d91/lfa 0 2024-04-02T04:43:06.323 INFO:tasks.workunit.client.1.smithi027.stdout:5/896: truncate d0/d1a/d20/f93 4897482 0 2024-04-02T04:43:06.325 INFO:tasks.workunit.client.1.smithi027.stdout:5/897: fsync d0/df/d11/d13/dce/fbc 0 2024-04-02T04:43:06.327 INFO:tasks.workunit.client.1.smithi027.stdout:9/913: dwrite d1a/d1d/d29/d50/d73/f7b [0,4194304] 0 2024-04-02T04:43:06.327 INFO:tasks.workunit.client.1.smithi027.stdout:7/775: creat d2/dba/dc/d1f/ffb x:0 0 0 2024-04-02T04:43:06.334 INFO:tasks.workunit.client.1.smithi027.stdout:9/914: creat d1a/d1d/d29/d51/d105/d102/f12c x:0 0 0 2024-04-02T04:43:06.335 INFO:tasks.workunit.client.1.smithi027.stdout:7/776: symlink d2/dba/dc/d16/d3f/d79/lfc 0 2024-04-02T04:43:06.336 INFO:tasks.workunit.client.1.smithi027.stdout:5/898: dwrite d0/df/d2f/d3b/dca/dea/f12c [0,4194304] 0 2024-04-02T04:43:06.340 INFO:tasks.workunit.client.1.smithi027.stdout:6/694: link d13/c1c d13/d1b/d48/d9c/cec 0 2024-04-02T04:43:06.344 INFO:tasks.workunit.client.1.smithi027.stdout:5/899: dwrite d0/d1a/d20/fde [0,4194304] 0 2024-04-02T04:43:06.350 INFO:tasks.workunit.client.1.smithi027.stdout:3/720: sync 2024-04-02T04:43:06.350 INFO:tasks.workunit.client.1.smithi027.stdout:1/731: sync 2024-04-02T04:43:06.350 INFO:tasks.workunit.client.1.smithi027.stdout:2/859: sync 2024-04-02T04:43:06.354 INFO:tasks.workunit.client.1.smithi027.stdout:9/915: dread d1a/d1d/d29/d51/fa6 [0,4194304] 0 2024-04-02T04:43:06.356 INFO:tasks.workunit.client.1.smithi027.stdout:1/732: read - d0/d1e/d2e/d3e/d5f/fb6 zero size 2024-04-02T04:43:06.358 INFO:tasks.workunit.client.1.smithi027.stdout:3/721: dread d1/f94 [0,4194304] 0 2024-04-02T04:43:06.361 INFO:tasks.workunit.client.1.smithi027.stdout:1/733: unlink d0/f3d 0 2024-04-02T04:43:06.362 INFO:tasks.workunit.client.1.smithi027.stdout:5/900: dwrite d0/df/d11/d13/d9a/d9c/dbe/fe7 [0,4194304] 0 2024-04-02T04:43:06.368 INFO:tasks.workunit.client.1.smithi027.stdout:4/701: dwrite d24/d2d/f43 [0,4194304] 0 2024-04-02T04:43:06.368 INFO:tasks.workunit.client.1.smithi027.stdout:9/916: rename d1a/d1d/d29/d50/da2 to d1a/d1d/d29/d51/d105/d8c/dcf/d12d 0 2024-04-02T04:43:06.371 INFO:tasks.workunit.client.1.smithi027.stdout:5/901: chown d0/df/d11/d37/c8f 715 1 2024-04-02T04:43:06.374 INFO:tasks.workunit.client.1.smithi027.stdout:8/766: write d5/d8/db/dd/f1f [1488164,109801] 0 2024-04-02T04:43:06.375 INFO:tasks.workunit.client.1.smithi027.stdout:4/702: symlink d24/d38/d44/d41/d5a/le6 0 2024-04-02T04:43:06.377 INFO:tasks.workunit.client.1.smithi027.stdout:8/767: truncate d5/d8/db/dd5/fde 975013 0 2024-04-02T04:43:06.377 INFO:tasks.workunit.client.1.smithi027.stdout:4/703: write f23 [4480022,87079] 0 2024-04-02T04:43:06.377 INFO:tasks.workunit.client.1.smithi027.stdout:9/917: mknod d1a/dce/c12e 0 2024-04-02T04:43:06.380 INFO:tasks.workunit.client.1.smithi027.stdout:5/902: creat d0/df/d11/d8c/f12d x:0 0 0 2024-04-02T04:43:06.381 INFO:tasks.workunit.client.1.smithi027.stdout:9/918: readlink d1a/d1d/d29/d56/d8b/le3 0 2024-04-02T04:43:06.382 INFO:tasks.workunit.client.1.smithi027.stdout:3/722: symlink d1/de/df/da5/leb 0 2024-04-02T04:43:06.386 INFO:tasks.workunit.client.1.smithi027.stdout:4/704: rename d24/d38/d44/d60/cde to d24/d38/d47/dc0/dca/ce7 0 2024-04-02T04:43:06.386 INFO:tasks.workunit.client.1.smithi027.stdout:4/705: fsync f21 0 2024-04-02T04:43:06.388 INFO:tasks.workunit.client.1.smithi027.stdout:5/903: mkdir d0/df/d11/d37/d12e 0 2024-04-02T04:43:06.392 INFO:tasks.workunit.client.1.smithi027.stdout:0/823: sync 2024-04-02T04:43:06.393 INFO:tasks.workunit.client.1.smithi027.stdout:9/919: rename d1a/l7f to d1a/dce/l12f 0 2024-04-02T04:43:06.394 INFO:tasks.workunit.client.1.smithi027.stdout:1/734: getdents d0/d1e/d25 0 2024-04-02T04:43:06.398 INFO:tasks.workunit.client.1.smithi027.stdout:4/706: stat d24/d38/d47/d62/f9a 0 2024-04-02T04:43:06.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:06 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:43:06.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:06 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:43:06.405 INFO:tasks.workunit.client.1.smithi027.stdout:4/707: readlink d24/d38/d44/d41/d49/lac 0 2024-04-02T04:43:06.406 INFO:tasks.workunit.client.1.smithi027.stdout:1/735: dread d0/f1d [0,4194304] 0 2024-04-02T04:43:06.409 INFO:tasks.workunit.client.1.smithi027.stdout:4/708: fdatasync d24/d38/d44/d82/d95/fb6 0 2024-04-02T04:43:06.412 INFO:tasks.workunit.client.1.smithi027.stdout:0/824: readlink d4/d16/d1b/l43 0 2024-04-02T04:43:06.412 INFO:tasks.workunit.client.1.smithi027.stdout:9/920: creat d1a/d1d/d29/d51/d105/dec/f130 x:0 0 0 2024-04-02T04:43:06.413 INFO:tasks.workunit.client.1.smithi027.stdout:6/695: dwrite d13/f16 [0,4194304] 0 2024-04-02T04:43:06.415 INFO:tasks.workunit.client.1.smithi027.stdout:3/723: rename d1/de/df/d13/d73/d77/d7f to d1/de/df/dec 0 2024-04-02T04:43:06.417 INFO:tasks.workunit.client.1.smithi027.stdout:5/904: dread d0/d68/dd7/fe2 [0,4194304] 0 2024-04-02T04:43:06.417 INFO:tasks.workunit.client.1.smithi027.stdout:7/777: dwrite d2/dba/dc/fab [0,4194304] 0 2024-04-02T04:43:06.418 INFO:tasks.workunit.client.1.smithi027.stdout:6/696: readlink d13/d1b/d48/d58/l5f 0 2024-04-02T04:43:06.421 INFO:tasks.workunit.client.1.smithi027.stdout:5/905: dread - d0/df/d11/d8c/f12d zero size 2024-04-02T04:43:06.423 INFO:tasks.workunit.client.1.smithi027.stdout:9/921: truncate d1a/d1d/d29/d50/d73/ffe 253204 0 2024-04-02T04:43:06.425 INFO:tasks.workunit.client.1.smithi027.stdout:2/860: dwrite d1/f88 [0,4194304] 0 2024-04-02T04:43:06.425 INFO:tasks.workunit.client.1.smithi027.stdout:0/825: rmdir d4/d38 39 2024-04-02T04:43:06.428 INFO:tasks.workunit.client.1.smithi027.stdout:0/826: truncate d4/d21/d28/f40 1823162 0 2024-04-02T04:43:06.440 INFO:tasks.workunit.client.1.smithi027.stdout:7/778: symlink d2/dba/dc/d16/d3f/d79/lfd 0 2024-04-02T04:43:06.448 INFO:tasks.workunit.client.1.smithi027.stdout:7/779: dwrite d2/dba/dc/d1f/d61/fca [0,4194304] 0 2024-04-02T04:43:06.450 INFO:tasks.workunit.client.1.smithi027.stdout:0/827: readlink d4/d38/d66/d4a/l92 0 2024-04-02T04:43:06.450 INFO:tasks.workunit.client.1.smithi027.stdout:3/724: rmdir d1/de/df/d9e/da0 39 2024-04-02T04:43:06.452 INFO:tasks.workunit.client.1.smithi027.stdout:3/725: write d1/de/df/d9e/fb4 [80285,107008] 0 2024-04-02T04:43:06.453 INFO:tasks.workunit.client.1.smithi027.stdout:3/726: dread - d1/d1a/fe0 zero size 2024-04-02T04:43:06.454 INFO:tasks.workunit.client.1.smithi027.stdout:3/727: write d1/de/df/d9e/fbe [2126830,92773] 0 2024-04-02T04:43:06.455 INFO:tasks.workunit.client.1.smithi027.stdout:4/709: creat d24/fe8 x:0 0 0 2024-04-02T04:43:06.457 INFO:tasks.workunit.client.1.smithi027.stdout:6/697: mkdir d13/d1b/d24/d25/ded 0 2024-04-02T04:43:06.459 INFO:tasks.workunit.client.1.smithi027.stdout:6/698: fsync d13/d1b/d24/d25/d27/f81 0 2024-04-02T04:43:06.465 INFO:tasks.workunit.client.1.smithi027.stdout:7/780: mknod d2/dba/dc/d1f/cfe 0 2024-04-02T04:43:06.470 INFO:tasks.workunit.client.1.smithi027.stdout:8/768: write d5/d8/db/dd/d5f/d27/d31/f9c [100661,113724] 0 2024-04-02T04:43:06.471 INFO:tasks.workunit.client.1.smithi027.stdout:0/828: dread d4/d76/dc4/dfe/dd1/f100 [0,4194304] 0 2024-04-02T04:43:06.471 INFO:tasks.workunit.client.1.smithi027.stdout:3/728: unlink d1/cd8 0 2024-04-02T04:43:06.472 INFO:tasks.workunit.client.1.smithi027.stdout:3/729: chown d1/c83 765 1 2024-04-02T04:43:06.473 INFO:tasks.workunit.client.1.smithi027.stdout:4/710: chown d24/f63 2 1 2024-04-02T04:43:06.478 INFO:tasks.workunit.client.1.smithi027.stdout:6/699: creat d13/d1b/d24/d25/d56/dc8/dc9/fee x:0 0 0 2024-04-02T04:43:06.483 INFO:tasks.workunit.client.1.smithi027.stdout:9/922: link d1a/d1d/d29/d37/c11e d1a/d1d/d29/d37/d9b/c131 0 2024-04-02T04:43:06.486 INFO:tasks.workunit.client.1.smithi027.stdout:7/781: chown d2/dba/dc/f7b 1228499 1 2024-04-02T04:43:06.488 INFO:tasks.workunit.client.1.smithi027.stdout:6/700: dwrite d13/d1b/d48/d58/f64 [0,4194304] 0 2024-04-02T04:43:06.490 INFO:tasks.workunit.client.1.smithi027.stdout:6/701: chown d13/d1b/d48/d58/db9 12 1 2024-04-02T04:43:06.496 INFO:tasks.workunit.client.1.smithi027.stdout:8/769: fsync d5/d8/db/dd/d5f/db2/fe5 0 2024-04-02T04:43:06.504 INFO:tasks.workunit.client.1.smithi027.stdout:0/829: rename d4/d21/d93/d10d to d4/d76/dc4/dfe/d117 0 2024-04-02T04:43:06.508 INFO:tasks.workunit.client.1.smithi027.stdout:7/782: mkdir d2/df/d89/dff 0 2024-04-02T04:43:06.512 INFO:tasks.workunit.client.1.smithi027.stdout:1/736: write d0/de/d52/d74/d97/fc2 [620030,86099] 0 2024-04-02T04:43:06.515 INFO:tasks.workunit.client.1.smithi027.stdout:6/702: creat d13/d1b/d48/d75/d9b/de2/fef x:0 0 0 2024-04-02T04:43:06.519 INFO:tasks.workunit.client.1.smithi027.stdout:7/783: dwrite d2/df/d89/d8d/fc8 [0,4194304] 0 2024-04-02T04:43:06.524 INFO:tasks.workunit.client.1.smithi027.stdout:4/711: getdents d24/d38/d44/dbc 0 2024-04-02T04:43:06.532 INFO:tasks.workunit.client.1.smithi027.stdout:7/784: dwrite d2/dba/dc/fab [0,4194304] 0 2024-04-02T04:43:06.540 INFO:tasks.workunit.client.1.smithi027.stdout:4/712: creat d24/d38/d47/dc0/fe9 x:0 0 0 2024-04-02T04:43:06.541 INFO:tasks.workunit.client.1.smithi027.stdout:1/737: truncate d0/d1e/d2e/f6d 1612682 0 2024-04-02T04:43:06.543 INFO:tasks.workunit.client.1.smithi027.stdout:2/861: dwrite d1/db/d3b/d17/d6d/d127/f71 [0,4194304] 0 2024-04-02T04:43:06.544 INFO:tasks.workunit.client.1.smithi027.stdout:4/713: read - d24/d38/d47/dc0/fe9 zero size 2024-04-02T04:43:06.547 INFO:tasks.workunit.client.1.smithi027.stdout:6/703: creat d13/d1b/d24/d25/d56/ff0 x:0 0 0 2024-04-02T04:43:06.547 INFO:tasks.workunit.client.1.smithi027.stdout:1/738: chown d0/d1e/d2e/f6b 0 1 2024-04-02T04:43:06.548 INFO:tasks.workunit.client.1.smithi027.stdout:7/785: write d2/dba/dc/d16/f2c [2231815,15757] 0 2024-04-02T04:43:06.550 INFO:tasks.workunit.client.1.smithi027.stdout:2/862: write d1/db/d3b/d17/d6d/dd9/dd5/dda/d8a/f10f [5092844,16821] 0 2024-04-02T04:43:06.551 INFO:tasks.workunit.client.1.smithi027.stdout:6/704: write d13/d1b/d24/d25/d56/ff0 [904205,124602] 0 2024-04-02T04:43:06.556 INFO:tasks.workunit.client.1.smithi027.stdout:7/786: unlink d2/d1d/d92/fc4 0 2024-04-02T04:43:06.558 INFO:tasks.workunit.client.1.smithi027.stdout:6/705: symlink d13/d1b/d48/d58/lf1 0 2024-04-02T04:43:06.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:06 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:43:06.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:06 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:43:06.563 INFO:tasks.workunit.client.1.smithi027.stdout:4/714: getdents d24/d38/d47 0 2024-04-02T04:43:06.566 INFO:tasks.workunit.client.1.smithi027.stdout:1/739: creat d0/d1e/d2e/d3b/fee x:0 0 0 2024-04-02T04:43:06.568 INFO:tasks.workunit.client.1.smithi027.stdout:7/787: dwrite d2/dba/dc/d16/d3f/d44/fe0 [0,4194304] 0 2024-04-02T04:43:06.569 INFO:tasks.workunit.client.1.smithi027.stdout:4/715: chown d24/d38/d44/d60/l69 133774 1 2024-04-02T04:43:06.569 INFO:tasks.workunit.client.1.smithi027.stdout:1/740: readlink d0/d1e/d2e/d3b/d4f/l83 0 2024-04-02T04:43:06.572 INFO:tasks.workunit.client.1.smithi027.stdout:3/730: write d1/f2e [508899,93291] 0 2024-04-02T04:43:06.575 INFO:tasks.workunit.client.1.smithi027.stdout:2/863: dread d1/db/d3b/d17/d2b/f77 [0,4194304] 0 2024-04-02T04:43:06.576 INFO:tasks.workunit.client.1.smithi027.stdout:4/716: mkdir d24/d48/db0/dea 0 2024-04-02T04:43:06.577 INFO:tasks.workunit.client.1.smithi027.stdout:7/788: creat d2/dba/dc/d16/d3a/d4d/dcb/f100 x:0 0 0 2024-04-02T04:43:06.580 INFO:tasks.workunit.client.1.smithi027.stdout:0/830: rename d4/d76/dc4 to d4/d21/d28/d4e/db5/d118 0 2024-04-02T04:43:06.582 INFO:tasks.workunit.client.1.smithi027.stdout:1/741: symlink d0/d1e/d2e/d3e/d5f/lef 0 2024-04-02T04:43:06.583 INFO:tasks.workunit.client.1.smithi027.stdout:3/731: dwrite d1/d1a/fbd [0,4194304] 0 2024-04-02T04:43:06.586 INFO:tasks.workunit.client.1.smithi027.stdout:7/789: truncate d2/dba/dc/d16/d3f/d50/f66 953271 0 2024-04-02T04:43:06.589 INFO:tasks.workunit.client.1.smithi027.stdout:4/717: symlink d24/d38/d44/d41/d49/leb 0 2024-04-02T04:43:06.591 INFO:tasks.workunit.client.1.smithi027.stdout:2/864: truncate d1/db/d3b/d17/d6d/dd9/dd5/f4f 1258009 0 2024-04-02T04:43:06.594 INFO:tasks.workunit.client.1.smithi027.stdout:6/706: dread d13/d1b/d24/d28/d3c/f78 [8388608,4194304] 0 2024-04-02T04:43:06.595 INFO:tasks.workunit.client.1.smithi027.stdout:0/831: dwrite d4/d38/d66/d4a/f61 [0,4194304] 0 2024-04-02T04:43:06.599 INFO:tasks.workunit.client.1.smithi027.stdout:3/732: creat d1/d1a/fed x:0 0 0 2024-04-02T04:43:06.600 INFO:tasks.workunit.client.1.smithi027.stdout:6/707: fsync d13/d1b/d24/d25/d56/ff0 0 2024-04-02T04:43:06.600 INFO:tasks.workunit.client.1.smithi027.stdout:8/770: dwrite d5/d8/f47 [0,4194304] 0 2024-04-02T04:43:06.604 INFO:tasks.workunit.client.1.smithi027.stdout:6/708: truncate d13/d1b/d24/d25/d56/dc8/fe9 713511 0 2024-04-02T04:43:06.605 INFO:tasks.workunit.client.1.smithi027.stdout:1/742: creat d0/d1e/d2e/d3b/d4f/d9b/ff0 x:0 0 0 2024-04-02T04:43:06.607 INFO:tasks.workunit.client.1.smithi027.stdout:6/709: write d13/d1b/d24/d25/d56/d6d/fcf [79190,33907] 0 2024-04-02T04:43:06.608 INFO:tasks.workunit.client.1.smithi027.stdout:7/790: rename d2/d1d/d57/f6d to d2/d7/df3/f101 0 2024-04-02T04:43:06.608 INFO:tasks.workunit.client.1.smithi027.stdout:6/710: write d13/fbf [757470,55460] 0 2024-04-02T04:43:06.610 INFO:tasks.workunit.client.1.smithi027.stdout:6/711: read d13/d1b/d24/d25/d42/fa9 [2430077,103908] 0 2024-04-02T04:43:06.616 INFO:tasks.workunit.client.1.smithi027.stdout:2/865: creat d1/db/d3b/d17/d6d/dd9/dd5/dda/dbb/f134 x:0 0 0 2024-04-02T04:43:06.622 INFO:tasks.workunit.client.1.smithi027.stdout:1/743: unlink d0/d1e/d2e/f4a 0 2024-04-02T04:43:06.623 INFO:tasks.workunit.client.1.smithi027.stdout:3/733: dread d1/f7 [0,4194304] 0 2024-04-02T04:43:06.626 INFO:tasks.workunit.client.1.smithi027.stdout:4/718: creat d24/d48/de4/fec x:0 0 0 2024-04-02T04:43:06.627 INFO:tasks.workunit.client.1.smithi027.stdout:7/791: rename d2/dba/dc/f7b to d2/dba/dc/d1f/d33/d36/f102 0 2024-04-02T04:43:06.628 INFO:tasks.workunit.client.1.smithi027.stdout:6/712: symlink d13/d1b/d24/d28/d46/dcc/da7/lf2 0 2024-04-02T04:43:06.632 INFO:tasks.workunit.client.1.smithi027.stdout:1/744: chown d0/d1e/d25/d3c/d64/cdc 16 1 2024-04-02T04:43:06.633 INFO:tasks.workunit.client.1.smithi027.stdout:1/745: read d0/f69 [4233944,27986] 0 2024-04-02T04:43:06.635 INFO:tasks.workunit.client.1.smithi027.stdout:4/719: unlink d24/d38/d44/l67 0 2024-04-02T04:43:06.637 INFO:tasks.workunit.client.1.smithi027.stdout:7/792: creat d2/d7/df3/f103 x:0 0 0 2024-04-02T04:43:06.639 INFO:tasks.workunit.client.1.smithi027.stdout:0/832: write d4/f60 [1495545,83023] 0 2024-04-02T04:43:06.640 INFO:tasks.workunit.client.1.smithi027.stdout:0/833: stat d4/d38/d66/d4a/l45 0 2024-04-02T04:43:06.641 INFO:tasks.workunit.client.1.smithi027.stdout:6/713: creat d13/d1b/d48/d9c/ff3 x:0 0 0 2024-04-02T04:43:06.642 INFO:tasks.workunit.client.1.smithi027.stdout:0/834: write d4/d21/d28/d4e/d5f/f79 [341069,67610] 0 2024-04-02T04:43:06.642 INFO:tasks.workunit.client.1.smithi027.stdout:6/714: chown d13/d1b/d24/d25/d42/l61 6885 1 2024-04-02T04:43:06.643 INFO:tasks.workunit.client.1.smithi027.stdout:0/835: dread - d4/d5d/fe5 zero size 2024-04-02T04:43:06.643 INFO:tasks.workunit.client.1.smithi027.stdout:8/771: link d5/d8/db/dd/d5f/d27/d31/d5b/f67 d5/db5/fff 0 2024-04-02T04:43:06.644 INFO:tasks.workunit.client.1.smithi027.stdout:6/715: stat d13/d1b/d24/cd9 0 2024-04-02T04:43:06.644 INFO:tasks.workunit.client.1.smithi027.stdout:3/734: symlink d1/de/df/d9e/lee 0 2024-04-02T04:43:06.646 INFO:tasks.workunit.client.1.smithi027.stdout:2/866: dread d1/db/d3b/d17/d2b/db8/fa4 [0,4194304] 0 2024-04-02T04:43:06.646 INFO:tasks.workunit.client.1.smithi027.stdout:0/836: chown d4/d38/d66/d2d/f3c 0 1 2024-04-02T04:43:06.653 INFO:tasks.workunit.client.1.smithi027.stdout:7/793: mkdir d2/df/d89/d8d/d104 0 2024-04-02T04:43:06.655 INFO:tasks.workunit.client.1.smithi027.stdout:1/746: rename d0/d1e to d0/de/d52/d74/db7/df1 0 2024-04-02T04:43:06.662 INFO:tasks.workunit.client.1.smithi027.stdout:0/837: creat d4/d21/d28/d4e/db5/f119 x:0 0 0 2024-04-02T04:43:06.668 INFO:tasks.workunit.client.1.smithi027.stdout:5/906: sync 2024-04-02T04:43:06.669 INFO:tasks.workunit.client.1.smithi027.stdout:1/747: dread d0/de/d52/d74/db7/df1/d2e/d3b/f77 [0,4194304] 0 2024-04-02T04:43:06.671 INFO:tasks.workunit.client.1.smithi027.stdout:0/838: dwrite d4/d21/d28/f48 [0,4194304] 0 2024-04-02T04:43:06.679 INFO:tasks.workunit.client.1.smithi027.stdout:4/720: write d24/d2d/f8a [1417457,114631] 0 2024-04-02T04:43:06.694 INFO:tasks.workunit.client.1.smithi027.stdout:2/867: dwrite d1/db/d3b/d17/d12/f7e [0,4194304] 0 2024-04-02T04:43:06.717 INFO:tasks.workunit.client.1.smithi027.stdout:8/772: fsync d5/d8/db/dd/f4f 0 2024-04-02T04:43:06.719 INFO:tasks.workunit.client.1.smithi027.stdout:3/735: symlink d1/de/df/d9e/lef 0 2024-04-02T04:43:06.720 INFO:tasks.workunit.client.1.smithi027.stdout:3/736: chown d1/l6 641130 1 2024-04-02T04:43:06.723 INFO:tasks.workunit.client.1.smithi027.stdout:0/839: write d4/d21/d28/d4e/d8c/db0/f114 [668640,124376] 0 2024-04-02T04:43:06.725 INFO:tasks.workunit.client.1.smithi027.stdout:4/721: creat d24/d38/d44/d60/fed x:0 0 0 2024-04-02T04:43:06.728 INFO:tasks.workunit.client.1.smithi027.stdout:7/794: creat d2/df/dc7/f105 x:0 0 0 2024-04-02T04:43:06.732 INFO:tasks.workunit.client.1.smithi027.stdout:6/716: creat d13/d1b/d48/d75/ff4 x:0 0 0 2024-04-02T04:43:06.733 INFO:tasks.workunit.client.1.smithi027.stdout:6/717: chown d13/d1b/d24/d25/d56/l68 6 1 2024-04-02T04:43:06.733 INFO:tasks.workunit.client.1.smithi027.stdout:9/923: sync 2024-04-02T04:43:06.734 INFO:tasks.workunit.client.1.smithi027.stdout:9/924: fsync d1a/d1d/d29/d51/d105/d102/f12c 0 2024-04-02T04:43:06.736 INFO:tasks.workunit.client.1.smithi027.stdout:3/737: symlink d1/de/df/d9e/dc6/lf0 0 2024-04-02T04:43:06.737 INFO:tasks.workunit.client.1.smithi027.stdout:5/907: creat d0/df/d11/d13/d9a/d9c/db8/d11e/f12f x:0 0 0 2024-04-02T04:43:06.739 INFO:tasks.workunit.client.1.smithi027.stdout:5/908: chown d0/df/d29/ce0 659 1 2024-04-02T04:43:06.740 INFO:tasks.workunit.client.1.smithi027.stdout:4/722: truncate d24/d38/d47/d62/fd0 244763 0 2024-04-02T04:43:06.745 INFO:tasks.workunit.client.1.smithi027.stdout:9/925: symlink d1a/d1d/d29/d50/d73/dbd/d10a/l132 0 2024-04-02T04:43:06.745 INFO:tasks.workunit.client.1.smithi027.stdout:3/738: dwrite d1/f46 [4194304,4194304] 0 2024-04-02T04:43:06.748 INFO:tasks.workunit.client.1.smithi027.stdout:0/840: mknod d4/d21/d28/d4e/d5f/c11a 0 2024-04-02T04:43:06.750 INFO:tasks.workunit.client.1.smithi027.stdout:9/926: write d1a/d1d/d29/d56/d8b/fa4 [2155008,67772] 0 2024-04-02T04:43:06.751 INFO:tasks.workunit.client.1.smithi027.stdout:6/718: mknod d13/d1b/d24/d28/cf5 0 2024-04-02T04:43:06.753 INFO:tasks.workunit.client.1.smithi027.stdout:5/909: dwrite d0/df/f121 [0,4194304] 0 2024-04-02T04:43:06.753 INFO:tasks.workunit.client.1.smithi027.stdout:1/748: truncate d0/de/d52/d74/db7/df1/fa3 2865081 0 2024-04-02T04:43:06.765 INFO:tasks.workunit.client.1.smithi027.stdout:9/927: rename d1a/d1d/d29/d37/d9b to d1a/d1d/d29/d51/dfa/d133 0 2024-04-02T04:43:06.766 INFO:tasks.workunit.client.1.smithi027.stdout:6/719: dwrite d13/d1b/d24/d25/d56/ff0 [0,4194304] 0 2024-04-02T04:43:06.767 INFO:tasks.workunit.client.1.smithi027.stdout:4/723: dread d24/d38/d44/d82/d95/fc2 [0,4194304] 0 2024-04-02T04:43:06.770 INFO:tasks.workunit.client.1.smithi027.stdout:4/724: truncate f13 4245449 0 2024-04-02T04:43:06.770 INFO:tasks.workunit.client.1.smithi027.stdout:9/928: write d1a/d1d/d29/d50/d73/fb1 [419725,120790] 0 2024-04-02T04:43:06.771 INFO:tasks.workunit.client.1.smithi027.stdout:5/910: write d0/d68/d6b/d74/fb1 [100559,129745] 0 2024-04-02T04:43:06.772 INFO:tasks.workunit.client.1.smithi027.stdout:1/749: symlink d0/de/d52/d74/db7/df1/d25/lf2 0 2024-04-02T04:43:06.773 INFO:tasks.workunit.client.1.smithi027.stdout:7/795: getdents d2/dba/dc/d1f/d33/d36 0 2024-04-02T04:43:06.775 INFO:tasks.workunit.client.1.smithi027.stdout:0/841: symlink d4/d21/d85/dfb/l11b 0 2024-04-02T04:43:06.778 INFO:tasks.workunit.client.1.smithi027.stdout:0/842: write d4/d38/f7f [407925,20080] 0 2024-04-02T04:43:06.782 INFO:tasks.workunit.client.1.smithi027.stdout:6/720: unlink d13/d1b/d24/d28/d46/f8f 0 2024-04-02T04:43:06.784 INFO:tasks.workunit.client.1.smithi027.stdout:2/868: write d1/db/d3b/d17/d68/f94 [1128585,127816] 0 2024-04-02T04:43:06.788 INFO:tasks.workunit.client.1.smithi027.stdout:9/929: dwrite d1a/d1d/d29/d51/d105/d8c/dcf/d12d/dfb/f123 [0,4194304] 0 2024-04-02T04:43:06.789 INFO:tasks.workunit.client.1.smithi027.stdout:9/930: stat d1a 0 2024-04-02T04:43:06.790 INFO:tasks.workunit.client.1.smithi027.stdout:4/725: rename d24/d38/d44/d41/d5a/lbe to d24/d38/d44/d41/d49/lee 0 2024-04-02T04:43:06.791 INFO:tasks.workunit.client.1.smithi027.stdout:1/750: dwrite d0/de/d52/d74/db7/df1/d2e/d3e/d5f/d8b/d9d/fad [0,4194304] 0 2024-04-02T04:43:06.798 INFO:tasks.workunit.client.1.smithi027.stdout:0/843: dread d4/d38/fa1 [0,4194304] 0 2024-04-02T04:43:06.799 INFO:tasks.workunit.client.1.smithi027.stdout:0/844: read d4/d16/d1b/fc2 [2287521,47582] 0 2024-04-02T04:43:06.802 INFO:tasks.workunit.client.1.smithi027.stdout:8/773: truncate d5/d8/db/dd/d5f/d27/f3a 3817592 0 2024-04-02T04:43:06.809 INFO:tasks.workunit.client.1.smithi027.stdout:7/796: rename d2/dba/dc/d1f/d33 to d2/d7/df3/d106 0 2024-04-02T04:43:06.815 INFO:tasks.workunit.client.1.smithi027.stdout:9/931: truncate d1a/d1d/d29/d37/d99/fbe 1926973 0 2024-04-02T04:43:06.815 INFO:tasks.workunit.client.1.smithi027.stdout:0/845: symlink d4/d21/d85/db3/l11c 0 2024-04-02T04:43:06.820 INFO:tasks.workunit.client.1.smithi027.stdout:2/869: unlink d1/db/d3b/d83/laa 0 2024-04-02T04:43:06.820 INFO:tasks.workunit.client.1.smithi027.stdout:3/739: dwrite d1/f45 [0,4194304] 0 2024-04-02T04:43:06.822 INFO:tasks.workunit.client.1.smithi027.stdout:2/870: chown d1/db/d3b/d61/d75/d100/l12d 309 1 2024-04-02T04:43:06.822 INFO:tasks.workunit.client.1.smithi027.stdout:7/797: creat d2/dba/dc/d16/d3f/d50/d8b/db6/f107 x:0 0 0 2024-04-02T04:43:06.823 INFO:tasks.workunit.client.1.smithi027.stdout:7/798: chown d2/l6c 181 1 2024-04-02T04:43:06.824 INFO:tasks.workunit.client.1.smithi027.stdout:2/871: chown d1/db/d3b/d17/f6a 45851 1 2024-04-02T04:43:06.824 INFO:tasks.workunit.client.1.smithi027.stdout:7/799: readlink d2/dba/le8 0 2024-04-02T04:43:06.824 INFO:tasks.workunit.client.1.smithi027.stdout:9/932: rmdir d1a/d1d/d29/d37/d20/dcc/dd5 39 2024-04-02T04:43:06.826 INFO:tasks.workunit.client.1.smithi027.stdout:7/800: chown d2/d1c/f7a 6 1 2024-04-02T04:43:06.827 INFO:tasks.workunit.client.1.smithi027.stdout:9/933: write d1a/d1d/d29/d51/d105/d8c/dcf/d12d/dfb/f10c [25125,38835] 0 2024-04-02T04:43:06.828 INFO:tasks.workunit.client.1.smithi027.stdout:5/911: write d0/df/d11/d13/dce/d55/f8a [226634,60341] 0 2024-04-02T04:43:06.838 INFO:tasks.workunit.client.1.smithi027.stdout:7/801: chown d2/dba/dc/d16/d3f/d50/d8b/d91/le9 1 1 2024-04-02T04:43:06.847 INFO:tasks.workunit.client.1.smithi027.stdout:8/774: link d5/d8/db/fca d5/d8/d15/d71/f100 0 2024-04-02T04:43:06.848 INFO:tasks.workunit.client.1.smithi027.stdout:3/740: creat d1/de/d55/de7/ff1 x:0 0 0 2024-04-02T04:43:06.850 INFO:tasks.workunit.client.1.smithi027.stdout:2/872: unlink d1/db/d3b/d17/d12/l48 0 2024-04-02T04:43:06.850 INFO:tasks.workunit.client.1.smithi027.stdout:2/873: chown d1/db/d3b/d61/d75/d81 28783544 1 2024-04-02T04:43:06.851 INFO:tasks.workunit.client.1.smithi027.stdout:7/802: mknod d2/dba/dc/d16/d3f/d44/c108 0 2024-04-02T04:43:06.854 INFO:tasks.workunit.client.1.smithi027.stdout:1/751: creat d0/de/d52/d74/db7/df1/d25/d3c/d64/dec/ff3 x:0 0 0 2024-04-02T04:43:06.855 INFO:tasks.workunit.client.1.smithi027.stdout:0/846: link d4/d38/d66/l26 d4/d38/d66/d2d/d81/l11d 0 2024-04-02T04:43:06.859 INFO:tasks.workunit.client.1.smithi027.stdout:9/934: dwrite d1a/d1d/d29/d37/d3a/f84 [0,4194304] 0 2024-04-02T04:43:06.861 INFO:tasks.workunit.client.1.smithi027.stdout:3/741: truncate d1/de/d55/d63/f91 2842089 0 2024-04-02T04:43:06.870 INFO:tasks.workunit.client.1.smithi027.stdout:1/752: symlink d0/de/d9e/lf4 0 2024-04-02T04:43:06.874 INFO:tasks.workunit.client.1.smithi027.stdout:9/935: rmdir d1a/d1d/d29/d50 39 2024-04-02T04:43:06.875 INFO:tasks.workunit.client.1.smithi027.stdout:9/936: readlink lf 0 2024-04-02T04:43:06.878 INFO:tasks.workunit.client.1.smithi027.stdout:9/937: chown d1a/d1d/d29/d37/d3a/fe6 1263049 1 2024-04-02T04:43:06.879 INFO:tasks.workunit.client.1.smithi027.stdout:8/775: dwrite d5/d8/db/d68/d95/fc5 [0,4194304] 0 2024-04-02T04:43:06.881 INFO:tasks.workunit.client.1.smithi027.stdout:1/753: creat d0/de/d52/d74/db7/df1/d2e/d3b/d4f/d9b/ff5 x:0 0 0 2024-04-02T04:43:06.892 INFO:tasks.workunit.client.1.smithi027.stdout:0/847: dwrite d4/d38/d66/f6c [0,4194304] 0 2024-04-02T04:43:06.894 INFO:tasks.workunit.client.1.smithi027.stdout:0/848: write d4/fec [798374,62976] 0 2024-04-02T04:43:06.901 INFO:tasks.workunit.client.1.smithi027.stdout:2/874: link d1/db/d3b/d17/d12/l20 d1/db/d3b/d17/d6d/dd9/dd5/dda/dbb/db4/l135 0 2024-04-02T04:43:06.906 INFO:tasks.workunit.client.1.smithi027.stdout:8/776: mknod d5/d8/d15/d71/db0/c101 0 2024-04-02T04:43:06.909 INFO:tasks.workunit.client.1.smithi027.stdout:7/803: getdents d2/dba/dc/d16/d2f 0 2024-04-02T04:43:06.910 INFO:tasks.workunit.client.1.smithi027.stdout:0/849: creat d4/d69/f11e x:0 0 0 2024-04-02T04:43:06.914 INFO:tasks.workunit.client.1.smithi027.stdout:9/938: dwrite d1a/d1d/d29/d56/d8b/fb4 [0,4194304] 0 2024-04-02T04:43:06.917 INFO:tasks.workunit.client.1.smithi027.stdout:0/850: dread d4/d38/ffd [0,4194304] 0 2024-04-02T04:43:06.918 INFO:tasks.workunit.client.1.smithi027.stdout:8/777: rename d5/l16 to d5/d8/db/dd/d5f/d27/d31/d5b/dc4/l102 0 2024-04-02T04:43:06.919 INFO:tasks.workunit.client.1.smithi027.stdout:0/851: chown d4/d21/d28/d4e/d5f/f79 250 1 2024-04-02T04:43:06.919 INFO:tasks.workunit.client.1.smithi027.stdout:1/754: dread d0/f37 [0,4194304] 0 2024-04-02T04:43:06.921 INFO:tasks.workunit.client.1.smithi027.stdout:1/755: stat d0/de/d52/d74/db7/df1/d2e/d3e/d5f/fb6 0 2024-04-02T04:43:06.941 INFO:tasks.workunit.client.1.smithi027.stdout:3/742: getdents d1/de/df/d13 0 2024-04-02T04:43:06.942 INFO:tasks.workunit.client.1.smithi027.stdout:2/875: getdents d1/db/d3b/d17/d6d/dd9/dd5/dda/dbb/db4/d12b 0 2024-04-02T04:43:06.947 INFO:tasks.workunit.client.1.smithi027.stdout:8/778: symlink d5/d8/db/df/l103 0 2024-04-02T04:43:06.956 INFO:tasks.workunit.client.1.smithi027.stdout:7/804: symlink d2/df/d89/dff/l109 0 2024-04-02T04:43:07.020 INFO:tasks.workunit.client.1.smithi027.stdout:1/756: write d0/fb [2383759,89293] 0 2024-04-02T04:43:07.020 INFO:tasks.workunit.client.1.smithi027.stdout:8/779: mknod d5/d8/db/dd/d5f/d27/d31/db9/dc1/c104 0 2024-04-02T04:43:07.020 INFO:tasks.workunit.client.1.smithi027.stdout:2/876: write d1/db/d3b/d17/d12/f62 [3368994,111922] 0 2024-04-02T04:43:07.021 INFO:tasks.workunit.client.1.smithi027.stdout:9/939: rename d1a/d1d/d29/d51/dfa/d133/ff2 to d1a/d1d/f134 0 2024-04-02T04:43:07.023 INFO:tasks.workunit.client.1.smithi027.stdout:3/743: link d1/de/df/d13/f36 d1/de/df/ff2 0 2024-04-02T04:43:07.025 INFO:tasks.workunit.client.1.smithi027.stdout:1/757: fdatasync d0/de/d52/d74/d97/fc2 0 2024-04-02T04:43:07.025 INFO:tasks.workunit.client.1.smithi027.stdout:8/780: chown d5/d8/db/dd/d4a/cbb 448 1 2024-04-02T04:43:07.027 INFO:tasks.workunit.client.1.smithi027.stdout:0/852: getdents d4/d38/d66 0 2024-04-02T04:43:07.030 INFO:tasks.workunit.client.1.smithi027.stdout:7/805: dwrite d2/dba/dc/d16/f2c [4194304,4194304] 0 2024-04-02T04:43:07.030 INFO:tasks.workunit.client.1.smithi027.stdout:1/758: readlink d0/de/d52/d74/db7/df1/d2e/l66 0 2024-04-02T04:43:07.031 INFO:tasks.workunit.client.1.smithi027.stdout:9/940: truncate d1a/d1d/d29/d51/d105/d8c/dcf/d12d/fba 536511 0 2024-04-02T04:43:07.037 INFO:tasks.workunit.client.1.smithi027.stdout:0/853: rename d4/d21/d85/dfb/l11b to d4/de0/l11f 0 2024-04-02T04:43:07.038 INFO:tasks.workunit.client.1.smithi027.stdout:7/806: creat d2/dba/dc/d16/d2f/dcd/f10a x:0 0 0 2024-04-02T04:43:07.038 INFO:tasks.workunit.client.1.smithi027.stdout:7/807: chown d2/df/d89/dde 172164 1 2024-04-02T04:43:07.043 INFO:tasks.workunit.client.1.smithi027.stdout:0/854: symlink d4/d16/d71/l120 0 2024-04-02T04:43:07.046 INFO:tasks.workunit.client.1.smithi027.stdout:0/855: read - d4/d21/d28/d4e/db5/f119 zero size 2024-04-02T04:43:07.046 INFO:tasks.workunit.client.1.smithi027.stdout:7/808: creat d2/d1d/f10b x:0 0 0 2024-04-02T04:43:07.046 INFO:tasks.workunit.client.1.smithi027.stdout:0/856: readlink d4/d21/d28/lbe 0 2024-04-02T04:43:07.046 INFO:tasks.workunit.client.1.smithi027.stdout:0/857: dread - d4/d69/f115 zero size 2024-04-02T04:43:07.046 INFO:tasks.workunit.client.1.smithi027.stdout:0/858: fdatasync d4/d38/d66/d4a/f42 0 2024-04-02T04:43:07.046 INFO:tasks.workunit.client.1.smithi027.stdout:3/744: write d1/de/df/f67 [599610,1699] 0 2024-04-02T04:43:07.050 INFO:tasks.workunit.client.1.smithi027.stdout:1/759: mknod d0/de/d52/d74/db7/df1/d25/d2b/d59/cf6 0 2024-04-02T04:43:07.053 INFO:tasks.workunit.client.1.smithi027.stdout:8/781: dwrite d5/d8/db/dd/d5f/db2/fe5 [0,4194304] 0 2024-04-02T04:43:07.061 INFO:tasks.workunit.client.1.smithi027.stdout:8/782: dwrite d5/f7d [0,4194304] 0 2024-04-02T04:43:07.062 INFO:tasks.workunit.client.1.smithi027.stdout:6/721: sync 2024-04-02T04:43:07.063 INFO:tasks.workunit.client.1.smithi027.stdout:8/783: dread - d5/d8/db/dd/d4a/dc9/feb zero size 2024-04-02T04:43:07.066 INFO:tasks.workunit.client.1.smithi027.stdout:7/809: write d2/d1d/f80 [913408,62608] 0 2024-04-02T04:43:07.069 INFO:tasks.workunit.client.1.smithi027.stdout:9/941: dwrite d1a/d1d/d29/d50/d73/dbd/fbf [0,4194304] 0 2024-04-02T04:43:07.070 INFO:tasks.workunit.client.1.smithi027.stdout:3/745: mknod d1/de/df/d9e/dc6/de4/cf3 0 2024-04-02T04:43:07.071 INFO:tasks.workunit.client.1.smithi027.stdout:7/810: dread - d2/dba/dc/d16/d3f/d79/fec zero size 2024-04-02T04:43:07.073 INFO:tasks.workunit.client.1.smithi027.stdout:9/942: dread - d1a/d1d/d29/d37/d3a/fe6 zero size 2024-04-02T04:43:07.074 INFO:tasks.workunit.client.1.smithi027.stdout:9/943: chown d1a/d1d/f104 13560 1 2024-04-02T04:43:07.082 INFO:tasks.workunit.client.1.smithi027.stdout:1/760: dwrite d0/de/d52/d74/d97/fea [0,4194304] 0 2024-04-02T04:43:07.085 INFO:tasks.workunit.client.1.smithi027.stdout:8/784: dread d5/d8/d15/d71/f100 [0,4194304] 0 2024-04-02T04:43:07.087 INFO:tasks.workunit.client.1.smithi027.stdout:1/761: truncate d0/de/d52/d74/db7/df1/d2e/d3b/d4f/d9b/ff5 632804 0 2024-04-02T04:43:07.089 INFO:tasks.workunit.client.1.smithi027.stdout:3/746: symlink d1/de/df/d13/lf4 0 2024-04-02T04:43:07.091 INFO:tasks.workunit.client.1.smithi027.stdout:7/811: unlink d2/d7/df3/d106/d36/cbc 0 2024-04-02T04:43:07.097 INFO:tasks.workunit.client.1.smithi027.stdout:0/859: truncate d4/d16/d71/fdf 1684414 0 2024-04-02T04:43:07.098 INFO:tasks.workunit.client.1.smithi027.stdout:0/860: readlink d4/d21/d85/db3/l11c 0 2024-04-02T04:43:07.102 INFO:tasks.workunit.client.1.smithi027.stdout:3/747: creat d1/de/d74/ff5 x:0 0 0 2024-04-02T04:43:07.105 INFO:tasks.workunit.client.1.smithi027.stdout:7/812: unlink d2/dba/dc/d16/d2f/c54 0 2024-04-02T04:43:07.107 INFO:tasks.workunit.client.1.smithi027.stdout:9/944: write d1a/d1d/d29/f46 [807438,8051] 0 2024-04-02T04:43:07.112 INFO:tasks.workunit.client.1.smithi027.stdout:1/762: write d0/f7 [7955405,118760] 0 2024-04-02T04:43:07.117 INFO:tasks.workunit.client.1.smithi027.stdout:0/861: dread d4/d16/d1b/f20 [0,4194304] 0 2024-04-02T04:43:07.118 INFO:tasks.workunit.client.1.smithi027.stdout:6/722: dwrite d13/d1b/f1e [0,4194304] 0 2024-04-02T04:43:07.121 INFO:tasks.workunit.client.1.smithi027.stdout:8/785: rename d5/d8/db/d68/f9b to d5/d8/db/dd/d5f/d27/f105 0 2024-04-02T04:43:07.121 INFO:tasks.workunit.client.1.smithi027.stdout:7/813: dwrite d2/dba/dc/d16/d3f/d50/d8b/db6/fd6 [0,4194304] 0 2024-04-02T04:43:07.125 INFO:tasks.workunit.client.1.smithi027.stdout:7/814: fsync d2/dba/dc/d16/d3a/d4d/dcb/f100 0 2024-04-02T04:43:07.126 INFO:tasks.workunit.client.1.smithi027.stdout:3/748: mknod d1/de/df/cf6 0 2024-04-02T04:43:07.135 INFO:tasks.workunit.client.1.smithi027.stdout:1/763: rename d0/de/d52/d74/db7/df1/d2e to d0/de/d52/d74/df7 0 2024-04-02T04:43:07.138 INFO:tasks.workunit.client.1.smithi027.stdout:8/786: mkdir d5/d8/db/dd/d5f/db2/dc6/d106 0 2024-04-02T04:43:07.146 INFO:tasks.workunit.client.1.smithi027.stdout:9/945: dread fa [0,4194304] 0 2024-04-02T04:43:07.147 INFO:tasks.workunit.client.1.smithi027.stdout:8/787: dwrite d5/d8/db/dd/d5f/db2/fe5 [0,4194304] 0 2024-04-02T04:43:07.149 INFO:tasks.workunit.client.1.smithi027.stdout:8/788: write d5/d8/db/d57/fa1 [895836,68487] 0 2024-04-02T04:43:07.153 INFO:tasks.workunit.client.1.smithi027.stdout:0/862: fsync d4/d21/d28/f54 0 2024-04-02T04:43:07.158 INFO:tasks.workunit.client.1.smithi027.stdout:6/723: getdents d13 0 2024-04-02T04:43:07.158 INFO:tasks.workunit.client.1.smithi027.stdout:0/863: write d4/d21/d28/d4e/db5/d118/dfe/fb4 [631631,119949] 0 2024-04-02T04:43:07.159 INFO:tasks.workunit.client.1.smithi027.stdout:7/815: dwrite d2/dba/d35/f60 [0,4194304] 0 2024-04-02T04:43:07.164 INFO:tasks.workunit.client.1.smithi027.stdout:3/749: truncate d1/de/df/f17 2822386 0 2024-04-02T04:43:07.165 INFO:tasks.workunit.client.1.smithi027.stdout:8/789: write d5/d8/f2e [4769267,52601] 0 2024-04-02T04:43:07.170 INFO:tasks.workunit.client.1.smithi027.stdout:6/724: truncate d13/d8d/fa4 64554 0 2024-04-02T04:43:07.171 INFO:tasks.workunit.client.1.smithi027.stdout:9/946: getdents d1a/d1d/d29/d50/d73/dbd 0 2024-04-02T04:43:07.173 INFO:tasks.workunit.client.1.smithi027.stdout:9/947: fdatasync d1a/d1d/d29/d56/ff8 0 2024-04-02T04:43:07.174 INFO:tasks.workunit.client.1.smithi027.stdout:1/764: dwrite d0/de/d52/d74/db7/fc6 [0,4194304] 0 2024-04-02T04:43:07.176 INFO:tasks.workunit.client.1.smithi027.stdout:6/725: chown d13/d1b/d48/d58/dd3/fe6 207309 1 2024-04-02T04:43:07.180 INFO:tasks.workunit.client.1.smithi027.stdout:8/790: getdents d5/d8/db/dd/d4a/dc9 0 2024-04-02T04:43:07.182 INFO:tasks.workunit.client.1.smithi027.stdout:6/726: write d13/d1b/d24/d25/d56/d73/d96/fb2 [1526226,52566] 0 2024-04-02T04:43:07.184 INFO:tasks.workunit.client.1.smithi027.stdout:7/816: dread d2/d7/df3/d106/d36/f102 [0,4194304] 0 2024-04-02T04:43:07.186 INFO:tasks.workunit.client.1.smithi027.stdout:1/765: dread - d0/de/d52/d74/db7/df1/d5d/f71 zero size 2024-04-02T04:43:07.187 INFO:tasks.workunit.client.1.smithi027.stdout:9/948: mkdir d1a/d1d/d135 0 2024-04-02T04:43:07.190 INFO:tasks.workunit.client.1.smithi027.stdout:8/791: mknod d5/db5/ddc/c107 0 2024-04-02T04:43:07.192 INFO:tasks.workunit.client.1.smithi027.stdout:6/727: rmdir d13/d1b/d24/d25/d42 39 2024-04-02T04:43:07.194 INFO:tasks.workunit.client.1.smithi027.stdout:0/864: dwrite d4/d16/dd5/f47 [0,4194304] 0 2024-04-02T04:43:07.197 INFO:tasks.workunit.client.1.smithi027.stdout:0/865: dread - d4/d21/d28/d4e/db5/d118/dfe/d117/f10e zero size 2024-04-02T04:43:07.198 INFO:tasks.workunit.client.1.smithi027.stdout:1/766: mkdir d0/de/d52/d74/df7/d3e/df8 0 2024-04-02T04:43:07.201 INFO:tasks.workunit.client.1.smithi027.stdout:4/726: sync 2024-04-02T04:43:07.201 INFO:tasks.workunit.client.1.smithi027.stdout:1/767: write d0/de/d52/d74/db7/df1/d5d/fa1 [3274874,4218] 0 2024-04-02T04:43:07.202 INFO:tasks.workunit.client.1.smithi027.stdout:9/949: rename d1a/dce to d1a/d1d/d29/d51/d105/d8c/dcf/d12d/dde/d136 0 2024-04-02T04:43:07.203 INFO:tasks.workunit.client.1.smithi027.stdout:6/728: dwrite d13/d1b/d24/d25/d56/ff0 [0,4194304] 0 2024-04-02T04:43:07.205 INFO:tasks.workunit.client.1.smithi027.stdout:1/768: fdatasync d0/de/d52/d74/db7/df1/f9f 0 2024-04-02T04:43:07.205 INFO:tasks.workunit.client.1.smithi027.stdout:9/950: write d1a/d1d/d3c/d6e/f97 [4039861,72332] 0 2024-04-02T04:43:07.206 INFO:tasks.workunit.client.1.smithi027.stdout:7/817: creat d2/dba/dc/d16/d3a/d4d/df4/f10c x:0 0 0 2024-04-02T04:43:07.210 INFO:tasks.workunit.client.1.smithi027.stdout:7/818: chown d2/dba/dc/d16/d2f/de5 60402176 1 2024-04-02T04:43:07.210 INFO:tasks.workunit.client.1.smithi027.stdout:9/951: read d1a/d1d/d29/d37/d20/f30 [577533,76471] 0 2024-04-02T04:43:07.210 INFO:tasks.workunit.client.1.smithi027.stdout:7/819: readlink d2/d7/df3/d106/d36/lf2 0 2024-04-02T04:43:07.223 INFO:tasks.workunit.client.1.smithi027.stdout:9/952: dwrite d1a/d1d/d29/d51/d105/d8c/dcf/d12d/dfb/f10c [0,4194304] 0 2024-04-02T04:43:07.224 INFO:tasks.workunit.client.1.smithi027.stdout:1/769: mknod d0/de/d52/d74/df7/d3e/d5f/d8b/d9d/cf9 0 2024-04-02T04:43:07.224 INFO:tasks.workunit.client.1.smithi027.stdout:1/770: stat d0/de/d52/d76 0 2024-04-02T04:43:07.230 INFO:tasks.workunit.client.1.smithi027.stdout:6/729: dread d13/d1b/d48/d75/f83 [0,4194304] 0 2024-04-02T04:43:07.232 INFO:tasks.workunit.client.1.smithi027.stdout:4/727: symlink d24/d38/d47/d62/lef 0 2024-04-02T04:43:07.233 INFO:tasks.workunit.client.1.smithi027.stdout:7/820: truncate d2/dba/dc/d16/d3a/d4d/f9d 679505 0 2024-04-02T04:43:07.236 INFO:tasks.workunit.client.1.smithi027.stdout:8/792: link d5/d8/db/dd/d5f/d27/d31/d5b/c78 d5/d8/db/dd/d5f/d27/d31/db9/c108 0 2024-04-02T04:43:07.238 INFO:tasks.workunit.client.1.smithi027.stdout:1/771: unlink d0/de/d52/d74/df7/d3e/d5f/fca 0 2024-04-02T04:43:07.241 INFO:tasks.workunit.client.1.smithi027.stdout:9/953: dread d1a/d1d/d29/d37/d20/f30 [0,4194304] 0 2024-04-02T04:43:07.243 INFO:tasks.workunit.client.1.smithi027.stdout:8/793: dread d5/d8/db/dd/d5f/d27/d31/d5b/dc4/f8b [0,4194304] 0 2024-04-02T04:43:07.244 INFO:tasks.workunit.client.1.smithi027.stdout:1/772: rmdir d0/de/d52/d74/df7/d3e/d5f/d8b/d9d 39 2024-04-02T04:43:07.245 INFO:tasks.workunit.client.1.smithi027.stdout:8/794: dread - d5/d8/db/dd/d5f/d27/d31/db9/dc1/fe1 zero size 2024-04-02T04:43:07.245 INFO:tasks.workunit.client.1.smithi027.stdout:1/773: fdatasync d0/de/d52/d74/d97/fea 0 2024-04-02T04:43:07.246 INFO:tasks.workunit.client.1.smithi027.stdout:6/730: rename d13/d1b/d24/f70 to d13/d1b/d24/d25/d56/d6d/daa/ff6 0 2024-04-02T04:43:07.246 INFO:tasks.workunit.client.1.smithi027.stdout:8/795: stat d5/d8/db/dd/d4a 0 2024-04-02T04:43:07.251 INFO:tasks.workunit.client.1.smithi027.stdout:9/954: chown d1a/d1d/d29/d37/d20/dcc/dd5/de2/d126 37 1 2024-04-02T04:43:07.252 INFO:tasks.workunit.client.1.smithi027.stdout:0/866: link d4/d76/f99 d4/d16/f121 0 2024-04-02T04:43:07.263 INFO:tasks.workunit.client.1.smithi027.stdout:7/821: creat d2/dba/dc/f10d x:0 0 0 2024-04-02T04:43:07.267 INFO:tasks.workunit.client.1.smithi027.stdout:6/731: rename d13/d1b/d24/d25/d56/d73 to d13/d1b/d48/d9c/df7 0 2024-04-02T04:43:07.271 INFO:tasks.workunit.client.1.smithi027.stdout:8/796: dread d5/d8/d15/d71/d96/fcc [0,4194304] 0 2024-04-02T04:43:07.273 INFO:tasks.workunit.client.1.smithi027.stdout:9/955: unlink d1a/d1d/d29/d37/d3a/ff0 0 2024-04-02T04:43:07.277 INFO:tasks.workunit.client.1.smithi027.stdout:0/867: dread - d4/d21/d85/fae zero size 2024-04-02T04:43:07.278 INFO:tasks.workunit.client.1.smithi027.stdout:1/774: mkdir d0/de/d52/d74/d97/dfa 0 2024-04-02T04:43:07.279 INFO:tasks.workunit.client.1.smithi027.stdout:9/956: dwrite f15 [4194304,4194304] 0 2024-04-02T04:43:07.284 INFO:tasks.workunit.client.1.smithi027.stdout:4/728: dwrite d24/d38/d44/d82/f96 [0,4194304] 0 2024-04-02T04:43:07.284 INFO:tasks.workunit.client.1.smithi027.stdout:6/732: mkdir d13/d1b/d48/d58/dd3/df8 0 2024-04-02T04:43:07.288 INFO:tasks.workunit.client.1.smithi027.stdout:8/797: creat d5/d8/db/dd/d4a/dc9/f109 x:0 0 0 2024-04-02T04:43:07.290 INFO:tasks.workunit.client.1.smithi027.stdout:8/798: write d5/d8/db/dd/d5f/d27/f50 [554357,52083] 0 2024-04-02T04:43:07.291 INFO:tasks.workunit.client.1.smithi027.stdout:0/868: mkdir d4/d69/d122 0 2024-04-02T04:43:07.293 INFO:tasks.workunit.client.1.smithi027.stdout:7/822: mknod d2/d1d/dd7/c10e 0 2024-04-02T04:43:07.302 INFO:tasks.workunit.client.1.smithi027.stdout:0/869: dwrite d4/d21/fb2 [0,4194304] 0 2024-04-02T04:43:07.302 INFO:tasks.workunit.client.1.smithi027.stdout:4/729: dread d24/d38/d44/d41/f87 [0,4194304] 0 2024-04-02T04:43:07.304 INFO:tasks.workunit.client.1.smithi027.stdout:6/733: mknod d13/d1b/d24/d28/d46/cf9 0 2024-04-02T04:43:07.305 INFO:tasks.workunit.client.1.smithi027.stdout:6/734: truncate d13/d1b/d24/d25/d56/fa0 4777365 0 2024-04-02T04:43:07.308 INFO:tasks.workunit.client.1.smithi027.stdout:8/799: unlink d5/d8/db/dd/d5f/d27/d31/d5b/dc4/fef 0 2024-04-02T04:43:07.314 INFO:tasks.workunit.client.1.smithi027.stdout:7/823: fsync d2/df/f85 0 2024-04-02T04:43:07.319 INFO:tasks.workunit.client.1.smithi027.stdout:1/775: link d0/de/d52/d74/df7/d3b/d4f/d9b/ff0 d0/de/d52/d74/dd9/ffb 0 2024-04-02T04:43:07.320 INFO:tasks.workunit.client.1.smithi027.stdout:0/870: symlink d4/d76/l123 0 2024-04-02T04:43:07.322 INFO:tasks.workunit.client.1.smithi027.stdout:0/871: write d4/d38/d66/d2d/f3c [584811,128326] 0 2024-04-02T04:43:07.326 INFO:tasks.workunit.client.1.smithi027.stdout:0/872: read d4/d38/d66/f2a [3489948,13412] 0 2024-04-02T04:43:07.326 INFO:tasks.workunit.client.1.smithi027.stdout:9/957: dwrite d1a/d1d/d29/d37/d3a/fa3 [0,4194304] 0 2024-04-02T04:43:07.327 INFO:tasks.workunit.client.1.smithi027.stdout:2/877: sync 2024-04-02T04:43:07.328 INFO:tasks.workunit.client.1.smithi027.stdout:5/912: sync 2024-04-02T04:43:07.328 INFO:tasks.workunit.client.1.smithi027.stdout:2/878: chown d1/db/c15 232821 1 2024-04-02T04:43:07.329 INFO:tasks.workunit.client.1.smithi027.stdout:9/958: dread - d1a/d1d/d29/d37/d3a/fc8 zero size 2024-04-02T04:43:07.331 INFO:tasks.workunit.client.1.smithi027.stdout:5/913: write d0/df/d11/d37/f113 [503300,98813] 0 2024-04-02T04:43:07.336 INFO:tasks.workunit.client.1.smithi027.stdout:7/824: rename d2/dba/dc/d16/d3f/d50/d8b/d91/fb1 to d2/df/dc7/f10f 0 2024-04-02T04:43:07.338 INFO:tasks.workunit.client.1.smithi027.stdout:7/825: write d2/d1d/f80 [1028245,78000] 0 2024-04-02T04:43:07.345 INFO:tasks.workunit.client.1.smithi027.stdout:4/730: write d24/f39 [135283,36323] 0 2024-04-02T04:43:07.349 INFO:tasks.workunit.client.1.smithi027.stdout:8/800: symlink d5/d8/db/dd/d5f/d27/d31/d5b/l10a 0 2024-04-02T04:43:07.362 INFO:tasks.workunit.client.1.smithi027.stdout:2/879: creat d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/de1/f136 x:0 0 0 2024-04-02T04:43:07.365 INFO:tasks.workunit.client.1.smithi027.stdout:5/914: unlink d0/df/d11/c50 0 2024-04-02T04:43:07.368 INFO:tasks.workunit.client.1.smithi027.stdout:1/776: mkdir d0/de/d52/d74/db7/df1/d25/dfc 0 2024-04-02T04:43:07.370 INFO:tasks.workunit.client.1.smithi027.stdout:7/826: unlink d2/dba/d35/lf7 0 2024-04-02T04:43:07.371 INFO:tasks.workunit.client.1.smithi027.stdout:6/735: creat d13/d1b/d24/d25/ffa x:0 0 0 2024-04-02T04:43:07.374 INFO:tasks.workunit.client.1.smithi027.stdout:9/959: mknod d1a/d1d/d135/c137 0 2024-04-02T04:43:07.376 INFO:tasks.workunit.client.1.smithi027.stdout:2/880: mkdir d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/de1/d137 0 2024-04-02T04:43:07.377 INFO:tasks.workunit.client.1.smithi027.stdout:5/915: read d0/d1a/d20/f73 [286625,114665] 0 2024-04-02T04:43:07.383 INFO:tasks.workunit.client.1.smithi027.stdout:9/960: creat d1a/d1d/d29/d51/d105/d8c/dcf/f138 x:0 0 0 2024-04-02T04:43:07.386 INFO:tasks.workunit.client.1.smithi027.stdout:1/777: creat d0/de/d52/d74/db7/df1/d25/d2b/d59/ffd x:0 0 0 2024-04-02T04:43:07.390 INFO:tasks.workunit.client.1.smithi027.stdout:6/736: creat d13/d1b/d24/d25/ded/ffb x:0 0 0 2024-04-02T04:43:07.391 INFO:tasks.workunit.client.1.smithi027.stdout:2/881: dwrite d1/db/d3b/d17/f4a [0,4194304] 0 2024-04-02T04:43:07.392 INFO:tasks.workunit.client.1.smithi027.stdout:6/737: write d13/d1b/f8e [1662980,52999] 0 2024-04-02T04:43:07.403 INFO:tasks.workunit.client.1.smithi027.stdout:5/916: rename d0/df/d11/d13/dce/f63 to d0/df/d11/d13/dce/f130 0 2024-04-02T04:43:07.406 INFO:tasks.workunit.client.1.smithi027.stdout:2/882: mknod d1/db/d3b/d17/d6d/dd9/c138 0 2024-04-02T04:43:07.408 INFO:tasks.workunit.client.1.smithi027.stdout:9/961: dread d1a/d1d/d29/d37/d3a/f54 [0,4194304] 0 2024-04-02T04:43:07.411 INFO:tasks.workunit.client.1.smithi027.stdout:5/917: truncate d0/d68/d6b/d74/fb6 4493576 0 2024-04-02T04:43:07.414 INFO:tasks.workunit.client.1.smithi027.stdout:9/962: unlink d1a/d1d/c8e 0 2024-04-02T04:43:07.419 INFO:tasks.workunit.client.1.smithi027.stdout:0/873: truncate d4/d16/dd5/f47 3034419 0 2024-04-02T04:43:07.419 INFO:tasks.workunit.client.1.smithi027.stdout:6/738: getdents d13/d1b/d24/d25/d56/d6d/daa 0 2024-04-02T04:43:07.421 INFO:tasks.workunit.client.1.smithi027.stdout:6/739: dread - d13/d1b/d24/d25/ded/ffb zero size 2024-04-02T04:43:07.426 INFO:tasks.workunit.client.1.smithi027.stdout:5/918: symlink d0/d68/l131 0 2024-04-02T04:43:07.434 INFO:tasks.workunit.client.1.smithi027.stdout:9/963: symlink d1a/d1d/d29/d37/l139 0 2024-04-02T04:43:07.436 INFO:tasks.workunit.client.1.smithi027.stdout:9/964: dread d1a/d1d/d29/d37/d20/f30 [0,4194304] 0 2024-04-02T04:43:07.438 INFO:tasks.workunit.client.1.smithi027.stdout:6/740: truncate d13/d1b/d24/f84 114120 0 2024-04-02T04:43:07.439 INFO:tasks.workunit.client.1.smithi027.stdout:9/965: truncate d1a/d1d/d29/d37/d3a/fe6 49680 0 2024-04-02T04:43:07.441 INFO:tasks.workunit.client.1.smithi027.stdout:5/919: symlink d0/df/d11/d13/d9a/d9c/dbe/l132 0 2024-04-02T04:43:07.443 INFO:tasks.workunit.client.1.smithi027.stdout:5/920: read d0/d1d/d61/f62 [169610,88091] 0 2024-04-02T04:43:07.449 INFO:tasks.workunit.client.1.smithi027.stdout:6/741: rename d13/d1b/l2f to d13/d1b/d48/d75/d9b/lfc 0 2024-04-02T04:43:07.451 INFO:tasks.workunit.client.1.smithi027.stdout:8/801: dwrite d5/d8/db/dd/d5f/f14 [4194304,4194304] 0 2024-04-02T04:43:07.457 INFO:tasks.workunit.client.1.smithi027.stdout:5/921: creat d0/d1a/f133 x:0 0 0 2024-04-02T04:43:07.460 INFO:tasks.workunit.client.1.smithi027.stdout:8/802: readlink l1 0 2024-04-02T04:43:07.464 INFO:tasks.workunit.client.1.smithi027.stdout:5/922: mkdir d0/d1d/d61/d72/d134 0 2024-04-02T04:43:07.468 INFO:tasks.workunit.client.1.smithi027.stdout:7/827: dwrite d2/dba/dc/d16/d3a/d4d/f9d [0,4194304] 0 2024-04-02T04:43:07.473 INFO:tasks.workunit.client.1.smithi027.stdout:7/828: write d2/dba/dc/d16/d3f/d50/f66 [819859,68537] 0 2024-04-02T04:43:07.474 INFO:tasks.workunit.client.1.smithi027.stdout:5/923: dwrite d0/d1d/d61/f62 [4194304,4194304] 0 2024-04-02T04:43:07.477 INFO:tasks.workunit.client.1.smithi027.stdout:5/924: readlink d0/df/d11/d13/lba 0 2024-04-02T04:43:07.480 INFO:tasks.workunit.client.1.smithi027.stdout:7/829: fdatasync d2/dba/dc/f30 0 2024-04-02T04:43:07.482 INFO:tasks.workunit.client.1.smithi027.stdout:3/750: sync 2024-04-02T04:43:07.483 INFO:tasks.workunit.client.1.smithi027.stdout:3/751: dread - d1/de/fa7 zero size 2024-04-02T04:43:07.507 INFO:tasks.workunit.client.1.smithi027.stdout:0/874: write d4/d38/d66/f2a [5484422,47820] 0 2024-04-02T04:43:07.508 INFO:tasks.workunit.client.1.smithi027.stdout:0/875: truncate d4/d69/f11e 482692 0 2024-04-02T04:43:07.514 INFO:tasks.workunit.client.1.smithi027.stdout:0/876: dwrite d4/d38/f91 [0,4194304] 0 2024-04-02T04:43:07.514 INFO:tasks.workunit.client.1.smithi027.stdout:9/966: truncate d1a/f38 3685580 0 2024-04-02T04:43:07.517 INFO:tasks.workunit.client.1.smithi027.stdout:0/877: read - d4/d16/dd5/fd6 zero size 2024-04-02T04:43:07.518 INFO:tasks.workunit.client.1.smithi027.stdout:6/742: dwrite f12 [0,4194304] 0 2024-04-02T04:43:07.519 INFO:tasks.workunit.client.1.smithi027.stdout:0/878: chown d4/d38/d66/d2d/d4c/c6e 6532688 1 2024-04-02T04:43:07.521 INFO:tasks.workunit.client.1.smithi027.stdout:6/743: chown d13/d1b/d48/d9c/df7/fa1 370053 1 2024-04-02T04:43:07.524 INFO:tasks.workunit.client.1.smithi027.stdout:0/879: write d4/d21/d28/d4e/db5/f119 [641344,80940] 0 2024-04-02T04:43:07.528 INFO:tasks.workunit.client.1.smithi027.stdout:9/967: rename d1a/d1d/d29/d51/d105/d8c/de1/d100 to d1a/d1d/d29/d51/d105/d13a 0 2024-04-02T04:43:07.530 INFO:tasks.workunit.client.1.smithi027.stdout:3/752: write d1/de/f71 [3753727,61545] 0 2024-04-02T04:43:07.531 INFO:tasks.workunit.client.1.smithi027.stdout:7/830: write d2/d7/f18 [1038652,13563] 0 2024-04-02T04:43:07.532 INFO:tasks.workunit.client.1.smithi027.stdout:5/925: dwrite d0/d68/fa4 [0,4194304] 0 2024-04-02T04:43:07.536 INFO:tasks.workunit.client.1.smithi027.stdout:6/744: dwrite d13/d1b/d48/d75/ff4 [0,4194304] 0 2024-04-02T04:43:07.541 INFO:tasks.workunit.client.1.smithi027.stdout:5/926: write d0/d1a/d20/fde [4086165,44905] 0 2024-04-02T04:43:07.546 INFO:tasks.workunit.client.1.smithi027.stdout:9/968: mkdir d1a/d1d/d3c/d6e/db8/d13b 0 2024-04-02T04:43:07.547 INFO:tasks.workunit.client.1.smithi027.stdout:5/927: write d0/d1d/d61/f62 [7559365,39177] 0 2024-04-02T04:43:07.550 INFO:tasks.workunit.client.1.smithi027.stdout:3/753: dwrite d1/d1a/da6/fc8 [0,4194304] 0 2024-04-02T04:43:07.562 INFO:tasks.workunit.client.1.smithi027.stdout:7/831: dread d2/dba/dc/faf [0,4194304] 0 2024-04-02T04:43:07.567 INFO:tasks.workunit.client.1.smithi027.stdout:9/969: dwrite d1a/d1d/d29/d51/dfa/d133/fdc [0,4194304] 0 2024-04-02T04:43:07.569 INFO:tasks.workunit.client.1.smithi027.stdout:3/754: dwrite d1/de/df/d13/f21 [0,4194304] 0 2024-04-02T04:43:07.571 INFO:tasks.workunit.client.1.smithi027.stdout:7/832: write d2/dba/dc/d16/f4a [2776851,21480] 0 2024-04-02T04:43:07.573 INFO:tasks.workunit.client.1.smithi027.stdout:0/880: dwrite d4/d38/fb6 [0,4194304] 0 2024-04-02T04:43:07.577 INFO:tasks.workunit.client.1.smithi027.stdout:5/928: unlink d0/df/d2f/f11d 0 2024-04-02T04:43:07.586 INFO:tasks.workunit.client.1.smithi027.stdout:3/755: creat d1/d1a/ff7 x:0 0 0 2024-04-02T04:43:07.589 INFO:tasks.workunit.client.1.smithi027.stdout:9/970: chown d1a/d1d/d29/d56/de4/f10f 347779 1 2024-04-02T04:43:07.589 INFO:tasks.workunit.client.1.smithi027.stdout:0/881: read - d4/d16/d1b/fc7 zero size 2024-04-02T04:43:07.594 INFO:tasks.workunit.client.1.smithi027.stdout:6/745: creat d13/d1b/d48/ffd x:0 0 0 2024-04-02T04:43:07.597 INFO:tasks.workunit.client.1.smithi027.stdout:5/929: symlink d0/df/d29/da5/l135 0 2024-04-02T04:43:07.598 INFO:tasks.workunit.client.1.smithi027.stdout:5/930: readlink d0/df/d76/l83 0 2024-04-02T04:43:07.602 INFO:tasks.workunit.client.1.smithi027.stdout:9/971: creat d1a/d1d/d29/d37/d20/dcc/dd5/de2/d126/f13c x:0 0 0 2024-04-02T04:43:07.604 INFO:tasks.workunit.client.1.smithi027.stdout:9/972: dread d1a/d1d/d29/d50/d73/ffe [0,4194304] 0 2024-04-02T04:43:07.606 INFO:tasks.workunit.client.1.smithi027.stdout:7/833: truncate d2/d1c/f40 2751612 0 2024-04-02T04:43:07.609 INFO:tasks.workunit.client.1.smithi027.stdout:9/973: dread d1a/d1d/d29/d37/d4c/f62 [0,4194304] 0 2024-04-02T04:43:07.609 INFO:tasks.workunit.client.1.smithi027.stdout:0/882: truncate d4/d38/d66/f2a 2766002 0 2024-04-02T04:43:07.612 INFO:tasks.workunit.client.1.smithi027.stdout:3/756: rename d1/de/df/d9e/dc6/fd4 to d1/de/df/d13/d73/d77/ff8 0 2024-04-02T04:43:07.615 INFO:tasks.workunit.client.1.smithi027.stdout:5/931: write d0/d1d/d61/d7e/faf [1817724,68282] 0 2024-04-02T04:43:07.619 INFO:tasks.workunit.client.1.smithi027.stdout:7/834: dread - d2/dba/dc/d16/d3f/d50/d8b/f8f zero size 2024-04-02T04:43:07.620 INFO:tasks.workunit.client.1.smithi027.stdout:7/835: chown d2/df/d89/faa 1085659057 1 2024-04-02T04:43:07.622 INFO:tasks.workunit.client.1.smithi027.stdout:0/883: symlink d4/d21/d85/dfb/l124 0 2024-04-02T04:43:07.625 INFO:tasks.workunit.client.1.smithi027.stdout:9/974: mknod d1a/d1d/d29/d50/d73/dbd/dd1/c13d 0 2024-04-02T04:43:07.626 INFO:tasks.workunit.client.1.smithi027.stdout:3/757: mknod d1/de/d74/cf9 0 2024-04-02T04:43:07.632 INFO:tasks.workunit.client.1.smithi027.stdout:9/975: read d1a/d1d/d29/d51/d105/fca [1067234,125056] 0 2024-04-02T04:43:07.632 INFO:tasks.workunit.client.1.smithi027.stdout:7/836: mknod d2/dba/dc/d1f/c110 0 2024-04-02T04:43:07.641 INFO:tasks.workunit.client.1.smithi027.stdout:5/932: dwrite d0/df/d2f/d3b/d51/f8e [0,4194304] 0 2024-04-02T04:43:07.641 INFO:tasks.workunit.client.1.smithi027.stdout:6/746: getdents d13/d1b/d24/d28/d3c 0 2024-04-02T04:43:07.643 INFO:tasks.workunit.client.1.smithi027.stdout:6/747: write d13/d1b/d24/f3a [783759,109867] 0 2024-04-02T04:43:07.644 INFO:tasks.workunit.client.1.smithi027.stdout:9/976: creat d1a/d1d/d29/d37/d3a/f13e x:0 0 0 2024-04-02T04:43:07.650 INFO:tasks.workunit.client.1.smithi027.stdout:7/837: dread d2/d1d/d90/f5a [0,4194304] 0 2024-04-02T04:43:07.651 INFO:tasks.workunit.client.1.smithi027.stdout:5/933: rename d0/df/d11/d13/dce/f130 to d0/df/d11/d13/d9a/d9c/db8/f136 0 2024-04-02T04:43:07.653 INFO:tasks.workunit.client.1.smithi027.stdout:7/838: dread - d2/dba/dc/d16/d2f/fa4 zero size 2024-04-02T04:43:07.661 INFO:tasks.workunit.client.1.smithi027.stdout:3/758: dwrite d1/de/d55/d99/fad [0,4194304] 0 2024-04-02T04:43:07.661 INFO:tasks.workunit.client.1.smithi027.stdout:6/748: creat d13/d1b/d24/d25/d56/dc8/dc9/ffe x:0 0 0 2024-04-02T04:43:07.666 INFO:tasks.workunit.client.1.smithi027.stdout:6/749: write d13/d1b/d48/d75/fd5 [969717,17197] 0 2024-04-02T04:43:07.668 INFO:tasks.workunit.client.1.smithi027.stdout:0/884: dwrite d4/d16/dd5/f47 [0,4194304] 0 2024-04-02T04:43:07.710 INFO:tasks.workunit.client.1.smithi027.stdout:3/759: rmdir d1/de/d55/d63/db9 39 2024-04-02T04:43:07.711 INFO:tasks.workunit.client.1.smithi027.stdout:3/760: fsync d1/de/df/d13/d73/fa9 0 2024-04-02T04:43:07.712 INFO:tasks.workunit.client.1.smithi027.stdout:6/750: symlink d13/d1b/d48/d9c/df7/lff 0 2024-04-02T04:43:07.714 INFO:tasks.workunit.client.1.smithi027.stdout:0/885: write d4/d38/d66/d2d/fac [1572895,54411] 0 2024-04-02T04:43:07.718 INFO:tasks.workunit.client.1.smithi027.stdout:3/761: mknod d1/de/d55/de7/cfa 0 2024-04-02T04:43:07.718 INFO:tasks.workunit.client.1.smithi027.stdout:3/762: stat d1/de/d55/fc0 0 2024-04-02T04:43:07.721 INFO:tasks.workunit.client.1.smithi027.stdout:0/886: fdatasync d4/d21/d85/fae 0 2024-04-02T04:43:07.722 INFO:tasks.workunit.client.1.smithi027.stdout:0/887: readlink d4/d21/d28/ld9 0 2024-04-02T04:43:07.725 INFO:tasks.workunit.client.1.smithi027.stdout:6/751: mknod d13/d1b/d48/d75/c100 0 2024-04-02T04:43:07.725 INFO:tasks.workunit.client.1.smithi027.stdout:6/752: readlink d13/d1b/d48/d9c/df7/lff 0 2024-04-02T04:43:07.727 INFO:tasks.workunit.client.1.smithi027.stdout:7/839: getdents d2/dba/dc/d16/d3f/d79 0 2024-04-02T04:43:07.730 INFO:tasks.workunit.client.1.smithi027.stdout:3/763: dread d1/de/f71 [0,4194304] 0 2024-04-02T04:43:07.735 INFO:tasks.workunit.client.1.smithi027.stdout:6/753: mkdir d13/d1b/d24/d25/d56/dc8/d101 0 2024-04-02T04:43:07.740 INFO:tasks.workunit.client.1.smithi027.stdout:3/764: mknod d1/d1a/cfb 0 2024-04-02T04:43:07.744 INFO:tasks.workunit.client.1.smithi027.stdout:7/840: truncate d2/d1c/f40 620615 0 2024-04-02T04:43:07.745 INFO:tasks.workunit.client.1.smithi027.stdout:0/888: creat d4/d16/f125 x:0 0 0 2024-04-02T04:43:07.746 INFO:tasks.workunit.client.1.smithi027.stdout:9/977: truncate d1a/d1d/d29/d37/d20/dad/fdf 41166 0 2024-04-02T04:43:07.747 INFO:tasks.workunit.client.1.smithi027.stdout:3/765: unlink d1/d1a/f3f 0 2024-04-02T04:43:07.749 INFO:tasks.workunit.client.1.smithi027.stdout:6/754: mknod d13/d1b/d24/d25/d56/c102 0 2024-04-02T04:43:07.751 INFO:tasks.workunit.client.1.smithi027.stdout:0/889: chown d4/d16/f121 7824121 1 2024-04-02T04:43:07.752 INFO:tasks.workunit.client.1.smithi027.stdout:9/978: mkdir d1a/d1d/d29/d37/d20/d7e/d13f 0 2024-04-02T04:43:07.753 INFO:tasks.workunit.client.1.smithi027.stdout:9/979: dread - d1a/d1d/d29/d51/d105/d102/f11c zero size 2024-04-02T04:43:07.756 INFO:tasks.workunit.client.1.smithi027.stdout:6/755: getdents d13/d1b/d48/d58/db9 0 2024-04-02T04:43:07.758 INFO:tasks.workunit.client.1.smithi027.stdout:0/890: rename d4/d38/l8e to d4/d69/d122/l126 0 2024-04-02T04:43:07.760 INFO:tasks.workunit.client.1.smithi027.stdout:9/980: mkdir d1a/d1d/d29/d51/d105/d8c/dcf/d12d/dde/d140 0 2024-04-02T04:43:07.761 INFO:tasks.workunit.client.1.smithi027.stdout:6/756: truncate d13/d1b/d24/d25/d56/fa6 1417167 0 2024-04-02T04:43:07.762 INFO:tasks.workunit.client.1.smithi027.stdout:0/891: readlink d4/d38/d66/d2d/d81/l11d 0 2024-04-02T04:43:07.776 INFO:tasks.workunit.client.1.smithi027.stdout:7/841: dread d2/d1c/f40 [0,4194304] 0 2024-04-02T04:43:07.777 INFO:tasks.workunit.client.1.smithi027.stdout:9/981: getdents d1a/d1d/d29/d56 0 2024-04-02T04:43:07.778 INFO:tasks.workunit.client.1.smithi027.stdout:7/842: chown d2/dba/dc/d16/d3f/d50/f66 1045151 1 2024-04-02T04:43:07.780 INFO:tasks.workunit.client.1.smithi027.stdout:0/892: link d4/d16/d1b/c46 d4/d38/d66/d2d/d81/c127 0 2024-04-02T04:43:07.783 INFO:tasks.workunit.client.1.smithi027.stdout:9/982: creat d1a/d1d/d29/d50/d73/dbd/f141 x:0 0 0 2024-04-02T04:43:07.790 INFO:tasks.workunit.client.1.smithi027.stdout:9/983: dwrite d1a/d1d/d29/d50/d73/dbd/fbf [0,4194304] 0 2024-04-02T04:43:07.794 INFO:tasks.workunit.client.1.smithi027.stdout:9/984: mkdir d1a/d1d/d29/d50/d117/d142 0 2024-04-02T04:43:07.797 INFO:tasks.workunit.client.1.smithi027.stdout:9/985: unlink d1a/d1d/d29/d37/c4d 0 2024-04-02T04:43:07.803 INFO:tasks.workunit.client.1.smithi027.stdout:9/986: dwrite d1a/d1d/d29/d56/de4/d103/f127 [0,4194304] 0 2024-04-02T04:43:07.806 INFO:tasks.workunit.client.1.smithi027.stdout:9/987: creat d1a/d1d/d29/d51/f143 x:0 0 0 2024-04-02T04:43:07.809 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:07 smithi016 ceph-mon[28969]: pgmap v32: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 84 MiB/s rd, 211 MiB/s wr, 501 op/s 2024-04-02T04:43:07.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:07 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:43:07.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:07 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:43:07.822 INFO:tasks.workunit.client.1.smithi027.stdout:4/731: sync 2024-04-02T04:43:07.828 INFO:tasks.workunit.client.1.smithi027.stdout:4/732: dwrite f21 [0,4194304] 0 2024-04-02T04:43:07.838 INFO:tasks.workunit.client.1.smithi027.stdout:4/733: dread d24/d38/d47/d5c/f7e [0,4194304] 0 2024-04-02T04:43:07.895 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:07 smithi027 ceph-mon[37550]: pgmap v32: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 84 MiB/s rd, 211 MiB/s wr, 501 op/s 2024-04-02T04:43:07.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:07 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:43:07.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:07 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:43:07.902 INFO:tasks.workunit.client.1.smithi027.stdout:0/893: dread d4/d16/f121 [0,4194304] 0 2024-04-02T04:43:07.906 INFO:tasks.workunit.client.1.smithi027.stdout:7/843: dwrite d2/dba/d35/f7c [0,4194304] 0 2024-04-02T04:43:07.914 INFO:tasks.workunit.client.1.smithi027.stdout:9/988: dread d1a/d1d/d29/d51/d105/d8c/de1/fe9 [0,4194304] 0 2024-04-02T04:43:07.917 INFO:tasks.workunit.client.1.smithi027.stdout:7/844: write d2/d1c/f8a [5242087,48265] 0 2024-04-02T04:43:07.919 INFO:tasks.workunit.client.1.smithi027.stdout:9/989: unlink d1a/d1d/d29/d51/d105/dec/f130 0 2024-04-02T04:43:07.920 INFO:tasks.workunit.client.1.smithi027.stdout:0/894: truncate d4/d21/d28/d4e/db5/f104 3866897 0 2024-04-02T04:43:07.921 INFO:tasks.workunit.client.1.smithi027.stdout:4/734: dwrite d24/f2c [0,4194304] 0 2024-04-02T04:43:07.922 INFO:tasks.workunit.client.1.smithi027.stdout:0/895: chown d4/d21/d85/db3/l11c 1030198070 1 2024-04-02T04:43:07.924 INFO:tasks.workunit.client.1.smithi027.stdout:7/845: fdatasync d2/dba/dc/d16/d3a/d4d/f58 0 2024-04-02T04:43:07.925 INFO:tasks.workunit.client.1.smithi027.stdout:0/896: fsync d4/d16/dd5/fe8 0 2024-04-02T04:43:07.927 INFO:tasks.workunit.client.1.smithi027.stdout:0/897: chown d4/de0/de2 1545 1 2024-04-02T04:43:07.927 INFO:tasks.workunit.client.1.smithi027.stdout:9/990: creat d1a/d1d/d29/d37/d20/d7e/f144 x:0 0 0 2024-04-02T04:43:07.928 INFO:tasks.workunit.client.1.smithi027.stdout:7/846: read d2/d1c/f37 [780282,65183] 0 2024-04-02T04:43:07.929 INFO:tasks.workunit.client.1.smithi027.stdout:0/898: readlink d4/d21/d28/d4e/db5/d118/dfe/l88 0 2024-04-02T04:43:07.929 INFO:tasks.workunit.client.1.smithi027.stdout:7/847: readlink d2/dba/dc/d16/d3f/d50/d8b/d91/lfa 0 2024-04-02T04:43:07.930 INFO:tasks.workunit.client.1.smithi027.stdout:4/735: mknod d24/d38/d44/d82/d95/cf0 0 2024-04-02T04:43:07.932 INFO:tasks.workunit.client.1.smithi027.stdout:4/736: write d24/d2d/f43 [1295505,126477] 0 2024-04-02T04:43:07.935 INFO:tasks.workunit.client.1.smithi027.stdout:9/991: symlink d1a/d1d/d29/d50/l145 0 2024-04-02T04:43:07.939 INFO:tasks.workunit.client.1.smithi027.stdout:4/737: dwrite d24/d38/d44/d60/fb8 [0,4194304] 0 2024-04-02T04:43:07.939 INFO:tasks.workunit.client.1.smithi027.stdout:0/899: creat d4/d38/d66/d2d/f128 x:0 0 0 2024-04-02T04:43:07.941 INFO:tasks.workunit.client.1.smithi027.stdout:0/900: write d4/d21/d28/d4e/db5/f119 [196079,27603] 0 2024-04-02T04:43:07.945 INFO:tasks.workunit.client.1.smithi027.stdout:7/848: truncate d2/dba/f1e 1208164 0 2024-04-02T04:43:07.946 INFO:tasks.workunit.client.1.smithi027.stdout:7/849: stat d2/dba/dc/d1f/d61/f77 0 2024-04-02T04:43:07.947 INFO:tasks.workunit.client.1.smithi027.stdout:0/901: dread d4/fec [0,4194304] 0 2024-04-02T04:43:07.948 INFO:tasks.workunit.client.1.smithi027.stdout:7/850: fdatasync d2/dba/dc/d16/d3f/dc0/fed 0 2024-04-02T04:43:07.949 INFO:tasks.workunit.client.1.smithi027.stdout:0/902: chown d4/de0/le4 1 1 2024-04-02T04:43:07.949 INFO:tasks.workunit.client.1.smithi027.stdout:7/851: stat d2/dba/dc/d1f/cfe 0 2024-04-02T04:43:07.951 INFO:tasks.workunit.client.1.smithi027.stdout:7/852: dread - d2/d1d/f10b zero size 2024-04-02T04:43:07.951 INFO:tasks.workunit.client.1.smithi027.stdout:9/992: rename lf to d1a/d1d/d3c/d6e/db8/d13b/l146 0 2024-04-02T04:43:07.952 INFO:tasks.workunit.client.1.smithi027.stdout:7/853: write d2/dba/da6/fcc [2557756,125363] 0 2024-04-02T04:43:07.959 INFO:tasks.workunit.client.1.smithi027.stdout:7/854: dwrite d2/d1c/f4b [0,4194304] 0 2024-04-02T04:43:07.961 INFO:tasks.workunit.client.1.smithi027.stdout:0/903: creat d4/d69/f129 x:0 0 0 2024-04-02T04:43:07.967 INFO:tasks.workunit.client.1.smithi027.stdout:1/778: sync 2024-04-02T04:43:07.967 INFO:tasks.workunit.client.1.smithi027.stdout:4/738: dread f15 [0,4194304] 0 2024-04-02T04:43:07.968 INFO:tasks.workunit.client.1.smithi027.stdout:9/993: rmdir d1a/d1d/d29/d51/d105/d8c/dcf/d12d/dde 39 2024-04-02T04:43:07.970 INFO:tasks.workunit.client.1.smithi027.stdout:4/739: write d24/d38/d44/f52 [1230021,23768] 0 2024-04-02T04:43:07.977 INFO:tasks.workunit.client.1.smithi027.stdout:7/855: mknod d2/dba/d35/c111 0 2024-04-02T04:43:07.979 INFO:tasks.workunit.client.1.smithi027.stdout:0/904: creat d4/d5d/f12a x:0 0 0 2024-04-02T04:43:07.979 INFO:tasks.workunit.client.1.smithi027.stdout:0/905: fdatasync d4/d21/d28/f54 0 2024-04-02T04:43:07.983 INFO:tasks.workunit.client.1.smithi027.stdout:7/856: rmdir d2/dba/dc/d16/d3a/d4d/d6f 39 2024-04-02T04:43:07.984 INFO:tasks.workunit.client.1.smithi027.stdout:0/906: dread - d4/d38/d66/d2d/ff4 zero size 2024-04-02T04:43:07.985 INFO:tasks.workunit.client.1.smithi027.stdout:7/857: read d2/df/dc7/f10f [3355457,126823] 0 2024-04-02T04:43:07.985 INFO:tasks.workunit.client.1.smithi027.stdout:7/858: readlink d2/d1c/l34 0 2024-04-02T04:43:07.987 INFO:tasks.workunit.client.1.smithi027.stdout:4/740: write d24/d38/d44/d82/da9/faa [1047349,76659] 0 2024-04-02T04:43:07.992 INFO:tasks.workunit.client.1.smithi027.stdout:1/779: creat d0/de/d52/d74/db7/df1/d25/d2b/ffe x:0 0 0 2024-04-02T04:43:07.993 INFO:tasks.workunit.client.1.smithi027.stdout:0/907: dwrite d4/d16/f27 [0,4194304] 0 2024-04-02T04:43:07.994 INFO:tasks.workunit.client.1.smithi027.stdout:0/908: chown d4/d16/d1b 104 1 2024-04-02T04:43:07.996 INFO:tasks.workunit.client.1.smithi027.stdout:7/859: rename d2/dba/dc/d16/d3f/d79/lfd to d2/dba/dc/d16/d2f/de5/l112 0 2024-04-02T04:43:07.996 INFO:tasks.workunit.client.1.smithi027.stdout:9/994: write d1a/d1d/d29/d56/d8b/f90 [972167,118551] 0 2024-04-02T04:43:08.005 INFO:tasks.workunit.client.1.smithi027.stdout:1/780: dwrite d0/de/d52/d74/db7/df1/f98 [0,4194304] 0 2024-04-02T04:43:08.008 INFO:tasks.workunit.client.1.smithi027.stdout:7/860: dwrite d2/dba/dc/d16/d2f/dcd/fcf [0,4194304] 0 2024-04-02T04:43:08.011 INFO:tasks.workunit.client.1.smithi027.stdout:9/995: rmdir d1a 39 2024-04-02T04:43:08.012 INFO:tasks.workunit.client.1.smithi027.stdout:2/883: sync 2024-04-02T04:43:08.013 INFO:tasks.workunit.client.1.smithi027.stdout:8/803: sync 2024-04-02T04:43:08.014 INFO:tasks.workunit.client.1.smithi027.stdout:2/884: write d1/db/d3b/d17/dec/f122 [1492590,55225] 0 2024-04-02T04:43:08.019 INFO:tasks.workunit.client.1.smithi027.stdout:4/741: write d24/d38/d44/d41/d49/f8b [963592,80645] 0 2024-04-02T04:43:08.022 INFO:tasks.workunit.client.1.smithi027.stdout:1/781: mknod d0/de/d52/d74/dbc/de2/cff 0 2024-04-02T04:43:08.028 INFO:tasks.workunit.client.1.smithi027.stdout:4/742: chown d24/d38/d44/d41/d49/lee 0 1 2024-04-02T04:43:08.029 INFO:tasks.workunit.client.1.smithi027.stdout:7/861: fsync d2/d1d/d92/f9b 0 2024-04-02T04:43:08.030 INFO:tasks.workunit.client.1.smithi027.stdout:0/909: truncate d4/d38/fa9 9081032 0 2024-04-02T04:43:08.031 INFO:tasks.workunit.client.1.smithi027.stdout:0/910: chown d4/d38/d66/d2d/d81/cc1 14851307 1 2024-04-02T04:43:08.031 INFO:tasks.workunit.client.1.smithi027.stdout:0/911: readlink d4/d69/l9a 0 2024-04-02T04:43:08.034 INFO:tasks.workunit.client.1.smithi027.stdout:2/885: mkdir d1/dfe/d139 0 2024-04-02T04:43:08.036 INFO:tasks.workunit.client.1.smithi027.stdout:7/862: creat d2/d1d/d90/f113 x:0 0 0 2024-04-02T04:43:08.037 INFO:tasks.workunit.client.1.smithi027.stdout:7/863: fsync d2/dba/dc/d16/fbd 0 2024-04-02T04:43:08.039 INFO:tasks.workunit.client.1.smithi027.stdout:0/912: dread d4/fec [0,4194304] 0 2024-04-02T04:43:08.040 INFO:tasks.workunit.client.1.smithi027.stdout:0/913: fsync d4/d38/d66/d4a/f70 0 2024-04-02T04:43:08.041 INFO:tasks.workunit.client.1.smithi027.stdout:2/886: rmdir d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/dc0/dc3 39 2024-04-02T04:43:08.043 INFO:tasks.workunit.client.1.smithi027.stdout:2/887: chown d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/d10d 4378853 1 2024-04-02T04:43:08.043 INFO:tasks.workunit.client.1.smithi027.stdout:1/782: creat d0/f100 x:0 0 0 2024-04-02T04:43:08.049 INFO:tasks.workunit.client.1.smithi027.stdout:9/996: link d1a/d1d/d29/d56/f88 d1a/d1d/d29/d51/d105/f147 0 2024-04-02T04:43:08.050 INFO:tasks.workunit.client.1.smithi027.stdout:9/997: truncate d1a/d1d/d29/d56/d8b/f90 1580822 0 2024-04-02T04:43:08.051 INFO:tasks.workunit.client.1.smithi027.stdout:4/743: creat d24/d38/d47/d62/ff1 x:0 0 0 2024-04-02T04:43:08.054 INFO:tasks.workunit.client.1.smithi027.stdout:0/914: creat d4/d21/d28/d4e/d8c/f12b x:0 0 0 2024-04-02T04:43:08.057 INFO:tasks.workunit.client.1.smithi027.stdout:2/888: creat d1/db/d3b/d17/d6d/dd9/dd5/dda/dbb/db4/ddf/f13a x:0 0 0 2024-04-02T04:43:08.062 INFO:tasks.workunit.client.1.smithi027.stdout:4/744: rename d24/d38/d44/dbc/lc7 to d24/d38/d44/d60/lf2 0 2024-04-02T04:43:08.066 INFO:tasks.workunit.client.1.smithi027.stdout:1/783: getdents d0/de/d52/dd1 0 2024-04-02T04:43:08.069 INFO:tasks.workunit.client.1.smithi027.stdout:7/864: rmdir d2/df/d89/d8d/d104 0 2024-04-02T04:43:08.072 INFO:tasks.workunit.client.1.smithi027.stdout:1/784: dread d0/f7 [8388608,4194304] 0 2024-04-02T04:43:08.073 INFO:tasks.workunit.client.1.smithi027.stdout:9/998: mknod d1a/d1d/c148 0 2024-04-02T04:43:08.074 INFO:tasks.workunit.client.1.smithi027.stdout:9/999: stat d1a/d1d/d29/d56/d8b/le3 0 2024-04-02T04:43:08.079 INFO:tasks.workunit.client.1.smithi027.stdout:2/889: mkdir d1/db/de5/de6/d13b 0 2024-04-02T04:43:08.083 INFO:tasks.workunit.client.1.smithi027.stdout:4/745: mkdir d24/d38/df3 0 2024-04-02T04:43:08.083 INFO:tasks.workunit.client.1.smithi027.stdout:7/865: dread d2/d1d/d92/f9b [0,4194304] 0 2024-04-02T04:43:08.091 INFO:tasks.workunit.client.1.smithi027.stdout:4/746: unlink lb 0 2024-04-02T04:43:08.094 INFO:tasks.workunit.client.1.smithi027.stdout:0/915: write d4/d69/f8a [2458555,105119] 0 2024-04-02T04:43:08.099 INFO:tasks.workunit.client.1.smithi027.stdout:4/747: dwrite d24/d38/d44/d41/d49/f77 [0,4194304] 0 2024-04-02T04:43:08.100 INFO:tasks.workunit.client.1.smithi027.stdout:1/785: symlink d0/de/d52/d74/db7/df1/d25/d2b/l101 0 2024-04-02T04:43:08.100 INFO:tasks.workunit.client.1.smithi027.stdout:1/786: readlink d0/ld7 0 2024-04-02T04:43:08.108 INFO:tasks.workunit.client.1.smithi027.stdout:0/916: mkdir d4/d21/d28/d12c 0 2024-04-02T04:43:08.111 INFO:tasks.workunit.client.1.smithi027.stdout:2/890: dwrite d1/db/d3b/d17/d2b/f114 [4194304,4194304] 0 2024-04-02T04:43:08.113 INFO:tasks.workunit.client.1.smithi027.stdout:1/787: mkdir d0/de/d52/d74/df7/d3b/d4f/d102 0 2024-04-02T04:43:08.113 INFO:tasks.workunit.client.1.smithi027.stdout:7/866: link d2/dba/dc/d16/d2f/de5/l112 d2/d1c/l114 0 2024-04-02T04:43:08.120 INFO:tasks.workunit.client.1.smithi027.stdout:7/867: symlink d2/d7/df3/l115 0 2024-04-02T04:43:08.123 INFO:tasks.workunit.client.1.smithi027.stdout:4/748: dwrite d24/d38/d44/d41/f87 [0,4194304] 0 2024-04-02T04:43:08.125 INFO:tasks.workunit.client.1.smithi027.stdout:7/868: fsync d2/dba/dc/d16/d2f/dcd/fcf 0 2024-04-02T04:43:08.126 INFO:tasks.workunit.client.1.smithi027.stdout:4/749: write d24/d48/fc9 [956706,48082] 0 2024-04-02T04:43:08.131 INFO:tasks.workunit.client.1.smithi027.stdout:0/917: dwrite d4/d21/fc5 [4194304,4194304] 0 2024-04-02T04:43:08.132 INFO:tasks.workunit.client.1.smithi027.stdout:0/918: read - d4/d16/dd5/fd6 zero size 2024-04-02T04:43:08.133 INFO:tasks.workunit.client.1.smithi027.stdout:0/919: write d4/d69/f11e [1011363,55749] 0 2024-04-02T04:43:08.135 INFO:tasks.workunit.client.1.smithi027.stdout:2/891: link d1/db/d3b/d17/d68/d69/f8d d1/db/d3b/d17/d6d/dd9/dd5/dda/f13c 0 2024-04-02T04:43:08.144 INFO:tasks.workunit.client.1.smithi027.stdout:0/920: mknod d4/d16/d71/c12d 0 2024-04-02T04:43:08.146 INFO:tasks.workunit.client.1.smithi027.stdout:7/869: mkdir d2/dba/dc/d16/d3a/db2/d116 0 2024-04-02T04:43:08.150 INFO:tasks.workunit.client.1.smithi027.stdout:0/921: symlink d4/d38/d66/l12e 0 2024-04-02T04:43:08.153 INFO:tasks.workunit.client.1.smithi027.stdout:7/870: mkdir d2/dba/dc/d16/d3f/d50/d8b/d91/d117 0 2024-04-02T04:43:08.155 INFO:tasks.workunit.client.1.smithi027.stdout:7/871: write d2/dba/dc/f10d [281739,19628] 0 2024-04-02T04:43:08.156 INFO:tasks.workunit.client.1.smithi027.stdout:1/788: getdents d0/de/d52/d74/db7/df1/d25/d3c/d64/dec 0 2024-04-02T04:43:08.161 INFO:tasks.workunit.client.1.smithi027.stdout:2/892: dread d1/db/d3b/d17/d2b/f53 [0,4194304] 0 2024-04-02T04:43:08.164 INFO:tasks.workunit.client.1.smithi027.stdout:1/789: unlink d0/de/d52/d74/df7/d3e/l85 0 2024-04-02T04:43:08.167 INFO:tasks.workunit.client.1.smithi027.stdout:7/872: rmdir d2/dba/dc/d16/d3a/db2/d116 0 2024-04-02T04:43:08.169 INFO:tasks.workunit.client.1.smithi027.stdout:0/922: creat d4/d38/d66/d2d/d4c/f12f x:0 0 0 2024-04-02T04:43:08.171 INFO:tasks.workunit.client.1.smithi027.stdout:1/790: mknod d0/de/d52/d74/db7/df1/c103 0 2024-04-02T04:43:08.173 INFO:tasks.workunit.client.1.smithi027.stdout:0/923: creat d4/d21/d28/d4e/db5/d118/dfe/dd1/f130 x:0 0 0 2024-04-02T04:43:08.173 INFO:tasks.workunit.client.1.smithi027.stdout:0/924: readlink d4/d21/d28/ld9 0 2024-04-02T04:43:08.175 INFO:tasks.workunit.client.1.smithi027.stdout:2/893: creat d1/db/d3b/d17/d6d/dd9/f13d x:0 0 0 2024-04-02T04:43:08.176 INFO:tasks.workunit.client.1.smithi027.stdout:1/791: rename d0/de/d52/d74/df7/d3b/dda/ce0 to d0/de/d52/d74/df7/d3b/dda/c104 0 2024-04-02T04:43:08.180 INFO:tasks.workunit.client.1.smithi027.stdout:7/873: getdents d2/df/d89 0 2024-04-02T04:43:08.181 INFO:tasks.workunit.client.1.smithi027.stdout:2/894: mkdir d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/de1/d137/d13e 0 2024-04-02T04:43:08.184 INFO:tasks.workunit.client.1.smithi027.stdout:7/874: symlink d2/dba/dc/d16/d3a/l118 0 2024-04-02T04:43:08.185 INFO:tasks.workunit.client.1.smithi027.stdout:5/934: sync 2024-04-02T04:43:08.185 INFO:tasks.workunit.client.1.smithi027.stdout:6/757: sync 2024-04-02T04:43:08.185 INFO:tasks.workunit.client.1.smithi027.stdout:3/766: sync 2024-04-02T04:43:08.187 INFO:tasks.workunit.client.1.smithi027.stdout:0/925: link d4/d21/d85/c112 d4/c131 0 2024-04-02T04:43:08.187 INFO:tasks.workunit.client.1.smithi027.stdout:7/875: dread d2/d1d/d92/f9b [0,4194304] 0 2024-04-02T04:43:08.187 INFO:tasks.workunit.client.1.smithi027.stdout:6/758: stat d13/d1b/d24/d25/d56/d6d 0 2024-04-02T04:43:08.190 INFO:tasks.workunit.client.1.smithi027.stdout:3/767: truncate d1/de/d74/fcc 713542 0 2024-04-02T04:43:08.192 INFO:tasks.workunit.client.1.smithi027.stdout:5/935: read d0/d1d/f91 [3162407,57426] 0 2024-04-02T04:43:08.193 INFO:tasks.workunit.client.1.smithi027.stdout:2/895: dread d1/db/d3b/d17/d2b/db8/f9f [0,4194304] 0 2024-04-02T04:43:08.194 INFO:tasks.workunit.client.1.smithi027.stdout:1/792: getdents d0/de/d52/d74/db7/df1 0 2024-04-02T04:43:08.195 INFO:tasks.workunit.client.1.smithi027.stdout:2/896: chown d1/db/d3b/d17/d6d/dd9/dd5/l67 293502744 1 2024-04-02T04:43:08.195 INFO:tasks.workunit.client.1.smithi027.stdout:2/897: readlink d1/db/d57/lce 0 2024-04-02T04:43:08.200 INFO:tasks.workunit.client.1.smithi027.stdout:3/768: mknod d1/de/df/d13/cfc 0 2024-04-02T04:43:08.207 INFO:tasks.workunit.client.1.smithi027.stdout:6/759: mknod d13/d1b/d24/d25/d42/c103 0 2024-04-02T04:43:08.208 INFO:tasks.workunit.client.1.smithi027.stdout:6/760: write f7 [2828862,126989] 0 2024-04-02T04:43:08.210 INFO:tasks.workunit.client.1.smithi027.stdout:0/926: mknod d4/d21/d28/d4e/d5f/c132 0 2024-04-02T04:43:08.214 INFO:tasks.workunit.client.1.smithi027.stdout:2/898: fsync d1/db/d3b/d17/d6d/dd9/dd5/dda/dbb/db4/ddf/f102 0 2024-04-02T04:43:08.218 INFO:tasks.workunit.client.1.smithi027.stdout:7/876: link d2/dba/d35/f67 d2/d1d/f119 0 2024-04-02T04:43:08.221 INFO:tasks.workunit.client.1.smithi027.stdout:5/936: fdatasync d0/df/d11/d13/d9a/db7/fdc 0 2024-04-02T04:43:08.225 INFO:tasks.workunit.client.1.smithi027.stdout:3/769: write d1/d1a/f79 [60004,58768] 0 2024-04-02T04:43:08.227 INFO:tasks.workunit.client.1.smithi027.stdout:6/761: rename d13/d1b/d24/d25/d42/cc7 to d13/d1b/d24/d28/d46/dcc/db7/c104 0 2024-04-02T04:43:08.230 INFO:tasks.workunit.client.1.smithi027.stdout:0/927: write d4/de0/de2/ff2 [908110,68059] 0 2024-04-02T04:43:08.240 INFO:tasks.workunit.client.1.smithi027.stdout:1/793: rename d0/de/d52/d74/df7/d3b/d4f/d9b/ff0 to d0/de/d52/d74/dbc/f105 0 2024-04-02T04:43:08.241 INFO:tasks.workunit.client.1.smithi027.stdout:6/762: fdatasync d13/d1b/d48/d58/fa2 0 2024-04-02T04:43:08.249 INFO:tasks.workunit.client.1.smithi027.stdout:7/877: dread d2/d1d/d57/f5e [0,4194304] 0 2024-04-02T04:43:08.249 INFO:tasks.workunit.client.1.smithi027.stdout:7/878: fsync d2/dba/da6/fcc 0 2024-04-02T04:43:08.249 INFO:tasks.workunit.client.1.smithi027.stdout:2/899: mkdir d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/d10d/d11f/d13f 0 2024-04-02T04:43:08.254 INFO:tasks.workunit.client.1.smithi027.stdout:5/937: write d0/d68/d6b/d74/fb6 [1163773,23506] 0 2024-04-02T04:43:08.258 INFO:tasks.workunit.client.1.smithi027.stdout:7/879: fdatasync d2/df/dc7/f10f 0 2024-04-02T04:43:08.260 INFO:tasks.workunit.client.1.smithi027.stdout:3/770: creat d1/de/df/d9e/ffd x:0 0 0 2024-04-02T04:43:08.264 INFO:tasks.workunit.client.1.smithi027.stdout:2/900: write d1/db/d3b/d17/f6a [767215,122792] 0 2024-04-02T04:43:08.266 INFO:tasks.workunit.client.1.smithi027.stdout:0/928: link d4/d21/d28/d4e/d5f/f5a d4/d21/d28/d4e/d5f/f133 0 2024-04-02T04:43:08.267 INFO:tasks.workunit.client.1.smithi027.stdout:0/929: chown d4/d21/d85/f86 16 1 2024-04-02T04:43:08.267 INFO:tasks.workunit.client.1.smithi027.stdout:2/901: dread d1/db/d3b/d17/d2b/db8/fd0 [0,4194304] 0 2024-04-02T04:43:08.270 INFO:tasks.workunit.client.1.smithi027.stdout:3/771: chown d1/de/d55/d63/f91 118938 1 2024-04-02T04:43:08.272 INFO:tasks.workunit.client.1.smithi027.stdout:5/938: mkdir d0/df/d11/d13/d137 0 2024-04-02T04:43:08.273 INFO:tasks.workunit.client.1.smithi027.stdout:1/794: dread d0/de/d52/d74/db7/df1/d25/f31 [4194304,4194304] 0 2024-04-02T04:43:08.273 INFO:tasks.workunit.client.1.smithi027.stdout:6/763: creat d13/d1b/f105 x:0 0 0 2024-04-02T04:43:08.276 INFO:tasks.workunit.client.1.smithi027.stdout:0/930: dwrite d4/d16/f125 [0,4194304] 0 2024-04-02T04:43:08.277 INFO:tasks.workunit.client.1.smithi027.stdout:2/902: mknod d1/db/d3b/d17/d6d/dd9/dd5/dda/c140 0 2024-04-02T04:43:08.278 INFO:tasks.workunit.client.1.smithi027.stdout:7/880: write d2/dba/dc/d16/d3f/d44/f6a [3782465,99528] 0 2024-04-02T04:43:08.283 INFO:tasks.workunit.client.1.smithi027.stdout:3/772: creat d1/de/df/d9e/dc6/de4/ffe x:0 0 0 2024-04-02T04:43:08.285 INFO:tasks.workunit.client.1.smithi027.stdout:5/939: fsync d0/df/d2f/d3b/f3d 0 2024-04-02T04:43:08.291 INFO:tasks.workunit.client.1.smithi027.stdout:5/940: dread d0/df/d2f/d3b/d51/f8e [0,4194304] 0 2024-04-02T04:43:08.291 INFO:tasks.workunit.client.1.smithi027.stdout:6/764: fdatasync d13/d1b/f2b 0 2024-04-02T04:43:08.292 INFO:tasks.workunit.client.1.smithi027.stdout:5/941: readlink d0/l1 0 2024-04-02T04:43:08.294 INFO:tasks.workunit.client.1.smithi027.stdout:5/942: write d0/d68/d6b/f117 [2616278,32872] 0 2024-04-02T04:43:08.294 INFO:tasks.workunit.client.1.smithi027.stdout:0/931: stat d4/d21/d28/d4e/d5f/ced 0 2024-04-02T04:43:08.296 INFO:tasks.workunit.client.1.smithi027.stdout:2/903: chown d1/db/d3b/d17/d6d/cf7 25413807 1 2024-04-02T04:43:08.296 INFO:tasks.workunit.client.1.smithi027.stdout:5/943: chown d0 7208475 1 2024-04-02T04:43:08.298 INFO:tasks.workunit.client.1.smithi027.stdout:2/904: read d1/db/d3b/d17/d12/f41 [1448725,34504] 0 2024-04-02T04:43:08.304 INFO:tasks.workunit.client.1.smithi027.stdout:7/881: dwrite d2/dba/dc/faf [0,4194304] 0 2024-04-02T04:43:08.305 INFO:tasks.workunit.client.1.smithi027.stdout:3/773: dread d1/d1a/f86 [0,4194304] 0 2024-04-02T04:43:08.315 INFO:tasks.workunit.client.1.smithi027.stdout:0/932: dread d4/f90 [0,4194304] 0 2024-04-02T04:43:08.319 INFO:tasks.workunit.client.1.smithi027.stdout:5/944: rename d0/df/d2f/d3b/d51/lcc to d0/df/d2f/d107/l138 0 2024-04-02T04:43:08.320 INFO:tasks.workunit.client.1.smithi027.stdout:6/765: write d13/d1b/d48/f66 [1431944,11044] 0 2024-04-02T04:43:08.321 INFO:tasks.workunit.client.1.smithi027.stdout:1/795: rmdir d0/de/dc0 0 2024-04-02T04:43:08.324 INFO:tasks.workunit.client.1.smithi027.stdout:0/933: dwrite d4/d21/d28/d4e/d8c/db0/f114 [0,4194304] 0 2024-04-02T04:43:08.331 INFO:tasks.workunit.client.1.smithi027.stdout:0/934: dread d4/d21/d28/f54 [0,4194304] 0 2024-04-02T04:43:08.334 INFO:tasks.workunit.client.1.smithi027.stdout:0/935: write d4/d21/d28/f54 [735531,33326] 0 2024-04-02T04:43:08.335 INFO:tasks.workunit.client.1.smithi027.stdout:0/936: chown d4/d21 60328189 1 2024-04-02T04:43:08.337 INFO:tasks.workunit.client.1.smithi027.stdout:0/937: chown d4/d38/fa1 485 1 2024-04-02T04:43:08.354 INFO:tasks.workunit.client.1.smithi027.stdout:0/938: dread d4/fb [0,4194304] 0 2024-04-02T04:43:08.379 INFO:tasks.workunit.client.1.smithi027.stdout:8/804: sync 2024-04-02T04:43:08.425 INFO:tasks.workunit.client.1.smithi027.stdout:7/882: symlink d2/dba/dc/d1f/d61/l11a 0 2024-04-02T04:43:08.426 INFO:tasks.workunit.client.1.smithi027.stdout:7/883: fsync d2/d1d/f80 0 2024-04-02T04:43:08.438 INFO:tasks.workunit.client.1.smithi027.stdout:4/750: sync 2024-04-02T04:43:08.439 INFO:tasks.workunit.client.1.smithi027.stdout:5/945: creat d0/df/d2f/d107/f139 x:0 0 0 2024-04-02T04:43:08.439 INFO:tasks.workunit.client.1.smithi027.stdout:5/946: readlink d0/df/d29/l4a 0 2024-04-02T04:43:08.440 INFO:tasks.workunit.client.1.smithi027.stdout:5/947: stat d0/d1d/l5d 0 2024-04-02T04:43:08.440 INFO:tasks.workunit.client.1.smithi027.stdout:6/766: readlink d13/d1b/d24/d25/d42/l69 0 2024-04-02T04:43:08.442 INFO:tasks.workunit.client.1.smithi027.stdout:6/767: write d13/d1b/f8e [4225688,53767] 0 2024-04-02T04:43:08.462 INFO:tasks.workunit.client.1.smithi027.stdout:7/884: mknod d2/dba/dc/d16/d2f/dcd/c11b 0 2024-04-02T04:43:08.482 INFO:tasks.workunit.client.1.smithi027.stdout:0/939: truncate d4/d5d/fd2 152759 0 2024-04-02T04:43:08.483 INFO:tasks.workunit.client.1.smithi027.stdout:8/805: unlink d5/d8/db/dd/d5f/f7e 0 2024-04-02T04:43:08.500 INFO:tasks.workunit.client.1.smithi027.stdout:1/796: getdents d0/de/d52/d74/dbc 0 2024-04-02T04:43:08.504 INFO:tasks.workunit.client.1.smithi027.stdout:4/751: getdents d24/d38/d47/d5c 0 2024-04-02T04:43:08.504 INFO:tasks.workunit.client.1.smithi027.stdout:4/752: chown d24/d48/fc9 1731 1 2024-04-02T04:43:08.505 INFO:tasks.workunit.client.1.smithi027.stdout:4/753: write d24/f2c [1698773,119389] 0 2024-04-02T04:43:08.506 INFO:tasks.workunit.client.1.smithi027.stdout:1/797: symlink d0/de/d52/d74/d97/l106 0 2024-04-02T04:43:08.508 INFO:tasks.workunit.client.1.smithi027.stdout:1/798: readlink d0/de/d52/d74/df7/d3b/d4f/l73 0 2024-04-02T04:43:08.511 INFO:tasks.workunit.client.1.smithi027.stdout:1/799: stat d0/de/f17 0 2024-04-02T04:43:08.513 INFO:tasks.workunit.client.1.smithi027.stdout:1/800: readlink d0/l8 0 2024-04-02T04:43:08.515 INFO:tasks.workunit.client.1.smithi027.stdout:1/801: read d0/de/f80 [3538663,6776] 0 2024-04-02T04:43:08.517 INFO:tasks.workunit.client.1.smithi027.stdout:6/768: rename d13/d1b/d24/d25/d56/fa6 to d13/d1b/d48/d9c/df7/f106 0 2024-04-02T04:43:08.518 INFO:tasks.workunit.client.1.smithi027.stdout:6/769: dread - d13/d1b/d24/d25/d27/f91 zero size 2024-04-02T04:43:08.519 INFO:tasks.workunit.client.1.smithi027.stdout:6/770: stat d13/d1b/d24/d25/d56/fa0 0 2024-04-02T04:43:08.526 INFO:tasks.workunit.client.1.smithi027.stdout:6/771: dwrite d13/d1b/d24/f41 [4194304,4194304] 0 2024-04-02T04:43:08.527 INFO:tasks.workunit.client.1.smithi027.stdout:8/806: rename d5/d8/db/dd/d5f/d91/dc2/dfd to d5/d8/db/dd/d5f/db2/d10b 0 2024-04-02T04:43:08.529 INFO:tasks.workunit.client.1.smithi027.stdout:6/772: write d13/d1b/d24/d25/d27/f97 [4911375,39537] 0 2024-04-02T04:43:08.533 INFO:tasks.workunit.client.1.smithi027.stdout:1/802: creat d0/de/d52/d74/db7/df1/d25/d3c/d64/da6/f107 x:0 0 0 2024-04-02T04:43:08.538 INFO:tasks.workunit.client.1.smithi027.stdout:6/773: creat d13/d1b/d24/d28/d46/dcc/f107 x:0 0 0 2024-04-02T04:43:08.553 INFO:tasks.workunit.client.1.smithi027.stdout:8/807: truncate d5/d8/db/dd/d5f/d27/d31/f89 475556 0 2024-04-02T04:43:08.553 INFO:tasks.workunit.client.1.smithi027.stdout:1/803: dwrite d0/de/d52/d74/db7/df1/d25/d2b/fce [4194304,4194304] 0 2024-04-02T04:43:08.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:08 smithi016 ceph-mon[28969]: Standby manager daemon smithi016.eqnebx restarted 2024-04-02T04:43:08.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:08 smithi016 ceph-mon[28969]: Standby manager daemon smithi016.eqnebx started 2024-04-02T04:43:08.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:08 smithi016 ceph-mon[28969]: from='mgr.? 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi016.eqnebx/crt"}]: dispatch 2024-04-02T04:43:08.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:08 smithi016 ceph-mon[28969]: from='mgr.? 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T04:43:08.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:08 smithi016 ceph-mon[28969]: from='mgr.? 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi016.eqnebx/key"}]: dispatch 2024-04-02T04:43:08.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:08 smithi016 ceph-mon[28969]: from='mgr.? 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T04:43:08.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:08 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:43:08.569 INFO:tasks.workunit.client.1.smithi027.stdout:1/804: creat d0/de/d52/d74/df7/d3e/d5f/d8b/d9d/f108 x:0 0 0 2024-04-02T04:43:08.572 INFO:tasks.workunit.client.1.smithi027.stdout:1/805: chown d0/de/d52/d74/c8e 0 1 2024-04-02T04:43:08.577 INFO:tasks.workunit.client.1.smithi027.stdout:1/806: dread - d0/de/d52/d74/df7/d3b/fee zero size 2024-04-02T04:43:08.582 INFO:tasks.workunit.client.1.smithi027.stdout:1/807: mkdir d0/de/d52/d76/d109 0 2024-04-02T04:43:08.583 INFO:tasks.workunit.client.1.smithi027.stdout:1/808: chown d0/de/d52/d76 10 1 2024-04-02T04:43:08.587 INFO:tasks.workunit.client.1.smithi027.stdout:1/809: chown d0/de/d52/d74/df7/d3e/d5f/l8a 7910476 1 2024-04-02T04:43:08.587 INFO:tasks.workunit.client.1.smithi027.stdout:1/810: stat d0/de/l35 0 2024-04-02T04:43:08.591 INFO:tasks.workunit.client.1.smithi027.stdout:1/811: mknod d0/de/d52/d74/df7/d3b/dda/c10a 0 2024-04-02T04:43:08.592 INFO:tasks.workunit.client.1.smithi027.stdout:1/812: readlink d0/de/d52/d74/df7/d3e/lb9 0 2024-04-02T04:43:08.592 INFO:tasks.workunit.client.1.smithi027.stdout:1/813: readlink d0/l8 0 2024-04-02T04:43:08.593 INFO:tasks.workunit.client.1.smithi027.stdout:1/814: chown d0/de/d52/d74/dd9/ldf 55210 1 2024-04-02T04:43:08.594 INFO:tasks.workunit.client.1.smithi027.stdout:1/815: fdatasync d0/de/d52/d74/db7/df1/d25/d3c/d64/dec/ff3 0 2024-04-02T04:43:08.601 INFO:tasks.workunit.client.1.smithi027.stdout:3/774: write d1/de/df/d13/d73/d77/f7a [53513,50595] 0 2024-04-02T04:43:08.603 INFO:tasks.workunit.client.1.smithi027.stdout:2/905: dwrite d1/d91/f9c [0,4194304] 0 2024-04-02T04:43:08.605 INFO:tasks.workunit.client.1.smithi027.stdout:1/816: dwrite d0/de/d52/d74/fb5 [0,4194304] 0 2024-04-02T04:43:08.607 INFO:tasks.workunit.client.1.smithi027.stdout:3/775: creat d1/d1a/fff x:0 0 0 2024-04-02T04:43:08.611 INFO:tasks.workunit.client.1.smithi027.stdout:2/906: mknod d1/db/d3b/d17/d6d/dd9/dd5/dda/d8a/c141 0 2024-04-02T04:43:08.613 INFO:tasks.workunit.client.1.smithi027.stdout:2/907: write d1/db/d3b/d17/d2b/f114 [1801432,63587] 0 2024-04-02T04:43:08.613 INFO:tasks.workunit.client.1.smithi027.stdout:3/776: truncate d1/de/d55/fb7 1186634 0 2024-04-02T04:43:08.615 INFO:tasks.workunit.client.1.smithi027.stdout:2/908: chown d1/db/d3b/d17/d6d/dd9/dd5/dda/dbb/fe4 317449 1 2024-04-02T04:43:08.621 INFO:tasks.workunit.client.1.smithi027.stdout:2/909: fdatasync d1/db/d3b/d17/d6d/dd9/dd5/f5a 0 2024-04-02T04:43:08.624 INFO:tasks.workunit.client.1.smithi027.stdout:3/777: dwrite d1/de/df/d13/d73/d77/fdf [0,4194304] 0 2024-04-02T04:43:08.625 INFO:tasks.workunit.client.1.smithi027.stdout:3/778: readlink d1/de/lce 0 2024-04-02T04:43:08.632 INFO:tasks.workunit.client.1.smithi027.stdout:3/779: mkdir d1/de/df/d9e/d100 0 2024-04-02T04:43:08.635 INFO:tasks.workunit.client.1.smithi027.stdout:3/780: creat d1/de/df/d13/db3/f101 x:0 0 0 2024-04-02T04:43:08.638 INFO:tasks.workunit.client.1.smithi027.stdout:3/781: truncate d1/de/f42 2423097 0 2024-04-02T04:43:08.639 INFO:tasks.workunit.client.1.smithi027.stdout:3/782: readlink d1/de/df/d13/l4d 0 2024-04-02T04:43:08.647 INFO:tasks.workunit.client.1.smithi027.stdout:5/948: dwrite d0/f47 [0,4194304] 0 2024-04-02T04:43:08.650 INFO:tasks.workunit.client.1.smithi027.stdout:3/783: dread d1/de/df/d61/f6c [0,4194304] 0 2024-04-02T04:43:08.655 INFO:tasks.workunit.client.1.smithi027.stdout:3/784: rmdir d1/de/d55 39 2024-04-02T04:43:08.657 INFO:tasks.workunit.client.1.smithi027.stdout:8/808: unlink l1 0 2024-04-02T04:43:08.668 INFO:tasks.workunit.client.1.smithi027.stdout:7/885: dwrite d2/f4c [0,4194304] 0 2024-04-02T04:43:08.673 INFO:tasks.workunit.client.1.smithi027.stdout:0/940: dwrite d4/d21/d28/f75 [0,4194304] 0 2024-04-02T04:43:08.674 INFO:tasks.workunit.client.1.smithi027.stdout:8/809: dread d5/d8/f17 [0,4194304] 0 2024-04-02T04:43:08.675 INFO:tasks.workunit.client.1.smithi027.stdout:0/941: write d4/d16/f125 [3895247,116701] 0 2024-04-02T04:43:08.676 INFO:tasks.workunit.client.1.smithi027.stdout:8/810: stat d5/d8/db/dd/fd2 0 2024-04-02T04:43:08.677 INFO:tasks.workunit.client.1.smithi027.stdout:0/942: fsync d4/d21/d28/d4e/d5f/f79 0 2024-04-02T04:43:08.687 INFO:tasks.workunit.client.1.smithi027.stdout:4/754: dwrite d24/d38/d44/d41/d5a/f88 [0,4194304] 0 2024-04-02T04:43:08.706 INFO:tasks.workunit.client.1.smithi027.stdout:2/910: rename d1/db/d3b/d17/d12/f7c to d1/db/d3b/d17/d6d/dd9/dd5/f142 0 2024-04-02T04:43:08.709 INFO:tasks.workunit.client.1.smithi027.stdout:2/911: fsync d1/db/d3b/d17/d6d/d84/fd8 0 2024-04-02T04:43:08.714 INFO:tasks.workunit.client.1.smithi027.stdout:3/785: rename d1/de/d74 to d1/de/df/d13/d73/d77/d102 0 2024-04-02T04:43:08.715 INFO:tasks.workunit.client.1.smithi027.stdout:4/755: mknod d24/d38/d44/d41/cf4 0 2024-04-02T04:43:08.715 INFO:tasks.workunit.client.1.smithi027.stdout:4/756: stat d24/d38/d44/f6e 0 2024-04-02T04:43:08.716 INFO:tasks.workunit.client.1.smithi027.stdout:2/912: symlink d1/db/d3b/d61/dd2/l143 0 2024-04-02T04:43:08.718 INFO:tasks.workunit.client.1.smithi027.stdout:3/786: mkdir d1/d1a/d103 0 2024-04-02T04:43:08.719 INFO:tasks.workunit.client.1.smithi027.stdout:6/774: rmdir d13/d1b/d48/d9c 39 2024-04-02T04:43:08.724 INFO:tasks.workunit.client.1.smithi027.stdout:7/886: rename d2/c6 to d2/dba/dc/d1f/c11c 0 2024-04-02T04:43:08.725 INFO:tasks.workunit.client.1.smithi027.stdout:7/887: chown d2/dba/dc/d16/d3f 63167 1 2024-04-02T04:43:08.726 INFO:tasks.workunit.client.1.smithi027.stdout:3/787: dwrite d1/de/df/d9e/ffd [0,4194304] 0 2024-04-02T04:43:08.726 INFO:tasks.workunit.client.1.smithi027.stdout:6/775: readlink d13/d1b/d48/d75/l86 0 2024-04-02T04:43:08.729 INFO:tasks.workunit.client.1.smithi027.stdout:6/776: write d13/d1b/d24/d25/d56/d6d/fcf [495279,101286] 0 2024-04-02T04:43:08.736 INFO:tasks.workunit.client.1.smithi027.stdout:3/788: creat d1/de/df/d9e/dc6/de4/f104 x:0 0 0 2024-04-02T04:43:08.737 INFO:tasks.workunit.client.1.smithi027.stdout:2/913: creat d1/db/d3b/d17/d6d/dd9/dd5/f144 x:0 0 0 2024-04-02T04:43:08.746 INFO:tasks.workunit.client.1.smithi027.stdout:2/914: mkdir d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/de1/d145 0 2024-04-02T04:43:08.750 INFO:tasks.workunit.client.1.smithi027.stdout:1/817: truncate d0/de/d52/d74/df7/d3e/f82 2215041 0 2024-04-02T04:43:08.751 INFO:tasks.workunit.client.1.smithi027.stdout:6/777: link d13/d1b/d24/cd9 d13/d1b/d24/d25/d42/c108 0 2024-04-02T04:43:08.756 INFO:tasks.workunit.client.1.smithi027.stdout:1/818: readlink d0/de/d52/d74/db7/df1/d25/d2b/d59/la9 0 2024-04-02T04:43:08.757 INFO:tasks.workunit.client.1.smithi027.stdout:1/819: chown d0/f54 6628297 1 2024-04-02T04:43:08.760 INFO:tasks.workunit.client.1.smithi027.stdout:6/778: creat d13/d1b/d48/d9c/df7/f109 x:0 0 0 2024-04-02T04:43:08.762 INFO:tasks.workunit.client.1.smithi027.stdout:1/820: creat d0/de/d52/d74/db7/df1/d25/d3c/d64/dec/f10b x:0 0 0 2024-04-02T04:43:08.764 INFO:tasks.workunit.client.1.smithi027.stdout:6/779: rename d13/d1b/d24/d28/d3c/f78 to d13/d1b/d24/d25/d56/dc8/f10a 0 2024-04-02T04:43:08.772 INFO:tasks.workunit.client.1.smithi027.stdout:6/780: fdatasync d13/d1b/d24/d25/d56/dc8/dc9/fda 0 2024-04-02T04:43:08.774 INFO:tasks.workunit.client.1.smithi027.stdout:5/949: write d0/df/d76/f85 [152828,114071] 0 2024-04-02T04:43:08.790 INFO:tasks.workunit.client.1.smithi027.stdout:6/781: creat d13/d1b/d48/f10b x:0 0 0 2024-04-02T04:43:08.797 INFO:tasks.workunit.client.1.smithi027.stdout:6/782: mkdir d13/d1b/d48/d75/d10c 0 2024-04-02T04:43:08.799 INFO:tasks.workunit.client.1.smithi027.stdout:6/783: truncate d13/d1b/d24/d25/d56/dc8/dc9/fd8 747734 0 2024-04-02T04:43:08.799 INFO:tasks.workunit.client.1.smithi027.stdout:6/784: fsync f7 0 2024-04-02T04:43:08.804 INFO:tasks.workunit.client.1.smithi027.stdout:6/785: symlink d13/d1b/l10d 0 2024-04-02T04:43:08.808 INFO:tasks.workunit.client.1.smithi027.stdout:6/786: mknod d13/d1b/d24/d25/d56/dc8/ddb/c10e 0 2024-04-02T04:43:08.814 INFO:tasks.workunit.client.1.smithi027.stdout:6/787: symlink d13/d1b/l10f 0 2024-04-02T04:43:08.821 INFO:tasks.workunit.client.1.smithi027.stdout:6/788: creat d13/d1b/d48/d58/db9/f110 x:0 0 0 2024-04-02T04:43:08.825 INFO:tasks.workunit.client.1.smithi027.stdout:8/811: dwrite d5/d8/db/dd/d5f/d91/d9f/fb4 [0,4194304] 0 2024-04-02T04:43:08.832 INFO:tasks.workunit.client.1.smithi027.stdout:8/812: creat d5/d8/d5c/de6/f10c x:0 0 0 2024-04-02T04:43:08.836 INFO:tasks.workunit.client.1.smithi027.stdout:8/813: write d5/d8/db/dd/d5f/f14 [423340,88571] 0 2024-04-02T04:43:08.837 INFO:tasks.workunit.client.1.smithi027.stdout:6/789: dread d13/d1b/d48/d75/f83 [0,4194304] 0 2024-04-02T04:43:08.841 INFO:tasks.workunit.client.1.smithi027.stdout:4/757: dwrite d24/d38/d47/d5c/f7e [0,4194304] 0 2024-04-02T04:43:08.843 INFO:tasks.workunit.client.1.smithi027.stdout:7/888: write d2/dba/dc/d16/d2f/fc2 [981298,94950] 0 2024-04-02T04:43:08.849 INFO:tasks.workunit.client.1.smithi027.stdout:6/790: dwrite d13/d1b/d24/d25/d27/f81 [0,4194304] 0 2024-04-02T04:43:08.852 INFO:tasks.workunit.client.1.smithi027.stdout:7/889: symlink d2/dba/dc/d16/d3f/d44/l11d 0 2024-04-02T04:43:08.853 INFO:tasks.workunit.client.1.smithi027.stdout:7/890: write d2/d1c/f4b [1399325,44953] 0 2024-04-02T04:43:08.862 INFO:tasks.workunit.client.1.smithi027.stdout:3/789: write d1/de/df/da5/fb2 [858791,47774] 0 2024-04-02T04:43:08.864 INFO:tasks.workunit.client.1.smithi027.stdout:6/791: fdatasync d13/d1b/d24/d25/d56/fc0 0 2024-04-02T04:43:08.872 INFO:tasks.workunit.client.1.smithi027.stdout:2/915: dwrite d1/db/d3b/d17/d6d/df1/f103 [0,4194304] 0 2024-04-02T04:43:08.873 INFO:tasks.workunit.client.1.smithi027.stdout:3/790: symlink d1/de/df/d13/d73/d77/d102/l105 0 2024-04-02T04:43:08.877 INFO:tasks.workunit.client.1.smithi027.stdout:6/792: rename d13/d1b/d48/d58/l6c to d13/l111 0 2024-04-02T04:43:08.879 INFO:tasks.workunit.client.1.smithi027.stdout:1/821: dwrite d0/de/d52/d74/db7/df1/d5d/f81 [0,4194304] 0 2024-04-02T04:43:08.890 INFO:tasks.workunit.client.1.smithi027.stdout:0/943: dwrite d4/d21/fbb [0,4194304] 0 2024-04-02T04:43:08.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:08 smithi027 ceph-mon[37550]: Standby manager daemon smithi016.eqnebx restarted 2024-04-02T04:43:08.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:08 smithi027 ceph-mon[37550]: Standby manager daemon smithi016.eqnebx started 2024-04-02T04:43:08.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:08 smithi027 ceph-mon[37550]: from='mgr.? 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi016.eqnebx/crt"}]: dispatch 2024-04-02T04:43:08.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:08 smithi027 ceph-mon[37550]: from='mgr.? 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T04:43:08.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:08 smithi027 ceph-mon[37550]: from='mgr.? 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi016.eqnebx/key"}]: dispatch 2024-04-02T04:43:08.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:08 smithi027 ceph-mon[37550]: from='mgr.? 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T04:43:08.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:08 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:43:08.897 INFO:tasks.workunit.client.1.smithi027.stdout:2/916: rename d1/d91 to d1/db/d10a/d146 0 2024-04-02T04:43:08.901 INFO:tasks.workunit.client.1.smithi027.stdout:3/791: rmdir d1/de/d55/d63/da4/de3 39 2024-04-02T04:43:08.903 INFO:tasks.workunit.client.1.smithi027.stdout:3/792: write d1/de/df/d13/d73/d77/f7a [5023036,67427] 0 2024-04-02T04:43:08.910 INFO:tasks.workunit.client.1.smithi027.stdout:2/917: readlink d1/db/d3b/d83/l90 0 2024-04-02T04:43:08.918 INFO:tasks.workunit.client.1.smithi027.stdout:8/814: write d5/d8/db/f29 [733622,42110] 0 2024-04-02T04:43:08.919 INFO:tasks.workunit.client.1.smithi027.stdout:1/822: dread d0/de/d52/d74/db7/df1/d25/d2b/fae [0,4194304] 0 2024-04-02T04:43:08.920 INFO:tasks.workunit.client.1.smithi027.stdout:3/793: dread d1/fab [0,4194304] 0 2024-04-02T04:43:08.921 INFO:tasks.workunit.client.1.smithi027.stdout:2/918: unlink d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/d106/l108 0 2024-04-02T04:43:08.922 INFO:tasks.workunit.client.1.smithi027.stdout:8/815: dread d5/d8/db/f76 [0,4194304] 0 2024-04-02T04:43:08.922 INFO:tasks.workunit.client.1.smithi027.stdout:4/758: truncate d24/d48/f75 2107651 0 2024-04-02T04:43:08.925 INFO:tasks.workunit.client.1.smithi027.stdout:0/944: dread d4/d38/d66/f6c [0,4194304] 0 2024-04-02T04:43:08.927 INFO:tasks.workunit.client.1.smithi027.stdout:8/816: read d5/d8/db/dd/d5f/d91/d9f/fb4 [3400658,6952] 0 2024-04-02T04:43:08.932 INFO:tasks.workunit.client.1.smithi027.stdout:3/794: dwrite d1/d1a/ff7 [0,4194304] 0 2024-04-02T04:43:08.932 INFO:tasks.workunit.client.1.smithi027.stdout:4/759: mknod d24/d38/d44/d82/cf5 0 2024-04-02T04:43:08.935 INFO:tasks.workunit.client.1.smithi027.stdout:7/891: dwrite d2/dba/d35/f67 [0,4194304] 0 2024-04-02T04:43:08.936 INFO:tasks.workunit.client.1.smithi027.stdout:7/892: write d2/d7/f18 [752426,79354] 0 2024-04-02T04:43:08.946 INFO:tasks.workunit.client.1.smithi027.stdout:0/945: unlink d4/d21/d28/d4e/fd0 0 2024-04-02T04:43:08.946 INFO:tasks.workunit.client.1.smithi027.stdout:7/893: dwrite d2/d7/df3/f103 [0,4194304] 0 2024-04-02T04:43:08.959 INFO:tasks.workunit.client.1.smithi027.stdout:4/760: creat d24/d38/d47/d5c/ff6 x:0 0 0 2024-04-02T04:43:08.959 INFO:tasks.workunit.client.1.smithi027.stdout:6/793: dread d13/f14 [0,4194304] 0 2024-04-02T04:43:08.963 INFO:tasks.workunit.client.1.smithi027.stdout:3/795: creat d1/de/d55/d99/f106 x:0 0 0 2024-04-02T04:43:08.964 INFO:tasks.workunit.client.1.smithi027.stdout:7/894: dread d2/dba/dc/d1f/fa1 [0,4194304] 0 2024-04-02T04:43:08.973 INFO:tasks.workunit.client.1.smithi027.stdout:1/823: rename d0/de/d52/d74/db7/df1/d25/d3c/d64/fd8 to d0/de/d52/d74/df7/d3b/f10c 0 2024-04-02T04:43:08.976 INFO:tasks.workunit.client.1.smithi027.stdout:8/817: dwrite d5/d8/db/f76 [0,4194304] 0 2024-04-02T04:43:08.981 INFO:tasks.workunit.client.1.smithi027.stdout:0/946: write d4/f90 [506455,60939] 0 2024-04-02T04:43:08.982 INFO:tasks.workunit.client.1.smithi027.stdout:0/947: stat d4/d21/d28/f113 0 2024-04-02T04:43:08.986 INFO:tasks.workunit.client.1.smithi027.stdout:4/761: write d24/d38/d44/fa5 [1301093,113794] 0 2024-04-02T04:43:08.988 INFO:tasks.workunit.client.1.smithi027.stdout:4/762: write d24/d38/d44/d41/d5a/f88 [4855435,129277] 0 2024-04-02T04:43:08.991 INFO:tasks.workunit.client.1.smithi027.stdout:6/794: mknod d13/d1b/d24/d25/d56/dc8/dc9/c112 0 2024-04-02T04:43:08.991 INFO:tasks.workunit.client.1.smithi027.stdout:6/795: stat d13/d1b/c2e 0 2024-04-02T04:43:08.996 INFO:tasks.workunit.client.1.smithi027.stdout:7/895: mkdir d2/dba/dc/d16/d3a/d4d/d11e 0 2024-04-02T04:43:08.998 INFO:tasks.workunit.client.1.smithi027.stdout:3/796: write d1/de/df/d9e/da0/fb5 [22489,14324] 0 2024-04-02T04:43:08.999 INFO:tasks.workunit.client.1.smithi027.stdout:4/763: chown d24/d38/d47/d62/l7a 7 1 2024-04-02T04:43:08.999 INFO:tasks.workunit.client.1.smithi027.stdout:3/797: chown d1/de/df/d13/lf4 2708122 1 2024-04-02T04:43:09.003 INFO:tasks.workunit.client.1.smithi027.stdout:2/919: rename d1/db/d3b/d17/d6d/dd9/dd5/dda/dbb/db4/l119 to d1/db/d3b/d17/d6d/d127/d110/l147 0 2024-04-02T04:43:09.004 INFO:tasks.workunit.client.1.smithi027.stdout:8/818: symlink d5/d8/db/dd/d5f/db2/dcf/l10d 0 2024-04-02T04:43:09.005 INFO:tasks.workunit.client.1.smithi027.stdout:2/920: fsync d1/db/d3b/d17/d6d/dd9/dd5/dda/dbb/db4/ddf/f13a 0 2024-04-02T04:43:09.007 INFO:tasks.workunit.client.1.smithi027.stdout:7/896: chown d2/dba/dc/d16/d3a/d4d/d6f/l75 18 1 2024-04-02T04:43:09.010 INFO:tasks.workunit.client.1.smithi027.stdout:7/897: truncate d2/dba/dc/d16/d3f/d50/d8b/db6/fd6 5234608 0 2024-04-02T04:43:09.011 INFO:tasks.workunit.client.1.smithi027.stdout:3/798: fsync d1/de/df/d13/f36 0 2024-04-02T04:43:09.011 INFO:tasks.workunit.client.1.smithi027.stdout:7/898: chown d2/df/dc7 3443527 1 2024-04-02T04:43:09.013 INFO:tasks.workunit.client.1.smithi027.stdout:4/764: write d24/d38/d44/d60/fa2 [602840,26318] 0 2024-04-02T04:43:09.015 INFO:tasks.workunit.client.1.smithi027.stdout:1/824: rename d0/fd to d0/de/d52/d74/dbc/f10d 0 2024-04-02T04:43:09.021 INFO:tasks.workunit.client.1.smithi027.stdout:3/799: chown d1/de/df/f5d 7 1 2024-04-02T04:43:09.023 INFO:tasks.workunit.client.1.smithi027.stdout:3/800: read d1/de/df/d13/f21 [2527692,64066] 0 2024-04-02T04:43:09.029 INFO:tasks.workunit.client.1.smithi027.stdout:0/948: rename d4/d69/f129 to d4/d38/d66/d2d/d81/f134 0 2024-04-02T04:43:09.033 INFO:tasks.workunit.client.1.smithi027.stdout:2/921: dwrite d1/db/d3b/d17/dec/f109 [0,4194304] 0 2024-04-02T04:43:09.033 INFO:tasks.workunit.client.1.smithi027.stdout:3/801: dwrite d1/f2a [0,4194304] 0 2024-04-02T04:43:09.038 INFO:tasks.workunit.client.1.smithi027.stdout:2/922: readlink d1/db/d3b/d17/d6d/dd9/dd5/l45 0 2024-04-02T04:43:09.039 INFO:tasks.workunit.client.1.smithi027.stdout:4/765: dwrite d24/d48/f7d [0,4194304] 0 2024-04-02T04:43:09.041 INFO:tasks.workunit.client.1.smithi027.stdout:4/766: fsync d24/d38/d47/fa3 0 2024-04-02T04:43:09.042 INFO:tasks.workunit.client.1.smithi027.stdout:8/819: creat d5/d8/f10e x:0 0 0 2024-04-02T04:43:09.047 INFO:tasks.workunit.client.1.smithi027.stdout:7/899: rename d2/d7/df3/d106/f55 to d2/dba/dc/d16/d3a/db2/f11f 0 2024-04-02T04:43:09.051 INFO:tasks.workunit.client.1.smithi027.stdout:1/825: mknod d0/de/d52/d74/d97/dfa/c10e 0 2024-04-02T04:43:09.051 INFO:tasks.workunit.client.1.smithi027.stdout:0/949: rmdir d4/d21/d85/dfb 39 2024-04-02T04:43:09.054 INFO:tasks.workunit.client.1.smithi027.stdout:3/802: stat d1/l90 0 2024-04-02T04:43:09.055 INFO:tasks.workunit.client.1.smithi027.stdout:3/803: stat d1/f2e 0 2024-04-02T04:43:09.056 INFO:tasks.workunit.client.1.smithi027.stdout:3/804: dread - d1/de/df/d9e/da0/fe5 zero size 2024-04-02T04:43:09.057 INFO:tasks.workunit.client.1.smithi027.stdout:2/923: creat d1/dfe/f148 x:0 0 0 2024-04-02T04:43:09.059 INFO:tasks.workunit.client.1.smithi027.stdout:7/900: dwrite d2/dba/dc/d16/d3f/d79/fec [0,4194304] 0 2024-04-02T04:43:09.059 INFO:tasks.workunit.client.1.smithi027.stdout:2/924: fdatasync d1/db/d3b/f1c 0 2024-04-02T04:43:09.072 INFO:tasks.workunit.client.1.smithi027.stdout:8/820: creat d5/d8/db/dd/d5f/db2/dc6/d106/f10f x:0 0 0 2024-04-02T04:43:09.072 INFO:tasks.workunit.client.1.smithi027.stdout:2/925: creat d1/db/d3b/d61/d75/d100/f149 x:0 0 0 2024-04-02T04:43:09.076 INFO:tasks.workunit.client.1.smithi027.stdout:8/821: chown d5/d8/db/dd/f51 178735370 1 2024-04-02T04:43:09.077 INFO:tasks.workunit.client.1.smithi027.stdout:2/926: rmdir d1/db/d10a 39 2024-04-02T04:43:09.079 INFO:tasks.workunit.client.1.smithi027.stdout:1/826: getdents d0/de/d52/d74/db7/df1/d5d 0 2024-04-02T04:43:09.086 INFO:tasks.workunit.client.1.smithi027.stdout:1/827: creat d0/de/d52/dd1/f10f x:0 0 0 2024-04-02T04:43:09.087 INFO:tasks.workunit.client.1.smithi027.stdout:1/828: chown d0/de/d52/d74/db7/df1/f3a 2661077 1 2024-04-02T04:43:09.089 INFO:tasks.workunit.client.1.smithi027.stdout:4/767: write d24/f3a [708270,83035] 0 2024-04-02T04:43:09.090 INFO:tasks.workunit.client.1.smithi027.stdout:2/927: getdents d1/db/d3b/d83 0 2024-04-02T04:43:09.090 INFO:tasks.workunit.client.1.smithi027.stdout:1/829: dread - d0/de/d52/d74/db7/df1/d25/d3c/d64/dec/ff3 zero size 2024-04-02T04:43:09.092 INFO:tasks.workunit.client.1.smithi027.stdout:4/768: write d24/d38/d47/d5c/ff6 [408933,65718] 0 2024-04-02T04:43:09.093 INFO:tasks.workunit.client.1.smithi027.stdout:1/830: read d0/de/f80 [3104408,18941] 0 2024-04-02T04:43:09.093 INFO:tasks.workunit.client.1.smithi027.stdout:2/928: stat d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/dc0 0 2024-04-02T04:43:09.097 INFO:tasks.workunit.client.1.smithi027.stdout:4/769: rmdir d24/d38/d44/d60 39 2024-04-02T04:43:09.099 INFO:tasks.workunit.client.1.smithi027.stdout:3/805: dwrite d1/de/df/d13/f43 [4194304,4194304] 0 2024-04-02T04:43:09.099 INFO:tasks.workunit.client.1.smithi027.stdout:8/822: write d5/d8/db/df/f79 [1333075,82378] 0 2024-04-02T04:43:09.102 INFO:tasks.workunit.client.1.smithi027.stdout:1/831: creat d0/de/d52/d74/db7/df1/d25/d3c/d64/f110 x:0 0 0 2024-04-02T04:43:09.102 INFO:tasks.workunit.client.1.smithi027.stdout:3/806: dread - d1/fe6 zero size 2024-04-02T04:43:09.106 INFO:tasks.workunit.client.1.smithi027.stdout:4/770: creat d24/d38/d47/ff7 x:0 0 0 2024-04-02T04:43:09.107 INFO:tasks.workunit.client.1.smithi027.stdout:2/929: write d1/f56 [3954985,117422] 0 2024-04-02T04:43:09.107 INFO:tasks.workunit.client.1.smithi027.stdout:0/950: dwrite d4/d38/fa9 [4194304,4194304] 0 2024-04-02T04:43:09.109 INFO:tasks.workunit.client.1.smithi027.stdout:2/930: write d1/f56 [3470456,115356] 0 2024-04-02T04:43:09.112 INFO:tasks.workunit.client.1.smithi027.stdout:3/807: rename d1/c57 to d1/de/df/d13/c107 0 2024-04-02T04:43:09.116 INFO:tasks.workunit.client.1.smithi027.stdout:4/771: chown d24/d38/d47/dc0/dca/ce7 35 1 2024-04-02T04:43:09.116 INFO:tasks.workunit.client.1.smithi027.stdout:1/832: symlink d0/de/d52/d74/df7/d3e/d5f/d8b/d9d/l111 0 2024-04-02T04:43:09.116 INFO:tasks.workunit.client.1.smithi027.stdout:3/808: truncate d1/fab 327602 0 2024-04-02T04:43:09.120 INFO:tasks.workunit.client.1.smithi027.stdout:4/772: write d24/d38/d44/fa5 [319851,69908] 0 2024-04-02T04:43:09.122 INFO:tasks.workunit.client.1.smithi027.stdout:0/951: rename d4/d16/f27 to d4/d21/d28/f135 0 2024-04-02T04:43:09.124 INFO:tasks.workunit.client.1.smithi027.stdout:2/931: dwrite d1/db/d3b/d17/d6d/dd9/dd5/dda/dbb/db4/ddf/f13a [0,4194304] 0 2024-04-02T04:43:09.125 INFO:tasks.workunit.client.1.smithi027.stdout:8/823: mknod d5/d8/d15/d71/d96/c110 0 2024-04-02T04:43:09.125 INFO:tasks.workunit.client.1.smithi027.stdout:0/952: stat d4/d38/d66/d2d/d81 0 2024-04-02T04:43:09.127 INFO:tasks.workunit.client.1.smithi027.stdout:1/833: dread - d0/de/fa8 zero size 2024-04-02T04:43:09.130 INFO:tasks.workunit.client.1.smithi027.stdout:3/809: dwrite d1/de/df/d9e/ffd [4194304,4194304] 0 2024-04-02T04:43:09.136 INFO:tasks.workunit.client.1.smithi027.stdout:2/932: creat d1/db/d3b/d17/d12/f14a x:0 0 0 2024-04-02T04:43:09.142 INFO:tasks.workunit.client.1.smithi027.stdout:8/824: mknod d5/d8/db/dd/d4a/c111 0 2024-04-02T04:43:09.145 INFO:tasks.workunit.client.1.smithi027.stdout:8/825: fdatasync d5/d8/db/f29 0 2024-04-02T04:43:09.146 INFO:tasks.workunit.client.1.smithi027.stdout:0/953: dwrite d4/d21/d28/f40 [0,4194304] 0 2024-04-02T04:43:09.146 INFO:tasks.workunit.client.1.smithi027.stdout:1/834: dread d0/de/f23 [0,4194304] 0 2024-04-02T04:43:09.155 INFO:tasks.workunit.client.1.smithi027.stdout:4/773: creat d24/d38/d44/d41/ff8 x:0 0 0 2024-04-02T04:43:09.155 INFO:tasks.workunit.client.1.smithi027.stdout:3/810: write d1/de/d55/d63/f91 [96936,34755] 0 2024-04-02T04:43:09.158 INFO:tasks.workunit.client.1.smithi027.stdout:2/933: rename d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/dc0/dc3 to d1/dfe/d14b 0 2024-04-02T04:43:09.160 INFO:tasks.workunit.client.1.smithi027.stdout:2/934: fdatasync d1/db/d3b/d17/d2b/f114 0 2024-04-02T04:43:09.161 INFO:tasks.workunit.client.1.smithi027.stdout:2/935: readlink d1/db/d3b/d17/d6d/l124 0 2024-04-02T04:43:09.163 INFO:tasks.workunit.client.1.smithi027.stdout:3/811: creat d1/de/df/d9e/dc6/f108 x:0 0 0 2024-04-02T04:43:09.164 INFO:tasks.workunit.client.1.smithi027.stdout:3/812: readlink d1/de/d55/l60 0 2024-04-02T04:43:09.169 INFO:tasks.workunit.client.1.smithi027.stdout:0/954: dread d4/d16/f1c [0,4194304] 0 2024-04-02T04:43:09.169 INFO:tasks.workunit.client.1.smithi027.stdout:3/813: dread d1/de/d55/fc0 [0,4194304] 0 2024-04-02T04:43:09.170 INFO:tasks.workunit.client.1.smithi027.stdout:0/955: chown d4/d76/fa8 31355330 1 2024-04-02T04:43:09.171 INFO:tasks.workunit.client.1.smithi027.stdout:1/835: fsync d0/de/d52/d74/df7/d3e/f82 0 2024-04-02T04:43:09.173 INFO:tasks.workunit.client.1.smithi027.stdout:4/774: link d24/d2d/f8a d24/d38/d47/d62/db4/ff9 0 2024-04-02T04:43:09.179 INFO:tasks.workunit.client.1.smithi027.stdout:8/826: write d5/d8/db/dd/d5f/d27/f3a [1826321,26889] 0 2024-04-02T04:43:09.181 INFO:tasks.workunit.client.1.smithi027.stdout:8/827: readlink d5/d8/db/dd/d5f/db2/dcf/l10d 0 2024-04-02T04:43:09.183 INFO:tasks.workunit.client.1.smithi027.stdout:3/814: dread d1/de/d55/d99/fad [0,4194304] 0 2024-04-02T04:43:09.213 INFO:tasks.workunit.client.1.smithi027.stdout:0/956: unlink d4/d38/d66/f2a 0 2024-04-02T04:43:09.213 INFO:tasks.workunit.client.1.smithi027.stdout:3/815: truncate d1/f8 2712522 0 2024-04-02T04:43:09.213 INFO:tasks.workunit.client.1.smithi027.stdout:2/936: link d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/de1/fbd d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/de1/d137/d13e/f14c 0 2024-04-02T04:43:09.213 INFO:tasks.workunit.client.1.smithi027.stdout:3/816: stat d1 0 2024-04-02T04:43:09.216 INFO:tasks.workunit.client.1.smithi027.stdout:3/817: symlink d1/de/df/d13/db3/l109 0 2024-04-02T04:43:09.218 INFO:tasks.workunit.client.1.smithi027.stdout:0/957: truncate d4/d21/d85/f86 529253 0 2024-04-02T04:43:09.219 INFO:tasks.workunit.client.1.smithi027.stdout:8/828: rename d5/d8/db/dd/d5f/d91/cd1 to d5/d8/db/dd/d5f/d27/d31/d5b/dc4/c112 0 2024-04-02T04:43:09.221 INFO:tasks.workunit.client.1.smithi027.stdout:0/958: fsync d4/d21/d28/d4e/d5f/fea 0 2024-04-02T04:43:09.223 INFO:tasks.workunit.client.1.smithi027.stdout:8/829: creat d5/d8/d5c/de6/f113 x:0 0 0 2024-04-02T04:43:09.223 INFO:tasks.workunit.client.1.smithi027.stdout:8/830: chown d5/d8/db/dd/d4a 0 1 2024-04-02T04:43:09.231 INFO:tasks.workunit.client.1.smithi027.stdout:2/937: rename d1/db/d3b/d17/d6d/dd9/dd5/dda/dbb to d1/db/d3b/d17/d14d 0 2024-04-02T04:43:09.236 INFO:tasks.workunit.client.1.smithi027.stdout:3/818: rename d1/de/df/d13/f85 to d1/de/df/d13/d73/de2/f10a 0 2024-04-02T04:43:09.237 INFO:tasks.workunit.client.1.smithi027.stdout:3/819: read d1/d1a/f79 [2571418,34145] 0 2024-04-02T04:43:09.238 INFO:tasks.workunit.client.1.smithi027.stdout:3/820: write d1/f2e [61475,95163] 0 2024-04-02T04:43:09.242 INFO:tasks.workunit.client.1.smithi027.stdout:0/959: creat d4/d38/d66/f136 x:0 0 0 2024-04-02T04:43:09.249 INFO:tasks.workunit.client.1.smithi027.stdout:8/831: getdents d5/d8/db/dd/d5f/d27/d31/db9/dc1 0 2024-04-02T04:43:09.252 INFO:tasks.workunit.client.1.smithi027.stdout:3/821: mknod d1/c10b 0 2024-04-02T04:43:09.253 INFO:tasks.workunit.client.1.smithi027.stdout:3/822: fdatasync d1/de/df/d13/d73/db1/fcb 0 2024-04-02T04:43:09.255 INFO:tasks.workunit.client.1.smithi027.stdout:8/832: creat d5/db5/ddc/f114 x:0 0 0 2024-04-02T04:43:09.257 INFO:tasks.workunit.client.1.smithi027.stdout:8/833: write d5/d8/db/dd/d5f/d27/d31/fd7 [282053,38339] 0 2024-04-02T04:43:09.258 INFO:tasks.workunit.client.1.smithi027.stdout:2/938: getdents d1/db/d3b/d61/dd2 0 2024-04-02T04:43:09.259 INFO:tasks.workunit.client.1.smithi027.stdout:8/834: chown d5/d8/db/dd/d5f/d91/dc2 225 1 2024-04-02T04:43:09.264 INFO:tasks.workunit.client.1.smithi027.stdout:2/939: rmdir d1/db/d3b/d61/d75 39 2024-04-02T04:43:09.264 INFO:tasks.workunit.client.1.smithi027.stdout:8/835: creat d5/d8/db/d68/f115 x:0 0 0 2024-04-02T04:43:09.266 INFO:tasks.workunit.client.1.smithi027.stdout:3/823: symlink d1/de/d55/d63/l10c 0 2024-04-02T04:43:09.269 INFO:tasks.workunit.client.1.smithi027.stdout:2/940: creat d1/db/d3b/d17/d6d/df1/f14e x:0 0 0 2024-04-02T04:43:09.270 INFO:tasks.workunit.client.1.smithi027.stdout:3/824: unlink d1/d1a/l56 0 2024-04-02T04:43:09.271 INFO:tasks.workunit.client.1.smithi027.stdout:8/836: dwrite d5/d8/db/dd/d5f/db2/dcf/ff1 [0,4194304] 0 2024-04-02T04:43:09.271 INFO:tasks.workunit.client.1.smithi027.stdout:0/960: symlink d4/d21/l137 0 2024-04-02T04:43:09.274 INFO:tasks.workunit.client.1.smithi027.stdout:8/837: chown d5/d8/db/dd/d5f/d91/d9f/cae 30597 1 2024-04-02T04:43:09.275 INFO:tasks.workunit.client.1.smithi027.stdout:2/941: creat d1/db/d3b/d17/dcf/f14f x:0 0 0 2024-04-02T04:43:09.284 INFO:tasks.workunit.client.1.smithi027.stdout:0/961: dread d4/d5d/fd2 [0,4194304] 0 2024-04-02T04:43:09.287 INFO:tasks.workunit.client.1.smithi027.stdout:2/942: mkdir d1/db/d3b/d61/d150 0 2024-04-02T04:43:09.288 INFO:tasks.workunit.client.1.smithi027.stdout:3/825: dread d1/de/df/f67 [0,4194304] 0 2024-04-02T04:43:09.291 INFO:tasks.workunit.client.1.smithi027.stdout:2/943: symlink d1/db/d3b/d17/d14d/l151 0 2024-04-02T04:43:09.292 INFO:tasks.workunit.client.1.smithi027.stdout:2/944: chown d1/db/d3b/d17/d2b/db8/fd0 29444 1 2024-04-02T04:43:09.293 INFO:tasks.workunit.client.1.smithi027.stdout:3/826: mknod d1/de/d55/d99/c10d 0 2024-04-02T04:43:09.294 INFO:tasks.workunit.client.1.smithi027.stdout:3/827: write d1/de/df/d61/fb8 [3131441,39141] 0 2024-04-02T04:43:09.295 INFO:tasks.workunit.client.1.smithi027.stdout:8/838: rename d5/d8/f10e to d5/d8/d15/d71/d96/f116 0 2024-04-02T04:43:09.297 INFO:tasks.workunit.client.1.smithi027.stdout:8/839: stat d5/d8/db/d68/d95/cbf 0 2024-04-02T04:43:09.297 INFO:tasks.workunit.client.1.smithi027.stdout:2/945: truncate d1/db/f30 6189323 0 2024-04-02T04:43:09.298 INFO:tasks.workunit.client.1.smithi027.stdout:2/946: chown d1 47 1 2024-04-02T04:43:09.298 INFO:tasks.workunit.client.1.smithi027.stdout:8/840: chown d5/d8/d15/f39 704 1 2024-04-02T04:43:09.299 INFO:tasks.workunit.client.1.smithi027.stdout:3/828: creat d1/de/df/d13/d73/db1/f10e x:0 0 0 2024-04-02T04:43:09.301 INFO:tasks.workunit.client.1.smithi027.stdout:3/829: truncate d1/de/fa7 1047299 0 2024-04-02T04:43:09.301 INFO:tasks.workunit.client.1.smithi027.stdout:0/962: rename d4/d16/f121 to d4/d21/d85/db3/f138 0 2024-04-02T04:43:09.303 INFO:tasks.workunit.client.1.smithi027.stdout:8/841: creat d5/db5/f117 x:0 0 0 2024-04-02T04:43:09.310 INFO:tasks.workunit.client.1.smithi027.stdout:8/842: truncate d5/d8/d15/f37 2333808 0 2024-04-02T04:43:09.310 INFO:tasks.workunit.client.1.smithi027.stdout:8/843: fdatasync d5/d8/db/d68/f115 0 2024-04-02T04:43:09.316 INFO:tasks.workunit.client.1.smithi027.stdout:2/947: write d1/db/d3b/d17/d6d/dd9/dd5/fa6 [407871,50027] 0 2024-04-02T04:43:09.317 INFO:tasks.workunit.client.1.smithi027.stdout:0/963: mkdir d4/d21/d85/d139 0 2024-04-02T04:43:09.318 INFO:tasks.workunit.client.1.smithi027.stdout:8/844: mknod d5/d8/db/dd/d5f/db2/dcf/c118 0 2024-04-02T04:43:09.332 INFO:tasks.workunit.client.1.smithi027.stdout:3/830: dwrite d1/de/df/f6b [0,4194304] 0 2024-04-02T04:43:09.332 INFO:tasks.workunit.client.1.smithi027.stdout:2/948: fdatasync d1/db/d3b/d17/d2b/f53 0 2024-04-02T04:43:09.334 INFO:tasks.workunit.client.1.smithi027.stdout:2/949: truncate d1/dfe/f148 874028 0 2024-04-02T04:43:09.335 INFO:tasks.workunit.client.1.smithi027.stdout:0/964: dread d4/d21/d28/d4e/fbd [0,4194304] 0 2024-04-02T04:43:09.336 INFO:tasks.workunit.client.1.smithi027.stdout:2/950: chown d1/db/d3b/d17/dec/f109 136309058 1 2024-04-02T04:43:09.340 INFO:tasks.workunit.client.1.smithi027.stdout:3/831: mknod d1/de/df/d9e/dc6/c10f 0 2024-04-02T04:43:09.346 INFO:tasks.workunit.client.1.smithi027.stdout:2/951: mknod d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/c152 0 2024-04-02T04:43:09.350 INFO:tasks.workunit.client.1.smithi027.stdout:8/845: creat d5/d8/db/dd/d5f/d27/d31/d5b/dc4/dea/f119 x:0 0 0 2024-04-02T04:43:09.351 INFO:tasks.workunit.client.1.smithi027.stdout:8/846: readlink d5/d8/db/l2a 0 2024-04-02T04:43:09.351 INFO:tasks.workunit.client.1.smithi027.stdout:8/847: write d5/d8/f2e [1696592,49523] 0 2024-04-02T04:43:09.357 INFO:tasks.workunit.client.1.smithi027.stdout:2/952: creat d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/d10d/d11f/f153 x:0 0 0 2024-04-02T04:43:09.358 INFO:tasks.workunit.client.1.smithi027.stdout:8/848: dread d5/d8/db/fca [0,4194304] 0 2024-04-02T04:43:09.359 INFO:tasks.workunit.client.1.smithi027.stdout:0/965: creat d4/d21/d85/f13a x:0 0 0 2024-04-02T04:43:09.369 INFO:tasks.workunit.client.1.smithi027.stdout:0/966: creat d4/d69/d122/f13b x:0 0 0 2024-04-02T04:43:09.369 INFO:tasks.workunit.client.1.smithi027.stdout:2/953: truncate d1/db/f30 5080196 0 2024-04-02T04:43:09.369 INFO:tasks.workunit.client.1.smithi027.stdout:2/954: read d1/db/d3b/d17/d2b/db8/fd0 [199941,21488] 0 2024-04-02T04:43:09.370 INFO:tasks.workunit.client.1.smithi027.stdout:2/955: read d1/db/d3b/d17/d6d/dd9/dd5/dda/f13c [2700194,81095] 0 2024-04-02T04:43:09.370 INFO:tasks.workunit.client.1.smithi027.stdout:2/956: chown d1/dfe 131120 1 2024-04-02T04:43:09.376 INFO:tasks.workunit.client.1.smithi027.stdout:3/832: truncate d1/de/f30 2477821 0 2024-04-02T04:43:09.377 INFO:tasks.workunit.client.1.smithi027.stdout:3/833: write d1/de/df/d13/d73/d77/fc4 [174150,80392] 0 2024-04-02T04:43:09.378 INFO:tasks.workunit.client.1.smithi027.stdout:3/834: write d1/fe6 [877839,57094] 0 2024-04-02T04:43:09.380 INFO:tasks.workunit.client.1.smithi027.stdout:3/835: write d1/de/fa7 [1801673,80720] 0 2024-04-02T04:43:09.382 INFO:tasks.workunit.client.1.smithi027.stdout:2/957: creat d1/db/d3b/d17/d6d/dd9/dd5/d125/f154 x:0 0 0 2024-04-02T04:43:09.390 INFO:tasks.workunit.client.1.smithi027.stdout:3/836: mkdir d1/d110 0 2024-04-02T04:43:09.392 INFO:tasks.workunit.client.1.smithi027.stdout:3/837: write d1/de/d55/f5f [2879435,99671] 0 2024-04-02T04:43:09.393 INFO:tasks.workunit.client.1.smithi027.stdout:8/849: write d5/d8/db/dd/d5f/d91/fb3 [212129,116293] 0 2024-04-02T04:43:09.394 INFO:tasks.workunit.client.1.smithi027.stdout:0/967: link d4/d76/f99 d4/d21/d85/dfb/f13c 0 2024-04-02T04:43:09.396 INFO:tasks.workunit.client.1.smithi027.stdout:8/850: creat d5/d8/db/d68/f11a x:0 0 0 2024-04-02T04:43:09.398 INFO:tasks.workunit.client.1.smithi027.stdout:3/838: mknod d1/de/d55/d99/c111 0 2024-04-02T04:43:09.399 INFO:tasks.workunit.client.1.smithi027.stdout:3/839: write d1/d1a/da6/fc8 [3264146,69884] 0 2024-04-02T04:43:09.400 INFO:tasks.workunit.client.1.smithi027.stdout:0/968: dread - d4/d21/d28/fde zero size 2024-04-02T04:43:09.400 INFO:tasks.workunit.client.1.smithi027.stdout:3/840: readlink d1/de/d55/l88 0 2024-04-02T04:43:09.400 INFO:tasks.workunit.client.1.smithi027.stdout:5/950: sync 2024-04-02T04:43:09.402 INFO:tasks.workunit.client.1.smithi027.stdout:3/841: chown d1/de/df/d61/cd9 222558352 1 2024-04-02T04:43:09.403 INFO:tasks.workunit.client.1.smithi027.stdout:8/851: symlink d5/d8/db/dd/d5f/d91/l11b 0 2024-04-02T04:43:09.403 INFO:tasks.workunit.client.1.smithi027.stdout:8/852: dread - d5/db5/f117 zero size 2024-04-02T04:43:09.406 INFO:tasks.workunit.client.1.smithi027.stdout:0/969: fdatasync d4/d16/d1b/f20 0 2024-04-02T04:43:09.407 INFO:tasks.workunit.client.1.smithi027.stdout:5/951: creat d0/df/d11/d37/d120/f13a x:0 0 0 2024-04-02T04:43:09.411 INFO:tasks.workunit.client.1.smithi027.stdout:0/970: chown d4/l1a 13 1 2024-04-02T04:43:09.414 INFO:tasks.workunit.client.1.smithi027.stdout:3/842: dread d1/de/df/f5d [0,4194304] 0 2024-04-02T04:43:09.414 INFO:tasks.workunit.client.1.smithi027.stdout:3/843: readlink d1/l23 0 2024-04-02T04:43:09.418 INFO:tasks.workunit.client.1.smithi027.stdout:0/971: truncate d4/d21/d85/fae 703956 0 2024-04-02T04:43:09.423 INFO:tasks.workunit.client.1.smithi027.stdout:0/972: mkdir d4/d76/d13d 0 2024-04-02T04:43:09.423 INFO:tasks.workunit.client.1.smithi027.stdout:5/952: dread d0/df/d11/d13/d9a/db7/fdc [0,4194304] 0 2024-04-02T04:43:09.424 INFO:tasks.workunit.client.1.smithi027.stdout:5/953: readlink d0/df/d11/d41/l70 0 2024-04-02T04:43:09.440 INFO:tasks.workunit.client.1.smithi027.stdout:5/954: link d0/d68/d6b/d74/la7 d0/df/d11/d13/dce/d55/l13b 0 2024-04-02T04:43:09.441 INFO:tasks.workunit.client.1.smithi027.stdout:2/958: write d1/db/d3b/d17/d6d/dd9/dd5/dda/d8a/f10e [314346,30623] 0 2024-04-02T04:43:09.447 INFO:tasks.workunit.client.1.smithi027.stdout:5/955: rmdir d0/df/d11/d8c 39 2024-04-02T04:43:09.464 INFO:tasks.workunit.client.1.smithi027.stdout:8/853: write d5/d8/db/dd/fda [791770,129365] 0 2024-04-02T04:43:09.473 INFO:tasks.workunit.client.1.smithi027.stdout:6/796: sync 2024-04-02T04:43:09.473 INFO:tasks.workunit.client.1.smithi027.stdout:7/901: sync 2024-04-02T04:43:09.481 INFO:tasks.workunit.client.1.smithi027.stdout:7/902: dwrite d2/dba/d35/f8c [0,4194304] 0 2024-04-02T04:43:09.483 INFO:tasks.workunit.client.1.smithi027.stdout:3/844: dwrite d1/de/d55/fc0 [0,4194304] 0 2024-04-02T04:43:09.483 INFO:tasks.workunit.client.1.smithi027.stdout:7/903: fdatasync d2/dba/dc/fc3 0 2024-04-02T04:43:09.484 INFO:tasks.workunit.client.1.smithi027.stdout:8/854: link d5/d8/db/df/l103 d5/d8/db/d68/l11c 0 2024-04-02T04:43:09.484 INFO:tasks.workunit.client.1.smithi027.stdout:8/855: readlink d5/lce 0 2024-04-02T04:43:09.486 INFO:tasks.workunit.client.1.smithi027.stdout:0/973: write d4/d21/fb2 [5079942,120163] 0 2024-04-02T04:43:09.488 INFO:tasks.workunit.client.1.smithi027.stdout:0/974: stat d4/d16/d1b/cca 0 2024-04-02T04:43:09.494 INFO:tasks.workunit.client.1.smithi027.stdout:3/845: mknod d1/d1a/c112 0 2024-04-02T04:43:09.495 INFO:tasks.workunit.client.1.smithi027.stdout:3/846: chown d1/de/df/d9e/cb6 880 1 2024-04-02T04:43:09.496 INFO:tasks.workunit.client.1.smithi027.stdout:6/797: unlink d13/d8d/fa4 0 2024-04-02T04:43:09.499 INFO:tasks.workunit.client.1.smithi027.stdout:5/956: write d0/df/d11/d13/d9a/d9f/f10a [402765,76074] 0 2024-04-02T04:43:09.500 INFO:tasks.workunit.client.1.smithi027.stdout:7/904: creat d2/dba/dc/d16/d2f/dcd/f120 x:0 0 0 2024-04-02T04:43:09.503 INFO:tasks.workunit.client.1.smithi027.stdout:7/905: chown d2/dba/dc/d16/d3f/d50/fea 17148 1 2024-04-02T04:43:09.504 INFO:tasks.workunit.client.1.smithi027.stdout:2/959: dwrite d1/db/d3b/d83/fb0 [0,4194304] 0 2024-04-02T04:43:09.507 INFO:tasks.workunit.client.1.smithi027.stdout:8/856: mknod d5/d8/db/dd/d5f/d27/d31/c11d 0 2024-04-02T04:43:09.512 INFO:tasks.workunit.client.1.smithi027.stdout:6/798: rename d13/c3b to d13/d1b/d24/d25/d56/dc8/c113 0 2024-04-02T04:43:09.514 INFO:tasks.workunit.client.1.smithi027.stdout:2/960: dwrite d1/db/d3b/d17/d68/f94 [0,4194304] 0 2024-04-02T04:43:09.517 INFO:tasks.workunit.client.1.smithi027.stdout:3/847: creat d1/de/d55/d63/f113 x:0 0 0 2024-04-02T04:43:09.520 INFO:tasks.workunit.client.1.smithi027.stdout:5/957: write d0/df/d11/d65/fdd [3705682,4876] 0 2024-04-02T04:43:09.523 INFO:tasks.workunit.client.1.smithi027.stdout:7/906: rename d2/dba/dc/d16/d2f/dcd/f10a to d2/d7/df3/d106/f121 0 2024-04-02T04:43:09.524 INFO:tasks.workunit.client.1.smithi027.stdout:5/958: truncate d0/df/d11/d13/d9a/d9c/dbe/feb 4932142 0 2024-04-02T04:43:09.529 INFO:tasks.workunit.client.1.smithi027.stdout:8/857: mkdir d5/d8/db/dd/d11e 0 2024-04-02T04:43:09.534 INFO:tasks.workunit.client.1.smithi027.stdout:3/848: stat d1/de/d55/d63/db9/lba 0 2024-04-02T04:43:09.536 INFO:tasks.workunit.client.1.smithi027.stdout:8/858: dwrite d5/d8/db/dd/f10 [0,4194304] 0 2024-04-02T04:43:09.537 INFO:tasks.workunit.client.1.smithi027.stdout:5/959: mkdir d0/d1d/d13c 0 2024-04-02T04:43:09.539 INFO:tasks.workunit.client.1.smithi027.stdout:2/961: mkdir d1/db/d3b/d17/d155 0 2024-04-02T04:43:09.546 INFO:tasks.workunit.client.1.smithi027.stdout:8/859: mknod d5/d8/db/df/c11f 0 2024-04-02T04:43:09.546 INFO:tasks.workunit.client.1.smithi027.stdout:2/962: dwrite d1/db/d3b/d17/d12/f65 [0,4194304] 0 2024-04-02T04:43:09.548 INFO:tasks.workunit.client.1.smithi027.stdout:5/960: symlink d0/df/d29/da5/da6/l13d 0 2024-04-02T04:43:09.549 INFO:tasks.workunit.client.1.smithi027.stdout:5/961: chown d0/df/d11/d41/f90 330040771 1 2024-04-02T04:43:09.550 INFO:tasks.workunit.client.1.smithi027.stdout:5/962: stat d0/d1d/f35 0 2024-04-02T04:43:09.550 INFO:tasks.workunit.client.1.smithi027.stdout:5/963: stat d0/f6 0 2024-04-02T04:43:09.555 INFO:tasks.workunit.client.1.smithi027.stdout:3/849: rename d1/de/df/c27 to d1/de/df/d9e/da0/c114 0 2024-04-02T04:43:09.556 INFO:tasks.workunit.client.1.smithi027.stdout:8/860: read - d5/d8/db/dd/d5f/d27/f105 zero size 2024-04-02T04:43:09.556 INFO:tasks.workunit.client.1.smithi027.stdout:3/850: dread - d1/d1a/fed zero size 2024-04-02T04:43:09.558 INFO:tasks.workunit.client.1.smithi027.stdout:5/964: symlink d0/df/d2f/d3b/d51/l13e 0 2024-04-02T04:43:09.559 INFO:tasks.workunit.client.1.smithi027.stdout:5/965: write d0/d1d/d61/d7e/faf [2828811,75165] 0 2024-04-02T04:43:09.560 INFO:tasks.workunit.client.1.smithi027.stdout:6/799: unlink d13/d1b/d24/d25/d56/cba 0 2024-04-02T04:43:09.561 INFO:tasks.workunit.client.1.smithi027.stdout:5/966: write d0/f47 [5188843,121909] 0 2024-04-02T04:43:09.562 INFO:tasks.workunit.client.1.smithi027.stdout:7/907: getdents d2/dba/dc/d16/d3a/d4d/dcb 0 2024-04-02T04:43:09.562 INFO:tasks.workunit.client.1.smithi027.stdout:6/800: readlink d13/d1b/d48/d58/lf1 0 2024-04-02T04:43:09.565 INFO:tasks.workunit.client.1.smithi027.stdout:8/861: symlink d5/d8/db/dd/d5f/d91/d9f/l120 0 2024-04-02T04:43:09.568 INFO:tasks.workunit.client.1.smithi027.stdout:8/862: dread d5/d8/f17 [0,4194304] 0 2024-04-02T04:43:09.570 INFO:tasks.workunit.client.1.smithi027.stdout:5/967: symlink d0/df/d2f/d3b/dca/l13f 0 2024-04-02T04:43:09.570 INFO:tasks.workunit.client.1.smithi027.stdout:6/801: stat d13/f17 0 2024-04-02T04:43:09.573 INFO:tasks.workunit.client.1.smithi027.stdout:3/851: unlink d1/de/f42 0 2024-04-02T04:43:09.574 INFO:tasks.workunit.client.1.smithi027.stdout:8/863: creat d5/d8/d15/d71/db0/f121 x:0 0 0 2024-04-02T04:43:09.577 INFO:tasks.workunit.client.1.smithi027.stdout:6/802: truncate d13/d1b/d24/d28/f65 874713 0 2024-04-02T04:43:09.579 INFO:tasks.workunit.client.1.smithi027.stdout:3/852: dwrite d1/de/df/f6b [0,4194304] 0 2024-04-02T04:43:09.585 INFO:tasks.workunit.client.1.smithi027.stdout:7/908: link d2/d7/df3/d106/f71 d2/df/d89/dff/f122 0 2024-04-02T04:43:09.587 INFO:tasks.workunit.client.1.smithi027.stdout:8/864: symlink d5/d8/db/dd/d5f/l122 0 2024-04-02T04:43:09.588 INFO:tasks.workunit.client.1.smithi027.stdout:5/968: creat d0/df/d11/d65/f140 x:0 0 0 2024-04-02T04:43:09.589 INFO:tasks.workunit.client.1.smithi027.stdout:6/803: dwrite d13/d1b/d48/f10b [0,4194304] 0 2024-04-02T04:43:09.590 INFO:tasks.workunit.client.1.smithi027.stdout:5/969: fdatasync d0/f6 0 2024-04-02T04:43:09.593 INFO:tasks.workunit.client.1.smithi027.stdout:5/970: dread - d0/df/d2f/d3b/d105/f118 zero size 2024-04-02T04:43:09.597 INFO:tasks.workunit.client.1.smithi027.stdout:8/865: dread d5/d8/db/dd5/fde [0,4194304] 0 2024-04-02T04:43:09.599 INFO:tasks.workunit.client.1.smithi027.stdout:8/866: chown d5/d8/d5c/lb6 36255382 1 2024-04-02T04:43:09.604 INFO:tasks.workunit.client.1.smithi027.stdout:5/971: truncate d0/df/f80 1790473 0 2024-04-02T04:43:09.609 INFO:tasks.workunit.client.1.smithi027.stdout:7/909: creat d2/dba/dc/d16/f123 x:0 0 0 2024-04-02T04:43:09.610 INFO:tasks.workunit.client.1.smithi027.stdout:7/910: fsync d2/d1c/f4b 0 2024-04-02T04:43:09.611 INFO:tasks.workunit.client.1.smithi027.stdout:5/972: symlink d0/d1d/d61/d7e/dff/l141 0 2024-04-02T04:43:09.619 INFO:tasks.workunit.client.1.smithi027.stdout:7/911: mknod d2/c124 0 2024-04-02T04:43:09.622 INFO:tasks.workunit.client.1.smithi027.stdout:3/853: dwrite d1/de/df/f14 [4194304,4194304] 0 2024-04-02T04:43:09.624 INFO:tasks.workunit.client.1.smithi027.stdout:3/854: chown d1/de/f58 1964024 1 2024-04-02T04:43:09.624 INFO:tasks.workunit.client.1.smithi027.stdout:8/867: write d5/d8/db/df/f22 [3736051,16366] 0 2024-04-02T04:43:09.627 INFO:tasks.workunit.client.1.smithi027.stdout:3/855: write d1/de/df/d13/d73/db1/fcf [270225,85775] 0 2024-04-02T04:43:09.631 INFO:tasks.workunit.client.1.smithi027.stdout:6/804: dwrite d13/d1b/d24/d25/d27/f3f [0,4194304] 0 2024-04-02T04:43:09.631 INFO:tasks.workunit.client.1.smithi027.stdout:3/856: write d1/de/df/d13/d73/d77/d102/ff5 [503607,128193] 0 2024-04-02T04:43:09.632 INFO:tasks.workunit.client.1.smithi027.stdout:7/912: rename d2/dba/d35/f67 to d2/dba/dc/d16/d2f/dcd/df8/f125 0 2024-04-02T04:43:09.635 INFO:tasks.workunit.client.1.smithi027.stdout:7/913: readlink d2/dba/dc/d16/d3f/dc0/ld8 0 2024-04-02T04:43:09.635 INFO:tasks.workunit.client.1.smithi027.stdout:8/868: mkdir d5/d8/db/dd/d5f/db2/d10b/d123 0 2024-04-02T04:43:09.636 INFO:tasks.workunit.client.1.smithi027.stdout:6/805: write d13/d1b/d24/f3a [3063131,41530] 0 2024-04-02T04:43:09.637 INFO:tasks.workunit.client.1.smithi027.stdout:6/806: write f7 [2214505,29394] 0 2024-04-02T04:43:09.640 INFO:tasks.workunit.client.1.smithi027.stdout:5/973: link d0/df/d2f/d3b/d51/dc0/f119 d0/df/d11/d8c/dd1/f142 0 2024-04-02T04:43:09.643 INFO:tasks.workunit.client.1.smithi027.stdout:8/869: truncate d5/db5/ddc/ff4 949065 0 2024-04-02T04:43:09.644 INFO:tasks.workunit.client.1.smithi027.stdout:6/807: read - d13/d1b/d48/d9c/ff3 zero size 2024-04-02T04:43:09.647 INFO:tasks.workunit.client.1.smithi027.stdout:6/808: write f12 [2380318,130931] 0 2024-04-02T04:43:09.648 INFO:tasks.workunit.client.1.smithi027.stdout:7/914: dwrite d2/dba/dc/d16/d3f/d50/d8b/db6/fd9 [0,4194304] 0 2024-04-02T04:43:09.648 INFO:tasks.workunit.client.1.smithi027.stdout:6/809: stat d13/d1b/d24/fd1 0 2024-04-02T04:43:09.658 INFO:tasks.workunit.client.1.smithi027.stdout:3/857: dread d1/de/df/d61/fb8 [0,4194304] 0 2024-04-02T04:43:09.661 INFO:tasks.workunit.client.1.smithi027.stdout:7/915: readlink d2/d1c/l114 0 2024-04-02T04:43:09.663 INFO:tasks.workunit.client.1.smithi027.stdout:6/810: rmdir d13/d1b/d24/d25/d56 39 2024-04-02T04:43:09.663 INFO:tasks.workunit.client.1.smithi027.stdout:3/858: dread d1/f2a [0,4194304] 0 2024-04-02T04:43:09.672 INFO:tasks.workunit.client.1.smithi027.stdout:3/859: read d1/d1a/fbd [651939,69516] 0 2024-04-02T04:43:09.672 INFO:tasks.workunit.client.1.smithi027.stdout:5/974: creat d0/d1d/d61/d72/d134/f143 x:0 0 0 2024-04-02T04:43:09.674 INFO:tasks.workunit.client.1.smithi027.stdout:5/975: read - d0/df/d11/d65/f140 zero size 2024-04-02T04:43:09.674 INFO:tasks.workunit.client.1.smithi027.stdout:6/811: read d13/d1b/d48/d9c/df7/d96/fac [1392611,3146] 0 2024-04-02T04:43:09.689 INFO:tasks.workunit.client.1.smithi027.stdout:1/836: sync 2024-04-02T04:43:09.689 INFO:tasks.workunit.client.1.smithi027.stdout:4/775: sync 2024-04-02T04:43:09.691 INFO:tasks.workunit.client.1.smithi027.stdout:1/837: dread - d0/de/d52/d74/db7/df1/d25/d3c/d64/dec/f10b zero size 2024-04-02T04:43:09.693 INFO:tasks.workunit.client.1.smithi027.stdout:1/838: truncate d0/de/d52/d74/dbc/f105 952198 0 2024-04-02T04:43:09.696 INFO:tasks.workunit.client.1.smithi027.stdout:3/860: rename d1/de/df/d13/lf4 to d1/de/df/d13/d73/db1/l115 0 2024-04-02T04:43:09.699 INFO:tasks.workunit.client.1.smithi027.stdout:6/812: read d13/fbf [1975282,6574] 0 2024-04-02T04:43:09.702 INFO:tasks.workunit.client.1.smithi027.stdout:8/870: getdents d5/d8/db/d68/d95 0 2024-04-02T04:43:09.706 INFO:tasks.workunit.client.1.smithi027.stdout:7/916: creat d2/dba/dc/f126 x:0 0 0 2024-04-02T04:43:09.709 INFO:tasks.workunit.client.1.smithi027.stdout:1/839: dread - d0/de/d52/d74/df7/d3e/fcb zero size 2024-04-02T04:43:09.710 INFO:tasks.workunit.client.1.smithi027.stdout:3/861: chown d1/de/df/d13/c107 91 1 2024-04-02T04:43:09.717 INFO:tasks.workunit.client.1.smithi027.stdout:1/840: dwrite d0/de/d52/d74/df7/d3e/d5f/d8b/d9d/f108 [0,4194304] 0 2024-04-02T04:43:09.717 INFO:tasks.workunit.client.1.smithi027.stdout:7/917: creat d2/dba/dc/d16/d2f/dcd/df8/f127 x:0 0 0 2024-04-02T04:43:09.718 INFO:tasks.workunit.client.1.smithi027.stdout:3/862: creat d1/de/df/d13/d73/db1/f116 x:0 0 0 2024-04-02T04:43:09.724 INFO:tasks.workunit.client.1.smithi027.stdout:1/841: readlink d0/l4e 0 2024-04-02T04:43:09.730 INFO:tasks.workunit.client.1.smithi027.stdout:7/918: dread d2/df/f85 [0,4194304] 0 2024-04-02T04:43:09.731 INFO:tasks.workunit.client.1.smithi027.stdout:5/976: write d0/df/d11/d13/d9a/d9c/dbe/fd2 [523741,113472] 0 2024-04-02T04:43:09.734 INFO:tasks.workunit.client.1.smithi027.stdout:1/842: dwrite d0/f6 [4194304,4194304] 0 2024-04-02T04:43:09.740 INFO:tasks.workunit.client.1.smithi027.stdout:7/919: getdents d2/df/d89/dde 0 2024-04-02T04:43:09.742 INFO:tasks.workunit.client.1.smithi027.stdout:3/863: dread d1/f32 [0,4194304] 0 2024-04-02T04:43:09.742 INFO:tasks.workunit.client.1.smithi027.stdout:5/977: mkdir d0/df/d11/d13/d9a/db7/d144 0 2024-04-02T04:43:09.743 INFO:tasks.workunit.client.1.smithi027.stdout:8/871: dread d5/d8/d15/d71/d96/fcc [0,4194304] 0 2024-04-02T04:43:09.743 INFO:tasks.workunit.client.1.smithi027.stdout:4/776: dwrite d24/d38/d47/d5c/f8c [0,4194304] 0 2024-04-02T04:43:09.746 INFO:tasks.workunit.client.1.smithi027.stdout:1/843: truncate d0/de/d52/d74/dbc/f105 1588433 0 2024-04-02T04:43:09.749 INFO:tasks.workunit.client.1.smithi027.stdout:7/920: creat d2/d1d/f128 x:0 0 0 2024-04-02T04:43:09.750 INFO:tasks.workunit.client.1.smithi027.stdout:7/921: chown d2/dba/dc 21 1 2024-04-02T04:43:09.752 INFO:tasks.workunit.client.1.smithi027.stdout:4/777: read d24/d38/d44/d60/fa2 [617246,110179] 0 2024-04-02T04:43:09.756 INFO:tasks.workunit.client.1.smithi027.stdout:3/864: rename d1/de/df/da5/fb2 to d1/de/df/d9e/dc6/de4/f117 0 2024-04-02T04:43:09.756 INFO:tasks.workunit.client.1.smithi027.stdout:8/872: dread d5/d8/db/dd/d5f/d27/d31/f89 [0,4194304] 0 2024-04-02T04:43:09.757 INFO:tasks.workunit.client.1.smithi027.stdout:1/844: mknod d0/de/d52/d76/d109/c112 0 2024-04-02T04:43:09.759 INFO:tasks.workunit.client.1.smithi027.stdout:1/845: chown d0/de/d52/d74/db7/df1/d25/d3c/d64 81052 1 2024-04-02T04:43:09.759 INFO:tasks.workunit.client.1.smithi027.stdout:4/778: read d24/d38/d44/d82/d95/fc2 [329787,71939] 0 2024-04-02T04:43:09.760 INFO:tasks.workunit.client.1.smithi027.stdout:3/865: write d1/f32 [3286856,121612] 0 2024-04-02T04:43:09.762 INFO:tasks.workunit.client.1.smithi027.stdout:7/922: unlink d2/d1d/d90/lf5 0 2024-04-02T04:43:09.766 INFO:tasks.workunit.client.1.smithi027.stdout:4/779: fdatasync d24/f63 0 2024-04-02T04:43:09.767 INFO:tasks.workunit.client.1.smithi027.stdout:4/780: chown d24/f2c 0 1 2024-04-02T04:43:09.769 INFO:tasks.workunit.client.1.smithi027.stdout:3/866: symlink d1/de/df/d9e/da0/l118 0 2024-04-02T04:43:09.769 INFO:tasks.workunit.client.1.smithi027.stdout:7/923: dwrite d2/dba/dc/d16/fbd [0,4194304] 0 2024-04-02T04:43:09.777 INFO:tasks.workunit.client.1.smithi027.stdout:4/781: mknod d24/d48/db0/dea/cfa 0 2024-04-02T04:43:09.778 INFO:tasks.workunit.client.1.smithi027.stdout:3/867: mknod d1/de/d55/d63/da4/c119 0 2024-04-02T04:43:09.780 INFO:tasks.workunit.client.1.smithi027.stdout:5/978: dwrite d0/df/d11/d8c/f11f [0,4194304] 0 2024-04-02T04:43:09.780 INFO:tasks.workunit.client.1.smithi027.stdout:3/868: fsync d1/d1a/fed 0 2024-04-02T04:43:09.782 INFO:tasks.workunit.client.1.smithi027.stdout:3/869: dread d1/f94 [0,4194304] 0 2024-04-02T04:43:09.790 INFO:tasks.workunit.client.1.smithi027.stdout:7/924: dread d2/dba/dc/f30 [0,4194304] 0 2024-04-02T04:43:09.793 INFO:tasks.workunit.client.1.smithi027.stdout:7/925: dread d2/d1c/f3b [4194304,4194304] 0 2024-04-02T04:43:09.809 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:09 smithi016 ceph-mon[28969]: pgmap v33: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 58 MiB/s rd, 146 MiB/s wr, 348 op/s 2024-04-02T04:43:09.809 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:09 smithi016 ceph-mon[28969]: mgrmap e26: smithi027.kjzcsc(active, since 60s), standbys: smithi016.eqnebx 2024-04-02T04:43:09.836 INFO:tasks.workunit.client.1.smithi027.stdout:1/846: dwrite d0/f54 [4194304,4194304] 0 2024-04-02T04:43:09.836 INFO:tasks.workunit.client.1.smithi027.stdout:7/926: write d2/dba/dc/d16/d3f/d50/f69 [1247566,2566] 0 2024-04-02T04:43:09.837 INFO:tasks.workunit.client.1.smithi027.stdout:5/979: truncate d0/df/d2f/f67 1191251 0 2024-04-02T04:43:09.850 INFO:tasks.workunit.client.1.smithi027.stdout:3/870: dwrite d1/de/df/f5d [4194304,4194304] 0 2024-04-02T04:43:09.851 INFO:tasks.workunit.client.1.smithi027.stdout:7/927: creat d2/dba/dc/d16/d2f/de5/f129 x:0 0 0 2024-04-02T04:43:09.853 INFO:tasks.workunit.client.1.smithi027.stdout:1/847: creat d0/de/d52/d74/df7/d3b/d4f/d9b/f113 x:0 0 0 2024-04-02T04:43:09.854 INFO:tasks.workunit.client.1.smithi027.stdout:1/848: chown d0/de/d52/f65 0 1 2024-04-02T04:43:09.855 INFO:tasks.workunit.client.1.smithi027.stdout:5/980: write d0/d1d/f79 [2184472,30837] 0 2024-04-02T04:43:09.856 INFO:tasks.workunit.client.1.smithi027.stdout:5/981: fdatasync d0/df/d11/d13/d9a/d9c/dbe/feb 0 2024-04-02T04:43:09.862 INFO:tasks.workunit.client.1.smithi027.stdout:1/849: symlink d0/de/d52/d74/df7/d3b/d4f/d9b/l114 0 2024-04-02T04:43:09.862 INFO:tasks.workunit.client.1.smithi027.stdout:1/850: fsync d0/f6 0 2024-04-02T04:43:09.865 INFO:tasks.workunit.client.1.smithi027.stdout:1/851: write d0/de/d52/d74/df7/d3b/d4f/d9b/f113 [344249,106586] 0 2024-04-02T04:43:09.875 INFO:tasks.workunit.client.1.smithi027.stdout:7/928: dread d2/dba/dc/d16/d3f/d44/fb4 [0,4194304] 0 2024-04-02T04:43:09.875 INFO:tasks.workunit.client.1.smithi027.stdout:1/852: write d0/de/d52/d74/df7/d3e/f6e [570330,101134] 0 2024-04-02T04:43:09.877 INFO:tasks.workunit.client.1.smithi027.stdout:7/929: chown d2/d1d/d92 129916 1 2024-04-02T04:43:09.879 INFO:tasks.workunit.client.1.smithi027.stdout:5/982: dwrite d0/df/d11/d13/f28 [0,4194304] 0 2024-04-02T04:43:09.879 INFO:tasks.workunit.client.1.smithi027.stdout:7/930: readlink d2/d1d/d92/lce 0 2024-04-02T04:43:09.880 INFO:tasks.workunit.client.1.smithi027.stdout:3/871: dread d1/de/df/d13/f2d [0,4194304] 0 2024-04-02T04:43:09.883 INFO:tasks.workunit.client.1.smithi027.stdout:1/853: mkdir d0/de/d52/d74/df7/d3b/d4f/d9b/d115 0 2024-04-02T04:43:09.886 INFO:tasks.workunit.client.1.smithi027.stdout:7/931: mkdir d2/d1d/dd7/d12a 0 2024-04-02T04:43:09.887 INFO:tasks.workunit.client.1.smithi027.stdout:7/932: chown d2/dba/dc/d16/d3a/d4d/d6f/l75 6519 1 2024-04-02T04:43:09.889 INFO:tasks.workunit.client.1.smithi027.stdout:3/872: chown d1/de/df/d13/d73/db1/l115 10 1 2024-04-02T04:43:09.889 INFO:tasks.workunit.client.1.smithi027.stdout:5/983: mkdir d0/df/d11/d13/d9a/d9c/dd0/d145 0 2024-04-02T04:43:09.891 INFO:tasks.workunit.client.1.smithi027.stdout:1/854: truncate d0/de/f17 2084998 0 2024-04-02T04:43:09.895 INFO:tasks.workunit.client.1.smithi027.stdout:3/873: truncate d1/de/df/d13/d73/db1/f10e 140645 0 2024-04-02T04:43:09.895 INFO:tasks.workunit.client.1.smithi027.stdout:7/933: dwrite d2/d1c/f8a [0,4194304] 0 2024-04-02T04:43:09.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:09 smithi027 ceph-mon[37550]: pgmap v33: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 58 MiB/s rd, 146 MiB/s wr, 348 op/s 2024-04-02T04:43:09.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:09 smithi027 ceph-mon[37550]: mgrmap e26: smithi027.kjzcsc(active, since 60s), standbys: smithi016.eqnebx 2024-04-02T04:43:09.896 INFO:tasks.workunit.client.1.smithi027.stdout:5/984: symlink d0/df/d11/d13/d9a/d9c/dd0/d145/l146 0 2024-04-02T04:43:09.898 INFO:tasks.workunit.client.1.smithi027.stdout:3/874: stat d1/de/df/d13/d73/d77/d102/fd1 0 2024-04-02T04:43:09.899 INFO:tasks.workunit.client.1.smithi027.stdout:7/934: write d2/dba/dc/d16/d2f/fc2 [2077245,73233] 0 2024-04-02T04:43:09.899 INFO:tasks.workunit.client.1.smithi027.stdout:3/875: readlink d1/d1a/l62 0 2024-04-02T04:43:09.901 INFO:tasks.workunit.client.1.smithi027.stdout:7/935: fsync d2/dba/dc/d16/f2c 0 2024-04-02T04:43:09.901 INFO:tasks.workunit.client.1.smithi027.stdout:5/985: mkdir d0/df/d29/da5/da6/d147 0 2024-04-02T04:43:09.901 INFO:tasks.workunit.client.1.smithi027.stdout:3/876: chown d1/de/d55/d99/fad 98739 1 2024-04-02T04:43:09.902 INFO:tasks.workunit.client.1.smithi027.stdout:7/936: dread - d2/d1d/d90/f113 zero size 2024-04-02T04:43:09.907 INFO:tasks.workunit.client.1.smithi027.stdout:5/986: fdatasync d0/fc5 0 2024-04-02T04:43:09.925 INFO:tasks.workunit.client.1.smithi027.stdout:7/937: link d2/dba/dc/d16/d3f/d50/d8b/db6/f107 d2/dba/dc/d16/d3f/d79/f12b 0 2024-04-02T04:43:09.927 INFO:tasks.workunit.client.1.smithi027.stdout:5/987: dread d0/d1d/d61/d7e/faf [0,4194304] 0 2024-04-02T04:43:09.927 INFO:tasks.workunit.client.1.smithi027.stdout:7/938: symlink d2/dba/da6/l12c 0 2024-04-02T04:43:09.928 INFO:tasks.workunit.client.1.smithi027.stdout:1/855: truncate d0/de/d52/d74/db7/df1/d5d/f81 45489 0 2024-04-02T04:43:09.929 INFO:tasks.workunit.client.1.smithi027.stdout:7/939: stat d2/dba/dc/d16/d3a/d4d/df4 0 2024-04-02T04:43:09.930 INFO:tasks.workunit.client.1.smithi027.stdout:1/856: write d0/de/d52/d74/df7/d3b/d4f/d9b/f113 [316900,105548] 0 2024-04-02T04:43:09.931 INFO:tasks.workunit.client.1.smithi027.stdout:7/940: write d2/dba/dc/d16/d2f/dcd/fcf [2690723,28948] 0 2024-04-02T04:43:09.931 INFO:tasks.workunit.client.1.smithi027.stdout:5/988: truncate d0/df/d11/d13/d9a/d9c/db8/f101 114070 0 2024-04-02T04:43:09.935 INFO:tasks.workunit.client.1.smithi027.stdout:3/877: write d1/de/df/d13/d73/d77/d102/fcc [1719298,3045] 0 2024-04-02T04:43:09.941 INFO:tasks.workunit.client.1.smithi027.stdout:7/941: chown d2/dba/dc/d16/d3f/d44/cd2 2698 1 2024-04-02T04:43:09.944 INFO:tasks.workunit.client.1.smithi027.stdout:1/857: rename d0/de/d52/d74/db7/df1/d25/dfc to d0/de/d52/d74/dbc/d116 0 2024-04-02T04:43:09.951 INFO:tasks.workunit.client.1.smithi027.stdout:7/942: write d2/d1d/d90/f5a [352398,109542] 0 2024-04-02T04:43:09.954 INFO:tasks.workunit.client.1.smithi027.stdout:3/878: symlink d1/d1a/d103/l11a 0 2024-04-02T04:43:09.955 INFO:tasks.workunit.client.1.smithi027.stdout:5/989: write d0/d1d/f7c [3466972,66265] 0 2024-04-02T04:43:09.956 INFO:tasks.workunit.client.1.smithi027.stdout:5/990: stat d0/df/d11/d13 0 2024-04-02T04:43:09.957 INFO:tasks.workunit.client.1.smithi027.stdout:1/858: mkdir d0/de/d9e/d117 0 2024-04-02T04:43:09.966 INFO:tasks.workunit.client.1.smithi027.stdout:5/991: symlink d0/df/d29/da5/da6/de9/l148 0 2024-04-02T04:43:09.970 INFO:tasks.workunit.client.1.smithi027.stdout:1/859: dread d0/de/d52/d74/df7/d3b/d4f/fc8 [0,4194304] 0 2024-04-02T04:43:09.970 INFO:tasks.workunit.client.1.smithi027.stdout:3/879: mknod d1/de/d55/d63/da4/c11b 0 2024-04-02T04:43:09.971 INFO:tasks.workunit.client.1.smithi027.stdout:5/992: readlink d0/df/d2f/d107/l138 0 2024-04-02T04:43:09.972 INFO:tasks.workunit.client.1.smithi027.stdout:1/860: chown d0/de/d52/d74/df7/d3e/f42 28896 1 2024-04-02T04:43:09.972 INFO:tasks.workunit.client.1.smithi027.stdout:3/880: write d1/de/d55/de7/ff1 [686734,42660] 0 2024-04-02T04:43:09.973 INFO:tasks.workunit.client.1.smithi027.stdout:7/943: link d2/dba/dc/d16/d3f/d79/c86 d2/d7/df3/d106/c12d 0 2024-04-02T04:43:09.973 INFO:tasks.workunit.client.1.smithi027.stdout:3/881: dread - d1/d1a/fff zero size 2024-04-02T04:43:09.978 INFO:tasks.workunit.client.1.smithi027.stdout:7/944: rmdir d2/d1d 39 2024-04-02T04:43:09.980 INFO:tasks.workunit.client.1.smithi027.stdout:3/882: dwrite d1/de/df/d9e/ffd [0,4194304] 0 2024-04-02T04:43:09.980 INFO:tasks.workunit.client.1.smithi027.stdout:1/861: creat d0/de/d52/d76/d109/f118 x:0 0 0 2024-04-02T04:43:09.986 INFO:tasks.workunit.client.1.smithi027.stdout:7/945: symlink d2/dba/dc/d16/d2f/l12e 0 2024-04-02T04:43:09.988 INFO:tasks.workunit.client.1.smithi027.stdout:7/946: chown d2/dba/dc/d16/d3a/d4d 8803 1 2024-04-02T04:43:09.988 INFO:tasks.workunit.client.1.smithi027.stdout:7/947: write d2/dba/dc/d16/f2c [9055561,122990] 0 2024-04-02T04:43:09.994 INFO:tasks.workunit.client.1.smithi027.stdout:5/993: rmdir d0/df/d29/da5/da6/d147 0 2024-04-02T04:43:09.995 INFO:tasks.workunit.client.1.smithi027.stdout:1/862: mkdir d0/de/d119 0 2024-04-02T04:43:09.995 INFO:tasks.workunit.client.1.smithi027.stdout:7/948: dwrite d2/dba/d35/f7c [0,4194304] 0 2024-04-02T04:43:10.000 INFO:tasks.workunit.client.1.smithi027.stdout:7/949: write d2/d1d/f10b [996962,87462] 0 2024-04-02T04:43:10.001 INFO:tasks.workunit.client.1.smithi027.stdout:5/994: mkdir d0/df/d11/d13/d9a/d9c/db8/d11e/d149 0 2024-04-02T04:43:10.003 INFO:tasks.workunit.client.1.smithi027.stdout:7/950: write d2/df/d89/faa [2102173,6445] 0 2024-04-02T04:43:10.004 INFO:tasks.workunit.client.1.smithi027.stdout:0/975: sync 2024-04-02T04:43:10.005 INFO:tasks.workunit.client.1.smithi027.stdout:7/951: write d2/d1d/f10b [1709282,62502] 0 2024-04-02T04:43:10.013 INFO:tasks.workunit.client.1.smithi027.stdout:3/883: dwrite d1/de/df/d13/d73/d77/d102/f8e [4194304,4194304] 0 2024-04-02T04:43:10.022 INFO:tasks.workunit.client.1.smithi027.stdout:7/952: link d2/d1c/f37 d2/dba/dc/d16/d3f/d50/d8b/db6/f12f 0 2024-04-02T04:43:10.025 INFO:tasks.workunit.client.1.smithi027.stdout:0/976: link d4/d21/l7b d4/d21/d28/d4e/l13e 0 2024-04-02T04:43:10.026 INFO:tasks.workunit.client.1.smithi027.stdout:0/977: readlink d4/d16/d1b/l43 0 2024-04-02T04:43:10.029 INFO:tasks.workunit.client.1.smithi027.stdout:5/995: dwrite d0/df/d2f/d3b/d51/ff5 [4194304,4194304] 0 2024-04-02T04:43:10.032 INFO:tasks.workunit.client.1.smithi027.stdout:1/863: dread d0/de/d52/d74/db7/df1/d25/f31 [4194304,4194304] 0 2024-04-02T04:43:10.034 INFO:tasks.workunit.client.1.smithi027.stdout:0/978: rename d4/d21/d28/d4e/d5f/ced to d4/d69/c13f 0 2024-04-02T04:43:10.037 INFO:tasks.workunit.client.1.smithi027.stdout:7/953: write d2/dba/dc/d16/d3f/d79/f12b [828069,16138] 0 2024-04-02T04:43:10.040 INFO:tasks.workunit.client.1.smithi027.stdout:5/996: fdatasync d0/d1d/d61/d7e/dff/f11c 0 2024-04-02T04:43:10.042 INFO:tasks.workunit.client.1.smithi027.stdout:1/864: mkdir d0/de/d9e/d117/d11a 0 2024-04-02T04:43:10.043 INFO:tasks.workunit.client.1.smithi027.stdout:7/954: symlink d2/df/dc7/l130 0 2024-04-02T04:43:10.044 INFO:tasks.workunit.client.1.smithi027.stdout:1/865: write d0/de/d52/d74/df7/d3e/d5f/d8b/d9d/f108 [2198953,69712] 0 2024-04-02T04:43:10.047 INFO:tasks.workunit.client.1.smithi027.stdout:0/979: mknod d4/d21/d28/d4e/d5f/c140 0 2024-04-02T04:43:10.050 INFO:tasks.workunit.client.1.smithi027.stdout:5/997: symlink d0/df/d29/da5/da6/l14a 0 2024-04-02T04:43:10.056 INFO:tasks.workunit.client.1.smithi027.stdout:1/866: mknod d0/de/d52/c11b 0 2024-04-02T04:43:10.056 INFO:tasks.workunit.client.1.smithi027.stdout:5/998: dwrite d0/df/d11/d8c/f11f [0,4194304] 0 2024-04-02T04:43:10.059 INFO:tasks.workunit.client.1.smithi027.stdout:0/980: rename d4/d21/d85/db3/ccb to d4/d21/d28/d4e/db5/d118/c141 0 2024-04-02T04:43:10.060 INFO:tasks.workunit.client.1.smithi027.stdout:5/999: dread d0/d1d/d61/d7e/dff/f11c [0,4194304] 0 2024-04-02T04:43:10.062 INFO:tasks.workunit.client.1.smithi027.stdout:7/955: symlink d2/dba/dc/l131 0 2024-04-02T04:43:10.066 INFO:tasks.workunit.client.1.smithi027.stdout:7/956: dread d2/d1d/d92/f9b [0,4194304] 0 2024-04-02T04:43:10.067 INFO:tasks.workunit.client.1.smithi027.stdout:1/867: truncate d0/de/d52/fdd 222940 0 2024-04-02T04:43:10.068 INFO:tasks.workunit.client.1.smithi027.stdout:0/981: creat d4/d76/f142 x:0 0 0 2024-04-02T04:43:10.073 INFO:tasks.workunit.client.1.smithi027.stdout:1/868: creat d0/de/d52/d74/dbc/f11c x:0 0 0 2024-04-02T04:43:10.075 INFO:tasks.workunit.client.1.smithi027.stdout:7/957: write d2/dba/dc/d16/d2f/f73 [508010,61175] 0 2024-04-02T04:43:10.077 INFO:tasks.workunit.client.1.smithi027.stdout:1/869: fsync d0/de/d52/d74/db7/df1/d25/f50 0 2024-04-02T04:43:10.078 INFO:tasks.workunit.client.1.smithi027.stdout:7/958: mknod d2/dba/dc/d16/d2f/dcd/c132 0 2024-04-02T04:43:10.079 INFO:tasks.workunit.client.1.smithi027.stdout:0/982: rename d4/d21/c25 to d4/d21/d85/c143 0 2024-04-02T04:43:10.082 INFO:tasks.workunit.client.1.smithi027.stdout:0/983: chown d4/c131 768 1 2024-04-02T04:43:10.085 INFO:tasks.workunit.client.1.smithi027.stdout:7/959: rename d2/dba/dc/d16/d2f/dcd/c11b to d2/d7/df3/d106/d36/c133 0 2024-04-02T04:43:10.092 INFO:tasks.workunit.client.1.smithi027.stdout:0/984: chown d4/d38/d66/d2d/d81/f134 386199426 1 2024-04-02T04:43:10.093 INFO:tasks.workunit.client.1.smithi027.stdout:7/960: link d2/dba/dc/d16/d3a/d4d/f9d d2/dba/dc/d16/d3a/d4d/d11e/f134 0 2024-04-02T04:43:10.094 INFO:tasks.workunit.client.1.smithi027.stdout:1/870: dwrite d0/de/d52/d74/df7/d3e/fcb [0,4194304] 0 2024-04-02T04:43:10.099 INFO:tasks.workunit.client.1.smithi027.stdout:0/985: write d4/d21/d28/f135 [5057098,112251] 0 2024-04-02T04:43:10.100 INFO:tasks.workunit.client.1.smithi027.stdout:1/871: truncate d0/de/d52/d74/db7/df1/d25/f50 5145935 0 2024-04-02T04:43:10.104 INFO:tasks.workunit.client.1.smithi027.stdout:7/961: dwrite d2/d1d/f80 [0,4194304] 0 2024-04-02T04:43:10.105 INFO:tasks.workunit.client.1.smithi027.stdout:7/962: truncate d2/d1d/f10b 2277574 0 2024-04-02T04:43:10.106 INFO:tasks.workunit.client.1.smithi027.stdout:0/986: creat d4/d21/d28/d4e/db5/d118/f144 x:0 0 0 2024-04-02T04:43:10.110 INFO:tasks.workunit.client.1.smithi027.stdout:1/872: mkdir d0/de/d52/d74/df7/d11d 0 2024-04-02T04:43:10.115 INFO:tasks.workunit.client.1.smithi027.stdout:7/963: rename d2/dba/dc/d16/d3f/d79/fec to d2/d1d/d90/f135 0 2024-04-02T04:43:10.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.112+0000 7fb517428700 1 -- 172.21.15.16:0/2308669095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb510100bd0 msgr2=0x7fb510101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:10.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.112+0000 7fb517428700 1 --2- 172.21.15.16:0/2308669095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb510100bd0 0x7fb510101030 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb500009910 tx=0x7fb500009c20 comp rx=0 tx=0).stop 2024-04-02T04:43:10.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.112+0000 7fb517428700 1 -- 172.21.15.16:0/2308669095 shutdown_connections 2024-04-02T04:43:10.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.112+0000 7fb517428700 1 --2- 172.21.15.16:0/2308669095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb510100bd0 0x7fb510101030 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:10.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.112+0000 7fb517428700 1 --2- 172.21.15.16:0/2308669095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5100ff9d0 0x7fb5100ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:10.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.112+0000 7fb517428700 1 -- 172.21.15.16:0/2308669095 >> 172.21.15.16:0/2308669095 conn(0x7fb5100faf70 msgr2=0x7fb5100fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:10.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.112+0000 7fb517428700 1 -- 172.21.15.16:0/2308669095 shutdown_connections 2024-04-02T04:43:10.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.112+0000 7fb517428700 1 -- 172.21.15.16:0/2308669095 wait complete. 2024-04-02T04:43:10.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.112+0000 7fb517428700 1 Processor -- start 2024-04-02T04:43:10.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.112+0000 7fb517428700 1 -- start start 2024-04-02T04:43:10.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.113+0000 7fb517428700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5100ff9d0 0x7fb510198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:10.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.113+0000 7fb517428700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb510100bd0 0x7fb510198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:10.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.113+0000 7fb517428700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb510199270 con 0x7fb5100ff9d0 2024-04-02T04:43:10.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.113+0000 7fb517428700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5101993b0 con 0x7fb510100bd0 2024-04-02T04:43:10.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.113+0000 7fb5151c4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5100ff9d0 0x7fb510198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:10.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.113+0000 7fb5151c4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5100ff9d0 0x7fb510198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38378/0 (socket says 172.21.15.16:38378) 2024-04-02T04:43:10.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.113+0000 7fb5151c4700 1 -- 172.21.15.16:0/3106186457 learned_addr learned my addr 172.21.15.16:0/3106186457 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:43:10.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.113+0000 7fb5149c3700 1 --2- 172.21.15.16:0/3106186457 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb510100bd0 0x7fb510198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:10.118 INFO:tasks.workunit.client.1.smithi027.stdout:0/987: mkdir d4/d145 0 2024-04-02T04:43:10.118 INFO:tasks.workunit.client.1.smithi027.stdout:1/873: dread d0/de/d52/d74/db7/df1/f9f [0,4194304] 0 2024-04-02T04:43:10.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.113+0000 7fb5151c4700 1 -- 172.21.15.16:0/3106186457 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb510100bd0 msgr2=0x7fb510198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:10.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.113+0000 7fb5151c4700 1 --2- 172.21.15.16:0/3106186457 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb510100bd0 0x7fb510198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:10.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.113+0000 7fb5151c4700 1 -- 172.21.15.16:0/3106186457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5000095f0 con 0x7fb5100ff9d0 2024-04-02T04:43:10.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.113+0000 7fb5151c4700 1 --2- 172.21.15.16:0/3106186457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5100ff9d0 0x7fb510198710 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00e8f0 tx=0x7fb50c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:10.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.114+0000 7fb5067fc700 1 -- 172.21.15.16:0/3106186457 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb50c009790 con 0x7fb5100ff9d0 2024-04-02T04:43:10.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.114+0000 7fb517428700 1 -- 172.21.15.16:0/3106186457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb51019de10 con 0x7fb5100ff9d0 2024-04-02T04:43:10.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.114+0000 7fb5067fc700 1 -- 172.21.15.16:0/3106186457 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb50c00c810 con 0x7fb5100ff9d0 2024-04-02T04:43:10.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.114+0000 7fb5067fc700 1 -- 172.21.15.16:0/3106186457 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb50c003d20 con 0x7fb5100ff9d0 2024-04-02T04:43:10.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.114+0000 7fb517428700 1 -- 172.21.15.16:0/3106186457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb51019e350 con 0x7fb5100ff9d0 2024-04-02T04:43:10.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.115+0000 7fb517428700 1 -- 172.21.15.16:0/3106186457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb51004ea80 con 0x7fb5100ff9d0 2024-04-02T04:43:10.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.116+0000 7fb5067fc700 1 -- 172.21.15.16:0/3106186457 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) v1 ==== 91816+0+0 (secure 0 0 0) 0x7fb50c0041f0 con 0x7fb5100ff9d0 2024-04-02T04:43:10.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.116+0000 7fb5067fc700 1 --2- 172.21.15.16:0/3106186457 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7fb4fc06db10 0x7fb4fc06ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:10.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.116+0000 7fb5067fc700 1 -- 172.21.15.16:0/3106186457 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fb50c014070 con 0x7fb5100ff9d0 2024-04-02T04:43:10.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.116+0000 7fb5149c3700 1 --2- 172.21.15.16:0/3106186457 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7fb4fc06db10 0x7fb4fc06ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:10.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.117+0000 7fb5149c3700 1 --2- 172.21.15.16:0/3106186457 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7fb4fc06db10 0x7fb4fc06ffd0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb50000b800 tx=0x7fb500009f70 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:10.121 INFO:tasks.workunit.client.1.smithi027.stdout:7/964: mknod d2/dba/dc/d16/d3f/d50/d8b/d91/c136 0 2024-04-02T04:43:10.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.120+0000 7fb5067fc700 1 -- 172.21.15.16:0/3106186457 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184299 (secure 0 0 0) 0x7fb50c05afe0 con 0x7fb5100ff9d0 2024-04-02T04:43:10.125 INFO:tasks.workunit.client.1.smithi027.stdout:0/988: dwrite d4/d21/d28/d4e/db5/d118/dfe/fb4 [0,4194304] 0 2024-04-02T04:43:10.144 INFO:tasks.workunit.client.1.smithi027.stdout:7/965: dwrite d2/dba/dc/f30 [4194304,4194304] 0 2024-04-02T04:43:10.145 INFO:tasks.workunit.client.1.smithi027.stdout:0/989: dwrite d4/d21/d28/f44 [0,4194304] 0 2024-04-02T04:43:10.145 INFO:tasks.workunit.client.1.smithi027.stdout:1/874: dwrite d0/de/d52/d74/df7/d3b/f78 [0,4194304] 0 2024-04-02T04:43:10.147 INFO:tasks.workunit.client.1.smithi027.stdout:0/990: stat d4/d69/f108 0 2024-04-02T04:43:10.153 INFO:tasks.workunit.client.1.smithi027.stdout:7/966: creat d2/dba/dc/d16/d3f/d50/f137 x:0 0 0 2024-04-02T04:43:10.155 INFO:tasks.workunit.client.1.smithi027.stdout:7/967: chown d2/dba/dc/d16/d3f/d44/c108 12482 1 2024-04-02T04:43:10.159 INFO:tasks.workunit.client.1.smithi027.stdout:0/991: creat d4/d145/f146 x:0 0 0 2024-04-02T04:43:10.161 INFO:tasks.workunit.client.1.smithi027.stdout:1/875: fdatasync d0/de/d52/d74/df7/d3b/dda/fb0 0 2024-04-02T04:43:10.161 INFO:tasks.workunit.client.1.smithi027.stdout:0/992: dread - d4/d69/d122/f13b zero size 2024-04-02T04:43:10.162 INFO:tasks.workunit.client.1.smithi027.stdout:0/993: dread - d4/d69/f10b zero size 2024-04-02T04:43:10.164 INFO:tasks.workunit.client.1.smithi027.stdout:7/968: link d2/c3d d2/d1d/dd7/c138 0 2024-04-02T04:43:10.165 INFO:tasks.workunit.client.1.smithi027.stdout:1/876: truncate d0/de/d52/d74/db7/df1/d25/d3c/d64/da6/f107 433244 0 2024-04-02T04:43:10.166 INFO:tasks.workunit.client.1.smithi027.stdout:0/994: creat d4/d21/d28/d4e/db5/f147 x:0 0 0 2024-04-02T04:43:10.170 INFO:tasks.workunit.client.1.smithi027.stdout:0/995: creat d4/d21/d28/d4e/d8c/f148 x:0 0 0 2024-04-02T04:43:10.172 INFO:tasks.workunit.client.1.smithi027.stdout:4/782: sync 2024-04-02T04:43:10.172 INFO:tasks.workunit.client.1.smithi027.stdout:2/963: sync 2024-04-02T04:43:10.172 INFO:tasks.workunit.client.1.smithi027.stdout:6/813: sync 2024-04-02T04:43:10.173 INFO:tasks.workunit.client.1.smithi027.stdout:8/873: sync 2024-04-02T04:43:10.173 INFO:tasks.workunit.client.1.smithi027.stdout:7/969: rename d2/dba/dc/d16/d3f/d44/f6a to d2/dba/dc/d16/d3a/d4d/dcb/f139 0 2024-04-02T04:43:10.174 INFO:tasks.workunit.client.1.smithi027.stdout:4/783: dread - d24/d38/d44/d82/d95/faf zero size 2024-04-02T04:43:10.175 INFO:tasks.workunit.client.1.smithi027.stdout:0/996: unlink d4/d21/d85/f13a 0 2024-04-02T04:43:10.180 INFO:tasks.workunit.client.1.smithi027.stdout:7/970: rmdir d2/dba/dc/d16/d3f/d50/d8b/db6 39 2024-04-02T04:43:10.181 INFO:tasks.workunit.client.1.smithi027.stdout:1/877: write d0/de/d52/d74/db7/df1/d5d/fdb [322439,124496] 0 2024-04-02T04:43:10.181 INFO:tasks.workunit.client.1.smithi027.stdout:7/971: readlink d2/dba/dc/d16/d2f/dcd/lef 0 2024-04-02T04:43:10.183 INFO:tasks.workunit.client.1.smithi027.stdout:7/972: stat d2/dba/dc/d16/d81/c94 0 2024-04-02T04:43:10.183 INFO:tasks.workunit.client.1.smithi027.stdout:4/784: creat d24/d38/d47/ffb x:0 0 0 2024-04-02T04:43:10.185 INFO:tasks.workunit.client.1.smithi027.stdout:1/878: fdatasync d0/de/d52/d74/df7/d3b/d4f/d9b/f113 0 2024-04-02T04:43:10.186 INFO:tasks.workunit.client.1.smithi027.stdout:1/879: chown d0/de/d52/d74/db7/fc6 0 1 2024-04-02T04:43:10.191 INFO:tasks.workunit.client.1.smithi027.stdout:2/964: rename d1/db/le0 to d1/db/de5/de6/df0/l156 0 2024-04-02T04:43:10.194 INFO:tasks.workunit.client.1.smithi027.stdout:7/973: creat d2/dba/da6/f13a x:0 0 0 2024-04-02T04:43:10.196 INFO:tasks.workunit.client.1.smithi027.stdout:4/785: symlink d24/d38/d44/dbc/lfc 0 2024-04-02T04:43:10.196 INFO:tasks.workunit.client.1.smithi027.stdout:7/974: read d2/dba/dc/d16/d3f/d50/f69 [1231840,2931] 0 2024-04-02T04:43:10.198 INFO:tasks.workunit.client.1.smithi027.stdout:0/997: symlink d4/d38/d66/d2d/l149 0 2024-04-02T04:43:10.199 INFO:tasks.workunit.client.1.smithi027.stdout:0/998: read d4/d21/d28/d4e/db5/d118/dfe/fb4 [834544,18956] 0 2024-04-02T04:43:10.203 INFO:tasks.workunit.client.1.smithi027.stdout:8/874: creat d5/d8/db/dd/d5f/d27/d31/f124 x:0 0 0 2024-04-02T04:43:10.209 INFO:tasks.workunit.client.1.smithi027.stdout:6/814: rename d13/d1b/d24/d25/d27 to d13/d1b/d48/d75/d9b/de2/d114 0 2024-04-02T04:43:10.212 INFO:tasks.workunit.client.1.smithi027.stdout:2/965: rmdir d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/de1/d137/d13e 39 2024-04-02T04:43:10.216 INFO:tasks.workunit.client.1.smithi027.stdout:1/880: write d0/de/d52/d74/df7/f7a [3995012,98067] 0 2024-04-02T04:43:10.218 INFO:tasks.workunit.client.1.smithi027.stdout:7/975: unlink d2/d7/df3/f103 0 2024-04-02T04:43:10.223 INFO:tasks.workunit.client.1.smithi027.stdout:0/999: fsync d4/d38/d66/d4a/f70 0 2024-04-02T04:43:10.230 INFO:tasks.workunit.client.1.smithi027.stdout:4/786: dwrite d24/d38/d44/d60/f7c [4194304,4194304] 0 2024-04-02T04:43:10.230 INFO:tasks.workunit.client.1.smithi027.stdout:6/815: mknod d13/d1b/d48/d58/c115 0 2024-04-02T04:43:10.237 INFO:tasks.workunit.client.1.smithi027.stdout:8/875: dwrite d5/d8/db/f8e [0,4194304] 0 2024-04-02T04:43:10.243 INFO:tasks.workunit.client.1.smithi027.stdout:1/881: mknod d0/de/d52/d74/db7/df1/d25/d3c/d64/dec/c11e 0 2024-04-02T04:43:10.244 INFO:tasks.workunit.client.1.smithi027.stdout:1/882: stat d0/de/c2a 0 2024-04-02T04:43:10.245 INFO:tasks.workunit.client.1.smithi027.stdout:7/976: mkdir d2/dba/dc/d16/d3a/d4d/df4/d13b 0 2024-04-02T04:43:10.249 INFO:tasks.workunit.client.1.smithi027.stdout:8/876: dwrite d5/d8/db/dd/d5f/d27/d31/d5b/dc4/dea/f119 [0,4194304] 0 2024-04-02T04:43:10.250 INFO:tasks.workunit.client.1.smithi027.stdout:8/877: dread - d5/d8/db/d68/f11a zero size 2024-04-02T04:43:10.250 INFO:tasks.workunit.client.1.smithi027.stdout:4/787: symlink d24/d38/d47/lfd 0 2024-04-02T04:43:10.251 INFO:tasks.workunit.client.1.smithi027.stdout:6/816: write d13/d1b/d24/d25/d42/fa9 [74400,30290] 0 2024-04-02T04:43:10.259 INFO:tasks.workunit.client.1.smithi027.stdout:3/884: sync 2024-04-02T04:43:10.259 INFO:tasks.workunit.client.1.smithi027.stdout:7/977: rmdir d2/dba/dc/d16/d3a/db2 39 2024-04-02T04:43:10.260 INFO:tasks.workunit.client.1.smithi027.stdout:3/885: stat d1/de/df/d9e/dc6/lf0 0 2024-04-02T04:43:10.261 INFO:tasks.workunit.client.1.smithi027.stdout:3/886: write d1/de/df/d13/db3/f101 [985377,25215] 0 2024-04-02T04:43:10.262 INFO:tasks.workunit.client.1.smithi027.stdout:8/878: mkdir d5/d8/d15/d71/db0/d125 0 2024-04-02T04:43:10.263 INFO:tasks.workunit.client.1.smithi027.stdout:6/817: symlink d13/d1b/d24/d25/ded/l116 0 2024-04-02T04:43:10.263 INFO:tasks.workunit.client.1.smithi027.stdout:2/966: link d1/db/d3b/d17/d6d/d84/f8e d1/db/d3b/d17/d6d/d84/f157 0 2024-04-02T04:43:10.264 INFO:tasks.workunit.client.1.smithi027.stdout:2/967: readlink d1/db/d3b/l3c 0 2024-04-02T04:43:10.266 INFO:tasks.workunit.client.1.smithi027.stdout:6/818: truncate d13/d1b/d48/d9c/df7/f109 434399 0 2024-04-02T04:43:10.275 INFO:tasks.workunit.client.1.smithi027.stdout:8/879: stat d5/l12 0 2024-04-02T04:43:10.276 INFO:tasks.workunit.client.1.smithi027.stdout:3/887: rmdir d1/de/df/da5 39 2024-04-02T04:43:10.276 INFO:tasks.workunit.client.1.smithi027.stdout:1/883: dread d0/de/d52/d74/df7/f7a [0,4194304] 0 2024-04-02T04:43:10.276 INFO:tasks.workunit.client.1.smithi027.stdout:8/880: chown d5/d8/db/f8e 15212700 1 2024-04-02T04:43:10.276 INFO:tasks.workunit.client.1.smithi027.stdout:1/884: readlink d0/l8 0 2024-04-02T04:43:10.277 INFO:tasks.workunit.client.1.smithi027.stdout:3/888: stat d1/de/df/f5d 0 2024-04-02T04:43:10.279 INFO:tasks.workunit.client.1.smithi027.stdout:3/889: chown d1/de/d55/d63/db9/lbf 2087052 1 2024-04-02T04:43:10.281 INFO:tasks.workunit.client.1.smithi027.stdout:6/819: symlink d13/d1b/d24/d25/ded/l117 0 2024-04-02T04:43:10.282 INFO:tasks.workunit.client.1.smithi027.stdout:7/978: write d2/dba/dc/d1f/f9e [1344799,51131] 0 2024-04-02T04:43:10.285 INFO:tasks.workunit.client.1.smithi027.stdout:4/788: truncate d24/f2c 3809147 0 2024-04-02T04:43:10.286 INFO:tasks.workunit.client.1.smithi027.stdout:8/881: creat d5/d8/db/dd/d5f/d27/d31/d5b/dc4/dea/f126 x:0 0 0 2024-04-02T04:43:10.292 INFO:tasks.workunit.client.1.smithi027.stdout:7/979: creat d2/d1d/dd7/f13c x:0 0 0 2024-04-02T04:43:10.294 INFO:tasks.workunit.client.1.smithi027.stdout:8/882: mknod d5/d8/db/df/c127 0 2024-04-02T04:43:10.295 INFO:tasks.workunit.client.1.smithi027.stdout:6/820: dread d13/d1b/d24/d25/d56/dc8/fe9 [0,4194304] 0 2024-04-02T04:43:10.297 INFO:tasks.workunit.client.1.smithi027.stdout:4/789: dwrite d24/d38/d44/d82/fd5 [0,4194304] 0 2024-04-02T04:43:10.300 INFO:tasks.workunit.client.1.smithi027.stdout:2/968: dwrite d1/db/d57/f99 [0,4194304] 0 2024-04-02T04:43:10.301 INFO:tasks.workunit.client.1.smithi027.stdout:3/890: link d1/de/d55/lb0 d1/de/df/d9e/d100/l11c 0 2024-04-02T04:43:10.303 INFO:tasks.workunit.client.1.smithi027.stdout:1/885: dwrite d0/de/d52/d74/df7/d3e/d5f/fb6 [0,4194304] 0 2024-04-02T04:43:10.303 INFO:tasks.workunit.client.1.smithi027.stdout:4/790: dread - d24/d38/d47/ff7 zero size 2024-04-02T04:43:10.306 INFO:tasks.workunit.client.1.smithi027.stdout:7/980: unlink d2/dba/dc/d1f/d61/lc5 0 2024-04-02T04:43:10.314 INFO:tasks.workunit.client.1.smithi027.stdout:4/791: write d24/d48/db0/fc3 [2951672,91362] 0 2024-04-02T04:43:10.320 INFO:tasks.workunit.client.1.smithi027.stdout:3/891: creat d1/de/df/d13/d73/f11d x:0 0 0 2024-04-02T04:43:10.322 INFO:tasks.workunit.client.1.smithi027.stdout:2/969: mknod d1/db/de5/c158 0 2024-04-02T04:43:10.323 INFO:tasks.workunit.client.1.smithi027.stdout:7/981: dwrite d2/dba/dc/d16/d3f/d44/fe0 [0,4194304] 0 2024-04-02T04:43:10.325 INFO:tasks.workunit.client.1.smithi027.stdout:1/886: creat d0/de/d52/d74/db7/df1/d25/d3c/d64/da6/f11f x:0 0 0 2024-04-02T04:43:10.326 INFO:tasks.workunit.client.1.smithi027.stdout:4/792: dwrite d24/d38/d44/d41/d49/f4c [0,4194304] 0 2024-04-02T04:43:10.326 INFO:tasks.workunit.client.1.smithi027.stdout:7/982: dread - d2/dba/fdf zero size 2024-04-02T04:43:10.326 INFO:tasks.workunit.client.1.smithi027.stdout:2/970: dread - d1/db/d3b/d17/d6d/dd9/f13d zero size 2024-04-02T04:43:10.327 INFO:tasks.workunit.client.1.smithi027.stdout:2/971: readlink d1/db/d3b/d61/la8 0 2024-04-02T04:43:10.330 INFO:tasks.workunit.client.1.smithi027.stdout:4/793: chown d24/d48/db0/ld6 24261041 1 2024-04-02T04:43:10.332 INFO:tasks.workunit.client.1.smithi027.stdout:1/887: dread - d0/de/d52/d74/db7/df1/d25/d3c/d64/dec/f10b zero size 2024-04-02T04:43:10.332 INFO:tasks.workunit.client.1.smithi027.stdout:1/888: readlink d0/l8 0 2024-04-02T04:43:10.343 INFO:tasks.workunit.client.1.smithi027.stdout:7/983: mkdir d2/dba/dc/d16/d2f/d13d 0 2024-04-02T04:43:10.350 INFO:tasks.workunit.client.1.smithi027.stdout:8/883: rename d5/d8/db/dd/d5f/d27/d31/d5b/dc4/f8b to d5/d8/d15/d71/db0/d125/f128 0 2024-04-02T04:43:10.352 INFO:tasks.workunit.client.1.smithi027.stdout:6/821: getdents d13/d1b/d24/d28/d3c 0 2024-04-02T04:43:10.353 INFO:tasks.workunit.client.1.smithi027.stdout:6/822: chown d13/d1b/d48/d75/d9b/de2/d114/f76 54279323 1 2024-04-02T04:43:10.355 INFO:tasks.workunit.client.1.smithi027.stdout:6/823: write d13/d1b/d48/d75/d9b/de2/d114/f81 [3413679,59992] 0 2024-04-02T04:43:10.358 INFO:tasks.workunit.client.1.smithi027.stdout:1/889: creat d0/de/d52/d74/db7/df1/d25/d2b/d59/dcf/f120 x:0 0 0 2024-04-02T04:43:10.360 INFO:tasks.workunit.client.1.smithi027.stdout:3/892: creat d1/de/f11e x:0 0 0 2024-04-02T04:43:10.360 INFO:tasks.workunit.client.1.smithi027.stdout:7/984: mknod d2/d1d/dd7/d12a/c13e 0 2024-04-02T04:43:10.360 INFO:tasks.workunit.client.1.smithi027.stdout:3/893: truncate d1/de/df/f67 5093374 0 2024-04-02T04:43:10.364 INFO:tasks.workunit.client.1.smithi027.stdout:4/794: creat d24/ffe x:0 0 0 2024-04-02T04:43:10.365 INFO:tasks.workunit.client.1.smithi027.stdout:8/884: creat d5/d8/db/dd/d5f/db2/d10b/d123/f129 x:0 0 0 2024-04-02T04:43:10.366 INFO:tasks.workunit.client.1.smithi027.stdout:1/890: creat d0/de/d52/d74/db7/df1/d25/d2b/d59/dcf/f121 x:0 0 0 2024-04-02T04:43:10.369 INFO:tasks.workunit.client.1.smithi027.stdout:3/894: dwrite d1/de/df/d13/d73/db1/f10e [0,4194304] 0 2024-04-02T04:43:10.370 INFO:tasks.workunit.client.1.smithi027.stdout:7/985: rmdir d2/dba/dc/d16/d3f/dc0 39 2024-04-02T04:43:10.374 INFO:tasks.workunit.client.1.smithi027.stdout:2/972: dwrite d1/db/d3b/d17/d6d/dd9/dd5/dda/f13c [0,4194304] 0 2024-04-02T04:43:10.377 INFO:tasks.workunit.client.1.smithi027.stdout:6/824: truncate d13/d1b/d24/d28/f65 1135374 0 2024-04-02T04:43:10.384 INFO:tasks.workunit.client.1.smithi027.stdout:8/885: rmdir d5/d8/db/dd/d5f/db2/dc6/d106 39 2024-04-02T04:43:10.389 INFO:tasks.workunit.client.1.smithi027.stdout:2/973: symlink d1/db/d3b/d83/l159 0 2024-04-02T04:43:10.391 INFO:tasks.workunit.client.1.smithi027.stdout:6/825: mknod d13/d1b/d24/d25/ded/c118 0 2024-04-02T04:43:10.394 INFO:tasks.workunit.client.1.smithi027.stdout:7/986: creat d2/dba/dc/d16/d2f/d13d/f13f x:0 0 0 2024-04-02T04:43:10.395 INFO:tasks.workunit.client.1.smithi027.stdout:7/987: fdatasync d2/dba/fdf 0 2024-04-02T04:43:10.397 INFO:tasks.workunit.client.1.smithi027.stdout:1/891: creat d0/de/d52/d74/df7/f122 x:0 0 0 2024-04-02T04:43:10.397 INFO:tasks.workunit.client.1.smithi027.stdout:7/988: chown d2/dba/dc/d16/d3a/d4d/f58 26 1 2024-04-02T04:43:10.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.396+0000 7fb517428700 1 -- 172.21.15.16:0/3106186457 --> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5100fcf20 con 0x7fb4fc06db10 2024-04-02T04:43:10.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.398+0000 7fb5067fc700 1 -- 172.21.15.16:0/3106186457 <== mgr.24415 v2:172.21.15.27:6828/1554339939 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fb5100fcf20 con 0x7fb4fc06db10 2024-04-02T04:43:10.402 INFO:tasks.workunit.client.1.smithi027.stdout:8/886: symlink d5/d8/db/dd/d5f/db2/l12a 0 2024-04-02T04:43:10.402 INFO:tasks.workunit.client.1.smithi027.stdout:2/974: dread d1/db/d3b/d17/d68/d69/fe9 [0,4194304] 0 2024-04-02T04:43:10.403 INFO:tasks.workunit.client.1.smithi027.stdout:2/975: readlink d1/l35 0 2024-04-02T04:43:10.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.400+0000 7fb517428700 1 -- 172.21.15.16:0/3106186457 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7fb4fc06db10 msgr2=0x7fb4fc06ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:10.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.400+0000 7fb517428700 1 --2- 172.21.15.16:0/3106186457 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7fb4fc06db10 0x7fb4fc06ffd0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb50000b800 tx=0x7fb500009f70 comp rx=0 tx=0).stop 2024-04-02T04:43:10.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.400+0000 7fb517428700 1 -- 172.21.15.16:0/3106186457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5100ff9d0 msgr2=0x7fb510198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:10.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.400+0000 7fb517428700 1 --2- 172.21.15.16:0/3106186457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5100ff9d0 0x7fb510198710 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00e8f0 tx=0x7fb50c00ec00 comp rx=0 tx=0).stop 2024-04-02T04:43:10.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.401+0000 7fb517428700 1 -- 172.21.15.16:0/3106186457 shutdown_connections 2024-04-02T04:43:10.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.401+0000 7fb517428700 1 --2- 172.21.15.16:0/3106186457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5100ff9d0 0x7fb510198710 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:10.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.401+0000 7fb517428700 1 --2- 172.21.15.16:0/3106186457 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7fb4fc06db10 0x7fb4fc06ffd0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:10.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.401+0000 7fb517428700 1 --2- 172.21.15.16:0/3106186457 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb510100bd0 0x7fb510198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:10.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.401+0000 7fb517428700 1 -- 172.21.15.16:0/3106186457 >> 172.21.15.16:0/3106186457 conn(0x7fb5100faf70 msgr2=0x7fb5100fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:10.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.401+0000 7fb517428700 1 -- 172.21.15.16:0/3106186457 shutdown_connections 2024-04-02T04:43:10.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.401+0000 7fb517428700 1 -- 172.21.15.16:0/3106186457 wait complete. 2024-04-02T04:43:10.405 INFO:tasks.workunit.client.1.smithi027.stdout:4/795: dread d24/f2c [0,4194304] 0 2024-04-02T04:43:10.405 INFO:tasks.workunit.client.1.smithi027.stdout:1/892: dread d0/de/d52/d74/db7/df1/f9c [0,4194304] 0 2024-04-02T04:43:10.405 INFO:tasks.workunit.client.1.smithi027.stdout:8/887: fsync d5/d8/db/dd/d5f/d91/fb3 0 2024-04-02T04:43:10.406 INFO:tasks.workunit.client.1.smithi027.stdout:1/893: write d0/f1d [1816974,126150] 0 2024-04-02T04:43:10.407 INFO:tasks.workunit.client.1.smithi027.stdout:7/989: mkdir d2/d1d/d140 0 2024-04-02T04:43:10.414 INFO:tasks.workunit.client.1.smithi027.stdout:2/976: read d1/db/d3b/d17/d6d/d84/f157 [3787806,97614] 0 2024-04-02T04:43:10.414 INFO:teuthology.orchestra.run.smithi016.stdout:true 2024-04-02T04:43:10.415 INFO:tasks.workunit.client.1.smithi027.stdout:4/796: creat d24/d48/db0/fff x:0 0 0 2024-04-02T04:43:10.416 INFO:tasks.workunit.client.1.smithi027.stdout:8/888: truncate d5/d8/db/dd/f1d 894262 0 2024-04-02T04:43:10.418 INFO:tasks.workunit.client.1.smithi027.stdout:8/889: dread d5/db5/ddc/ff4 [0,4194304] 0 2024-04-02T04:43:10.421 INFO:tasks.workunit.client.1.smithi027.stdout:8/890: dread d5/d8/db/dd/d5f/d27/d31/f84 [0,4194304] 0 2024-04-02T04:43:10.422 INFO:tasks.workunit.client.1.smithi027.stdout:7/990: rename d2/d7/df3/d106/d36/l56 to d2/dba/da6/l141 0 2024-04-02T04:43:10.423 INFO:tasks.workunit.client.1.smithi027.stdout:2/977: mkdir d1/db/d3b/d17/d14d/d15a 0 2024-04-02T04:43:10.429 INFO:tasks.workunit.client.1.smithi027.stdout:4/797: fdatasync d24/fb5 0 2024-04-02T04:43:10.430 INFO:tasks.workunit.client.1.smithi027.stdout:4/798: dread - d24/d38/d47/fdf zero size 2024-04-02T04:43:10.431 INFO:tasks.workunit.client.1.smithi027.stdout:4/799: truncate f13 5039729 0 2024-04-02T04:43:10.436 INFO:tasks.workunit.client.1.smithi027.stdout:8/891: write d5/d8/db/dd/f1f [1552651,100786] 0 2024-04-02T04:43:10.438 INFO:tasks.workunit.client.1.smithi027.stdout:3/895: dwrite d1/de/df/d13/f34 [0,4194304] 0 2024-04-02T04:43:10.443 INFO:tasks.workunit.client.1.smithi027.stdout:2/978: mkdir d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/d10d/d11f/d15b 0 2024-04-02T04:43:10.454 INFO:tasks.workunit.client.1.smithi027.stdout:4/800: mkdir d24/d2d/d100 0 2024-04-02T04:43:10.457 INFO:tasks.workunit.client.1.smithi027.stdout:8/892: creat d5/d8/db/d68/d95/f12b x:0 0 0 2024-04-02T04:43:10.469 INFO:tasks.workunit.client.1.smithi027.stdout:3/896: rename d1/d1a/cbc to d1/de/df/d9e/c11f 0 2024-04-02T04:43:10.480 INFO:tasks.workunit.client.1.smithi027.stdout:4/801: dread d24/d38/d44/d60/f6f [0,4194304] 0 2024-04-02T04:43:10.481 INFO:tasks.workunit.client.1.smithi027.stdout:6/826: truncate d13/d1b/d48/d58/f64 2378374 0 2024-04-02T04:43:10.482 INFO:tasks.workunit.client.1.smithi027.stdout:3/897: symlink d1/de/df/da5/l120 0 2024-04-02T04:43:10.482 INFO:tasks.workunit.client.1.smithi027.stdout:6/827: write d13/d1b/d24/d25/ded/ffb [463759,45809] 0 2024-04-02T04:43:10.485 INFO:tasks.workunit.client.1.smithi027.stdout:2/979: rename d1/db/d3b/d17/d6d/df1/f103 to d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/dc0/f15c 0 2024-04-02T04:43:10.495 INFO:tasks.workunit.client.1.smithi027.stdout:2/980: unlink d1/db/d3b/d17/d68/la7 0 2024-04-02T04:43:10.500 INFO:tasks.workunit.client.1.smithi027.stdout:3/898: link d1/de/d55/de7/ff1 d1/de/df/d13/d73/d77/d102/f121 0 2024-04-02T04:43:10.502 INFO:tasks.workunit.client.1.smithi027.stdout:2/981: truncate d1/db/d3b/d17/d14d/db4/ddf/ffd 155382 0 2024-04-02T04:43:10.505 INFO:tasks.workunit.client.1.smithi027.stdout:6/828: rename d13/d1b/d48/f66 to d13/d1b/d48/f119 0 2024-04-02T04:43:10.506 INFO:tasks.workunit.client.1.smithi027.stdout:1/894: dwrite d0/de/d52/d74/df7/f6b [0,4194304] 0 2024-04-02T04:43:10.508 INFO:tasks.workunit.client.1.smithi027.stdout:3/899: rename d1/d1a/ca8 to d1/de/df/d9e/dc6/de4/c122 0 2024-04-02T04:43:10.521 INFO:tasks.workunit.client.1.smithi027.stdout:3/900: symlink d1/de/df/da5/l123 0 2024-04-02T04:43:10.522 INFO:tasks.workunit.client.1.smithi027.stdout:7/991: write d2/d1c/f6e [102251,124475] 0 2024-04-02T04:43:10.523 INFO:tasks.workunit.client.1.smithi027.stdout:1/895: creat d0/de/d9e/d117/d11a/f123 x:0 0 0 2024-04-02T04:43:10.526 INFO:tasks.workunit.client.1.smithi027.stdout:3/901: mknod d1/de/df/d9e/dc6/c124 0 2024-04-02T04:43:10.529 INFO:tasks.workunit.client.1.smithi027.stdout:1/896: mknod d0/de/d52/d74/df7/d3b/d4f/dd3/c124 0 2024-04-02T04:43:10.531 INFO:tasks.workunit.client.1.smithi027.stdout:1/897: write d0/de/d52/d74/db7/df1/d25/d3c/d64/f110 [943192,37127] 0 2024-04-02T04:43:10.533 INFO:tasks.workunit.client.1.smithi027.stdout:8/893: truncate d5/d8/db/dd/d5f/db2/dcf/ff1 3968539 0 2024-04-02T04:43:10.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.530+0000 7f2df7ac7700 1 -- 172.21.15.16:0/2943002761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df0100bd0 msgr2=0x7f2df0101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:10.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.530+0000 7f2df7ac7700 1 --2- 172.21.15.16:0/2943002761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df0100bd0 0x7f2df0101030 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2de0009910 tx=0x7f2de0009c20 comp rx=0 tx=0).stop 2024-04-02T04:43:10.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.530+0000 7f2df7ac7700 1 -- 172.21.15.16:0/2943002761 shutdown_connections 2024-04-02T04:43:10.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.530+0000 7f2df7ac7700 1 --2- 172.21.15.16:0/2943002761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df0100bd0 0x7f2df0101030 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:10.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.530+0000 7f2df7ac7700 1 --2- 172.21.15.16:0/2943002761 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2df00ff9d0 0x7f2df00ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:10.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.530+0000 7f2df7ac7700 1 -- 172.21.15.16:0/2943002761 >> 172.21.15.16:0/2943002761 conn(0x7f2df00faf70 msgr2=0x7f2df00fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:10.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.530+0000 7f2df7ac7700 1 -- 172.21.15.16:0/2943002761 shutdown_connections 2024-04-02T04:43:10.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.531+0000 7f2df7ac7700 1 -- 172.21.15.16:0/2943002761 wait complete. 2024-04-02T04:43:10.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.531+0000 7f2df7ac7700 1 Processor -- start 2024-04-02T04:43:10.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.531+0000 7f2df7ac7700 1 -- start start 2024-04-02T04:43:10.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.531+0000 7f2df7ac7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2df00ff9d0 0x7f2df0068f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:10.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.531+0000 7f2df7ac7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df0100bd0 0x7f2df019b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:10.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.531+0000 7f2df7ac7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df019b7b0 con 0x7f2df0100bd0 2024-04-02T04:43:10.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.531+0000 7f2df7ac7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df019b920 con 0x7f2df00ff9d0 2024-04-02T04:43:10.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.531+0000 7f2df5062700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df0100bd0 0x7f2df019b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:10.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.531+0000 7f2df5062700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df0100bd0 0x7f2df019b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38388/0 (socket says 172.21.15.16:38388) 2024-04-02T04:43:10.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.531+0000 7f2df5062700 1 -- 172.21.15.16:0/385390432 learned_addr learned my addr 172.21.15.16:0/385390432 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:43:10.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.531+0000 7f2df5863700 1 --2- 172.21.15.16:0/385390432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2df00ff9d0 0x7f2df0068f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:10.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.532+0000 7f2df5062700 1 -- 172.21.15.16:0/385390432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2df00ff9d0 msgr2=0x7f2df0068f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:10.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.532+0000 7f2df5062700 1 --2- 172.21.15.16:0/385390432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2df00ff9d0 0x7f2df0068f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:10.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.532+0000 7f2df5062700 1 -- 172.21.15.16:0/385390432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de00095f0 con 0x7f2df0100bd0 2024-04-02T04:43:10.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.532+0000 7f2df5062700 1 --2- 172.21.15.16:0/385390432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df0100bd0 0x7f2df019b270 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2de0006010 tx=0x7f2de000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:10.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.532+0000 7f2de6ffd700 1 -- 172.21.15.16:0/385390432 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2de001d070 con 0x7f2df0100bd0 2024-04-02T04:43:10.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.532+0000 7f2de6ffd700 1 -- 172.21.15.16:0/385390432 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2de000f460 con 0x7f2df0100bd0 2024-04-02T04:43:10.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.532+0000 7f2df7ac7700 1 -- 172.21.15.16:0/385390432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2df019bba0 con 0x7f2df0100bd0 2024-04-02T04:43:10.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.532+0000 7f2de6ffd700 1 -- 172.21.15.16:0/385390432 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2de0021620 con 0x7f2df0100bd0 2024-04-02T04:43:10.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.532+0000 7f2df7ac7700 1 -- 172.21.15.16:0/385390432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2df019c110 con 0x7f2df0100bd0 2024-04-02T04:43:10.540 INFO:tasks.workunit.client.1.smithi027.stdout:7/992: symlink d2/dba/dc/d16/l142 0 2024-04-02T04:43:10.540 INFO:tasks.workunit.client.1.smithi027.stdout:8/894: chown d5/d8/db/dd/d5f/d27/d31/db9/le9 455404094 1 2024-04-02T04:43:10.540 INFO:tasks.workunit.client.1.smithi027.stdout:3/902: mkdir d1/de/df/d13/d73/de2/d125 0 2024-04-02T04:43:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.533+0000 7f2df7ac7700 1 -- 172.21.15.16:0/385390432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2df018f550 con 0x7f2df0100bd0 2024-04-02T04:43:10.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.534+0000 7f2de6ffd700 1 -- 172.21.15.16:0/385390432 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) v1 ==== 91816+0+0 (secure 0 0 0) 0x7f2de000f5d0 con 0x7f2df0100bd0 2024-04-02T04:43:10.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.534+0000 7f2de6ffd700 1 --2- 172.21.15.16:0/385390432 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7f2ddc06db20 0x7f2ddc06ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:10.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.534+0000 7f2de6ffd700 1 -- 172.21.15.16:0/385390432 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f2de008e930 con 0x7f2df0100bd0 2024-04-02T04:43:10.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.534+0000 7f2df5863700 1 --2- 172.21.15.16:0/385390432 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7f2ddc06db20 0x7f2ddc06ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:10.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.536+0000 7f2df5863700 1 --2- 172.21.15.16:0/385390432 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7f2ddc06db20 0x7f2ddc06ffe0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2df0100a30 tx=0x7f2dec00b410 comp rx=0 tx=0).ready entity=mgr.24415 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:10.542 INFO:tasks.workunit.client.1.smithi027.stdout:4/802: write d24/d38/d44/d41/f80 [302164,39035] 0 2024-04-02T04:43:10.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.540+0000 7f2de6ffd700 1 -- 172.21.15.16:0/385390432 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184299 (secure 0 0 0) 0x7f2de005be00 con 0x7f2df0100bd0 2024-04-02T04:43:10.552 INFO:tasks.workunit.client.1.smithi027.stdout:7/993: mknod d2/d7/c143 0 2024-04-02T04:43:10.555 INFO:tasks.workunit.client.1.smithi027.stdout:8/895: mknod d5/d8/db/dd/d5f/db2/dc6/d106/c12c 0 2024-04-02T04:43:10.556 INFO:tasks.workunit.client.1.smithi027.stdout:8/896: fdatasync d5/d8/db/fa3 0 2024-04-02T04:43:10.560 INFO:tasks.workunit.client.1.smithi027.stdout:7/994: symlink d2/d1d/l144 0 2024-04-02T04:43:10.566 INFO:tasks.workunit.client.1.smithi027.stdout:2/982: dwrite d1/db/d3b/d61/fdd [0,4194304] 0 2024-04-02T04:43:10.571 INFO:tasks.workunit.client.1.smithi027.stdout:6/829: dwrite d13/d1b/d48/d75/f83 [0,4194304] 0 2024-04-02T04:43:10.574 INFO:tasks.workunit.client.1.smithi027.stdout:4/803: rmdir d24/d38/df3 0 2024-04-02T04:43:10.578 INFO:tasks.workunit.client.1.smithi027.stdout:4/804: dread f21 [0,4194304] 0 2024-04-02T04:43:10.580 INFO:tasks.workunit.client.1.smithi027.stdout:4/805: read - d24/d38/d47/d62/ff1 zero size 2024-04-02T04:43:10.581 INFO:tasks.workunit.client.1.smithi027.stdout:3/903: link d1/de/df/c48 d1/d110/c126 0 2024-04-02T04:43:10.585 INFO:tasks.workunit.client.1.smithi027.stdout:7/995: creat d2/dba/dc/d16/d3f/d50/d8b/db6/f145 x:0 0 0 2024-04-02T04:43:10.585 INFO:tasks.workunit.client.1.smithi027.stdout:7/996: chown d2/df/dc7 46 1 2024-04-02T04:43:10.587 INFO:tasks.workunit.client.1.smithi027.stdout:2/983: mkdir d1/db/d3b/d61/d75/d81/d15d 0 2024-04-02T04:43:10.589 INFO:tasks.workunit.client.1.smithi027.stdout:2/984: write d1/db/d3b/d17/d6d/dd9/dd5/d125/f154 [45050,15891] 0 2024-04-02T04:43:10.593 INFO:tasks.workunit.client.1.smithi027.stdout:1/898: truncate d0/de/d52/d74/fb5 722479 0 2024-04-02T04:43:10.600 INFO:tasks.workunit.client.1.smithi027.stdout:4/806: mkdir d24/d38/d44/d41/d101 0 2024-04-02T04:43:10.601 INFO:tasks.workunit.client.1.smithi027.stdout:4/807: stat d24/d38/d44/d60/fb8 0 2024-04-02T04:43:10.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.601+0000 7f2de6ffd700 1 -- 172.21.15.16:0/385390432 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 27) v1 ==== 45778+0+0 (secure 0 0 0) 0x7f2de006fef0 con 0x7f2df0100bd0 2024-04-02T04:43:10.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.602+0000 7f2de6ffd700 1 -- 172.21.15.16:0/385390432 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7f2ddc06db20 msgr2=0x7f2ddc06ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:10.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:10.602+0000 7f2de6ffd700 1 --2- 172.21.15.16:0/385390432 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7f2ddc06db20 0x7f2ddc06ffe0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2df0100a30 tx=0x7f2dec00b410 comp rx=0 tx=0).stop 2024-04-02T04:43:10.608 INFO:tasks.workunit.client.1.smithi027.stdout:8/897: write d5/d8/db/dd/d5f/d27/d31/d5b/f67 [3164600,67789] 0 2024-04-02T04:43:10.610 INFO:tasks.workunit.client.1.smithi027.stdout:8/898: write d5/d8/d15/d71/db0/f121 [604850,118115] 0 2024-04-02T04:43:10.614 INFO:tasks.workunit.client.1.smithi027.stdout:2/985: mkdir d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/dc0/d15e 0 2024-04-02T04:43:10.615 INFO:tasks.workunit.client.1.smithi027.stdout:1/899: getdents d0/de/d52/d74/df7/d3b/d4f/dd3/dde 0 2024-04-02T04:43:10.618 INFO:tasks.workunit.client.1.smithi027.stdout:7/997: getdents d2/dba/dc/d16/d3a/d4d 0 2024-04-02T04:43:10.620 INFO:tasks.workunit.client.1.smithi027.stdout:8/899: mknod d5/d8/db/dd/d5f/db2/c12d 0 2024-04-02T04:43:10.622 INFO:tasks.workunit.client.1.smithi027.stdout:7/998: rmdir d2/df/d89/d8d 39 2024-04-02T04:43:10.630 INFO:tasks.workunit.client.1.smithi027.stdout:3/904: dwrite d1/de/f54 [0,4194304] 0 2024-04-02T04:43:10.630 INFO:tasks.workunit.client.1.smithi027.stdout:1/900: rename d0/de/d52/d74/db7/df1/d25/d2b/c7e to d0/de/d52/d74/dd9/c125 0 2024-04-02T04:43:10.631 INFO:tasks.workunit.client.1.smithi027.stdout:8/900: mkdir d5/d8/d15/d71/d96/d12e 0 2024-04-02T04:43:10.634 INFO:tasks.workunit.client.1.smithi027.stdout:8/901: write d5/d8/db/dd/d4a/dc9/feb [849445,94472] 0 2024-04-02T04:43:10.634 INFO:tasks.workunit.client.1.smithi027.stdout:1/901: truncate d0/de/d9e/d117/d11a/f123 445874 0 2024-04-02T04:43:10.635 INFO:tasks.workunit.client.1.smithi027.stdout:3/905: readlink d1/de/df/lae 0 2024-04-02T04:43:10.637 INFO:tasks.workunit.client.1.smithi027.stdout:4/808: write f11 [5592482,94865] 0 2024-04-02T04:43:10.637 INFO:tasks.workunit.client.1.smithi027.stdout:3/906: fsync d1/de/d55/d63/f113 0 2024-04-02T04:43:10.639 INFO:tasks.workunit.client.1.smithi027.stdout:4/809: read - d24/ffe zero size 2024-04-02T04:43:10.640 INFO:tasks.workunit.client.1.smithi027.stdout:4/810: stat d24/d38/d44/d41/la1 0 2024-04-02T04:43:10.641 INFO:tasks.workunit.client.1.smithi027.stdout:2/986: write d1/db/d3b/d17/d6d/dd9/dd5/dda/dbc/f105 [292021,61501] 0 2024-04-02T04:43:10.643 INFO:tasks.workunit.client.1.smithi027.stdout:8/902: creat d5/d8/d15/d71/db0/d125/f12f x:0 0 0 2024-04-02T04:43:10.644 INFO:tasks.workunit.client.1.smithi027.stdout:8/903: chown d5/l12 45879 1 2024-04-02T04:43:10.645 INFO:tasks.workunit.client.1.smithi027.stdout:3/907: mknod d1/de/d55/d63/da4/de3/c127 0 2024-04-02T04:43:10.649 INFO:tasks.workunit.client.1.smithi027.stdout:1/902: rename d0/de/d52/d74/df7/d3b/l68 to d0/de/d52/d74/db7/df1/d25/d3c/l126 0 2024-04-02T04:43:10.650 INFO:tasks.workunit.client.1.smithi027.stdout:2/987: creat d1/db/de5/de6/d13b/f15f x:0 0 0 2024-04-02T04:43:10.653 INFO:tasks.workunit.client.1.smithi027.stdout:4/811: rename d24/f46 to d24/d38/d44/d82/f102 0 2024-04-02T04:43:10.656 INFO:tasks.workunit.client.1.smithi027.stdout:1/903: symlink d0/de/d52/d74/db7/df1/d25/d3c/d64/l127 0 2024-04-02T04:43:10.657 INFO:tasks.workunit.client.1.smithi027.stdout:2/988: symlink d1/db/d3b/d17/d14d/db4/ddf/d113/l160 0 2024-04-02T04:43:10.660 INFO:tasks.workunit.client.1.smithi027.stdout:1/904: mknod d0/de/d52/d74/dd9/c128 0 2024-04-02T04:43:10.663 INFO:tasks.workunit.client.1.smithi027.stdout:1/905: read d0/de/d52/d74/db7/df1/d25/d2b/fce [4881746,123018] 0 2024-04-02T04:43:10.671 INFO:tasks.workunit.client.1.smithi027.stdout:3/908: truncate d1/de/df/d13/f34 1730879 0 2024-04-02T04:43:10.671 INFO:tasks.workunit.client.1.smithi027.stdout:2/989: dread d1/db/d3b/d17/d2b/f114 [0,4194304] 0 2024-04-02T04:43:10.679 INFO:tasks.workunit.client.1.smithi027.stdout:4/812: truncate d24/f2c 4727552 0 2024-04-02T04:43:10.681 INFO:tasks.workunit.client.1.smithi027.stdout:2/990: chown d1/db/d3b/d17/d14d/db4/ddf/d113 4 1 2024-04-02T04:43:10.681 INFO:tasks.workunit.client.1.smithi027.stdout:4/813: truncate d24/d48/de4/fec 301483 0 2024-04-02T04:43:10.738 INFO:tasks.workunit.client.1.smithi027.stdout:1/906: rename d0/la to d0/de/d52/d74/df7/d3b/l129 0 2024-04-02T04:43:10.739 INFO:tasks.workunit.client.1.smithi027.stdout:2/991: mkdir d1/db/d3b/d17/d6d/d84/d161 0 2024-04-02T04:43:10.740 INFO:tasks.workunit.client.1.smithi027.stdout:2/992: readlink d1/db/d3b/d61/d75/d81/l12f 0 2024-04-02T04:43:10.744 INFO:tasks.workunit.client.1.smithi027.stdout:4/814: mkdir d24/d38/d44/d41/d101/d103 0 2024-04-02T04:43:10.746 INFO:tasks.workunit.client.1.smithi027.stdout:3/909: link d1/de/df/c15 d1/de/df/da5/c128 0 2024-04-02T04:43:10.748 INFO:tasks.workunit.client.1.smithi027.stdout:2/993: symlink d1/db/d3b/d17/d155/l162 0 2024-04-02T04:43:10.755 INFO:tasks.workunit.client.1.smithi027.stdout:2/994: creat d1/db/d3b/d17/dcf/f163 x:0 0 0 2024-04-02T04:43:10.767 INFO:tasks.workunit.client.1.smithi027.stdout:2/995: mkdir d1/db/de5/d164 0 2024-04-02T04:43:10.768 INFO:tasks.workunit.client.1.smithi027.stdout:2/996: fsync d1/dfe/d14b/fe8 0 2024-04-02T04:43:10.782 INFO:tasks.workunit.client.1.smithi027.stdout:1/907: dwrite d0/de/f38 [0,4194304] 0 2024-04-02T04:43:10.784 INFO:tasks.workunit.client.1.smithi027.stdout:1/908: readlink d0/de/d52/d74/dd9/le8 0 2024-04-02T04:43:10.787 INFO:tasks.workunit.client.1.smithi027.stdout:4/815: dwrite d24/d38/d47/d62/db4/ff9 [0,4194304] 0 2024-04-02T04:43:10.787 INFO:tasks.workunit.client.1.smithi027.stdout:3/910: truncate d1/de/f54 3270385 0 2024-04-02T04:43:10.792 INFO:tasks.workunit.client.1.smithi027.stdout:3/911: creat d1/de/df/da5/f129 x:0 0 0 2024-04-02T04:43:10.794 INFO:tasks.workunit.client.1.smithi027.stdout:1/909: creat d0/de/d52/d74/df7/d11d/f12a x:0 0 0 2024-04-02T04:43:10.794 INFO:tasks.workunit.client.1.smithi027.stdout:1/910: stat d0/de 0 2024-04-02T04:43:10.801 INFO:tasks.workunit.client.1.smithi027.stdout:1/911: creat d0/de/d52/d74/db7/f12b x:0 0 0 2024-04-02T04:43:10.814 INFO:tasks.workunit.client.1.smithi027.stdout:1/912: unlink d0/l3 0 2024-04-02T04:43:10.820 INFO:tasks.workunit.client.1.smithi027.stdout:1/913: creat d0/de/d52/d74/db7/df1/d25/d2b/d59/dcf/f12c x:0 0 0 2024-04-02T04:43:10.822 INFO:tasks.workunit.client.1.smithi027.stdout:4/816: truncate d24/d38/d44/d60/fb8 1287424 0 2024-04-02T04:43:10.824 INFO:tasks.workunit.client.1.smithi027.stdout:1/914: dread d0/de/d52/d74/df7/d3b/d4f/d9b/f113 [0,4194304] 0 2024-04-02T04:43:10.827 INFO:tasks.workunit.client.1.smithi027.stdout:3/912: truncate d1/de/df/f14 3468897 0 2024-04-02T04:43:10.830 INFO:tasks.workunit.client.1.smithi027.stdout:1/915: creat d0/de/d52/d74/db7/df1/f12d x:0 0 0 2024-04-02T04:43:10.840 INFO:tasks.workunit.client.1.smithi027.stdout:4/817: dread f1e [0,4194304] 0 2024-04-02T04:43:10.843 INFO:tasks.workunit.client.1.smithi027.stdout:1/916: dread d0/de/d52/d74/df7/f6d [0,4194304] 0 2024-04-02T04:43:10.850 INFO:tasks.workunit.client.1.smithi027.stdout:1/917: creat d0/de/d52/d74/df7/d3b/d4f/d9b/f12e x:0 0 0 2024-04-02T04:43:10.850 INFO:tasks.workunit.client.1.smithi027.stdout:3/913: write d1/de/df/d13/d73/d77/d102/f121 [1652692,31159] 0 2024-04-02T04:43:10.852 INFO:tasks.workunit.client.1.smithi027.stdout:4/818: mkdir d24/d38/d44/d82/d97/d9b/d104 0 2024-04-02T04:43:10.857 INFO:tasks.workunit.client.1.smithi027.stdout:3/914: creat d1/de/df/d9e/dc6/de4/f12a x:0 0 0 2024-04-02T04:43:10.860 INFO:tasks.workunit.client.1.smithi027.stdout:4/819: dread - d24/d38/d44/d41/d5a/fab zero size 2024-04-02T04:43:10.861 INFO:tasks.workunit.client.1.smithi027.stdout:4/820: chown f15 95 1 2024-04-02T04:43:10.869 INFO:tasks.workunit.client.1.smithi027.stdout:4/821: creat d24/d38/d44/d41/d101/f105 x:0 0 0 2024-04-02T04:43:10.870 INFO:tasks.workunit.client.1.smithi027.stdout:3/915: mknod d1/d110/c12b 0 2024-04-02T04:43:10.871 INFO:tasks.workunit.client.1.smithi027.stdout:1/918: dwrite d0/de/d52/f67 [4194304,4194304] 0 2024-04-02T04:43:10.874 INFO:tasks.workunit.client.1.smithi027.stdout:3/916: creat d1/de/df/d13/f12c x:0 0 0 2024-04-02T04:43:10.875 INFO:tasks.workunit.client.1.smithi027.stdout:4/822: truncate d24/d38/d44/d60/fa2 737716 0 2024-04-02T04:43:10.877 INFO:tasks.workunit.client.1.smithi027.stdout:4/823: read f11 [5462717,118868] 0 2024-04-02T04:43:10.878 INFO:tasks.workunit.client.1.smithi027.stdout:3/917: mkdir d1/de/df/d13/db3/d12d 0 2024-04-02T04:43:10.883 INFO:tasks.workunit.client.1.smithi027.stdout:1/919: rename d0/de/d52/d74/db7/df1/d25/d3c/d64/lbf to d0/de/d119/l12f 0 2024-04-02T04:43:10.884 INFO:tasks.workunit.client.1.smithi027.stdout:4/824: mkdir d24/d38/d47/dc0/d106 0 2024-04-02T04:43:10.888 INFO:tasks.workunit.client.1.smithi027.stdout:4/825: mknod d24/d38/d47/d5c/c107 0 2024-04-02T04:43:10.891 INFO:tasks.workunit.client.1.smithi027.stdout:4/826: mknod d24/d38/d44/d41/d101/c108 0 2024-04-02T04:43:10.893 INFO:tasks.workunit.client.1.smithi027.stdout:3/918: rename d1/de/d55/d99/fad to d1/de/df/dec/f12e 0 2024-04-02T04:43:10.893 INFO:tasks.workunit.client.1.smithi027.stdout:4/827: truncate d24/d38/d44/d41/ff8 65134 0 2024-04-02T04:43:10.894 INFO:tasks.workunit.client.1.smithi027.stdout:4/828: dread - d24/d38/d47/d62/ff1 zero size 2024-04-02T04:43:10.895 INFO:tasks.workunit.client.1.smithi027.stdout:4/829: write d24/d48/db0/fc3 [3343829,84943] 0 2024-04-02T04:43:10.897 INFO:tasks.workunit.client.1.smithi027.stdout:1/920: rename d0/de/d9e/fe7 to d0/de/d52/d74/db7/df1/d25/d3c/d64/dec/f130 0 2024-04-02T04:43:10.898 INFO:tasks.workunit.client.1.smithi027.stdout:3/919: creat d1/de/df/d9e/dc6/f12f x:0 0 0 2024-04-02T04:43:10.903 INFO:tasks.workunit.client.1.smithi027.stdout:1/921: mkdir d0/de/d52/d74/df7/d3e/d5f/d8b/d9d/d131 0 2024-04-02T04:43:10.904 INFO:tasks.workunit.client.1.smithi027.stdout:4/830: mkdir d24/d38/d44/d82/d97/d9b/d104/d109 0 2024-04-02T04:43:10.906 INFO:tasks.workunit.client.1.smithi027.stdout:3/920: truncate d1/de/df/f6f 4353545 0 2024-04-02T04:43:10.907 INFO:tasks.workunit.client.1.smithi027.stdout:3/921: fsync d1/de/df/d13/d73/db1/fcb 0 2024-04-02T04:43:10.907 INFO:tasks.workunit.client.1.smithi027.stdout:1/922: creat d0/de/d52/d74/db7/df1/d25/d3c/d64/f132 x:0 0 0 2024-04-02T04:43:10.908 INFO:tasks.workunit.client.1.smithi027.stdout:4/831: truncate d24/d38/d44/d41/d5a/fab 789607 0 2024-04-02T04:43:10.910 INFO:tasks.workunit.client.1.smithi027.stdout:3/922: write d1/de/df/d13/d73/fa9 [217780,76767] 0 2024-04-02T04:43:10.910 INFO:tasks.workunit.client.1.smithi027.stdout:1/923: rmdir d0/de/d52/d74/d97/dfa 39 2024-04-02T04:43:10.910 INFO:tasks.workunit.client.1.smithi027.stdout:4/832: chown d24/f28 13345 1 2024-04-02T04:43:10.911 INFO:tasks.workunit.client.1.smithi027.stdout:1/924: chown d0/de/d52/d74/df7/d3e/d5f/d8b/d9d/fad 11 1 2024-04-02T04:43:10.912 INFO:tasks.workunit.client.1.smithi027.stdout:4/833: truncate d24/d38/d44/d41/d101/f105 307630 0 2024-04-02T04:43:10.914 INFO:tasks.workunit.client.1.smithi027.stdout:3/923: rmdir d1/de/df/d13/d73/d77 39 2024-04-02T04:43:10.914 INFO:tasks.workunit.client.1.smithi027.stdout:4/834: chown d24/d38/d47/d62/fd0 52 1 2024-04-02T04:43:10.914 INFO:tasks.workunit.client.1.smithi027.stdout:3/924: stat d1/c10b 0 2024-04-02T04:43:10.914 INFO:tasks.workunit.client.1.smithi027.stdout:3/925: stat d1/f2a 0 2024-04-02T04:43:10.914 INFO:tasks.workunit.client.1.smithi027.stdout:4/835: fsync d24/d38/d47/ff7 0 2024-04-02T04:43:10.915 INFO:tasks.workunit.client.1.smithi027.stdout:4/836: write f13 [3815872,114005] 0 2024-04-02T04:43:10.915 INFO:tasks.workunit.client.1.smithi027.stdout:1/925: fdatasync d0/de/d52/d74/df7/d3e/f42 0 2024-04-02T04:43:10.917 INFO:tasks.workunit.client.1.smithi027.stdout:4/837: stat d24/d38/d44/d41/d101/d103 0 2024-04-02T04:43:10.918 INFO:tasks.workunit.client.1.smithi027.stdout:3/926: chown d1/de/df/d9e/lc7 189 1 2024-04-02T04:43:10.925 INFO:tasks.workunit.client.1.smithi027.stdout:1/926: fdatasync d0/de/d52/d74/db7/df1/d25/d3c/f48 0 2024-04-02T04:43:10.930 INFO:tasks.workunit.client.1.smithi027.stdout:4/838: creat d24/d38/d44/d82/d97/d9b/f10a x:0 0 0 2024-04-02T04:43:10.938 INFO:tasks.workunit.client.1.smithi027.stdout:1/927: symlink d0/l133 0 2024-04-02T04:43:10.941 INFO:tasks.workunit.client.1.smithi027.stdout:4/839: creat d24/d38/d44/f10b x:0 0 0 2024-04-02T04:43:10.943 INFO:tasks.workunit.client.1.smithi027.stdout:1/928: symlink d0/de/d52/d74/df7/d3b/dda/l134 0 2024-04-02T04:43:10.944 INFO:tasks.workunit.client.1.smithi027.stdout:1/929: truncate d0/f100 881566 0 2024-04-02T04:43:10.947 INFO:tasks.workunit.client.1.smithi027.stdout:1/930: dread - d0/de/d52/d74/df7/d3b/f10c zero size 2024-04-02T04:43:10.949 INFO:tasks.workunit.client.1.smithi027.stdout:1/931: chown d0/de/d52/d74/db7/df1/d25/d3c/d64/f132 95084466 1 2024-04-02T04:43:10.956 INFO:tasks.workunit.client.1.smithi027.stdout:4/840: getdents d24/d38/d47 0 2024-04-02T04:43:10.957 INFO:tasks.workunit.client.1.smithi027.stdout:1/932: creat d0/de/d52/d74/df7/d3b/d4f/d9b/d115/f135 x:0 0 0 2024-04-02T04:43:10.964 INFO:tasks.workunit.client.1.smithi027.stdout:1/933: creat d0/de/d52/d74/d97/dfa/f136 x:0 0 0 2024-04-02T04:43:10.968 INFO:tasks.workunit.client.1.smithi027.stdout:1/934: symlink d0/de/d52/d74/df7/d3e/d5f/d8b/d9d/l137 0 2024-04-02T04:43:10.968 INFO:tasks.workunit.client.1.smithi027.stdout:4/841: write d24/d38/d44/d82/f102 [4457825,20236] 0 2024-04-02T04:43:10.971 INFO:tasks.workunit.client.1.smithi027.stdout:4/842: read - d24/d38/d44/d82/f90 zero size 2024-04-02T04:43:10.971 INFO:tasks.workunit.client.1.smithi027.stdout:1/935: mknod d0/de/d52/d74/df7/d11d/c138 0 2024-04-02T04:43:10.974 INFO:tasks.workunit.client.1.smithi027.stdout:4/843: dread d24/d38/d44/d82/d95/fb6 [0,4194304] 0 2024-04-02T04:43:10.979 INFO:tasks.workunit.client.1.smithi027.stdout:4/844: rename d24/d38/d44/d41/d5a/fcd to d24/d48/f10c 0 2024-04-02T04:43:10.988 INFO:tasks.workunit.client.1.smithi027.stdout:1/936: dwrite d0/de/d52/d74/df7/d3e/f82 [0,4194304] 0 2024-04-02T04:43:10.990 INFO:tasks.workunit.client.1.smithi027.stdout:4/845: truncate d24/d2d/f8a 2621494 0 2024-04-02T04:43:10.991 INFO:tasks.workunit.client.1.smithi027.stdout:4/846: creat d24/d38/d44/dbc/f10d x:0 0 0 2024-04-02T04:43:10.994 INFO:tasks.workunit.client.1.smithi027.stdout:1/937: dread d0/de/f23 [0,4194304] 0 2024-04-02T04:43:10.995 INFO:tasks.workunit.client.1.smithi027.stdout:1/938: readlink d0/de/d9e/lf4 0 2024-04-02T04:43:11.014 INFO:tasks.workunit.client.1.smithi027.stdout:1/939: dwrite d0/de/d52/d74/df7/d3b/dda/fa5 [0,4194304] 0 2024-04-02T04:43:11.021 INFO:tasks.workunit.client.1.smithi027.stdout:4/847: dread d24/f32 [0,4194304] 0 2024-04-02T04:43:11.022 INFO:tasks.workunit.client.1.smithi027.stdout:4/848: write d24/f39 [1610427,61703] 0 2024-04-02T04:43:11.031 INFO:tasks.workunit.client.1.smithi027.stdout:1/940: dwrite d0/de/d52/d74/db7/df1/d25/d2b/d59/fa4 [0,4194304] 0 2024-04-02T04:43:11.038 INFO:tasks.workunit.client.1.smithi027.stdout:1/941: mknod d0/de/d52/d74/d97/c139 0 2024-04-02T04:43:11.053 INFO:tasks.workunit.client.1.smithi027.stdout:1/942: dwrite d0/de/d52/d74/df7/d3e/f42 [0,4194304] 0 2024-04-02T04:43:11.132 INFO:tasks.workunit.client.1.smithi027.stdout:1/943: rename d0/de/d52/d74/df7/d3e/d5f/d8b/d9d/faf to d0/de/d52/d74/f13a 0 2024-04-02T04:43:11.138 INFO:tasks.workunit.client.1.smithi027.stdout:1/944: write d0/de/f23 [132583,122934] 0 2024-04-02T04:43:11.140 INFO:tasks.workunit.client.1.smithi027.stdout:1/945: creat d0/de/d52/d74/dbc/d116/f13b x:0 0 0 2024-04-02T04:43:11.141 INFO:tasks.workunit.client.1.smithi027.stdout:1/946: chown d0/de/f92 12700534 1 2024-04-02T04:43:11.142 INFO:tasks.workunit.client.1.smithi027.stdout:1/947: truncate d0/de/d52/d74/db7/f12b 623093 0 2024-04-02T04:43:11.145 INFO:tasks.workunit.client.1.smithi027.stdout:1/948: creat d0/de/d52/d74/f13c x:0 0 0 2024-04-02T04:43:11.148 INFO:tasks.workunit.client.1.smithi027.stdout:1/949: write d0/de/d52/d74/db7/df1/f9f [949019,39066] 0 2024-04-02T04:43:11.153 INFO:tasks.workunit.client.1.smithi027.stdout:1/950: dread d0/de/d52/d74/df7/f7a [0,4194304] 0 2024-04-02T04:43:11.155 INFO:tasks.workunit.client.1.smithi027.stdout:1/951: creat d0/de/d9e/f13d x:0 0 0 2024-04-02T04:43:11.157 INFO:tasks.workunit.client.1.smithi027.stdout:1/952: fsync d0/de/d52/d74/db7/df1/d25/d2b/d59/dcf/f121 0 2024-04-02T04:43:11.158 INFO:tasks.workunit.client.1.smithi027.stdout:1/953: write d0/de/d52/d74/df7/d3e/f42 [2789283,112012] 0 2024-04-02T04:43:11.162 INFO:tasks.workunit.client.1.smithi027.stdout:1/954: truncate d0/de/d52/d74/dbc/d116/f13b 134242 0 2024-04-02T04:43:11.164 INFO:tasks.workunit.client.1.smithi027.stdout:1/955: mknod d0/de/d52/d74/db7/df1/d25/d3c/d64/c13e 0 2024-04-02T04:43:11.166 INFO:tasks.workunit.client.1.smithi027.stdout:1/956: write d0/de/d52/d74/df7/d3e/d5f/d8b/d9d/f108 [361008,46558] 0 2024-04-02T04:43:11.171 INFO:tasks.workunit.client.1.smithi027.stdout:1/957: rename d0/de/d52/f67 to d0/de/d52/d74/df7/d11d/f13f 0 2024-04-02T04:43:11.185 INFO:tasks.workunit.client.1.smithi027.stdout:1/958: dread d0/de/d52/d74/db7/df1/d25/d3c/f48 [0,4194304] 0 2024-04-02T04:43:11.189 INFO:tasks.workunit.client.1.smithi027.stdout:1/959: read d0/de/d52/d74/db7/df1/d25/d2b/f84 [16502,24] 0 2024-04-02T04:43:11.190 INFO:tasks.workunit.client.1.smithi027.stdout:1/960: dread - d0/de/d52/d74/d97/dfa/f136 zero size 2024-04-02T04:43:11.191 INFO:tasks.workunit.client.1.smithi027.stdout:1/961: write d0/fb [356112,34308] 0 2024-04-02T04:43:11.196 INFO:tasks.workunit.client.1.smithi027.stdout:1/962: rmdir d0/de/d52/d74/df7 39 2024-04-02T04:43:11.198 INFO:tasks.workunit.client.1.smithi027.stdout:1/963: chown d0/de/d52/d74/db7/df1/d5d/fdb 21800 1 2024-04-02T04:43:11.200 INFO:tasks.workunit.client.1.smithi027.stdout:1/964: symlink d0/de/d52/d76/l140 0 2024-04-02T04:43:11.205 INFO:tasks.workunit.client.1.smithi027.stdout:1/965: mknod d0/de/d52/d74/df7/d11d/c141 0 2024-04-02T04:43:11.208 INFO:tasks.workunit.client.1.smithi027.stdout:1/966: creat d0/de/d119/f142 x:0 0 0 2024-04-02T04:43:11.211 INFO:tasks.workunit.client.1.smithi027.stdout:1/967: write d0/de/d52/d74/df7/d3b/fee [84340,99691] 0 2024-04-02T04:43:11.214 INFO:tasks.workunit.client.1.smithi027.stdout:1/968: creat d0/de/d52/d74/df7/d3e/f143 x:0 0 0 2024-04-02T04:43:11.216 INFO:tasks.workunit.client.1.smithi027.stdout:1/969: mknod d0/de/d52/d74/db7/df1/d25/d2b/d59/dcf/c144 0 2024-04-02T04:43:11.230 INFO:tasks.workunit.client.1.smithi027.stdout:1/970: dwrite d0/de/d52/d74/df7/d3e/d5f/f79 [0,4194304] 0 2024-04-02T04:43:11.231 INFO:tasks.workunit.client.1.smithi027.stdout:1/971: read - d0/de/d52/d74/df7/d3b/d4f/d9b/d115/f135 zero size 2024-04-02T04:43:11.237 INFO:tasks.workunit.client.1.smithi027.stdout:6/830: sync 2024-04-02T04:43:11.238 INFO:tasks.workunit.client.1.smithi027.stdout:1/972: dwrite d0/de/d52/d74/db7/df1/f12d [0,4194304] 0 2024-04-02T04:43:11.241 INFO:tasks.workunit.client.1.smithi027.stdout:6/831: fsync d13/d1b/d24/f4a 0 2024-04-02T04:43:11.243 INFO:tasks.workunit.client.1.smithi027.stdout:1/973: mkdir d0/de/d52/d74/dbc/d145 0 2024-04-02T04:43:11.245 INFO:tasks.workunit.client.1.smithi027.stdout:1/974: write d0/de/d52/d74/df7/d3b/d4f/d9b/f113 [1488596,92885] 0 2024-04-02T04:43:11.247 INFO:tasks.workunit.client.1.smithi027.stdout:6/832: write d13/d1b/d48/f119 [530596,71964] 0 2024-04-02T04:43:11.247 INFO:tasks.workunit.client.1.smithi027.stdout:6/833: dread - d13/d1b/d48/ffd zero size 2024-04-02T04:43:11.250 INFO:tasks.workunit.client.1.smithi027.stdout:1/975: chown d0/de/d52/d74/fb5 4 1 2024-04-02T04:43:11.251 INFO:tasks.workunit.client.1.smithi027.stdout:1/976: chown d0/de/d52/d74/df7/d3e/df8 89400 1 2024-04-02T04:43:11.265 INFO:tasks.workunit.client.1.smithi027.stdout:1/977: dread d0/de/d52/d74/db7/df1/fa3 [0,4194304] 0 2024-04-02T04:43:11.269 INFO:tasks.workunit.client.1.smithi027.stdout:6/834: dread d13/f1a [0,4194304] 0 2024-04-02T04:43:11.269 INFO:tasks.workunit.client.1.smithi027.stdout:1/978: creat d0/de/d52/d76/f146 x:0 0 0 2024-04-02T04:43:11.281 INFO:tasks.workunit.client.1.smithi027.stdout:1/979: link d0/de/fcc d0/de/d52/d74/db7/df1/d25/d3c/d64/da6/f147 0 2024-04-02T04:43:11.283 INFO:tasks.workunit.client.1.smithi027.stdout:6/835: dwrite d13/d1b/f8a [0,4194304] 0 2024-04-02T04:43:11.285 INFO:tasks.workunit.client.1.smithi027.stdout:1/980: mkdir d0/de/d52/d74/df7/d3b/dda/d148 0 2024-04-02T04:43:11.286 INFO:tasks.workunit.client.1.smithi027.stdout:1/981: chown d0/de/d52/d74/db7/df1/d25/d2b/f84 0 1 2024-04-02T04:43:11.291 INFO:tasks.workunit.client.1.smithi027.stdout:1/982: mkdir d0/de/d52/d74/db7/df1/d25/d3c/d64/d149 0 2024-04-02T04:43:11.293 INFO:tasks.workunit.client.1.smithi027.stdout:1/983: write d0/de/d52/d74/db7/df1/f12d [1426245,81145] 0 2024-04-02T04:43:11.304 INFO:tasks.workunit.client.1.smithi027.stdout:6/836: dwrite d13/d1b/d48/d75/f7d [0,4194304] 0 2024-04-02T04:43:11.308 INFO:tasks.workunit.client.1.smithi027.stdout:1/984: dwrite d0/de/f80 [0,4194304] 0 2024-04-02T04:43:11.320 INFO:tasks.workunit.client.1.smithi027.stdout:1/985: mknod d0/c14a 0 2024-04-02T04:43:11.320 INFO:tasks.workunit.client.1.smithi027.stdout:6/837: write d13/d1b/f2b [5713191,53798] 0 2024-04-02T04:43:11.322 INFO:tasks.workunit.client.1.smithi027.stdout:1/986: creat d0/de/d52/d76/f14b x:0 0 0 2024-04-02T04:43:11.327 INFO:tasks.workunit.client.1.smithi027.stdout:4/849: sync 2024-04-02T04:43:11.327 INFO:tasks.workunit.client.1.smithi027.stdout:2/997: sync 2024-04-02T04:43:11.327 INFO:tasks.workunit.client.1.smithi027.stdout:8/904: sync 2024-04-02T04:43:11.327 INFO:tasks.workunit.client.1.smithi027.stdout:3/927: sync 2024-04-02T04:43:11.327 INFO:tasks.workunit.client.1.smithi027.stdout:7/999: sync 2024-04-02T04:43:11.328 INFO:tasks.workunit.client.1.smithi027.stdout:6/838: getdents d13/d1b/d48/d75/d9b/de2/d114 0 2024-04-02T04:43:11.329 INFO:tasks.workunit.client.1.smithi027.stdout:2/998: stat d1/db/d3b/ccc 0 2024-04-02T04:43:11.330 INFO:tasks.workunit.client.1.smithi027.stdout:3/928: chown d1/de/d55/d63/da4/c119 133224 1 2024-04-02T04:43:11.331 INFO:tasks.workunit.client.1.smithi027.stdout:6/839: dread - d13/d1b/d24/d28/d3c/fea zero size 2024-04-02T04:43:11.337 INFO:tasks.workunit.client.1.smithi027.stdout:4/850: mknod d24/d38/d44/d82/da9/c10e 0 2024-04-02T04:43:11.345 INFO:tasks.workunit.client.1.smithi027.stdout:2/999: unlink d1/db/d3b/d17/d155/l162 0 2024-04-02T04:43:11.346 INFO:tasks.workunit.client.1.smithi027.stdout:1/987: dwrite d0/de/d52/d74/db7/df1/d25/d2b/fae [4194304,4194304] 0 2024-04-02T04:43:11.348 INFO:tasks.workunit.client.1.smithi027.stdout:8/905: symlink d5/d8/db/l130 0 2024-04-02T04:43:11.355 INFO:tasks.workunit.client.1.smithi027.stdout:4/851: mknod d24/d38/d44/d41/d49/c10f 0 2024-04-02T04:43:11.355 INFO:tasks.workunit.client.1.smithi027.stdout:6/840: mknod d13/d1b/d48/d75/d9b/de2/d114/c11a 0 2024-04-02T04:43:11.356 INFO:tasks.workunit.client.1.smithi027.stdout:3/929: dwrite d1/de/df/d13/f2d [0,4194304] 0 2024-04-02T04:43:11.356 INFO:tasks.workunit.client.1.smithi027.stdout:4/852: chown d24/d38/d44/d60 1323371 1 2024-04-02T04:43:11.365 INFO:tasks.workunit.client.1.smithi027.stdout:1/988: creat d0/de/d52/d74/df7/d3e/f14c x:0 0 0 2024-04-02T04:43:11.367 INFO:tasks.workunit.client.1.smithi027.stdout:8/906: symlink d5/d8/d15/d71/db0/l131 0 2024-04-02T04:43:11.371 INFO:tasks.workunit.client.1.smithi027.stdout:3/930: stat d1/l37 0 2024-04-02T04:43:11.373 INFO:tasks.workunit.client.1.smithi027.stdout:6/841: truncate d13/d1b/d48/d75/f9f 4577538 0 2024-04-02T04:43:11.374 INFO:tasks.workunit.client.1.smithi027.stdout:6/842: stat d13/d1b/d24/d28/d46/cf9 0 2024-04-02T04:43:11.375 INFO:tasks.workunit.client.1.smithi027.stdout:1/989: symlink d0/de/d52/d74/db7/df1/d25/d3c/d64/l14d 0 2024-04-02T04:43:11.377 INFO:tasks.workunit.client.1.smithi027.stdout:8/907: symlink d5/d8/d5c/de6/l132 0 2024-04-02T04:43:11.382 INFO:tasks.workunit.client.1.smithi027.stdout:1/990: creat d0/de/d52/d74/dbc/de2/f14e x:0 0 0 2024-04-02T04:43:11.385 INFO:tasks.workunit.client.1.smithi027.stdout:6/843: fdatasync d13/d1b/d24/d28/f65 0 2024-04-02T04:43:11.390 INFO:tasks.workunit.client.1.smithi027.stdout:6/844: rmdir d13/d1b/d24/d25/d56/d6d 39 2024-04-02T04:43:11.393 INFO:tasks.workunit.client.1.smithi027.stdout:6/845: read - d13/d1b/d48/d75/d9b/de2/fef zero size 2024-04-02T04:43:11.394 INFO:tasks.workunit.client.1.smithi027.stdout:4/853: dwrite d24/f3f [0,4194304] 0 2024-04-02T04:43:11.394 INFO:tasks.workunit.client.1.smithi027.stdout:6/846: chown d13/d1b/d48/d9c/db8 230879407 1 2024-04-02T04:43:11.399 INFO:tasks.workunit.client.1.smithi027.stdout:8/908: getdents d5/d8/db/dd/d4a/dc9 0 2024-04-02T04:43:11.402 INFO:tasks.workunit.client.1.smithi027.stdout:1/991: rmdir d0/de/d52/d74/dbc/d145 0 2024-04-02T04:43:11.404 INFO:tasks.workunit.client.1.smithi027.stdout:6/847: mknod d13/d1b/d24/d25/d42/c11b 0 2024-04-02T04:43:11.406 INFO:tasks.workunit.client.1.smithi027.stdout:4/854: creat d24/d38/d47/dc0/dca/f110 x:0 0 0 2024-04-02T04:43:11.409 INFO:tasks.workunit.client.1.smithi027.stdout:6/848: dread - d13/d1b/d48/d58/fa2 zero size 2024-04-02T04:43:11.410 INFO:tasks.workunit.client.1.smithi027.stdout:8/909: dread d5/d8/db/dd/d5f/d91/fb3 [0,4194304] 0 2024-04-02T04:43:11.411 INFO:tasks.workunit.client.1.smithi027.stdout:6/849: fdatasync d13/d1b/d24/d25/d42/fa9 0 2024-04-02T04:43:11.413 INFO:tasks.workunit.client.1.smithi027.stdout:8/910: write d5/d8/db/dd/fd2 [1686960,20526] 0 2024-04-02T04:43:11.414 INFO:tasks.workunit.client.1.smithi027.stdout:6/850: readlink d13/d1b/d24/d25/d42/l7b 0 2024-04-02T04:43:11.414 INFO:tasks.workunit.client.1.smithi027.stdout:8/911: fsync d5/d8/d15/d71/f72 0 2024-04-02T04:43:11.418 INFO:tasks.workunit.client.1.smithi027.stdout:4/855: truncate d24/d48/f10c 588554 0 2024-04-02T04:43:11.421 INFO:tasks.workunit.client.1.smithi027.stdout:6/851: chown d13/d1b/d24/d25/d56/d6d/daa/cbe 2422675 1 2024-04-02T04:43:11.422 INFO:tasks.workunit.client.1.smithi027.stdout:8/912: creat d5/d8/db/dd/d5f/d91/d9f/f133 x:0 0 0 2024-04-02T04:43:11.425 INFO:tasks.workunit.client.1.smithi027.stdout:4/856: unlink d24/fa8 0 2024-04-02T04:43:11.426 INFO:tasks.workunit.client.1.smithi027.stdout:6/852: rmdir d13/d1b/d48/d75/d9b/de2/d114 39 2024-04-02T04:43:11.428 INFO:tasks.workunit.client.1.smithi027.stdout:8/913: truncate d5/d8/db/dd/f51 32696 0 2024-04-02T04:43:11.428 INFO:tasks.workunit.client.1.smithi027.stdout:1/992: dread d0/de/d52/d74/df7/d3b/f77 [4194304,4194304] 0 2024-04-02T04:43:11.430 INFO:tasks.workunit.client.1.smithi027.stdout:8/914: write d5/d8/db/f29 [385870,28089] 0 2024-04-02T04:43:11.437 INFO:tasks.workunit.client.1.smithi027.stdout:8/915: chown d5/d8/l42 39 1 2024-04-02T04:43:11.438 INFO:tasks.workunit.client.1.smithi027.stdout:4/857: dread d24/d38/d44/fa5 [0,4194304] 0 2024-04-02T04:43:11.449 INFO:tasks.workunit.client.1.smithi027.stdout:4/858: rename d24/d38/d47/f93 to d24/d38/d44/d41/d101/d103/f111 0 2024-04-02T04:43:11.462 INFO:tasks.workunit.client.1.smithi027.stdout:6/853: dread d13/d1b/f2b [4194304,4194304] 0 2024-04-02T04:43:11.463 INFO:tasks.workunit.client.1.smithi027.stdout:4/859: creat d24/d2d/d100/f112 x:0 0 0 2024-04-02T04:43:11.464 INFO:tasks.workunit.client.1.smithi027.stdout:4/860: chown d24/fe8 1 1 2024-04-02T04:43:11.474 INFO:tasks.workunit.client.1.smithi027.stdout:1/993: dread d0/f69 [4194304,4194304] 0 2024-04-02T04:43:11.475 INFO:tasks.workunit.client.1.smithi027.stdout:6/854: dwrite d13/d1b/d48/d75/f83 [0,4194304] 0 2024-04-02T04:43:11.476 INFO:tasks.workunit.client.1.smithi027.stdout:4/861: link d24/d38/d44/d82/d97/lda d24/d38/d44/d82/d97/l113 0 2024-04-02T04:43:11.480 INFO:tasks.workunit.client.1.smithi027.stdout:1/994: creat d0/de/d52/d74/df7/d3b/d4f/f14f x:0 0 0 2024-04-02T04:43:11.481 INFO:tasks.workunit.client.1.smithi027.stdout:4/862: creat d24/d38/d47/dc0/dca/f114 x:0 0 0 2024-04-02T04:43:11.482 INFO:tasks.workunit.client.1.smithi027.stdout:6/855: truncate d13/d1b/d48/d58/dd3/fe6 511394 0 2024-04-02T04:43:11.486 INFO:tasks.workunit.client.1.smithi027.stdout:1/995: symlink d0/de/d52/d74/df7/d3b/d4f/l150 0 2024-04-02T04:43:11.489 INFO:tasks.workunit.client.1.smithi027.stdout:4/863: fdatasync d24/d38/d47/d62/db4/ff9 0 2024-04-02T04:43:11.491 INFO:tasks.workunit.client.1.smithi027.stdout:6/856: creat d13/d1b/d48/d75/d9b/de2/d114/f11c x:0 0 0 2024-04-02T04:43:11.493 INFO:tasks.workunit.client.1.smithi027.stdout:6/857: write d13/d1b/d48/d9c/ff3 [706795,64462] 0 2024-04-02T04:43:11.496 INFO:tasks.workunit.client.1.smithi027.stdout:1/996: rename d0/de/d52/d74/fc1 to d0/de/d52/d74/db7/df1/d25/d3c/d64/d149/f151 0 2024-04-02T04:43:11.505 INFO:tasks.workunit.client.1.smithi027.stdout:1/997: dwrite d0/de/d52/d74/df7/d3b/d4f/d9b/f12e [0,4194304] 0 2024-04-02T04:43:11.507 INFO:tasks.workunit.client.1.smithi027.stdout:1/998: truncate d0/de/d52/d74/df7/d3e/f14c 586505 0 2024-04-02T04:43:11.509 INFO:tasks.workunit.client.1.smithi027.stdout:4/864: getdents d24/d38/d44/d82/d97 0 2024-04-02T04:43:11.513 INFO:tasks.workunit.client.1.smithi027.stdout:4/865: mknod d24/c115 0 2024-04-02T04:43:11.525 INFO:tasks.workunit.client.1.smithi027.stdout:6/858: dwrite d13/fbf [0,4194304] 0 2024-04-02T04:43:11.529 INFO:tasks.workunit.client.1.smithi027.stdout:4/866: dread d24/d38/d44/d41/d101/d103/f111 [0,4194304] 0 2024-04-02T04:43:11.530 INFO:tasks.workunit.client.1.smithi027.stdout:1/999: dread d0/de/d52/d74/dbc/f10d [0,4194304] 0 2024-04-02T04:43:11.532 INFO:tasks.workunit.client.1.smithi027.stdout:6/859: mknod d13/d1b/d24/d28/d46/dcc/da7/c11d 0 2024-04-02T04:43:11.533 INFO:tasks.workunit.client.1.smithi027.stdout:6/860: chown d13/d1b/d48/d58/l85 4 1 2024-04-02T04:43:11.534 INFO:tasks.workunit.client.1.smithi027.stdout:4/867: mknod d24/d38/d44/d41/d49/c116 0 2024-04-02T04:43:11.539 INFO:tasks.workunit.client.1.smithi027.stdout:4/868: mknod d24/d38/d47/dc0/d106/c117 0 2024-04-02T04:43:11.539 INFO:tasks.workunit.client.1.smithi027.stdout:4/869: read - d24/d38/d44/d82/d95/faf zero size 2024-04-02T04:43:11.540 INFO:tasks.workunit.client.1.smithi027.stdout:4/870: fsync d24/d38/d44/d41/f80 0 2024-04-02T04:43:11.542 INFO:tasks.workunit.client.1.smithi027.stdout:4/871: dread d24/f33 [4194304,4194304] 0 2024-04-02T04:43:11.543 INFO:tasks.workunit.client.1.smithi027.stdout:4/872: dread - d24/d48/db0/fff zero size 2024-04-02T04:43:11.547 INFO:tasks.workunit.client.1.smithi027.stdout:4/873: mkdir d24/d38/d44/d41/d101/d118 0 2024-04-02T04:43:11.548 INFO:tasks.workunit.client.1.smithi027.stdout:4/874: write d24/ffe [427556,20550] 0 2024-04-02T04:43:11.552 INFO:tasks.workunit.client.1.smithi027.stdout:6/861: dread d13/d8d/fa3 [0,4194304] 0 2024-04-02T04:43:11.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.603+0000 7f2de6ffd700 1 -- 172.21.15.16:0/385390432 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mgrmap(e 28) v1 ==== 45905+0+0 (secure 0 0 0) 0x7f2de005d000 con 0x7f2df0100bd0 2024-04-02T04:43:11.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.603+0000 7f2de6ffd700 1 --2- 172.21.15.16:0/385390432 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f2ddc005e80 0x7f2ddc035ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:11.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.603+0000 7f2de6ffd700 1 -- 172.21.15.16:0/385390432 --> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ddc036e70 con 0x7f2ddc005e80 2024-04-02T04:43:11.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.611+0000 7f2df5863700 1 --2- 172.21.15.16:0/385390432 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f2ddc005e80 0x7f2ddc035ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:11.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.612+0000 7f2df5863700 1 --2- 172.21.15.16:0/385390432 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f2ddc005e80 0x7f2ddc035ba0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2dec00bfd0 tx=0x7f2dec016000 comp rx=0 tx=0).ready entity=mgr.14744 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:11.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.614+0000 7f2de6ffd700 1 -- 172.21.15.16:0/385390432 <== mgr.14744 v2:172.21.15.16:6800/2776599951 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f2ddc036e70 con 0x7f2ddc005e80 2024-04-02T04:43:11.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.616+0000 7f2df7ac7700 1 -- 172.21.15.16:0/385390432 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f2ddc005e80 msgr2=0x7f2ddc035ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:11.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.616+0000 7f2df7ac7700 1 --2- 172.21.15.16:0/385390432 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f2ddc005e80 0x7f2ddc035ba0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2dec00bfd0 tx=0x7f2dec016000 comp rx=0 tx=0).stop 2024-04-02T04:43:11.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.616+0000 7f2df7ac7700 1 -- 172.21.15.16:0/385390432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df0100bd0 msgr2=0x7f2df019b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:11.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.616+0000 7f2df7ac7700 1 --2- 172.21.15.16:0/385390432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df0100bd0 0x7f2df019b270 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2de0006010 tx=0x7f2de000bd70 comp rx=0 tx=0).stop 2024-04-02T04:43:11.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.616+0000 7f2df7ac7700 1 -- 172.21.15.16:0/385390432 shutdown_connections 2024-04-02T04:43:11.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.616+0000 7f2df7ac7700 1 --2- 172.21.15.16:0/385390432 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f2ddc005e80 0x7f2ddc035ba0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:11.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.616+0000 7f2df7ac7700 1 --2- 172.21.15.16:0/385390432 >> [v2:172.21.15.27:6828/1554339939,v1:172.21.15.27:6829/1554339939] conn(0x7f2ddc06db20 0x7f2ddc06ffe0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:11.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.616+0000 7f2df7ac7700 1 --2- 172.21.15.16:0/385390432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2df00ff9d0 0x7f2df0068f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:11.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.616+0000 7f2df7ac7700 1 --2- 172.21.15.16:0/385390432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df0100bd0 0x7f2df019b270 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:11.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.616+0000 7f2df7ac7700 1 -- 172.21.15.16:0/385390432 >> 172.21.15.16:0/385390432 conn(0x7f2df00faf70 msgr2=0x7f2df0104120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:11.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.616+0000 7f2df7ac7700 1 -- 172.21.15.16:0/385390432 shutdown_connections 2024-04-02T04:43:11.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.616+0000 7f2df7ac7700 1 -- 172.21.15.16:0/385390432 wait complete. 2024-04-02T04:43:11.635 INFO:tasks.workunit.client.1.smithi027.stdout:6/862: creat d13/d1b/d24/d28/f11e x:0 0 0 2024-04-02T04:43:11.637 INFO:tasks.workunit.client.1.smithi027.stdout:6/863: truncate d13/d1b/d48/d58/fa2 387851 0 2024-04-02T04:43:11.639 INFO:tasks.workunit.client.1.smithi027.stdout:6/864: chown d13/d1b/d48/d75/d9b/de2/d114/l79 765128 1 2024-04-02T04:43:11.641 INFO:tasks.workunit.client.1.smithi027.stdout:6/865: chown d13/d1b/d24/d28/d46/dcc/da7/ce8 93035293 1 2024-04-02T04:43:11.643 INFO:tasks.workunit.client.1.smithi027.stdout:6/866: creat d13/d1b/d24/d28/d46/f11f x:0 0 0 2024-04-02T04:43:11.663 INFO:tasks.workunit.client.1.smithi027.stdout:6/867: write d13/d1b/d24/f84 [1010077,94746] 0 2024-04-02T04:43:11.672 INFO:tasks.workunit.client.1.smithi027.stdout:6/868: write d13/d1b/d48/d58/f6a [3541108,120569] 0 2024-04-02T04:43:11.685 INFO:tasks.workunit.client.1.smithi027.stdout:3/931: sync 2024-04-02T04:43:11.685 INFO:tasks.workunit.client.1.smithi027.stdout:8/916: sync 2024-04-02T04:43:11.687 INFO:tasks.workunit.client.1.smithi027.stdout:6/869: dwrite d13/d1b/d48/d58/db9/f110 [0,4194304] 0 2024-04-02T04:43:11.688 INFO:tasks.workunit.client.1.smithi027.stdout:3/932: chown d1/de/df/d13/f5c 5578 1 2024-04-02T04:43:11.689 INFO:tasks.workunit.client.1.smithi027.stdout:8/917: dread - d5/d8/db/dd/d4a/dc9/fa6 zero size 2024-04-02T04:43:11.691 INFO:tasks.workunit.client.1.smithi027.stdout:8/918: chown d5/d8/db/dd/d5f/db2/dc6/d106 126558193 1 2024-04-02T04:43:11.737 INFO:tasks.workunit.client.1.smithi027.stdout:4/875: sync 2024-04-02T04:43:11.744 INFO:tasks.workunit.client.1.smithi027.stdout:4/876: dwrite d24/d38/d47/fdf [0,4194304] 0 2024-04-02T04:43:11.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.742+0000 7fe8fac0e700 1 -- 172.21.15.16:0/3442627490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8f4101cd0 msgr2=0x7fe8f4102130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:11.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.742+0000 7fe8fac0e700 1 --2- 172.21.15.16:0/3442627490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8f4101cd0 0x7fe8f4102130 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009910 tx=0x7fe8e0009c20 comp rx=0 tx=0).stop 2024-04-02T04:43:11.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.742+0000 7fe8fac0e700 1 -- 172.21.15.16:0/3442627490 shutdown_connections 2024-04-02T04:43:11.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.742+0000 7fe8fac0e700 1 --2- 172.21.15.16:0/3442627490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8f4101cd0 0x7fe8f4102130 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:11.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.742+0000 7fe8fac0e700 1 --2- 172.21.15.16:0/3442627490 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8f4100ad0 0x7fe8f4100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:11.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.742+0000 7fe8fac0e700 1 -- 172.21.15.16:0/3442627490 >> 172.21.15.16:0/3442627490 conn(0x7fe8f40fc090 msgr2=0x7fe8f40fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:11.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.742+0000 7fe8fac0e700 1 -- 172.21.15.16:0/3442627490 shutdown_connections 2024-04-02T04:43:11.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.742+0000 7fe8fac0e700 1 -- 172.21.15.16:0/3442627490 wait complete. 2024-04-02T04:43:11.746 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:11 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:43:11.746 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:11 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:43:11.746 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:11 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:43:11.746 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:11 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:43:11.746 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:11 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:43:11.746 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:11 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:43:11.747 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:11 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "mgr fail", "who": "smithi027.kjzcsc"}]: dispatch 2024-04-02T04:43:11.747 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:11 smithi016 ceph-mon[28969]: osdmap e48: 6 total, 6 up, 6 in 2024-04-02T04:43:11.747 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:11 smithi016 ceph-mon[28969]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd='[{"prefix": "mgr fail", "who": "smithi027.kjzcsc"}]': finished 2024-04-02T04:43:11.747 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:11 smithi016 ceph-mon[28969]: mgrmap e27: smithi016.eqnebx(active, starting, since 0.00792843s) 2024-04-02T04:43:11.747 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:11 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi016"}]: dispatch 2024-04-02T04:43:11.747 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:11 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2024-04-02T04:43:11.747 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:11 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi016.fyzrnm"}]: dispatch 2024-04-02T04:43:11.747 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:11 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi016.uhgkgg"}]: dispatch 2024-04-02T04:43:11.747 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:11 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.zwrfoo"}]: dispatch 2024-04-02T04:43:11.747 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:11 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.sdublg"}]: dispatch 2024-04-02T04:43:11.747 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:11 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mgr metadata", "who": "smithi016.eqnebx", "id": "smithi016.eqnebx"}]: dispatch 2024-04-02T04:43:11.747 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:11 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T04:43:11.747 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:11 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T04:43:11.747 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:11 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T04:43:11.748 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:11 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T04:43:11.748 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:11 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T04:43:11.748 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:11 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T04:43:11.748 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:11 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T04:43:11.748 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:11 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T04:43:11.748 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:11 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T04:43:11.748 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:11 smithi016 ceph-mon[28969]: Manager daemon smithi016.eqnebx is now available 2024-04-02T04:43:11.748 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:11 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:43:11.748 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:11 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:43:11.748 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:11 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi016.eqnebx/mirror_snapshot_schedule"}]: dispatch 2024-04-02T04:43:11.748 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:11 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi016.eqnebx/trash_purge_schedule"}]: dispatch 2024-04-02T04:43:11.748 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:11 smithi016 ceph-mon[28969]: [02/Apr/2024:04:43:11] ENGINE Bus STARTING 2024-04-02T04:43:11.748 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:11 smithi016 ceph-mon[28969]: [02/Apr/2024:04:43:11] ENGINE Serving on http://172.21.15.16:8765 2024-04-02T04:43:11.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.742+0000 7fe8fac0e700 1 Processor -- start 2024-04-02T04:43:11.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.743+0000 7fe8fac0e700 1 -- start start 2024-04-02T04:43:11.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.743+0000 7fe8fac0e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8f4100ad0 0x7fe8f4196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:11.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.743+0000 7fe8fac0e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8f4101cd0 0x7fe8f4196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:11.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.743+0000 7fe8fac0e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f4197080 con 0x7fe8f4101cd0 2024-04-02T04:43:11.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.743+0000 7fe8fac0e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f41971c0 con 0x7fe8f4100ad0 2024-04-02T04:43:11.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.743+0000 7fe8f3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8f4101cd0 0x7fe8f4196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:11.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.743+0000 7fe8f3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8f4101cd0 0x7fe8f4196a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34510/0 (socket says 172.21.15.16:34510) 2024-04-02T04:43:11.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.743+0000 7fe8f3fff700 1 -- 172.21.15.16:0/1576437582 learned_addr learned my addr 172.21.15.16:0/1576437582 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:43:11.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.743+0000 7fe8f89aa700 1 --2- 172.21.15.16:0/1576437582 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8f4100ad0 0x7fe8f4196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:11.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.743+0000 7fe8f3fff700 1 -- 172.21.15.16:0/1576437582 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8f4100ad0 msgr2=0x7fe8f4196520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:11.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.743+0000 7fe8f3fff700 1 --2- 172.21.15.16:0/1576437582 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8f4100ad0 0x7fe8f4196520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:11.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.743+0000 7fe8f3fff700 1 -- 172.21.15.16:0/1576437582 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e00095f0 con 0x7fe8f4101cd0 2024-04-02T04:43:11.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.743+0000 7fe8f3fff700 1 --2- 172.21.15.16:0/1576437582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8f4101cd0 0x7fe8f4196a60 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe8e000b800 tx=0x7fe8e000f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:11.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.744+0000 7fe8f1ffb700 1 -- 172.21.15.16:0/1576437582 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8e001d070 con 0x7fe8f4101cd0 2024-04-02T04:43:11.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.744+0000 7fe8fac0e700 1 -- 172.21.15.16:0/1576437582 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8f419bbc0 con 0x7fe8f4101cd0 2024-04-02T04:43:11.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.744+0000 7fe8f1ffb700 1 -- 172.21.15.16:0/1576437582 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe8e00052a0 con 0x7fe8f4101cd0 2024-04-02T04:43:11.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.744+0000 7fe8f1ffb700 1 -- 172.21.15.16:0/1576437582 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8e0017400 con 0x7fe8f4101cd0 2024-04-02T04:43:11.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.744+0000 7fe8fac0e700 1 -- 172.21.15.16:0/1576437582 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8f419c130 con 0x7fe8f4101cd0 2024-04-02T04:43:11.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.745+0000 7fe8f1ffb700 1 -- 172.21.15.16:0/1576437582 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 28) v1 ==== 45905+0+0 (secure 0 0 0) 0x7fe8e0017670 con 0x7fe8f4101cd0 2024-04-02T04:43:11.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.745+0000 7fe8fac0e700 1 -- 172.21.15.16:0/1576437582 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8f4190650 con 0x7fe8f4101cd0 2024-04-02T04:43:11.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.745+0000 7fe8f1ffb700 1 --2- 172.21.15.16:0/1576437582 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7fe8e4038d00 0x7fe8e403b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:11.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.745+0000 7fe8f1ffb700 1 -- 172.21.15.16:0/1576437582 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fe8e004e120 con 0x7fe8f4101cd0 2024-04-02T04:43:11.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.746+0000 7fe8f89aa700 1 --2- 172.21.15.16:0/1576437582 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7fe8e4038d00 0x7fe8e403b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:11.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.746+0000 7fe8f89aa700 1 --2- 172.21.15.16:0/1576437582 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7fe8e4038d00 0x7fe8e403b1c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4101b30 tx=0x7fe8e8009260 comp rx=0 tx=0).ready entity=mgr.14744 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:11.752 INFO:tasks.workunit.client.1.smithi027.stdout:4/877: read d24/d38/d44/d82/d95/fb6 [28731,71837] 0 2024-04-02T04:43:11.755 INFO:tasks.workunit.client.1.smithi027.stdout:3/933: creat d1/de/df/d13/d73/db1/f130 x:0 0 0 2024-04-02T04:43:11.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.752+0000 7fe8f1ffb700 1 -- 172.21.15.16:0/1576437582 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184299 (secure 0 0 0) 0x7fe8e0016e20 con 0x7fe8f4101cd0 2024-04-02T04:43:11.763 INFO:tasks.workunit.client.1.smithi027.stdout:6/870: rename d13/d1b/d24/d25/d56/dc8/c113 to d13/d1b/d24/d25/d42/c120 0 2024-04-02T04:43:11.764 INFO:tasks.workunit.client.1.smithi027.stdout:8/919: dwrite d5/d8/f47 [0,4194304] 0 2024-04-02T04:43:11.766 INFO:tasks.workunit.client.1.smithi027.stdout:3/934: mkdir d1/de/df/d9e/d131 0 2024-04-02T04:43:11.767 INFO:tasks.workunit.client.1.smithi027.stdout:6/871: dread - d13/d1b/d24/d25/d56/fdd zero size 2024-04-02T04:43:11.768 INFO:tasks.workunit.client.1.smithi027.stdout:3/935: dread - d1/de/df/d9e/dc6/de4/f12a zero size 2024-04-02T04:43:11.769 INFO:tasks.workunit.client.1.smithi027.stdout:3/936: chown d1/l31 645782 1 2024-04-02T04:43:11.770 INFO:tasks.workunit.client.1.smithi027.stdout:8/920: rename d5/d8/db/dd/d5f/d27/d31/lf0 to d5/d8/d15/d71/db0/l134 0 2024-04-02T04:43:11.775 INFO:tasks.workunit.client.1.smithi027.stdout:8/921: stat d5/d8/db/dd/d5f/d27/d31/d5b/dc4/l102 0 2024-04-02T04:43:11.783 INFO:tasks.workunit.client.1.smithi027.stdout:8/922: rename d5/d8/l49 to d5/d8/db/dd/d5f/db2/d10b/l135 0 2024-04-02T04:43:11.784 INFO:tasks.workunit.client.1.smithi027.stdout:6/872: link d13/d1b/d48/d9c/df7/lb6 d13/d1b/d24/d25/d56/l121 0 2024-04-02T04:43:11.788 INFO:tasks.workunit.client.1.smithi027.stdout:8/923: getdents d5/d8/db/dd/d4a 0 2024-04-02T04:43:11.793 INFO:tasks.workunit.client.1.smithi027.stdout:8/924: getdents d5/d8/d15/d71/d96 0 2024-04-02T04:43:11.795 INFO:tasks.workunit.client.1.smithi027.stdout:8/925: fsync d5/d8/db/dd/f10 0 2024-04-02T04:43:11.798 INFO:tasks.workunit.client.1.smithi027.stdout:8/926: chown d5/d8/c23 2657172 1 2024-04-02T04:43:11.819 INFO:tasks.workunit.client.1.smithi027.stdout:8/927: truncate d5/d8/db/dd/d5f/d27/f50 4123438 0 2024-04-02T04:43:11.820 INFO:tasks.workunit.client.1.smithi027.stdout:3/937: dwrite d1/fab [0,4194304] 0 2024-04-02T04:43:11.828 INFO:tasks.workunit.client.1.smithi027.stdout:8/928: creat d5/d8/db/dd/d5f/db2/d10b/d123/f136 x:0 0 0 2024-04-02T04:43:11.830 INFO:tasks.workunit.client.1.smithi027.stdout:3/938: fdatasync d1/de/f30 0 2024-04-02T04:43:11.832 INFO:tasks.workunit.client.1.smithi027.stdout:8/929: mkdir d5/d8/db/d68/d137 0 2024-04-02T04:43:11.835 INFO:tasks.workunit.client.1.smithi027.stdout:8/930: read d5/d8/db/dd/d4a/dc9/feb [327368,127741] 0 2024-04-02T04:43:11.837 INFO:tasks.workunit.client.1.smithi027.stdout:3/939: creat d1/de/df/d9e/dc6/f132 x:0 0 0 2024-04-02T04:43:11.838 INFO:tasks.workunit.client.1.smithi027.stdout:3/940: chown d1/de/df/d61/f4c 7203722 1 2024-04-02T04:43:11.841 INFO:tasks.workunit.client.1.smithi027.stdout:8/931: rename d5/d8/db/dd/d5f/d91/d9f/cc7 to d5/db5/c138 0 2024-04-02T04:43:11.846 INFO:tasks.workunit.client.1.smithi027.stdout:3/941: dwrite d1/de/df/da5/f129 [0,4194304] 0 2024-04-02T04:43:11.846 INFO:tasks.workunit.client.1.smithi027.stdout:8/932: fdatasync d5/d8/d15/d71/db0/d125/f128 0 2024-04-02T04:43:11.860 INFO:tasks.workunit.client.1.smithi027.stdout:8/933: symlink d5/d8/db/dd/d5f/db2/l139 0 2024-04-02T04:43:11.861 INFO:tasks.workunit.client.1.smithi027.stdout:3/942: link d1/de/l50 d1/d1a/l133 0 2024-04-02T04:43:11.863 INFO:tasks.workunit.client.1.smithi027.stdout:8/934: write d5/d8/d15/d71/db0/d125/f12f [35874,27652] 0 2024-04-02T04:43:11.865 INFO:tasks.workunit.client.1.smithi027.stdout:3/943: mknod d1/de/df/d13/d73/de2/c134 0 2024-04-02T04:43:11.866 INFO:tasks.workunit.client.1.smithi027.stdout:8/935: creat d5/d8/db/dd/d4a/dc9/f13a x:0 0 0 2024-04-02T04:43:11.868 INFO:tasks.workunit.client.1.smithi027.stdout:8/936: chown d5/d8/db/dd/d5f/db2/dc6/d106 452624521 1 2024-04-02T04:43:11.871 INFO:tasks.workunit.client.1.smithi027.stdout:3/944: dread - d1/de/df/d13/d73/d77/d102/f80 zero size 2024-04-02T04:43:11.872 INFO:tasks.workunit.client.1.smithi027.stdout:8/937: symlink d5/d8/db/dd/d5f/d91/l13b 0 2024-04-02T04:43:11.876 INFO:tasks.workunit.client.1.smithi027.stdout:8/938: truncate d5/d8/d15/d71/f100 1340512 0 2024-04-02T04:43:11.878 INFO:tasks.workunit.client.1.smithi027.stdout:8/939: chown d5/d8/db/dd/d5f/db2/d10b/d123/f129 70996 1 2024-04-02T04:43:11.881 INFO:tasks.workunit.client.1.smithi027.stdout:8/940: chown d5/d8/db/dd/d5f/d91/l11b 6699662 1 2024-04-02T04:43:11.881 INFO:tasks.workunit.client.1.smithi027.stdout:3/945: getdents d1 0 2024-04-02T04:43:11.882 INFO:tasks.workunit.client.1.smithi027.stdout:8/941: read - d5/d8/db/dd/d5f/db2/dc6/d106/f10f zero size 2024-04-02T04:43:11.892 INFO:tasks.workunit.client.1.smithi027.stdout:8/942: write d5/d8/db/dd/f3b [882111,83372] 0 2024-04-02T04:43:11.894 INFO:tasks.workunit.client.1.smithi027.stdout:3/946: creat d1/de/df/f135 x:0 0 0 2024-04-02T04:43:11.895 INFO:tasks.workunit.client.1.smithi027.stdout:3/947: fdatasync d1/de/df/d13/d73/db1/f116 0 2024-04-02T04:43:11.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:11 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:43:11.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:11 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:43:11.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:11 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:43:11.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:11 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:43:11.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:11 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' 2024-04-02T04:43:11.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:11 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:43:11.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:11 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "mgr fail", "who": "smithi027.kjzcsc"}]: dispatch 2024-04-02T04:43:11.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:11 smithi027 ceph-mon[37550]: osdmap e48: 6 total, 6 up, 6 in 2024-04-02T04:43:11.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:11 smithi027 ceph-mon[37550]: from='mgr.24415 172.21.15.27:0/2776367365' entity='mgr.smithi027.kjzcsc' cmd='[{"prefix": "mgr fail", "who": "smithi027.kjzcsc"}]': finished 2024-04-02T04:43:11.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:11 smithi027 ceph-mon[37550]: mgrmap e27: smithi016.eqnebx(active, starting, since 0.00792843s) 2024-04-02T04:43:11.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:11 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi016"}]: dispatch 2024-04-02T04:43:11.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:11 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2024-04-02T04:43:11.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:11 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi016.fyzrnm"}]: dispatch 2024-04-02T04:43:11.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:11 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi016.uhgkgg"}]: dispatch 2024-04-02T04:43:11.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:11 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.zwrfoo"}]: dispatch 2024-04-02T04:43:11.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:11 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.sdublg"}]: dispatch 2024-04-02T04:43:11.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:11 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mgr metadata", "who": "smithi016.eqnebx", "id": "smithi016.eqnebx"}]: dispatch 2024-04-02T04:43:11.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:11 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T04:43:11.898 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:11 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T04:43:11.898 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:11 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T04:43:11.898 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:11 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T04:43:11.898 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:11 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T04:43:11.898 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:11 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T04:43:11.898 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:11 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T04:43:11.898 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:11 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T04:43:11.898 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:11 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T04:43:11.898 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:11 smithi027 ceph-mon[37550]: Manager daemon smithi016.eqnebx is now available 2024-04-02T04:43:11.898 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:11 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:43:11.898 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:11 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:43:11.898 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:11 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi016.eqnebx/mirror_snapshot_schedule"}]: dispatch 2024-04-02T04:43:11.898 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:11 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi016.eqnebx/trash_purge_schedule"}]: dispatch 2024-04-02T04:43:11.898 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:11 smithi027 ceph-mon[37550]: [02/Apr/2024:04:43:11] ENGINE Bus STARTING 2024-04-02T04:43:11.898 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:11 smithi027 ceph-mon[37550]: [02/Apr/2024:04:43:11] ENGINE Serving on http://172.21.15.16:8765 2024-04-02T04:43:11.899 INFO:tasks.workunit.client.1.smithi027.stdout:3/948: fdatasync d1/de/df/d13/d73/db1/f130 0 2024-04-02T04:43:11.899 INFO:tasks.workunit.client.1.smithi027.stdout:8/943: creat d5/d8/db/dd/d5f/d27/d31/f13c x:0 0 0 2024-04-02T04:43:11.901 INFO:tasks.workunit.client.1.smithi027.stdout:3/949: mkdir d1/de/d55/d63/d136 0 2024-04-02T04:43:11.902 INFO:tasks.workunit.client.1.smithi027.stdout:8/944: fsync d5/f7a 0 2024-04-02T04:43:11.909 INFO:tasks.workunit.client.1.smithi027.stdout:8/945: rename d5/d8/db/f76 to d5/d8/db/dd/d5f/d27/d31/db9/f13d 0 2024-04-02T04:43:11.916 INFO:tasks.workunit.client.1.smithi027.stdout:3/950: link d1/de/d55/lb0 d1/de/d55/d63/da4/de3/l137 0 2024-04-02T04:43:11.916 INFO:tasks.workunit.client.1.smithi027.stdout:8/946: read d5/d8/db/dd/d5f/d27/d31/d5b/dc4/fe4 [2570858,4520] 0 2024-04-02T04:43:11.924 INFO:tasks.workunit.client.1.smithi027.stdout:3/951: mknod d1/de/df/d13/d73/de2/d125/c138 0 2024-04-02T04:43:11.928 INFO:tasks.workunit.client.1.smithi027.stdout:3/952: rmdir d1 39 2024-04-02T04:43:11.928 INFO:tasks.workunit.client.1.smithi027.stdout:4/878: sync 2024-04-02T04:43:11.928 INFO:tasks.workunit.client.1.smithi027.stdout:8/947: dwrite d5/db5/fd9 [0,4194304] 0 2024-04-02T04:43:11.936 INFO:tasks.workunit.client.1.smithi027.stdout:4/879: rmdir d24/d38/d47/d62 39 2024-04-02T04:43:11.941 INFO:tasks.workunit.client.1.smithi027.stdout:3/953: mkdir d1/de/d55/d63/da4/de3/d139 0 2024-04-02T04:43:11.943 INFO:tasks.workunit.client.1.smithi027.stdout:4/880: rmdir d24/d38/d44/d41 39 2024-04-02T04:43:11.947 INFO:tasks.workunit.client.1.smithi027.stdout:8/948: creat d5/d8/db/dd/d5f/d27/f13e x:0 0 0 2024-04-02T04:43:11.948 INFO:tasks.workunit.client.1.smithi027.stdout:8/949: stat d5/d8/db/dd/d5f/db2/dcf/c118 0 2024-04-02T04:43:11.952 INFO:tasks.workunit.client.1.smithi027.stdout:3/954: dread d1/de/df/d61/f6c [0,4194304] 0 2024-04-02T04:43:11.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.984+0000 7fe8fac0e700 1 -- 172.21.15.16:0/1576437582 --> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe8f402cdc0 con 0x7fe8e4038d00 2024-04-02T04:43:11.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.996+0000 7fe8f1ffb700 1 -- 172.21.15.16:0/1576437582 <== mgr.14744 v2:172.21.15.16:6800/2776599951 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fe8f402cdc0 con 0x7fe8e4038d00 2024-04-02T04:43:11.999 INFO:teuthology.orchestra.run.smithi016.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:43:11.999 INFO:teuthology.orchestra.run.smithi016.stdout:alertmanager.smithi016 smithi016 *:9093,9094 running (7m) 6s ago 9m 24.2M - 0.25.0 c8568f914cd2 b95d8354b016 2024-04-02T04:43:12.000 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi016 smithi016 running (9m) 6s ago 9m 7713k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 9de6c16a487f 2024-04-02T04:43:12.000 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi027 smithi027 running (8m) 52s ago 8m 7633k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 99956592471a 2024-04-02T04:43:12.000 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi016 smithi016 running (9m) 6s ago 9m 6656k - 18.2.2-1059-g4d05f4ca 909dce4fe53d bd73919b0711 2024-04-02T04:43:12.000 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi027 smithi027 running (8m) 52s ago 8m 6660k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 7a54c37f202e 2024-04-02T04:43:12.000 INFO:teuthology.orchestra.run.smithi016.stdout:grafana.smithi016 smithi016 *:3000 running (7m) 6s ago 8m 85.9M - 9.4.7 954c08fa6188 7c43d133569c 2024-04-02T04:43:12.000 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.fyzrnm smithi016 running (3m) 6s ago 3m 15.3M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 70f28b9572a6 2024-04-02T04:43:12.000 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.uhgkgg smithi016 running (3m) 6s ago 3m 13.5M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 0aa4d16562a6 2024-04-02T04:43:12.000 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.sdublg smithi027 running (3m) 52s ago 3m 17.2M - 18.2.2-1059-g4d05f4ca 909dce4fe53d a538451099db 2024-04-02T04:43:12.000 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.zwrfoo smithi027 running (3m) 52s ago 3m 174M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 4cb3fa428d71 2024-04-02T04:43:12.000 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi016.eqnebx smithi016 *:8443,9283,8765 running (13s) 6s ago 10m 65.6M - 19.0.0-1979-g6cec793d 78f830190839 d61ed91b4577 2024-04-02T04:43:12.001 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi027.kjzcsc smithi027 *:8443,9283,8765 running (96s) 52s ago 8m 488M - 19.0.0-1979-g6cec793d 78f830190839 a1e2b7522dcb 2024-04-02T04:43:12.001 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi016 smithi016 running (10m) 6s ago 10m 51.1M 2048M 18.2.2-1059-g4d05f4ca 909dce4fe53d 24d5820dc994 2024-04-02T04:43:12.001 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi027 smithi027 running (7m) 52s ago 7m 41.1M 2048M 18.2.2-1059-g4d05f4ca 909dce4fe53d b77563184a1e 2024-04-02T04:43:12.001 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi016 smithi016 *:9100 running (9m) 6s ago 9m 13.9M - 1.5.0 0da6a335fe13 09f54c65b53c 2024-04-02T04:43:12.001 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi027 smithi027 *:9100 running (8m) 52s ago 8m 15.4M - 1.5.0 0da6a335fe13 e41a657c7443 2024-04-02T04:43:12.001 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 smithi016 running (7m) 6s ago 7m 185M 1217M 18.2.2-1059-g4d05f4ca 909dce4fe53d 3f40108debd6 2024-04-02T04:43:12.001 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 smithi016 running (6m) 6s ago 6m 190M 1217M 18.2.2-1059-g4d05f4ca 909dce4fe53d 7361abc9e1b6 2024-04-02T04:43:12.001 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 smithi016 running (6m) 6s ago 6m 161M 1217M 18.2.2-1059-g4d05f4ca 909dce4fe53d 52e592b4d1f2 2024-04-02T04:43:12.001 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 smithi027 running (5m) 52s ago 5m 89.1M 2240M 18.2.2-1059-g4d05f4ca 909dce4fe53d cf7c7fe022f0 2024-04-02T04:43:12.001 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 smithi027 running (5m) 52s ago 5m 90.0M 2240M 18.2.2-1059-g4d05f4ca 909dce4fe53d 8cbbedf8cac9 2024-04-02T04:43:12.001 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 smithi027 running (5m) 52s ago 5m 84.8M 2240M 18.2.2-1059-g4d05f4ca 909dce4fe53d ac6094415529 2024-04-02T04:43:12.001 INFO:teuthology.orchestra.run.smithi016.stdout:prometheus.smithi016 smithi016 *:9095 running (25s) 6s ago 8m 44.6M - 2.43.0 a07b618ecd1d 80a629d224f2 2024-04-02T04:43:12.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.998+0000 7fe8fac0e700 1 -- 172.21.15.16:0/1576437582 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7fe8e4038d00 msgr2=0x7fe8e403b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:12.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.998+0000 7fe8fac0e700 1 --2- 172.21.15.16:0/1576437582 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7fe8e4038d00 0x7fe8e403b1c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4101b30 tx=0x7fe8e8009260 comp rx=0 tx=0).stop 2024-04-02T04:43:12.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.998+0000 7fe8fac0e700 1 -- 172.21.15.16:0/1576437582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8f4101cd0 msgr2=0x7fe8f4196a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:12.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.998+0000 7fe8fac0e700 1 --2- 172.21.15.16:0/1576437582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8f4101cd0 0x7fe8f4196a60 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe8e000b800 tx=0x7fe8e000f510 comp rx=0 tx=0).stop 2024-04-02T04:43:12.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.998+0000 7fe8fac0e700 1 -- 172.21.15.16:0/1576437582 shutdown_connections 2024-04-02T04:43:12.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.998+0000 7fe8fac0e700 1 --2- 172.21.15.16:0/1576437582 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7fe8e4038d00 0x7fe8e403b1c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:12.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.998+0000 7fe8fac0e700 1 --2- 172.21.15.16:0/1576437582 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8f4100ad0 0x7fe8f4196520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:12.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.998+0000 7fe8fac0e700 1 --2- 172.21.15.16:0/1576437582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8f4101cd0 0x7fe8f4196a60 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:12.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.998+0000 7fe8fac0e700 1 -- 172.21.15.16:0/1576437582 >> 172.21.15.16:0/1576437582 conn(0x7fe8f40fc090 msgr2=0x7fe8f4105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:12.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.999+0000 7fe8fac0e700 1 -- 172.21.15.16:0/1576437582 shutdown_connections 2024-04-02T04:43:12.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:11.999+0000 7fe8fac0e700 1 -- 172.21.15.16:0/1576437582 wait complete. 2024-04-02T04:43:12.040 INFO:tasks.workunit.client.1.smithi027.stdout:3/955: mknod d1/de/df/c13a 0 2024-04-02T04:43:12.086 INFO:tasks.workunit.client.1.smithi027.stdout:6/873: sync 2024-04-02T04:43:12.102 INFO:tasks.workunit.client.1.smithi027.stdout:3/956: creat d1/de/df/d9e/da0/f13b x:0 0 0 2024-04-02T04:43:12.105 INFO:tasks.workunit.client.1.smithi027.stdout:8/950: write d5/d8/f33 [4549741,87523] 0 2024-04-02T04:43:12.106 INFO:tasks.workunit.client.1.smithi027.stdout:3/957: truncate d1/f64 831714 0 2024-04-02T04:43:12.107 INFO:tasks.workunit.client.1.smithi027.stdout:6/874: mknod d13/d1b/d48/c122 0 2024-04-02T04:43:12.108 INFO:tasks.workunit.client.1.smithi027.stdout:8/951: chown d5/d8/db/dd/d5f/d27/d31/d5b/d6c/c73 652559 1 2024-04-02T04:43:12.109 INFO:tasks.workunit.client.1.smithi027.stdout:6/875: truncate d13/d1b/d48/f119 1688344 0 2024-04-02T04:43:12.111 INFO:tasks.workunit.client.1.smithi027.stdout:3/958: creat d1/de/df/dec/f13c x:0 0 0 2024-04-02T04:43:12.113 INFO:tasks.workunit.client.1.smithi027.stdout:8/952: mknod d5/d8/db/dd/d5f/d27/d31/d5b/dc4/dea/c13f 0 2024-04-02T04:43:12.114 INFO:tasks.workunit.client.1.smithi027.stdout:3/959: creat d1/de/df/d13/db3/f13d x:0 0 0 2024-04-02T04:43:12.116 INFO:tasks.workunit.client.1.smithi027.stdout:3/960: dread - d1/de/d55/d99/f106 zero size 2024-04-02T04:43:12.116 INFO:tasks.workunit.client.1.smithi027.stdout:6/876: dwrite d13/d1b/d24/d28/d46/dcc/f107 [0,4194304] 0 2024-04-02T04:43:12.118 INFO:tasks.workunit.client.1.smithi027.stdout:3/961: chown d1/de/c2f 0 1 2024-04-02T04:43:12.118 INFO:tasks.workunit.client.1.smithi027.stdout:6/877: stat d13/d1b/d24/d25/d42/c103 0 2024-04-02T04:43:12.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.116+0000 7f4197314700 1 -- 172.21.15.16:0/1725411581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4190101c60 msgr2=0x7f41901020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:12.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.116+0000 7f4197314700 1 --2- 172.21.15.16:0/1725411581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4190101c60 0x7f41901020c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4180009910 tx=0x7f4180009c20 comp rx=0 tx=0).stop 2024-04-02T04:43:12.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.116+0000 7f4197314700 1 -- 172.21.15.16:0/1725411581 shutdown_connections 2024-04-02T04:43:12.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.116+0000 7f4197314700 1 --2- 172.21.15.16:0/1725411581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4190101c60 0x7f41901020c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:12.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.116+0000 7f4197314700 1 --2- 172.21.15.16:0/1725411581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4190100a60 0x7f4190100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:12.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.116+0000 7f4197314700 1 -- 172.21.15.16:0/1725411581 >> 172.21.15.16:0/1725411581 conn(0x7f41900fc020 msgr2=0x7f41900fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:12.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.116+0000 7f4197314700 1 -- 172.21.15.16:0/1725411581 shutdown_connections 2024-04-02T04:43:12.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.116+0000 7f4197314700 1 -- 172.21.15.16:0/1725411581 wait complete. 2024-04-02T04:43:12.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.117+0000 7f4197314700 1 Processor -- start 2024-04-02T04:43:12.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.117+0000 7f4197314700 1 -- start start 2024-04-02T04:43:12.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.117+0000 7f4197314700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4190100a60 0x7f4190074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:12.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.117+0000 7f4197314700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4190101c60 0x7f41900731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:12.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.117+0000 7f4197314700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41900736e0 con 0x7f4190100a60 2024-04-02T04:43:12.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.117+0000 7f4197314700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4190073820 con 0x7f4190101c60 2024-04-02T04:43:12.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.117+0000 7f41950b0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4190100a60 0x7f4190074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:12.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.117+0000 7f41950b0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4190100a60 0x7f4190074b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34524/0 (socket says 172.21.15.16:34524) 2024-04-02T04:43:12.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.117+0000 7f41950b0700 1 -- 172.21.15.16:0/52599282 learned_addr learned my addr 172.21.15.16:0/52599282 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:43:12.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.117+0000 7f41948af700 1 --2- 172.21.15.16:0/52599282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4190101c60 0x7f41900731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:12.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.117+0000 7f41950b0700 1 -- 172.21.15.16:0/52599282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4190101c60 msgr2=0x7f41900731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:12.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.117+0000 7f41950b0700 1 --2- 172.21.15.16:0/52599282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4190101c60 0x7f41900731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:12.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.117+0000 7f41950b0700 1 -- 172.21.15.16:0/52599282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41800095f0 con 0x7f4190100a60 2024-04-02T04:43:12.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.118+0000 7f41950b0700 1 --2- 172.21.15.16:0/52599282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4190100a60 0x7f4190074b00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f418c00d900 tx=0x7f418c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:12.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.118+0000 7f41867fc700 1 -- 172.21.15.16:0/52599282 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f418c0041f0 con 0x7f4190100a60 2024-04-02T04:43:12.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.118+0000 7f4197314700 1 -- 172.21.15.16:0/52599282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4190073b00 con 0x7f4190100a60 2024-04-02T04:43:12.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.118+0000 7f41867fc700 1 -- 172.21.15.16:0/52599282 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f418c004350 con 0x7f4190100a60 2024-04-02T04:43:12.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.118+0000 7f41867fc700 1 -- 172.21.15.16:0/52599282 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f418c003cf0 con 0x7f4190100a60 2024-04-02T04:43:12.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.118+0000 7f4197314700 1 -- 172.21.15.16:0/52599282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f419019e4d0 con 0x7f4190100a60 2024-04-02T04:43:12.123 INFO:tasks.workunit.client.1.smithi027.stdout:6/878: read d13/d1b/d24/f84 [978451,102257] 0 2024-04-02T04:43:12.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.119+0000 7f41867fc700 1 -- 172.21.15.16:0/52599282 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 28) v1 ==== 45905+0+0 (secure 0 0 0) 0x7f418c010460 con 0x7f4190100a60 2024-04-02T04:43:12.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.119+0000 7f41867fc700 1 --2- 172.21.15.16:0/52599282 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f417c038d00 0x7f417c03b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:12.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.119+0000 7f41867fc700 1 -- 172.21.15.16:0/52599282 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f418c04cb30 con 0x7f4190100a60 2024-04-02T04:43:12.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.119+0000 7f41948af700 1 --2- 172.21.15.16:0/52599282 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f417c038d00 0x7f417c03b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:12.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.119+0000 7f4197314700 1 -- 172.21.15.16:0/52599282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41901905e0 con 0x7f4190100a60 2024-04-02T04:43:12.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.120+0000 7f41948af700 1 --2- 172.21.15.16:0/52599282 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f417c038d00 0x7f417c03b1c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4180005090 tx=0x7f4180005960 comp rx=0 tx=0).ready entity=mgr.14744 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:12.124 INFO:tasks.workunit.client.1.smithi027.stdout:8/953: read d5/db5/fd8 [140760,7935] 0 2024-04-02T04:43:12.126 INFO:tasks.workunit.client.1.smithi027.stdout:3/962: dwrite d1/de/df/d13/d73/db1/fcf [0,4194304] 0 2024-04-02T04:43:12.129 INFO:tasks.workunit.client.1.smithi027.stdout:3/963: write d1/fab [4401390,93115] 0 2024-04-02T04:43:12.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.126+0000 7f41867fc700 1 -- 172.21.15.16:0/52599282 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184299 (secure 0 0 0) 0x7f418c01a950 con 0x7f4190100a60 2024-04-02T04:43:12.130 INFO:tasks.workunit.client.1.smithi027.stdout:8/954: mknod d5/d8/db/df/c140 0 2024-04-02T04:43:12.130 INFO:tasks.workunit.client.1.smithi027.stdout:3/964: fdatasync d1/de/df/f5d 0 2024-04-02T04:43:12.142 INFO:tasks.workunit.client.1.smithi027.stdout:3/965: getdents d1/de/df/da5 0 2024-04-02T04:43:12.143 INFO:tasks.workunit.client.1.smithi027.stdout:3/966: readlink d1/d1a/l62 0 2024-04-02T04:43:12.146 INFO:tasks.workunit.client.1.smithi027.stdout:3/967: mknod d1/de/d55/c13e 0 2024-04-02T04:43:12.150 INFO:tasks.workunit.client.1.smithi027.stdout:8/955: dwrite d5/d8/db/dd/d5f/faf [0,4194304] 0 2024-04-02T04:43:12.153 INFO:tasks.workunit.client.1.smithi027.stdout:8/956: creat d5/d8/db/dd/d11e/f141 x:0 0 0 2024-04-02T04:43:12.155 INFO:tasks.workunit.client.1.smithi027.stdout:8/957: mkdir d5/d8/db/df/d142 0 2024-04-02T04:43:12.157 INFO:tasks.workunit.client.1.smithi027.stdout:3/968: write d1/de/df/f17 [3694636,38668] 0 2024-04-02T04:43:12.159 INFO:tasks.workunit.client.1.smithi027.stdout:8/958: mkdir d5/d8/d15/d71/d96/d143 0 2024-04-02T04:43:12.165 INFO:tasks.workunit.client.1.smithi027.stdout:3/969: dread d1/de/fa7 [0,4194304] 0 2024-04-02T04:43:12.169 INFO:tasks.workunit.client.1.smithi027.stdout:3/970: truncate d1/de/df/f24 904519 0 2024-04-02T04:43:12.174 INFO:tasks.workunit.client.1.smithi027.stdout:8/959: write d5/d8/db/d68/ffa [497861,78272] 0 2024-04-02T04:43:12.175 INFO:tasks.workunit.client.1.smithi027.stdout:3/971: symlink d1/de/d55/d63/da4/de3/d139/l13f 0 2024-04-02T04:43:12.176 INFO:tasks.workunit.client.1.smithi027.stdout:8/960: fsync d5/d8/db/dd/d5f/db2/fe5 0 2024-04-02T04:43:12.180 INFO:tasks.workunit.client.1.smithi027.stdout:8/961: creat d5/d8/db/dd/d4a/dc9/f144 x:0 0 0 2024-04-02T04:43:12.182 INFO:tasks.workunit.client.1.smithi027.stdout:8/962: fdatasync d5/d8/d15/d71/db0/d125/f128 0 2024-04-02T04:43:12.187 INFO:tasks.workunit.client.1.smithi027.stdout:8/963: symlink d5/d8/d5c/de6/l145 0 2024-04-02T04:43:12.193 INFO:tasks.workunit.client.1.smithi027.stdout:3/972: dread d1/de/df/d9e/fbe [0,4194304] 0 2024-04-02T04:43:12.195 INFO:tasks.workunit.client.1.smithi027.stdout:8/964: getdents d5/d8/d15/d71/db0/d125 0 2024-04-02T04:43:12.195 INFO:tasks.workunit.client.1.smithi027.stdout:3/973: write d1/de/df/d13/d73/d77/d102/ff5 [1536942,54701] 0 2024-04-02T04:43:12.202 INFO:tasks.workunit.client.1.smithi027.stdout:3/974: dwrite d1/de/df/d13/d73/db1/f116 [0,4194304] 0 2024-04-02T04:43:12.203 INFO:tasks.workunit.client.1.smithi027.stdout:3/975: readlink d1/de/df/d9e/lc7 0 2024-04-02T04:43:12.239 INFO:tasks.workunit.client.1.smithi027.stdout:3/976: getdents d1/de/d55/d63/da4 0 2024-04-02T04:43:12.240 INFO:tasks.workunit.client.1.smithi027.stdout:3/977: readlink d1/de/d55/d63/db9/lc3 0 2024-04-02T04:43:12.243 INFO:tasks.workunit.client.1.smithi027.stdout:3/978: symlink d1/de/d55/de7/l140 0 2024-04-02T04:43:12.245 INFO:tasks.workunit.client.1.smithi027.stdout:3/979: read d1/de/df/d13/f35 [953283,103296] 0 2024-04-02T04:43:12.255 INFO:tasks.workunit.client.1.smithi027.stdout:3/980: getdents d1/de/df/d61 0 2024-04-02T04:43:12.259 INFO:tasks.workunit.client.1.smithi027.stdout:3/981: write d1/de/df/d13/d73/d77/d102/fd1 [793609,26084] 0 2024-04-02T04:43:12.266 INFO:tasks.workunit.client.1.smithi027.stdout:3/982: dwrite d1/de/df/d9e/dc6/de4/ffe [0,4194304] 0 2024-04-02T04:43:12.266 INFO:tasks.workunit.client.1.smithi027.stdout:3/983: fdatasync d1/de/f8a 0 2024-04-02T04:43:12.276 INFO:tasks.workunit.client.1.smithi027.stdout:3/984: fsync d1/de/df/fc2 0 2024-04-02T04:43:12.277 INFO:tasks.workunit.client.1.smithi027.stdout:3/985: readlink d1/de/df/d61/l41 0 2024-04-02T04:43:12.279 INFO:tasks.workunit.client.1.smithi027.stdout:3/986: rmdir d1/d1a 39 2024-04-02T04:43:12.288 INFO:tasks.workunit.client.1.smithi027.stdout:3/987: dwrite d1/de/df/d9e/dc6/de4/ffe [0,4194304] 0 2024-04-02T04:43:12.305 INFO:tasks.workunit.client.1.smithi027.stdout:4/881: sync 2024-04-02T04:43:12.311 INFO:tasks.workunit.client.1.smithi027.stdout:3/988: dread d1/de/df/d13/d73/d77/d102/fd1 [0,4194304] 0 2024-04-02T04:43:12.314 INFO:tasks.workunit.client.1.smithi027.stdout:3/989: symlink d1/de/df/d13/l141 0 2024-04-02T04:43:12.314 INFO:tasks.workunit.client.1.smithi027.stdout:4/882: write f14 [981348,67548] 0 2024-04-02T04:43:12.315 INFO:tasks.workunit.client.1.smithi027.stdout:3/990: stat d1/de/df/d9e/da0/fb5 0 2024-04-02T04:43:12.318 INFO:tasks.workunit.client.1.smithi027.stdout:3/991: mkdir d1/de/d55/de7/d142 0 2024-04-02T04:43:12.319 INFO:tasks.workunit.client.1.smithi027.stdout:3/992: chown d1/de/df/d9e/da0/cc1 956781 1 2024-04-02T04:43:12.324 INFO:tasks.workunit.client.1.smithi027.stdout:3/993: unlink d1/de/d55/l88 0 2024-04-02T04:43:12.326 INFO:tasks.workunit.client.1.smithi027.stdout:4/883: write d24/d38/d44/d41/f89 [796706,87789] 0 2024-04-02T04:43:12.328 INFO:tasks.workunit.client.1.smithi027.stdout:4/884: write d24/f2c [1160738,70200] 0 2024-04-02T04:43:12.328 INFO:tasks.workunit.client.1.smithi027.stdout:3/994: mknod d1/de/df/d13/d73/de2/d125/c143 0 2024-04-02T04:43:12.331 INFO:tasks.workunit.client.1.smithi027.stdout:4/885: rmdir d24/d38/d44/d60 39 2024-04-02T04:43:12.333 INFO:tasks.workunit.client.1.smithi027.stdout:3/995: mknod d1/de/df/dec/c144 0 2024-04-02T04:43:12.335 INFO:tasks.workunit.client.1.smithi027.stdout:4/886: truncate d24/d38/d44/d41/d49/f83 3076346 0 2024-04-02T04:43:12.335 INFO:tasks.workunit.client.1.smithi027.stdout:4/887: chown d24/d48/f4e 547395 1 2024-04-02T04:43:12.336 INFO:tasks.workunit.client.1.smithi027.stdout:3/996: mknod d1/de/d55/de7/c145 0 2024-04-02T04:43:12.341 INFO:tasks.workunit.client.1.smithi027.stdout:4/888: mkdir d24/d38/d44/d41/d101/d119 0 2024-04-02T04:43:12.343 INFO:tasks.workunit.client.1.smithi027.stdout:3/997: mknod d1/de/df/d9e/d131/c146 0 2024-04-02T04:43:12.344 INFO:tasks.workunit.client.1.smithi027.stdout:4/889: symlink d24/d38/d44/d82/da9/l11a 0 2024-04-02T04:43:12.345 INFO:tasks.workunit.client.1.smithi027.stdout:3/998: fdatasync d1/de/f71 0 2024-04-02T04:43:12.347 INFO:tasks.workunit.client.1.smithi027.stdout:4/890: creat d24/d38/d44/d82/f11b x:0 0 0 2024-04-02T04:43:12.353 INFO:tasks.workunit.client.1.smithi027.stdout:3/999: dwrite d1/de/df/f67 [0,4194304] 0 2024-04-02T04:43:12.354 INFO:tasks.workunit.client.1.smithi027.stdout:4/891: rmdir d24/d38/d47/dc0/ddd 0 2024-04-02T04:43:12.358 INFO:tasks.workunit.client.1.smithi027.stdout:4/892: mknod d24/d38/d44/d60/c11c 0 2024-04-02T04:43:12.368 INFO:tasks.workunit.client.1.smithi027.stdout:4/893: link d24/d38/d44/d60/fa2 d24/d2d/d100/f11d 0 2024-04-02T04:43:12.369 INFO:tasks.workunit.client.1.smithi027.stdout:4/894: chown d24/d38/d47/fa3 0 1 2024-04-02T04:43:12.370 INFO:tasks.workunit.client.1.smithi027.stdout:8/965: sync 2024-04-02T04:43:12.370 INFO:tasks.workunit.client.1.smithi027.stdout:6/879: sync 2024-04-02T04:43:12.371 INFO:tasks.workunit.client.1.smithi027.stdout:4/895: truncate f2 119890 0 2024-04-02T04:43:12.373 INFO:tasks.workunit.client.1.smithi027.stdout:4/896: chown d24/d38/d44/d60/c11c 508865146 1 2024-04-02T04:43:12.373 INFO:tasks.workunit.client.1.smithi027.stdout:4/897: readlink le 0 2024-04-02T04:43:12.377 INFO:tasks.workunit.client.1.smithi027.stdout:6/880: creat d13/d1b/d24/d25/d56/dc8/d101/f123 x:0 0 0 2024-04-02T04:43:12.380 INFO:tasks.workunit.client.1.smithi027.stdout:8/966: creat d5/d8/d15/d71/d96/d12e/f146 x:0 0 0 2024-04-02T04:43:12.385 INFO:tasks.workunit.client.1.smithi027.stdout:4/898: dread d24/d48/db0/fc3 [0,4194304] 0 2024-04-02T04:43:12.387 INFO:tasks.workunit.client.1.smithi027.stdout:8/967: unlink d5/d8/d15/d71/db0/l134 0 2024-04-02T04:43:12.387 INFO:tasks.workunit.client.1.smithi027.stdout:8/968: stat d5/d8/d15/d71/db0 0 2024-04-02T04:43:12.388 INFO:tasks.workunit.client.1.smithi027.stdout:4/899: mkdir d24/d48/db0/dea/d11e 0 2024-04-02T04:43:12.391 INFO:tasks.workunit.client.1.smithi027.stdout:8/969: unlink d5/d8/db/dd/d5f/d27/d31/d5b/ca9 0 2024-04-02T04:43:12.393 INFO:tasks.workunit.client.1.smithi027.stdout:8/970: fdatasync d5/d8/db/dd/d5f/d27/d31/d5b/d6c/f7b 0 2024-04-02T04:43:12.393 INFO:tasks.workunit.client.1.smithi027.stdout:4/900: mkdir d24/d38/d47/d62/d11f 0 2024-04-02T04:43:12.397 INFO:tasks.workunit.client.1.smithi027.stdout:4/901: mkdir d24/d38/d120 0 2024-04-02T04:43:12.398 INFO:tasks.workunit.client.1.smithi027.stdout:8/971: symlink d5/d8/db/dd/d5f/db2/dc6/l147 0 2024-04-02T04:43:12.409 INFO:tasks.workunit.client.1.smithi027.stdout:8/972: getdents d5/d8/db/dd/d4a/dc9 0 2024-04-02T04:43:12.417 INFO:tasks.workunit.client.1.smithi027.stdout:8/973: dwrite d5/d8/db/dd/fda [0,4194304] 0 2024-04-02T04:43:12.426 INFO:tasks.workunit.client.1.smithi027.stdout:8/974: dwrite d5/d8/db/d68/f11a [0,4194304] 0 2024-04-02T04:43:12.438 INFO:tasks.workunit.client.1.smithi027.stdout:8/975: unlink d5/d8/db/dd/d5f/d27/d31/d5b/lf7 0 2024-04-02T04:43:12.439 INFO:tasks.workunit.client.1.smithi027.stdout:6/881: dwrite d13/f16 [0,4194304] 0 2024-04-02T04:43:12.443 INFO:tasks.workunit.client.1.smithi027.stdout:8/976: truncate d5/d8/db/dd/d5f/db2/dc6/d106/f10f 225996 0 2024-04-02T04:43:12.445 INFO:tasks.workunit.client.1.smithi027.stdout:8/977: readlink d5/d8/d5c/lb6 0 2024-04-02T04:43:12.445 INFO:tasks.workunit.client.1.smithi027.stdout:6/882: mknod d13/d1b/d24/d28/d46/dcc/da7/c124 0 2024-04-02T04:43:12.448 INFO:tasks.workunit.client.1.smithi027.stdout:8/978: write d5/d8/db/df/f22 [4002544,120406] 0 2024-04-02T04:43:12.457 INFO:tasks.workunit.client.1.smithi027.stdout:6/883: creat d13/d1b/d48/d75/d9b/de2/d114/f125 x:0 0 0 2024-04-02T04:43:12.461 INFO:tasks.workunit.client.1.smithi027.stdout:6/884: creat d13/d1b/d24/d25/d56/dc8/dc9/f126 x:0 0 0 2024-04-02T04:43:12.461 INFO:tasks.workunit.client.1.smithi027.stdout:4/902: write d24/d38/d44/d82/f90 [787496,101088] 0 2024-04-02T04:43:12.465 INFO:tasks.workunit.client.1.smithi027.stdout:6/885: mkdir d13/d1b/d48/d9c/df7/d96/d127 0 2024-04-02T04:43:12.468 INFO:tasks.workunit.client.1.smithi027.stdout:4/903: symlink d24/d38/d44/d82/d97/d9b/d104/l121 0 2024-04-02T04:43:12.471 INFO:tasks.workunit.client.1.smithi027.stdout:8/979: getdents d5/db5 0 2024-04-02T04:43:12.484 INFO:tasks.workunit.client.1.smithi027.stdout:4/904: creat d24/d38/d44/d41/f122 x:0 0 0 2024-04-02T04:43:12.490 INFO:tasks.workunit.client.1.smithi027.stdout:6/886: dwrite d13/d1b/d24/d25/d56/dc8/dc9/fee [0,4194304] 0 2024-04-02T04:43:12.493 INFO:tasks.workunit.client.1.smithi027.stdout:8/980: dread d5/d8/db/dd/d5f/d27/f50 [0,4194304] 0 2024-04-02T04:43:12.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.491+0000 7f4197314700 1 -- 172.21.15.16:0/52599282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f419004ea80 con 0x7f4190100a60 2024-04-02T04:43:12.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.492+0000 7f41867fc700 1 -- 172.21.15.16:0/52599282 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+735 (secure 0 0 0) 0x7f418c048070 con 0x7f4190100a60 2024-04-02T04:43:12.495 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:43:12.495 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": { 2024-04-02T04:43:12.495 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 2 2024-04-02T04:43:12.495 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:43:12.495 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": { 2024-04-02T04:43:12.495 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 1 2024-04-02T04:43:12.495 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:43:12.495 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": { 2024-04-02T04:43:12.495 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 6 2024-04-02T04:43:12.495 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:43:12.496 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-04-02T04:43:12.496 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 4 2024-04-02T04:43:12.496 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:43:12.496 INFO:teuthology.orchestra.run.smithi016.stdout: "overall": { 2024-04-02T04:43:12.496 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 12, 2024-04-02T04:43:12.496 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 1 2024-04-02T04:43:12.496 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-02T04:43:12.496 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:43:12.496 INFO:tasks.workunit.client.1.smithi027.stdout:6/887: rename d13/d1b/d48/d75/d9b/de2/d114/f11c to d13/d1b/d48/d9c/df7/f128 0 2024-04-02T04:43:12.497 INFO:tasks.workunit.client.1.smithi027.stdout:4/905: truncate d24/f3f 2994640 0 2024-04-02T04:43:12.499 INFO:tasks.workunit.client.1.smithi027.stdout:8/981: fdatasync d5/d8/db/dd/d5f/d91/d9f/fb4 0 2024-04-02T04:43:12.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.495+0000 7f4197314700 1 -- 172.21.15.16:0/52599282 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f417c038d00 msgr2=0x7f417c03b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:12.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.495+0000 7f4197314700 1 --2- 172.21.15.16:0/52599282 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f417c038d00 0x7f417c03b1c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4180005090 tx=0x7f4180005960 comp rx=0 tx=0).stop 2024-04-02T04:43:12.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.495+0000 7f4197314700 1 -- 172.21.15.16:0/52599282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4190100a60 msgr2=0x7f4190074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:12.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.495+0000 7f4197314700 1 --2- 172.21.15.16:0/52599282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4190100a60 0x7f4190074b00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f418c00d900 tx=0x7f418c00dc10 comp rx=0 tx=0).stop 2024-04-02T04:43:12.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.495+0000 7f4197314700 1 -- 172.21.15.16:0/52599282 shutdown_connections 2024-04-02T04:43:12.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.495+0000 7f4197314700 1 --2- 172.21.15.16:0/52599282 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f417c038d00 0x7f417c03b1c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:12.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.496+0000 7f4197314700 1 --2- 172.21.15.16:0/52599282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4190100a60 0x7f4190074b00 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:12.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.496+0000 7f4197314700 1 --2- 172.21.15.16:0/52599282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4190101c60 0x7f41900731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:12.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.496+0000 7f4197314700 1 -- 172.21.15.16:0/52599282 >> 172.21.15.16:0/52599282 conn(0x7f41900fc020 msgr2=0x7f4190105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:12.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.496+0000 7f4197314700 1 -- 172.21.15.16:0/52599282 shutdown_connections 2024-04-02T04:43:12.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.496+0000 7f4197314700 1 -- 172.21.15.16:0/52599282 wait complete. 2024-04-02T04:43:12.501 INFO:tasks.workunit.client.1.smithi027.stdout:6/888: creat d13/d1b/d24/d25/d56/d6d/daa/f129 x:0 0 0 2024-04-02T04:43:12.501 INFO:tasks.workunit.client.1.smithi027.stdout:8/982: write d5/d8/db/dd/d5f/db2/fe5 [4682623,113908] 0 2024-04-02T04:43:12.502 INFO:tasks.workunit.client.1.smithi027.stdout:4/906: truncate f13 4480582 0 2024-04-02T04:43:12.502 INFO:tasks.workunit.client.1.smithi027.stdout:6/889: stat d13/d1b/d48/d58/dd3 0 2024-04-02T04:43:12.510 INFO:tasks.workunit.client.1.smithi027.stdout:6/890: fdatasync d13/f1a 0 2024-04-02T04:43:12.511 INFO:tasks.workunit.client.1.smithi027.stdout:8/983: dwrite d5/d8/db/dd/d5f/db2/dc6/d106/f10f [0,4194304] 0 2024-04-02T04:43:12.516 INFO:tasks.workunit.client.1.smithi027.stdout:6/891: unlink d13/d1b/f105 0 2024-04-02T04:43:12.518 INFO:tasks.workunit.client.1.smithi027.stdout:8/984: chown d5/d8/db/dd/d5f/d27/ce2 77893786 1 2024-04-02T04:43:12.524 INFO:tasks.workunit.client.1.smithi027.stdout:8/985: rmdir d5/d8/db 39 2024-04-02T04:43:12.527 INFO:tasks.workunit.client.1.smithi027.stdout:4/907: dwrite f1d [0,4194304] 0 2024-04-02T04:43:12.534 INFO:tasks.workunit.client.1.smithi027.stdout:6/892: creat d13/d1b/d24/d25/f12a x:0 0 0 2024-04-02T04:43:12.537 INFO:tasks.workunit.client.1.smithi027.stdout:8/986: creat d5/d8/db/d57/f148 x:0 0 0 2024-04-02T04:43:12.542 INFO:tasks.workunit.client.1.smithi027.stdout:6/893: readlink d13/d1b/d48/d9c/df7/lb6 0 2024-04-02T04:43:12.545 INFO:tasks.workunit.client.1.smithi027.stdout:6/894: fsync d13/d1b/d48/d75/d9b/de2/d114/f3f 0 2024-04-02T04:43:12.545 INFO:tasks.workunit.client.1.smithi027.stdout:8/987: unlink d5/d8/db/d68/lb7 0 2024-04-02T04:43:12.546 INFO:tasks.workunit.client.1.smithi027.stdout:4/908: dwrite f21 [4194304,4194304] 0 2024-04-02T04:43:12.550 INFO:tasks.workunit.client.1.smithi027.stdout:4/909: rmdir d24/d2d 39 2024-04-02T04:43:12.556 INFO:tasks.workunit.client.1.smithi027.stdout:4/910: symlink d24/d38/d44/d41/d101/l123 0 2024-04-02T04:43:12.562 INFO:tasks.workunit.client.1.smithi027.stdout:6/895: write d13/f93 [360208,102941] 0 2024-04-02T04:43:12.567 INFO:tasks.workunit.client.1.smithi027.stdout:4/911: write f2 [127200,7298] 0 2024-04-02T04:43:12.568 INFO:tasks.workunit.client.1.smithi027.stdout:4/912: chown d24/d38/d47/dc0/dca 23 1 2024-04-02T04:43:12.570 INFO:tasks.workunit.client.1.smithi027.stdout:6/896: write d13/d1b/d48/d9c/df7/d96/fac [4699677,111180] 0 2024-04-02T04:43:12.580 INFO:tasks.workunit.client.1.smithi027.stdout:4/913: dwrite d24/d48/f5d [0,4194304] 0 2024-04-02T04:43:12.585 INFO:tasks.workunit.client.1.smithi027.stdout:6/897: dwrite d13/d1b/d24/d25/d56/fc0 [0,4194304] 0 2024-04-02T04:43:12.586 INFO:tasks.workunit.client.1.smithi027.stdout:6/898: readlink d13/d1b/d24/d28/d46/lc6 0 2024-04-02T04:43:12.588 INFO:tasks.workunit.client.1.smithi027.stdout:4/914: rename d24/d38/d44/d82/d95/ca4 to d24/d48/db0/c124 0 2024-04-02T04:43:12.589 INFO:tasks.workunit.client.1.smithi027.stdout:4/915: stat d24/d38/d44/d41/d101/d103 0 2024-04-02T04:43:12.594 INFO:tasks.workunit.client.1.smithi027.stdout:4/916: getdents d24/d38/d44/d41/d101/d119 0 2024-04-02T04:43:12.597 INFO:tasks.workunit.client.1.smithi027.stdout:4/917: mkdir d24/d38/d44/d41/d101/d118/d125 0 2024-04-02T04:43:12.601 INFO:tasks.workunit.client.1.smithi027.stdout:4/918: mknod d24/d38/d44/d41/c126 0 2024-04-02T04:43:12.613 INFO:tasks.workunit.client.1.smithi027.stdout:6/899: dread d13/d1b/d24/fd1 [0,4194304] 0 2024-04-02T04:43:12.614 INFO:tasks.workunit.client.1.smithi027.stdout:4/919: getdents d24/d38/d47/d62/db4 0 2024-04-02T04:43:12.616 INFO:tasks.workunit.client.1.smithi027.stdout:4/920: fdatasync d24/fe8 0 2024-04-02T04:43:12.619 INFO:tasks.workunit.client.1.smithi027.stdout:4/921: dread f2 [0,4194304] 0 2024-04-02T04:43:12.627 INFO:tasks.workunit.client.1.smithi027.stdout:4/922: mkdir d24/d48/db0/dea/d11e/d127 0 2024-04-02T04:43:12.629 INFO:tasks.workunit.client.1.smithi027.stdout:6/900: getdents d13/d1b/d48/d75/d9b 0 2024-04-02T04:43:12.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.629+0000 7f5a70dfd700 1 -- 172.21.15.16:0/2919604286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6c101d10 msgr2=0x7f5a6c102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:12.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.629+0000 7f5a70dfd700 1 --2- 172.21.15.16:0/2919604286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6c101d10 0x7f5a6c102170 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5a60009910 tx=0x7f5a60009c20 comp rx=0 tx=0).stop 2024-04-02T04:43:12.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.629+0000 7f5a70dfd700 1 -- 172.21.15.16:0/2919604286 shutdown_connections 2024-04-02T04:43:12.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.629+0000 7f5a70dfd700 1 --2- 172.21.15.16:0/2919604286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6c101d10 0x7f5a6c102170 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:12.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.629+0000 7f5a70dfd700 1 --2- 172.21.15.16:0/2919604286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a6c100b10 0x7f5a6c100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:12.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.629+0000 7f5a70dfd700 1 -- 172.21.15.16:0/2919604286 >> 172.21.15.16:0/2919604286 conn(0x7f5a6c0fc0d0 msgr2=0x7f5a6c0fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:12.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.629+0000 7f5a70dfd700 1 -- 172.21.15.16:0/2919604286 shutdown_connections 2024-04-02T04:43:12.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.629+0000 7f5a70dfd700 1 -- 172.21.15.16:0/2919604286 wait complete. 2024-04-02T04:43:12.634 INFO:tasks.workunit.client.1.smithi027.stdout:6/901: mkdir d13/d1b/d48/d75/d12b 0 2024-04-02T04:43:12.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.630+0000 7f5a70dfd700 1 Processor -- start 2024-04-02T04:43:12.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.630+0000 7f5a70dfd700 1 -- start start 2024-04-02T04:43:12.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.630+0000 7f5a70dfd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a6c100b10 0x7f5a6c1964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:12.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.630+0000 7f5a70dfd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6c101d10 0x7f5a6c196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:12.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.630+0000 7f5a70dfd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6c197030 con 0x7f5a6c101d10 2024-04-02T04:43:12.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.630+0000 7f5a70dfd700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6c197170 con 0x7f5a6c100b10 2024-04-02T04:43:12.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.630+0000 7f5a69d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6c101d10 0x7f5a6c196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:12.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.630+0000 7f5a69d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6c101d10 0x7f5a6c196a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34540/0 (socket says 172.21.15.16:34540) 2024-04-02T04:43:12.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.630+0000 7f5a69d9b700 1 -- 172.21.15.16:0/2014799175 learned_addr learned my addr 172.21.15.16:0/2014799175 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:43:12.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.630+0000 7f5a6a59c700 1 --2- 172.21.15.16:0/2014799175 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a6c100b10 0x7f5a6c1964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:12.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.631+0000 7f5a69d9b700 1 -- 172.21.15.16:0/2014799175 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a6c100b10 msgr2=0x7f5a6c1964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:12.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.631+0000 7f5a69d9b700 1 --2- 172.21.15.16:0/2014799175 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a6c100b10 0x7f5a6c1964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:12.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.631+0000 7f5a69d9b700 1 -- 172.21.15.16:0/2014799175 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a600095f0 con 0x7f5a6c101d10 2024-04-02T04:43:12.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.631+0000 7f5a69d9b700 1 --2- 172.21.15.16:0/2014799175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6c101d10 0x7f5a6c196a10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000b800 tx=0x7f5a60005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:12.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.631+0000 7f5a5b7fe700 1 -- 172.21.15.16:0/2014799175 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a6001d070 con 0x7f5a6c101d10 2024-04-02T04:43:12.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.631+0000 7f5a70dfd700 1 -- 172.21.15.16:0/2014799175 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a6c19bb20 con 0x7f5a6c101d10 2024-04-02T04:43:12.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.631+0000 7f5a5b7fe700 1 -- 172.21.15.16:0/2014799175 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a600052a0 con 0x7f5a6c101d10 2024-04-02T04:43:12.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.631+0000 7f5a5b7fe700 1 -- 172.21.15.16:0/2014799175 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a60017400 con 0x7f5a6c101d10 2024-04-02T04:43:12.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.631+0000 7f5a70dfd700 1 -- 172.21.15.16:0/2014799175 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a6c19c090 con 0x7f5a6c101d10 2024-04-02T04:43:12.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.632+0000 7f5a5b7fe700 1 -- 172.21.15.16:0/2014799175 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 29) v1 ==== 46199+0+0 (secure 0 0 0) 0x7f5a600176c0 con 0x7f5a6c101d10 2024-04-02T04:43:12.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.632+0000 7f5a5b7fe700 1 --2- 172.21.15.16:0/2014799175 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f5a54039090 0x7f5a5403b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:12.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.632+0000 7f5a5b7fe700 1 -- 172.21.15.16:0/2014799175 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f5a6004f670 con 0x7f5a6c101d10 2024-04-02T04:43:12.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.632+0000 7f5a70dfd700 1 -- 172.21.15.16:0/2014799175 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a6c190690 con 0x7f5a6c101d10 2024-04-02T04:43:12.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.633+0000 7f5a6a59c700 1 --2- 172.21.15.16:0/2014799175 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f5a54039090 0x7f5a5403b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:12.638 INFO:tasks.workunit.client.1.smithi027.stdout:6/902: stat d13/d1b/d48/d75/d9b/de2/fef 0 2024-04-02T04:43:12.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.633+0000 7f5a6a59c700 1 --2- 172.21.15.16:0/2014799175 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f5a54039090 0x7f5a5403b550 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c101b70 tx=0x7f5a5c009260 comp rx=0 tx=0).ready entity=mgr.14744 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:12.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.639+0000 7f5a5b7fe700 1 -- 172.21.15.16:0/2014799175 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184299 (secure 0 0 0) 0x7f5a60020370 con 0x7f5a6c101d10 2024-04-02T04:43:12.642 INFO:tasks.workunit.client.1.smithi027.stdout:6/903: dwrite d13/d1b/d48/d75/d9b/de2/d114/f3f [4194304,4194304] 0 2024-04-02T04:43:12.644 INFO:tasks.workunit.client.1.smithi027.stdout:4/923: dread d24/d38/f8d [0,4194304] 0 2024-04-02T04:43:12.660 INFO:tasks.workunit.client.1.smithi027.stdout:4/924: dread d24/d38/d44/d82/fb7 [0,4194304] 0 2024-04-02T04:43:12.666 INFO:tasks.workunit.client.1.smithi027.stdout:4/925: dread d24/d38/d44/d41/d49/f8b [0,4194304] 0 2024-04-02T04:43:12.670 INFO:tasks.workunit.client.1.smithi027.stdout:6/904: dwrite d13/d8d/fa3 [4194304,4194304] 0 2024-04-02T04:43:12.671 INFO:tasks.workunit.client.1.smithi027.stdout:4/926: mkdir d24/d38/d44/d41/d101/d118/d128 0 2024-04-02T04:43:12.676 INFO:tasks.workunit.client.1.smithi027.stdout:6/905: read d13/d1b/d48/d9c/ff3 [634116,9824] 0 2024-04-02T04:43:12.676 INFO:tasks.workunit.client.1.smithi027.stdout:4/927: rename d24/d38/d47/dc0/dca/ce7 to d24/d38/d44/d41/d101/c129 0 2024-04-02T04:43:12.678 INFO:tasks.workunit.client.1.smithi027.stdout:6/906: dread - d13/d1b/d24/d25/d56/dc8/dc9/ffe zero size 2024-04-02T04:43:12.681 INFO:tasks.workunit.client.1.smithi027.stdout:4/928: truncate d24/f76 1447653 0 2024-04-02T04:43:12.683 INFO:tasks.workunit.client.1.smithi027.stdout:6/907: symlink d13/d1b/d48/d9c/df7/d96/l12c 0 2024-04-02T04:43:12.684 INFO:tasks.workunit.client.1.smithi027.stdout:6/908: chown d13/d1b/d24/d28/d3c/f7f 0 1 2024-04-02T04:43:12.705 INFO:tasks.workunit.client.1.smithi027.stdout:6/909: write d13/d1b/d24/f4a [1220610,6133] 0 2024-04-02T04:43:12.706 INFO:tasks.workunit.client.1.smithi027.stdout:4/929: dread d24/f76 [0,4194304] 0 2024-04-02T04:43:12.707 INFO:tasks.workunit.client.1.smithi027.stdout:4/930: truncate f14 4878360 0 2024-04-02T04:43:12.775 INFO:tasks.workunit.client.1.smithi027.stdout:4/931: unlink d24/d38/d44/d82/d97/cae 0 2024-04-02T04:43:12.777 INFO:tasks.workunit.client.1.smithi027.stdout:4/932: dread d24/f76 [0,4194304] 0 2024-04-02T04:43:12.777 INFO:tasks.workunit.client.1.smithi027.stdout:6/910: link d13/d1b/d24/d25/d56/ff0 d13/d8d/f12d 0 2024-04-02T04:43:12.779 INFO:tasks.workunit.client.1.smithi027.stdout:4/933: chown d24/d2d/d100 1 1 2024-04-02T04:43:12.780 INFO:tasks.workunit.client.1.smithi027.stdout:6/911: read d13/d1b/d24/d25/d56/fa0 [1012704,43829] 0 2024-04-02T04:43:12.780 INFO:tasks.workunit.client.1.smithi027.stdout:6/912: write f12 [4552823,81259] 0 2024-04-02T04:43:12.782 INFO:tasks.workunit.client.1.smithi027.stdout:4/934: mknod d24/d48/db0/dea/d11e/c12a 0 2024-04-02T04:43:12.785 INFO:tasks.workunit.client.1.smithi027.stdout:4/935: symlink d24/d38/d44/d41/d49/l12b 0 2024-04-02T04:43:12.787 INFO:tasks.workunit.client.1.smithi027.stdout:4/936: symlink d24/d48/de4/l12c 0 2024-04-02T04:43:12.790 INFO:tasks.workunit.client.1.smithi027.stdout:4/937: symlink d24/d38/d47/d62/db4/l12d 0 2024-04-02T04:43:12.790 INFO:tasks.workunit.client.1.smithi027.stdout:4/938: chown d24/d38/d47/d5c/cd8 30165041 1 2024-04-02T04:43:12.795 INFO:tasks.workunit.client.1.smithi027.stdout:4/939: mkdir d24/d38/d44/d82/d97/d9b/d12e 0 2024-04-02T04:43:12.796 INFO:tasks.workunit.client.1.smithi027.stdout:4/940: chown d24/c115 3460872 1 2024-04-02T04:43:12.799 INFO:tasks.workunit.client.1.smithi027.stdout:4/941: creat d24/d38/d44/d82/d95/f12f x:0 0 0 2024-04-02T04:43:12.809 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:12 smithi016 ceph-mon[28969]: [02/Apr/2024:04:43:11] ENGINE Serving on https://172.21.15.16:7150 2024-04-02T04:43:12.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:12 smithi016 ceph-mon[28969]: [02/Apr/2024:04:43:11] ENGINE Bus STARTED 2024-04-02T04:43:12.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:12 smithi016 ceph-mon[28969]: mgrmap e28: smithi016.eqnebx(active, since 1.0089s) 2024-04-02T04:43:12.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:12 smithi016 ceph-mon[28969]: from='client.14752 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:43:12.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:12 smithi016 ceph-mon[28969]: pgmap v3: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail 2024-04-02T04:43:12.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:12 smithi016 ceph-mon[28969]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:43:12.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:12 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/52599282' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:43:12.810 INFO:tasks.workunit.client.1.smithi027.stdout:4/942: getdents d24/d38/d44/d82/d97/d9b/d104 0 2024-04-02T04:43:12.816 INFO:tasks.workunit.client.1.smithi027.stdout:6/913: write d13/d1b/d24/d25/d56/fa0 [1419191,83175] 0 2024-04-02T04:43:12.819 INFO:tasks.workunit.client.1.smithi027.stdout:4/943: rmdir d24/d38/d44/d41/d101/d119 0 2024-04-02T04:43:12.820 INFO:tasks.workunit.client.1.smithi027.stdout:6/914: creat d13/d1b/d24/d25/d56/dc8/f12e x:0 0 0 2024-04-02T04:43:12.821 INFO:tasks.workunit.client.1.smithi027.stdout:4/944: mknod d24/d38/d44/d41/d101/d118/c130 0 2024-04-02T04:43:12.825 INFO:tasks.workunit.client.1.smithi027.stdout:6/915: creat d13/d1b/d24/d28/f12f x:0 0 0 2024-04-02T04:43:12.826 INFO:tasks.workunit.client.1.smithi027.stdout:6/916: chown d13/d1b/d24/d28/d46 1035645 1 2024-04-02T04:43:12.829 INFO:tasks.workunit.client.1.smithi027.stdout:4/945: dread d24/d38/d44/d60/fb8 [0,4194304] 0 2024-04-02T04:43:12.830 INFO:tasks.workunit.client.1.smithi027.stdout:6/917: link d13/d1b/d24/l82 d13/d1b/d24/d25/d56/dc8/l130 0 2024-04-02T04:43:12.833 INFO:tasks.workunit.client.1.smithi027.stdout:6/918: read d13/f14 [661794,30469] 0 2024-04-02T04:43:12.836 INFO:tasks.workunit.client.1.smithi027.stdout:4/946: dwrite d24/d38/d47/ffb [0,4194304] 0 2024-04-02T04:43:12.838 INFO:tasks.workunit.client.1.smithi027.stdout:6/919: rename d13/d1b/d24/d28/d46/cf9 to d13/d1b/d48/d75/d9b/c131 0 2024-04-02T04:43:12.843 INFO:tasks.workunit.client.1.smithi027.stdout:4/947: mknod d24/d2d/c131 0 2024-04-02T04:43:12.844 INFO:tasks.workunit.client.1.smithi027.stdout:6/920: mkdir d13/d1b/d24/d28/d46/d132 0 2024-04-02T04:43:12.852 INFO:tasks.workunit.client.1.smithi027.stdout:6/921: creat d13/d1b/d48/d58/dd3/df8/f133 x:0 0 0 2024-04-02T04:43:12.855 INFO:tasks.workunit.client.1.smithi027.stdout:4/948: creat d24/d38/d47/d62/f132 x:0 0 0 2024-04-02T04:43:12.873 INFO:tasks.workunit.client.1.smithi027.stdout:4/949: dread d24/d38/f5e [0,4194304] 0 2024-04-02T04:43:12.876 INFO:tasks.workunit.client.1.smithi027.stdout:4/950: rmdir d24/d48/db0 39 2024-04-02T04:43:12.885 INFO:tasks.workunit.client.1.smithi027.stdout:4/951: dwrite d24/d38/d47/dc0/dca/f114 [0,4194304] 0 2024-04-02T04:43:12.886 INFO:tasks.workunit.client.1.smithi027.stdout:6/922: dwrite d13/d1b/d48/d75/f8b [0,4194304] 0 2024-04-02T04:43:12.890 INFO:tasks.workunit.client.1.smithi027.stdout:6/923: mknod d13/d1b/d24/d28/d46/dcc/db7/c134 0 2024-04-02T04:43:12.896 INFO:tasks.workunit.client.1.smithi027.stdout:4/952: creat d24/d48/db0/dea/f133 x:0 0 0 2024-04-02T04:43:12.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:12 smithi027 ceph-mon[37550]: [02/Apr/2024:04:43:11] ENGINE Serving on https://172.21.15.16:7150 2024-04-02T04:43:12.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:12 smithi027 ceph-mon[37550]: [02/Apr/2024:04:43:11] ENGINE Bus STARTED 2024-04-02T04:43:12.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:12 smithi027 ceph-mon[37550]: mgrmap e28: smithi016.eqnebx(active, since 1.0089s) 2024-04-02T04:43:12.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:12 smithi027 ceph-mon[37550]: from='client.14752 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:43:12.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:12 smithi027 ceph-mon[37550]: pgmap v3: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail 2024-04-02T04:43:12.898 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:12 smithi027 ceph-mon[37550]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:43:12.898 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:12 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/52599282' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:43:12.898 INFO:tasks.workunit.client.1.smithi027.stdout:6/924: symlink d13/d1b/d48/d58/dd3/l135 0 2024-04-02T04:43:12.904 INFO:tasks.workunit.client.1.smithi027.stdout:4/953: dwrite d24/d38/d44/d82/da9/faa [0,4194304] 0 2024-04-02T04:43:12.913 INFO:tasks.workunit.client.1.smithi027.stdout:4/954: truncate d24/d48/f10c 994011 0 2024-04-02T04:43:12.915 INFO:tasks.workunit.client.1.smithi027.stdout:6/925: write d13/d1b/d24/d28/f65 [1633274,89454] 0 2024-04-02T04:43:12.916 INFO:tasks.workunit.client.1.smithi027.stdout:8/988: sync 2024-04-02T04:43:12.918 INFO:tasks.workunit.client.1.smithi027.stdout:8/989: dread - d5/d8/db/dd/d5f/d27/f105 zero size 2024-04-02T04:43:12.922 INFO:tasks.workunit.client.1.smithi027.stdout:8/990: unlink d5/d8/d5c/l64 0 2024-04-02T04:43:12.922 INFO:tasks.workunit.client.1.smithi027.stdout:4/955: dread f1e [0,4194304] 0 2024-04-02T04:43:12.924 INFO:tasks.workunit.client.1.smithi027.stdout:8/991: chown d5/d8/db/dd/d5f/d27/d31/d5b/lbd 1164 1 2024-04-02T04:43:12.925 INFO:tasks.workunit.client.1.smithi027.stdout:8/992: write d5/d8/d15/d71/f72 [1528563,43677] 0 2024-04-02T04:43:12.927 INFO:tasks.workunit.client.1.smithi027.stdout:4/956: symlink d24/d38/d120/l134 0 2024-04-02T04:43:12.928 INFO:tasks.workunit.client.1.smithi027.stdout:4/957: chown d24/d38/d120 1280 1 2024-04-02T04:43:12.930 INFO:tasks.workunit.client.1.smithi027.stdout:8/993: read d5/d8/db/dd/d5f/d27/d31/f32 [1429871,128422] 0 2024-04-02T04:43:12.932 INFO:tasks.workunit.client.1.smithi027.stdout:8/994: write d5/d8/db/dd/d5f/d27/d31/d5b/d6c/f7b [4016803,37593] 0 2024-04-02T04:43:12.932 INFO:tasks.workunit.client.1.smithi027.stdout:6/926: truncate d13/d1b/d24/f4a 2400617 0 2024-04-02T04:43:12.936 INFO:tasks.workunit.client.1.smithi027.stdout:4/958: rename d24/d38/d44/d41/d5a/fab to d24/d38/d120/f135 0 2024-04-02T04:43:12.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.937+0000 7f5a70dfd700 1 -- 172.21.15.16:0/2014799175 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5a6c109670 con 0x7f5a6c101d10 2024-04-02T04:43:12.941 INFO:tasks.workunit.client.1.smithi027.stdout:4/959: rename d24/fe8 to d24/d38/d44/d82/da9/f136 0 2024-04-02T04:43:12.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.938+0000 7f5a5b7fe700 1 -- 172.21.15.16:0/2014799175 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1843 (secure 0 0 0) 0x7f5a6000ee60 con 0x7f5a6c101d10 2024-04-02T04:43:12.941 INFO:teuthology.orchestra.run.smithi016.stdout:e15 2024-04-02T04:43:12.941 INFO:teuthology.orchestra.run.smithi016.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:43:12.941 INFO:teuthology.orchestra.run.smithi016.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:43:12.941 INFO:teuthology.orchestra.run.smithi016.stdout:legacy client fscid: 1 2024-04-02T04:43:12.941 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:43:12.941 INFO:teuthology.orchestra.run.smithi016.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:43:12.941 INFO:teuthology.orchestra.run.smithi016.stdout:fs_name cephfs 2024-04-02T04:43:12.942 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 15 2024-04-02T04:43:12.942 INFO:teuthology.orchestra.run.smithi016.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:43:12.942 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-04-02T04:39:30.740168+0000 2024-04-02T04:43:12.942 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-04-02T04:39:49.875319+0000 2024-04-02T04:43:12.942 INFO:teuthology.orchestra.run.smithi016.stdout:tableserver 0 2024-04-02T04:43:12.942 INFO:teuthology.orchestra.run.smithi016.stdout:root 0 2024-04-02T04:43:12.942 INFO:teuthology.orchestra.run.smithi016.stdout:session_timeout 60 2024-04-02T04:43:12.942 INFO:teuthology.orchestra.run.smithi016.stdout:session_autoclose 300 2024-04-02T04:43:12.942 INFO:teuthology.orchestra.run.smithi016.stdout:max_file_size 1099511627776 2024-04-02T04:43:12.942 INFO:teuthology.orchestra.run.smithi016.stdout:max_xattr_size 65536 2024-04-02T04:43:12.942 INFO:teuthology.orchestra.run.smithi016.stdout:required_client_features {} 2024-04-02T04:43:12.942 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure 0 2024-04-02T04:43:12.942 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure_osd_epoch 46 2024-04-02T04:43:12.942 INFO:teuthology.orchestra.run.smithi016.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:43:12.943 INFO:teuthology.orchestra.run.smithi016.stdout:max_mds 1 2024-04-02T04:43:12.943 INFO:teuthology.orchestra.run.smithi016.stdout:in 0 2024-04-02T04:43:12.943 INFO:teuthology.orchestra.run.smithi016.stdout:up {0=24285} 2024-04-02T04:43:12.943 INFO:teuthology.orchestra.run.smithi016.stdout:failed 2024-04-02T04:43:12.943 INFO:teuthology.orchestra.run.smithi016.stdout:damaged 2024-04-02T04:43:12.943 INFO:teuthology.orchestra.run.smithi016.stdout:stopped 2024-04-02T04:43:12.943 INFO:teuthology.orchestra.run.smithi016.stdout:data_pools [3] 2024-04-02T04:43:12.943 INFO:teuthology.orchestra.run.smithi016.stdout:metadata_pool 2 2024-04-02T04:43:12.943 INFO:teuthology.orchestra.run.smithi016.stdout:inline_data enabled 2024-04-02T04:43:12.943 INFO:teuthology.orchestra.run.smithi016.stdout:balancer 2024-04-02T04:43:12.943 INFO:teuthology.orchestra.run.smithi016.stdout:bal_rank_mask -1 2024-04-02T04:43:12.943 INFO:teuthology.orchestra.run.smithi016.stdout:standby_count_wanted 1 2024-04-02T04:43:12.943 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.zwrfoo{0:24285} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.27:6824/1461455976,v1:172.21.15.27:6825/1461455976] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:43:12.943 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:43:12.943 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:43:12.944 INFO:teuthology.orchestra.run.smithi016.stdout:Standby daemons: 2024-04-02T04:43:12.944 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:43:12.944 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.fyzrnm{-1:14488} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.16:6828/2578719859,v1:172.21.15.16:6829/2578719859] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:43:12.944 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.uhgkgg{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.16:6826/601263579,v1:172.21.15.16:6827/601263579] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:43:12.944 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.sdublg{-1:24295} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.27:6826/4177192740,v1:172.21.15.27:6827/4177192740] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:43:12.945 INFO:tasks.workunit.client.1.smithi027.stdout:8/995: mkdir d5/d8/db/d68/d137/d149 0 2024-04-02T04:43:12.945 INFO:tasks.workunit.client.1.smithi027.stdout:6/927: dread d13/d1b/d48/d75/fd5 [0,4194304] 0 2024-04-02T04:43:12.945 INFO:tasks.workunit.client.1.smithi027.stdout:4/960: write d24/d38/d44/d82/f11b [686128,105770] 0 2024-04-02T04:43:12.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.940+0000 7f5a70dfd700 1 -- 172.21.15.16:0/2014799175 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f5a54039090 msgr2=0x7f5a5403b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:12.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.940+0000 7f5a70dfd700 1 --2- 172.21.15.16:0/2014799175 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f5a54039090 0x7f5a5403b550 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c101b70 tx=0x7f5a5c009260 comp rx=0 tx=0).stop 2024-04-02T04:43:12.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.940+0000 7f5a70dfd700 1 -- 172.21.15.16:0/2014799175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6c101d10 msgr2=0x7f5a6c196a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:12.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.940+0000 7f5a70dfd700 1 --2- 172.21.15.16:0/2014799175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6c101d10 0x7f5a6c196a10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000b800 tx=0x7f5a60005f00 comp rx=0 tx=0).stop 2024-04-02T04:43:12.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.940+0000 7f5a70dfd700 1 -- 172.21.15.16:0/2014799175 shutdown_connections 2024-04-02T04:43:12.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.940+0000 7f5a70dfd700 1 --2- 172.21.15.16:0/2014799175 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f5a54039090 0x7f5a5403b550 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:12.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.940+0000 7f5a70dfd700 1 --2- 172.21.15.16:0/2014799175 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a6c100b10 0x7f5a6c1964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:12.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.940+0000 7f5a70dfd700 1 --2- 172.21.15.16:0/2014799175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6c101d10 0x7f5a6c196a10 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:12.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.940+0000 7f5a70dfd700 1 -- 172.21.15.16:0/2014799175 >> 172.21.15.16:0/2014799175 conn(0x7f5a6c0fc0d0 msgr2=0x7f5a6c105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:12.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.940+0000 7f5a70dfd700 1 -- 172.21.15.16:0/2014799175 shutdown_connections 2024-04-02T04:43:12.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:12.940+0000 7f5a70dfd700 1 -- 172.21.15.16:0/2014799175 wait complete. 2024-04-02T04:43:12.946 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 15 2024-04-02T04:43:12.947 INFO:tasks.workunit.client.1.smithi027.stdout:8/996: creat d5/d8/db/dd/d5f/d91/dc2/f14a x:0 0 0 2024-04-02T04:43:12.949 INFO:tasks.workunit.client.1.smithi027.stdout:6/928: creat d13/d1b/d48/d9c/df7/d96/d127/f136 x:0 0 0 2024-04-02T04:43:12.950 INFO:tasks.workunit.client.1.smithi027.stdout:4/961: dwrite f21 [0,4194304] 0 2024-04-02T04:43:12.957 INFO:tasks.workunit.client.1.smithi027.stdout:4/962: dread d24/d38/d44/d82/f11b [0,4194304] 0 2024-04-02T04:43:12.958 INFO:tasks.workunit.client.1.smithi027.stdout:6/929: mknod d13/d1b/d24/d25/d56/d6d/c137 0 2024-04-02T04:43:12.964 INFO:tasks.workunit.client.1.smithi027.stdout:8/997: rename d5/d8/db/dd/d5f/d91/d9f/l120 to d5/d8/db/dd/d5f/d27/l14b 0 2024-04-02T04:43:12.965 INFO:tasks.workunit.client.1.smithi027.stdout:6/930: creat d13/d1b/d48/d9c/df7/f138 x:0 0 0 2024-04-02T04:43:12.968 INFO:tasks.workunit.client.1.smithi027.stdout:4/963: creat d24/d48/db0/dea/d11e/d127/f137 x:0 0 0 2024-04-02T04:43:12.971 INFO:tasks.workunit.client.1.smithi027.stdout:6/931: fdatasync d13/d1b/d48/d75/d9b/de2/d114/f44 0 2024-04-02T04:43:12.974 INFO:tasks.workunit.client.1.smithi027.stdout:8/998: dwrite d5/d8/db/dd/d5f/db2/d10b/d123/f129 [0,4194304] 0 2024-04-02T04:43:12.978 INFO:tasks.workunit.client.1.smithi027.stdout:4/964: creat d24/d48/db0/dea/d11e/d127/f138 x:0 0 0 2024-04-02T04:43:12.982 INFO:tasks.workunit.client.1.smithi027.stdout:6/932: dwrite d13/d1b/d48/d9c/df7/fa1 [0,4194304] 0 2024-04-02T04:43:12.984 INFO:tasks.workunit.client.1.smithi027.stdout:8/999: symlink d5/d8/d15/d71/db0/l14c 0 2024-04-02T04:43:12.987 INFO:tasks.workunit.client.1.smithi027.stdout:4/965: unlink d24/d48/db0/dea/d11e/d127/f138 0 2024-04-02T04:43:12.988 INFO:tasks.workunit.client.1.smithi027.stdout:6/933: fdatasync d13/d1b/f2b 0 2024-04-02T04:43:12.992 INFO:tasks.workunit.client.1.smithi027.stdout:4/966: rename d24/l55 to d24/d38/d47/d62/d11f/l139 0 2024-04-02T04:43:12.995 INFO:tasks.workunit.client.1.smithi027.stdout:4/967: creat d24/d38/d44/d41/d101/d118/f13a x:0 0 0 2024-04-02T04:43:12.998 INFO:tasks.workunit.client.1.smithi027.stdout:6/934: getdents d13/d1b/d48/d58 0 2024-04-02T04:43:12.999 INFO:tasks.workunit.client.1.smithi027.stdout:6/935: readlink d13/d1b/l10f 0 2024-04-02T04:43:13.004 INFO:tasks.workunit.client.1.smithi027.stdout:6/936: rename d13/d1b/d24/d25/ded/l116 to d13/d1b/d24/d28/d46/dcc/l139 0 2024-04-02T04:43:13.009 INFO:tasks.workunit.client.1.smithi027.stdout:6/937: mkdir d13/d1b/d48/d75/d13a 0 2024-04-02T04:43:13.017 INFO:tasks.workunit.client.1.smithi027.stdout:6/938: rename d13/l111 to d13/d1b/d48/d9c/db8/l13b 0 2024-04-02T04:43:13.018 INFO:tasks.workunit.client.1.smithi027.stdout:4/968: write f13 [3811123,125822] 0 2024-04-02T04:43:13.021 INFO:tasks.workunit.client.1.smithi027.stdout:4/969: fsync d24/d38/d44/fa5 0 2024-04-02T04:43:13.028 INFO:tasks.workunit.client.1.smithi027.stdout:4/970: link f2 d24/d38/d44/d41/d101/f13b 0 2024-04-02T04:43:13.033 INFO:tasks.workunit.client.1.smithi027.stdout:6/939: creat d13/d1b/d48/d9c/df7/d96/d127/f13c x:0 0 0 2024-04-02T04:43:13.038 INFO:tasks.workunit.client.1.smithi027.stdout:6/940: dwrite d13/d1b/d24/f3a [0,4194304] 0 2024-04-02T04:43:13.048 INFO:tasks.workunit.client.1.smithi027.stdout:6/941: write d13/d1b/d24/d25/d56/dc8/dc9/fda [559878,129746] 0 2024-04-02T04:43:13.051 INFO:tasks.workunit.client.1.smithi027.stdout:6/942: symlink d13/d1b/d48/d9c/l13d 0 2024-04-02T04:43:13.052 INFO:tasks.workunit.client.1.smithi027.stdout:6/943: write d13/d1b/d48/d75/d9b/de2/d114/f125 [939526,8628] 0 2024-04-02T04:43:13.053 INFO:tasks.workunit.client.1.smithi027.stdout:6/944: chown d13/d1b/d48/d75/d9b/de2/d114/f44 273141 1 2024-04-02T04:43:13.061 INFO:tasks.workunit.client.1.smithi027.stdout:6/945: creat d13/d1b/d24/d25/d56/d6d/f13e x:0 0 0 2024-04-02T04:43:13.062 INFO:tasks.workunit.client.1.smithi027.stdout:6/946: write d13/d1b/d48/d58/dd3/fe6 [640573,116893] 0 2024-04-02T04:43:13.065 INFO:tasks.workunit.client.1.smithi027.stdout:6/947: symlink d13/d1b/d48/d58/dd3/l13f 0 2024-04-02T04:43:13.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.063+0000 7f2c79efe700 1 -- 172.21.15.16:0/2714988692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c740ff9d0 msgr2=0x7f2c740ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:13.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.063+0000 7f2c79efe700 1 --2- 172.21.15.16:0/2714988692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c740ff9d0 0x7f2c740ffdf0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2c64009910 tx=0x7f2c64009c20 comp rx=0 tx=0).stop 2024-04-02T04:43:13.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.063+0000 7f2c79efe700 1 -- 172.21.15.16:0/2714988692 shutdown_connections 2024-04-02T04:43:13.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.063+0000 7f2c79efe700 1 --2- 172.21.15.16:0/2714988692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c74100bd0 0x7f2c74101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:13.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.063+0000 7f2c79efe700 1 --2- 172.21.15.16:0/2714988692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c740ff9d0 0x7f2c740ffdf0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:13.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.063+0000 7f2c79efe700 1 -- 172.21.15.16:0/2714988692 >> 172.21.15.16:0/2714988692 conn(0x7f2c740faf70 msgr2=0x7f2c740fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:13.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.063+0000 7f2c79efe700 1 -- 172.21.15.16:0/2714988692 shutdown_connections 2024-04-02T04:43:13.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.063+0000 7f2c79efe700 1 -- 172.21.15.16:0/2714988692 wait complete. 2024-04-02T04:43:13.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.063+0000 7f2c79efe700 1 Processor -- start 2024-04-02T04:43:13.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.064+0000 7f2c79efe700 1 -- start start 2024-04-02T04:43:13.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.064+0000 7f2c79efe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c740ff9d0 0x7f2c74196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:13.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.064+0000 7f2c79efe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c74100bd0 0x7f2c74196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:13.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.064+0000 7f2c79efe700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c741970e0 con 0x7f2c740ff9d0 2024-04-02T04:43:13.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.064+0000 7f2c79efe700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c74197220 con 0x7f2c74100bd0 2024-04-02T04:43:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.064+0000 7f2c737fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c740ff9d0 0x7f2c74196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.064+0000 7f2c737fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c740ff9d0 0x7f2c74196580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34568/0 (socket says 172.21.15.16:34568) 2024-04-02T04:43:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.064+0000 7f2c737fe700 1 -- 172.21.15.16:0/3100573547 learned_addr learned my addr 172.21.15.16:0/3100573547 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:43:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.064+0000 7f2c72ffd700 1 --2- 172.21.15.16:0/3100573547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c74100bd0 0x7f2c74196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.064+0000 7f2c737fe700 1 -- 172.21.15.16:0/3100573547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c74100bd0 msgr2=0x7f2c74196ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.064+0000 7f2c737fe700 1 --2- 172.21.15.16:0/3100573547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c74100bd0 0x7f2c74196ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.064+0000 7f2c737fe700 1 -- 172.21.15.16:0/3100573547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c640095f0 con 0x7f2c740ff9d0 2024-04-02T04:43:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.064+0000 7f2c737fe700 1 --2- 172.21.15.16:0/3100573547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c740ff9d0 0x7f2c74196580 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2c64005f00 tx=0x7f2c6400b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:13.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.065+0000 7f2c70ff9700 1 -- 172.21.15.16:0/3100573547 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c6401d070 con 0x7f2c740ff9d0 2024-04-02T04:43:13.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.065+0000 7f2c70ff9700 1 -- 172.21.15.16:0/3100573547 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2c640052a0 con 0x7f2c740ff9d0 2024-04-02T04:43:13.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.065+0000 7f2c79efe700 1 -- 172.21.15.16:0/3100573547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c7419bbd0 con 0x7f2c740ff9d0 2024-04-02T04:43:13.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.065+0000 7f2c70ff9700 1 -- 172.21.15.16:0/3100573547 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c64017400 con 0x7f2c740ff9d0 2024-04-02T04:43:13.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.065+0000 7f2c79efe700 1 -- 172.21.15.16:0/3100573547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c7419c0c0 con 0x7f2c740ff9d0 2024-04-02T04:43:13.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.066+0000 7f2c70ff9700 1 -- 172.21.15.16:0/3100573547 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 29) v1 ==== 46199+0+0 (secure 0 0 0) 0x7f2c64017560 con 0x7f2c740ff9d0 2024-04-02T04:43:13.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.066+0000 7f2c79efe700 1 -- 172.21.15.16:0/3100573547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c7404ea80 con 0x7f2c740ff9d0 2024-04-02T04:43:13.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.066+0000 7f2c70ff9700 1 --2- 172.21.15.16:0/3100573547 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f2c5c039090 0x7f2c5c03b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:13.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.066+0000 7f2c70ff9700 1 -- 172.21.15.16:0/3100573547 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f2c6404e5e0 con 0x7f2c740ff9d0 2024-04-02T04:43:13.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.066+0000 7f2c72ffd700 1 --2- 172.21.15.16:0/3100573547 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f2c5c039090 0x7f2c5c03b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:13.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.066+0000 7f2c72ffd700 1 --2- 172.21.15.16:0/3100573547 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f2c5c039090 0x7f2c5c03b550 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2c68009c60 tx=0x7f2c68009260 comp rx=0 tx=0).ready entity=mgr.14744 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:13.071 INFO:tasks.workunit.client.1.smithi027.stdout:6/948: dwrite f7 [0,4194304] 0 2024-04-02T04:43:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.071+0000 7f2c70ff9700 1 -- 172.21.15.16:0/3100573547 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184299 (secure 0 0 0) 0x7f2c64020370 con 0x7f2c740ff9d0 2024-04-02T04:43:13.091 INFO:tasks.workunit.client.1.smithi027.stdout:6/949: dread d13/d1b/d48/f10b [0,4194304] 0 2024-04-02T04:43:13.094 INFO:tasks.workunit.client.1.smithi027.stdout:6/950: mkdir d13/d1b/d48/d58/db9/d140 0 2024-04-02T04:43:13.095 INFO:tasks.workunit.client.1.smithi027.stdout:6/951: write d13/d1b/d48/d75/d9b/de2/d114/f125 [1262248,14954] 0 2024-04-02T04:43:13.097 INFO:tasks.workunit.client.1.smithi027.stdout:6/952: write d13/d1b/d24/d28/d46/f11f [707524,84376] 0 2024-04-02T04:43:13.099 INFO:tasks.workunit.client.1.smithi027.stdout:6/953: write d13/d1b/d24/d25/d56/dc8/dc9/fda [1619683,68386] 0 2024-04-02T04:43:13.106 INFO:tasks.workunit.client.1.smithi027.stdout:6/954: unlink d13/d1b/d24/d25/l37 0 2024-04-02T04:43:13.107 INFO:tasks.workunit.client.1.smithi027.stdout:6/955: stat d13/d1b/d24/c50 0 2024-04-02T04:43:13.109 INFO:tasks.workunit.client.1.smithi027.stdout:6/956: fdatasync d13/d1b/d24/d25/d56/dc8/f10a 0 2024-04-02T04:43:13.114 INFO:tasks.workunit.client.1.smithi027.stdout:6/957: dread d13/d1b/d48/d75/d9b/de2/d114/f3f [4194304,4194304] 0 2024-04-02T04:43:13.116 INFO:tasks.workunit.client.1.smithi027.stdout:6/958: chown d13/d1b/d48/d75/cb0 474 1 2024-04-02T04:43:13.117 INFO:tasks.workunit.client.1.smithi027.stdout:6/959: dread - d13/d1b/d48/fd6 zero size 2024-04-02T04:43:13.119 INFO:tasks.workunit.client.1.smithi027.stdout:6/960: symlink d13/d1b/d24/d25/d56/dc8/dc9/l141 0 2024-04-02T04:43:13.121 INFO:tasks.workunit.client.1.smithi027.stdout:6/961: creat d13/d1b/d48/d58/f142 x:0 0 0 2024-04-02T04:43:13.123 INFO:tasks.workunit.client.1.smithi027.stdout:6/962: symlink d13/d1b/d48/d75/d10c/l143 0 2024-04-02T04:43:13.136 INFO:tasks.workunit.client.1.smithi027.stdout:6/963: dread d13/d1b/d48/d9c/df7/d96/fac [0,4194304] 0 2024-04-02T04:43:13.146 INFO:tasks.workunit.client.1.smithi027.stdout:6/964: dwrite d13/f1a [0,4194304] 0 2024-04-02T04:43:13.148 INFO:tasks.workunit.client.1.smithi027.stdout:6/965: truncate d13/d1b/d24/d25/f12a 940322 0 2024-04-02T04:43:13.151 INFO:tasks.workunit.client.1.smithi027.stdout:6/966: write d13/f93 [1221247,61905] 0 2024-04-02T04:43:13.154 INFO:tasks.workunit.client.1.smithi027.stdout:6/967: mknod d13/d1b/d24/d25/d56/c144 0 2024-04-02T04:43:13.158 INFO:tasks.workunit.client.1.smithi027.stdout:6/968: dread d13/d1b/d48/d75/f8b [0,4194304] 0 2024-04-02T04:43:13.160 INFO:tasks.workunit.client.1.smithi027.stdout:6/969: write d13/d1b/d24/f3a [4071368,125717] 0 2024-04-02T04:43:13.166 INFO:tasks.workunit.client.1.smithi027.stdout:6/970: mkdir d13/d1b/d48/d9c/df7/d145 0 2024-04-02T04:43:13.166 INFO:tasks.workunit.client.1.smithi027.stdout:6/971: symlink d13/d1b/d48/d9c/df7/l146 0 2024-04-02T04:43:13.170 INFO:tasks.workunit.client.1.smithi027.stdout:6/972: truncate d13/d1b/d48/fd6 600758 0 2024-04-02T04:43:13.176 INFO:tasks.workunit.client.1.smithi027.stdout:6/973: dwrite d13/d1b/d48/d75/f83 [0,4194304] 0 2024-04-02T04:43:13.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.331+0000 7f2c79efe700 1 -- 172.21.15.16:0/3100573547 --> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c74068c40 con 0x7f2c5c039090 2024-04-02T04:43:13.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.332+0000 7f2c70ff9700 1 -- 172.21.15.16:0/3100573547 <== mgr.14744 v2:172.21.15.16:6800/2776599951 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f2c74068c40 con 0x7f2c5c039090 2024-04-02T04:43:13.336 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:43:13.336 INFO:teuthology.orchestra.run.smithi016.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:43:13.336 INFO:teuthology.orchestra.run.smithi016.stdout: "in_progress": true, 2024-04-02T04:43:13.336 INFO:teuthology.orchestra.run.smithi016.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-02T04:43:13.336 INFO:teuthology.orchestra.run.smithi016.stdout: "services_complete": [ 2024-04-02T04:43:13.336 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr" 2024-04-02T04:43:13.336 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-04-02T04:43:13.336 INFO:teuthology.orchestra.run.smithi016.stdout: "progress": "2/23 daemons upgraded", 2024-04-02T04:43:13.336 INFO:teuthology.orchestra.run.smithi016.stdout: "message": "", 2024-04-02T04:43:13.336 INFO:teuthology.orchestra.run.smithi016.stdout: "is_paused": false 2024-04-02T04:43:13.336 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:43:13.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.335+0000 7f2c79efe700 1 -- 172.21.15.16:0/3100573547 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f2c5c039090 msgr2=0x7f2c5c03b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:13.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.335+0000 7f2c79efe700 1 --2- 172.21.15.16:0/3100573547 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f2c5c039090 0x7f2c5c03b550 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2c68009c60 tx=0x7f2c68009260 comp rx=0 tx=0).stop 2024-04-02T04:43:13.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.335+0000 7f2c79efe700 1 -- 172.21.15.16:0/3100573547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c740ff9d0 msgr2=0x7f2c74196580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:13.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.335+0000 7f2c79efe700 1 --2- 172.21.15.16:0/3100573547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c740ff9d0 0x7f2c74196580 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2c64005f00 tx=0x7f2c6400b5e0 comp rx=0 tx=0).stop 2024-04-02T04:43:13.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.335+0000 7f2c79efe700 1 -- 172.21.15.16:0/3100573547 shutdown_connections 2024-04-02T04:43:13.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.335+0000 7f2c79efe700 1 --2- 172.21.15.16:0/3100573547 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f2c5c039090 0x7f2c5c03b550 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:13.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.335+0000 7f2c79efe700 1 --2- 172.21.15.16:0/3100573547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c740ff9d0 0x7f2c74196580 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:13.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.335+0000 7f2c79efe700 1 --2- 172.21.15.16:0/3100573547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c74100bd0 0x7f2c74196ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:13.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.335+0000 7f2c79efe700 1 -- 172.21.15.16:0/3100573547 >> 172.21.15.16:0/3100573547 conn(0x7f2c740faf70 msgr2=0x7f2c74068530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:13.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.336+0000 7f2c79efe700 1 -- 172.21.15.16:0/3100573547 shutdown_connections 2024-04-02T04:43:13.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.336+0000 7f2c79efe700 1 -- 172.21.15.16:0/3100573547 wait complete. 2024-04-02T04:43:13.477 INFO:tasks.workunit.client.1.smithi027.stdout:4/971: sync 2024-04-02T04:43:13.478 INFO:tasks.workunit.client.1.smithi027.stdout:6/974: sync 2024-04-02T04:43:13.480 INFO:tasks.workunit.client.1.smithi027.stdout:4/972: symlink d24/d38/d44/d82/d97/d9b/d104/d109/l13c 0 2024-04-02T04:43:13.481 INFO:tasks.workunit.client.1.smithi027.stdout:6/975: truncate d13/d1b/d24/d25/d56/dc8/fe9 404568 0 2024-04-02T04:43:13.482 INFO:tasks.workunit.client.1.smithi027.stdout:4/973: chown d24/d38/d44/d60/f6f 1341 1 2024-04-02T04:43:13.485 INFO:tasks.workunit.client.1.smithi027.stdout:4/974: truncate d24/d38/d44/d41/d101/f13b 1049502 0 2024-04-02T04:43:13.486 INFO:tasks.workunit.client.1.smithi027.stdout:6/976: link d13/d1b/d48/d75/l99 d13/d1b/d48/d58/db9/l147 0 2024-04-02T04:43:13.490 INFO:tasks.workunit.client.1.smithi027.stdout:6/977: dread d13/f1a [0,4194304] 0 2024-04-02T04:43:13.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.491+0000 7f18b0765700 1 -- 172.21.15.16:0/2051904774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18a80ff9e0 msgr2=0x7f18a80ffe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.491+0000 7f18b0765700 1 --2- 172.21.15.16:0/2051904774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18a80ff9e0 0x7f18a80ffe00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f18a4009910 tx=0x7f18a4009c20 comp rx=0 tx=0).stop 2024-04-02T04:43:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.491+0000 7f18b0765700 1 -- 172.21.15.16:0/2051904774 shutdown_connections 2024-04-02T04:43:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.491+0000 7f18b0765700 1 --2- 172.21.15.16:0/2051904774 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18a8100be0 0x7f18a8101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.491+0000 7f18b0765700 1 --2- 172.21.15.16:0/2051904774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18a80ff9e0 0x7f18a80ffe00 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.491+0000 7f18b0765700 1 -- 172.21.15.16:0/2051904774 >> 172.21.15.16:0/2051904774 conn(0x7f18a80faf80 msgr2=0x7f18a80fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.491+0000 7f18b0765700 1 -- 172.21.15.16:0/2051904774 shutdown_connections 2024-04-02T04:43:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.491+0000 7f18b0765700 1 -- 172.21.15.16:0/2051904774 wait complete. 2024-04-02T04:43:13.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.492+0000 7f18b0765700 1 Processor -- start 2024-04-02T04:43:13.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.492+0000 7f18b0765700 1 -- start start 2024-04-02T04:43:13.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.492+0000 7f18b0765700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18a80ff9e0 0x7f18a8103920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.492+0000 7f18b0765700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18a8100be0 0x7f18a8101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.492+0000 7f18b0765700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a819a140 con 0x7f18a80ff9e0 2024-04-02T04:43:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.492+0000 7f18b0765700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a819a280 con 0x7f18a8100be0 2024-04-02T04:43:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.493+0000 7f18ae501700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18a80ff9e0 0x7f18a8103920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.493+0000 7f18ae501700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18a80ff9e0 0x7f18a8103920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34582/0 (socket says 172.21.15.16:34582) 2024-04-02T04:43:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.493+0000 7f18add00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18a8100be0 0x7f18a8101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.493+0000 7f18add00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18a8100be0 0x7f18a8101fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.16:33984/0 (socket says 172.21.15.16:33984) 2024-04-02T04:43:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.493+0000 7f18add00700 1 -- 172.21.15.16:0/2921836604 learned_addr learned my addr 172.21.15.16:0/2921836604 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:43:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.493+0000 7f18add00700 1 -- 172.21.15.16:0/2921836604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18a80ff9e0 msgr2=0x7f18a8103920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.493+0000 7f18add00700 1 --2- 172.21.15.16:0/2921836604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18a80ff9e0 0x7f18a8103920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.493+0000 7f18add00700 1 -- 172.21.15.16:0/2921836604 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a40095f0 con 0x7f18a8100be0 2024-04-02T04:43:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.493+0000 7f18add00700 1 --2- 172.21.15.16:0/2921836604 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18a8100be0 0x7f18a8101fc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f189800e920 tx=0x7f189800ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.493+0000 7f189f7fe700 1 -- 172.21.15.16:0/2921836604 <== mon.1 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f189800c9d0 con 0x7f18a8100be0 2024-04-02T04:43:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.493+0000 7f18b0765700 1 -- 172.21.15.16:0/2921836604 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18a8102500 con 0x7f18a8100be0 2024-04-02T04:43:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.493+0000 7f18b0765700 1 -- 172.21.15.16:0/2921836604 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18a8102aa0 con 0x7f18a8100be0 2024-04-02T04:43:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.493+0000 7f189f7fe700 1 -- 172.21.15.16:0/2921836604 <== mon.1 v2:172.21.15.27:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1898010430 con 0x7f18a8100be0 2024-04-02T04:43:13.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.494+0000 7f189f7fe700 1 -- 172.21.15.16:0/2921836604 <== mon.1 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1898018630 con 0x7f18a8100be0 2024-04-02T04:43:13.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.495+0000 7f18b0765700 1 -- 172.21.15.16:0/2921836604 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18a8102d10 con 0x7f18a8100be0 2024-04-02T04:43:13.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.495+0000 7f189f7fe700 1 -- 172.21.15.16:0/2921836604 <== mon.1 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 29) v1 ==== 46199+0+0 (secure 0 0 0) 0x7f18980105a0 con 0x7f18a8100be0 2024-04-02T04:43:13.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.495+0000 7f189f7fe700 1 --2- 172.21.15.16:0/2921836604 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f1894039090 0x7f189403b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:13.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.495+0000 7f189f7fe700 1 -- 172.21.15.16:0/2921836604 <== mon.1 v2:172.21.15.27:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f1898014070 con 0x7f18a8100be0 2024-04-02T04:43:13.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.495+0000 7f18ae501700 1 --2- 172.21.15.16:0/2921836604 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f1894039090 0x7f189403b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:13.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.496+0000 7f18ae501700 1 --2- 172.21.15.16:0/2921836604 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f1894039090 0x7f189403b550 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f18a400b6f0 tx=0x7f18a4005fb0 comp rx=0 tx=0).ready entity=mgr.14744 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:13.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.500+0000 7f189f7fe700 1 -- 172.21.15.16:0/2921836604 <== mon.1 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184299 (secure 0 0 0) 0x7f1898016720 con 0x7f18a8100be0 2024-04-02T04:43:13.510 INFO:tasks.workunit.client.1.smithi027.stdout:4/975: write d24/d38/d44/d41/d5a/fd9 [1011706,116707] 0 2024-04-02T04:43:13.511 INFO:tasks.workunit.client.1.smithi027.stdout:6/978: write d13/d1b/f23 [352688,101312] 0 2024-04-02T04:43:13.516 INFO:tasks.workunit.client.1.smithi027.stdout:4/976: mknod d24/d38/d44/d82/d97/d9b/d104/c13d 0 2024-04-02T04:43:13.519 INFO:tasks.workunit.client.1.smithi027.stdout:4/977: symlink d24/d38/d44/d60/l13e 0 2024-04-02T04:43:13.521 INFO:tasks.workunit.client.1.smithi027.stdout:4/978: dread - d24/d38/d44/d82/d95/f12f zero size 2024-04-02T04:43:13.521 INFO:tasks.workunit.client.1.smithi027.stdout:4/979: stat d24/f3a 0 2024-04-02T04:43:13.522 INFO:tasks.workunit.client.1.smithi027.stdout:4/980: chown f11 1 1 2024-04-02T04:43:13.522 INFO:tasks.workunit.client.1.smithi027.stdout:6/979: dwrite d13/d1b/d48/d9c/df7/d96/fac [0,4194304] 0 2024-04-02T04:43:13.524 INFO:tasks.workunit.client.1.smithi027.stdout:4/981: rmdir d24/d48 39 2024-04-02T04:43:13.527 INFO:tasks.workunit.client.1.smithi027.stdout:6/980: rmdir d13/d1b/d48/d75/d9b/de2 39 2024-04-02T04:43:13.529 INFO:tasks.workunit.client.1.smithi027.stdout:4/982: mknod d24/d38/d44/dbc/c13f 0 2024-04-02T04:43:13.543 INFO:tasks.workunit.client.1.smithi027.stdout:4/983: truncate d24/d38/d44/d82/f90 95320 0 2024-04-02T04:43:13.550 INFO:tasks.workunit.client.1.smithi027.stdout:6/981: link d13/d1b/d24/d28/d46/dcc/l139 d13/d1b/d24/d25/l148 0 2024-04-02T04:43:13.553 INFO:tasks.workunit.client.1.smithi027.stdout:4/984: getdents d24/d38/d47/dc0 0 2024-04-02T04:43:13.555 INFO:tasks.workunit.client.1.smithi027.stdout:6/982: mkdir d13/d1b/d24/d149 0 2024-04-02T04:43:13.559 INFO:tasks.workunit.client.1.smithi027.stdout:4/985: dread - d24/d38/d47/d5c/fe3 zero size 2024-04-02T04:43:13.564 INFO:tasks.workunit.client.1.smithi027.stdout:4/986: dread d24/d38/d44/d41/f89 [0,4194304] 0 2024-04-02T04:43:13.566 INFO:tasks.workunit.client.1.smithi027.stdout:4/987: write d24/d38/d44/d82/d97/d9b/f10a [617486,95642] 0 2024-04-02T04:43:13.571 INFO:tasks.workunit.client.1.smithi027.stdout:4/988: mknod d24/d2d/d100/c140 0 2024-04-02T04:43:13.571 INFO:tasks.workunit.client.1.smithi027.stdout:6/983: dread d13/d1b/d24/f41 [4194304,4194304] 0 2024-04-02T04:43:13.571 INFO:tasks.workunit.client.1.smithi027.stdout:6/984: fsync d13/d1b/d24/d25/d56/d6d/fcf 0 2024-04-02T04:43:13.572 INFO:tasks.workunit.client.1.smithi027.stdout:6/985: chown d13/d1b/d24/d25/d42 28883616 1 2024-04-02T04:43:13.574 INFO:tasks.workunit.client.1.smithi027.stdout:6/986: mkdir d13/d1b/d48/d9c/df7/d14a 0 2024-04-02T04:43:13.577 INFO:tasks.workunit.client.1.smithi027.stdout:4/989: write d24/fbb [207460,1951] 0 2024-04-02T04:43:13.579 INFO:tasks.workunit.client.1.smithi027.stdout:6/987: symlink d13/d1b/d24/d149/l14b 0 2024-04-02T04:43:13.581 INFO:tasks.workunit.client.1.smithi027.stdout:4/990: mknod d24/d48/c141 0 2024-04-02T04:43:13.584 INFO:tasks.workunit.client.1.smithi027.stdout:4/991: mknod d24/d38/d44/d41/d101/c142 0 2024-04-02T04:43:13.585 INFO:tasks.workunit.client.1.smithi027.stdout:4/992: fsync d24/d38/d44/d41/f87 0 2024-04-02T04:43:13.587 INFO:tasks.workunit.client.1.smithi027.stdout:6/988: dwrite d13/d1b/d48/d9c/df7/f128 [0,4194304] 0 2024-04-02T04:43:13.589 INFO:tasks.workunit.client.1.smithi027.stdout:6/989: chown d13/d1b/d48/c67 559242008 1 2024-04-02T04:43:13.590 INFO:tasks.workunit.client.1.smithi027.stdout:4/993: mknod d24/d2d/d100/c143 0 2024-04-02T04:43:13.600 INFO:tasks.workunit.client.1.smithi027.stdout:6/990: creat d13/f14c x:0 0 0 2024-04-02T04:43:13.603 INFO:tasks.workunit.client.1.smithi027.stdout:4/994: link d24/d38/fcf d24/d38/d47/f144 0 2024-04-02T04:43:13.606 INFO:tasks.workunit.client.1.smithi027.stdout:4/995: rmdir d24/d38/d47/d5c 39 2024-04-02T04:43:13.608 INFO:tasks.workunit.client.1.smithi027.stdout:6/991: write d13/d1b/d48/d75/f8b [1288468,86806] 0 2024-04-02T04:43:13.613 INFO:tasks.workunit.client.1.smithi027.stdout:4/996: getdents d24/d38/d44/d41/d101/d118/d128 0 2024-04-02T04:43:13.616 INFO:tasks.workunit.client.1.smithi027.stdout:6/992: write d13/d1b/d24/fd1 [3393463,11951] 0 2024-04-02T04:43:13.617 INFO:tasks.workunit.client.1.smithi027.stdout:4/997: creat d24/d48/db0/f145 x:0 0 0 2024-04-02T04:43:13.618 INFO:tasks.workunit.client.1.smithi027.stdout:6/993: readlink d13/d1b/d48/d75/d9b/lfc 0 2024-04-02T04:43:13.618 INFO:tasks.workunit.client.1.smithi027.stdout:4/998: write f14 [4165032,110342] 0 2024-04-02T04:43:13.622 INFO:tasks.workunit.client.1.smithi027.stdout:4/999: truncate d24/f32 1540860 0 2024-04-02T04:43:13.633 INFO:tasks.workunit.client.1.smithi027.stdout:6/994: dread d13/d1b/d48/d75/d9b/de2/d114/f76 [0,4194304] 0 2024-04-02T04:43:13.641 INFO:tasks.workunit.client.1.smithi027.stdout:6/995: mkdir d13/d1b/d24/d28/d46/dcc/db7/d14d 0 2024-04-02T04:43:13.643 INFO:tasks.workunit.client.1.smithi027.stdout:6/996: mknod d13/d1b/d24/d25/ded/c14e 0 2024-04-02T04:43:13.647 INFO:tasks.workunit.client.1.smithi027.stdout:6/997: symlink d13/d1b/d24/d28/d46/dcc/da7/l14f 0 2024-04-02T04:43:13.648 INFO:tasks.workunit.client.1.smithi027.stdout:6/998: readlink d13/d1b/d48/d75/d10c/l143 0 2024-04-02T04:43:13.649 INFO:tasks.workunit.client.1.smithi027.stdout:6/999: symlink d13/d1b/d48/d58/db9/l150 0 2024-04-02T04:43:13.654 INFO:tasks.workunit.client.1.smithi027.stderr:+ rm -rf -- ./tmp.awxrg0SToX 2024-04-02T04:43:13.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.827+0000 7f18b0765700 1 -- 172.21.15.16:0/2921836604 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f18a8108540 con 0x7f18a8100be0 2024-04-02T04:43:13.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.828+0000 7f189f7fe700 1 -- 172.21.15.16:0/2921836604 <== mon.1 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f1898016330 con 0x7f18a8100be0 2024-04-02T04:43:13.831 INFO:teuthology.orchestra.run.smithi016.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-02T04:43:13.831 INFO:teuthology.orchestra.run.smithi016.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-02T04:43:13.831 INFO:teuthology.orchestra.run.smithi016.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-02T04:43:13.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.830+0000 7f18b0765700 1 -- 172.21.15.16:0/2921836604 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f1894039090 msgr2=0x7f189403b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:13.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.830+0000 7f18b0765700 1 --2- 172.21.15.16:0/2921836604 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f1894039090 0x7f189403b550 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f18a400b6f0 tx=0x7f18a4005fb0 comp rx=0 tx=0).stop 2024-04-02T04:43:13.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.830+0000 7f18b0765700 1 -- 172.21.15.16:0/2921836604 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18a8100be0 msgr2=0x7f18a8101fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:13.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.830+0000 7f18b0765700 1 --2- 172.21.15.16:0/2921836604 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18a8100be0 0x7f18a8101fc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f189800e920 tx=0x7f189800ec30 comp rx=0 tx=0).stop 2024-04-02T04:43:13.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.830+0000 7f18b0765700 1 -- 172.21.15.16:0/2921836604 shutdown_connections 2024-04-02T04:43:13.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.830+0000 7f18b0765700 1 --2- 172.21.15.16:0/2921836604 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f1894039090 0x7f189403b550 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:13.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.830+0000 7f18b0765700 1 --2- 172.21.15.16:0/2921836604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18a80ff9e0 0x7f18a8103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:13.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.830+0000 7f18b0765700 1 --2- 172.21.15.16:0/2921836604 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18a8100be0 0x7f18a8101fc0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:13.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.830+0000 7f18b0765700 1 -- 172.21.15.16:0/2921836604 >> 172.21.15.16:0/2921836604 conn(0x7f18a80faf80 msgr2=0x7f18a8068f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:13.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.830+0000 7f18b0765700 1 -- 172.21.15.16:0/2921836604 shutdown_connections 2024-04-02T04:43:13.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:13.830+0000 7f18b0765700 1 -- 172.21.15.16:0/2921836604 wait complete. 2024-04-02T04:43:13.871 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:13 smithi027 ceph-mon[37550]: pgmap v4: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail 2024-04-02T04:43:13.871 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:13 smithi027 ceph-mon[37550]: mgrmap e29: smithi016.eqnebx(active, since 2s) 2024-04-02T04:43:13.871 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:13 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/2014799175' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:43:13.975 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:13 smithi016 ceph-mon[28969]: pgmap v4: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail 2024-04-02T04:43:13.975 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:13 smithi016 ceph-mon[28969]: mgrmap e29: smithi016.eqnebx(active, since 2s) 2024-04-02T04:43:13.975 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:13 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/2014799175' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:43:15.040 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:14 smithi027 ceph-mon[37550]: from='client.14794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:43:15.040 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:14 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/2921836604' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:43:15.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:14 smithi016 ceph-mon[28969]: from='client.14794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:43:15.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:14 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/2921836604' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:43:15.895 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:15 smithi027 ceph-mon[37550]: pgmap v5: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail 2024-04-02T04:43:15.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:15 smithi027 ceph-mon[37550]: mgrmap e30: smithi016.eqnebx(active, since 4s) 2024-04-02T04:43:16.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:15 smithi016 ceph-mon[28969]: pgmap v5: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail 2024-04-02T04:43:16.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:15 smithi016 ceph-mon[28969]: mgrmap e30: smithi016.eqnebx(active, since 4s) 2024-04-02T04:43:16.974 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:16 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:16.975 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:16 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:16.975 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:16 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:16.975 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:16 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:17.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:16 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:17.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:16 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:17.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:16 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:17.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:16 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:18.051 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:17 smithi027 ceph-mon[37550]: pgmap v6: 65 pgs: 65 active+clean; 3.3 GiB data, 9.9 GiB used, 526 GiB / 536 GiB avail; 2.4 MiB/s rd, 7.0 MiB/s wr, 146 op/s 2024-04-02T04:43:18.052 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:17 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:18.052 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:17 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:18.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:17 smithi016 ceph-mon[28969]: pgmap v6: 65 pgs: 65 active+clean; 3.3 GiB data, 9.9 GiB used, 526 GiB / 536 GiB avail; 2.4 MiB/s rd, 7.0 MiB/s wr, 146 op/s 2024-04-02T04:43:18.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:17 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:18.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:17 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:19.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:19 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:19.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:19 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:19.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:19 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:19.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:19 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:20.140 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:20 smithi016 ceph-mon[28969]: pgmap v7: 65 pgs: 65 active+clean; 3.2 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 9.8 MiB/s rd, 22 MiB/s wr, 142 op/s 2024-04-02T04:43:20.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:20 smithi027 ceph-mon[37550]: pgmap v7: 65 pgs: 65 active+clean; 3.2 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 9.8 MiB/s rd, 22 MiB/s wr, 142 op/s 2024-04-02T04:43:21.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:21 smithi027 ceph-mon[37550]: Standby manager daemon smithi027.kjzcsc started 2024-04-02T04:43:21.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:21 smithi027 ceph-mon[37550]: from='mgr.? 172.21.15.27:0/4047030994' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi027.kjzcsc/crt"}]: dispatch 2024-04-02T04:43:21.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:21 smithi027 ceph-mon[37550]: from='mgr.? 172.21.15.27:0/4047030994' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T04:43:21.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:21 smithi027 ceph-mon[37550]: from='mgr.? 172.21.15.27:0/4047030994' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi027.kjzcsc/key"}]: dispatch 2024-04-02T04:43:21.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:21 smithi027 ceph-mon[37550]: from='mgr.? 172.21.15.27:0/4047030994' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T04:43:21.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:21 smithi016 ceph-mon[28969]: Standby manager daemon smithi027.kjzcsc started 2024-04-02T04:43:21.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:21 smithi016 ceph-mon[28969]: from='mgr.? 172.21.15.27:0/4047030994' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi027.kjzcsc/crt"}]: dispatch 2024-04-02T04:43:21.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:21 smithi016 ceph-mon[28969]: from='mgr.? 172.21.15.27:0/4047030994' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T04:43:21.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:21 smithi016 ceph-mon[28969]: from='mgr.? 172.21.15.27:0/4047030994' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi027.kjzcsc/key"}]: dispatch 2024-04-02T04:43:21.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:21 smithi016 ceph-mon[28969]: from='mgr.? 172.21.15.27:0/4047030994' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T04:43:22.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:22 smithi016 ceph-mon[28969]: pgmap v8: 65 pgs: 65 active+clean; 3.2 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 7.6 MiB/s rd, 17 MiB/s wr, 110 op/s 2024-04-02T04:43:22.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:22 smithi016 ceph-mon[28969]: mgrmap e31: smithi016.eqnebx(active, since 10s), standbys: smithi027.kjzcsc 2024-04-02T04:43:22.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:22 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mgr metadata", "who": "smithi027.kjzcsc", "id": "smithi027.kjzcsc"}]: dispatch 2024-04-02T04:43:22.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:22 smithi027 ceph-mon[37550]: pgmap v8: 65 pgs: 65 active+clean; 3.2 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 7.6 MiB/s rd, 17 MiB/s wr, 110 op/s 2024-04-02T04:43:22.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:22 smithi027 ceph-mon[37550]: mgrmap e31: smithi016.eqnebx(active, since 10s), standbys: smithi027.kjzcsc 2024-04-02T04:43:22.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:22 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mgr metadata", "who": "smithi027.kjzcsc", "id": "smithi027.kjzcsc"}]: dispatch 2024-04-02T04:43:23.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:23 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:23.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:23 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:23.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:23 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:43:23.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:23 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:43:23.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:23 smithi027 ceph-mon[37550]: Updating smithi016:/etc/ceph/ceph.conf 2024-04-02T04:43:23.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:23 smithi027 ceph-mon[37550]: Updating smithi027:/etc/ceph/ceph.conf 2024-04-02T04:43:23.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:23 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:23.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:23 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:23.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:23 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:43:23.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:23 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:43:23.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:23 smithi016 ceph-mon[28969]: Updating smithi016:/etc/ceph/ceph.conf 2024-04-02T04:43:23.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:23 smithi016 ceph-mon[28969]: Updating smithi027:/etc/ceph/ceph.conf 2024-04-02T04:43:23.731 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-02T04:43:23.731 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-02T04:43:24.376 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:24 smithi016 ceph-mon[28969]: Updating smithi027:/var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:43:24.376 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:24 smithi016 ceph-mon[28969]: Updating smithi016:/var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:43:24.376 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:24 smithi016 ceph-mon[28969]: pgmap v9: 65 pgs: 65 active+clean; 2.4 GiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 6.2 MiB/s rd, 15 MiB/s wr, 164 op/s 2024-04-02T04:43:24.376 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:24 smithi016 ceph-mon[28969]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-04-02T04:43:24.376 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:24 smithi016 ceph-mon[28969]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-04-02T04:43:24.376 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:24 smithi016 ceph-mon[28969]: Updating smithi016:/var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T04:43:24.376 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:24 smithi016 ceph-mon[28969]: Updating smithi027:/var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T04:43:24.377 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:24 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:24.377 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:24 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:24.377 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:24 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:24.377 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:24 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:24.377 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:24 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:24.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:24 smithi027 ceph-mon[37550]: Updating smithi027:/var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:43:24.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:24 smithi027 ceph-mon[37550]: Updating smithi016:/var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:43:24.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:24 smithi027 ceph-mon[37550]: pgmap v9: 65 pgs: 65 active+clean; 2.4 GiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 6.2 MiB/s rd, 15 MiB/s wr, 164 op/s 2024-04-02T04:43:24.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:24 smithi027 ceph-mon[37550]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-04-02T04:43:24.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:24 smithi027 ceph-mon[37550]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-04-02T04:43:24.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:24 smithi027 ceph-mon[37550]: Updating smithi016:/var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T04:43:24.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:24 smithi027 ceph-mon[37550]: Updating smithi027:/var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T04:43:24.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:24 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:24.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:24 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:24.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:24 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:24.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:24 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:24.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:24 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:25.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:25 smithi016 ceph-mon[28969]: Reconfiguring prometheus.smithi016 (dependencies changed)... 2024-04-02T04:43:25.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:25 smithi016 ceph-mon[28969]: Reconfiguring daemon prometheus.smithi016 on smithi016 2024-04-02T04:43:25.645 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:25 smithi027 ceph-mon[37550]: Reconfiguring prometheus.smithi016 (dependencies changed)... 2024-04-02T04:43:25.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:25 smithi027 ceph-mon[37550]: Reconfiguring daemon prometheus.smithi016 on smithi016 2024-04-02T04:43:26.424 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:26 smithi016 ceph-mon[28969]: pgmap v10: 65 pgs: 65 active+clean; 2.4 GiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 5.7 MiB/s rd, 14 MiB/s wr, 151 op/s 2024-04-02T04:43:26.424 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:26 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:43:26.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:26 smithi027 ceph-mon[37550]: pgmap v10: 65 pgs: 65 active+clean; 2.4 GiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 5.7 MiB/s rd, 14 MiB/s wr, 151 op/s 2024-04-02T04:43:26.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:26 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:43:28.522 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:28 smithi016 ceph-mon[28969]: pgmap v11: 65 pgs: 65 active+clean; 2.1 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 5.7 MiB/s rd, 14 MiB/s wr, 196 op/s 2024-04-02T04:43:28.645 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:28 smithi027 ceph-mon[37550]: pgmap v11: 65 pgs: 65 active+clean; 2.1 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 5.7 MiB/s rd, 14 MiB/s wr, 196 op/s 2024-04-02T04:43:30.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:30 smithi016 ceph-mon[28969]: pgmap v12: 65 pgs: 65 active+clean; 1.4 GiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 4.7 MiB/s rd, 11 MiB/s wr, 157 op/s 2024-04-02T04:43:30.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:30 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:30.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:30 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:30.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:30 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-02T04:43:30.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:30 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:43:30.618 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:30 smithi027 ceph-mon[37550]: pgmap v12: 65 pgs: 65 active+clean; 1.4 GiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 4.7 MiB/s rd, 11 MiB/s wr, 157 op/s 2024-04-02T04:43:30.618 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:30 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:30.618 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:30 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:30.618 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:30 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-02T04:43:30.618 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:30 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:43:31.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:31 smithi016 ceph-mon[28969]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-02T04:43:31.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:31 smithi027 ceph-mon[37550]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-02T04:43:32.292 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-02T04:43:32.292 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-04-02T04:43:32.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:32 smithi016 ceph-mon[28969]: pgmap v13: 65 pgs: 65 active+clean; 1.4 GiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 21 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-04-02T04:43:32.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:32 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi027.kjzcsc", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T04:43:32.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:32 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T04:43:32.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:32 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:43:32.644 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:32 smithi027 ceph-mon[37550]: pgmap v13: 65 pgs: 65 active+clean; 1.4 GiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 21 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-04-02T04:43:32.644 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:32 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi027.kjzcsc", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T04:43:32.644 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:32 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T04:43:32.644 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:32 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:43:33.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:33 smithi016 ceph-mon[28969]: Upgrade: Updating mgr.smithi027.kjzcsc 2024-04-02T04:43:33.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:33 smithi016 ceph-mon[28969]: Deploying daemon mgr.smithi027.kjzcsc on smithi027 2024-04-02T04:43:33.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:33 smithi027 ceph-mon[37550]: Upgrade: Updating mgr.smithi027.kjzcsc 2024-04-02T04:43:33.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:33 smithi027 ceph-mon[37550]: Deploying daemon mgr.smithi027.kjzcsc on smithi027 2024-04-02T04:43:34.331 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:34 smithi027 ceph-mon[37550]: pgmap v14: 65 pgs: 65 active+clean; 707 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 35 KiB/s rd, 1.8 MiB/s wr, 201 op/s 2024-04-02T04:43:34.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:34 smithi016 ceph-mon[28969]: pgmap v14: 65 pgs: 65 active+clean; 707 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 35 KiB/s rd, 1.8 MiB/s wr, 201 op/s 2024-04-02T04:43:36.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:36 smithi016 ceph-mon[28969]: pgmap v15: 65 pgs: 65 active+clean; 707 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 26 KiB/s rd, 1.2 MiB/s wr, 133 op/s 2024-04-02T04:43:36.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:36 smithi027 ceph-mon[37550]: pgmap v15: 65 pgs: 65 active+clean; 707 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 26 KiB/s rd, 1.2 MiB/s wr, 133 op/s 2024-04-02T04:43:38.551 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:38 smithi027 ceph-mon[37550]: pgmap v16: 65 pgs: 65 active+clean; 600 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 38 KiB/s rd, 1.3 MiB/s wr, 147 op/s 2024-04-02T04:43:38.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:38 smithi016 ceph-mon[28969]: pgmap v16: 65 pgs: 65 active+clean; 600 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 38 KiB/s rd, 1.3 MiB/s wr, 147 op/s 2024-04-02T04:43:40.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:40 smithi027 ceph-mon[37550]: pgmap v17: 65 pgs: 65 active+clean; 288 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-04-02T04:43:40.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:40 smithi016 ceph-mon[28969]: pgmap v17: 65 pgs: 65 active+clean; 288 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-04-02T04:43:41.515 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:41 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:43:41.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:41 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:43:42.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:42 smithi027 ceph-mon[37550]: pgmap v18: 65 pgs: 65 active+clean; 288 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 1.3 MiB/s wr, 120 op/s 2024-04-02T04:43:42.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:42 smithi016 ceph-mon[28969]: pgmap v18: 65 pgs: 65 active+clean; 288 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 1.3 MiB/s wr, 120 op/s 2024-04-02T04:43:43.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:43.941+0000 7fc56f209700 1 -- 172.21.15.16:0/1776939960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5680ffc90 msgr2=0x7fc568100110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:43.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:43.941+0000 7fc56f209700 1 --2- 172.21.15.16:0/1776939960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5680ffc90 0x7fc568100110 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc554009910 tx=0x7fc554009c20 comp rx=0 tx=0).stop 2024-04-02T04:43:43.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:43.941+0000 7fc56f209700 1 -- 172.21.15.16:0/1776939960 shutdown_connections 2024-04-02T04:43:43.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:43.941+0000 7fc56f209700 1 --2- 172.21.15.16:0/1776939960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5680ffc90 0x7fc568100110 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:43.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:43.941+0000 7fc56f209700 1 --2- 172.21.15.16:0/1776939960 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc568100fe0 0x7fc568101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:43.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:43.941+0000 7fc56f209700 1 -- 172.21.15.16:0/1776939960 >> 172.21.15.16:0/1776939960 conn(0x7fc5680fa700 msgr2=0x7fc5680fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:43.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:43.942+0000 7fc56f209700 1 -- 172.21.15.16:0/1776939960 shutdown_connections 2024-04-02T04:43:43.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:43.942+0000 7fc56f209700 1 -- 172.21.15.16:0/1776939960 wait complete. 2024-04-02T04:43:43.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:43.942+0000 7fc56f209700 1 Processor -- start 2024-04-02T04:43:43.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:43.942+0000 7fc56f209700 1 -- start start 2024-04-02T04:43:43.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:43.942+0000 7fc56f209700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5680ffc90 0x7fc568196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:43.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:43.942+0000 7fc56f209700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc568100fe0 0x7fc568196a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:43.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:43.942+0000 7fc56f209700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5681970a0 con 0x7fc5680ffc90 2024-04-02T04:43:43.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:43.942+0000 7fc56f209700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5681971e0 con 0x7fc568100fe0 2024-04-02T04:43:43.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:43.942+0000 7fc56cfa5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5680ffc90 0x7fc568196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:43.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:43.942+0000 7fc56cfa5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5680ffc90 0x7fc568196540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38268/0 (socket says 172.21.15.16:38268) 2024-04-02T04:43:43.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:43.942+0000 7fc56cfa5700 1 -- 172.21.15.16:0/995138614 learned_addr learned my addr 172.21.15.16:0/995138614 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:43:43.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:43.942+0000 7fc567fff700 1 --2- 172.21.15.16:0/995138614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc568100fe0 0x7fc568196a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:43.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:43.943+0000 7fc56cfa5700 1 -- 172.21.15.16:0/995138614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc568100fe0 msgr2=0x7fc568196a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:43.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:43.943+0000 7fc56cfa5700 1 --2- 172.21.15.16:0/995138614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc568100fe0 0x7fc568196a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:43.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:43.943+0000 7fc56cfa5700 1 -- 172.21.15.16:0/995138614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5540095f0 con 0x7fc5680ffc90 2024-04-02T04:43:43.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:43.943+0000 7fc567fff700 1 --2- 172.21.15.16:0/995138614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc568100fe0 0x7fc568196a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T04:43:43.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:43.943+0000 7fc56cfa5700 1 --2- 172.21.15.16:0/995138614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5680ffc90 0x7fc568196540 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc55c00d900 tx=0x7fc55c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:43.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:43.943+0000 7fc565ffb700 1 -- 172.21.15.16:0/995138614 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc55c00b920 con 0x7fc5680ffc90 2024-04-02T04:43:43.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:43.943+0000 7fc565ffb700 1 -- 172.21.15.16:0/995138614 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc55c010460 con 0x7fc5680ffc90 2024-04-02T04:43:43.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:43.943+0000 7fc565ffb700 1 -- 172.21.15.16:0/995138614 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc55c00f590 con 0x7fc5680ffc90 2024-04-02T04:43:43.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:43.943+0000 7fc56f209700 1 -- 172.21.15.16:0/995138614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc56819bc40 con 0x7fc5680ffc90 2024-04-02T04:43:43.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:43.943+0000 7fc56f209700 1 -- 172.21.15.16:0/995138614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc56819c140 con 0x7fc5680ffc90 2024-04-02T04:43:43.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:43.944+0000 7fc56f209700 1 -- 172.21.15.16:0/995138614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc56804ea80 con 0x7fc5680ffc90 2024-04-02T04:43:43.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:43.945+0000 7fc565ffb700 1 -- 172.21.15.16:0/995138614 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 31) v1 ==== 91816+0+0 (secure 0 0 0) 0x7fc55c0105d0 con 0x7fc5680ffc90 2024-04-02T04:43:43.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:43.945+0000 7fc565ffb700 1 --2- 172.21.15.16:0/995138614 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7fc55806db20 0x7fc55806ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:43.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:43.946+0000 7fc565ffb700 1 -- 172.21.15.16:0/995138614 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fc55c08d4d0 con 0x7fc5680ffc90 2024-04-02T04:43:43.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:43.946+0000 7fc567fff700 1 --2- 172.21.15.16:0/995138614 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7fc55806db20 0x7fc55806ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:43.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:43.946+0000 7fc567fff700 1 --2- 172.21.15.16:0/995138614 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7fc55806db20 0x7fc55806ffe0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc55400b5c0 tx=0x7fc55401a040 comp rx=0 tx=0).ready entity=mgr.14744 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:43.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:43.950+0000 7fc565ffb700 1 -- 172.21.15.16:0/995138614 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184299 (secure 0 0 0) 0x7fc55c05a850 con 0x7fc5680ffc90 2024-04-02T04:43:44.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.216+0000 7fc56f209700 1 -- 172.21.15.16:0/995138614 --> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5680ff2c0 con 0x7fc55806db20 2024-04-02T04:43:44.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.217+0000 7fc565ffb700 1 -- 172.21.15.16:0/995138614 <== mgr.14744 v2:172.21.15.16:6800/2776599951 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fc5680ff2c0 con 0x7fc55806db20 2024-04-02T04:43:44.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.219+0000 7fc56f209700 1 -- 172.21.15.16:0/995138614 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7fc55806db20 msgr2=0x7fc55806ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:44.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.219+0000 7fc56f209700 1 --2- 172.21.15.16:0/995138614 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7fc55806db20 0x7fc55806ffe0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc55400b5c0 tx=0x7fc55401a040 comp rx=0 tx=0).stop 2024-04-02T04:43:44.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.219+0000 7fc56f209700 1 -- 172.21.15.16:0/995138614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5680ffc90 msgr2=0x7fc568196540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:44.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.219+0000 7fc56f209700 1 --2- 172.21.15.16:0/995138614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5680ffc90 0x7fc568196540 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc55c00d900 tx=0x7fc55c00dc10 comp rx=0 tx=0).stop 2024-04-02T04:43:44.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.219+0000 7fc56f209700 1 -- 172.21.15.16:0/995138614 shutdown_connections 2024-04-02T04:43:44.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.219+0000 7fc56f209700 1 --2- 172.21.15.16:0/995138614 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7fc55806db20 0x7fc55806ffe0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:44.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.219+0000 7fc56f209700 1 --2- 172.21.15.16:0/995138614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5680ffc90 0x7fc568196540 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:44.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.219+0000 7fc56f209700 1 --2- 172.21.15.16:0/995138614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc568100fe0 0x7fc568196a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:44.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.219+0000 7fc56f209700 1 -- 172.21.15.16:0/995138614 >> 172.21.15.16:0/995138614 conn(0x7fc5680fa700 msgr2=0x7fc5680febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:44.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.220+0000 7fc56f209700 1 -- 172.21.15.16:0/995138614 shutdown_connections 2024-04-02T04:43:44.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.220+0000 7fc56f209700 1 -- 172.21.15.16:0/995138614 wait complete. 2024-04-02T04:43:44.233 INFO:teuthology.orchestra.run.smithi016.stdout:true 2024-04-02T04:43:44.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.328+0000 7fda8ba37700 1 -- 172.21.15.16:0/3572086821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda84100af0 msgr2=0x7fda84100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:44.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.328+0000 7fda8ba37700 1 --2- 172.21.15.16:0/3572086821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda84100af0 0x7fda84100f10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fda80009910 tx=0x7fda80009c20 comp rx=0 tx=0).stop 2024-04-02T04:43:44.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.328+0000 7fda8ba37700 1 -- 172.21.15.16:0/3572086821 shutdown_connections 2024-04-02T04:43:44.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.328+0000 7fda8ba37700 1 --2- 172.21.15.16:0/3572086821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda84101cf0 0x7fda84102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:44.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.328+0000 7fda8ba37700 1 --2- 172.21.15.16:0/3572086821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda84100af0 0x7fda84100f10 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:44.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.328+0000 7fda8ba37700 1 -- 172.21.15.16:0/3572086821 >> 172.21.15.16:0/3572086821 conn(0x7fda840fc0d0 msgr2=0x7fda840fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:44.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.328+0000 7fda8ba37700 1 -- 172.21.15.16:0/3572086821 shutdown_connections 2024-04-02T04:43:44.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.328+0000 7fda8ba37700 1 -- 172.21.15.16:0/3572086821 wait complete. 2024-04-02T04:43:44.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.328+0000 7fda8ba37700 1 Processor -- start 2024-04-02T04:43:44.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.329+0000 7fda8ba37700 1 -- start start 2024-04-02T04:43:44.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.329+0000 7fda8ba37700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda84100af0 0x7fda84198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:44.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.329+0000 7fda8ba37700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda84101cf0 0x7fda84198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:44.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.329+0000 7fda8ba37700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda84199290 con 0x7fda84101cf0 2024-04-02T04:43:44.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.329+0000 7fda8ba37700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda841993d0 con 0x7fda84100af0 2024-04-02T04:43:44.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.329+0000 7fda88fd2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda84101cf0 0x7fda84198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:44.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.329+0000 7fda897d3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda84100af0 0x7fda84198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:44.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.329+0000 7fda88fd2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda84101cf0 0x7fda84198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38282/0 (socket says 172.21.15.16:38282) 2024-04-02T04:43:44.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.329+0000 7fda897d3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda84100af0 0x7fda84198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.16:51902/0 (socket says 172.21.15.16:51902) 2024-04-02T04:43:44.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.329+0000 7fda88fd2700 1 -- 172.21.15.16:0/1765845980 learned_addr learned my addr 172.21.15.16:0/1765845980 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:43:44.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.329+0000 7fda88fd2700 1 -- 172.21.15.16:0/1765845980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda84100af0 msgr2=0x7fda84198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:44.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.329+0000 7fda88fd2700 1 --2- 172.21.15.16:0/1765845980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda84100af0 0x7fda84198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:44.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.329+0000 7fda88fd2700 1 -- 172.21.15.16:0/1765845980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda800095f0 con 0x7fda84101cf0 2024-04-02T04:43:44.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.329+0000 7fda897d3700 1 --2- 172.21.15.16:0/1765845980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda84100af0 0x7fda84198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T04:43:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.329+0000 7fda88fd2700 1 --2- 172.21.15.16:0/1765845980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda84101cf0 0x7fda84198c70 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fda7400e920 tx=0x7fda7400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.330+0000 7fda7a7fc700 1 -- 172.21.15.16:0/1765845980 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda7400c9d0 con 0x7fda84101cf0 2024-04-02T04:43:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.330+0000 7fda7a7fc700 1 -- 172.21.15.16:0/1765845980 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda74010430 con 0x7fda84101cf0 2024-04-02T04:43:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.330+0000 7fda8ba37700 1 -- 172.21.15.16:0/1765845980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda8419de30 con 0x7fda84101cf0 2024-04-02T04:43:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.330+0000 7fda7a7fc700 1 -- 172.21.15.16:0/1765845980 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda74018630 con 0x7fda84101cf0 2024-04-02T04:43:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.330+0000 7fda8ba37700 1 -- 172.21.15.16:0/1765845980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda8419e3d0 con 0x7fda84101cf0 2024-04-02T04:43:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.331+0000 7fda8ba37700 1 -- 172.21.15.16:0/1765845980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda841066c0 con 0x7fda84101cf0 2024-04-02T04:43:44.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.331+0000 7fda7a7fc700 1 -- 172.21.15.16:0/1765845980 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 31) v1 ==== 91816+0+0 (secure 0 0 0) 0x7fda740105a0 con 0x7fda84101cf0 2024-04-02T04:43:44.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.332+0000 7fda7a7fc700 1 --2- 172.21.15.16:0/1765845980 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7fda7006db20 0x7fda7006ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:44.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.332+0000 7fda7a7fc700 1 -- 172.21.15.16:0/1765845980 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fda74014070 con 0x7fda84101cf0 2024-04-02T04:43:44.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.332+0000 7fda897d3700 1 --2- 172.21.15.16:0/1765845980 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7fda7006db20 0x7fda7006ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:44.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.332+0000 7fda897d3700 1 --2- 172.21.15.16:0/1765845980 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7fda7006db20 0x7fda7006ffe0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fda8000b800 tx=0x7fda80009f70 comp rx=0 tx=0).ready entity=mgr.14744 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:44.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.336+0000 7fda7a7fc700 1 -- 172.21.15.16:0/1765845980 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184299 (secure 0 0 0) 0x7fda74053000 con 0x7fda84101cf0 2024-04-02T04:43:44.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:44 smithi016 ceph-mon[28969]: pgmap v19: 65 pgs: 65 active+clean; 294 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 41 KiB/s rd, 2.0 MiB/s wr, 159 op/s 2024-04-02T04:43:44.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.568+0000 7fda8ba37700 1 -- 172.21.15.16:0/1765845980 --> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fda8402cdc0 con 0x7fda7006db20 2024-04-02T04:43:44.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.569+0000 7fda7a7fc700 1 -- 172.21.15.16:0/1765845980 <== mgr.14744 v2:172.21.15.16:6800/2776599951 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fda8402cdc0 con 0x7fda7006db20 2024-04-02T04:43:44.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.571+0000 7fda8ba37700 1 -- 172.21.15.16:0/1765845980 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7fda7006db20 msgr2=0x7fda7006ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:44.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.571+0000 7fda8ba37700 1 --2- 172.21.15.16:0/1765845980 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7fda7006db20 0x7fda7006ffe0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fda8000b800 tx=0x7fda80009f70 comp rx=0 tx=0).stop 2024-04-02T04:43:44.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.571+0000 7fda8ba37700 1 -- 172.21.15.16:0/1765845980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda84101cf0 msgr2=0x7fda84198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:44.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.571+0000 7fda8ba37700 1 --2- 172.21.15.16:0/1765845980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda84101cf0 0x7fda84198c70 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fda7400e920 tx=0x7fda7400ec30 comp rx=0 tx=0).stop 2024-04-02T04:43:44.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.571+0000 7fda8ba37700 1 -- 172.21.15.16:0/1765845980 shutdown_connections 2024-04-02T04:43:44.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.571+0000 7fda8ba37700 1 --2- 172.21.15.16:0/1765845980 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7fda7006db20 0x7fda7006ffe0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:44.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.571+0000 7fda8ba37700 1 --2- 172.21.15.16:0/1765845980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda84100af0 0x7fda84198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:44.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.571+0000 7fda8ba37700 1 --2- 172.21.15.16:0/1765845980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda84101cf0 0x7fda84198c70 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:44.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.571+0000 7fda8ba37700 1 -- 172.21.15.16:0/1765845980 >> 172.21.15.16:0/1765845980 conn(0x7fda840fc0d0 msgr2=0x7fda84073330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:44.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.571+0000 7fda8ba37700 1 -- 172.21.15.16:0/1765845980 shutdown_connections 2024-04-02T04:43:44.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.571+0000 7fda8ba37700 1 -- 172.21.15.16:0/1765845980 wait complete. 2024-04-02T04:43:44.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:44 smithi027 ceph-mon[37550]: pgmap v19: 65 pgs: 65 active+clean; 294 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 41 KiB/s rd, 2.0 MiB/s wr, 159 op/s 2024-04-02T04:43:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.677+0000 7f98f4d2e700 1 -- 172.21.15.16:0/466186601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98f0103ea0 msgr2=0x7f98f0106290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.677+0000 7f98f4d2e700 1 --2- 172.21.15.16:0/466186601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98f0103ea0 0x7f98f0106290 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f98e4009910 tx=0x7f98e4009c20 comp rx=0 tx=0).stop 2024-04-02T04:43:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.678+0000 7f98f4d2e700 1 -- 172.21.15.16:0/466186601 shutdown_connections 2024-04-02T04:43:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.678+0000 7f98f4d2e700 1 --2- 172.21.15.16:0/466186601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98f0103ea0 0x7f98f0106290 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.678+0000 7f98f4d2e700 1 --2- 172.21.15.16:0/466186601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98f0101570 0x7f98f0103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.678+0000 7f98f4d2e700 1 -- 172.21.15.16:0/466186601 >> 172.21.15.16:0/466186601 conn(0x7f98f00faf80 msgr2=0x7f98f00fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.678+0000 7f98f4d2e700 1 -- 172.21.15.16:0/466186601 shutdown_connections 2024-04-02T04:43:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.678+0000 7f98f4d2e700 1 -- 172.21.15.16:0/466186601 wait complete. 2024-04-02T04:43:44.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.678+0000 7f98f4d2e700 1 Processor -- start 2024-04-02T04:43:44.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.678+0000 7f98f4d2e700 1 -- start start 2024-04-02T04:43:44.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.679+0000 7f98f4d2e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98f0101570 0x7f98f01964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:44.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.679+0000 7f98f4d2e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98f0103ea0 0x7f98f01969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:44.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.679+0000 7f98f4d2e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f0197000 con 0x7f98f0101570 2024-04-02T04:43:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.679+0000 7f98f4d2e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f0197140 con 0x7f98f0103ea0 2024-04-02T04:43:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.679+0000 7f98ee59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98f0101570 0x7f98f01964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.679+0000 7f98ee59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98f0101570 0x7f98f01964a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38308/0 (socket says 172.21.15.16:38308) 2024-04-02T04:43:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.679+0000 7f98ee59c700 1 -- 172.21.15.16:0/2305507930 learned_addr learned my addr 172.21.15.16:0/2305507930 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:43:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.679+0000 7f98edd9b700 1 --2- 172.21.15.16:0/2305507930 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98f0103ea0 0x7f98f01969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.679+0000 7f98ee59c700 1 -- 172.21.15.16:0/2305507930 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98f0103ea0 msgr2=0x7f98f01969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.679+0000 7f98ee59c700 1 --2- 172.21.15.16:0/2305507930 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98f0103ea0 0x7f98f01969e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.679+0000 7f98ee59c700 1 -- 172.21.15.16:0/2305507930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e40095f0 con 0x7f98f0101570 2024-04-02T04:43:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.679+0000 7f98ee59c700 1 --2- 172.21.15.16:0/2305507930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98f0101570 0x7f98f01964a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f98e000e920 tx=0x7f98e000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.679+0000 7f98df7fe700 1 -- 172.21.15.16:0/2305507930 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98e000c9d0 con 0x7f98f0101570 2024-04-02T04:43:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.679+0000 7f98df7fe700 1 -- 172.21.15.16:0/2305507930 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f98e0010430 con 0x7f98f0101570 2024-04-02T04:43:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.679+0000 7f98df7fe700 1 -- 172.21.15.16:0/2305507930 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98e0018630 con 0x7f98f0101570 2024-04-02T04:43:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.680+0000 7f98f4d2e700 1 -- 172.21.15.16:0/2305507930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98f019bba0 con 0x7f98f0101570 2024-04-02T04:43:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.680+0000 7f98f4d2e700 1 -- 172.21.15.16:0/2305507930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98f019c0f0 con 0x7f98f0101570 2024-04-02T04:43:44.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.681+0000 7f98f4d2e700 1 -- 172.21.15.16:0/2305507930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98f004ea80 con 0x7f98f0101570 2024-04-02T04:43:44.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.681+0000 7f98df7fe700 1 -- 172.21.15.16:0/2305507930 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 31) v1 ==== 91816+0+0 (secure 0 0 0) 0x7f98e00105a0 con 0x7f98f0101570 2024-04-02T04:43:44.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.682+0000 7f98df7fe700 1 --2- 172.21.15.16:0/2305507930 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f98d806db20 0x7f98d806ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:44.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.682+0000 7f98df7fe700 1 -- 172.21.15.16:0/2305507930 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f98e0014070 con 0x7f98f0101570 2024-04-02T04:43:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.682+0000 7f98edd9b700 1 --2- 172.21.15.16:0/2305507930 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f98d806db20 0x7f98d806ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.682+0000 7f98edd9b700 1 --2- 172.21.15.16:0/2305507930 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f98d806db20 0x7f98d806ffe0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f98e400b800 tx=0x7f98e4009f70 comp rx=0 tx=0).ready entity=mgr.14744 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:44.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.686+0000 7f98df7fe700 1 -- 172.21.15.16:0/2305507930 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184299 (secure 0 0 0) 0x7f98e0053000 con 0x7f98f0101570 2024-04-02T04:43:44.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.911+0000 7f98f4d2e700 1 -- 172.21.15.16:0/2305507930 --> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f98f0104840 con 0x7f98d806db20 2024-04-02T04:43:44.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.922+0000 7f98df7fe700 1 -- 172.21.15.16:0/2305507930 <== mgr.14744 v2:172.21.15.16:6800/2776599951 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f98f0104840 con 0x7f98d806db20 2024-04-02T04:43:44.925 INFO:teuthology.orchestra.run.smithi016.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:43:44.925 INFO:teuthology.orchestra.run.smithi016.stdout:alertmanager.smithi016 smithi016 *:9093,9094 running (8m) 28s ago 9m 24.2M - 0.25.0 c8568f914cd2 b95d8354b016 2024-04-02T04:43:44.925 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi016 smithi016 running (10m) 28s ago 9m 7726k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 9de6c16a487f 2024-04-02T04:43:44.925 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi027 smithi027 running (8m) 29s ago 8m 7772k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 99956592471a 2024-04-02T04:43:44.925 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi016 smithi016 running (9m) 28s ago 9m 6656k - 18.2.2-1059-g4d05f4ca 909dce4fe53d bd73919b0711 2024-04-02T04:43:44.926 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi027 smithi027 running (8m) 29s ago 8m 6660k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 7a54c37f202e 2024-04-02T04:43:44.926 INFO:teuthology.orchestra.run.smithi016.stdout:grafana.smithi016 smithi016 *:3000 running (7m) 28s ago 9m 85.0M - 9.4.7 954c08fa6188 7c43d133569c 2024-04-02T04:43:44.926 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.fyzrnm smithi016 running (4m) 28s ago 4m 15.4M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 70f28b9572a6 2024-04-02T04:43:44.926 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.uhgkgg smithi016 running (4m) 28s ago 4m 13.7M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 0aa4d16562a6 2024-04-02T04:43:44.926 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.sdublg smithi027 running (3m) 29s ago 3m 17.7M - 18.2.2-1059-g4d05f4ca 909dce4fe53d a538451099db 2024-04-02T04:43:44.926 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.zwrfoo smithi027 running (4m) 29s ago 4m 252M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 4cb3fa428d71 2024-04-02T04:43:44.926 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi016.eqnebx smithi016 *:8443,9283,8765 running (46s) 28s ago 11m 475M - 19.0.0-1979-g6cec793d 78f830190839 d61ed91b4577 2024-04-02T04:43:44.926 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi027.kjzcsc smithi027 *:8443,9283,8765 running (2m) 29s ago 8m 68.1M - 19.0.0-1979-g6cec793d 78f830190839 a1e2b7522dcb 2024-04-02T04:43:44.926 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi016 smithi016 running (11m) 28s ago 11m 53.5M 2048M 18.2.2-1059-g4d05f4ca 909dce4fe53d 24d5820dc994 2024-04-02T04:43:44.926 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi027 smithi027 running (8m) 29s ago 8m 42.2M 2048M 18.2.2-1059-g4d05f4ca 909dce4fe53d b77563184a1e 2024-04-02T04:43:44.926 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi016 smithi016 *:9100 running (9m) 28s ago 9m 13.7M - 1.5.0 0da6a335fe13 09f54c65b53c 2024-04-02T04:43:44.926 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi027 smithi027 *:9100 running (8m) 29s ago 8m 15.4M - 1.5.0 0da6a335fe13 e41a657c7443 2024-04-02T04:43:44.926 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 smithi016 running (7m) 28s ago 7m 189M 1217M 18.2.2-1059-g4d05f4ca 909dce4fe53d 3f40108debd6 2024-04-02T04:43:44.926 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 smithi016 running (7m) 28s ago 7m 198M 1217M 18.2.2-1059-g4d05f4ca 909dce4fe53d 7361abc9e1b6 2024-04-02T04:43:44.927 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 smithi016 running (6m) 28s ago 6m 165M 1217M 18.2.2-1059-g4d05f4ca 909dce4fe53d 52e592b4d1f2 2024-04-02T04:43:44.927 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 smithi027 running (6m) 29s ago 6m 218M 2240M 18.2.2-1059-g4d05f4ca 909dce4fe53d cf7c7fe022f0 2024-04-02T04:43:44.927 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 smithi027 running (6m) 29s ago 6m 201M 2240M 18.2.2-1059-g4d05f4ca 909dce4fe53d 8cbbedf8cac9 2024-04-02T04:43:44.927 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 smithi027 running (5m) 29s ago 5m 210M 2240M 18.2.2-1059-g4d05f4ca 909dce4fe53d ac6094415529 2024-04-02T04:43:44.927 INFO:teuthology.orchestra.run.smithi016.stdout:prometheus.smithi016 smithi016 *:9095 starting - - - - 2024-04-02T04:43:44.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.924+0000 7f98f4d2e700 1 -- 172.21.15.16:0/2305507930 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f98d806db20 msgr2=0x7f98d806ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:44.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.924+0000 7f98f4d2e700 1 --2- 172.21.15.16:0/2305507930 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f98d806db20 0x7f98d806ffe0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f98e400b800 tx=0x7f98e4009f70 comp rx=0 tx=0).stop 2024-04-02T04:43:44.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.924+0000 7f98f4d2e700 1 -- 172.21.15.16:0/2305507930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98f0101570 msgr2=0x7f98f01964a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:44.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.924+0000 7f98f4d2e700 1 --2- 172.21.15.16:0/2305507930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98f0101570 0x7f98f01964a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f98e000e920 tx=0x7f98e000ec30 comp rx=0 tx=0).stop 2024-04-02T04:43:44.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.924+0000 7f98f4d2e700 1 -- 172.21.15.16:0/2305507930 shutdown_connections 2024-04-02T04:43:44.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.924+0000 7f98f4d2e700 1 --2- 172.21.15.16:0/2305507930 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f98d806db20 0x7f98d806ffe0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:44.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.924+0000 7f98f4d2e700 1 --2- 172.21.15.16:0/2305507930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98f0101570 0x7f98f01964a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:44.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.924+0000 7f98f4d2e700 1 --2- 172.21.15.16:0/2305507930 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98f0103ea0 0x7f98f01969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:44.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.924+0000 7f98f4d2e700 1 -- 172.21.15.16:0/2305507930 >> 172.21.15.16:0/2305507930 conn(0x7f98f00faf80 msgr2=0x7f98f00fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:44.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.924+0000 7f98f4d2e700 1 -- 172.21.15.16:0/2305507930 shutdown_connections 2024-04-02T04:43:44.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:44.924+0000 7f98f4d2e700 1 -- 172.21.15.16:0/2305507930 wait complete. 2024-04-02T04:43:45.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.059+0000 7f630e19a700 1 -- 172.21.15.16:0/2251667730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6308100390 msgr2=0x7f63081007f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:45.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.059+0000 7f630e19a700 1 --2- 172.21.15.16:0/2251667730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6308100390 0x7f63081007f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f62fc009910 tx=0x7f62fc009c20 comp rx=0 tx=0).stop 2024-04-02T04:43:45.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.059+0000 7f630e19a700 1 -- 172.21.15.16:0/2251667730 shutdown_connections 2024-04-02T04:43:45.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.059+0000 7f630e19a700 1 --2- 172.21.15.16:0/2251667730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6308100390 0x7f63081007f0 secure :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f62fc009910 tx=0x7f62fc009c20 comp rx=0 tx=0).stop 2024-04-02T04:43:45.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.059+0000 7f630e19a700 1 --2- 172.21.15.16:0/2251667730 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63080ff190 0x7f63080ff5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:45.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.059+0000 7f630e19a700 1 -- 172.21.15.16:0/2251667730 >> 172.21.15.16:0/2251667730 conn(0x7f63080fa730 msgr2=0x7f63080fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:45.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.060+0000 7f630e19a700 1 -- 172.21.15.16:0/2251667730 shutdown_connections 2024-04-02T04:43:45.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.060+0000 7f630e19a700 1 -- 172.21.15.16:0/2251667730 wait complete. 2024-04-02T04:43:45.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.061+0000 7f630e19a700 1 Processor -- start 2024-04-02T04:43:45.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.061+0000 7f630e19a700 1 -- start start 2024-04-02T04:43:45.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.061+0000 7f630e19a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63080ff190 0x7f6308071330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:45.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.061+0000 7f630e19a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6308071870 0x7f6308071cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:45.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.061+0000 7f630e19a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6308072c90 con 0x7f6308071870 2024-04-02T04:43:45.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.061+0000 7f630e19a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6308072e00 con 0x7f63080ff190 2024-04-02T04:43:45.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.062+0000 7f63077fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63080ff190 0x7f6308071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:45.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.062+0000 7f63077fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63080ff190 0x7f6308071330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.16:51922/0 (socket says 172.21.15.16:51922) 2024-04-02T04:43:45.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.062+0000 7f63077fe700 1 -- 172.21.15.16:0/2820874025 learned_addr learned my addr 172.21.15.16:0/2820874025 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:43:45.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.062+0000 7f6306ffd700 1 --2- 172.21.15.16:0/2820874025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6308071870 0x7f6308071cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:45.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.062+0000 7f63077fe700 1 -- 172.21.15.16:0/2820874025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6308071870 msgr2=0x7f6308071cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:45.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.062+0000 7f63077fe700 1 --2- 172.21.15.16:0/2820874025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6308071870 0x7f6308071cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:45.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.062+0000 7f63077fe700 1 -- 172.21.15.16:0/2820874025 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62fc0095f0 con 0x7f63080ff190 2024-04-02T04:43:45.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.063+0000 7f63077fe700 1 --2- 172.21.15.16:0/2820874025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63080ff190 0x7f6308071330 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f630000e8f0 tx=0x7f630000ec00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:45.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.063+0000 7f6304ff9700 1 -- 172.21.15.16:0/2820874025 <== mon.1 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63000041f0 con 0x7f63080ff190 2024-04-02T04:43:45.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.063+0000 7f630e19a700 1 -- 172.21.15.16:0/2820874025 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f630810cb10 con 0x7f63080ff190 2024-04-02T04:43:45.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.063+0000 7f630e19a700 1 -- 172.21.15.16:0/2820874025 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63081a2e10 con 0x7f63080ff190 2024-04-02T04:43:45.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.063+0000 7f6304ff9700 1 -- 172.21.15.16:0/2820874025 <== mon.1 v2:172.21.15.27:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f630000ccd0 con 0x7f63080ff190 2024-04-02T04:43:45.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.063+0000 7f6304ff9700 1 -- 172.21.15.16:0/2820874025 <== mon.1 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6300003c80 con 0x7f63080ff190 2024-04-02T04:43:45.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.065+0000 7f630e19a700 1 -- 172.21.15.16:0/2820874025 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f630804ea80 con 0x7f63080ff190 2024-04-02T04:43:45.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.065+0000 7f6304ff9700 1 -- 172.21.15.16:0/2820874025 <== mon.1 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 31) v1 ==== 91816+0+0 (secure 0 0 0) 0x7f6300018430 con 0x7f63080ff190 2024-04-02T04:43:45.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.066+0000 7f6304ff9700 1 --2- 172.21.15.16:0/2820874025 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f62f006db20 0x7f62f006ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:45.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.066+0000 7f6304ff9700 1 -- 172.21.15.16:0/2820874025 <== mon.1 v2:172.21.15.27:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f6300014070 con 0x7f63080ff190 2024-04-02T04:43:45.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.066+0000 7f6306ffd700 1 --2- 172.21.15.16:0/2820874025 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f62f006db20 0x7f62f006ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:45.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.067+0000 7f6306ffd700 1 --2- 172.21.15.16:0/2820874025 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f62f006db20 0x7f62f006ffe0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f62fc00b7d0 tx=0x7f62fc005fb0 comp rx=0 tx=0).ready entity=mgr.14744 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:45.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.075+0000 7f6304ff9700 1 -- 172.21.15.16:0/2820874025 <== mon.1 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184299 (secure 0 0 0) 0x7f630005bf00 con 0x7f63080ff190 2024-04-02T04:43:45.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.377+0000 7f630e19a700 1 -- 172.21.15.16:0/2820874025 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6308107530 con 0x7f63080ff190 2024-04-02T04:43:45.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.377+0000 7f6304ff9700 1 -- 172.21.15.16:0/2820874025 <== mon.1 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+735 (secure 0 0 0) 0x7f630005ba90 con 0x7f63080ff190 2024-04-02T04:43:45.380 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:43:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": { 2024-04-02T04:43:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 2 2024-04-02T04:43:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:43:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": { 2024-04-02T04:43:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:43:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:43:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": { 2024-04-02T04:43:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 6 2024-04-02T04:43:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:43:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-04-02T04:43:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 4 2024-04-02T04:43:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:43:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: "overall": { 2024-04-02T04:43:45.381 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 12, 2024-04-02T04:43:45.382 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:43:45.382 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-02T04:43:45.382 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:43:45.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.380+0000 7f630e19a700 1 -- 172.21.15.16:0/2820874025 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f62f006db20 msgr2=0x7f62f006ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:45.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.380+0000 7f630e19a700 1 --2- 172.21.15.16:0/2820874025 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f62f006db20 0x7f62f006ffe0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f62fc00b7d0 tx=0x7f62fc005fb0 comp rx=0 tx=0).stop 2024-04-02T04:43:45.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.380+0000 7f630e19a700 1 -- 172.21.15.16:0/2820874025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63080ff190 msgr2=0x7f6308071330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:45.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.380+0000 7f630e19a700 1 --2- 172.21.15.16:0/2820874025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63080ff190 0x7f6308071330 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f630000e8f0 tx=0x7f630000ec00 comp rx=0 tx=0).stop 2024-04-02T04:43:45.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.380+0000 7f630e19a700 1 -- 172.21.15.16:0/2820874025 shutdown_connections 2024-04-02T04:43:45.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.380+0000 7f630e19a700 1 --2- 172.21.15.16:0/2820874025 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f62f006db20 0x7f62f006ffe0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:45.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.380+0000 7f630e19a700 1 --2- 172.21.15.16:0/2820874025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63080ff190 0x7f6308071330 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:45.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.380+0000 7f630e19a700 1 --2- 172.21.15.16:0/2820874025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6308071870 0x7f6308071cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:45.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.380+0000 7f630e19a700 1 -- 172.21.15.16:0/2820874025 >> 172.21.15.16:0/2820874025 conn(0x7f63080fa730 msgr2=0x7f63080fbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:45.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.380+0000 7f630e19a700 1 -- 172.21.15.16:0/2820874025 shutdown_connections 2024-04-02T04:43:45.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.380+0000 7f630e19a700 1 -- 172.21.15.16:0/2820874025 wait complete. 2024-04-02T04:43:45.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:45 smithi027 ceph-mon[37550]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:43:45.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.490+0000 7f110929f700 1 -- 172.21.15.16:0/4051173915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1104100bd0 msgr2=0x7f1104101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:45.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.490+0000 7f110929f700 1 --2- 172.21.15.16:0/4051173915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1104100bd0 0x7f1104101030 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f10f8009910 tx=0x7f10f8009c20 comp rx=0 tx=0).stop 2024-04-02T04:43:45.494 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:45 smithi016 ceph-mon[28969]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:43:45.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.490+0000 7f110929f700 1 -- 172.21.15.16:0/4051173915 shutdown_connections 2024-04-02T04:43:45.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.490+0000 7f110929f700 1 --2- 172.21.15.16:0/4051173915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1104100bd0 0x7f1104101030 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:45.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.490+0000 7f110929f700 1 --2- 172.21.15.16:0/4051173915 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11040ff9d0 0x7f11040ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:45.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.490+0000 7f110929f700 1 -- 172.21.15.16:0/4051173915 >> 172.21.15.16:0/4051173915 conn(0x7f11040faf70 msgr2=0x7f11040fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:45.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.490+0000 7f110929f700 1 -- 172.21.15.16:0/4051173915 shutdown_connections 2024-04-02T04:43:45.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.490+0000 7f110929f700 1 -- 172.21.15.16:0/4051173915 wait complete. 2024-04-02T04:43:45.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.491+0000 7f110929f700 1 Processor -- start 2024-04-02T04:43:45.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.491+0000 7f110929f700 1 -- start start 2024-04-02T04:43:45.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.491+0000 7f110929f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11040ff9d0 0x7f11041038c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:45.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.491+0000 7f110929f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1104100bd0 0x7f1104101fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:45.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.491+0000 7f110929f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11041024f0 con 0x7f11040ff9d0 2024-04-02T04:43:45.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.491+0000 7f110929f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1104102630 con 0x7f1104100bd0 2024-04-02T04:43:45.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.491+0000 7f1102ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11040ff9d0 0x7f11041038c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:45.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.491+0000 7f1102ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11040ff9d0 0x7f11041038c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38350/0 (socket says 172.21.15.16:38350) 2024-04-02T04:43:45.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.491+0000 7f1102ffd700 1 -- 172.21.15.16:0/65733 learned_addr learned my addr 172.21.15.16:0/65733 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:43:45.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.491+0000 7f11027fc700 1 --2- 172.21.15.16:0/65733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1104100bd0 0x7f1104101fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:45.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.492+0000 7f1102ffd700 1 -- 172.21.15.16:0/65733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1104100bd0 msgr2=0x7f1104101fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:45.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.492+0000 7f1102ffd700 1 --2- 172.21.15.16:0/65733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1104100bd0 0x7f1104101fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:45.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.492+0000 7f1102ffd700 1 -- 172.21.15.16:0/65733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f80095f0 con 0x7f11040ff9d0 2024-04-02T04:43:45.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.492+0000 7f1102ffd700 1 --2- 172.21.15.16:0/65733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11040ff9d0 0x7f11041038c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f10f400e8f0 tx=0x7f10f400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:45.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.492+0000 7f10ebfff700 1 -- 172.21.15.16:0/65733 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10f4009790 con 0x7f11040ff9d0 2024-04-02T04:43:45.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.492+0000 7f10ebfff700 1 -- 172.21.15.16:0/65733 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f10f400c810 con 0x7f11040ff9d0 2024-04-02T04:43:45.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.492+0000 7f110929f700 1 -- 172.21.15.16:0/65733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1104102910 con 0x7f11040ff9d0 2024-04-02T04:43:45.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.492+0000 7f10ebfff700 1 -- 172.21.15.16:0/65733 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10f4003d20 con 0x7f11040ff9d0 2024-04-02T04:43:45.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.492+0000 7f110929f700 1 -- 172.21.15.16:0/65733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f110419e5e0 con 0x7f11040ff9d0 2024-04-02T04:43:45.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.493+0000 7f110929f700 1 -- 172.21.15.16:0/65733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f110404ea80 con 0x7f11040ff9d0 2024-04-02T04:43:45.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.493+0000 7f10ebfff700 1 -- 172.21.15.16:0/65733 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 31) v1 ==== 91816+0+0 (secure 0 0 0) 0x7f10f40041f0 con 0x7f11040ff9d0 2024-04-02T04:43:45.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.494+0000 7f10ebfff700 1 --2- 172.21.15.16:0/65733 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f10ec06db20 0x7f10ec06ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:45.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.494+0000 7f10ebfff700 1 -- 172.21.15.16:0/65733 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f10f4014070 con 0x7f11040ff9d0 2024-04-02T04:43:45.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.494+0000 7f11027fc700 1 --2- 172.21.15.16:0/65733 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f10ec06db20 0x7f10ec06ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:45.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.494+0000 7f11027fc700 1 --2- 172.21.15.16:0/65733 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f10ec06db20 0x7f10ec06ffe0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f10f800b800 tx=0x7f10f8009f70 comp rx=0 tx=0).ready entity=mgr.14744 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:45.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.499+0000 7f10ebfff700 1 -- 172.21.15.16:0/65733 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184299 (secure 0 0 0) 0x7f10f405ae70 con 0x7f11040ff9d0 2024-04-02T04:43:45.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.757+0000 7f110929f700 1 -- 172.21.15.16:0/65733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1104108530 con 0x7f11040ff9d0 2024-04-02T04:43:45.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.757+0000 7f10ebfff700 1 -- 172.21.15.16:0/65733 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1843 (secure 0 0 0) 0x7f10f405aa00 con 0x7f11040ff9d0 2024-04-02T04:43:45.761 INFO:teuthology.orchestra.run.smithi016.stdout:e15 2024-04-02T04:43:45.761 INFO:teuthology.orchestra.run.smithi016.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:43:45.761 INFO:teuthology.orchestra.run.smithi016.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:43:45.761 INFO:teuthology.orchestra.run.smithi016.stdout:legacy client fscid: 1 2024-04-02T04:43:45.761 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:43:45.761 INFO:teuthology.orchestra.run.smithi016.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:43:45.761 INFO:teuthology.orchestra.run.smithi016.stdout:fs_name cephfs 2024-04-02T04:43:45.761 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 15 2024-04-02T04:43:45.761 INFO:teuthology.orchestra.run.smithi016.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:43:45.761 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-04-02T04:39:30.740168+0000 2024-04-02T04:43:45.761 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-04-02T04:39:49.875319+0000 2024-04-02T04:43:45.761 INFO:teuthology.orchestra.run.smithi016.stdout:tableserver 0 2024-04-02T04:43:45.762 INFO:teuthology.orchestra.run.smithi016.stdout:root 0 2024-04-02T04:43:45.762 INFO:teuthology.orchestra.run.smithi016.stdout:session_timeout 60 2024-04-02T04:43:45.762 INFO:teuthology.orchestra.run.smithi016.stdout:session_autoclose 300 2024-04-02T04:43:45.762 INFO:teuthology.orchestra.run.smithi016.stdout:max_file_size 1099511627776 2024-04-02T04:43:45.762 INFO:teuthology.orchestra.run.smithi016.stdout:max_xattr_size 65536 2024-04-02T04:43:45.762 INFO:teuthology.orchestra.run.smithi016.stdout:required_client_features {} 2024-04-02T04:43:45.762 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure 0 2024-04-02T04:43:45.762 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure_osd_epoch 46 2024-04-02T04:43:45.762 INFO:teuthology.orchestra.run.smithi016.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:43:45.762 INFO:teuthology.orchestra.run.smithi016.stdout:max_mds 1 2024-04-02T04:43:45.762 INFO:teuthology.orchestra.run.smithi016.stdout:in 0 2024-04-02T04:43:45.762 INFO:teuthology.orchestra.run.smithi016.stdout:up {0=24285} 2024-04-02T04:43:45.762 INFO:teuthology.orchestra.run.smithi016.stdout:failed 2024-04-02T04:43:45.762 INFO:teuthology.orchestra.run.smithi016.stdout:damaged 2024-04-02T04:43:45.763 INFO:teuthology.orchestra.run.smithi016.stdout:stopped 2024-04-02T04:43:45.763 INFO:teuthology.orchestra.run.smithi016.stdout:data_pools [3] 2024-04-02T04:43:45.763 INFO:teuthology.orchestra.run.smithi016.stdout:metadata_pool 2 2024-04-02T04:43:45.763 INFO:teuthology.orchestra.run.smithi016.stdout:inline_data enabled 2024-04-02T04:43:45.763 INFO:teuthology.orchestra.run.smithi016.stdout:balancer 2024-04-02T04:43:45.763 INFO:teuthology.orchestra.run.smithi016.stdout:bal_rank_mask -1 2024-04-02T04:43:45.763 INFO:teuthology.orchestra.run.smithi016.stdout:standby_count_wanted 1 2024-04-02T04:43:45.763 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.zwrfoo{0:24285} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.27:6824/1461455976,v1:172.21.15.27:6825/1461455976] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:43:45.763 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:43:45.763 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:43:45.763 INFO:teuthology.orchestra.run.smithi016.stdout:Standby daemons: 2024-04-02T04:43:45.763 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:43:45.763 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.fyzrnm{-1:14488} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.16:6828/2578719859,v1:172.21.15.16:6829/2578719859] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:43:45.764 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.uhgkgg{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.16:6826/601263579,v1:172.21.15.16:6827/601263579] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:43:45.764 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.sdublg{-1:24295} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.27:6826/4177192740,v1:172.21.15.27:6827/4177192740] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:43:45.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.760+0000 7f110929f700 1 -- 172.21.15.16:0/65733 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f10ec06db20 msgr2=0x7f10ec06ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:45.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.760+0000 7f110929f700 1 --2- 172.21.15.16:0/65733 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f10ec06db20 0x7f10ec06ffe0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f10f800b800 tx=0x7f10f8009f70 comp rx=0 tx=0).stop 2024-04-02T04:43:45.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.760+0000 7f110929f700 1 -- 172.21.15.16:0/65733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11040ff9d0 msgr2=0x7f11041038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:45.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.760+0000 7f110929f700 1 --2- 172.21.15.16:0/65733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11040ff9d0 0x7f11041038c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f10f400e8f0 tx=0x7f10f400ec00 comp rx=0 tx=0).stop 2024-04-02T04:43:45.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.760+0000 7f110929f700 1 -- 172.21.15.16:0/65733 shutdown_connections 2024-04-02T04:43:45.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.760+0000 7f110929f700 1 --2- 172.21.15.16:0/65733 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f10ec06db20 0x7f10ec06ffe0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:45.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.760+0000 7f110929f700 1 --2- 172.21.15.16:0/65733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11040ff9d0 0x7f11041038c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:45.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.760+0000 7f110929f700 1 --2- 172.21.15.16:0/65733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1104100bd0 0x7f1104101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:45.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.760+0000 7f110929f700 1 -- 172.21.15.16:0/65733 >> 172.21.15.16:0/65733 conn(0x7f11040faf70 msgr2=0x7f11040fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:45.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.760+0000 7f110929f700 1 -- 172.21.15.16:0/65733 shutdown_connections 2024-04-02T04:43:45.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.760+0000 7f110929f700 1 -- 172.21.15.16:0/65733 wait complete. 2024-04-02T04:43:45.765 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 15 2024-04-02T04:43:45.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.872+0000 7facca699700 1 -- 172.21.15.16:0/3760073844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc40ff9f0 msgr2=0x7facc40ffe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:45.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.872+0000 7facca699700 1 --2- 172.21.15.16:0/3760073844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc40ff9f0 0x7facc40ffe10 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7facb4009910 tx=0x7facb4009c20 comp rx=0 tx=0).stop 2024-04-02T04:43:45.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.872+0000 7facca699700 1 -- 172.21.15.16:0/3760073844 shutdown_connections 2024-04-02T04:43:45.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.872+0000 7facca699700 1 --2- 172.21.15.16:0/3760073844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7facc4100bf0 0x7facc4101050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:45.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.872+0000 7facca699700 1 --2- 172.21.15.16:0/3760073844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc40ff9f0 0x7facc40ffe10 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:45.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.872+0000 7facca699700 1 -- 172.21.15.16:0/3760073844 >> 172.21.15.16:0/3760073844 conn(0x7facc40faf90 msgr2=0x7facc40fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:45.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.872+0000 7facca699700 1 -- 172.21.15.16:0/3760073844 shutdown_connections 2024-04-02T04:43:45.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.872+0000 7facca699700 1 -- 172.21.15.16:0/3760073844 wait complete. 2024-04-02T04:43:45.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.872+0000 7facca699700 1 Processor -- start 2024-04-02T04:43:45.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.873+0000 7facca699700 1 -- start start 2024-04-02T04:43:45.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.873+0000 7facca699700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7facc40ff9f0 0x7facc4069a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:45.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.873+0000 7facca699700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc4100bf0 0x7facc419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:45.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.873+0000 7facca699700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc419b870 con 0x7facc4100bf0 2024-04-02T04:43:45.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.873+0000 7facca699700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc419b9b0 con 0x7facc40ff9f0 2024-04-02T04:43:45.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.873+0000 7facc37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc4100bf0 0x7facc419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:45.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.873+0000 7facc37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc4100bf0 0x7facc419b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38364/0 (socket says 172.21.15.16:38364) 2024-04-02T04:43:45.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.873+0000 7facc37fe700 1 -- 172.21.15.16:0/3163664587 learned_addr learned my addr 172.21.15.16:0/3163664587 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:43:45.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.873+0000 7facc3fff700 1 --2- 172.21.15.16:0/3163664587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7facc40ff9f0 0x7facc4069a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:45.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.873+0000 7facc37fe700 1 -- 172.21.15.16:0/3163664587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7facc40ff9f0 msgr2=0x7facc4069a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:45.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.873+0000 7facc37fe700 1 --2- 172.21.15.16:0/3163664587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7facc40ff9f0 0x7facc4069a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:45.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.873+0000 7facc37fe700 1 -- 172.21.15.16:0/3163664587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb40095f0 con 0x7facc4100bf0 2024-04-02T04:43:45.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.873+0000 7facc37fe700 1 --2- 172.21.15.16:0/3163664587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc4100bf0 0x7facc419b330 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7facb800e920 tx=0x7facb800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:45.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.874+0000 7facc17fa700 1 -- 172.21.15.16:0/3163664587 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7facb800c9d0 con 0x7facc4100bf0 2024-04-02T04:43:45.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.874+0000 7facc17fa700 1 -- 172.21.15.16:0/3163664587 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7facb8010430 con 0x7facc4100bf0 2024-04-02T04:43:45.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.874+0000 7facca699700 1 -- 172.21.15.16:0/3163664587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facc419bc10 con 0x7facc4100bf0 2024-04-02T04:43:45.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.874+0000 7facc17fa700 1 -- 172.21.15.16:0/3163664587 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7facb8018630 con 0x7facc4100bf0 2024-04-02T04:43:45.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.874+0000 7facca699700 1 -- 172.21.15.16:0/3163664587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc419c1e0 con 0x7facc4100bf0 2024-04-02T04:43:45.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.875+0000 7facca699700 1 -- 172.21.15.16:0/3163664587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facc418f570 con 0x7facc4100bf0 2024-04-02T04:43:45.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.875+0000 7facc17fa700 1 -- 172.21.15.16:0/3163664587 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 31) v1 ==== 91816+0+0 (secure 0 0 0) 0x7facb80105a0 con 0x7facc4100bf0 2024-04-02T04:43:45.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.875+0000 7facc17fa700 1 --2- 172.21.15.16:0/3163664587 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7facac06db20 0x7facac06ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:45.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.876+0000 7facc17fa700 1 -- 172.21.15.16:0/3163664587 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7facb8014070 con 0x7facc4100bf0 2024-04-02T04:43:45.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.876+0000 7facc3fff700 1 --2- 172.21.15.16:0/3163664587 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7facac06db20 0x7facac06ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:45.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.876+0000 7facc3fff700 1 --2- 172.21.15.16:0/3163664587 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7facac06db20 0x7facac06ffe0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7facb400b800 tx=0x7facb4009f70 comp rx=0 tx=0).ready entity=mgr.14744 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:45.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:45.880+0000 7facc17fa700 1 -- 172.21.15.16:0/3163664587 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184299 (secure 0 0 0) 0x7facb8053000 con 0x7facc4100bf0 2024-04-02T04:43:46.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.114+0000 7facca699700 1 -- 172.21.15.16:0/3163664587 --> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7facc402ce10 con 0x7facac06db20 2024-04-02T04:43:46.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.115+0000 7facc17fa700 1 -- 172.21.15.16:0/3163664587 <== mgr.14744 v2:172.21.15.16:6800/2776599951 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7facc402ce10 con 0x7facac06db20 2024-04-02T04:43:46.118 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:43:46.118 INFO:teuthology.orchestra.run.smithi016.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:43:46.118 INFO:teuthology.orchestra.run.smithi016.stdout: "in_progress": true, 2024-04-02T04:43:46.118 INFO:teuthology.orchestra.run.smithi016.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-02T04:43:46.118 INFO:teuthology.orchestra.run.smithi016.stdout: "services_complete": [ 2024-04-02T04:43:46.118 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr" 2024-04-02T04:43:46.118 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-04-02T04:43:46.118 INFO:teuthology.orchestra.run.smithi016.stdout: "progress": "2/23 daemons upgraded", 2024-04-02T04:43:46.118 INFO:teuthology.orchestra.run.smithi016.stdout: "message": "Currently upgrading mgr daemons", 2024-04-02T04:43:46.118 INFO:teuthology.orchestra.run.smithi016.stdout: "is_paused": false 2024-04-02T04:43:46.118 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:43:46.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.117+0000 7facca699700 1 -- 172.21.15.16:0/3163664587 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7facac06db20 msgr2=0x7facac06ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:46.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.117+0000 7facca699700 1 --2- 172.21.15.16:0/3163664587 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7facac06db20 0x7facac06ffe0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7facb400b800 tx=0x7facb4009f70 comp rx=0 tx=0).stop 2024-04-02T04:43:46.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.117+0000 7facca699700 1 -- 172.21.15.16:0/3163664587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc4100bf0 msgr2=0x7facc419b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:46.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.117+0000 7facca699700 1 --2- 172.21.15.16:0/3163664587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc4100bf0 0x7facc419b330 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7facb800e920 tx=0x7facb800ec30 comp rx=0 tx=0).stop 2024-04-02T04:43:46.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.117+0000 7facca699700 1 -- 172.21.15.16:0/3163664587 shutdown_connections 2024-04-02T04:43:46.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.117+0000 7facca699700 1 --2- 172.21.15.16:0/3163664587 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7facac06db20 0x7facac06ffe0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:46.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.117+0000 7facca699700 1 --2- 172.21.15.16:0/3163664587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7facc40ff9f0 0x7facc4069a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:46.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.117+0000 7facca699700 1 --2- 172.21.15.16:0/3163664587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc4100bf0 0x7facc419b330 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:46.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.117+0000 7facca699700 1 -- 172.21.15.16:0/3163664587 >> 172.21.15.16:0/3163664587 conn(0x7facc40faf90 msgr2=0x7facc4104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:46.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.117+0000 7facca699700 1 -- 172.21.15.16:0/3163664587 shutdown_connections 2024-04-02T04:43:46.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.117+0000 7facca699700 1 -- 172.21.15.16:0/3163664587 wait complete. 2024-04-02T04:43:46.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.261+0000 7f90ba8b3700 1 -- 172.21.15.16:0/1103979299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90b4101d90 msgr2=0x7f90b41021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:46.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.261+0000 7f90ba8b3700 1 --2- 172.21.15.16:0/1103979299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90b4101d90 0x7f90b41021f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f90a8009910 tx=0x7f90a8009c20 comp rx=0 tx=0).stop 2024-04-02T04:43:46.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.261+0000 7f90ba8b3700 1 -- 172.21.15.16:0/1103979299 shutdown_connections 2024-04-02T04:43:46.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.261+0000 7f90ba8b3700 1 --2- 172.21.15.16:0/1103979299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90b4101d90 0x7f90b41021f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:46.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.261+0000 7f90ba8b3700 1 --2- 172.21.15.16:0/1103979299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90b40fedb0 0x7f90b40ff1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:46.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.261+0000 7f90ba8b3700 1 -- 172.21.15.16:0/1103979299 >> 172.21.15.16:0/1103979299 conn(0x7f90b40fa710 msgr2=0x7f90b40fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:46.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.261+0000 7f90ba8b3700 1 -- 172.21.15.16:0/1103979299 shutdown_connections 2024-04-02T04:43:46.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.261+0000 7f90ba8b3700 1 -- 172.21.15.16:0/1103979299 wait complete. 2024-04-02T04:43:46.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.261+0000 7f90ba8b3700 1 Processor -- start 2024-04-02T04:43:46.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.261+0000 7f90ba8b3700 1 -- start start 2024-04-02T04:43:46.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.262+0000 7f90ba8b3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90b40fedb0 0x7f90b4198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:46.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.262+0000 7f90ba8b3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90b4101d90 0x7f90b4198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:46.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.262+0000 7f90ba8b3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b4199270 con 0x7f90b4101d90 2024-04-02T04:43:46.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.262+0000 7f90ba8b3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b41993b0 con 0x7f90b40fedb0 2024-04-02T04:43:46.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.262+0000 7f90b37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90b4101d90 0x7f90b4198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:46.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.262+0000 7f90b37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90b4101d90 0x7f90b4198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38370/0 (socket says 172.21.15.16:38370) 2024-04-02T04:43:46.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.262+0000 7f90b3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90b40fedb0 0x7f90b4198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:46.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.262+0000 7f90b37fe700 1 -- 172.21.15.16:0/251457514 learned_addr learned my addr 172.21.15.16:0/251457514 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:43:46.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.262+0000 7f90b37fe700 1 -- 172.21.15.16:0/251457514 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90b40fedb0 msgr2=0x7f90b4198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:46.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.262+0000 7f90b37fe700 1 --2- 172.21.15.16:0/251457514 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90b40fedb0 0x7f90b4198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:46.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.262+0000 7f90b37fe700 1 -- 172.21.15.16:0/251457514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a80095f0 con 0x7f90b4101d90 2024-04-02T04:43:46.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.262+0000 7f90b37fe700 1 --2- 172.21.15.16:0/251457514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90b4101d90 0x7f90b4198c50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f90a800b6f0 tx=0x7f90a8005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:46.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.262+0000 7f90b17fa700 1 -- 172.21.15.16:0/251457514 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90a801d070 con 0x7f90b4101d90 2024-04-02T04:43:46.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.263+0000 7f90b17fa700 1 -- 172.21.15.16:0/251457514 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f90a80052a0 con 0x7f90b4101d90 2024-04-02T04:43:46.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.263+0000 7f90ba8b3700 1 -- 172.21.15.16:0/251457514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90b419ddb0 con 0x7f90b4101d90 2024-04-02T04:43:46.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.263+0000 7f90b17fa700 1 -- 172.21.15.16:0/251457514 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90a8022470 con 0x7f90b4101d90 2024-04-02T04:43:46.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.263+0000 7f90ba8b3700 1 -- 172.21.15.16:0/251457514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90b419e320 con 0x7f90b4101d90 2024-04-02T04:43:46.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.264+0000 7f90ba8b3700 1 -- 172.21.15.16:0/251457514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90b4106760 con 0x7f90b4101d90 2024-04-02T04:43:46.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.264+0000 7f90b17fa700 1 -- 172.21.15.16:0/251457514 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 31) v1 ==== 91816+0+0 (secure 0 0 0) 0x7f90a800f770 con 0x7f90b4101d90 2024-04-02T04:43:46.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.265+0000 7f90b17fa700 1 --2- 172.21.15.16:0/251457514 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f909c06db20 0x7f909c06ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:43:46.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.265+0000 7f90b17fa700 1 -- 172.21.15.16:0/251457514 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f90a808eb90 con 0x7f90b4101d90 2024-04-02T04:43:46.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.265+0000 7f90b3fff700 1 --2- 172.21.15.16:0/251457514 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f909c06db20 0x7f909c06ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:43:46.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.265+0000 7f90b3fff700 1 --2- 172.21.15.16:0/251457514 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f909c06db20 0x7f909c06ffe0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f90b4068e70 tx=0x7f90a4009260 comp rx=0 tx=0).ready entity=mgr.14744 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:43:46.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.269+0000 7f90b17fa700 1 -- 172.21.15.16:0/251457514 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184299 (secure 0 0 0) 0x7f90a805bf90 con 0x7f90b4101d90 2024-04-02T04:43:46.368 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:46 smithi027 ceph-mon[37550]: from='client.14806 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:43:46.368 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:46 smithi027 ceph-mon[37550]: pgmap v20: 65 pgs: 65 active+clean; 294 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 1.3 MiB/s wr, 94 op/s 2024-04-02T04:43:46.368 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:46 smithi027 ceph-mon[37550]: from='client.14810 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:43:46.369 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:46 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/2820874025' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:43:46.369 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:46 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/65733' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:43:46.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:46 smithi016 ceph-mon[28969]: from='client.14806 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:43:46.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:46 smithi016 ceph-mon[28969]: pgmap v20: 65 pgs: 65 active+clean; 294 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 1.3 MiB/s wr, 94 op/s 2024-04-02T04:43:46.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:46 smithi016 ceph-mon[28969]: from='client.14810 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:43:46.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:46 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/2820874025' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:43:46.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:46 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/65733' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:43:46.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.579+0000 7f90ba8b3700 1 -- 172.21.15.16:0/251457514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f90b404ead0 con 0x7f90b4101d90 2024-04-02T04:43:46.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.579+0000 7f90b17fa700 1 -- 172.21.15.16:0/251457514 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f90a805bb20 con 0x7f90b4101d90 2024-04-02T04:43:46.582 INFO:teuthology.orchestra.run.smithi016.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-02T04:43:46.582 INFO:teuthology.orchestra.run.smithi016.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-02T04:43:46.582 INFO:teuthology.orchestra.run.smithi016.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-02T04:43:46.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.581+0000 7f90ba8b3700 1 -- 172.21.15.16:0/251457514 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f909c06db20 msgr2=0x7f909c06ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:46.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.581+0000 7f90ba8b3700 1 --2- 172.21.15.16:0/251457514 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f909c06db20 0x7f909c06ffe0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f90b4068e70 tx=0x7f90a4009260 comp rx=0 tx=0).stop 2024-04-02T04:43:46.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.581+0000 7f90ba8b3700 1 -- 172.21.15.16:0/251457514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90b4101d90 msgr2=0x7f90b4198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:43:46.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.581+0000 7f90ba8b3700 1 --2- 172.21.15.16:0/251457514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90b4101d90 0x7f90b4198c50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f90a800b6f0 tx=0x7f90a8005e30 comp rx=0 tx=0).stop 2024-04-02T04:43:46.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.581+0000 7f90ba8b3700 1 -- 172.21.15.16:0/251457514 shutdown_connections 2024-04-02T04:43:46.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.581+0000 7f90ba8b3700 1 --2- 172.21.15.16:0/251457514 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f909c06db20 0x7f909c06ffe0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:46.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.581+0000 7f90ba8b3700 1 --2- 172.21.15.16:0/251457514 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90b40fedb0 0x7f90b4198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:46.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.581+0000 7f90ba8b3700 1 --2- 172.21.15.16:0/251457514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90b4101d90 0x7f90b4198c50 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:43:46.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.581+0000 7f90ba8b3700 1 -- 172.21.15.16:0/251457514 >> 172.21.15.16:0/251457514 conn(0x7f90b40fa710 msgr2=0x7f90b40fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:43:46.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.581+0000 7f90ba8b3700 1 -- 172.21.15.16:0/251457514 shutdown_connections 2024-04-02T04:43:46.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:43:46.581+0000 7f90ba8b3700 1 -- 172.21.15.16:0/251457514 wait complete. 2024-04-02T04:43:47.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:47 smithi016 ceph-mon[28969]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:43:47.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:47 smithi016 ceph-mon[28969]: from='client.? 172.21.15.16:0/251457514' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:43:47.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:47 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:47.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:47 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:47.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:47 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:43:47.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:47 smithi027 ceph-mon[37550]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:43:47.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:47 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/251457514' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:43:47.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:47 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:47.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:47 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:47.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:47 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:43:48.488 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:48 smithi027 ceph-mon[37550]: pgmap v21: 65 pgs: 65 active+clean; 295 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 40 KiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-04-02T04:43:48.488 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:48 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:48.488 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:48 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:48.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:48 smithi016 ceph-mon[28969]: pgmap v21: 65 pgs: 65 active+clean; 295 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 40 KiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-04-02T04:43:48.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:48 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:48.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:48 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:50.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:50 smithi027 ceph-mon[37550]: pgmap v22: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 1.9 MiB/s wr, 116 op/s 2024-04-02T04:43:50.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:50 smithi016 ceph-mon[28969]: pgmap v22: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 1.9 MiB/s wr, 116 op/s 2024-04-02T04:43:52.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:52 smithi027 ceph-mon[37550]: pgmap v23: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 1.3 MiB/s wr, 75 op/s 2024-04-02T04:43:52.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:52 smithi016 ceph-mon[28969]: pgmap v23: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 1.3 MiB/s wr, 75 op/s 2024-04-02T04:43:54.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:53 smithi027 ceph-mon[37550]: pgmap v24: 65 pgs: 65 active+clean; 301 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 41 KiB/s rd, 2.0 MiB/s wr, 112 op/s 2024-04-02T04:43:54.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:53 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:54.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:53 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:54.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:53 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:54.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:53 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:54.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:53 smithi016 ceph-mon[28969]: pgmap v24: 65 pgs: 65 active+clean; 301 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 41 KiB/s rd, 2.0 MiB/s wr, 112 op/s 2024-04-02T04:43:54.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:53 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:54.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:53 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:54.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:53 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:54.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:53 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:56.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:55 smithi016 ceph-mon[28969]: pgmap v25: 65 pgs: 65 active+clean; 301 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 1.3 MiB/s wr, 73 op/s 2024-04-02T04:43:56.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:55 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:56.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:55 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:56.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:55 smithi016 ceph-mon[28969]: Standby manager daemon smithi027.kjzcsc restarted 2024-04-02T04:43:56.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:55 smithi016 ceph-mon[28969]: Standby manager daemon smithi027.kjzcsc started 2024-04-02T04:43:56.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:55 smithi016 ceph-mon[28969]: from='mgr.? 172.21.15.27:0/828228889' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi027.kjzcsc/crt"}]: dispatch 2024-04-02T04:43:56.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:55 smithi016 ceph-mon[28969]: from='mgr.? 172.21.15.27:0/828228889' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T04:43:56.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:55 smithi016 ceph-mon[28969]: from='mgr.? 172.21.15.27:0/828228889' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi027.kjzcsc/key"}]: dispatch 2024-04-02T04:43:56.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:55 smithi016 ceph-mon[28969]: from='mgr.? 172.21.15.27:0/828228889' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T04:43:56.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:55 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:43:56.118 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:55 smithi027 ceph-mon[37550]: pgmap v25: 65 pgs: 65 active+clean; 301 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 1.3 MiB/s wr, 73 op/s 2024-04-02T04:43:56.118 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:55 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:56.118 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:55 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:43:56.118 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:55 smithi027 ceph-mon[37550]: Standby manager daemon smithi027.kjzcsc restarted 2024-04-02T04:43:56.118 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:55 smithi027 ceph-mon[37550]: Standby manager daemon smithi027.kjzcsc started 2024-04-02T04:43:56.119 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:55 smithi027 ceph-mon[37550]: from='mgr.? 172.21.15.27:0/828228889' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi027.kjzcsc/crt"}]: dispatch 2024-04-02T04:43:56.119 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:55 smithi027 ceph-mon[37550]: from='mgr.? 172.21.15.27:0/828228889' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T04:43:56.119 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:55 smithi027 ceph-mon[37550]: from='mgr.? 172.21.15.27:0/828228889' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi027.kjzcsc/key"}]: dispatch 2024-04-02T04:43:56.119 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:55 smithi027 ceph-mon[37550]: from='mgr.? 172.21.15.27:0/828228889' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T04:43:56.119 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:55 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:43:57.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:56 smithi027 ceph-mon[37550]: mgrmap e32: smithi016.eqnebx(active, since 45s), standbys: smithi027.kjzcsc 2024-04-02T04:43:57.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:56 smithi016 ceph-mon[28969]: mgrmap e32: smithi016.eqnebx(active, since 45s), standbys: smithi027.kjzcsc 2024-04-02T04:43:58.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:57 smithi027 ceph-mon[37550]: pgmap v26: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 42 KiB/s rd, 2.1 MiB/s wr, 97 op/s 2024-04-02T04:43:58.149 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:57 smithi016 ceph-mon[28969]: pgmap v26: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 42 KiB/s rd, 2.1 MiB/s wr, 97 op/s 2024-04-02T04:44:00.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:43:59 smithi027 ceph-mon[37550]: pgmap v27: 65 pgs: 65 active+clean; 302 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.9 MiB/s wr, 110 op/s 2024-04-02T04:44:00.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:43:59 smithi016 ceph-mon[28969]: pgmap v27: 65 pgs: 65 active+clean; 302 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.9 MiB/s wr, 110 op/s 2024-04-02T04:44:02.145 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:01 smithi027 ceph-mon[37550]: pgmap v28: 65 pgs: 65 active+clean; 302 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.5 MiB/s wr, 84 op/s 2024-04-02T04:44:02.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:01 smithi016 ceph-mon[28969]: pgmap v28: 65 pgs: 65 active+clean; 302 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.5 MiB/s wr, 84 op/s 2024-04-02T04:44:03.003 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:02 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:03.003 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:02 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:03.003 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:02 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:44:03.003 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:02 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:44:03.003 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:02 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:03.004 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:02 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:44:03.004 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:02 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:44:03.004 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:02 smithi016 ceph-mon[28969]: Upgrade: Setting container_image for all mgr 2024-04-02T04:44:03.004 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:02 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:03.004 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:02 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi016.eqnebx"}]: dispatch 2024-04-02T04:44:03.004 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:02 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi016.eqnebx"}]': finished 2024-04-02T04:44:03.004 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:02 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi027.kjzcsc"}]: dispatch 2024-04-02T04:44:03.004 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:02 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi027.kjzcsc"}]': finished 2024-04-02T04:44:03.004 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:02 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-02T04:44:03.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:02 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:03.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:02 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:03.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:02 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:44:03.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:02 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:44:03.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:02 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:03.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:02 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:44:03.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:02 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:44:03.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:02 smithi027 ceph-mon[37550]: Upgrade: Setting container_image for all mgr 2024-04-02T04:44:03.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:02 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:03.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:02 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi016.eqnebx"}]: dispatch 2024-04-02T04:44:03.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:02 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi016.eqnebx"}]': finished 2024-04-02T04:44:03.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:02 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi027.kjzcsc"}]: dispatch 2024-04-02T04:44:03.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:02 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi027.kjzcsc"}]': finished 2024-04-02T04:44:03.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:02 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-02T04:44:04.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:04 smithi027 ceph-mon[37550]: pgmap v29: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46 KiB/s rd, 2.2 MiB/s wr, 122 op/s 2024-04-02T04:44:04.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:04 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:04.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:04 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T04:44:04.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:04 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T04:44:04.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:04 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:44:04.466 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:04 smithi016 ceph-mon[28969]: pgmap v29: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46 KiB/s rd, 2.2 MiB/s wr, 122 op/s 2024-04-02T04:44:04.466 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:04 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:04.466 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:04 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T04:44:04.466 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:04 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T04:44:04.466 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:04 smithi016 ceph-mon[28969]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:44:05.395 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:05 smithi027 ceph-mon[37550]: Upgrade: Updating mon.smithi016 2024-04-02T04:44:05.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:05 smithi027 ceph-mon[37550]: Deploying daemon mon.smithi016 on smithi016 2024-04-02T04:44:05.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:05 smithi016 ceph-mon[28969]: Upgrade: Updating mon.smithi016 2024-04-02T04:44:05.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:05 smithi016 ceph-mon[28969]: Deploying daemon mon.smithi016 on smithi016 2024-04-02T04:44:06.156 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:05 smithi016 systemd[1]: Stopping Ceph mon.smithi016 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4... 2024-04-02T04:44:06.157 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:06 smithi016 ceph-mon[28969]: pgmap v30: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 31 KiB/s rd, 1.5 MiB/s wr, 86 op/s 2024-04-02T04:44:06.395 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:06 smithi027 ceph-mon[37550]: pgmap v30: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 31 KiB/s rd, 1.5 MiB/s wr, 86 op/s 2024-04-02T04:44:06.482 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:06 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi016[28965]: 2024-04-02T04:44:06.152+0000 7f05457d8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi016 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T04:44:06.483 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:06 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi016[28965]: 2024-04-02T04:44:06.152+0000 7f05457d8700 -1 mon.smithi016@0(leader) e2 *** Got Signal Terminated *** 2024-04-02T04:44:06.483 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:06 smithi016 podman[93053]: 2024-04-02 04:44:06.174311744 +0000 UTC m=+0.133634825 container died 24d5820dc9942c9a49ee6314cd7bbf0c255fdbdf7bc7e5004950ab521f7caaf4 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi016, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=reef-4d05f4c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD) 2024-04-02T04:44:06.809 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:06 smithi016 podman[93053]: 2024-04-02 04:44:06.671270221 +0000 UTC m=+0.630593319 container cleanup 24d5820dc9942c9a49ee6314cd7bbf0c255fdbdf7bc7e5004950ab521f7caaf4 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi016, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-4d05f4c, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T04:44:06.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:06 smithi016 bash[93053]: ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi016 2024-04-02T04:44:07.274 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:06 smithi016 podman[93068]: 2024-04-02 04:44:06.914666284 +0000 UTC m=+0.740327645 container remove 24d5820dc9942c9a49ee6314cd7bbf0c255fdbdf7bc7e5004950ab521f7caaf4 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi016, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=reef-4d05f4c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-02T04:44:07.274 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 systemd[1]: ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@mon.smithi016.service: Deactivated successfully. 2024-04-02T04:44:07.274 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 systemd[1]: Stopped Ceph mon.smithi016 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4. 2024-04-02T04:44:07.274 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 systemd[1]: ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@mon.smithi016.service: Consumed 11.991s CPU time. 2024-04-02T04:44:07.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 systemd[1]: Starting Ceph mon.smithi016 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4... 2024-04-02T04:44:07.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 podman[93212]: 2024-04-02 04:44:07.391374929 +0000 UTC m=+0.017844174 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:44:07.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 podman[93212]: 2024-04-02 04:44:07.509874888 +0000 UTC m=+0.136344161 container create eabfc84f1cdff5cd60b50713584c6364a79145f2998e417d49bd432ff2679bf6 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi016, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-6cec793, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS) 2024-04-02T04:44:08.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 podman[93212]: 2024-04-02 04:44:07.788091149 +0000 UTC m=+0.414560381 container init eabfc84f1cdff5cd60b50713584c6364a79145f2998e417d49bd432ff2679bf6 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi016, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-6cec793, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-02T04:44:08.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 podman[93212]: 2024-04-02 04:44:07.792537835 +0000 UTC m=+0.419007051 container start eabfc84f1cdff5cd60b50713584c6364a79145f2998e417d49bd432ff2679bf6 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi016, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-6cec793, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-02T04:44:08.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: set uid:gid to 167:167 (ceph:ceph) 2024-04-02T04:44:08.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev), process ceph-mon, pid 2 2024-04-02T04:44:08.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: pidfile_write: ignore empty --pid-file 2024-04-02T04:44:08.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: load: jerasure load: lrc 2024-04-02T04:44:08.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: RocksDB version: 7.9.2 2024-04-02T04:44:08.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Git sha 0 2024-04-02T04:44:08.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Compile date 2024-04-01 15:23:26 2024-04-02T04:44:08.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: DB SUMMARY 2024-04-02T04:44:08.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: DB Session ID: 87EWKSD92JTXJZB353SH 2024-04-02T04:44:08.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: CURRENT file: CURRENT 2024-04-02T04:44:08.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: IDENTITY file: IDENTITY 2024-04-02T04:44:08.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1510 Bytes 2024-04-02T04:44:08.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi016/store.db dir, Total Num: 1, files: 000032.sst 2024-04-02T04:44:08.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi016/store.db: 000030.log size: 1363785 ; 2024-04-02T04:44:08.062 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.error_if_exists: 0 2024-04-02T04:44:08.062 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.create_if_missing: 0 2024-04-02T04:44:08.062 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.paranoid_checks: 1 2024-04-02T04:44:08.062 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-02T04:44:08.062 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-02T04:44:08.062 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-02T04:44:08.062 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.env: 0x557910378c60 2024-04-02T04:44:08.062 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.fs: PosixFileSystem 2024-04-02T04:44:08.062 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.info_log: 0x5579111739a0 2024-04-02T04:44:08.062 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.max_file_opening_threads: 16 2024-04-02T04:44:08.062 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.statistics: (nil) 2024-04-02T04:44:08.062 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.use_fsync: 0 2024-04-02T04:44:08.062 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.max_log_file_size: 0 2024-04-02T04:44:08.063 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-02T04:44:08.063 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-02T04:44:08.063 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.keep_log_file_num: 1000 2024-04-02T04:44:08.063 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.recycle_log_file_num: 0 2024-04-02T04:44:08.063 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.allow_fallocate: 1 2024-04-02T04:44:08.063 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.allow_mmap_reads: 0 2024-04-02T04:44:08.063 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.allow_mmap_writes: 0 2024-04-02T04:44:08.063 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.use_direct_reads: 0 2024-04-02T04:44:08.063 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-02T04:44:08.063 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.create_missing_column_families: 0 2024-04-02T04:44:08.063 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.db_log_dir: 2024-04-02T04:44:08.063 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.wal_dir: 2024-04-02T04:44:08.063 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-02T04:44:08.063 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-02T04:44:08.063 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-02T04:44:08.064 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-02T04:44:08.064 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-02T04:44:08.064 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-02T04:44:08.064 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.advise_random_on_open: 1 2024-04-02T04:44:08.064 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.db_write_buffer_size: 0 2024-04-02T04:44:08.064 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.write_buffer_manager: 0x5579111461e0 2024-04-02T04:44:08.064 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-02T04:44:08.064 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-02T04:44:08.064 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-02T04:44:08.064 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.rate_limiter: (nil) 2024-04-02T04:44:08.064 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-02T04:44:08.064 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.wal_recovery_mode: 2 2024-04-02T04:44:08.064 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.enable_thread_tracking: 0 2024-04-02T04:44:08.064 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.enable_pipelined_write: 0 2024-04-02T04:44:08.064 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.unordered_write: 0 2024-04-02T04:44:08.065 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-02T04:44:08.065 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-02T04:44:08.065 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-02T04:44:08.065 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-02T04:44:08.065 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.row_cache: None 2024-04-02T04:44:08.065 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.wal_filter: None 2024-04-02T04:44:08.066 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-02T04:44:08.066 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.allow_ingest_behind: 0 2024-04-02T04:44:08.066 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.two_write_queues: 0 2024-04-02T04:44:08.066 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.manual_wal_flush: 0 2024-04-02T04:44:08.067 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.wal_compression: 0 2024-04-02T04:44:08.067 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.atomic_flush: 0 2024-04-02T04:44:08.067 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-02T04:44:08.067 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-02T04:44:08.067 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-02T04:44:08.067 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.log_readahead_size: 0 2024-04-02T04:44:08.067 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-02T04:44:08.067 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.best_efforts_recovery: 0 2024-04-02T04:44:08.067 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-02T04:44:08.067 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-02T04:44:08.067 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.allow_data_in_errors: 0 2024-04-02T04:44:08.067 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.db_host_id: __hostname__ 2024-04-02T04:44:08.067 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-02T04:44:08.068 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.max_background_jobs: 2 2024-04-02T04:44:08.068 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.max_background_compactions: -1 2024-04-02T04:44:08.068 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.max_subcompactions: 1 2024-04-02T04:44:08.068 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-02T04:44:08.068 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-02T04:44:08.068 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-02T04:44:08.068 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.max_total_wal_size: 0 2024-04-02T04:44:08.068 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-02T04:44:08.068 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-02T04:44:08.068 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-02T04:44:08.068 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-02T04:44:08.068 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.max_open_files: -1 2024-04-02T04:44:08.068 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.bytes_per_sync: 0 2024-04-02T04:44:08.068 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-02T04:44:08.069 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-02T04:44:08.069 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.compaction_readahead_size: 0 2024-04-02T04:44:08.069 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.max_background_flushes: -1 2024-04-02T04:44:08.069 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Compression algorithms supported: 2024-04-02T04:44:08.069 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-02T04:44:08.069 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: kZSTD supported: 0 2024-04-02T04:44:08.069 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: kXpressCompression supported: 0 2024-04-02T04:44:08.069 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: kLZ4HCCompression supported: 1 2024-04-02T04:44:08.069 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: kZlibCompression supported: 1 2024-04-02T04:44:08.069 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: kSnappyCompression supported: 1 2024-04-02T04:44:08.069 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: kLZ4Compression supported: 1 2024-04-02T04:44:08.069 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: kBZip2Compression supported: 0 2024-04-02T04:44:08.069 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-02T04:44:08.069 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-02T04:44:08.070 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi016/store.db/MANIFEST-000015 2024-04-02T04:44:08.070 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-02T04:44:08.070 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-02T04:44:08.070 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.merge_operator: 2024-04-02T04:44:08.070 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.compaction_filter: None 2024-04-02T04:44:08.070 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.compaction_filter_factory: None 2024-04-02T04:44:08.070 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.sst_partitioner_factory: None 2024-04-02T04:44:08.070 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-02T04:44:08.070 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-02T04:44:08.070 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557911173aa0) 2024-04-02T04:44:08.070 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: cache_index_and_filter_blocks: 1 2024-04-02T04:44:08.070 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-02T04:44:08.070 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-02T04:44:08.070 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: pin_top_level_index_and_filter: 1 2024-04-02T04:44:08.071 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: index_type: 0 2024-04-02T04:44:08.071 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: data_block_index_type: 0 2024-04-02T04:44:08.071 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: index_shortening: 1 2024-04-02T04:44:08.071 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-02T04:44:08.071 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: checksum: 4 2024-04-02T04:44:08.071 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: no_block_cache: 0 2024-04-02T04:44:08.071 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: block_cache: 0x557911177090 2024-04-02T04:44:08.071 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: block_cache_name: BinnedLRUCache 2024-04-02T04:44:08.071 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: block_cache_options: 2024-04-02T04:44:08.071 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: capacity : 536870912 2024-04-02T04:44:08.071 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: num_shard_bits : 4 2024-04-02T04:44:08.071 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: strict_capacity_limit : 0 2024-04-02T04:44:08.072 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: high_pri_pool_ratio: 0.000 2024-04-02T04:44:08.072 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: block_cache_compressed: (nil) 2024-04-02T04:44:08.072 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: persistent_cache: (nil) 2024-04-02T04:44:08.072 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: block_size: 4096 2024-04-02T04:44:08.072 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: block_size_deviation: 10 2024-04-02T04:44:08.072 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: block_restart_interval: 16 2024-04-02T04:44:08.072 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: index_block_restart_interval: 1 2024-04-02T04:44:08.072 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: metadata_block_size: 4096 2024-04-02T04:44:08.072 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: partition_filters: 0 2024-04-02T04:44:08.072 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: use_delta_encoding: 1 2024-04-02T04:44:08.072 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: filter_policy: bloomfilter 2024-04-02T04:44:08.072 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: whole_key_filtering: 1 2024-04-02T04:44:08.072 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: verify_compression: 0 2024-04-02T04:44:08.072 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: read_amp_bytes_per_bit: 0 2024-04-02T04:44:08.072 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: format_version: 5 2024-04-02T04:44:08.073 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: enable_index_compression: 1 2024-04-02T04:44:08.073 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: block_align: 0 2024-04-02T04:44:08.073 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: max_auto_readahead_size: 262144 2024-04-02T04:44:08.073 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: prepopulate_block_cache: 0 2024-04-02T04:44:08.073 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: initial_auto_readahead_size: 8192 2024-04-02T04:44:08.073 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: num_file_reads_for_auto_readahead: 2 2024-04-02T04:44:08.073 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.write_buffer_size: 33554432 2024-04-02T04:44:08.073 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.max_write_buffer_number: 2 2024-04-02T04:44:08.073 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.compression: NoCompression 2024-04-02T04:44:08.073 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.bottommost_compression: Disabled 2024-04-02T04:44:08.073 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.prefix_extractor: nullptr 2024-04-02T04:44:08.073 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-02T04:44:08.073 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.num_levels: 7 2024-04-02T04:44:08.073 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-02T04:44:08.073 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-02T04:44:08.074 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-02T04:44:08.074 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-02T04:44:08.074 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-02T04:44:08.074 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-02T04:44:08.074 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-02T04:44:08.074 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-02T04:44:08.074 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-02T04:44:08.074 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-02T04:44:08.074 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-02T04:44:08.074 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-02T04:44:08.074 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-02T04:44:08.074 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.compression_opts.level: 32767 2024-04-02T04:44:08.074 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.compression_opts.strategy: 0 2024-04-02T04:44:08.074 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-02T04:44:08.075 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-02T04:44:08.075 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-02T04:44:08.075 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-02T04:44:08.075 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.compression_opts.enabled: false 2024-04-02T04:44:08.075 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-02T04:44:08.075 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-02T04:44:08.075 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-02T04:44:08.075 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-02T04:44:08.075 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.target_file_size_base: 67108864 2024-04-02T04:44:08.075 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-02T04:44:08.075 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-02T04:44:08.075 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-02T04:44:08.075 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-02T04:44:08.075 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-02T04:44:08.075 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-02T04:44:08.076 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-02T04:44:08.076 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-02T04:44:08.076 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-02T04:44:08.076 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-02T04:44:08.076 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-02T04:44:08.076 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-02T04:44:08.076 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-02T04:44:08.076 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-02T04:44:08.076 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.arena_block_size: 1048576 2024-04-02T04:44:08.076 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-02T04:44:08.076 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-02T04:44:08.076 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.disable_auto_compactions: 0 2024-04-02T04:44:08.076 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-02T04:44:08.076 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-02T04:44:08.076 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-02T04:44:08.077 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-02T04:44:08.077 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-02T04:44:08.077 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-02T04:44:08.077 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-02T04:44:08.077 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-02T04:44:08.077 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-02T04:44:08.077 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-02T04:44:08.077 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.table_properties_collectors: 2024-04-02T04:44:08.077 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.inplace_update_support: 0 2024-04-02T04:44:08.077 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-02T04:44:08.077 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-02T04:44:08.077 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-02T04:44:08.077 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-02T04:44:08.077 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.bloom_locality: 0 2024-04-02T04:44:08.078 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.max_successive_merges: 0 2024-04-02T04:44:08.078 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-02T04:44:08.078 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.paranoid_file_checks: 0 2024-04-02T04:44:08.078 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.force_consistency_checks: 1 2024-04-02T04:44:08.078 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.report_bg_io_stats: 0 2024-04-02T04:44:08.078 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.ttl: 2592000 2024-04-02T04:44:08.078 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-02T04:44:08.078 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-02T04:44:08.078 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-02T04:44:08.078 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.enable_blob_files: false 2024-04-02T04:44:08.078 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.min_blob_size: 0 2024-04-02T04:44:08.078 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.blob_file_size: 268435456 2024-04-02T04:44:08.078 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-02T04:44:08.079 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-02T04:44:08.079 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-02T04:44:08.079 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-02T04:44:08.079 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-02T04:44:08.079 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.blob_file_starting_level: 0 2024-04-02T04:44:08.079 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-02T04:44:08.079 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-02T04:44:08.079 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi016/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 34, last_sequence is 12092, log_number is 30,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 30 2024-04-02T04:44:08.079 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 30 2024-04-02T04:44:08.079 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 05c51ba7-789d-46bb-92d6-8d939af0a557 2024-04-02T04:44:08.079 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712033047822738, "job": 1, "event": "recovery_started", "wal_files": [30]} 2024-04-02T04:44:08.079 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #30 mode 2 2024-04-02T04:44:08.079 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712033047827701, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 35, "file_size": 1225559, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12093, "largest_seqno": 12905, "table_properties": {"data_size": 1221341, "index_size": 2308, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 901, "raw_key_size": 7763, "raw_average_key_size": 23, "raw_value_size": 1213958, "raw_average_value_size": 3612, "num_data_blocks": 105, "num_entries": 336, "num_filter_entries": 336, "num_deletions": 2, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1712033047, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "05c51ba7-789d-46bb-92d6-8d939af0a557", "db_session_id": "87EWKSD92JTXJZB353SH", "orig_file_number": 35, "seqno_to_time_mapping": "N/A"}} 2024-04-02T04:44:08.080 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712033047827801, "job": 1, "event": "recovery_finished"} 2024-04-02T04:44:08.080 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: [db/version_set.cc:5047] Creating manifest 37 2024-04-02T04:44:08.080 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-02T04:44:08.080 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi016/store.db/000030.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-02T04:44:08.080 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x557911268000 2024-04-02T04:44:08.080 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: DB pointer 0x557911252000 2024-04-02T04:44:08.080 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-02T04:44:08.080 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-02T04:44:08.080 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: ** DB Stats ** 2024-04-02T04:44:08.080 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-02T04:44:08.080 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-02T04:44:08.080 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-02T04:44:08.080 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-02T04:44:08.080 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-02T04:44:08.081 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-02T04:44:08.081 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-02T04:44:08.081 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: 2024-04-02T04:44:08.081 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: ** Compaction Stats [default] ** 2024-04-02T04:44:08.081 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-02T04:44:08.081 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-02T04:44:08.081 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: L0 1/0 1.17 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 431.3 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-04-02T04:44:08.081 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: L6 1/0 5.63 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-04-02T04:44:08.081 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Sum 2/0 6.80 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 431.3 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-04-02T04:44:08.081 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 431.3 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-04-02T04:44:08.081 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: 2024-04-02T04:44:08.081 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: ** Compaction Stats [default] ** 2024-04-02T04:44:08.081 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-02T04:44:08.081 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-02T04:44:08.082 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 431.3 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-04-02T04:44:08.082 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: 2024-04-02T04:44:08.082 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-02T04:44:08.082 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: 2024-04-02T04:44:08.082 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-02T04:44:08.082 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Flush(GB): cumulative 0.001, interval 0.001 2024-04-02T04:44:08.082 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-02T04:44:08.082 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-02T04:44:08.082 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-02T04:44:08.082 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-02T04:44:08.082 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Cumulative compaction: 0.00 GB write, 155.09 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-02T04:44:08.082 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Interval compaction: 0.00 GB write, 155.09 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-02T04:44:08.082 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-02T04:44:08.083 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Block cache BinnedLRUCache@0x557911177090#2 capacity: 512.00 MB usage: 3.61 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-02T04:44:08.083 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.03 KB,0.000196695%) IndexBlock(1,2.58 KB,0.000491738%) Misc(1,0.00 KB,0%) 2024-04-02T04:44:08.083 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: 2024-04-02T04:44:08.083 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-02T04:44:08.083 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: starting mon.smithi016 rank 0 at public addrs [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] at bind addrs [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi016 fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 2024-04-02T04:44:08.083 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: mon.smithi016@-1(???) e2 preinit fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 2024-04-02T04:44:08.083 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: mon.smithi016@-1(???).mds e15 new map 2024-04-02T04:44:08.083 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: mon.smithi016@-1(???).mds e15 print_map 2024-04-02T04:44:08.083 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: e15 2024-04-02T04:44:08.083 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:44:08.083 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:44:08.083 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: legacy client fscid: 1 2024-04-02T04:44:08.083 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: 2024-04-02T04:44:08.084 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Filesystem 'cephfs' (1) 2024-04-02T04:44:08.084 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: fs_name cephfs 2024-04-02T04:44:08.084 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: epoch 15 2024-04-02T04:44:08.084 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:44:08.084 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: created 2024-04-02T04:39:30.740168+0000 2024-04-02T04:44:08.084 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: modified 2024-04-02T04:39:49.875319+0000 2024-04-02T04:44:08.084 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: tableserver 0 2024-04-02T04:44:08.084 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: root 0 2024-04-02T04:44:08.084 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: session_timeout 60 2024-04-02T04:44:08.084 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: session_autoclose 300 2024-04-02T04:44:08.084 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: max_file_size 1099511627776 2024-04-02T04:44:08.084 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: max_xattr_size 65536 2024-04-02T04:44:08.084 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: required_client_features {} 2024-04-02T04:44:08.084 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: last_failure 0 2024-04-02T04:44:08.084 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: last_failure_osd_epoch 46 2024-04-02T04:44:08.085 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:44:08.085 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: max_mds 1 2024-04-02T04:44:08.085 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: in 0 2024-04-02T04:44:08.085 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: up {0=24285} 2024-04-02T04:44:08.085 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: failed 2024-04-02T04:44:08.085 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: damaged 2024-04-02T04:44:08.085 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: stopped 2024-04-02T04:44:08.085 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: data_pools [3] 2024-04-02T04:44:08.085 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: metadata_pool 2 2024-04-02T04:44:08.085 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: inline_data enabled 2024-04-02T04:44:08.085 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: balancer 2024-04-02T04:44:08.085 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: bal_rank_mask -1 2024-04-02T04:44:08.085 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: standby_count_wanted 1 2024-04-02T04:44:08.085 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: qdb_cluster leader: 0 members: 2024-04-02T04:44:08.085 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: [mds.cephfs.smithi027.zwrfoo{0:24285} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.27:6824/1461455976,v1:172.21.15.27:6825/1461455976] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:44:08.086 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: 2024-04-02T04:44:08.086 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: 2024-04-02T04:44:08.086 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: Standby daemons: 2024-04-02T04:44:08.086 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: 2024-04-02T04:44:08.086 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: [mds.cephfs.smithi016.fyzrnm{-1:14488} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.16:6828/2578719859,v1:172.21.15.16:6829/2578719859] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:44:08.086 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: [mds.cephfs.smithi016.uhgkgg{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.16:6826/601263579,v1:172.21.15.16:6827/601263579] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:44:08.086 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout: [mds.cephfs.smithi027.sdublg{-1:24295} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.27:6826/4177192740,v1:172.21.15.27:6827/4177192740] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:44:08.086 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: mon.smithi016@-1(???).osd e48 crush map has features 3314933000852226048, adjusting msgr requires 2024-04-02T04:44:08.086 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: mon.smithi016@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-02T04:44:08.086 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: mon.smithi016@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-02T04:44:08.086 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: mon.smithi016@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-02T04:44:08.086 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 ceph-mon[93230]: mon.smithi016@-1(???).paxosservice(auth 1..22) refresh upgraded, format 0 -> 3 2024-04-02T04:44:08.086 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 bash[93212]: eabfc84f1cdff5cd60b50713584c6364a79145f2998e417d49bd432ff2679bf6 2024-04-02T04:44:08.086 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:07 smithi016 systemd[1]: Started Ceph mon.smithi016 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4. 2024-04-02T04:44:09.351 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:09 smithi016 ceph-mon[93230]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi016"}]: dispatch 2024-04-02T04:44:09.351 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:09 smithi016 ceph-mon[93230]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2024-04-02T04:44:09.351 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:09 smithi016 ceph-mon[93230]: pgmap v31: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 47 KiB/s rd, 1.9 MiB/s wr, 101 op/s 2024-04-02T04:44:09.351 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:09 smithi016 ceph-mon[93230]: mon.smithi016 calling monitor election 2024-04-02T04:44:09.351 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:09 smithi016 ceph-mon[93230]: mon.smithi016 is new leader, mons smithi016,smithi027 in quorum (ranks 0,1) 2024-04-02T04:44:09.351 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:09 smithi016 ceph-mon[93230]: monmap epoch 2 2024-04-02T04:44:09.352 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:09 smithi016 ceph-mon[93230]: fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 2024-04-02T04:44:09.352 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:09 smithi016 ceph-mon[93230]: last_changed 2024-04-02T04:35:14.456167+0000 2024-04-02T04:44:09.352 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:09 smithi016 ceph-mon[93230]: created 2024-04-02T04:32:20.006984+0000 2024-04-02T04:44:09.352 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:09 smithi016 ceph-mon[93230]: min_mon_release 18 (reef) 2024-04-02T04:44:09.352 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:09 smithi016 ceph-mon[93230]: election_strategy: 1 2024-04-02T04:44:09.352 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:09 smithi016 ceph-mon[93230]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.smithi016 2024-04-02T04:44:09.352 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:09 smithi016 ceph-mon[93230]: 1: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.smithi027 2024-04-02T04:44:09.352 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:09 smithi016 ceph-mon[93230]: fsmap cephfs:1 {0=cephfs.smithi027.zwrfoo=up:active} 3 up:standby 2024-04-02T04:44:09.352 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:09 smithi016 ceph-mon[93230]: osdmap e48: 6 total, 6 up, 6 in 2024-04-02T04:44:09.352 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:09 smithi016 ceph-mon[93230]: mgrmap e32: smithi016.eqnebx(active, since 58s), standbys: smithi027.kjzcsc 2024-04-02T04:44:09.352 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:09 smithi016 ceph-mon[93230]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-02T04:44:09.352 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:09 smithi016 ceph-mon[93230]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-02T04:44:09.352 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:09 smithi016 ceph-mon[93230]: fs cephfs has deprecated feature inline_data enabled. 2024-04-02T04:44:09.352 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:09 smithi016 ceph-mon[93230]: from='mgr.14744 ' entity='' 2024-04-02T04:44:09.352 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:09 smithi016 ceph-mon[93230]: mgrmap e33: smithi016.eqnebx(active, since 58s), standbys: smithi027.kjzcsc 2024-04-02T04:44:09.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:09 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi016"}]: dispatch 2024-04-02T04:44:09.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:09 smithi027 ceph-mon[37550]: from='mgr.14744 172.21.15.16:0/982152607' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2024-04-02T04:44:09.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:09 smithi027 ceph-mon[37550]: pgmap v31: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 47 KiB/s rd, 1.9 MiB/s wr, 101 op/s 2024-04-02T04:44:09.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:09 smithi027 ceph-mon[37550]: mon.smithi016 calling monitor election 2024-04-02T04:44:09.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:09 smithi027 ceph-mon[37550]: mon.smithi016 is new leader, mons smithi016,smithi027 in quorum (ranks 0,1) 2024-04-02T04:44:09.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:09 smithi027 ceph-mon[37550]: monmap epoch 2 2024-04-02T04:44:09.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:09 smithi027 ceph-mon[37550]: fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 2024-04-02T04:44:09.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:09 smithi027 ceph-mon[37550]: last_changed 2024-04-02T04:35:14.456167+0000 2024-04-02T04:44:09.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:09 smithi027 ceph-mon[37550]: created 2024-04-02T04:32:20.006984+0000 2024-04-02T04:44:09.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:09 smithi027 ceph-mon[37550]: min_mon_release 18 (reef) 2024-04-02T04:44:09.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:09 smithi027 ceph-mon[37550]: election_strategy: 1 2024-04-02T04:44:09.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:09 smithi027 ceph-mon[37550]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.smithi016 2024-04-02T04:44:09.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:09 smithi027 ceph-mon[37550]: 1: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.smithi027 2024-04-02T04:44:09.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:09 smithi027 ceph-mon[37550]: fsmap cephfs:1 {0=cephfs.smithi027.zwrfoo=up:active} 3 up:standby 2024-04-02T04:44:09.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:09 smithi027 ceph-mon[37550]: osdmap e48: 6 total, 6 up, 6 in 2024-04-02T04:44:09.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:09 smithi027 ceph-mon[37550]: mgrmap e32: smithi016.eqnebx(active, since 58s), standbys: smithi027.kjzcsc 2024-04-02T04:44:09.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:09 smithi027 ceph-mon[37550]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-02T04:44:09.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:09 smithi027 ceph-mon[37550]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-02T04:44:09.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:09 smithi027 ceph-mon[37550]: fs cephfs has deprecated feature inline_data enabled. 2024-04-02T04:44:09.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:09 smithi027 ceph-mon[37550]: from='mgr.14744 ' entity='' 2024-04-02T04:44:09.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:09 smithi027 ceph-mon[37550]: mgrmap e33: smithi016.eqnebx(active, since 58s), standbys: smithi027.kjzcsc 2024-04-02T04:44:16.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:16.705+0000 7f2435252700 1 -- 172.21.15.16:0/678550105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2430100ad0 msgr2=0x7f2430100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:16.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:16.705+0000 7f2435252700 1 --2- 172.21.15.16:0/678550105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2430100ad0 0x7f2430100ef0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2420009910 tx=0x7f2420009c20 comp rx=0 tx=0).stop 2024-04-02T04:44:16.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:16.705+0000 7f2435252700 1 -- 172.21.15.16:0/678550105 shutdown_connections 2024-04-02T04:44:16.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:16.705+0000 7f2435252700 1 --2- 172.21.15.16:0/678550105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2430101cd0 0x7f2430102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:16.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:16.705+0000 7f2435252700 1 --2- 172.21.15.16:0/678550105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2430100ad0 0x7f2430100ef0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:16.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:16.705+0000 7f2435252700 1 -- 172.21.15.16:0/678550105 >> 172.21.15.16:0/678550105 conn(0x7f24300fc090 msgr2=0x7f24300fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:16.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:16.705+0000 7f2435252700 1 -- 172.21.15.16:0/678550105 shutdown_connections 2024-04-02T04:44:16.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:16.705+0000 7f2435252700 1 -- 172.21.15.16:0/678550105 wait complete. 2024-04-02T04:44:16.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:16.706+0000 7f2435252700 1 Processor -- start 2024-04-02T04:44:16.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:16.706+0000 7f2435252700 1 -- start start 2024-04-02T04:44:16.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:16.706+0000 7f2435252700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2430100ad0 0x7f2430074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:16.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:16.706+0000 7f2435252700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2430101cd0 0x7f24300731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:16.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:16.706+0000 7f2435252700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24300736e0 con 0x7f2430101cd0 2024-04-02T04:44:16.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:16.706+0000 7f2435252700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2430073820 con 0x7f2430100ad0 2024-04-02T04:44:16.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:16.706+0000 7f242ed9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2430100ad0 0x7f2430074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:16.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:16.706+0000 7f242ed9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2430100ad0 0x7f2430074b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.16:45808/0 (socket says 172.21.15.16:45808) 2024-04-02T04:44:16.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:16.706+0000 7f242e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2430101cd0 0x7f24300731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:16.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:16.706+0000 7f242ed9d700 1 -- 172.21.15.16:0/3066276005 learned_addr learned my addr 172.21.15.16:0/3066276005 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:44:16.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:16.707+0000 7f242ed9d700 1 -- 172.21.15.16:0/3066276005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2430101cd0 msgr2=0x7f24300731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:16.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:16.707+0000 7f242ed9d700 1 --2- 172.21.15.16:0/3066276005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2430101cd0 0x7f24300731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:16.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:16.707+0000 7f242ed9d700 1 -- 172.21.15.16:0/3066276005 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24200095f0 con 0x7f2430100ad0 2024-04-02T04:44:16.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:16.707+0000 7f242ed9d700 1 --2- 172.21.15.16:0/3066276005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2430100ad0 0x7f2430074b00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f24200059d0 tx=0x7f242000bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:16.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:16.707+0000 7f2427fff700 1 -- 172.21.15.16:0/3066276005 <== mon.1 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f242001d070 con 0x7f2430100ad0 2024-04-02T04:44:16.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:16.707+0000 7f2435252700 1 -- 172.21.15.16:0/3066276005 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2430073aa0 con 0x7f2430100ad0 2024-04-02T04:44:16.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:16.707+0000 7f2435252700 1 -- 172.21.15.16:0/3066276005 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2430074010 con 0x7f2430100ad0 2024-04-02T04:44:16.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:16.707+0000 7f2427fff700 1 -- 172.21.15.16:0/3066276005 <== mon.1 v2:172.21.15.27:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f242000f460 con 0x7f2430100ad0 2024-04-02T04:44:16.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:16.707+0000 7f2427fff700 1 -- 172.21.15.16:0/3066276005 <== mon.1 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2420021640 con 0x7f2430100ad0 2024-04-02T04:44:16.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:16.708+0000 7f2435252700 1 -- 172.21.15.16:0/3066276005 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2430190650 con 0x7f2430100ad0 2024-04-02T04:44:16.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:16.709+0000 7f2427fff700 1 -- 172.21.15.16:0/3066276005 <== mon.1 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 33) v1 ==== 91955+0+0 (secure 0 0 0) 0x7f242000f5d0 con 0x7f2430100ad0 2024-04-02T04:44:16.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:16.709+0000 7f2427fff700 1 --2- 172.21.15.16:0/3066276005 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f241c06dea0 0x7f241c070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:16.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:16.709+0000 7f242e59c700 1 -- 172.21.15.16:0/3066276005 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f241c06dea0 msgr2=0x7f241c070360 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/2776599951 2024-04-02T04:44:16.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:16.709+0000 7f2427fff700 1 -- 172.21.15.16:0/3066276005 <== mon.1 v2:172.21.15.27:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f242008ecf0 con 0x7f2430100ad0 2024-04-02T04:44:16.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:16.710+0000 7f242e59c700 1 --2- 172.21.15.16:0/3066276005 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f241c06dea0 0x7f241c070360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T04:44:16.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:16.714+0000 7f2427fff700 1 -- 172.21.15.16:0/3066276005 <== mon.1 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f242005bd70 con 0x7f2430100ad0 2024-04-02T04:44:16.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:16.911+0000 7f242e59c700 1 -- 172.21.15.16:0/3066276005 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f241c06dea0 msgr2=0x7f241c070360 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/2776599951 2024-04-02T04:44:16.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:16.911+0000 7f242e59c700 1 --2- 172.21.15.16:0/3066276005 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f241c06dea0 0x7f241c070360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T04:44:16.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:16.951+0000 7f2435252700 1 -- 172.21.15.16:0/3066276005 --> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2430074010 con 0x7f241c06dea0 2024-04-02T04:44:17.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:17.023+0000 7f2427fff700 1 -- 172.21.15.16:0/3066276005 <== mon.1 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 34) v1 ==== 91478+0+0 (secure 0 0 0) 0x7f242008d050 con 0x7f2430100ad0 2024-04-02T04:44:17.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:17.023+0000 7f2427fff700 1 -- 172.21.15.16:0/3066276005 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f241c06dea0 msgr2=0x7f241c070360 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T04:44:17.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:17.023+0000 7f2427fff700 1 --2- 172.21.15.16:0/3066276005 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f241c06dea0 0x7f241c070360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:17.222 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:17 smithi016 ceph-mon[93230]: Active manager daemon smithi016.eqnebx restarted 2024-04-02T04:44:17.222 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:17 smithi016 ceph-mon[93230]: Activating manager daemon smithi016.eqnebx 2024-04-02T04:44:17.222 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:17 smithi016 ceph-mon[93230]: osdmap e49: 6 total, 6 up, 6 in 2024-04-02T04:44:17.222 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:17 smithi016 ceph-mon[93230]: mgrmap e34: smithi016.eqnebx(active, starting, since 0.00558826s), standbys: smithi027.kjzcsc 2024-04-02T04:44:17.223 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:17 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi016"}]: dispatch 2024-04-02T04:44:17.223 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:17 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2024-04-02T04:44:17.223 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:17 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi016.fyzrnm"}]: dispatch 2024-04-02T04:44:17.223 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:17 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi016.uhgkgg"}]: dispatch 2024-04-02T04:44:17.223 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:17 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.zwrfoo"}]: dispatch 2024-04-02T04:44:17.223 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:17 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.sdublg"}]: dispatch 2024-04-02T04:44:17.223 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:17 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mgr metadata", "who": "smithi016.eqnebx", "id": "smithi016.eqnebx"}]: dispatch 2024-04-02T04:44:17.223 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:17 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mgr metadata", "who": "smithi027.kjzcsc", "id": "smithi027.kjzcsc"}]: dispatch 2024-04-02T04:44:17.223 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:17 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T04:44:17.223 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:17 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T04:44:17.223 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:17 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T04:44:17.223 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:17 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T04:44:17.223 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:17 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T04:44:17.223 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:17 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T04:44:17.224 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:17 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T04:44:17.224 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:17 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T04:44:17.224 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:17 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T04:44:17.224 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:17 smithi016 ceph-mon[93230]: Manager daemon smithi016.eqnebx is now available 2024-04-02T04:44:17.224 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:17 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:44:17.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:17 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi027[37522]: 2024-04-02T04:44:17.013+0000 7f269cde0700 -1 mon.smithi027@1(peon).osd e49 update_from_paxos full map CRC mismatch, resetting to canonical 2024-04-02T04:44:17.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:17 smithi027 ceph-mon[37550]: Active manager daemon smithi016.eqnebx restarted 2024-04-02T04:44:17.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:17 smithi027 ceph-mon[37550]: Activating manager daemon smithi016.eqnebx 2024-04-02T04:44:17.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:17 smithi027 ceph-mon[37550]: osdmap e49: 6 total, 6 up, 6 in 2024-04-02T04:44:17.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:17 smithi027 ceph-mon[37550]: mgrmap e34: smithi016.eqnebx(active, starting, since 0.00558826s), standbys: smithi027.kjzcsc 2024-04-02T04:44:17.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:17 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi016"}]: dispatch 2024-04-02T04:44:17.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:17 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2024-04-02T04:44:17.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:17 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi016.fyzrnm"}]: dispatch 2024-04-02T04:44:17.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:17 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi016.uhgkgg"}]: dispatch 2024-04-02T04:44:17.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:17 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.zwrfoo"}]: dispatch 2024-04-02T04:44:17.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:17 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.sdublg"}]: dispatch 2024-04-02T04:44:17.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:17 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mgr metadata", "who": "smithi016.eqnebx", "id": "smithi016.eqnebx"}]: dispatch 2024-04-02T04:44:17.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:17 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mgr metadata", "who": "smithi027.kjzcsc", "id": "smithi027.kjzcsc"}]: dispatch 2024-04-02T04:44:17.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:17 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T04:44:17.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:17 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T04:44:17.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:17 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T04:44:17.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:17 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T04:44:17.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:17 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T04:44:17.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:17 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T04:44:17.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:17 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T04:44:17.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:17 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T04:44:17.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:17 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T04:44:17.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:17 smithi027 ceph-mon[37550]: Manager daemon smithi016.eqnebx is now available 2024-04-02T04:44:17.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:17 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:44:18.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.024+0000 7f2427fff700 1 -- 172.21.15.16:0/3066276005 <== mon.1 v2:172.21.15.27:3300/0 8 ==== mgrmap(e 35) v1 ==== 91605+0+0 (secure 0 0 0) 0x7f2420059d20 con 0x7f2430100ad0 2024-04-02T04:44:18.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.024+0000 7f2427fff700 1 --2- 172.21.15.16:0/3066276005 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f241c036ae0 0x7f241c077900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:18.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.024+0000 7f2427fff700 1 -- 172.21.15.16:0/3066276005 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2430074010 con 0x7f241c036ae0 2024-04-02T04:44:18.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.025+0000 7f242e59c700 1 --2- 172.21.15.16:0/3066276005 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f241c036ae0 0x7f241c077900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:18.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.025+0000 7f242e59c700 1 --2- 172.21.15.16:0/3066276005 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f241c036ae0 0x7f241c077900 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2418005950 tx=0x7f241800b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:18.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.027+0000 7f2427fff700 1 -- 172.21.15.16:0/3066276005 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f2430074010 con 0x7f241c036ae0 2024-04-02T04:44:18.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.030+0000 7f2435252700 1 -- 172.21.15.16:0/3066276005 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f241c036ae0 msgr2=0x7f241c077900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:18.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.030+0000 7f2435252700 1 --2- 172.21.15.16:0/3066276005 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f241c036ae0 0x7f241c077900 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2418005950 tx=0x7f241800b410 comp rx=0 tx=0).stop 2024-04-02T04:44:18.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.030+0000 7f2435252700 1 -- 172.21.15.16:0/3066276005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2430100ad0 msgr2=0x7f2430074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:18.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.030+0000 7f2435252700 1 --2- 172.21.15.16:0/3066276005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2430100ad0 0x7f2430074b00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f24200059d0 tx=0x7f242000bd70 comp rx=0 tx=0).stop 2024-04-02T04:44:18.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.030+0000 7f2435252700 1 -- 172.21.15.16:0/3066276005 shutdown_connections 2024-04-02T04:44:18.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.030+0000 7f2435252700 1 --2- 172.21.15.16:0/3066276005 >> [v2:172.21.15.16:6800/2776599951,v1:172.21.15.16:6801/2776599951] conn(0x7f241c06dea0 0x7f241c070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:18.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.030+0000 7f2435252700 1 --2- 172.21.15.16:0/3066276005 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f241c036ae0 0x7f241c077900 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:18.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.030+0000 7f2435252700 1 --2- 172.21.15.16:0/3066276005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2430100ad0 0x7f2430074b00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:18.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.030+0000 7f2435252700 1 --2- 172.21.15.16:0/3066276005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2430101cd0 0x7f24300731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:18.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.030+0000 7f2435252700 1 -- 172.21.15.16:0/3066276005 >> 172.21.15.16:0/3066276005 conn(0x7f24300fc090 msgr2=0x7f24301052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:18.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.030+0000 7f2435252700 1 -- 172.21.15.16:0/3066276005 shutdown_connections 2024-04-02T04:44:18.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.030+0000 7f2435252700 1 -- 172.21.15.16:0/3066276005 wait complete. 2024-04-02T04:44:18.046 INFO:teuthology.orchestra.run.smithi016.stdout:true 2024-04-02T04:44:18.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.144+0000 7f224f961700 1 -- 172.21.15.16:0/1800453944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2248100ae0 msgr2=0x7f2248100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:18.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.144+0000 7f224f961700 1 --2- 172.21.15.16:0/1800453944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2248100ae0 0x7f2248100f00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2244009910 tx=0x7f2244009c20 comp rx=0 tx=0).stop 2024-04-02T04:44:18.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.144+0000 7f224f961700 1 -- 172.21.15.16:0/1800453944 shutdown_connections 2024-04-02T04:44:18.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.144+0000 7f224f961700 1 --2- 172.21.15.16:0/1800453944 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2248101ce0 0x7f2248102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:18.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.144+0000 7f224f961700 1 --2- 172.21.15.16:0/1800453944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2248100ae0 0x7f2248100f00 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:18.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.144+0000 7f224f961700 1 -- 172.21.15.16:0/1800453944 >> 172.21.15.16:0/1800453944 conn(0x7f22480fc0a0 msgr2=0x7f22480fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:18.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.144+0000 7f224f961700 1 -- 172.21.15.16:0/1800453944 shutdown_connections 2024-04-02T04:44:18.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.144+0000 7f224f961700 1 -- 172.21.15.16:0/1800453944 wait complete. 2024-04-02T04:44:18.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.145+0000 7f224f961700 1 Processor -- start 2024-04-02T04:44:18.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.145+0000 7f224f961700 1 -- start start 2024-04-02T04:44:18.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.145+0000 7f224f961700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2248100ae0 0x7f2248074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:18.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.145+0000 7f224f961700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2248101ce0 0x7f22480731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:18.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.145+0000 7f224f961700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22480736e0 con 0x7f2248100ae0 2024-04-02T04:44:18.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.145+0000 7f224f961700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2248073820 con 0x7f2248101ce0 2024-04-02T04:44:18.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.145+0000 7f224d6fd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2248100ae0 0x7f2248074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:18.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.145+0000 7f224d6fd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2248100ae0 0x7f2248074b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59742/0 (socket says 172.21.15.16:59742) 2024-04-02T04:44:18.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.145+0000 7f224d6fd700 1 -- 172.21.15.16:0/256320389 learned_addr learned my addr 172.21.15.16:0/256320389 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:44:18.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.145+0000 7f224cefc700 1 --2- 172.21.15.16:0/256320389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2248101ce0 0x7f22480731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:18.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.146+0000 7f224d6fd700 1 -- 172.21.15.16:0/256320389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2248101ce0 msgr2=0x7f22480731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:18.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.146+0000 7f224d6fd700 1 --2- 172.21.15.16:0/256320389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2248101ce0 0x7f22480731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:18.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.146+0000 7f224d6fd700 1 -- 172.21.15.16:0/256320389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22440095f0 con 0x7f2248100ae0 2024-04-02T04:44:18.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.146+0000 7f224d6fd700 1 --2- 172.21.15.16:0/256320389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2248100ae0 0x7f2248074b00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2244005f00 tx=0x7f224400b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:18.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.146+0000 7f223e7fc700 1 -- 172.21.15.16:0/256320389 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f224401d070 con 0x7f2248100ae0 2024-04-02T04:44:18.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.146+0000 7f223e7fc700 1 -- 172.21.15.16:0/256320389 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f22440052a0 con 0x7f2248100ae0 2024-04-02T04:44:18.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.146+0000 7f224f961700 1 -- 172.21.15.16:0/256320389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2248073aa0 con 0x7f2248100ae0 2024-04-02T04:44:18.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.146+0000 7f223e7fc700 1 -- 172.21.15.16:0/256320389 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2244017400 con 0x7f2248100ae0 2024-04-02T04:44:18.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.146+0000 7f224f961700 1 -- 172.21.15.16:0/256320389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2248074010 con 0x7f2248100ae0 2024-04-02T04:44:18.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.147+0000 7f224f961700 1 -- 172.21.15.16:0/256320389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2248190660 con 0x7f2248100ae0 2024-04-02T04:44:18.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.148+0000 7f223e7fc700 1 -- 172.21.15.16:0/256320389 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 35) v1 ==== 91613+0+0 (secure 0 0 0) 0x7f2244003680 con 0x7f2248100ae0 2024-04-02T04:44:18.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.148+0000 7f223e7fc700 1 --2- 172.21.15.16:0/256320389 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f223406da90 0x7f223406ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:18.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.148+0000 7f223e7fc700 1 -- 172.21.15.16:0/256320389 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f224408e8d0 con 0x7f2248100ae0 2024-04-02T04:44:18.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.148+0000 7f224cefc700 1 --2- 172.21.15.16:0/256320389 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f223406da90 0x7f223406ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:18.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.149+0000 7f224cefc700 1 --2- 172.21.15.16:0/256320389 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f223406da90 0x7f223406ff50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2238009c60 tx=0x7f2238009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:18.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.153+0000 7f223e7fc700 1 -- 172.21.15.16:0/256320389 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f22440580c0 con 0x7f2248100ae0 2024-04-02T04:44:18.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:18 smithi016 ceph-mon[93230]: failed to encode map e49 with expected crc 2024-04-02T04:44:18.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:18 smithi016 ceph-mon[93230]: failed to encode map e49 with expected crc 2024-04-02T04:44:18.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:18 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:44:18.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:18 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi016.eqnebx/mirror_snapshot_schedule"}]: dispatch 2024-04-02T04:44:18.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:18 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi016.eqnebx/trash_purge_schedule"}]: dispatch 2024-04-02T04:44:18.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:18 smithi016 ceph-mon[93230]: mgrmap e35: smithi016.eqnebx(active, since 1.00983s), standbys: smithi027.kjzcsc 2024-04-02T04:44:18.342 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:18 smithi027 ceph-mon[37550]: failed to encode map e49 with expected crc 2024-04-02T04:44:18.342 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:18 smithi027 ceph-mon[37550]: failed to encode map e49 with expected crc 2024-04-02T04:44:18.342 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:18 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:44:18.342 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:18 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi016.eqnebx/mirror_snapshot_schedule"}]: dispatch 2024-04-02T04:44:18.343 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:18 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi016.eqnebx/trash_purge_schedule"}]: dispatch 2024-04-02T04:44:18.343 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:18 smithi027 ceph-mon[37550]: mgrmap e35: smithi016.eqnebx(active, since 1.00983s), standbys: smithi027.kjzcsc 2024-04-02T04:44:18.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.389+0000 7f224f961700 1 -- 172.21.15.16:0/256320389 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2248074010 con 0x7f223406da90 2024-04-02T04:44:18.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.390+0000 7f223e7fc700 1 -- 172.21.15.16:0/256320389 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f2248074010 con 0x7f223406da90 2024-04-02T04:44:18.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.393+0000 7f224f961700 1 -- 172.21.15.16:0/256320389 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f223406da90 msgr2=0x7f223406ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:18.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.393+0000 7f224f961700 1 --2- 172.21.15.16:0/256320389 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f223406da90 0x7f223406ff50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2238009c60 tx=0x7f2238009260 comp rx=0 tx=0).stop 2024-04-02T04:44:18.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.393+0000 7f224f961700 1 -- 172.21.15.16:0/256320389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2248100ae0 msgr2=0x7f2248074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:18.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.393+0000 7f224f961700 1 --2- 172.21.15.16:0/256320389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2248100ae0 0x7f2248074b00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2244005f00 tx=0x7f224400b5e0 comp rx=0 tx=0).stop 2024-04-02T04:44:18.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.393+0000 7f224f961700 1 -- 172.21.15.16:0/256320389 shutdown_connections 2024-04-02T04:44:18.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.393+0000 7f224f961700 1 --2- 172.21.15.16:0/256320389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2248100ae0 0x7f2248074b00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:18.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.393+0000 7f224f961700 1 --2- 172.21.15.16:0/256320389 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f223406da90 0x7f223406ff50 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:18.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.393+0000 7f224f961700 1 --2- 172.21.15.16:0/256320389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2248101ce0 0x7f22480731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:18.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.393+0000 7f224f961700 1 -- 172.21.15.16:0/256320389 >> 172.21.15.16:0/256320389 conn(0x7f22480fc0a0 msgr2=0x7f22481052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:18.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.393+0000 7f224f961700 1 -- 172.21.15.16:0/256320389 shutdown_connections 2024-04-02T04:44:18.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.393+0000 7f224f961700 1 -- 172.21.15.16:0/256320389 wait complete. 2024-04-02T04:44:18.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.503+0000 7fe6ccf78700 1 -- 172.21.15.16:0/665896458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c80ff9d0 msgr2=0x7fe6c80ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:18.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.503+0000 7fe6ccf78700 1 --2- 172.21.15.16:0/665896458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c80ff9d0 0x7fe6c80ffdf0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8009910 tx=0x7fe6b8009c20 comp rx=0 tx=0).stop 2024-04-02T04:44:18.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.503+0000 7fe6ccf78700 1 -- 172.21.15.16:0/665896458 shutdown_connections 2024-04-02T04:44:18.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.503+0000 7fe6ccf78700 1 --2- 172.21.15.16:0/665896458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6c8100bd0 0x7fe6c8101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:18.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.503+0000 7fe6ccf78700 1 --2- 172.21.15.16:0/665896458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c80ff9d0 0x7fe6c80ffdf0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:18.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.503+0000 7fe6ccf78700 1 -- 172.21.15.16:0/665896458 >> 172.21.15.16:0/665896458 conn(0x7fe6c80faf70 msgr2=0x7fe6c80fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:18.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.503+0000 7fe6ccf78700 1 -- 172.21.15.16:0/665896458 shutdown_connections 2024-04-02T04:44:18.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.503+0000 7fe6ccf78700 1 -- 172.21.15.16:0/665896458 wait complete. 2024-04-02T04:44:18.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.504+0000 7fe6ccf78700 1 Processor -- start 2024-04-02T04:44:18.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.504+0000 7fe6ccf78700 1 -- start start 2024-04-02T04:44:18.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.504+0000 7fe6ccf78700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c80ff9d0 0x7fe6c8103870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:18.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.504+0000 7fe6ccf78700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6c8100bd0 0x7fe6c8101fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:18.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.504+0000 7fe6ccf78700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c8103e90 con 0x7fe6c80ff9d0 2024-04-02T04:44:18.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.504+0000 7fe6ccf78700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c81024f0 con 0x7fe6c8100bd0 2024-04-02T04:44:18.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.504+0000 7fe6c659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c80ff9d0 0x7fe6c8103870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:18.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.504+0000 7fe6c659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c80ff9d0 0x7fe6c8103870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59762/0 (socket says 172.21.15.16:59762) 2024-04-02T04:44:18.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.504+0000 7fe6c659c700 1 -- 172.21.15.16:0/4243177220 learned_addr learned my addr 172.21.15.16:0/4243177220 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:44:18.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.504+0000 7fe6c5d9b700 1 --2- 172.21.15.16:0/4243177220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6c8100bd0 0x7fe6c8101fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:18.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.504+0000 7fe6c5d9b700 1 -- 172.21.15.16:0/4243177220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c80ff9d0 msgr2=0x7fe6c8103870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:18.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.504+0000 7fe6c5d9b700 1 --2- 172.21.15.16:0/4243177220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c80ff9d0 0x7fe6c8103870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:18.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.504+0000 7fe6c5d9b700 1 -- 172.21.15.16:0/4243177220 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b80095f0 con 0x7fe6c8100bd0 2024-04-02T04:44:18.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.504+0000 7fe6c659c700 1 --2- 172.21.15.16:0/4243177220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c80ff9d0 0x7fe6c8103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T04:44:18.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.505+0000 7fe6c5d9b700 1 --2- 172.21.15.16:0/4243177220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6c8100bd0 0x7fe6c8101fb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc00e970 tx=0x7fe6bc00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:18.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.505+0000 7fe6b77fe700 1 -- 172.21.15.16:0/4243177220 <== mon.1 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6bc00c9d0 con 0x7fe6c8100bd0 2024-04-02T04:44:18.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.505+0000 7fe6ccf78700 1 -- 172.21.15.16:0/4243177220 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6c81027a0 con 0x7fe6c8100bd0 2024-04-02T04:44:18.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.505+0000 7fe6b77fe700 1 -- 172.21.15.16:0/4243177220 <== mon.1 v2:172.21.15.27:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe6bc010430 con 0x7fe6c8100bd0 2024-04-02T04:44:18.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.505+0000 7fe6ccf78700 1 -- 172.21.15.16:0/4243177220 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6c8102d70 con 0x7fe6c8100bd0 2024-04-02T04:44:18.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.505+0000 7fe6b77fe700 1 -- 172.21.15.16:0/4243177220 <== mon.1 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6bc018630 con 0x7fe6c8100bd0 2024-04-02T04:44:18.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.506+0000 7fe6ccf78700 1 -- 172.21.15.16:0/4243177220 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6c804ea80 con 0x7fe6c8100bd0 2024-04-02T04:44:18.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.506+0000 7fe6b77fe700 1 -- 172.21.15.16:0/4243177220 <== mon.1 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 35) v1 ==== 91605+0+0 (secure 0 0 0) 0x7fe6bc0105a0 con 0x7fe6c8100bd0 2024-04-02T04:44:18.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.507+0000 7fe6b77fe700 1 --2- 172.21.15.16:0/4243177220 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fe6b006da90 0x7fe6b006ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:18.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.507+0000 7fe6b77fe700 1 -- 172.21.15.16:0/4243177220 <== mon.1 v2:172.21.15.27:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fe6bc014070 con 0x7fe6c8100bd0 2024-04-02T04:44:18.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.507+0000 7fe6c659c700 1 --2- 172.21.15.16:0/4243177220 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fe6b006da90 0x7fe6b006ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:18.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.507+0000 7fe6c659c700 1 --2- 172.21.15.16:0/4243177220 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fe6b006da90 0x7fe6b006ff50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe6b800b6f0 tx=0x7fe6b8005fb0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:18.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.516+0000 7fe6b77fe700 1 -- 172.21.15.16:0/4243177220 <== mon.1 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe6bc053280 con 0x7fe6c8100bd0 2024-04-02T04:44:18.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.788+0000 7fe6ccf78700 1 -- 172.21.15.16:0/4243177220 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6c80fcee0 con 0x7fe6b006da90 2024-04-02T04:44:18.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.801+0000 7fe6b77fe700 1 -- 172.21.15.16:0/4243177220 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fe6c80fcee0 con 0x7fe6b006da90 2024-04-02T04:44:18.805 INFO:teuthology.orchestra.run.smithi016.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:44:18.805 INFO:teuthology.orchestra.run.smithi016.stdout:alertmanager.smithi016 smithi016 *:9093,9094 running (8m) 25s ago 10m 24.2M - 0.25.0 c8568f914cd2 b95d8354b016 2024-04-02T04:44:18.805 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi016 smithi016 running (10m) 25s ago 10m 7776k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 9de6c16a487f 2024-04-02T04:44:18.805 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi027 smithi027 running (9m) 25s ago 9m 7818k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 99956592471a 2024-04-02T04:44:18.805 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi016 smithi016 running (10m) 25s ago 10m 6656k - 18.2.2-1059-g4d05f4ca 909dce4fe53d bd73919b0711 2024-04-02T04:44:18.805 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi027 smithi027 running (9m) 25s ago 9m 6660k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 7a54c37f202e 2024-04-02T04:44:18.805 INFO:teuthology.orchestra.run.smithi016.stdout:grafana.smithi016 smithi016 *:3000 running (8m) 25s ago 10m 85.0M - 9.4.7 954c08fa6188 7c43d133569c 2024-04-02T04:44:18.805 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.fyzrnm smithi016 running (4m) 25s ago 4m 15.7M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 70f28b9572a6 2024-04-02T04:44:18.805 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.uhgkgg smithi016 running (4m) 25s ago 4m 14.0M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 0aa4d16562a6 2024-04-02T04:44:18.805 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.sdublg smithi027 running (4m) 25s ago 4m 18.0M - 18.2.2-1059-g4d05f4ca 909dce4fe53d a538451099db 2024-04-02T04:44:18.805 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.zwrfoo smithi027 running (4m) 25s ago 4m 254M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 4cb3fa428d71 2024-04-02T04:44:18.805 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi016.eqnebx smithi016 *:8443,9283,8765 running (79s) 25s ago 11m 495M - 19.0.0-1979-g6cec793d 78f830190839 d61ed91b4577 2024-04-02T04:44:18.805 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi027.kjzcsc smithi027 *:8443,9283,8765 running (36s) 25s ago 9m 108M - 19.0.0-1979-g6cec793d 78f830190839 5072a5ffc761 2024-04-02T04:44:18.806 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi016 smithi016 running (11m) 25s ago 11m 55.7M 2048M 18.2.2-1059-g4d05f4ca 909dce4fe53d 24d5820dc994 2024-04-02T04:44:18.806 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi027 smithi027 running (9m) 25s ago 9m 42.0M 2048M 18.2.2-1059-g4d05f4ca 909dce4fe53d b77563184a1e 2024-04-02T04:44:18.806 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi016 smithi016 *:9100 running (10m) 25s ago 10m 14.6M - 1.5.0 0da6a335fe13 09f54c65b53c 2024-04-02T04:44:18.806 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi027 smithi027 *:9100 running (9m) 25s ago 9m 15.5M - 1.5.0 0da6a335fe13 e41a657c7443 2024-04-02T04:44:18.806 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 smithi016 running (8m) 25s ago 8m 179M 1217M 18.2.2-1059-g4d05f4ca 909dce4fe53d 3f40108debd6 2024-04-02T04:44:18.806 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 smithi016 running (7m) 25s ago 7m 189M 1217M 18.2.2-1059-g4d05f4ca 909dce4fe53d 7361abc9e1b6 2024-04-02T04:44:18.806 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 smithi016 running (7m) 25s ago 7m 154M 1217M 18.2.2-1059-g4d05f4ca 909dce4fe53d 52e592b4d1f2 2024-04-02T04:44:18.806 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 smithi027 running (7m) 25s ago 7m 203M 2240M 18.2.2-1059-g4d05f4ca 909dce4fe53d cf7c7fe022f0 2024-04-02T04:44:18.806 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 smithi027 running (6m) 25s ago 6m 184M 2240M 18.2.2-1059-g4d05f4ca 909dce4fe53d 8cbbedf8cac9 2024-04-02T04:44:18.806 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 smithi027 running (6m) 25s ago 6m 168M 2240M 18.2.2-1059-g4d05f4ca 909dce4fe53d ac6094415529 2024-04-02T04:44:18.806 INFO:teuthology.orchestra.run.smithi016.stdout:prometheus.smithi016 smithi016 *:9095 running (50s) 25s ago 9m 48.4M - 2.43.0 a07b618ecd1d 466f00d73ddd 2024-04-02T04:44:18.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.804+0000 7fe6ccf78700 1 -- 172.21.15.16:0/4243177220 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fe6b006da90 msgr2=0x7fe6b006ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:18.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.804+0000 7fe6ccf78700 1 --2- 172.21.15.16:0/4243177220 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fe6b006da90 0x7fe6b006ff50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe6b800b6f0 tx=0x7fe6b8005fb0 comp rx=0 tx=0).stop 2024-04-02T04:44:18.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.804+0000 7fe6ccf78700 1 -- 172.21.15.16:0/4243177220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6c8100bd0 msgr2=0x7fe6c8101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:18.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.804+0000 7fe6ccf78700 1 --2- 172.21.15.16:0/4243177220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6c8100bd0 0x7fe6c8101fb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc00e970 tx=0x7fe6bc00ec80 comp rx=0 tx=0).stop 2024-04-02T04:44:18.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.805+0000 7fe6ccf78700 1 -- 172.21.15.16:0/4243177220 shutdown_connections 2024-04-02T04:44:18.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.805+0000 7fe6ccf78700 1 --2- 172.21.15.16:0/4243177220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c80ff9d0 0x7fe6c8103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:18.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.805+0000 7fe6ccf78700 1 --2- 172.21.15.16:0/4243177220 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fe6b006da90 0x7fe6b006ff50 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:18.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.805+0000 7fe6ccf78700 1 --2- 172.21.15.16:0/4243177220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6c8100bd0 0x7fe6c8101fb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:18.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.805+0000 7fe6ccf78700 1 -- 172.21.15.16:0/4243177220 >> 172.21.15.16:0/4243177220 conn(0x7fe6c80faf70 msgr2=0x7fe6c80fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:18.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.805+0000 7fe6ccf78700 1 -- 172.21.15.16:0/4243177220 shutdown_connections 2024-04-02T04:44:18.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.805+0000 7fe6ccf78700 1 -- 172.21.15.16:0/4243177220 wait complete. 2024-04-02T04:44:18.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.920+0000 7f32350dd700 1 -- 172.21.15.16:0/487588681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3230101db0 msgr2=0x7f3230102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:18.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.920+0000 7f32350dd700 1 --2- 172.21.15.16:0/487588681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3230101db0 0x7f3230102210 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3224009910 tx=0x7f3224009c20 comp rx=0 tx=0).stop 2024-04-02T04:44:18.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.920+0000 7f32350dd700 1 -- 172.21.15.16:0/487588681 shutdown_connections 2024-04-02T04:44:18.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.920+0000 7f32350dd700 1 --2- 172.21.15.16:0/487588681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3230101db0 0x7f3230102210 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:18.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.920+0000 7f32350dd700 1 --2- 172.21.15.16:0/487588681 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32300fedd0 0x7f32300ff1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:18.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.920+0000 7f32350dd700 1 -- 172.21.15.16:0/487588681 >> 172.21.15.16:0/487588681 conn(0x7f32300fa6f0 msgr2=0x7f32300fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:18.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.920+0000 7f32350dd700 1 -- 172.21.15.16:0/487588681 shutdown_connections 2024-04-02T04:44:18.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.920+0000 7f32350dd700 1 -- 172.21.15.16:0/487588681 wait complete. 2024-04-02T04:44:18.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.921+0000 7f32350dd700 1 Processor -- start 2024-04-02T04:44:18.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.921+0000 7f32350dd700 1 -- start start 2024-04-02T04:44:18.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.921+0000 7f32350dd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32300fedd0 0x7f3230196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:18.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.921+0000 7f32350dd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3230101db0 0x7f32301969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:18.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.921+0000 7f32350dd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3230196ff0 con 0x7f32300fedd0 2024-04-02T04:44:18.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.921+0000 7f32350dd700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3230197130 con 0x7f3230101db0 2024-04-02T04:44:18.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.921+0000 7f322ed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32300fedd0 0x7f3230196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:18.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.921+0000 7f322ed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32300fedd0 0x7f3230196490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59778/0 (socket says 172.21.15.16:59778) 2024-04-02T04:44:18.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.921+0000 7f322ed9d700 1 -- 172.21.15.16:0/1776359316 learned_addr learned my addr 172.21.15.16:0/1776359316 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:44:18.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.921+0000 7f322e59c700 1 --2- 172.21.15.16:0/1776359316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3230101db0 0x7f32301969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:18.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.922+0000 7f322ed9d700 1 -- 172.21.15.16:0/1776359316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3230101db0 msgr2=0x7f32301969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:18.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.922+0000 7f322ed9d700 1 --2- 172.21.15.16:0/1776359316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3230101db0 0x7f32301969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:18.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.922+0000 7f322ed9d700 1 -- 172.21.15.16:0/1776359316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32240095f0 con 0x7f32300fedd0 2024-04-02T04:44:18.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.922+0000 7f322ed9d700 1 --2- 172.21.15.16:0/1776359316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32300fedd0 0x7f3230196490 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f322000e8f0 tx=0x7f322000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:18.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.922+0000 7f3217fff700 1 -- 172.21.15.16:0/1776359316 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3220009790 con 0x7f32300fedd0 2024-04-02T04:44:18.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.922+0000 7f3217fff700 1 -- 172.21.15.16:0/1776359316 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f322000c810 con 0x7f32300fedd0 2024-04-02T04:44:18.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.922+0000 7f32350dd700 1 -- 172.21.15.16:0/1776359316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f323019bb90 con 0x7f32300fedd0 2024-04-02T04:44:18.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.922+0000 7f3217fff700 1 -- 172.21.15.16:0/1776359316 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3220003d20 con 0x7f32300fedd0 2024-04-02T04:44:18.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.922+0000 7f32350dd700 1 -- 172.21.15.16:0/1776359316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f323019c0e0 con 0x7f32300fedd0 2024-04-02T04:44:18.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.923+0000 7f32350dd700 1 -- 172.21.15.16:0/1776359316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3230190690 con 0x7f32300fedd0 2024-04-02T04:44:18.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.924+0000 7f3217fff700 1 -- 172.21.15.16:0/1776359316 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 35) v1 ==== 91613+0+0 (secure 0 0 0) 0x7f32200041f0 con 0x7f32300fedd0 2024-04-02T04:44:18.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.924+0000 7f3217fff700 1 --2- 172.21.15.16:0/1776359316 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f321806da80 0x7f321806ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:18.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.924+0000 7f3217fff700 1 -- 172.21.15.16:0/1776359316 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f3220014070 con 0x7f32300fedd0 2024-04-02T04:44:18.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.924+0000 7f322e59c700 1 --2- 172.21.15.16:0/1776359316 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f321806da80 0x7f321806ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:18.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.925+0000 7f322e59c700 1 --2- 172.21.15.16:0/1776359316 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f321806da80 0x7f321806ff40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f322400b800 tx=0x7f3224009f70 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:18.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:18.929+0000 7f3217fff700 1 -- 172.21.15.16:0/1776359316 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3220057160 con 0x7f32300fedd0 2024-04-02T04:44:19.070 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:19 smithi027 ceph-mon[37550]: [02/Apr/2024:04:44:17] ENGINE Bus STARTING 2024-04-02T04:44:19.070 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:19 smithi027 ceph-mon[37550]: from='client.24589 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:44:19.070 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:19 smithi027 ceph-mon[37550]: pgmap v3: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:44:19.070 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:19 smithi027 ceph-mon[37550]: [02/Apr/2024:04:44:18] ENGINE Serving on http://172.21.15.16:8765 2024-04-02T04:44:19.070 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:19 smithi027 ceph-mon[37550]: [02/Apr/2024:04:44:18] ENGINE Serving on https://172.21.15.16:7150 2024-04-02T04:44:19.070 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:19 smithi027 ceph-mon[37550]: [02/Apr/2024:04:44:18] ENGINE Bus STARTED 2024-04-02T04:44:19.071 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:19 smithi027 ceph-mon[37550]: Standby manager daemon smithi027.kjzcsc restarted 2024-04-02T04:44:19.071 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:19 smithi027 ceph-mon[37550]: Standby manager daemon smithi027.kjzcsc started 2024-04-02T04:44:19.071 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:19 smithi027 ceph-mon[37550]: from='mgr.? 172.21.15.27:0/2031678168' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi027.kjzcsc/crt"}]: dispatch 2024-04-02T04:44:19.071 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:19 smithi027 ceph-mon[37550]: from='mgr.? 172.21.15.27:0/2031678168' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T04:44:19.071 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:19 smithi027 ceph-mon[37550]: from='mgr.? 172.21.15.27:0/2031678168' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi027.kjzcsc/key"}]: dispatch 2024-04-02T04:44:19.071 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:19 smithi027 ceph-mon[37550]: from='mgr.? 172.21.15.27:0/2031678168' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T04:44:19.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.073+0000 7f3217fff700 1 -- 172.21.15.16:0/1776359316 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 36) v1 ==== 91907+0+0 (secure 0 0 0) 0x7f3220057530 con 0x7f32300fedd0 2024-04-02T04:44:19.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.257+0000 7f32350dd700 1 -- 172.21.15.16:0/1776359316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f323004ea80 con 0x7f32300fedd0 2024-04-02T04:44:19.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.257+0000 7f3217fff700 1 -- 172.21.15.16:0/1776359316 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f322007ba70 con 0x7f32300fedd0 2024-04-02T04:44:19.260 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:44:19.260 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": { 2024-04-02T04:44:19.261 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 1, 2024-04-02T04:44:19.261 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 1 2024-04-02T04:44:19.261 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:44:19.261 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": { 2024-04-02T04:44:19.261 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:44:19.261 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:44:19.261 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": { 2024-04-02T04:44:19.261 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 6 2024-04-02T04:44:19.261 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:44:19.261 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-04-02T04:44:19.261 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 4 2024-04-02T04:44:19.261 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:44:19.261 INFO:teuthology.orchestra.run.smithi016.stdout: "overall": { 2024-04-02T04:44:19.261 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 11, 2024-04-02T04:44:19.261 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 3 2024-04-02T04:44:19.262 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-02T04:44:19.262 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:44:19.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.259+0000 7f32350dd700 1 -- 172.21.15.16:0/1776359316 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f321806da80 msgr2=0x7f321806ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:19.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.259+0000 7f32350dd700 1 --2- 172.21.15.16:0/1776359316 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f321806da80 0x7f321806ff40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f322400b800 tx=0x7f3224009f70 comp rx=0 tx=0).stop 2024-04-02T04:44:19.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.259+0000 7f32350dd700 1 -- 172.21.15.16:0/1776359316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32300fedd0 msgr2=0x7f3230196490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:19.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.259+0000 7f32350dd700 1 --2- 172.21.15.16:0/1776359316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32300fedd0 0x7f3230196490 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f322000e8f0 tx=0x7f322000ec00 comp rx=0 tx=0).stop 2024-04-02T04:44:19.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.259+0000 7f32350dd700 1 -- 172.21.15.16:0/1776359316 shutdown_connections 2024-04-02T04:44:19.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.259+0000 7f32350dd700 1 --2- 172.21.15.16:0/1776359316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32300fedd0 0x7f3230196490 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:19.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.259+0000 7f32350dd700 1 --2- 172.21.15.16:0/1776359316 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f321806da80 0x7f321806ff40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:19.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.259+0000 7f32350dd700 1 --2- 172.21.15.16:0/1776359316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3230101db0 0x7f32301969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:19.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.259+0000 7f32350dd700 1 -- 172.21.15.16:0/1776359316 >> 172.21.15.16:0/1776359316 conn(0x7f32300fa6f0 msgr2=0x7f3230105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:19.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.259+0000 7f32350dd700 1 -- 172.21.15.16:0/1776359316 shutdown_connections 2024-04-02T04:44:19.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.260+0000 7f32350dd700 1 -- 172.21.15.16:0/1776359316 wait complete. 2024-04-02T04:44:19.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.372+0000 7f582ed9e700 1 -- 172.21.15.16:0/4195573035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5828100ad0 msgr2=0x7f5828100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:19.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.372+0000 7f582ed9e700 1 --2- 172.21.15.16:0/4195573035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5828100ad0 0x7f5828100ef0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f581c009910 tx=0x7f581c009c20 comp rx=0 tx=0).stop 2024-04-02T04:44:19.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.372+0000 7f582ed9e700 1 -- 172.21.15.16:0/4195573035 shutdown_connections 2024-04-02T04:44:19.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.372+0000 7f582ed9e700 1 --2- 172.21.15.16:0/4195573035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5828101cd0 0x7f5828102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:19.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.372+0000 7f582ed9e700 1 --2- 172.21.15.16:0/4195573035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5828100ad0 0x7f5828100ef0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:19.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.372+0000 7f582ed9e700 1 -- 172.21.15.16:0/4195573035 >> 172.21.15.16:0/4195573035 conn(0x7f58280fc090 msgr2=0x7f58280fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:19.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.372+0000 7f582ed9e700 1 -- 172.21.15.16:0/4195573035 shutdown_connections 2024-04-02T04:44:19.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.372+0000 7f582ed9e700 1 -- 172.21.15.16:0/4195573035 wait complete. 2024-04-02T04:44:19.376 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:19 smithi016 ceph-mon[93230]: [02/Apr/2024:04:44:17] ENGINE Bus STARTING 2024-04-02T04:44:19.376 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:19 smithi016 ceph-mon[93230]: from='client.24589 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:44:19.376 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:19 smithi016 ceph-mon[93230]: pgmap v3: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:44:19.376 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:19 smithi016 ceph-mon[93230]: [02/Apr/2024:04:44:18] ENGINE Serving on http://172.21.15.16:8765 2024-04-02T04:44:19.377 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:19 smithi016 ceph-mon[93230]: [02/Apr/2024:04:44:18] ENGINE Serving on https://172.21.15.16:7150 2024-04-02T04:44:19.377 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:19 smithi016 ceph-mon[93230]: [02/Apr/2024:04:44:18] ENGINE Bus STARTED 2024-04-02T04:44:19.377 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:19 smithi016 ceph-mon[93230]: Standby manager daemon smithi027.kjzcsc restarted 2024-04-02T04:44:19.377 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:19 smithi016 ceph-mon[93230]: Standby manager daemon smithi027.kjzcsc started 2024-04-02T04:44:19.377 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:19 smithi016 ceph-mon[93230]: from='mgr.? 172.21.15.27:0/2031678168' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi027.kjzcsc/crt"}]: dispatch 2024-04-02T04:44:19.377 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:19 smithi016 ceph-mon[93230]: from='mgr.? 172.21.15.27:0/2031678168' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T04:44:19.377 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:19 smithi016 ceph-mon[93230]: from='mgr.? 172.21.15.27:0/2031678168' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi027.kjzcsc/key"}]: dispatch 2024-04-02T04:44:19.377 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:19 smithi016 ceph-mon[93230]: from='mgr.? 172.21.15.27:0/2031678168' entity='mgr.smithi027.kjzcsc' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T04:44:19.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.373+0000 7f582ed9e700 1 Processor -- start 2024-04-02T04:44:19.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.373+0000 7f582ed9e700 1 -- start start 2024-04-02T04:44:19.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.373+0000 7f582ed9e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5828100ad0 0x7f58281964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:19.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.373+0000 7f582ed9e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5828101cd0 0x7f5828196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:19.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.373+0000 7f582ed9e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5828197050 con 0x7f5828100ad0 2024-04-02T04:44:19.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.373+0000 7f582ed9e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5828197190 con 0x7f5828101cd0 2024-04-02T04:44:19.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.373+0000 7f582cb3a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5828100ad0 0x7f58281964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.373+0000 7f582cb3a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5828100ad0 0x7f58281964f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59814/0 (socket says 172.21.15.16:59814) 2024-04-02T04:44:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.373+0000 7f582cb3a700 1 -- 172.21.15.16:0/1893108049 learned_addr learned my addr 172.21.15.16:0/1893108049 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:44:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.373+0000 7f5827fff700 1 --2- 172.21.15.16:0/1893108049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5828101cd0 0x7f5828196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.373+0000 7f582cb3a700 1 -- 172.21.15.16:0/1893108049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5828101cd0 msgr2=0x7f5828196a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.373+0000 7f582cb3a700 1 --2- 172.21.15.16:0/1893108049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5828101cd0 0x7f5828196a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.374+0000 7f582cb3a700 1 -- 172.21.15.16:0/1893108049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581c0095f0 con 0x7f5828100ad0 2024-04-02T04:44:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.374+0000 7f582cb3a700 1 --2- 172.21.15.16:0/1893108049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5828100ad0 0x7f58281964f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f581c005f00 tx=0x7f581c00b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.374+0000 7f5825ffb700 1 -- 172.21.15.16:0/1893108049 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f581c01d070 con 0x7f5828100ad0 2024-04-02T04:44:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.374+0000 7f5825ffb700 1 -- 172.21.15.16:0/1893108049 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f581c0052a0 con 0x7f5828100ad0 2024-04-02T04:44:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.374+0000 7f582ed9e700 1 -- 172.21.15.16:0/1893108049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f582819bb90 con 0x7f5828100ad0 2024-04-02T04:44:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.374+0000 7f5825ffb700 1 -- 172.21.15.16:0/1893108049 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f581c017400 con 0x7f5828100ad0 2024-04-02T04:44:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.374+0000 7f582ed9e700 1 -- 172.21.15.16:0/1893108049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f582819c080 con 0x7f5828100ad0 2024-04-02T04:44:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.375+0000 7f582ed9e700 1 -- 172.21.15.16:0/1893108049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5828190650 con 0x7f5828100ad0 2024-04-02T04:44:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.376+0000 7f5825ffb700 1 -- 172.21.15.16:0/1893108049 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 36) v1 ==== 91907+0+0 (secure 0 0 0) 0x7f581c003680 con 0x7f5828100ad0 2024-04-02T04:44:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.376+0000 7f5825ffb700 1 --2- 172.21.15.16:0/1893108049 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f581806de10 0x7f58180702d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.376+0000 7f5825ffb700 1 -- 172.21.15.16:0/1893108049 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f581c08efa0 con 0x7f5828100ad0 2024-04-02T04:44:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.376+0000 7f5827fff700 1 --2- 172.21.15.16:0/1893108049 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f581806de10 0x7f58180702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.377+0000 7f5827fff700 1 --2- 172.21.15.16:0/1893108049 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f581806de10 0x7f58180702d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5814009c60 tx=0x7f5814009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:19.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.381+0000 7f5825ffb700 1 -- 172.21.15.16:0/1893108049 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f581c05bfa0 con 0x7f5828100ad0 2024-04-02T04:44:19.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.701+0000 7f582ed9e700 1 -- 172.21.15.16:0/1893108049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5828109630 con 0x7f5828100ad0 2024-04-02T04:44:19.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.701+0000 7f5825ffb700 1 -- 172.21.15.16:0/1893108049 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1875 (secure 0 0 0) 0x7f581c05bb30 con 0x7f5828100ad0 2024-04-02T04:44:19.705 INFO:teuthology.orchestra.run.smithi016.stdout:e15 2024-04-02T04:44:19.705 INFO:teuthology.orchestra.run.smithi016.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:44:19.705 INFO:teuthology.orchestra.run.smithi016.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:44:19.705 INFO:teuthology.orchestra.run.smithi016.stdout:legacy client fscid: 1 2024-04-02T04:44:19.705 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:44:19.705 INFO:teuthology.orchestra.run.smithi016.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:44:19.705 INFO:teuthology.orchestra.run.smithi016.stdout:fs_name cephfs 2024-04-02T04:44:19.705 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 15 2024-04-02T04:44:19.705 INFO:teuthology.orchestra.run.smithi016.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:44:19.705 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-04-02T04:39:30.740168+0000 2024-04-02T04:44:19.705 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-04-02T04:39:49.875319+0000 2024-04-02T04:44:19.706 INFO:teuthology.orchestra.run.smithi016.stdout:tableserver 0 2024-04-02T04:44:19.706 INFO:teuthology.orchestra.run.smithi016.stdout:root 0 2024-04-02T04:44:19.706 INFO:teuthology.orchestra.run.smithi016.stdout:session_timeout 60 2024-04-02T04:44:19.706 INFO:teuthology.orchestra.run.smithi016.stdout:session_autoclose 300 2024-04-02T04:44:19.706 INFO:teuthology.orchestra.run.smithi016.stdout:max_file_size 1099511627776 2024-04-02T04:44:19.706 INFO:teuthology.orchestra.run.smithi016.stdout:max_xattr_size 65536 2024-04-02T04:44:19.706 INFO:teuthology.orchestra.run.smithi016.stdout:required_client_features {} 2024-04-02T04:44:19.706 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure 0 2024-04-02T04:44:19.706 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure_osd_epoch 46 2024-04-02T04:44:19.706 INFO:teuthology.orchestra.run.smithi016.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:44:19.706 INFO:teuthology.orchestra.run.smithi016.stdout:max_mds 1 2024-04-02T04:44:19.706 INFO:teuthology.orchestra.run.smithi016.stdout:in 0 2024-04-02T04:44:19.707 INFO:teuthology.orchestra.run.smithi016.stdout:up {0=24285} 2024-04-02T04:44:19.707 INFO:teuthology.orchestra.run.smithi016.stdout:failed 2024-04-02T04:44:19.707 INFO:teuthology.orchestra.run.smithi016.stdout:damaged 2024-04-02T04:44:19.707 INFO:teuthology.orchestra.run.smithi016.stdout:stopped 2024-04-02T04:44:19.707 INFO:teuthology.orchestra.run.smithi016.stdout:data_pools [3] 2024-04-02T04:44:19.707 INFO:teuthology.orchestra.run.smithi016.stdout:metadata_pool 2 2024-04-02T04:44:19.707 INFO:teuthology.orchestra.run.smithi016.stdout:inline_data enabled 2024-04-02T04:44:19.707 INFO:teuthology.orchestra.run.smithi016.stdout:balancer 2024-04-02T04:44:19.707 INFO:teuthology.orchestra.run.smithi016.stdout:bal_rank_mask -1 2024-04-02T04:44:19.707 INFO:teuthology.orchestra.run.smithi016.stdout:standby_count_wanted 1 2024-04-02T04:44:19.707 INFO:teuthology.orchestra.run.smithi016.stdout:qdb_cluster leader: 0 members: 2024-04-02T04:44:19.707 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.zwrfoo{0:24285} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.27:6824/1461455976,v1:172.21.15.27:6825/1461455976] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:44:19.708 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:44:19.708 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:44:19.708 INFO:teuthology.orchestra.run.smithi016.stdout:Standby daemons: 2024-04-02T04:44:19.708 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:44:19.708 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.fyzrnm{-1:14488} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.16:6828/2578719859,v1:172.21.15.16:6829/2578719859] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:44:19.708 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.uhgkgg{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.16:6826/601263579,v1:172.21.15.16:6827/601263579] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:44:19.708 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.sdublg{-1:24295} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.27:6826/4177192740,v1:172.21.15.27:6827/4177192740] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:44:19.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.704+0000 7f582ed9e700 1 -- 172.21.15.16:0/1893108049 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f581806de10 msgr2=0x7f58180702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:19.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.704+0000 7f582ed9e700 1 --2- 172.21.15.16:0/1893108049 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f581806de10 0x7f58180702d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5814009c60 tx=0x7f5814009260 comp rx=0 tx=0).stop 2024-04-02T04:44:19.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.704+0000 7f582ed9e700 1 -- 172.21.15.16:0/1893108049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5828100ad0 msgr2=0x7f58281964f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:19.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.704+0000 7f582ed9e700 1 --2- 172.21.15.16:0/1893108049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5828100ad0 0x7f58281964f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f581c005f00 tx=0x7f581c00b5e0 comp rx=0 tx=0).stop 2024-04-02T04:44:19.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.704+0000 7f582ed9e700 1 -- 172.21.15.16:0/1893108049 shutdown_connections 2024-04-02T04:44:19.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.704+0000 7f582ed9e700 1 --2- 172.21.15.16:0/1893108049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5828100ad0 0x7f58281964f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:19.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.704+0000 7f582ed9e700 1 --2- 172.21.15.16:0/1893108049 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f581806de10 0x7f58180702d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:19.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.704+0000 7f582ed9e700 1 --2- 172.21.15.16:0/1893108049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5828101cd0 0x7f5828196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:19.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.704+0000 7f582ed9e700 1 -- 172.21.15.16:0/1893108049 >> 172.21.15.16:0/1893108049 conn(0x7f58280fc090 msgr2=0x7f58281052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:19.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.704+0000 7f582ed9e700 1 -- 172.21.15.16:0/1893108049 shutdown_connections 2024-04-02T04:44:19.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.704+0000 7f582ed9e700 1 -- 172.21.15.16:0/1893108049 wait complete. 2024-04-02T04:44:19.710 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 15 2024-04-02T04:44:19.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.821+0000 7f219d889700 1 -- 172.21.15.16:0/1901462180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2198103f40 msgr2=0x7f2198106330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:19.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.821+0000 7f219d889700 1 --2- 172.21.15.16:0/1901462180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2198103f40 0x7f2198106330 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f218c009910 tx=0x7f218c009c20 comp rx=0 tx=0).stop 2024-04-02T04:44:19.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.821+0000 7f219d889700 1 -- 172.21.15.16:0/1901462180 shutdown_connections 2024-04-02T04:44:19.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.821+0000 7f219d889700 1 --2- 172.21.15.16:0/1901462180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2198103f40 0x7f2198106330 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:19.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.821+0000 7f219d889700 1 --2- 172.21.15.16:0/1901462180 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2198101610 0x7f2198103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:19.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.821+0000 7f219d889700 1 -- 172.21.15.16:0/1901462180 >> 172.21.15.16:0/1901462180 conn(0x7f21980faf80 msgr2=0x7f21980fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:19.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.821+0000 7f219d889700 1 -- 172.21.15.16:0/1901462180 shutdown_connections 2024-04-02T04:44:19.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.821+0000 7f219d889700 1 -- 172.21.15.16:0/1901462180 wait complete. 2024-04-02T04:44:19.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.822+0000 7f219d889700 1 Processor -- start 2024-04-02T04:44:19.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.822+0000 7f219d889700 1 -- start start 2024-04-02T04:44:19.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.822+0000 7f219d889700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2198101610 0x7f21981964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:19.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.822+0000 7f219d889700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2198103f40 0x7f21981969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:19.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.822+0000 7f219d889700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2198197010 con 0x7f2198101610 2024-04-02T04:44:19.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.822+0000 7f219d889700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2198197150 con 0x7f2198103f40 2024-04-02T04:44:19.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.823+0000 7f2196ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2198101610 0x7f21981964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:19.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.823+0000 7f2196ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2198101610 0x7f21981964b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59826/0 (socket says 172.21.15.16:59826) 2024-04-02T04:44:19.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.823+0000 7f2196ffd700 1 -- 172.21.15.16:0/1250041121 learned_addr learned my addr 172.21.15.16:0/1250041121 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:44:19.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.823+0000 7f21967fc700 1 --2- 172.21.15.16:0/1250041121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2198103f40 0x7f21981969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:19.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.823+0000 7f2196ffd700 1 -- 172.21.15.16:0/1250041121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2198103f40 msgr2=0x7f21981969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:19.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.823+0000 7f2196ffd700 1 --2- 172.21.15.16:0/1250041121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2198103f40 0x7f21981969f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:19.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.823+0000 7f2196ffd700 1 -- 172.21.15.16:0/1250041121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f218c0095f0 con 0x7f2198101610 2024-04-02T04:44:19.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.823+0000 7f2196ffd700 1 --2- 172.21.15.16:0/1250041121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2198101610 0x7f21981964b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f218800e920 tx=0x7f218800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:19.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.823+0000 7f219c887700 1 -- 172.21.15.16:0/1250041121 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f218800c9d0 con 0x7f2198101610 2024-04-02T04:44:19.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.823+0000 7f219c887700 1 -- 172.21.15.16:0/1250041121 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2188010430 con 0x7f2198101610 2024-04-02T04:44:19.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.823+0000 7f219d889700 1 -- 172.21.15.16:0/1250041121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f219819bbb0 con 0x7f2198101610 2024-04-02T04:44:19.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.823+0000 7f219c887700 1 -- 172.21.15.16:0/1250041121 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2188018630 con 0x7f2198101610 2024-04-02T04:44:19.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.823+0000 7f219d889700 1 -- 172.21.15.16:0/1250041121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f219819c100 con 0x7f2198101610 2024-04-02T04:44:19.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.824+0000 7f219d889700 1 -- 172.21.15.16:0/1250041121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f219804ea80 con 0x7f2198101610 2024-04-02T04:44:19.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.825+0000 7f219c887700 1 -- 172.21.15.16:0/1250041121 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 36) v1 ==== 91907+0+0 (secure 0 0 0) 0x7f21880105a0 con 0x7f2198101610 2024-04-02T04:44:19.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.825+0000 7f219c887700 1 --2- 172.21.15.16:0/1250041121 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f218006de10 0x7f21800702d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:19.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.826+0000 7f219c887700 1 -- 172.21.15.16:0/1250041121 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f2188014070 con 0x7f2198101610 2024-04-02T04:44:19.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.826+0000 7f21967fc700 1 --2- 172.21.15.16:0/1250041121 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f218006de10 0x7f21800702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:19.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.826+0000 7f21967fc700 1 --2- 172.21.15.16:0/1250041121 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f218006de10 0x7f21800702d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f218c00b800 tx=0x7f218c009f70 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:19.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:19.830+0000 7f219c887700 1 -- 172.21.15.16:0/1250041121 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2188053280 con 0x7f2198101610 2024-04-02T04:44:20.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.077+0000 7f219d889700 1 -- 172.21.15.16:0/1250041121 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f21981048e0 con 0x7f218006de10 2024-04-02T04:44:20.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.078+0000 7f219c887700 1 -- 172.21.15.16:0/1250041121 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f21981048e0 con 0x7f218006de10 2024-04-02T04:44:20.081 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:44:20.081 INFO:teuthology.orchestra.run.smithi016.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:44:20.081 INFO:teuthology.orchestra.run.smithi016.stdout: "in_progress": true, 2024-04-02T04:44:20.082 INFO:teuthology.orchestra.run.smithi016.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-02T04:44:20.082 INFO:teuthology.orchestra.run.smithi016.stdout: "services_complete": [ 2024-04-02T04:44:20.082 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr" 2024-04-02T04:44:20.082 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-04-02T04:44:20.082 INFO:teuthology.orchestra.run.smithi016.stdout: "progress": "2/23 daemons upgraded", 2024-04-02T04:44:20.082 INFO:teuthology.orchestra.run.smithi016.stdout: "message": "", 2024-04-02T04:44:20.082 INFO:teuthology.orchestra.run.smithi016.stdout: "is_paused": false 2024-04-02T04:44:20.082 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:44:20.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.080+0000 7f219d889700 1 -- 172.21.15.16:0/1250041121 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f218006de10 msgr2=0x7f21800702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:20.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.080+0000 7f219d889700 1 --2- 172.21.15.16:0/1250041121 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f218006de10 0x7f21800702d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f218c00b800 tx=0x7f218c009f70 comp rx=0 tx=0).stop 2024-04-02T04:44:20.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.080+0000 7f219d889700 1 -- 172.21.15.16:0/1250041121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2198101610 msgr2=0x7f21981964b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:20.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.080+0000 7f219d889700 1 --2- 172.21.15.16:0/1250041121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2198101610 0x7f21981964b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f218800e920 tx=0x7f218800ec30 comp rx=0 tx=0).stop 2024-04-02T04:44:20.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.080+0000 7f219d889700 1 -- 172.21.15.16:0/1250041121 shutdown_connections 2024-04-02T04:44:20.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.080+0000 7f219d889700 1 --2- 172.21.15.16:0/1250041121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2198101610 0x7f21981964b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:20.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.080+0000 7f219d889700 1 --2- 172.21.15.16:0/1250041121 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f218006de10 0x7f21800702d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:20.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.080+0000 7f219d889700 1 --2- 172.21.15.16:0/1250041121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2198103f40 0x7f21981969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:20.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.080+0000 7f219d889700 1 -- 172.21.15.16:0/1250041121 >> 172.21.15.16:0/1250041121 conn(0x7f21980faf80 msgr2=0x7f21980fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:20.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.081+0000 7f219d889700 1 -- 172.21.15.16:0/1250041121 shutdown_connections 2024-04-02T04:44:20.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.081+0000 7f219d889700 1 -- 172.21.15.16:0/1250041121 wait complete. 2024-04-02T04:44:20.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.193+0000 7fc5fd588700 1 -- 172.21.15.16:0/1451408151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5f8100ad0 msgr2=0x7fc5f8100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:20.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.193+0000 7fc5fd588700 1 --2- 172.21.15.16:0/1451408151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5f8100ad0 0x7fc5f8100ef0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8009910 tx=0x7fc5e8009c20 comp rx=0 tx=0).stop 2024-04-02T04:44:20.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.193+0000 7fc5fd588700 1 -- 172.21.15.16:0/1451408151 shutdown_connections 2024-04-02T04:44:20.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.193+0000 7fc5fd588700 1 --2- 172.21.15.16:0/1451408151 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5f8101cd0 0x7fc5f8102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:20.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.193+0000 7fc5fd588700 1 --2- 172.21.15.16:0/1451408151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5f8100ad0 0x7fc5f8100ef0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:20.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.193+0000 7fc5fd588700 1 -- 172.21.15.16:0/1451408151 >> 172.21.15.16:0/1451408151 conn(0x7fc5f80fc090 msgr2=0x7fc5f80fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:20.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.193+0000 7fc5fd588700 1 -- 172.21.15.16:0/1451408151 shutdown_connections 2024-04-02T04:44:20.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.193+0000 7fc5fd588700 1 -- 172.21.15.16:0/1451408151 wait complete. 2024-04-02T04:44:20.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.194+0000 7fc5fd588700 1 Processor -- start 2024-04-02T04:44:20.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.194+0000 7fc5fd588700 1 -- start start 2024-04-02T04:44:20.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.194+0000 7fc5fd588700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5f8100ad0 0x7fc5f81964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:20.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.194+0000 7fc5fd588700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5f8101cd0 0x7fc5f8196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:20.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.194+0000 7fc5fd588700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f8197050 con 0x7fc5f8101cd0 2024-04-02T04:44:20.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.194+0000 7fc5fd588700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f8197190 con 0x7fc5f8100ad0 2024-04-02T04:44:20.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.194+0000 7fc5f67fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5f8101cd0 0x7fc5f8196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:20.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.194+0000 7fc5f6ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5f8100ad0 0x7fc5f81964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:20.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.194+0000 7fc5f67fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5f8101cd0 0x7fc5f8196a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59850/0 (socket says 172.21.15.16:59850) 2024-04-02T04:44:20.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.194+0000 7fc5f67fc700 1 -- 172.21.15.16:0/2135410344 learned_addr learned my addr 172.21.15.16:0/2135410344 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:44:20.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.194+0000 7fc5f6ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5f8100ad0 0x7fc5f81964f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.16:46020/0 (socket says 172.21.15.16:46020) 2024-04-02T04:44:20.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.194+0000 7fc5f67fc700 1 -- 172.21.15.16:0/2135410344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5f8100ad0 msgr2=0x7fc5f81964f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:20.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.194+0000 7fc5f67fc700 1 --2- 172.21.15.16:0/2135410344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5f8100ad0 0x7fc5f81964f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:20.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.194+0000 7fc5f67fc700 1 -- 172.21.15.16:0/2135410344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e80095f0 con 0x7fc5f8101cd0 2024-04-02T04:44:20.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.194+0000 7fc5f6ffd700 1 --2- 172.21.15.16:0/2135410344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5f8100ad0 0x7fc5f81964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T04:44:20.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.195+0000 7fc5f67fc700 1 --2- 172.21.15.16:0/2135410344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5f8101cd0 0x7fc5f8196a30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec00e920 tx=0x7fc5ec00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:20.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.195+0000 7fc5dffff700 1 -- 172.21.15.16:0/2135410344 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5ec00c9d0 con 0x7fc5f8101cd0 2024-04-02T04:44:20.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.195+0000 7fc5dffff700 1 -- 172.21.15.16:0/2135410344 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc5ec010430 con 0x7fc5f8101cd0 2024-04-02T04:44:20.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.195+0000 7fc5fd588700 1 -- 172.21.15.16:0/2135410344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f819bbf0 con 0x7fc5f8101cd0 2024-04-02T04:44:20.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.195+0000 7fc5dffff700 1 -- 172.21.15.16:0/2135410344 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5ec018630 con 0x7fc5f8101cd0 2024-04-02T04:44:20.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.195+0000 7fc5fd588700 1 -- 172.21.15.16:0/2135410344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f819c1c0 con 0x7fc5f8101cd0 2024-04-02T04:44:20.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.196+0000 7fc5fd588700 1 -- 172.21.15.16:0/2135410344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f8190650 con 0x7fc5f8101cd0 2024-04-02T04:44:20.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.196+0000 7fc5dffff700 1 -- 172.21.15.16:0/2135410344 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 36) v1 ==== 91907+0+0 (secure 0 0 0) 0x7fc5ec0105a0 con 0x7fc5f8101cd0 2024-04-02T04:44:20.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.197+0000 7fc5dffff700 1 --2- 172.21.15.16:0/2135410344 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc5e006de10 0x7fc5e00702d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:20.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.197+0000 7fc5dffff700 1 -- 172.21.15.16:0/2135410344 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fc5ec014070 con 0x7fc5f8101cd0 2024-04-02T04:44:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.197+0000 7fc5f6ffd700 1 --2- 172.21.15.16:0/2135410344 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc5e006de10 0x7fc5e00702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.197+0000 7fc5f6ffd700 1 --2- 172.21.15.16:0/2135410344 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc5e006de10 0x7fc5e00702d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc5e800b800 tx=0x7fc5e8009f70 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:20.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.202+0000 7fc5dffff700 1 -- 172.21.15.16:0/2135410344 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc5ec053280 con 0x7fc5f8101cd0 2024-04-02T04:44:20.226 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:20 smithi027 ceph-mon[37550]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:44:20.226 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:20 smithi027 ceph-mon[37550]: from='client.24617 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:44:20.226 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:20 smithi027 ceph-mon[37550]: pgmap v4: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:44:20.226 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:20 smithi027 ceph-mon[37550]: mgrmap e36: smithi016.eqnebx(active, since 2s), standbys: smithi027.kjzcsc 2024-04-02T04:44:20.226 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:20 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/1776359316' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:44:20.226 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:20 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/1893108049' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:44:20.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:20 smithi016 ceph-mon[93230]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:44:20.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:20 smithi016 ceph-mon[93230]: from='client.24617 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:44:20.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:20 smithi016 ceph-mon[93230]: pgmap v4: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:44:20.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:20 smithi016 ceph-mon[93230]: mgrmap e36: smithi016.eqnebx(active, since 2s), standbys: smithi027.kjzcsc 2024-04-02T04:44:20.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:20 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/1776359316' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:44:20.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:20 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/1893108049' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:44:20.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.553+0000 7fc5fd588700 1 -- 172.21.15.16:0/2135410344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc5f8109630 con 0x7fc5f8101cd0 2024-04-02T04:44:20.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.553+0000 7fc5dffff700 1 -- 172.21.15.16:0/2135410344 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fc5ec05aef0 con 0x7fc5f8101cd0 2024-04-02T04:44:20.558 INFO:teuthology.orchestra.run.smithi016.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-02T04:44:20.558 INFO:teuthology.orchestra.run.smithi016.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-02T04:44:20.558 INFO:teuthology.orchestra.run.smithi016.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-02T04:44:20.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.556+0000 7fc5fd588700 1 -- 172.21.15.16:0/2135410344 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc5e006de10 msgr2=0x7fc5e00702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:20.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.556+0000 7fc5fd588700 1 --2- 172.21.15.16:0/2135410344 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc5e006de10 0x7fc5e00702d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc5e800b800 tx=0x7fc5e8009f70 comp rx=0 tx=0).stop 2024-04-02T04:44:20.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.556+0000 7fc5fd588700 1 -- 172.21.15.16:0/2135410344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5f8101cd0 msgr2=0x7fc5f8196a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:20.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.556+0000 7fc5fd588700 1 --2- 172.21.15.16:0/2135410344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5f8101cd0 0x7fc5f8196a30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec00e920 tx=0x7fc5ec00ec30 comp rx=0 tx=0).stop 2024-04-02T04:44:20.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.556+0000 7fc5fd588700 1 -- 172.21.15.16:0/2135410344 shutdown_connections 2024-04-02T04:44:20.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.556+0000 7fc5fd588700 1 --2- 172.21.15.16:0/2135410344 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc5e006de10 0x7fc5e00702d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:20.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.556+0000 7fc5fd588700 1 --2- 172.21.15.16:0/2135410344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5f8100ad0 0x7fc5f81964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:20.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.556+0000 7fc5fd588700 1 --2- 172.21.15.16:0/2135410344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5f8101cd0 0x7fc5f8196a30 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:20.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.556+0000 7fc5fd588700 1 -- 172.21.15.16:0/2135410344 >> 172.21.15.16:0/2135410344 conn(0x7fc5f80fc090 msgr2=0x7fc5f81052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:20.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.556+0000 7fc5fd588700 1 -- 172.21.15.16:0/2135410344 shutdown_connections 2024-04-02T04:44:20.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:20.556+0000 7fc5fd588700 1 -- 172.21.15.16:0/2135410344 wait complete. 2024-04-02T04:44:21.394 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:21 smithi016 ceph-mon[93230]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:44:21.394 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:21 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/2135410344' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:44:21.395 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:21 smithi027 ceph-mon[37550]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:44:21.395 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:21 smithi027 ceph-mon[37550]: from='client.? 172.21.15.16:0/2135410344' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:44:22.332 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:22 smithi016 ceph-mon[93230]: pgmap v5: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:44:22.332 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:22 smithi016 ceph-mon[93230]: mgrmap e37: smithi016.eqnebx(active, since 4s), standbys: smithi027.kjzcsc 2024-04-02T04:44:22.332 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:22 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:22.332 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:22 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:22.390 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:22 smithi027 ceph-mon[37550]: pgmap v5: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:44:22.390 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:22 smithi027 ceph-mon[37550]: mgrmap e37: smithi016.eqnebx(active, since 4s), standbys: smithi027.kjzcsc 2024-04-02T04:44:22.390 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:22 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:22.390 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:22 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:24.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:24 smithi016 ceph-mon[93230]: pgmap v6: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:44:24.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:24 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:24.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:24 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:24.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:24 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:24.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:24 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:24.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:24 smithi027 ceph-mon[37550]: pgmap v6: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:44:24.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:24 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:24.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:24 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:24.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:24 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:24.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:24 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:26.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:26 smithi016 ceph-mon[93230]: pgmap v7: 65 pgs: 65 active+clean; 310 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 52 KiB/s rd, 1.1 MiB/s wr, 145 op/s 2024-04-02T04:44:26.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:26 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:26.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:26 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:26.643 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:26 smithi027 ceph-mon[37550]: pgmap v7: 65 pgs: 65 active+clean; 310 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 52 KiB/s rd, 1.1 MiB/s wr, 145 op/s 2024-04-02T04:44:26.643 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:26 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:26.644 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:26 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:28.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:28 smithi027 ceph-mon[37550]: pgmap v8: 65 pgs: 65 active+clean; 310 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 40 KiB/s rd, 883 KiB/s wr, 113 op/s 2024-04-02T04:44:28.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:28 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:28.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:28 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:28.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:28 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:44:28.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:28 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:44:28.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:28 smithi027 ceph-mon[37550]: Updating smithi016:/etc/ceph/ceph.conf 2024-04-02T04:44:28.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:28 smithi027 ceph-mon[37550]: Updating smithi027:/etc/ceph/ceph.conf 2024-04-02T04:44:28.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:28 smithi016 ceph-mon[93230]: pgmap v8: 65 pgs: 65 active+clean; 310 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 40 KiB/s rd, 883 KiB/s wr, 113 op/s 2024-04-02T04:44:28.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:28 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:28.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:28 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:28.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:28 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:44:28.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:28 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:44:28.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:28 smithi016 ceph-mon[93230]: Updating smithi016:/etc/ceph/ceph.conf 2024-04-02T04:44:28.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:28 smithi016 ceph-mon[93230]: Updating smithi027:/etc/ceph/ceph.conf 2024-04-02T04:44:29.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:29 smithi027 ceph-mon[37550]: Updating smithi027:/var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:44:29.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:29 smithi027 ceph-mon[37550]: Updating smithi016:/var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:44:29.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:29 smithi027 ceph-mon[37550]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-04-02T04:44:29.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:29 smithi027 ceph-mon[37550]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-04-02T04:44:29.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:29 smithi027 ceph-mon[37550]: Updating smithi027:/var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T04:44:29.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:29 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:29.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:29 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:29.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:29 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:29.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:29 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:29.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:29 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:29.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:29 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:44:29.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:29 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:44:29.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:29 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-02T04:44:29.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:29 smithi016 ceph-mon[93230]: Updating smithi027:/var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:44:29.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:29 smithi016 ceph-mon[93230]: Updating smithi016:/var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T04:44:29.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:29 smithi016 ceph-mon[93230]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-04-02T04:44:29.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:29 smithi016 ceph-mon[93230]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-04-02T04:44:29.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:29 smithi016 ceph-mon[93230]: Updating smithi027:/var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T04:44:29.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:29 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:29.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:29 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:29.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:29 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:29.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:29 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:29.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:29 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:29.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:29 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:44:29.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:29 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:44:29.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:29 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-02T04:44:30.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:30 smithi016 ceph-mon[93230]: Updating smithi016:/var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T04:44:30.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:30 smithi016 ceph-mon[93230]: pgmap v9: 65 pgs: 65 active+clean; 292 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 48 KiB/s rd, 1.3 MiB/s wr, 175 op/s 2024-04-02T04:44:30.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:30 smithi027 ceph-mon[37550]: Updating smithi016:/var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T04:44:30.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:30 smithi027 ceph-mon[37550]: pgmap v9: 65 pgs: 65 active+clean; 292 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 48 KiB/s rd, 1.3 MiB/s wr, 175 op/s 2024-04-02T04:44:32.177 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:32 smithi027 ceph-mon[37550]: pgmap v10: 65 pgs: 65 active+clean; 292 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 44 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-04-02T04:44:32.177 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:32 smithi027 ceph-mon[37550]: Upgrade: Updating mon.smithi027 2024-04-02T04:44:32.177 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:32 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:32.177 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:32 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T04:44:32.177 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:32 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T04:44:32.177 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:32 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:44:32.177 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:32 smithi027 ceph-mon[37550]: Deploying daemon mon.smithi027 on smithi027 2024-04-02T04:44:32.177 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:32 smithi027 ceph-mon[37550]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:44:32.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:32 smithi016 ceph-mon[93230]: pgmap v10: 65 pgs: 65 active+clean; 292 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 44 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-04-02T04:44:32.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:32 smithi016 ceph-mon[93230]: Upgrade: Updating mon.smithi027 2024-04-02T04:44:32.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:32 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:32.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:32 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T04:44:32.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:32 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T04:44:32.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:32 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:44:32.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:32 smithi016 ceph-mon[93230]: Deploying daemon mon.smithi027 on smithi027 2024-04-02T04:44:32.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:32 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:44:33.303 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:33 smithi027 systemd[1]: Stopping Ceph mon.smithi027 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4... 2024-04-02T04:44:33.870 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:33 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi027[37522]: 2024-04-02T04:44:33.600+0000 7f26a25eb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi027 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T04:44:33.870 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:33 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi027[37522]: 2024-04-02T04:44:33.600+0000 7f26a25eb700 -1 mon.smithi027@1(peon) e2 *** Got Signal Terminated *** 2024-04-02T04:44:34.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:33 smithi027 podman[82876]: 2024-04-02 04:44:33.867035876 +0000 UTC m=+0.386308087 container died b77563184a1e85886b48e078ea5bcaaa6a23dcf9594b86bafda62c96cb1b39d8 (image=quay.ceph.io/ceph-ci/ceph@sha256:397d0c1d908c002c558906e3ea05982bc336d403ee4d900400cc3bdd9243aa2e, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi027, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, RELEASE=reef-4d05f4c, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , ceph=True) 2024-04-02T04:44:34.546 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:34 smithi027 podman[82876]: 2024-04-02 04:44:34.267397541 +0000 UTC m=+0.786669738 container cleanup b77563184a1e85886b48e078ea5bcaaa6a23dcf9594b86bafda62c96cb1b39d8 (image=quay.ceph.io/ceph-ci/ceph@sha256:397d0c1d908c002c558906e3ea05982bc336d403ee4d900400cc3bdd9243aa2e, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi027, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, RELEASE=reef-4d05f4c, maintainer=Guillaume Abrioux ) 2024-04-02T04:44:34.546 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:34 smithi027 bash[82876]: ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi027 2024-04-02T04:44:34.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:34 smithi027 podman[82890]: 2024-04-02 04:44:34.544052713 +0000 UTC m=+0.678904232 container remove b77563184a1e85886b48e078ea5bcaaa6a23dcf9594b86bafda62c96cb1b39d8 (image=quay.ceph.io/ceph-ci/ceph@sha256:397d0c1d908c002c558906e3ea05982bc336d403ee4d900400cc3bdd9243aa2e, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi027, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=reef-4d05f4c, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-02T04:44:34.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:34 smithi027 systemd[1]: ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@mon.smithi027.service: Deactivated successfully. 2024-04-02T04:44:34.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:34 smithi027 systemd[1]: Stopped Ceph mon.smithi027 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4. 2024-04-02T04:44:34.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:34 smithi027 systemd[1]: ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@mon.smithi027.service: Consumed 7.032s CPU time. 2024-04-02T04:44:35.306 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 systemd[1]: Starting Ceph mon.smithi027 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4... 2024-04-02T04:44:35.306 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 podman[83017]: 2024-04-02 04:44:35.153970116 +0000 UTC m=+0.019778506 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:44:35.582 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 podman[83017]: 2024-04-02 04:44:35.303893327 +0000 UTC m=+0.169701689 container create fcd68b20dcc5e6d1d630af96bc8cb0ac28513a73371305294bea2f8d6f363a08 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi027, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-6cec793, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-02T04:44:35.582 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 podman[83017]: 2024-04-02 04:44:35.580218143 +0000 UTC m=+0.446026521 container init fcd68b20dcc5e6d1d630af96bc8cb0ac28513a73371305294bea2f8d6f363a08 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi027, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=squid-6cec793, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-02T04:44:35.900 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 podman[83017]: 2024-04-02 04:44:35.585337869 +0000 UTC m=+0.451146238 container start fcd68b20dcc5e6d1d630af96bc8cb0ac28513a73371305294bea2f8d6f363a08 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi027, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240326, RELEASE=squid-6cec793, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-02T04:44:35.900 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: set uid:gid to 167:167 (ceph:ceph) 2024-04-02T04:44:35.900 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev), process ceph-mon, pid 2 2024-04-02T04:44:35.900 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: pidfile_write: ignore empty --pid-file 2024-04-02T04:44:35.900 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: load: jerasure load: lrc 2024-04-02T04:44:35.900 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: RocksDB version: 7.9.2 2024-04-02T04:44:35.900 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Git sha 0 2024-04-02T04:44:35.900 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Compile date 2024-04-01 15:23:26 2024-04-02T04:44:35.901 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: DB SUMMARY 2024-04-02T04:44:35.901 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: DB Session ID: WRVDBLON3AAFFR6J42PV 2024-04-02T04:44:35.901 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: CURRENT file: CURRENT 2024-04-02T04:44:35.901 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: IDENTITY file: IDENTITY 2024-04-02T04:44:35.901 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: MANIFEST file: MANIFEST-000010 size: 1404 Bytes 2024-04-02T04:44:35.901 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi027/store.db dir, Total Num: 1, files: 000027.sst 2024-04-02T04:44:35.901 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi027/store.db: 000025.log size: 4669586 ; 2024-04-02T04:44:35.901 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.error_if_exists: 0 2024-04-02T04:44:35.901 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.create_if_missing: 0 2024-04-02T04:44:35.901 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.paranoid_checks: 1 2024-04-02T04:44:35.901 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-02T04:44:35.901 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-02T04:44:35.901 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-02T04:44:35.901 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.env: 0x556a2f8c8c60 2024-04-02T04:44:35.902 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.fs: PosixFileSystem 2024-04-02T04:44:35.902 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.info_log: 0x556a30a179a0 2024-04-02T04:44:35.902 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.max_file_opening_threads: 16 2024-04-02T04:44:35.902 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.statistics: (nil) 2024-04-02T04:44:35.902 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.use_fsync: 0 2024-04-02T04:44:35.902 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.max_log_file_size: 0 2024-04-02T04:44:35.902 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-02T04:44:35.902 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-02T04:44:35.902 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.keep_log_file_num: 1000 2024-04-02T04:44:35.902 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.recycle_log_file_num: 0 2024-04-02T04:44:35.902 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.allow_fallocate: 1 2024-04-02T04:44:35.902 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.allow_mmap_reads: 0 2024-04-02T04:44:35.902 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.allow_mmap_writes: 0 2024-04-02T04:44:35.902 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.use_direct_reads: 0 2024-04-02T04:44:35.903 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-02T04:44:35.903 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.create_missing_column_families: 0 2024-04-02T04:44:35.903 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.db_log_dir: 2024-04-02T04:44:35.903 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.wal_dir: 2024-04-02T04:44:35.903 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-02T04:44:35.903 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-02T04:44:35.903 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-02T04:44:35.903 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-02T04:44:35.903 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-02T04:44:35.903 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-02T04:44:35.903 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.advise_random_on_open: 1 2024-04-02T04:44:35.903 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.db_write_buffer_size: 0 2024-04-02T04:44:35.903 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.write_buffer_manager: 0x556a309ea1e0 2024-04-02T04:44:35.903 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-02T04:44:35.903 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-02T04:44:35.903 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-02T04:44:35.903 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.rate_limiter: (nil) 2024-04-02T04:44:35.904 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-02T04:44:35.904 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.wal_recovery_mode: 2 2024-04-02T04:44:35.904 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.enable_thread_tracking: 0 2024-04-02T04:44:35.904 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.enable_pipelined_write: 0 2024-04-02T04:44:35.904 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.unordered_write: 0 2024-04-02T04:44:35.904 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-02T04:44:35.904 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-02T04:44:35.904 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-02T04:44:35.904 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-02T04:44:35.904 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.row_cache: None 2024-04-02T04:44:35.904 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.wal_filter: None 2024-04-02T04:44:35.904 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-02T04:44:35.904 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.allow_ingest_behind: 0 2024-04-02T04:44:35.904 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.two_write_queues: 0 2024-04-02T04:44:35.904 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.manual_wal_flush: 0 2024-04-02T04:44:35.904 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.wal_compression: 0 2024-04-02T04:44:35.905 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.atomic_flush: 0 2024-04-02T04:44:35.905 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-02T04:44:35.905 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-02T04:44:35.905 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-02T04:44:35.905 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.log_readahead_size: 0 2024-04-02T04:44:35.905 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-02T04:44:35.905 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.best_efforts_recovery: 0 2024-04-02T04:44:35.905 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-02T04:44:35.905 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-02T04:44:35.905 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.allow_data_in_errors: 0 2024-04-02T04:44:35.905 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.db_host_id: __hostname__ 2024-04-02T04:44:35.905 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-02T04:44:35.905 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.max_background_jobs: 2 2024-04-02T04:44:35.905 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.max_background_compactions: -1 2024-04-02T04:44:35.906 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.max_subcompactions: 1 2024-04-02T04:44:35.906 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-02T04:44:35.906 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-02T04:44:35.906 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-02T04:44:35.906 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.max_total_wal_size: 0 2024-04-02T04:44:35.906 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-02T04:44:35.906 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-02T04:44:35.906 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-02T04:44:35.906 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-02T04:44:35.907 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.max_open_files: -1 2024-04-02T04:44:35.907 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.bytes_per_sync: 0 2024-04-02T04:44:35.907 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-02T04:44:35.907 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-02T04:44:35.907 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.compaction_readahead_size: 0 2024-04-02T04:44:35.907 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.max_background_flushes: -1 2024-04-02T04:44:35.907 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Compression algorithms supported: 2024-04-02T04:44:35.907 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-02T04:44:35.907 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: kZSTD supported: 0 2024-04-02T04:44:35.907 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: kXpressCompression supported: 0 2024-04-02T04:44:35.907 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: kLZ4HCCompression supported: 1 2024-04-02T04:44:35.907 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: kZlibCompression supported: 1 2024-04-02T04:44:35.908 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: kSnappyCompression supported: 1 2024-04-02T04:44:35.908 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: kLZ4Compression supported: 1 2024-04-02T04:44:35.908 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: kBZip2Compression supported: 0 2024-04-02T04:44:35.908 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-02T04:44:35.908 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-02T04:44:35.908 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi027/store.db/MANIFEST-000010 2024-04-02T04:44:35.908 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-02T04:44:35.908 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-02T04:44:35.908 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.merge_operator: 2024-04-02T04:44:35.908 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.compaction_filter: None 2024-04-02T04:44:35.908 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.compaction_filter_factory: None 2024-04-02T04:44:35.908 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.sst_partitioner_factory: None 2024-04-02T04:44:35.908 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-02T04:44:35.908 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-02T04:44:35.909 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556a30a17aa0) 2024-04-02T04:44:35.909 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: cache_index_and_filter_blocks: 1 2024-04-02T04:44:35.909 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-02T04:44:35.909 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-02T04:44:35.909 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: pin_top_level_index_and_filter: 1 2024-04-02T04:44:35.909 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: index_type: 0 2024-04-02T04:44:35.909 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: data_block_index_type: 0 2024-04-02T04:44:35.909 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: index_shortening: 1 2024-04-02T04:44:35.909 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-02T04:44:35.909 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: checksum: 4 2024-04-02T04:44:35.909 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: no_block_cache: 0 2024-04-02T04:44:35.909 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: block_cache: 0x556a30a1b090 2024-04-02T04:44:35.909 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: block_cache_name: BinnedLRUCache 2024-04-02T04:44:35.909 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: block_cache_options: 2024-04-02T04:44:35.910 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: capacity : 536870912 2024-04-02T04:44:35.910 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: num_shard_bits : 4 2024-04-02T04:44:35.910 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: strict_capacity_limit : 0 2024-04-02T04:44:35.910 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: high_pri_pool_ratio: 0.000 2024-04-02T04:44:35.910 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: block_cache_compressed: (nil) 2024-04-02T04:44:35.910 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: persistent_cache: (nil) 2024-04-02T04:44:35.910 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: block_size: 4096 2024-04-02T04:44:35.910 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: block_size_deviation: 10 2024-04-02T04:44:35.910 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: block_restart_interval: 16 2024-04-02T04:44:35.910 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: index_block_restart_interval: 1 2024-04-02T04:44:35.910 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: metadata_block_size: 4096 2024-04-02T04:44:35.910 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: partition_filters: 0 2024-04-02T04:44:35.910 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: use_delta_encoding: 1 2024-04-02T04:44:35.910 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: filter_policy: bloomfilter 2024-04-02T04:44:35.910 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: whole_key_filtering: 1 2024-04-02T04:44:35.911 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: verify_compression: 0 2024-04-02T04:44:35.911 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: read_amp_bytes_per_bit: 0 2024-04-02T04:44:35.911 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: format_version: 5 2024-04-02T04:44:35.911 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: enable_index_compression: 1 2024-04-02T04:44:35.911 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: block_align: 0 2024-04-02T04:44:35.911 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: max_auto_readahead_size: 262144 2024-04-02T04:44:35.911 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: prepopulate_block_cache: 0 2024-04-02T04:44:35.911 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: initial_auto_readahead_size: 8192 2024-04-02T04:44:35.911 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: num_file_reads_for_auto_readahead: 2 2024-04-02T04:44:35.911 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.write_buffer_size: 33554432 2024-04-02T04:44:35.911 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.max_write_buffer_number: 2 2024-04-02T04:44:35.911 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.compression: NoCompression 2024-04-02T04:44:35.911 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.bottommost_compression: Disabled 2024-04-02T04:44:35.911 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.prefix_extractor: nullptr 2024-04-02T04:44:35.911 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-02T04:44:35.911 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.num_levels: 7 2024-04-02T04:44:35.912 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-02T04:44:35.912 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-02T04:44:35.912 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-02T04:44:35.912 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-02T04:44:35.912 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-02T04:44:35.912 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-02T04:44:35.912 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-02T04:44:35.912 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-02T04:44:35.912 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-02T04:44:35.912 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-02T04:44:35.912 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-02T04:44:35.912 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-02T04:44:35.912 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-02T04:44:35.912 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.compression_opts.level: 32767 2024-04-02T04:44:35.913 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.compression_opts.strategy: 0 2024-04-02T04:44:35.913 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-02T04:44:35.913 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-02T04:44:35.913 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-02T04:44:35.913 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-02T04:44:35.913 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.compression_opts.enabled: false 2024-04-02T04:44:35.913 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-02T04:44:35.913 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-02T04:44:35.913 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-02T04:44:35.913 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-02T04:44:35.913 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.target_file_size_base: 67108864 2024-04-02T04:44:35.913 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-02T04:44:35.913 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-02T04:44:35.913 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-02T04:44:35.914 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-02T04:44:35.914 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-02T04:44:35.914 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-02T04:44:35.914 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-02T04:44:35.914 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-02T04:44:35.914 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-02T04:44:35.914 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-02T04:44:35.914 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-02T04:44:35.914 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-02T04:44:35.914 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-02T04:44:35.914 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-02T04:44:35.915 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.arena_block_size: 1048576 2024-04-02T04:44:35.915 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-02T04:44:35.915 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-02T04:44:35.915 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.disable_auto_compactions: 0 2024-04-02T04:44:35.916 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-02T04:44:35.916 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-02T04:44:35.916 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-02T04:44:35.916 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-02T04:44:35.916 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-02T04:44:35.916 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-02T04:44:35.916 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-02T04:44:35.916 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-02T04:44:35.916 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-02T04:44:35.916 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-02T04:44:35.916 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.table_properties_collectors: 2024-04-02T04:44:35.916 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.inplace_update_support: 0 2024-04-02T04:44:35.916 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-02T04:44:35.917 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-02T04:44:35.917 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-02T04:44:35.917 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-02T04:44:35.917 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.bloom_locality: 0 2024-04-02T04:44:35.917 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.max_successive_merges: 0 2024-04-02T04:44:35.917 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-02T04:44:35.917 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.paranoid_file_checks: 0 2024-04-02T04:44:35.917 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.force_consistency_checks: 1 2024-04-02T04:44:35.917 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.report_bg_io_stats: 0 2024-04-02T04:44:35.917 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.ttl: 2592000 2024-04-02T04:44:35.917 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-02T04:44:35.917 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-02T04:44:35.917 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-02T04:44:35.917 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.enable_blob_files: false 2024-04-02T04:44:35.918 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.min_blob_size: 0 2024-04-02T04:44:35.918 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.blob_file_size: 268435456 2024-04-02T04:44:35.918 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-02T04:44:35.918 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-02T04:44:35.918 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-02T04:44:35.918 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-02T04:44:35.918 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-02T04:44:35.918 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.blob_file_starting_level: 0 2024-04-02T04:44:35.918 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-02T04:44:35.918 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-02T04:44:35.918 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi027/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 29, last_sequence is 12090, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 25 2024-04-02T04:44:35.918 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-04-02T04:44:35.918 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: de102378-e598-4743-8929-633e7d254a4b 2024-04-02T04:44:35.918 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712033075622413, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-04-02T04:44:35.919 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-04-02T04:44:35.919 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712033075637460, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 2870423, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12095, "largest_seqno": 13494, "table_properties": {"data_size": 2864108, "index_size": 3888, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1413, "raw_key_size": 12981, "raw_average_key_size": 23, "raw_value_size": 2851914, "raw_average_value_size": 5120, "num_data_blocks": 178, "num_entries": 557, "num_filter_entries": 557, "num_deletions": 2, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1712033075, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "de102378-e598-4743-8929-633e7d254a4b", "db_session_id": "WRVDBLON3AAFFR6J42PV", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-04-02T04:44:35.919 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712033075637571, "job": 1, "event": "recovery_finished"} 2024-04-02T04:44:35.919 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-04-02T04:44:35.919 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-02T04:44:35.919 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi027/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-02T04:44:35.919 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556a30b0c000 2024-04-02T04:44:35.919 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: DB pointer 0x556a30af6000 2024-04-02T04:44:35.919 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-02T04:44:35.919 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-02T04:44:35.919 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: ** DB Stats ** 2024-04-02T04:44:35.919 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-02T04:44:35.919 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-02T04:44:35.919 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-02T04:44:35.920 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-02T04:44:35.920 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-02T04:44:35.920 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-02T04:44:35.920 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-02T04:44:35.920 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: 2024-04-02T04:44:35.920 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: ** Compaction Stats [default] ** 2024-04-02T04:44:35.920 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-02T04:44:35.920 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-02T04:44:35.920 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: L0 1/0 2.74 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 481.9 0.01 0.00 1 0.006 0 0 0.0 0.0 2024-04-02T04:44:35.920 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: L6 1/0 5.63 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-04-02T04:44:35.920 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Sum 2/0 8.37 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 481.9 0.01 0.00 1 0.006 0 0 0.0 0.0 2024-04-02T04:44:35.920 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 481.9 0.01 0.00 1 0.006 0 0 0.0 0.0 2024-04-02T04:44:35.920 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: 2024-04-02T04:44:35.920 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: ** Compaction Stats [default] ** 2024-04-02T04:44:35.921 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-02T04:44:35.921 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-02T04:44:35.921 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 481.9 0.01 0.00 1 0.006 0 0 0.0 0.0 2024-04-02T04:44:35.921 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: 2024-04-02T04:44:35.921 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-02T04:44:35.921 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: 2024-04-02T04:44:35.921 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-02T04:44:35.921 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Flush(GB): cumulative 0.003, interval 0.003 2024-04-02T04:44:35.921 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-02T04:44:35.921 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-02T04:44:35.921 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-02T04:44:35.921 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-02T04:44:35.921 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Cumulative compaction: 0.00 GB write, 126.37 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-02T04:44:35.922 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Interval compaction: 0.00 GB write, 126.37 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-02T04:44:35.922 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-02T04:44:35.922 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Block cache BinnedLRUCache@0x556a30a1b090#2 capacity: 512.00 MB usage: 5.61 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.3e-05 secs_since: 0 2024-04-02T04:44:35.922 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.53 KB,0.000292063%) IndexBlock(1,4.08 KB,0.000777841%) Misc(1,0.00 KB,0%) 2024-04-02T04:44:35.922 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: 2024-04-02T04:44:35.922 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-02T04:44:35.922 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: starting mon.smithi027 rank 1 at public addrs [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] at bind addrs [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi027 fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 2024-04-02T04:44:35.922 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: mon.smithi027@-1(???) e2 preinit fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 2024-04-02T04:44:35.922 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: mon.smithi027@-1(???).mds e15 new map 2024-04-02T04:44:35.922 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: mon.smithi027@-1(???).mds e15 print_map 2024-04-02T04:44:35.922 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: e15 2024-04-02T04:44:35.922 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:44:35.922 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:44:35.923 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: legacy client fscid: 1 2024-04-02T04:44:35.923 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: 2024-04-02T04:44:35.923 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Filesystem 'cephfs' (1) 2024-04-02T04:44:35.923 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: fs_name cephfs 2024-04-02T04:44:35.923 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: epoch 15 2024-04-02T04:44:35.923 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:44:35.923 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: created 2024-04-02T04:39:30.740168+0000 2024-04-02T04:44:35.923 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: modified 2024-04-02T04:39:49.875319+0000 2024-04-02T04:44:35.923 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: tableserver 0 2024-04-02T04:44:35.923 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: root 0 2024-04-02T04:44:35.923 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: session_timeout 60 2024-04-02T04:44:35.923 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: session_autoclose 300 2024-04-02T04:44:35.923 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: max_file_size 1099511627776 2024-04-02T04:44:35.923 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: max_xattr_size 65536 2024-04-02T04:44:35.923 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: required_client_features {} 2024-04-02T04:44:35.924 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: last_failure 0 2024-04-02T04:44:35.924 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: last_failure_osd_epoch 46 2024-04-02T04:44:35.924 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:44:35.924 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: max_mds 1 2024-04-02T04:44:35.924 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: in 0 2024-04-02T04:44:35.924 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: up {0=24285} 2024-04-02T04:44:35.924 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: failed 2024-04-02T04:44:35.924 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: damaged 2024-04-02T04:44:35.924 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: stopped 2024-04-02T04:44:35.924 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: data_pools [3] 2024-04-02T04:44:35.924 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: metadata_pool 2 2024-04-02T04:44:35.924 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: inline_data enabled 2024-04-02T04:44:35.924 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: balancer 2024-04-02T04:44:35.924 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: bal_rank_mask -1 2024-04-02T04:44:35.924 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: standby_count_wanted 1 2024-04-02T04:44:35.925 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: qdb_cluster leader: 0 members: 2024-04-02T04:44:35.925 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: [mds.cephfs.smithi027.zwrfoo{0:24285} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.27:6824/1461455976,v1:172.21.15.27:6825/1461455976] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:44:35.925 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: 2024-04-02T04:44:35.925 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: 2024-04-02T04:44:35.925 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Standby daemons: 2024-04-02T04:44:35.925 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: 2024-04-02T04:44:35.925 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: [mds.cephfs.smithi016.fyzrnm{-1:14488} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.16:6828/2578719859,v1:172.21.15.16:6829/2578719859] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:44:35.925 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: [mds.cephfs.smithi016.uhgkgg{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.16:6826/601263579,v1:172.21.15.16:6827/601263579] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:44:35.925 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: [mds.cephfs.smithi027.sdublg{-1:24295} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.27:6826/4177192740,v1:172.21.15.27:6827/4177192740] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:44:35.925 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: mon.smithi027@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-04-02T04:44:35.925 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: mon.smithi027@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-04-02T04:44:35.925 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: mon.smithi027@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-04-02T04:44:35.925 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: mon.smithi027@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-04-02T04:44:35.925 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 ceph-mon[83036]: mon.smithi027@-1(???).paxosservice(auth 1..24) refresh upgraded, format 0 -> 3 2024-04-02T04:44:35.925 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 bash[83017]: fcd68b20dcc5e6d1d630af96bc8cb0ac28513a73371305294bea2f8d6f363a08 2024-04-02T04:44:35.926 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:35 smithi027 systemd[1]: Started Ceph mon.smithi027 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4. 2024-04-02T04:44:37.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:37 smithi027 ceph-mon[83036]: pgmap v11: 65 pgs: 65 active+clean; 292 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 44 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-04-02T04:44:37.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:37 smithi016 ceph-mon[93230]: pgmap v11: 65 pgs: 65 active+clean; 292 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 44 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-04-02T04:44:38.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:38 smithi027 ceph-mon[83036]: pgmap v13: 65 pgs: 65 active+clean; 288 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 1.2 MiB/s wr, 162 op/s 2024-04-02T04:44:38.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:38 smithi027 ceph-mon[83036]: fsmap cephfs:1 {0=cephfs.smithi027.zwrfoo=up:active} 3 up:standby 2024-04-02T04:44:38.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:38 smithi027 ceph-mon[83036]: osdmap e49: 6 total, 6 up, 6 in 2024-04-02T04:44:38.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:38 smithi027 ceph-mon[83036]: mgrmap e37: smithi016.eqnebx(active, since 20s), standbys: smithi027.kjzcsc 2024-04-02T04:44:38.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:38 smithi027 ceph-mon[83036]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-02T04:44:38.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:38 smithi027 ceph-mon[83036]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-02T04:44:38.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:38 smithi027 ceph-mon[83036]: fs cephfs has deprecated feature inline_data enabled. 2024-04-02T04:44:38.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:38 smithi027 ceph-mon[83036]: mon.smithi016 calling monitor election 2024-04-02T04:44:38.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:38 smithi027 ceph-mon[83036]: mon.smithi027 calling monitor election 2024-04-02T04:44:38.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:38 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi016"}]: dispatch 2024-04-02T04:44:38.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:38 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2024-04-02T04:44:38.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:38 smithi027 ceph-mon[83036]: mon.smithi016 is new leader, mons smithi016,smithi027 in quorum (ranks 0,1) 2024-04-02T04:44:38.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:38 smithi027 ceph-mon[83036]: monmap epoch 3 2024-04-02T04:44:38.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:38 smithi027 ceph-mon[83036]: fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 2024-04-02T04:44:38.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:38 smithi027 ceph-mon[83036]: last_changed 2024-04-02T04:44:37.142925+0000 2024-04-02T04:44:38.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:38 smithi027 ceph-mon[83036]: created 2024-04-02T04:32:20.006984+0000 2024-04-02T04:44:38.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:38 smithi027 ceph-mon[83036]: min_mon_release 19 (squid) 2024-04-02T04:44:38.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:38 smithi027 ceph-mon[83036]: election_strategy: 1 2024-04-02T04:44:38.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:38 smithi027 ceph-mon[83036]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.smithi016 2024-04-02T04:44:38.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:38 smithi027 ceph-mon[83036]: 1: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.smithi027 2024-04-02T04:44:38.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:38 smithi027 ceph-mon[83036]: fsmap cephfs:1 {0=cephfs.smithi027.zwrfoo=up:active} 3 up:standby 2024-04-02T04:44:38.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:38 smithi027 ceph-mon[83036]: osdmap e49: 6 total, 6 up, 6 in 2024-04-02T04:44:38.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:38 smithi027 ceph-mon[83036]: mgrmap e37: smithi016.eqnebx(active, since 20s), standbys: smithi027.kjzcsc 2024-04-02T04:44:38.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:38 smithi027 ceph-mon[83036]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-02T04:44:38.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:38 smithi027 ceph-mon[83036]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-02T04:44:38.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:38 smithi027 ceph-mon[83036]: fs cephfs has deprecated feature inline_data enabled. 2024-04-02T04:44:38.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:38 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:38.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:38 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:38.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:38 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:44:38.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:38 smithi016 ceph-mon[93230]: pgmap v13: 65 pgs: 65 active+clean; 288 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 1.2 MiB/s wr, 162 op/s 2024-04-02T04:44:38.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:38 smithi016 ceph-mon[93230]: fsmap cephfs:1 {0=cephfs.smithi027.zwrfoo=up:active} 3 up:standby 2024-04-02T04:44:38.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:38 smithi016 ceph-mon[93230]: osdmap e49: 6 total, 6 up, 6 in 2024-04-02T04:44:38.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:38 smithi016 ceph-mon[93230]: mgrmap e37: smithi016.eqnebx(active, since 20s), standbys: smithi027.kjzcsc 2024-04-02T04:44:38.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:38 smithi016 ceph-mon[93230]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-02T04:44:38.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:38 smithi016 ceph-mon[93230]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-02T04:44:38.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:38 smithi016 ceph-mon[93230]: fs cephfs has deprecated feature inline_data enabled. 2024-04-02T04:44:38.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:38 smithi016 ceph-mon[93230]: mon.smithi016 calling monitor election 2024-04-02T04:44:38.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:38 smithi016 ceph-mon[93230]: mon.smithi027 calling monitor election 2024-04-02T04:44:38.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:38 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi016"}]: dispatch 2024-04-02T04:44:38.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:38 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2024-04-02T04:44:38.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:38 smithi016 ceph-mon[93230]: mon.smithi016 is new leader, mons smithi016,smithi027 in quorum (ranks 0,1) 2024-04-02T04:44:38.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:38 smithi016 ceph-mon[93230]: monmap epoch 3 2024-04-02T04:44:38.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:38 smithi016 ceph-mon[93230]: fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 2024-04-02T04:44:38.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:38 smithi016 ceph-mon[93230]: last_changed 2024-04-02T04:44:37.142925+0000 2024-04-02T04:44:38.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:38 smithi016 ceph-mon[93230]: created 2024-04-02T04:32:20.006984+0000 2024-04-02T04:44:38.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:38 smithi016 ceph-mon[93230]: min_mon_release 19 (squid) 2024-04-02T04:44:38.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:38 smithi016 ceph-mon[93230]: election_strategy: 1 2024-04-02T04:44:38.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:38 smithi016 ceph-mon[93230]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.smithi016 2024-04-02T04:44:38.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:38 smithi016 ceph-mon[93230]: 1: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.smithi027 2024-04-02T04:44:38.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:38 smithi016 ceph-mon[93230]: fsmap cephfs:1 {0=cephfs.smithi027.zwrfoo=up:active} 3 up:standby 2024-04-02T04:44:38.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:38 smithi016 ceph-mon[93230]: osdmap e49: 6 total, 6 up, 6 in 2024-04-02T04:44:38.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:38 smithi016 ceph-mon[93230]: mgrmap e37: smithi016.eqnebx(active, since 20s), standbys: smithi027.kjzcsc 2024-04-02T04:44:38.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:38 smithi016 ceph-mon[93230]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-02T04:44:38.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:38 smithi016 ceph-mon[93230]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-02T04:44:38.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:38 smithi016 ceph-mon[93230]: fs cephfs has deprecated feature inline_data enabled. 2024-04-02T04:44:38.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:38 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:38.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:38 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:38.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:38 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:44:38.710 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-04-02T04:44:38.710 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-02T04:44:39.061 DEBUG:teuthology.parallel:result is None 2024-04-02T04:44:40.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:40 smithi016 ceph-mon[93230]: pgmap v14: 65 pgs: 65 active+clean; 279 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 40 KiB/s rd, 1.8 MiB/s wr, 263 op/s 2024-04-02T04:44:40.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:40 smithi027 ceph-mon[83036]: pgmap v14: 65 pgs: 65 active+clean; 279 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 40 KiB/s rd, 1.8 MiB/s wr, 263 op/s 2024-04-02T04:44:41.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:41 smithi027 ceph-mon[83036]: pgmap v15: 65 pgs: 65 active+clean; 279 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.2 MiB/s wr, 187 op/s 2024-04-02T04:44:42.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:41 smithi016 ceph-mon[93230]: pgmap v15: 65 pgs: 65 active+clean; 279 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.2 MiB/s wr, 187 op/s 2024-04-02T04:44:43.860 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-04-02T04:44:43.860 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-04-02T04:44:44.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:44 smithi027 ceph-mon[83036]: pgmap v16: 65 pgs: 65 active+clean; 279 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.2 MiB/s wr, 187 op/s 2024-04-02T04:44:44.242 DEBUG:teuthology.parallel:result is None 2024-04-02T04:44:44.243 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-02T04:44:44.282 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-02T04:44:44.283 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-04-02T04:44:44.322 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-04-02T04:44:44.323 DEBUG:teuthology.parallel:result is None 2024-04-02T04:44:44.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:44 smithi016 ceph-mon[93230]: pgmap v16: 65 pgs: 65 active+clean; 279 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.2 MiB/s wr, 187 op/s 2024-04-02T04:44:46.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:46 smithi027 ceph-mon[83036]: pgmap v17: 65 pgs: 65 active+clean; 267 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 38 KiB/s rd, 1.6 MiB/s wr, 267 op/s 2024-04-02T04:44:46.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:46 smithi016 ceph-mon[93230]: pgmap v17: 65 pgs: 65 active+clean; 267 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 38 KiB/s rd, 1.6 MiB/s wr, 267 op/s 2024-04-02T04:44:48.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:48 smithi027 ceph-mon[83036]: pgmap v18: 65 pgs: 65 active+clean; 267 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 25 KiB/s rd, 1.0 MiB/s wr, 180 op/s 2024-04-02T04:44:48.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:48 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:44:48.445 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:48 smithi016 ceph-mon[93230]: pgmap v18: 65 pgs: 65 active+clean; 267 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 25 KiB/s rd, 1.0 MiB/s wr, 180 op/s 2024-04-02T04:44:48.445 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:48 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:44:50.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:50 smithi016 ceph-mon[93230]: pgmap v19: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 31 KiB/s rd, 1.1 MiB/s wr, 209 op/s 2024-04-02T04:44:50.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:50 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:50.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:50 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:50.395 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:50 smithi027 ceph-mon[83036]: pgmap v19: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 31 KiB/s rd, 1.1 MiB/s wr, 209 op/s 2024-04-02T04:44:50.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:50 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:50.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:50 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:50.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.679+0000 7f003d434700 1 -- 172.21.15.16:0/436572465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0038101560 msgr2=0x7f0038103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:50.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.679+0000 7f003d434700 1 --2- 172.21.15.16:0/436572465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0038101560 0x7f0038103950 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f002800b0a0 tx=0x7f002800b3b0 comp rx=0 tx=0).stop 2024-04-02T04:44:50.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.679+0000 7f003d434700 1 -- 172.21.15.16:0/436572465 shutdown_connections 2024-04-02T04:44:50.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.679+0000 7f003d434700 1 --2- 172.21.15.16:0/436572465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0038103e90 0x7f0038106280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:50.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.679+0000 7f003d434700 1 --2- 172.21.15.16:0/436572465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0038101560 0x7f0038103950 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:50.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.679+0000 7f003d434700 1 -- 172.21.15.16:0/436572465 >> 172.21.15.16:0/436572465 conn(0x7f00380faf70 msgr2=0x7f00380fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:50.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.679+0000 7f003d434700 1 -- 172.21.15.16:0/436572465 shutdown_connections 2024-04-02T04:44:50.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.679+0000 7f003d434700 1 -- 172.21.15.16:0/436572465 wait complete. 2024-04-02T04:44:50.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.680+0000 7f003d434700 1 Processor -- start 2024-04-02T04:44:50.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.680+0000 7f003d434700 1 -- start start 2024-04-02T04:44:50.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.680+0000 7f003d434700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0038101560 0x7f00381986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:50.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.680+0000 7f003d434700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0038103e90 0x7f0038198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:50.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.680+0000 7f003d434700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0038199230 con 0x7f0038101560 2024-04-02T04:44:50.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.680+0000 7f003d434700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0038199370 con 0x7f0038103e90 2024-04-02T04:44:50.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.680+0000 7f0036ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0038101560 0x7f00381986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:50.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.680+0000 7f00367fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0038103e90 0x7f0038198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:50.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.680+0000 7f0036ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0038101560 0x7f00381986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50218/0 (socket says 172.21.15.16:50218) 2024-04-02T04:44:50.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.680+0000 7f0036ffd700 1 -- 172.21.15.16:0/1235884794 learned_addr learned my addr 172.21.15.16:0/1235884794 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:44:50.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.681+0000 7f00367fc700 1 -- 172.21.15.16:0/1235884794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0038101560 msgr2=0x7f00381986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:50.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.681+0000 7f00367fc700 1 --2- 172.21.15.16:0/1235884794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0038101560 0x7f00381986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:50.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.681+0000 7f00367fc700 1 -- 172.21.15.16:0/1235884794 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0028009d00 con 0x7f0038103e90 2024-04-02T04:44:50.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.681+0000 7f0036ffd700 1 --2- 172.21.15.16:0/1235884794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0038101560 0x7f00381986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T04:44:50.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.681+0000 7f00367fc700 1 --2- 172.21.15.16:0/1235884794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0038103e90 0x7f0038198c10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f002000e970 tx=0x7f002000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:50.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.681+0000 7f002ffff700 1 -- 172.21.15.16:0/1235884794 <== mon.1 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f002000c9d0 con 0x7f0038103e90 2024-04-02T04:44:50.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.681+0000 7f002ffff700 1 -- 172.21.15.16:0/1235884794 <== mon.1 v2:172.21.15.27:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0020010430 con 0x7f0038103e90 2024-04-02T04:44:50.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.681+0000 7f003d434700 1 -- 172.21.15.16:0/1235884794 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f003819ddd0 con 0x7f0038103e90 2024-04-02T04:44:50.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.681+0000 7f002ffff700 1 -- 172.21.15.16:0/1235884794 <== mon.1 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0020018630 con 0x7f0038103e90 2024-04-02T04:44:50.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.681+0000 7f003d434700 1 -- 172.21.15.16:0/1235884794 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f003819e370 con 0x7f0038103e90 2024-04-02T04:44:50.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.682+0000 7f003d434700 1 -- 172.21.15.16:0/1235884794 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f003804ea80 con 0x7f0038103e90 2024-04-02T04:44:50.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.683+0000 7f002ffff700 1 -- 172.21.15.16:0/1235884794 <== mon.1 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 37) v1 ==== 91963+0+0 (secure 0 0 0) 0x7f002000cb30 con 0x7f0038103e90 2024-04-02T04:44:50.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.683+0000 7f002ffff700 1 --2- 172.21.15.16:0/1235884794 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f002406dea0 0x7f0024070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:50.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.683+0000 7f002ffff700 1 -- 172.21.15.16:0/1235884794 <== mon.1 v2:172.21.15.27:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f0020014070 con 0x7f0038103e90 2024-04-02T04:44:50.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.683+0000 7f0036ffd700 1 --2- 172.21.15.16:0/1235884794 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f002406dea0 0x7f0024070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:50.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.683+0000 7f0036ffd700 1 --2- 172.21.15.16:0/1235884794 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f002406dea0 0x7f0024070360 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0028007840 tx=0x7f00280060c0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:50.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.688+0000 7f002ffff700 1 -- 172.21.15.16:0/1235884794 <== mon.1 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0020053280 con 0x7f0038103e90 2024-04-02T04:44:50.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.916+0000 7f003d434700 1 -- 172.21.15.16:0/1235884794 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f00380ffd20 con 0x7f002406dea0 2024-04-02T04:44:50.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.917+0000 7f002ffff700 1 -- 172.21.15.16:0/1235884794 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f00380ffd20 con 0x7f002406dea0 2024-04-02T04:44:50.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.919+0000 7f003d434700 1 -- 172.21.15.16:0/1235884794 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f002406dea0 msgr2=0x7f0024070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:50.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.919+0000 7f003d434700 1 --2- 172.21.15.16:0/1235884794 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f002406dea0 0x7f0024070360 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0028007840 tx=0x7f00280060c0 comp rx=0 tx=0).stop 2024-04-02T04:44:50.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.919+0000 7f003d434700 1 -- 172.21.15.16:0/1235884794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0038103e90 msgr2=0x7f0038198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:50.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.919+0000 7f003d434700 1 --2- 172.21.15.16:0/1235884794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0038103e90 0x7f0038198c10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f002000e970 tx=0x7f002000ec80 comp rx=0 tx=0).stop 2024-04-02T04:44:50.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.919+0000 7f003d434700 1 -- 172.21.15.16:0/1235884794 shutdown_connections 2024-04-02T04:44:50.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.919+0000 7f003d434700 1 --2- 172.21.15.16:0/1235884794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0038101560 0x7f00381986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:50.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.919+0000 7f003d434700 1 --2- 172.21.15.16:0/1235884794 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f002406dea0 0x7f0024070360 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:50.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.919+0000 7f003d434700 1 --2- 172.21.15.16:0/1235884794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0038103e90 0x7f0038198c10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:50.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.919+0000 7f003d434700 1 -- 172.21.15.16:0/1235884794 >> 172.21.15.16:0/1235884794 conn(0x7f00380faf70 msgr2=0x7f00380ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:50.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.919+0000 7f003d434700 1 -- 172.21.15.16:0/1235884794 shutdown_connections 2024-04-02T04:44:50.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:50.919+0000 7f003d434700 1 -- 172.21.15.16:0/1235884794 wait complete. 2024-04-02T04:44:50.932 INFO:teuthology.orchestra.run.smithi016.stdout:true 2024-04-02T04:44:51.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.029+0000 7fac283f4700 1 -- 172.21.15.16:0/701355920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac20103e90 msgr2=0x7fac20106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:51.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.029+0000 7fac283f4700 1 --2- 172.21.15.16:0/701355920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac20103e90 0x7fac20106280 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fac10009910 tx=0x7fac10009c20 comp rx=0 tx=0).stop 2024-04-02T04:44:51.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.029+0000 7fac283f4700 1 -- 172.21.15.16:0/701355920 shutdown_connections 2024-04-02T04:44:51.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.029+0000 7fac283f4700 1 --2- 172.21.15.16:0/701355920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac20103e90 0x7fac20106280 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:51.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.029+0000 7fac283f4700 1 --2- 172.21.15.16:0/701355920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac20101560 0x7fac20103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:51.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.029+0000 7fac283f4700 1 -- 172.21.15.16:0/701355920 >> 172.21.15.16:0/701355920 conn(0x7fac200faf70 msgr2=0x7fac200fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:51.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.029+0000 7fac283f4700 1 -- 172.21.15.16:0/701355920 shutdown_connections 2024-04-02T04:44:51.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.029+0000 7fac283f4700 1 -- 172.21.15.16:0/701355920 wait complete. 2024-04-02T04:44:51.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.030+0000 7fac283f4700 1 Processor -- start 2024-04-02T04:44:51.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.030+0000 7fac283f4700 1 -- start start 2024-04-02T04:44:51.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.030+0000 7fac283f4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac20101560 0x7fac20196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:51.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.030+0000 7fac283f4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac20103e90 0x7fac201969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:51.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.030+0000 7fac283f4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac20196fd0 con 0x7fac20103e90 2024-04-02T04:44:51.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.030+0000 7fac283f4700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac20197110 con 0x7fac20101560 2024-04-02T04:44:51.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.030+0000 7fac2598f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac20103e90 0x7fac201969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:51.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.031+0000 7fac2598f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac20103e90 0x7fac201969b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50242/0 (socket says 172.21.15.16:50242) 2024-04-02T04:44:51.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.031+0000 7fac2598f700 1 -- 172.21.15.16:0/845643208 learned_addr learned my addr 172.21.15.16:0/845643208 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:44:51.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.031+0000 7fac26190700 1 --2- 172.21.15.16:0/845643208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac20101560 0x7fac20196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:51.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.031+0000 7fac2598f700 1 -- 172.21.15.16:0/845643208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac20101560 msgr2=0x7fac20196470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:51.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.031+0000 7fac2598f700 1 --2- 172.21.15.16:0/845643208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac20101560 0x7fac20196470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:51.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.031+0000 7fac2598f700 1 -- 172.21.15.16:0/845643208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac100095f0 con 0x7fac20103e90 2024-04-02T04:44:51.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.031+0000 7fac2598f700 1 --2- 172.21.15.16:0/845643208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac20103e90 0x7fac201969b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fac10006010 tx=0x7fac1000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:51.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.031+0000 7fac177fe700 1 -- 172.21.15.16:0/845643208 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac1001d070 con 0x7fac20103e90 2024-04-02T04:44:51.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.031+0000 7fac177fe700 1 -- 172.21.15.16:0/845643208 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac10005dc0 con 0x7fac20103e90 2024-04-02T04:44:51.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.031+0000 7fac283f4700 1 -- 172.21.15.16:0/845643208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac2019bb10 con 0x7fac20103e90 2024-04-02T04:44:51.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.031+0000 7fac177fe700 1 -- 172.21.15.16:0/845643208 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac10021410 con 0x7fac20103e90 2024-04-02T04:44:51.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.031+0000 7fac283f4700 1 -- 172.21.15.16:0/845643208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac2019c080 con 0x7fac20103e90 2024-04-02T04:44:51.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.032+0000 7fac283f4700 1 -- 172.21.15.16:0/845643208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac2004ea80 con 0x7fac20103e90 2024-04-02T04:44:51.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.033+0000 7fac177fe700 1 -- 172.21.15.16:0/845643208 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 37) v1 ==== 91963+0+0 (secure 0 0 0) 0x7fac1000f460 con 0x7fac20103e90 2024-04-02T04:44:51.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.034+0000 7fac177fe700 1 --2- 172.21.15.16:0/845643208 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fac0c06de90 0x7fac0c070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:51.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.034+0000 7fac177fe700 1 -- 172.21.15.16:0/845643208 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fac1008f030 con 0x7fac20103e90 2024-04-02T04:44:51.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.034+0000 7fac26190700 1 --2- 172.21.15.16:0/845643208 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fac0c06de90 0x7fac0c070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:51.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.034+0000 7fac26190700 1 --2- 172.21.15.16:0/845643208 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fac0c06de90 0x7fac0c070350 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fac1c005950 tx=0x7fac1c00b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:51.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.040+0000 7fac177fe700 1 -- 172.21.15.16:0/845643208 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fac1005bed0 con 0x7fac20103e90 2024-04-02T04:44:51.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.261+0000 7fac283f4700 1 -- 172.21.15.16:0/845643208 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fac20105d50 con 0x7fac0c06de90 2024-04-02T04:44:51.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.262+0000 7fac177fe700 1 -- 172.21.15.16:0/845643208 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fac20105d50 con 0x7fac0c06de90 2024-04-02T04:44:51.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.264+0000 7fac283f4700 1 -- 172.21.15.16:0/845643208 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fac0c06de90 msgr2=0x7fac0c070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:51.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.264+0000 7fac283f4700 1 --2- 172.21.15.16:0/845643208 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fac0c06de90 0x7fac0c070350 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fac1c005950 tx=0x7fac1c00b410 comp rx=0 tx=0).stop 2024-04-02T04:44:51.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.264+0000 7fac283f4700 1 -- 172.21.15.16:0/845643208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac20103e90 msgr2=0x7fac201969b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:51.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.264+0000 7fac283f4700 1 --2- 172.21.15.16:0/845643208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac20103e90 0x7fac201969b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fac10006010 tx=0x7fac1000bd70 comp rx=0 tx=0).stop 2024-04-02T04:44:51.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.264+0000 7fac283f4700 1 -- 172.21.15.16:0/845643208 shutdown_connections 2024-04-02T04:44:51.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.264+0000 7fac283f4700 1 --2- 172.21.15.16:0/845643208 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fac0c06de90 0x7fac0c070350 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:51.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.264+0000 7fac283f4700 1 --2- 172.21.15.16:0/845643208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac20101560 0x7fac20196470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:51.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.264+0000 7fac283f4700 1 --2- 172.21.15.16:0/845643208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac20103e90 0x7fac201969b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:51.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.264+0000 7fac283f4700 1 -- 172.21.15.16:0/845643208 >> 172.21.15.16:0/845643208 conn(0x7fac200faf70 msgr2=0x7fac200fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:51.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.264+0000 7fac283f4700 1 -- 172.21.15.16:0/845643208 shutdown_connections 2024-04-02T04:44:51.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.264+0000 7fac283f4700 1 -- 172.21.15.16:0/845643208 wait complete. 2024-04-02T04:44:51.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.369+0000 7f1981ad6700 1 -- 172.21.15.16:0/1431150008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f197c102160 msgr2=0x7f197c102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:51.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.369+0000 7f1981ad6700 1 --2- 172.21.15.16:0/1431150008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f197c102160 0x7f197c102580 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f197400b0a0 tx=0x7f197400b3b0 comp rx=0 tx=0).stop 2024-04-02T04:44:51.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.369+0000 7f1981ad6700 1 -- 172.21.15.16:0/1431150008 shutdown_connections 2024-04-02T04:44:51.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.369+0000 7f1981ad6700 1 --2- 172.21.15.16:0/1431150008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f197c1006a0 0x7f197c100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:51.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.369+0000 7f1981ad6700 1 --2- 172.21.15.16:0/1431150008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f197c102160 0x7f197c102580 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:51.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.369+0000 7f1981ad6700 1 -- 172.21.15.16:0/1431150008 >> 172.21.15.16:0/1431150008 conn(0x7f197c0fc040 msgr2=0x7f197c0fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:51.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.369+0000 7f1981ad6700 1 -- 172.21.15.16:0/1431150008 shutdown_connections 2024-04-02T04:44:51.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.369+0000 7f1981ad6700 1 -- 172.21.15.16:0/1431150008 wait complete. 2024-04-02T04:44:51.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.370+0000 7f1981ad6700 1 Processor -- start 2024-04-02T04:44:51.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.370+0000 7f1981ad6700 1 -- start start 2024-04-02T04:44:51.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.370+0000 7f1981ad6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f197c1006a0 0x7f197c074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:51.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.370+0000 7f1981ad6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f197c102160 0x7f197c0731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:51.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.370+0000 7f1981ad6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c0736e0 con 0x7f197c102160 2024-04-02T04:44:51.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.370+0000 7f1981ad6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c073820 con 0x7f197c1006a0 2024-04-02T04:44:51.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.370+0000 7f197affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f197c102160 0x7f197c0731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:51.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.370+0000 7f197b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f197c1006a0 0x7f197c074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:51.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.370+0000 7f197affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f197c102160 0x7f197c0731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50266/0 (socket says 172.21.15.16:50266) 2024-04-02T04:44:51.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.370+0000 7f197affd700 1 -- 172.21.15.16:0/844076473 learned_addr learned my addr 172.21.15.16:0/844076473 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:44:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.371+0000 7f197affd700 1 -- 172.21.15.16:0/844076473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f197c1006a0 msgr2=0x7f197c074b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.371+0000 7f197affd700 1 --2- 172.21.15.16:0/844076473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f197c1006a0 0x7f197c074b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.371+0000 7f197affd700 1 -- 172.21.15.16:0/844076473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1974009d00 con 0x7f197c102160 2024-04-02T04:44:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.371+0000 7f197b7fe700 1 --2- 172.21.15.16:0/844076473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f197c1006a0 0x7f197c074b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T04:44:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.371+0000 7f197affd700 1 --2- 172.21.15.16:0/844076473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f197c102160 0x7f197c0731a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f196400e920 tx=0x7f196400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.371+0000 7f1978ff9700 1 -- 172.21.15.16:0/844076473 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f196400c9d0 con 0x7f197c102160 2024-04-02T04:44:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.371+0000 7f1981ad6700 1 -- 172.21.15.16:0/844076473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f197c073b00 con 0x7f197c102160 2024-04-02T04:44:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.371+0000 7f1978ff9700 1 -- 172.21.15.16:0/844076473 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1964010430 con 0x7f197c102160 2024-04-02T04:44:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.371+0000 7f1978ff9700 1 -- 172.21.15.16:0/844076473 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1964018630 con 0x7f197c102160 2024-04-02T04:44:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.371+0000 7f1981ad6700 1 -- 172.21.15.16:0/844076473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f197c19e5d0 con 0x7f197c102160 2024-04-02T04:44:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.372+0000 7f1981ad6700 1 -- 172.21.15.16:0/844076473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f197c190700 con 0x7f197c102160 2024-04-02T04:44:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.373+0000 7f1978ff9700 1 -- 172.21.15.16:0/844076473 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 37) v1 ==== 91963+0+0 (secure 0 0 0) 0x7f19640105a0 con 0x7f197c102160 2024-04-02T04:44:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.373+0000 7f1978ff9700 1 --2- 172.21.15.16:0/844076473 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f196806dea0 0x7f1968070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.373+0000 7f1978ff9700 1 -- 172.21.15.16:0/844076473 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f1964014070 con 0x7f197c102160 2024-04-02T04:44:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.373+0000 7f197b7fe700 1 --2- 172.21.15.16:0/844076473 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f196806dea0 0x7f1968070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.374+0000 7f197b7fe700 1 --2- 172.21.15.16:0/844076473 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f196806dea0 0x7f1968070360 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1974009fd0 tx=0x7f1974006000 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:51.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.378+0000 7f1978ff9700 1 -- 172.21.15.16:0/844076473 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1964053280 con 0x7f197c102160 2024-04-02T04:44:51.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.590+0000 7f1981ad6700 1 -- 172.21.15.16:0/844076473 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f197c0611f0 con 0x7f196806dea0 2024-04-02T04:44:51.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.601+0000 7f1978ff9700 1 -- 172.21.15.16:0/844076473 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f197c0611f0 con 0x7f196806dea0 2024-04-02T04:44:51.604 INFO:teuthology.orchestra.run.smithi016.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:44:51.604 INFO:teuthology.orchestra.run.smithi016.stdout:alertmanager.smithi016 smithi016 *:9093,9094 running (9m) 27s ago 10m 24.2M - 0.25.0 c8568f914cd2 b95d8354b016 2024-04-02T04:44:51.604 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi016 smithi016 running (11m) 27s ago 11m 7814k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 9de6c16a487f 2024-04-02T04:44:51.604 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi027 smithi027 running (9m) 2s ago 9m 7884k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 99956592471a 2024-04-02T04:44:51.604 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi016 smithi016 running (11m) 27s ago 11m 6660k - 18.2.2-1059-g4d05f4ca 909dce4fe53d bd73919b0711 2024-04-02T04:44:51.604 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi027 smithi027 running (9m) 2s ago 9m 6660k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 7a54c37f202e 2024-04-02T04:44:51.604 INFO:teuthology.orchestra.run.smithi016.stdout:grafana.smithi016 smithi016 *:3000 running (9m) 27s ago 10m 85.6M - 9.4.7 954c08fa6188 7c43d133569c 2024-04-02T04:44:51.604 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.fyzrnm smithi016 running (5m) 27s ago 5m 16.6M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 70f28b9572a6 2024-04-02T04:44:51.604 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.uhgkgg smithi016 running (5m) 27s ago 5m 14.9M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 0aa4d16562a6 2024-04-02T04:44:51.605 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.sdublg smithi027 running (5m) 2s ago 5m 18.8M - 18.2.2-1059-g4d05f4ca 909dce4fe53d a538451099db 2024-04-02T04:44:51.605 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.zwrfoo smithi027 running (5m) 2s ago 5m 180M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 4cb3fa428d71 2024-04-02T04:44:51.605 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi016.eqnebx smithi016 *:8443,9283,8765 running (112s) 27s ago 12m 471M - 19.0.0-1979-g6cec793d 78f830190839 d61ed91b4577 2024-04-02T04:44:51.605 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi027.kjzcsc smithi027 *:8443,9283,8765 running (68s) 2s ago 9m 448M - 19.0.0-1979-g6cec793d 78f830190839 5072a5ffc761 2024-04-02T04:44:51.605 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi016 smithi016 running (44s) 27s ago 12m 27.7M 2048M 19.0.0-1979-g6cec793d 78f830190839 eabfc84f1cdf 2024-04-02T04:44:51.605 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi027 smithi027 running (16s) 2s ago 9m 24.4M 2048M 19.0.0-1979-g6cec793d 78f830190839 fcd68b20dcc5 2024-04-02T04:44:51.605 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi016 smithi016 *:9100 running (10m) 27s ago 10m 15.0M - 1.5.0 0da6a335fe13 09f54c65b53c 2024-04-02T04:44:51.605 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi027 smithi027 *:9100 running (9m) 2s ago 9m 15.2M - 1.5.0 0da6a335fe13 e41a657c7443 2024-04-02T04:44:51.605 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 smithi016 running (8m) 27s ago 8m 193M 1217M 18.2.2-1059-g4d05f4ca 909dce4fe53d 3f40108debd6 2024-04-02T04:44:51.605 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 smithi016 running (8m) 27s ago 8m 206M 1217M 18.2.2-1059-g4d05f4ca 909dce4fe53d 7361abc9e1b6 2024-04-02T04:44:51.605 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 smithi016 running (8m) 27s ago 7m 169M 1217M 18.2.2-1059-g4d05f4ca 909dce4fe53d 52e592b4d1f2 2024-04-02T04:44:51.605 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 smithi027 running (7m) 2s ago 7m 232M 2240M 18.2.2-1059-g4d05f4ca 909dce4fe53d cf7c7fe022f0 2024-04-02T04:44:51.606 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 smithi027 running (7m) 2s ago 7m 207M 2240M 18.2.2-1059-g4d05f4ca 909dce4fe53d 8cbbedf8cac9 2024-04-02T04:44:51.606 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 smithi027 running (6m) 2s ago 6m 190M 2240M 18.2.2-1059-g4d05f4ca 909dce4fe53d ac6094415529 2024-04-02T04:44:51.606 INFO:teuthology.orchestra.run.smithi016.stdout:prometheus.smithi016 smithi016 *:9095 running (83s) 27s ago 10m 51.1M - 2.43.0 a07b618ecd1d 466f00d73ddd 2024-04-02T04:44:51.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.603+0000 7f1981ad6700 1 -- 172.21.15.16:0/844076473 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f196806dea0 msgr2=0x7f1968070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:51.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.603+0000 7f1981ad6700 1 --2- 172.21.15.16:0/844076473 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f196806dea0 0x7f1968070360 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1974009fd0 tx=0x7f1974006000 comp rx=0 tx=0).stop 2024-04-02T04:44:51.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.603+0000 7f1981ad6700 1 -- 172.21.15.16:0/844076473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f197c102160 msgr2=0x7f197c0731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:51.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.603+0000 7f1981ad6700 1 --2- 172.21.15.16:0/844076473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f197c102160 0x7f197c0731a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f196400e920 tx=0x7f196400ec30 comp rx=0 tx=0).stop 2024-04-02T04:44:51.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.603+0000 7f1981ad6700 1 -- 172.21.15.16:0/844076473 shutdown_connections 2024-04-02T04:44:51.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.603+0000 7f1981ad6700 1 --2- 172.21.15.16:0/844076473 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f196806dea0 0x7f1968070360 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:51.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.603+0000 7f1981ad6700 1 --2- 172.21.15.16:0/844076473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f197c1006a0 0x7f197c074b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:51.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.603+0000 7f1981ad6700 1 --2- 172.21.15.16:0/844076473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f197c102160 0x7f197c0731a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:51.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.603+0000 7f1981ad6700 1 -- 172.21.15.16:0/844076473 >> 172.21.15.16:0/844076473 conn(0x7f197c0fc040 msgr2=0x7f197c1052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:51.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.603+0000 7f1981ad6700 1 -- 172.21.15.16:0/844076473 shutdown_connections 2024-04-02T04:44:51.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.603+0000 7f1981ad6700 1 -- 172.21.15.16:0/844076473 wait complete. 2024-04-02T04:44:51.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:51 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:51.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:51 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:51.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:51 smithi027 ceph-mon[83036]: from='client.44101 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:44:51.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:51 smithi027 ceph-mon[83036]: pgmap v20: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 524 KiB/s wr, 108 op/s 2024-04-02T04:44:51.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:51 smithi027 ceph-mon[83036]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:44:51.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.707+0000 7ffb6a19b700 1 -- 172.21.15.16:0/1121758368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb64103f00 msgr2=0x7ffb641062f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:51.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.707+0000 7ffb6a19b700 1 --2- 172.21.15.16:0/1121758368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb64103f00 0x7ffb641062f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ffb58009910 tx=0x7ffb58009c20 comp rx=0 tx=0).stop 2024-04-02T04:44:51.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.707+0000 7ffb6a19b700 1 -- 172.21.15.16:0/1121758368 shutdown_connections 2024-04-02T04:44:51.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.707+0000 7ffb6a19b700 1 --2- 172.21.15.16:0/1121758368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb64103f00 0x7ffb641062f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:51.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.707+0000 7ffb6a19b700 1 --2- 172.21.15.16:0/1121758368 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb641015d0 0x7ffb641039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:51.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.707+0000 7ffb6a19b700 1 -- 172.21.15.16:0/1121758368 >> 172.21.15.16:0/1121758368 conn(0x7ffb640faf80 msgr2=0x7ffb640fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:51.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.707+0000 7ffb6a19b700 1 -- 172.21.15.16:0/1121758368 shutdown_connections 2024-04-02T04:44:51.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.707+0000 7ffb6a19b700 1 -- 172.21.15.16:0/1121758368 wait complete. 2024-04-02T04:44:51.711 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:51 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:51.711 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:51 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:51.712 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:51 smithi016 ceph-mon[93230]: from='client.44101 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:44:51.712 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:51 smithi016 ceph-mon[93230]: pgmap v20: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 524 KiB/s wr, 108 op/s 2024-04-02T04:44:51.712 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:51 smithi016 ceph-mon[93230]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:44:51.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.708+0000 7ffb6a19b700 1 Processor -- start 2024-04-02T04:44:51.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.708+0000 7ffb6a19b700 1 -- start start 2024-04-02T04:44:51.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.708+0000 7ffb6a19b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb641015d0 0x7ffb641964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:51.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.708+0000 7ffb6a19b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb64103f00 0x7ffb641969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:51.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.708+0000 7ffb6a19b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb64197000 con 0x7ffb64103f00 2024-04-02T04:44:51.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.708+0000 7ffb6a19b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb64197140 con 0x7ffb641015d0 2024-04-02T04:44:51.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.709+0000 7ffb62ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb64103f00 0x7ffb641969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:51.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.709+0000 7ffb62ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb64103f00 0x7ffb641969e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35030/0 (socket says 172.21.15.16:35030) 2024-04-02T04:44:51.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.709+0000 7ffb62ffd700 1 -- 172.21.15.16:0/2790844070 learned_addr learned my addr 172.21.15.16:0/2790844070 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:44:51.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.709+0000 7ffb637fe700 1 --2- 172.21.15.16:0/2790844070 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb641015d0 0x7ffb641964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:51.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.709+0000 7ffb62ffd700 1 -- 172.21.15.16:0/2790844070 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb641015d0 msgr2=0x7ffb641964a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:51.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.709+0000 7ffb62ffd700 1 --2- 172.21.15.16:0/2790844070 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb641015d0 0x7ffb641964a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:51.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.709+0000 7ffb62ffd700 1 -- 172.21.15.16:0/2790844070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb580095f0 con 0x7ffb64103f00 2024-04-02T04:44:51.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.709+0000 7ffb62ffd700 1 --2- 172.21.15.16:0/2790844070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb64103f00 0x7ffb641969e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ffb580092e0 tx=0x7ffb5800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:51.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.709+0000 7ffb60ff9700 1 -- 172.21.15.16:0/2790844070 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb5801d070 con 0x7ffb64103f00 2024-04-02T04:44:51.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.709+0000 7ffb6a19b700 1 -- 172.21.15.16:0/2790844070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb6419bb40 con 0x7ffb64103f00 2024-04-02T04:44:51.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.709+0000 7ffb60ff9700 1 -- 172.21.15.16:0/2790844070 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffb5800f460 con 0x7ffb64103f00 2024-04-02T04:44:51.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.710+0000 7ffb60ff9700 1 -- 172.21.15.16:0/2790844070 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb58021640 con 0x7ffb64103f00 2024-04-02T04:44:51.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.710+0000 7ffb6a19b700 1 -- 172.21.15.16:0/2790844070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb6419c0b0 con 0x7ffb64103f00 2024-04-02T04:44:51.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.711+0000 7ffb6a19b700 1 -- 172.21.15.16:0/2790844070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb6404ea80 con 0x7ffb64103f00 2024-04-02T04:44:51.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.711+0000 7ffb60ff9700 1 -- 172.21.15.16:0/2790844070 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 37) v1 ==== 91963+0+0 (secure 0 0 0) 0x7ffb5800f5d0 con 0x7ffb64103f00 2024-04-02T04:44:51.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.711+0000 7ffb60ff9700 1 --2- 172.21.15.16:0/2790844070 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ffb4c06dea0 0x7ffb4c070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:51.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.712+0000 7ffb60ff9700 1 -- 172.21.15.16:0/2790844070 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7ffb5808f020 con 0x7ffb64103f00 2024-04-02T04:44:51.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.712+0000 7ffb637fe700 1 --2- 172.21.15.16:0/2790844070 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ffb4c06dea0 0x7ffb4c070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:51.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.712+0000 7ffb637fe700 1 --2- 172.21.15.16:0/2790844070 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ffb4c06dea0 0x7ffb4c070360 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ffb54005950 tx=0x7ffb5400a300 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:51.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:51.717+0000 7ffb60ff9700 1 -- 172.21.15.16:0/2790844070 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ffb5805bf40 con 0x7ffb64103f00 2024-04-02T04:44:52.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.011+0000 7ffb6a19b700 1 -- 172.21.15.16:0/2790844070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ffb641097f0 con 0x7ffb64103f00 2024-04-02T04:44:52.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.011+0000 7ffb60ff9700 1 -- 172.21.15.16:0/2790844070 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+733 (secure 0 0 0) 0x7ffb5805bad0 con 0x7ffb64103f00 2024-04-02T04:44:52.014 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:44:52.015 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": { 2024-04-02T04:44:52.015 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:44:52.015 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:44:52.015 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": { 2024-04-02T04:44:52.015 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:44:52.015 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:44:52.015 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": { 2024-04-02T04:44:52.015 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 6 2024-04-02T04:44:52.015 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:44:52.015 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-04-02T04:44:52.015 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 4 2024-04-02T04:44:52.015 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:44:52.015 INFO:teuthology.orchestra.run.smithi016.stdout: "overall": { 2024-04-02T04:44:52.016 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 10, 2024-04-02T04:44:52.016 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 4 2024-04-02T04:44:52.016 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-02T04:44:52.016 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:44:52.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.014+0000 7ffb6a19b700 1 -- 172.21.15.16:0/2790844070 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ffb4c06dea0 msgr2=0x7ffb4c070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:52.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.014+0000 7ffb6a19b700 1 --2- 172.21.15.16:0/2790844070 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ffb4c06dea0 0x7ffb4c070360 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ffb54005950 tx=0x7ffb5400a300 comp rx=0 tx=0).stop 2024-04-02T04:44:52.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.014+0000 7ffb6a19b700 1 -- 172.21.15.16:0/2790844070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb64103f00 msgr2=0x7ffb641969e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:52.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.014+0000 7ffb6a19b700 1 --2- 172.21.15.16:0/2790844070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb64103f00 0x7ffb641969e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ffb580092e0 tx=0x7ffb5800bd70 comp rx=0 tx=0).stop 2024-04-02T04:44:52.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.014+0000 7ffb6a19b700 1 -- 172.21.15.16:0/2790844070 shutdown_connections 2024-04-02T04:44:52.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.014+0000 7ffb6a19b700 1 --2- 172.21.15.16:0/2790844070 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ffb4c06dea0 0x7ffb4c070360 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:52.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.014+0000 7ffb6a19b700 1 --2- 172.21.15.16:0/2790844070 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb641015d0 0x7ffb641964a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:52.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.014+0000 7ffb6a19b700 1 --2- 172.21.15.16:0/2790844070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb64103f00 0x7ffb641969e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:52.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.014+0000 7ffb6a19b700 1 -- 172.21.15.16:0/2790844070 >> 172.21.15.16:0/2790844070 conn(0x7ffb640faf80 msgr2=0x7ffb640fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:52.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.014+0000 7ffb6a19b700 1 -- 172.21.15.16:0/2790844070 shutdown_connections 2024-04-02T04:44:52.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.014+0000 7ffb6a19b700 1 -- 172.21.15.16:0/2790844070 wait complete. 2024-04-02T04:44:52.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.118+0000 7fca6011c700 1 -- 172.21.15.16:0/1450777308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca58103ed0 msgr2=0x7fca581062c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:52.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.118+0000 7fca6011c700 1 --2- 172.21.15.16:0/1450777308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca58103ed0 0x7fca581062c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fca48009910 tx=0x7fca48009c20 comp rx=0 tx=0).stop 2024-04-02T04:44:52.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.118+0000 7fca6011c700 1 -- 172.21.15.16:0/1450777308 shutdown_connections 2024-04-02T04:44:52.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.118+0000 7fca6011c700 1 --2- 172.21.15.16:0/1450777308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca58103ed0 0x7fca581062c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:52.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.118+0000 7fca6011c700 1 --2- 172.21.15.16:0/1450777308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca581015a0 0x7fca58103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:52.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.118+0000 7fca6011c700 1 -- 172.21.15.16:0/1450777308 >> 172.21.15.16:0/1450777308 conn(0x7fca580fafb0 msgr2=0x7fca580fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:52.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.119+0000 7fca6011c700 1 -- 172.21.15.16:0/1450777308 shutdown_connections 2024-04-02T04:44:52.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.119+0000 7fca6011c700 1 -- 172.21.15.16:0/1450777308 wait complete. 2024-04-02T04:44:52.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.119+0000 7fca6011c700 1 Processor -- start 2024-04-02T04:44:52.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.119+0000 7fca6011c700 1 -- start start 2024-04-02T04:44:52.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.119+0000 7fca6011c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca581015a0 0x7fca58196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:52.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.119+0000 7fca6011c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca58103ed0 0x7fca58196a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:52.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.119+0000 7fca6011c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca581970a0 con 0x7fca58103ed0 2024-04-02T04:44:52.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.119+0000 7fca6011c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca581971e0 con 0x7fca581015a0 2024-04-02T04:44:52.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.120+0000 7fca5d6b7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca58103ed0 0x7fca58196a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:52.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.120+0000 7fca5d6b7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca58103ed0 0x7fca58196a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35052/0 (socket says 172.21.15.16:35052) 2024-04-02T04:44:52.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.120+0000 7fca5d6b7700 1 -- 172.21.15.16:0/3101256433 learned_addr learned my addr 172.21.15.16:0/3101256433 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:44:52.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.120+0000 7fca5deb8700 1 --2- 172.21.15.16:0/3101256433 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca581015a0 0x7fca58196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:52.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.120+0000 7fca5d6b7700 1 -- 172.21.15.16:0/3101256433 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca581015a0 msgr2=0x7fca58196540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:52.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.120+0000 7fca5d6b7700 1 --2- 172.21.15.16:0/3101256433 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca581015a0 0x7fca58196540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:52.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.120+0000 7fca5d6b7700 1 -- 172.21.15.16:0/3101256433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca480095f0 con 0x7fca58103ed0 2024-04-02T04:44:52.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.120+0000 7fca5d6b7700 1 --2- 172.21.15.16:0/3101256433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca58103ed0 0x7fca58196a80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fca4800bd70 tx=0x7fca4800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:52.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.120+0000 7fca4effd700 1 -- 172.21.15.16:0/3101256433 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca4801d070 con 0x7fca58103ed0 2024-04-02T04:44:52.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.120+0000 7fca6011c700 1 -- 172.21.15.16:0/3101256433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca5819bb90 con 0x7fca58103ed0 2024-04-02T04:44:52.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.120+0000 7fca4effd700 1 -- 172.21.15.16:0/3101256433 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca4800f460 con 0x7fca58103ed0 2024-04-02T04:44:52.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.120+0000 7fca4effd700 1 -- 172.21.15.16:0/3101256433 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca48017630 con 0x7fca58103ed0 2024-04-02T04:44:52.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.120+0000 7fca6011c700 1 -- 172.21.15.16:0/3101256433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca5819c100 con 0x7fca58103ed0 2024-04-02T04:44:52.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.121+0000 7fca6011c700 1 -- 172.21.15.16:0/3101256433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca5804ea80 con 0x7fca58103ed0 2024-04-02T04:44:52.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.122+0000 7fca4effd700 1 -- 172.21.15.16:0/3101256433 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 37) v1 ==== 91963+0+0 (secure 0 0 0) 0x7fca4800fad0 con 0x7fca58103ed0 2024-04-02T04:44:52.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.122+0000 7fca4effd700 1 --2- 172.21.15.16:0/3101256433 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fca4406dea0 0x7fca44070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:52.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.122+0000 7fca4effd700 1 -- 172.21.15.16:0/3101256433 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fca4808f150 con 0x7fca58103ed0 2024-04-02T04:44:52.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.122+0000 7fca5deb8700 1 --2- 172.21.15.16:0/3101256433 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fca4406dea0 0x7fca44070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:52.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.123+0000 7fca5deb8700 1 --2- 172.21.15.16:0/3101256433 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fca4406dea0 0x7fca44070360 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fca54005950 tx=0x7fca5400b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:52.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.127+0000 7fca4effd700 1 -- 172.21.15.16:0/3101256433 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fca4805bff0 con 0x7fca58103ed0 2024-04-02T04:44:52.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.377+0000 7fca6011c700 1 -- 172.21.15.16:0/3101256433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fca58109840 con 0x7fca58103ed0 2024-04-02T04:44:52.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.378+0000 7fca4effd700 1 -- 172.21.15.16:0/3101256433 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1875 (secure 0 0 0) 0x7fca4805bb80 con 0x7fca58103ed0 2024-04-02T04:44:52.381 INFO:teuthology.orchestra.run.smithi016.stdout:e15 2024-04-02T04:44:52.381 INFO:teuthology.orchestra.run.smithi016.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:44:52.381 INFO:teuthology.orchestra.run.smithi016.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:44:52.381 INFO:teuthology.orchestra.run.smithi016.stdout:legacy client fscid: 1 2024-04-02T04:44:52.381 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:44:52.382 INFO:teuthology.orchestra.run.smithi016.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:44:52.382 INFO:teuthology.orchestra.run.smithi016.stdout:fs_name cephfs 2024-04-02T04:44:52.382 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 15 2024-04-02T04:44:52.382 INFO:teuthology.orchestra.run.smithi016.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:44:52.382 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-04-02T04:39:30.740168+0000 2024-04-02T04:44:52.382 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-04-02T04:39:49.875319+0000 2024-04-02T04:44:52.382 INFO:teuthology.orchestra.run.smithi016.stdout:tableserver 0 2024-04-02T04:44:52.382 INFO:teuthology.orchestra.run.smithi016.stdout:root 0 2024-04-02T04:44:52.382 INFO:teuthology.orchestra.run.smithi016.stdout:session_timeout 60 2024-04-02T04:44:52.382 INFO:teuthology.orchestra.run.smithi016.stdout:session_autoclose 300 2024-04-02T04:44:52.382 INFO:teuthology.orchestra.run.smithi016.stdout:max_file_size 1099511627776 2024-04-02T04:44:52.382 INFO:teuthology.orchestra.run.smithi016.stdout:max_xattr_size 65536 2024-04-02T04:44:52.382 INFO:teuthology.orchestra.run.smithi016.stdout:required_client_features {} 2024-04-02T04:44:52.382 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure 0 2024-04-02T04:44:52.383 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure_osd_epoch 46 2024-04-02T04:44:52.383 INFO:teuthology.orchestra.run.smithi016.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:44:52.383 INFO:teuthology.orchestra.run.smithi016.stdout:max_mds 1 2024-04-02T04:44:52.383 INFO:teuthology.orchestra.run.smithi016.stdout:in 0 2024-04-02T04:44:52.383 INFO:teuthology.orchestra.run.smithi016.stdout:up {0=24285} 2024-04-02T04:44:52.383 INFO:teuthology.orchestra.run.smithi016.stdout:failed 2024-04-02T04:44:52.383 INFO:teuthology.orchestra.run.smithi016.stdout:damaged 2024-04-02T04:44:52.383 INFO:teuthology.orchestra.run.smithi016.stdout:stopped 2024-04-02T04:44:52.383 INFO:teuthology.orchestra.run.smithi016.stdout:data_pools [3] 2024-04-02T04:44:52.383 INFO:teuthology.orchestra.run.smithi016.stdout:metadata_pool 2 2024-04-02T04:44:52.383 INFO:teuthology.orchestra.run.smithi016.stdout:inline_data enabled 2024-04-02T04:44:52.383 INFO:teuthology.orchestra.run.smithi016.stdout:balancer 2024-04-02T04:44:52.383 INFO:teuthology.orchestra.run.smithi016.stdout:bal_rank_mask -1 2024-04-02T04:44:52.383 INFO:teuthology.orchestra.run.smithi016.stdout:standby_count_wanted 1 2024-04-02T04:44:52.384 INFO:teuthology.orchestra.run.smithi016.stdout:qdb_cluster leader: 0 members: 2024-04-02T04:44:52.384 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.zwrfoo{0:24285} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.27:6824/1461455976,v1:172.21.15.27:6825/1461455976] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:44:52.384 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:44:52.384 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:44:52.384 INFO:teuthology.orchestra.run.smithi016.stdout:Standby daemons: 2024-04-02T04:44:52.384 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:44:52.384 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.fyzrnm{-1:14488} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.16:6828/2578719859,v1:172.21.15.16:6829/2578719859] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:44:52.384 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.uhgkgg{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.16:6826/601263579,v1:172.21.15.16:6827/601263579] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:44:52.384 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.sdublg{-1:24295} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.27:6826/4177192740,v1:172.21.15.27:6827/4177192740] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:44:52.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.380+0000 7fca6011c700 1 -- 172.21.15.16:0/3101256433 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fca4406dea0 msgr2=0x7fca44070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:52.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.380+0000 7fca6011c700 1 --2- 172.21.15.16:0/3101256433 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fca4406dea0 0x7fca44070360 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fca54005950 tx=0x7fca5400b410 comp rx=0 tx=0).stop 2024-04-02T04:44:52.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.380+0000 7fca6011c700 1 -- 172.21.15.16:0/3101256433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca58103ed0 msgr2=0x7fca58196a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:52.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.380+0000 7fca6011c700 1 --2- 172.21.15.16:0/3101256433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca58103ed0 0x7fca58196a80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fca4800bd70 tx=0x7fca4800bda0 comp rx=0 tx=0).stop 2024-04-02T04:44:52.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.380+0000 7fca6011c700 1 -- 172.21.15.16:0/3101256433 shutdown_connections 2024-04-02T04:44:52.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.380+0000 7fca6011c700 1 --2- 172.21.15.16:0/3101256433 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fca4406dea0 0x7fca44070360 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:52.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.380+0000 7fca6011c700 1 --2- 172.21.15.16:0/3101256433 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca581015a0 0x7fca58196540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:52.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.380+0000 7fca6011c700 1 --2- 172.21.15.16:0/3101256433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca58103ed0 0x7fca58196a80 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:52.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.380+0000 7fca6011c700 1 -- 172.21.15.16:0/3101256433 >> 172.21.15.16:0/3101256433 conn(0x7fca580fafb0 msgr2=0x7fca580fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:52.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.380+0000 7fca6011c700 1 -- 172.21.15.16:0/3101256433 shutdown_connections 2024-04-02T04:44:52.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.380+0000 7fca6011c700 1 -- 172.21.15.16:0/3101256433 wait complete. 2024-04-02T04:44:52.386 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 15 2024-04-02T04:44:52.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.486+0000 7fe425a21700 1 -- 172.21.15.16:0/4216027973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe420101db0 msgr2=0x7fe420102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:52.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.486+0000 7fe425a21700 1 --2- 172.21.15.16:0/4216027973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe420101db0 0x7fe420102210 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe410009910 tx=0x7fe410009c20 comp rx=0 tx=0).stop 2024-04-02T04:44:52.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.486+0000 7fe425a21700 1 -- 172.21.15.16:0/4216027973 shutdown_connections 2024-04-02T04:44:52.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.486+0000 7fe425a21700 1 --2- 172.21.15.16:0/4216027973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe420101db0 0x7fe420102210 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:52.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.486+0000 7fe425a21700 1 --2- 172.21.15.16:0/4216027973 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4200fedd0 0x7fe4200ff1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:52.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.486+0000 7fe425a21700 1 -- 172.21.15.16:0/4216027973 >> 172.21.15.16:0/4216027973 conn(0x7fe4200fa730 msgr2=0x7fe4200fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:52.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.486+0000 7fe425a21700 1 -- 172.21.15.16:0/4216027973 shutdown_connections 2024-04-02T04:44:52.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.486+0000 7fe425a21700 1 -- 172.21.15.16:0/4216027973 wait complete. 2024-04-02T04:44:52.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.486+0000 7fe425a21700 1 Processor -- start 2024-04-02T04:44:52.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.487+0000 7fe425a21700 1 -- start start 2024-04-02T04:44:52.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.487+0000 7fe425a21700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4200fedd0 0x7fe420101670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:52.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.487+0000 7fe425a21700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe420101db0 0x7fe4200ffd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:52.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.487+0000 7fe425a21700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4201002a0 con 0x7fe420101db0 2024-04-02T04:44:52.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.487+0000 7fe425a21700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4201003e0 con 0x7fe4200fedd0 2024-04-02T04:44:52.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.487+0000 7fe41e7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe420101db0 0x7fe4200ffd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:52.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.487+0000 7fe41e7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe420101db0 0x7fe4200ffd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35062/0 (socket says 172.21.15.16:35062) 2024-04-02T04:44:52.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.487+0000 7fe41e7fc700 1 -- 172.21.15.16:0/1931697111 learned_addr learned my addr 172.21.15.16:0/1931697111 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:44:52.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.487+0000 7fe41effd700 1 --2- 172.21.15.16:0/1931697111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4200fedd0 0x7fe420101670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:52.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.487+0000 7fe41e7fc700 1 -- 172.21.15.16:0/1931697111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4200fedd0 msgr2=0x7fe420101670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:52.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.487+0000 7fe41e7fc700 1 --2- 172.21.15.16:0/1931697111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4200fedd0 0x7fe420101670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:52.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.487+0000 7fe41e7fc700 1 -- 172.21.15.16:0/1931697111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4100095f0 con 0x7fe420101db0 2024-04-02T04:44:52.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.487+0000 7fe41e7fc700 1 --2- 172.21.15.16:0/1931697111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe420101db0 0x7fe4200ffd60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe41000b6f0 tx=0x7fe410005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:52.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.488+0000 7fe424a1f700 1 -- 172.21.15.16:0/1931697111 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe41001d070 con 0x7fe420101db0 2024-04-02T04:44:52.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.488+0000 7fe424a1f700 1 -- 172.21.15.16:0/1931697111 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe4100052a0 con 0x7fe420101db0 2024-04-02T04:44:52.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.488+0000 7fe425a21700 1 -- 172.21.15.16:0/1931697111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe420100660 con 0x7fe420101db0 2024-04-02T04:44:52.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.488+0000 7fe424a1f700 1 -- 172.21.15.16:0/1931697111 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe410022470 con 0x7fe420101db0 2024-04-02T04:44:52.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.488+0000 7fe425a21700 1 -- 172.21.15.16:0/1931697111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe420100bd0 con 0x7fe420101db0 2024-04-02T04:44:52.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.489+0000 7fe425a21700 1 -- 172.21.15.16:0/1931697111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe420190690 con 0x7fe420101db0 2024-04-02T04:44:52.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.489+0000 7fe424a1f700 1 -- 172.21.15.16:0/1931697111 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 37) v1 ==== 91963+0+0 (secure 0 0 0) 0x7fe41000f770 con 0x7fe420101db0 2024-04-02T04:44:52.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.489+0000 7fe424a1f700 1 --2- 172.21.15.16:0/1931697111 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fe40c06dea0 0x7fe40c070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:52.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.490+0000 7fe424a1f700 1 -- 172.21.15.16:0/1931697111 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fe41008ef70 con 0x7fe420101db0 2024-04-02T04:44:52.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.490+0000 7fe41effd700 1 --2- 172.21.15.16:0/1931697111 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fe40c06dea0 0x7fe40c070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:52.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.490+0000 7fe41effd700 1 --2- 172.21.15.16:0/1931697111 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fe40c06dea0 0x7fe40c070360 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe420068e10 tx=0x7fe408009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:52.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.494+0000 7fe424a1f700 1 -- 172.21.15.16:0/1931697111 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe4100c3280 con 0x7fe420101db0 2024-04-02T04:44:52.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.718+0000 7fe425a21700 1 -- 172.21.15.16:0/1931697111 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe420100bd0 con 0x7fe40c06dea0 2024-04-02T04:44:52.722 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:52 smithi016 ceph-mon[93230]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:44:52.722 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:52 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/2790844070' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:44:52.722 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:52 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/3101256433' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:44:52.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.720+0000 7fe424a1f700 1 -- 172.21.15.16:0/1931697111 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fe420100bd0 con 0x7fe40c06dea0 2024-04-02T04:44:52.723 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:44:52.723 INFO:teuthology.orchestra.run.smithi016.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:44:52.723 INFO:teuthology.orchestra.run.smithi016.stdout: "in_progress": true, 2024-04-02T04:44:52.723 INFO:teuthology.orchestra.run.smithi016.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-02T04:44:52.723 INFO:teuthology.orchestra.run.smithi016.stdout: "services_complete": [ 2024-04-02T04:44:52.723 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr", 2024-04-02T04:44:52.723 INFO:teuthology.orchestra.run.smithi016.stdout: "mon" 2024-04-02T04:44:52.723 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-04-02T04:44:52.723 INFO:teuthology.orchestra.run.smithi016.stdout: "progress": "4/23 daemons upgraded", 2024-04-02T04:44:52.723 INFO:teuthology.orchestra.run.smithi016.stdout: "message": "Currently upgrading mon daemons", 2024-04-02T04:44:52.723 INFO:teuthology.orchestra.run.smithi016.stdout: "is_paused": false 2024-04-02T04:44:52.723 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:44:52.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.723+0000 7fe425a21700 1 -- 172.21.15.16:0/1931697111 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fe40c06dea0 msgr2=0x7fe40c070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:52.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.723+0000 7fe425a21700 1 --2- 172.21.15.16:0/1931697111 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fe40c06dea0 0x7fe40c070360 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe420068e10 tx=0x7fe408009260 comp rx=0 tx=0).stop 2024-04-02T04:44:52.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.723+0000 7fe425a21700 1 -- 172.21.15.16:0/1931697111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe420101db0 msgr2=0x7fe4200ffd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:52.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.723+0000 7fe425a21700 1 --2- 172.21.15.16:0/1931697111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe420101db0 0x7fe4200ffd60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe41000b6f0 tx=0x7fe410005e30 comp rx=0 tx=0).stop 2024-04-02T04:44:52.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.723+0000 7fe425a21700 1 -- 172.21.15.16:0/1931697111 shutdown_connections 2024-04-02T04:44:52.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.723+0000 7fe425a21700 1 --2- 172.21.15.16:0/1931697111 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fe40c06dea0 0x7fe40c070360 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:52.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.723+0000 7fe425a21700 1 --2- 172.21.15.16:0/1931697111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4200fedd0 0x7fe420101670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:52.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.723+0000 7fe425a21700 1 --2- 172.21.15.16:0/1931697111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe420101db0 0x7fe4200ffd60 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:52.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.723+0000 7fe425a21700 1 -- 172.21.15.16:0/1931697111 >> 172.21.15.16:0/1931697111 conn(0x7fe4200fa730 msgr2=0x7fe420105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:52.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.723+0000 7fe425a21700 1 -- 172.21.15.16:0/1931697111 shutdown_connections 2024-04-02T04:44:52.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.723+0000 7fe425a21700 1 -- 172.21.15.16:0/1931697111 wait complete. 2024-04-02T04:44:52.758 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:52 smithi027 ceph-mon[83036]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:44:52.759 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:52 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/2790844070' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:44:52.759 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:52 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/3101256433' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:44:52.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.829+0000 7f60fccdd700 1 -- 172.21.15.16:0/2318100653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60f8100bd0 msgr2=0x7f60f8101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:52.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.829+0000 7f60fccdd700 1 --2- 172.21.15.16:0/2318100653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60f8100bd0 0x7f60f8101030 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f60e8009910 tx=0x7f60e8009c20 comp rx=0 tx=0).stop 2024-04-02T04:44:52.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.829+0000 7f60fccdd700 1 -- 172.21.15.16:0/2318100653 shutdown_connections 2024-04-02T04:44:52.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.829+0000 7f60fccdd700 1 --2- 172.21.15.16:0/2318100653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60f8100bd0 0x7f60f8101030 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:52.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.829+0000 7f60fccdd700 1 --2- 172.21.15.16:0/2318100653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60f80ff9d0 0x7f60f80ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:52.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.829+0000 7f60fccdd700 1 -- 172.21.15.16:0/2318100653 >> 172.21.15.16:0/2318100653 conn(0x7f60f80faf70 msgr2=0x7f60f80fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:52.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.829+0000 7f60fccdd700 1 -- 172.21.15.16:0/2318100653 shutdown_connections 2024-04-02T04:44:52.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.829+0000 7f60fccdd700 1 -- 172.21.15.16:0/2318100653 wait complete. 2024-04-02T04:44:52.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.829+0000 7f60fccdd700 1 Processor -- start 2024-04-02T04:44:52.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.830+0000 7f60fccdd700 1 -- start start 2024-04-02T04:44:52.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.830+0000 7f60fccdd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60f80ff9d0 0x7f60f8198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:52.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.830+0000 7f60fccdd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60f8100bd0 0x7f60f8198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:52.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.830+0000 7f60fccdd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f8199290 con 0x7f60f8100bd0 2024-04-02T04:44:52.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.830+0000 7f60fccdd700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f81993d0 con 0x7f60f80ff9d0 2024-04-02T04:44:52.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.830+0000 7f60f5d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60f8100bd0 0x7f60f8198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:52.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.830+0000 7f60f5d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60f8100bd0 0x7f60f8198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35086/0 (socket says 172.21.15.16:35086) 2024-04-02T04:44:52.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.830+0000 7f60f5d9b700 1 -- 172.21.15.16:0/1028480122 learned_addr learned my addr 172.21.15.16:0/1028480122 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:44:52.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.830+0000 7f60f659c700 1 --2- 172.21.15.16:0/1028480122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60f80ff9d0 0x7f60f8198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:52.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.830+0000 7f60f5d9b700 1 -- 172.21.15.16:0/1028480122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60f80ff9d0 msgr2=0x7f60f8198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:52.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.830+0000 7f60f5d9b700 1 --2- 172.21.15.16:0/1028480122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60f80ff9d0 0x7f60f8198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:52.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.830+0000 7f60f5d9b700 1 -- 172.21.15.16:0/1028480122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e80095f0 con 0x7f60f8100bd0 2024-04-02T04:44:52.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.830+0000 7f60f5d9b700 1 --2- 172.21.15.16:0/1028480122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60f8100bd0 0x7f60f8198c70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f60e8005e50 tx=0x7f60e8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:52.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.831+0000 7f60e77fe700 1 -- 172.21.15.16:0/1028480122 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60e801d070 con 0x7f60f8100bd0 2024-04-02T04:44:52.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.831+0000 7f60fccdd700 1 -- 172.21.15.16:0/1028480122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60f819ddd0 con 0x7f60f8100bd0 2024-04-02T04:44:52.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.831+0000 7f60fccdd700 1 -- 172.21.15.16:0/1028480122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60f819e340 con 0x7f60f8100bd0 2024-04-02T04:44:52.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.831+0000 7f60e77fe700 1 -- 172.21.15.16:0/1028480122 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f60e80052a0 con 0x7f60f8100bd0 2024-04-02T04:44:52.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.831+0000 7f60e77fe700 1 -- 172.21.15.16:0/1028480122 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60e8003e80 con 0x7f60f8100bd0 2024-04-02T04:44:52.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.832+0000 7f60fccdd700 1 -- 172.21.15.16:0/1028480122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60d80052f0 con 0x7f60f8100bd0 2024-04-02T04:44:52.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.832+0000 7f60e77fe700 1 -- 172.21.15.16:0/1028480122 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 37) v1 ==== 91963+0+0 (secure 0 0 0) 0x7f60e8003680 con 0x7f60f8100bd0 2024-04-02T04:44:52.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.832+0000 7f60e77fe700 1 --2- 172.21.15.16:0/1028480122 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f60e006de90 0x7f60e0070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:44:52.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.833+0000 7f60e77fe700 1 -- 172.21.15.16:0/1028480122 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f60e808f200 con 0x7f60f8100bd0 2024-04-02T04:44:52.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.833+0000 7f60f659c700 1 --2- 172.21.15.16:0/1028480122 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f60e006de90 0x7f60e0070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:44:52.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.833+0000 7f60f659c700 1 --2- 172.21.15.16:0/1028480122 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f60e006de90 0x7f60e0070350 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f60f8100a30 tx=0x7f60ec009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:44:52.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:52.837+0000 7f60e77fe700 1 -- 172.21.15.16:0/1028480122 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f60e805c040 con 0x7f60f8100bd0 2024-04-02T04:44:53.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:53.127+0000 7f60fccdd700 1 -- 172.21.15.16:0/1028480122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f60d8005160 con 0x7f60f8100bd0 2024-04-02T04:44:53.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:53.127+0000 7f60e77fe700 1 -- 172.21.15.16:0/1028480122 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f60e805bbd0 con 0x7f60f8100bd0 2024-04-02T04:44:53.130 INFO:teuthology.orchestra.run.smithi016.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-02T04:44:53.130 INFO:teuthology.orchestra.run.smithi016.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-02T04:44:53.131 INFO:teuthology.orchestra.run.smithi016.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-02T04:44:53.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:53.129+0000 7f60fccdd700 1 -- 172.21.15.16:0/1028480122 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f60e006de90 msgr2=0x7f60e0070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:53.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:53.129+0000 7f60fccdd700 1 --2- 172.21.15.16:0/1028480122 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f60e006de90 0x7f60e0070350 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f60f8100a30 tx=0x7f60ec009260 comp rx=0 tx=0).stop 2024-04-02T04:44:53.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:53.129+0000 7f60fccdd700 1 -- 172.21.15.16:0/1028480122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60f8100bd0 msgr2=0x7f60f8198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:44:53.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:53.129+0000 7f60fccdd700 1 --2- 172.21.15.16:0/1028480122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60f8100bd0 0x7f60f8198c70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f60e8005e50 tx=0x7f60e8005e80 comp rx=0 tx=0).stop 2024-04-02T04:44:53.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:53.129+0000 7f60fccdd700 1 -- 172.21.15.16:0/1028480122 shutdown_connections 2024-04-02T04:44:53.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:53.129+0000 7f60fccdd700 1 --2- 172.21.15.16:0/1028480122 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f60e006de90 0x7f60e0070350 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:53.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:53.129+0000 7f60fccdd700 1 --2- 172.21.15.16:0/1028480122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60f80ff9d0 0x7f60f8198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:53.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:53.129+0000 7f60fccdd700 1 --2- 172.21.15.16:0/1028480122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60f8100bd0 0x7f60f8198c70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:44:53.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:53.129+0000 7f60fccdd700 1 -- 172.21.15.16:0/1028480122 >> 172.21.15.16:0/1028480122 conn(0x7f60f80faf70 msgr2=0x7f60f8068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:44:53.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:53.129+0000 7f60fccdd700 1 -- 172.21.15.16:0/1028480122 shutdown_connections 2024-04-02T04:44:53.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:44:53.129+0000 7f60fccdd700 1 -- 172.21.15.16:0/1028480122 wait complete. 2024-04-02T04:44:53.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:53 smithi027 ceph-mon[83036]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:44:53.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:53 smithi027 ceph-mon[83036]: pgmap v21: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 524 KiB/s wr, 108 op/s 2024-04-02T04:44:53.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:53 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/1028480122' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:44:54.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:53 smithi016 ceph-mon[93230]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:44:54.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:53 smithi016 ceph-mon[93230]: pgmap v21: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 524 KiB/s wr, 108 op/s 2024-04-02T04:44:54.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:53 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/1028480122' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:44:55.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:55 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:55.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:55 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:55.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:55 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:44:55.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:55 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:44:55.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:55 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:55.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:55 smithi027 ceph-mon[83036]: Reconfiguring mon.smithi016 (monmap changed)... 2024-04-02T04:44:55.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:55 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T04:44:55.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:55 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T04:44:55.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:55 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:44:55.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:55 smithi027 ceph-mon[83036]: Reconfiguring daemon mon.smithi016 on smithi016 2024-04-02T04:44:55.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:55 smithi027 ceph-mon[83036]: pgmap v22: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 548 KiB/s wr, 109 op/s 2024-04-02T04:44:55.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:55 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:55.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:55 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:55.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:55 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:44:55.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:55 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:44:55.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:55 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:55.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:55 smithi016 ceph-mon[93230]: Reconfiguring mon.smithi016 (monmap changed)... 2024-04-02T04:44:55.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:55 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T04:44:55.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:55 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T04:44:55.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:55 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:44:55.811 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:55 smithi016 ceph-mon[93230]: Reconfiguring daemon mon.smithi016 on smithi016 2024-04-02T04:44:55.811 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:55 smithi016 ceph-mon[93230]: pgmap v22: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 548 KiB/s wr, 109 op/s 2024-04-02T04:44:57.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:57 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:57.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:57 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:57.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:57 smithi027 ceph-mon[83036]: Reconfiguring mgr.smithi016.eqnebx (monmap changed)... 2024-04-02T04:44:57.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:57 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi016.eqnebx", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T04:44:57.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:57 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T04:44:57.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:57 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:44:57.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:57 smithi027 ceph-mon[83036]: Reconfiguring daemon mgr.smithi016.eqnebx on smithi016 2024-04-02T04:44:57.474 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:57 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:57.475 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:57 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:57.475 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:57 smithi016 ceph-mon[93230]: Reconfiguring mgr.smithi016.eqnebx (monmap changed)... 2024-04-02T04:44:57.475 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:57 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi016.eqnebx", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T04:44:57.475 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:57 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T04:44:57.475 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:57 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:44:57.475 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:57 smithi016 ceph-mon[93230]: Reconfiguring daemon mgr.smithi016.eqnebx on smithi016 2024-04-02T04:44:58.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:58 smithi027 ceph-mon[83036]: pgmap v23: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 6.1 KiB/s rd, 122 KiB/s wr, 29 op/s 2024-04-02T04:44:58.511 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:58 smithi016 ceph-mon[93230]: pgmap v23: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 6.1 KiB/s rd, 122 KiB/s wr, 29 op/s 2024-04-02T04:44:59.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:59 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:59.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:59 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:59.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:59 smithi016 ceph-mon[93230]: Reconfiguring ceph-exporter.smithi016 (monmap changed)... 2024-04-02T04:44:59.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:59 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi016", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-02T04:44:59.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:59 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi016", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-02T04:44:59.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:59 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi016", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-02T04:44:59.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:59 smithi016 ceph-mon[93230]: Unable to update caps for client.ceph-exporter.smithi016 2024-04-02T04:44:59.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:59 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi016"}]: dispatch 2024-04-02T04:44:59.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:59 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:44:59.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:59 smithi016 ceph-mon[93230]: Reconfiguring daemon ceph-exporter.smithi016 on smithi016 2024-04-02T04:44:59.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:44:59 smithi016 ceph-mon[93230]: pgmap v24: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 6.1 KiB/s rd, 122 KiB/s wr, 30 op/s 2024-04-02T04:44:59.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:59 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:59.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:59 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:44:59.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:59 smithi027 ceph-mon[83036]: Reconfiguring ceph-exporter.smithi016 (monmap changed)... 2024-04-02T04:44:59.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:59 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi016", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-02T04:44:59.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:59 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi016", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-02T04:44:59.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:59 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi016", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-02T04:44:59.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:59 smithi027 ceph-mon[83036]: Unable to update caps for client.ceph-exporter.smithi016 2024-04-02T04:44:59.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:59 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi016"}]: dispatch 2024-04-02T04:44:59.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:59 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:44:59.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:59 smithi027 ceph-mon[83036]: Reconfiguring daemon ceph-exporter.smithi016 on smithi016 2024-04-02T04:44:59.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:44:59 smithi027 ceph-mon[83036]: pgmap v24: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 6.1 KiB/s rd, 122 KiB/s wr, 30 op/s 2024-04-02T04:45:01.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:01 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:01.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:01 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:01.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:01 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi016", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-02T04:45:01.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:01 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:01.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:01 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:01.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:01 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:01.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:01 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi016", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-02T04:45:01.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:01 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:02.645 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:02 smithi016 ceph-mon[93230]: Reconfiguring crash.smithi016 (monmap changed)... 2024-04-02T04:45:02.645 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:02 smithi016 ceph-mon[93230]: Reconfiguring daemon crash.smithi016 on smithi016 2024-04-02T04:45:02.645 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:02 smithi016 ceph-mon[93230]: pgmap v25: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 24 KiB/s wr, 1 op/s 2024-04-02T04:45:02.645 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:02 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:02.645 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:02 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:02.645 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:02 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-02T04:45:02.645 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:02 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:02.645 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:02 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:45:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:02 smithi027 ceph-mon[83036]: Reconfiguring crash.smithi016 (monmap changed)... 2024-04-02T04:45:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:02 smithi027 ceph-mon[83036]: Reconfiguring daemon crash.smithi016 on smithi016 2024-04-02T04:45:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:02 smithi027 ceph-mon[83036]: pgmap v25: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 24 KiB/s wr, 1 op/s 2024-04-02T04:45:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:02 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:02 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:02 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-02T04:45:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:02 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:02 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:45:03.602 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:03 smithi016 ceph-mon[93230]: Reconfiguring osd.0 (monmap changed)... 2024-04-02T04:45:03.602 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:03 smithi016 ceph-mon[93230]: Reconfiguring daemon osd.0 on smithi016 2024-04-02T04:45:03.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:03 smithi027 ceph-mon[83036]: Reconfiguring osd.0 (monmap changed)... 2024-04-02T04:45:03.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:03 smithi027 ceph-mon[83036]: Reconfiguring daemon osd.0 on smithi016 2024-04-02T04:45:04.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:04 smithi016 ceph-mon[93230]: pgmap v26: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 24 KiB/s wr, 1 op/s 2024-04-02T04:45:04.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:04 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:04.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:04 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:04.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:04 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-02T04:45:04.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:04 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:04.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:04 smithi027 ceph-mon[83036]: pgmap v26: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 24 KiB/s wr, 1 op/s 2024-04-02T04:45:04.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:04 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:04.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:04 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:04.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:04 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-02T04:45:04.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:04 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:05.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:05 smithi027 ceph-mon[83036]: Reconfiguring osd.1 (monmap changed)... 2024-04-02T04:45:05.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:05 smithi027 ceph-mon[83036]: Reconfiguring daemon osd.1 on smithi016 2024-04-02T04:45:05.809 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:05 smithi016 ceph-mon[93230]: Reconfiguring osd.1 (monmap changed)... 2024-04-02T04:45:05.809 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:05 smithi016 ceph-mon[93230]: Reconfiguring daemon osd.1 on smithi016 2024-04-02T04:45:06.463 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:06 smithi016 ceph-mon[93230]: pgmap v27: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 24 KiB/s wr, 1 op/s 2024-04-02T04:45:06.463 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:06.463 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:06.463 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-02T04:45:06.463 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:06.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:06 smithi027 ceph-mon[83036]: pgmap v27: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 24 KiB/s wr, 1 op/s 2024-04-02T04:45:06.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:06.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:06.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-02T04:45:06.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:07.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:07 smithi027 ceph-mon[83036]: Reconfiguring osd.2 (monmap changed)... 2024-04-02T04:45:07.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:07 smithi027 ceph-mon[83036]: Reconfiguring daemon osd.2 on smithi016 2024-04-02T04:45:07.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:07 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:07.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:07 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:07.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:07 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi016.uhgkgg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:45:07.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:07 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:07.794 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:07 smithi016 ceph-mon[93230]: Reconfiguring osd.2 (monmap changed)... 2024-04-02T04:45:07.794 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:07 smithi016 ceph-mon[93230]: Reconfiguring daemon osd.2 on smithi016 2024-04-02T04:45:07.795 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:07 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:07.795 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:07 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:07.795 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:07 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi016.uhgkgg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:45:07.795 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:07 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:08.482 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:08 smithi016 ceph-mon[93230]: pgmap v28: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-02T04:45:08.482 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:08 smithi016 ceph-mon[93230]: Reconfiguring mds.cephfs.smithi016.uhgkgg (monmap changed)... 2024-04-02T04:45:08.482 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:08 smithi016 ceph-mon[93230]: Reconfiguring daemon mds.cephfs.smithi016.uhgkgg on smithi016 2024-04-02T04:45:08.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:08 smithi027 ceph-mon[83036]: pgmap v28: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-02T04:45:08.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:08 smithi027 ceph-mon[83036]: Reconfiguring mds.cephfs.smithi016.uhgkgg (monmap changed)... 2024-04-02T04:45:08.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:08 smithi027 ceph-mon[83036]: Reconfiguring daemon mds.cephfs.smithi016.uhgkgg on smithi016 2024-04-02T04:45:10.205 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:09 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:10.205 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:09 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:10.206 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:09 smithi016 ceph-mon[93230]: Reconfiguring mds.cephfs.smithi016.fyzrnm (monmap changed)... 2024-04-02T04:45:10.206 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:09 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi016.fyzrnm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:45:10.206 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:09 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:10.206 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:09 smithi016 ceph-mon[93230]: Reconfiguring daemon mds.cephfs.smithi016.fyzrnm on smithi016 2024-04-02T04:45:10.206 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:09 smithi016 ceph-mon[93230]: pgmap v29: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-02T04:45:10.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:09 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:10.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:09 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:10.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:09 smithi027 ceph-mon[83036]: Reconfiguring mds.cephfs.smithi016.fyzrnm (monmap changed)... 2024-04-02T04:45:10.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:09 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi016.fyzrnm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:45:10.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:09 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:10.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:09 smithi027 ceph-mon[83036]: Reconfiguring daemon mds.cephfs.smithi016.fyzrnm on smithi016 2024-04-02T04:45:10.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:09 smithi027 ceph-mon[83036]: pgmap v29: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-02T04:45:11.590 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:11 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:11.590 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:11 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:11.590 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:11 smithi027 ceph-mon[83036]: Reconfiguring ceph-exporter.smithi027 (monmap changed)... 2024-04-02T04:45:11.590 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:11 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi027", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-02T04:45:11.590 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:11 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi027", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-02T04:45:11.590 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:11 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi027", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-02T04:45:11.591 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:11 smithi027 ceph-mon[83036]: Unable to update caps for client.ceph-exporter.smithi027 2024-04-02T04:45:11.591 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:11 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi027"}]: dispatch 2024-04-02T04:45:11.591 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:11 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:11.591 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:11 smithi027 ceph-mon[83036]: Reconfiguring daemon ceph-exporter.smithi027 on smithi027 2024-04-02T04:45:11.591 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:11 smithi027 ceph-mon[83036]: pgmap v30: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:45:11.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:11 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:11.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:11 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:11.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:11 smithi016 ceph-mon[93230]: Reconfiguring ceph-exporter.smithi027 (monmap changed)... 2024-04-02T04:45:11.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:11 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi027", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-02T04:45:11.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:11 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi027", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-02T04:45:11.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:11 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi027", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-02T04:45:11.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:11 smithi016 ceph-mon[93230]: Unable to update caps for client.ceph-exporter.smithi027 2024-04-02T04:45:11.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:11 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi027"}]: dispatch 2024-04-02T04:45:11.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:11 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:11.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:11 smithi016 ceph-mon[93230]: Reconfiguring daemon ceph-exporter.smithi027 on smithi027 2024-04-02T04:45:11.811 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:11 smithi016 ceph-mon[93230]: pgmap v30: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:45:13.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:13 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:13.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:13 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:13.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:13 smithi027 ceph-mon[83036]: Reconfiguring crash.smithi027 (monmap changed)... 2024-04-02T04:45:13.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:13 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi027", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-02T04:45:13.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:13 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:13.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:13 smithi027 ceph-mon[83036]: Reconfiguring daemon crash.smithi027 on smithi027 2024-04-02T04:45:13.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:13 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:13.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:13 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:13.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:13 smithi016 ceph-mon[93230]: Reconfiguring crash.smithi027 (monmap changed)... 2024-04-02T04:45:13.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:13 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi027", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-02T04:45:13.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:13 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:13.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:13 smithi016 ceph-mon[93230]: Reconfiguring daemon crash.smithi027 on smithi027 2024-04-02T04:45:14.544 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:14 smithi027 ceph-mon[83036]: pgmap v31: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:45:14.545 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:14 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:14.545 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:14 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:14.545 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:14 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi027.kjzcsc", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T04:45:14.545 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:14 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T04:45:14.545 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:14 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:14.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:14 smithi016 ceph-mon[93230]: pgmap v31: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:45:14.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:14 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:14.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:14 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:14.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:14 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi027.kjzcsc", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T04:45:14.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:14 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T04:45:14.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:14 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:15.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:15 smithi027 ceph-mon[83036]: Reconfiguring mgr.smithi027.kjzcsc (monmap changed)... 2024-04-02T04:45:15.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:15 smithi027 ceph-mon[83036]: Reconfiguring daemon mgr.smithi027.kjzcsc on smithi027 2024-04-02T04:45:15.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:15 smithi016 ceph-mon[93230]: Reconfiguring mgr.smithi027.kjzcsc (monmap changed)... 2024-04-02T04:45:15.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:15 smithi016 ceph-mon[93230]: Reconfiguring daemon mgr.smithi027.kjzcsc on smithi027 2024-04-02T04:45:16.489 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:16 smithi027 ceph-mon[83036]: pgmap v32: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:45:16.489 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:16 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:16.489 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:16 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:16.489 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:16 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T04:45:16.489 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:16 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T04:45:16.489 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:16 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:16.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:16 smithi016 ceph-mon[93230]: pgmap v32: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:45:16.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:16 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:16.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:16 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:16.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:16 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T04:45:16.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:16 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T04:45:16.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:16 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:17.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:17 smithi016 ceph-mon[93230]: Reconfiguring mon.smithi027 (monmap changed)... 2024-04-02T04:45:17.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:17 smithi016 ceph-mon[93230]: Reconfiguring daemon mon.smithi027 on smithi027 2024-04-02T04:45:17.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:17 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:45:17.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:17 smithi027 ceph-mon[83036]: Reconfiguring mon.smithi027 (monmap changed)... 2024-04-02T04:45:17.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:17 smithi027 ceph-mon[83036]: Reconfiguring daemon mon.smithi027 on smithi027 2024-04-02T04:45:17.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:17 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:45:18.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:18 smithi027 ceph-mon[83036]: pgmap v33: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:45:18.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:18 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:18.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:18 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:18.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:18 smithi027 ceph-mon[83036]: Reconfiguring osd.3 (monmap changed)... 2024-04-02T04:45:18.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:18 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-02T04:45:18.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:18 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:18.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:18 smithi027 ceph-mon[83036]: Reconfiguring daemon osd.3 on smithi027 2024-04-02T04:45:18.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:18 smithi016 ceph-mon[93230]: pgmap v33: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:45:18.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:18 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:18.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:18 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:18.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:18 smithi016 ceph-mon[93230]: Reconfiguring osd.3 (monmap changed)... 2024-04-02T04:45:18.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:18 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-02T04:45:18.563 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:18 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:18.563 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:18 smithi016 ceph-mon[93230]: Reconfiguring daemon osd.3 on smithi027 2024-04-02T04:45:20.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:20 smithi027 ceph-mon[83036]: pgmap v34: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:45:20.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:20 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:20.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:20 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:20.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:20 smithi027 ceph-mon[83036]: Reconfiguring osd.4 (monmap changed)... 2024-04-02T04:45:20.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:20 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-02T04:45:20.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:20 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:20.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:20 smithi027 ceph-mon[83036]: Reconfiguring daemon osd.4 on smithi027 2024-04-02T04:45:20.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:20 smithi016 ceph-mon[93230]: pgmap v34: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:45:20.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:20 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:20.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:20 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:20.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:20 smithi016 ceph-mon[93230]: Reconfiguring osd.4 (monmap changed)... 2024-04-02T04:45:20.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:20 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-02T04:45:20.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:20 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:20.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:20 smithi016 ceph-mon[93230]: Reconfiguring daemon osd.4 on smithi027 2024-04-02T04:45:22.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:21 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:22.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:21 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:22.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:21 smithi027 ceph-mon[83036]: Reconfiguring osd.5 (monmap changed)... 2024-04-02T04:45:22.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:21 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-02T04:45:22.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:21 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:22.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:21 smithi027 ceph-mon[83036]: Reconfiguring daemon osd.5 on smithi027 2024-04-02T04:45:22.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:21 smithi027 ceph-mon[83036]: pgmap v35: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:45:22.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:21 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:22.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:21 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:22.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:21 smithi016 ceph-mon[93230]: Reconfiguring osd.5 (monmap changed)... 2024-04-02T04:45:22.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:21 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-02T04:45:22.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:21 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:22.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:21 smithi016 ceph-mon[93230]: Reconfiguring daemon osd.5 on smithi027 2024-04-02T04:45:22.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:21 smithi016 ceph-mon[93230]: pgmap v35: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:45:23.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.249+0000 7fbb10851700 1 -- 172.21.15.16:0/2345737413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb0c100ae0 msgr2=0x7fbb0c100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:23.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.249+0000 7fbb10851700 1 --2- 172.21.15.16:0/2345737413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb0c100ae0 0x7fbb0c100f00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbafc009870 tx=0x7fbafc009b80 comp rx=0 tx=0).stop 2024-04-02T04:45:23.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.249+0000 7fbb10851700 1 -- 172.21.15.16:0/2345737413 shutdown_connections 2024-04-02T04:45:23.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.249+0000 7fbb10851700 1 --2- 172.21.15.16:0/2345737413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb0c101ce0 0x7fbb0c102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:23.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.249+0000 7fbb10851700 1 --2- 172.21.15.16:0/2345737413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb0c100ae0 0x7fbb0c100f00 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:23.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.249+0000 7fbb10851700 1 -- 172.21.15.16:0/2345737413 >> 172.21.15.16:0/2345737413 conn(0x7fbb0c0fc0a0 msgr2=0x7fbb0c0fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:23.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.249+0000 7fbb10851700 1 -- 172.21.15.16:0/2345737413 shutdown_connections 2024-04-02T04:45:23.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.249+0000 7fbb10851700 1 -- 172.21.15.16:0/2345737413 wait complete. 2024-04-02T04:45:23.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.250+0000 7fbb10851700 1 Processor -- start 2024-04-02T04:45:23.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.250+0000 7fbb10851700 1 -- start start 2024-04-02T04:45:23.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.250+0000 7fbb10851700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb0c100ae0 0x7fbb0c1964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:23.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.250+0000 7fbb10851700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb0c101ce0 0x7fbb0c196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:23.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.250+0000 7fbb10851700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0c197030 con 0x7fbb0c101ce0 2024-04-02T04:45:23.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.250+0000 7fbb10851700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0c197170 con 0x7fbb0c100ae0 2024-04-02T04:45:23.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.250+0000 7fbb09d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb0c101ce0 0x7fbb0c196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:23.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.250+0000 7fbb0a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb0c100ae0 0x7fbb0c1964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:23.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.250+0000 7fbb09d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb0c101ce0 0x7fbb0c196a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44630/0 (socket says 172.21.15.16:44630) 2024-04-02T04:45:23.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.250+0000 7fbb09d9b700 1 -- 172.21.15.16:0/480173232 learned_addr learned my addr 172.21.15.16:0/480173232 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:45:23.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.251+0000 7fbb0a59c700 1 -- 172.21.15.16:0/480173232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb0c101ce0 msgr2=0x7fbb0c196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:23.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.251+0000 7fbb0a59c700 1 --2- 172.21.15.16:0/480173232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb0c101ce0 0x7fbb0c196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:23.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.251+0000 7fbb0a59c700 1 -- 172.21.15.16:0/480173232 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb000095f0 con 0x7fbb0c100ae0 2024-04-02T04:45:23.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.251+0000 7fbb0a59c700 1 --2- 172.21.15.16:0/480173232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb0c100ae0 0x7fbb0c1964d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbafc009230 tx=0x7fbafc00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:23.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.251+0000 7fbafb7fe700 1 -- 172.21.15.16:0/480173232 <== mon.1 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbafc01c070 con 0x7fbb0c100ae0 2024-04-02T04:45:23.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.251+0000 7fbafb7fe700 1 -- 172.21.15.16:0/480173232 <== mon.1 v2:172.21.15.27:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbafc00fd70 con 0x7fbb0c100ae0 2024-04-02T04:45:23.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.251+0000 7fbb10851700 1 -- 172.21.15.16:0/480173232 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbafc009520 con 0x7fbb0c100ae0 2024-04-02T04:45:23.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.251+0000 7fbafb7fe700 1 -- 172.21.15.16:0/480173232 <== mon.1 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbafc022d70 con 0x7fbb0c100ae0 2024-04-02T04:45:23.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.251+0000 7fbb10851700 1 -- 172.21.15.16:0/480173232 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb0c19bf50 con 0x7fbb0c100ae0 2024-04-02T04:45:23.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.252+0000 7fbb10851700 1 -- 172.21.15.16:0/480173232 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb0c190660 con 0x7fbb0c100ae0 2024-04-02T04:45:23.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.253+0000 7fbafb7fe700 1 -- 172.21.15.16:0/480173232 <== mon.1 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 37) v1 ==== 91963+0+0 (secure 0 0 0) 0x7fbafc020070 con 0x7fbb0c100ae0 2024-04-02T04:45:23.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.253+0000 7fbafb7fe700 1 --2- 172.21.15.16:0/480173232 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fbaf406dea0 0x7fbaf4070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:23.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.253+0000 7fbb09d9b700 1 --2- 172.21.15.16:0/480173232 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fbaf406dea0 0x7fbaf4070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:23.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.253+0000 7fbafb7fe700 1 -- 172.21.15.16:0/480173232 <== mon.1 v2:172.21.15.27:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fbafc08ee80 con 0x7fbb0c100ae0 2024-04-02T04:45:23.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.254+0000 7fbb09d9b700 1 --2- 172.21.15.16:0/480173232 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fbaf406dea0 0x7fbaf4070360 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbb00005d10 tx=0x7fbb00009310 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:23.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.259+0000 7fbafb7fe700 1 -- 172.21.15.16:0/480173232 <== mon.1 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbafc05bda0 con 0x7fbb0c100ae0 2024-04-02T04:45:23.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.479+0000 7fbb10851700 1 -- 172.21.15.16:0/480173232 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb0c02cd20 con 0x7fbaf406dea0 2024-04-02T04:45:23.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.481+0000 7fbafb7fe700 1 -- 172.21.15.16:0/480173232 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fbb0c02cd20 con 0x7fbaf406dea0 2024-04-02T04:45:23.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.483+0000 7fbb10851700 1 -- 172.21.15.16:0/480173232 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fbaf406dea0 msgr2=0x7fbaf4070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:23.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.483+0000 7fbb10851700 1 --2- 172.21.15.16:0/480173232 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fbaf406dea0 0x7fbaf4070360 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbb00005d10 tx=0x7fbb00009310 comp rx=0 tx=0).stop 2024-04-02T04:45:23.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.483+0000 7fbb10851700 1 -- 172.21.15.16:0/480173232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb0c100ae0 msgr2=0x7fbb0c1964d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:23.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.483+0000 7fbb10851700 1 --2- 172.21.15.16:0/480173232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb0c100ae0 0x7fbb0c1964d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbafc009230 tx=0x7fbafc00f690 comp rx=0 tx=0).stop 2024-04-02T04:45:23.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.483+0000 7fbb10851700 1 -- 172.21.15.16:0/480173232 shutdown_connections 2024-04-02T04:45:23.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.483+0000 7fbb10851700 1 --2- 172.21.15.16:0/480173232 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fbaf406dea0 0x7fbaf4070360 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:23.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.483+0000 7fbb10851700 1 --2- 172.21.15.16:0/480173232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb0c100ae0 0x7fbb0c1964d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:23.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.483+0000 7fbb10851700 1 --2- 172.21.15.16:0/480173232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb0c101ce0 0x7fbb0c196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:23.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.483+0000 7fbb10851700 1 -- 172.21.15.16:0/480173232 >> 172.21.15.16:0/480173232 conn(0x7fbb0c0fc0a0 msgr2=0x7fbb0c105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:23.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.483+0000 7fbb10851700 1 -- 172.21.15.16:0/480173232 shutdown_connections 2024-04-02T04:45:23.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.483+0000 7fbb10851700 1 -- 172.21.15.16:0/480173232 wait complete. 2024-04-02T04:45:23.497 INFO:teuthology.orchestra.run.smithi016.stdout:true 2024-04-02T04:45:23.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.593+0000 7f7e2f03e700 1 -- 172.21.15.16:0/138724939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e28100bd0 msgr2=0x7f7e28101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:23.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.593+0000 7f7e2f03e700 1 --2- 172.21.15.16:0/138724939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e28100bd0 0x7f7e28101030 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009910 tx=0x7f7e10009c20 comp rx=0 tx=0).stop 2024-04-02T04:45:23.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.593+0000 7f7e2f03e700 1 -- 172.21.15.16:0/138724939 shutdown_connections 2024-04-02T04:45:23.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.593+0000 7f7e2f03e700 1 --2- 172.21.15.16:0/138724939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e28100bd0 0x7f7e28101030 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:23.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.593+0000 7f7e2f03e700 1 --2- 172.21.15.16:0/138724939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e280ff9d0 0x7f7e280ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:23.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.593+0000 7f7e2f03e700 1 -- 172.21.15.16:0/138724939 >> 172.21.15.16:0/138724939 conn(0x7f7e280faf70 msgr2=0x7f7e280fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:23.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.593+0000 7f7e2f03e700 1 -- 172.21.15.16:0/138724939 shutdown_connections 2024-04-02T04:45:23.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.593+0000 7f7e2f03e700 1 -- 172.21.15.16:0/138724939 wait complete. 2024-04-02T04:45:23.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.593+0000 7f7e2f03e700 1 Processor -- start 2024-04-02T04:45:23.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.593+0000 7f7e2f03e700 1 -- start start 2024-04-02T04:45:23.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.594+0000 7f7e2f03e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e280ff9d0 0x7f7e28196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:23.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.594+0000 7f7e2f03e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e28100bd0 0x7f7e28196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:23.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.594+0000 7f7e2f03e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e281970e0 con 0x7f7e28100bd0 2024-04-02T04:45:23.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.594+0000 7f7e2f03e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e28197220 con 0x7f7e280ff9d0 2024-04-02T04:45:23.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.594+0000 7f7e27fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e28100bd0 0x7f7e28196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:23.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.594+0000 7f7e27fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e28100bd0 0x7f7e28196ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44652/0 (socket says 172.21.15.16:44652) 2024-04-02T04:45:23.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.594+0000 7f7e27fff700 1 -- 172.21.15.16:0/921557040 learned_addr learned my addr 172.21.15.16:0/921557040 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:45:23.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.594+0000 7f7e2cdda700 1 --2- 172.21.15.16:0/921557040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e280ff9d0 0x7f7e28196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:23.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.594+0000 7f7e27fff700 1 -- 172.21.15.16:0/921557040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e280ff9d0 msgr2=0x7f7e28196580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:23.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.594+0000 7f7e27fff700 1 --2- 172.21.15.16:0/921557040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e280ff9d0 0x7f7e28196580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:23.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.594+0000 7f7e27fff700 1 -- 172.21.15.16:0/921557040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e100095f0 con 0x7f7e28100bd0 2024-04-02T04:45:23.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.594+0000 7f7e27fff700 1 --2- 172.21.15.16:0/921557040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e28100bd0 0x7f7e28196ac0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7e100092e0 tx=0x7f7e1000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:23.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.594+0000 7f7e25ffb700 1 -- 172.21.15.16:0/921557040 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e1001d070 con 0x7f7e28100bd0 2024-04-02T04:45:23.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.594+0000 7f7e2f03e700 1 -- 172.21.15.16:0/921557040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e2819bbd0 con 0x7f7e28100bd0 2024-04-02T04:45:23.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.594+0000 7f7e25ffb700 1 -- 172.21.15.16:0/921557040 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e1000f460 con 0x7f7e28100bd0 2024-04-02T04:45:23.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.594+0000 7f7e25ffb700 1 -- 172.21.15.16:0/921557040 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e10021640 con 0x7f7e28100bd0 2024-04-02T04:45:23.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.595+0000 7f7e2f03e700 1 -- 172.21.15.16:0/921557040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e2819c140 con 0x7f7e28100bd0 2024-04-02T04:45:23.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.596+0000 7f7e2f03e700 1 -- 172.21.15.16:0/921557040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e2804ea80 con 0x7f7e28100bd0 2024-04-02T04:45:23.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.598+0000 7f7e25ffb700 1 -- 172.21.15.16:0/921557040 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 37) v1 ==== 91963+0+0 (secure 0 0 0) 0x7f7e1000fad0 con 0x7f7e28100bd0 2024-04-02T04:45:23.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.598+0000 7f7e25ffb700 1 --2- 172.21.15.16:0/921557040 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f7e1406de90 0x7f7e14070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:23.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.598+0000 7f7e25ffb700 1 -- 172.21.15.16:0/921557040 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f7e1008f440 con 0x7f7e28100bd0 2024-04-02T04:45:23.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.598+0000 7f7e2cdda700 1 --2- 172.21.15.16:0/921557040 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f7e1406de90 0x7f7e14070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:23.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.599+0000 7f7e2cdda700 1 --2- 172.21.15.16:0/921557040 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f7e1406de90 0x7f7e14070350 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7e28100a30 tx=0x7f7e1800c040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:23.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.602+0000 7f7e25ffb700 1 -- 172.21.15.16:0/921557040 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7e1005c360 con 0x7f7e28100bd0 2024-04-02T04:45:23.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.828+0000 7f7e2f03e700 1 -- 172.21.15.16:0/921557040 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e280fcf20 con 0x7f7e1406de90 2024-04-02T04:45:23.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.830+0000 7f7e25ffb700 1 -- 172.21.15.16:0/921557040 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f7e280fcf20 con 0x7f7e1406de90 2024-04-02T04:45:23.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.832+0000 7f7e2f03e700 1 -- 172.21.15.16:0/921557040 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f7e1406de90 msgr2=0x7f7e14070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:23.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.832+0000 7f7e2f03e700 1 --2- 172.21.15.16:0/921557040 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f7e1406de90 0x7f7e14070350 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7e28100a30 tx=0x7f7e1800c040 comp rx=0 tx=0).stop 2024-04-02T04:45:23.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.832+0000 7f7e2f03e700 1 -- 172.21.15.16:0/921557040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e28100bd0 msgr2=0x7f7e28196ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:23.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.832+0000 7f7e2f03e700 1 --2- 172.21.15.16:0/921557040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e28100bd0 0x7f7e28196ac0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7e100092e0 tx=0x7f7e1000bd70 comp rx=0 tx=0).stop 2024-04-02T04:45:23.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.832+0000 7f7e2f03e700 1 -- 172.21.15.16:0/921557040 shutdown_connections 2024-04-02T04:45:23.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.832+0000 7f7e2f03e700 1 --2- 172.21.15.16:0/921557040 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f7e1406de90 0x7f7e14070350 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:23.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.832+0000 7f7e2f03e700 1 --2- 172.21.15.16:0/921557040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e280ff9d0 0x7f7e28196580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:23.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.832+0000 7f7e2f03e700 1 --2- 172.21.15.16:0/921557040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e28100bd0 0x7f7e28196ac0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:23.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.832+0000 7f7e2f03e700 1 -- 172.21.15.16:0/921557040 >> 172.21.15.16:0/921557040 conn(0x7f7e280faf70 msgr2=0x7f7e280fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:23.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.832+0000 7f7e2f03e700 1 -- 172.21.15.16:0/921557040 shutdown_connections 2024-04-02T04:45:23.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.832+0000 7f7e2f03e700 1 -- 172.21.15.16:0/921557040 wait complete. 2024-04-02T04:45:23.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:23 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:23.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:23 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:23.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:23 smithi027 ceph-mon[83036]: Reconfiguring mds.cephfs.smithi027.zwrfoo (monmap changed)... 2024-04-02T04:45:23.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:23 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi027.zwrfoo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:45:23.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:23 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:23.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:23 smithi027 ceph-mon[83036]: Reconfiguring daemon mds.cephfs.smithi027.zwrfoo on smithi027 2024-04-02T04:45:23.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:23 smithi027 ceph-mon[83036]: pgmap v36: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:45:23.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.937+0000 7f55f1180700 1 -- 172.21.15.16:0/1797480916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55ec100ad0 msgr2=0x7f55ec100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:23.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.937+0000 7f55f1180700 1 --2- 172.21.15.16:0/1797480916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55ec100ad0 0x7f55ec100ef0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f55d4009910 tx=0x7f55d4009c20 comp rx=0 tx=0).stop 2024-04-02T04:45:23.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.937+0000 7f55f1180700 1 -- 172.21.15.16:0/1797480916 shutdown_connections 2024-04-02T04:45:23.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.937+0000 7f55f1180700 1 --2- 172.21.15.16:0/1797480916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55ec101cd0 0x7f55ec102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:23.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.937+0000 7f55f1180700 1 --2- 172.21.15.16:0/1797480916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55ec100ad0 0x7f55ec100ef0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:23.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.937+0000 7f55f1180700 1 -- 172.21.15.16:0/1797480916 >> 172.21.15.16:0/1797480916 conn(0x7f55ec0fc070 msgr2=0x7f55ec0fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:23.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.937+0000 7f55f1180700 1 -- 172.21.15.16:0/1797480916 shutdown_connections 2024-04-02T04:45:23.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.937+0000 7f55f1180700 1 -- 172.21.15.16:0/1797480916 wait complete. 2024-04-02T04:45:23.941 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:23 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:23.941 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:23 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:23.941 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:23 smithi016 ceph-mon[93230]: Reconfiguring mds.cephfs.smithi027.zwrfoo (monmap changed)... 2024-04-02T04:45:23.941 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:23 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi027.zwrfoo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:45:23.941 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:23 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:23.941 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:23 smithi016 ceph-mon[93230]: Reconfiguring daemon mds.cephfs.smithi027.zwrfoo on smithi027 2024-04-02T04:45:23.942 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:23 smithi016 ceph-mon[93230]: pgmap v36: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:45:23.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.938+0000 7f55f1180700 1 Processor -- start 2024-04-02T04:45:23.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.938+0000 7f55f1180700 1 -- start start 2024-04-02T04:45:23.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.938+0000 7f55f1180700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55ec100ad0 0x7f55ec1964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:23.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.938+0000 7f55f1180700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55ec101cd0 0x7f55ec1969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:23.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.938+0000 7f55f1180700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55ec197010 con 0x7f55ec100ad0 2024-04-02T04:45:23.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.938+0000 7f55f1180700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55ec197150 con 0x7f55ec101cd0 2024-04-02T04:45:23.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.938+0000 7f55ead9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55ec100ad0 0x7f55ec1964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:23.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.939+0000 7f55ead9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55ec100ad0 0x7f55ec1964b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44664/0 (socket says 172.21.15.16:44664) 2024-04-02T04:45:23.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.939+0000 7f55ea59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55ec101cd0 0x7f55ec1969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:23.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.939+0000 7f55ead9d700 1 -- 172.21.15.16:0/2351771516 learned_addr learned my addr 172.21.15.16:0/2351771516 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:45:23.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.939+0000 7f55ead9d700 1 -- 172.21.15.16:0/2351771516 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55ec101cd0 msgr2=0x7f55ec1969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:23.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.939+0000 7f55ead9d700 1 --2- 172.21.15.16:0/2351771516 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55ec101cd0 0x7f55ec1969f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:23.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.939+0000 7f55ead9d700 1 -- 172.21.15.16:0/2351771516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d40095f0 con 0x7f55ec100ad0 2024-04-02T04:45:23.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.939+0000 7f55ea59c700 1 --2- 172.21.15.16:0/2351771516 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55ec101cd0 0x7f55ec1969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T04:45:23.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.939+0000 7f55ead9d700 1 --2- 172.21.15.16:0/2351771516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55ec100ad0 0x7f55ec1964b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f55d4005f00 tx=0x7f55d400b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:23.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.939+0000 7f55e3fff700 1 -- 172.21.15.16:0/2351771516 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55d401d070 con 0x7f55ec100ad0 2024-04-02T04:45:23.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.939+0000 7f55f1180700 1 -- 172.21.15.16:0/2351771516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55ec19bb50 con 0x7f55ec100ad0 2024-04-02T04:45:23.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.939+0000 7f55e3fff700 1 -- 172.21.15.16:0/2351771516 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f55d40052a0 con 0x7f55ec100ad0 2024-04-02T04:45:23.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.939+0000 7f55e3fff700 1 -- 172.21.15.16:0/2351771516 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55d4017400 con 0x7f55ec100ad0 2024-04-02T04:45:23.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.939+0000 7f55f1180700 1 -- 172.21.15.16:0/2351771516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55ec19c040 con 0x7f55ec100ad0 2024-04-02T04:45:23.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.940+0000 7f55f1180700 1 -- 172.21.15.16:0/2351771516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55ec190650 con 0x7f55ec100ad0 2024-04-02T04:45:23.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.941+0000 7f55e3fff700 1 -- 172.21.15.16:0/2351771516 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 37) v1 ==== 91963+0+0 (secure 0 0 0) 0x7f55d400f820 con 0x7f55ec100ad0 2024-04-02T04:45:23.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.941+0000 7f55e3fff700 1 --2- 172.21.15.16:0/2351771516 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f55d806dea0 0x7f55d8070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:23.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.941+0000 7f55e3fff700 1 -- 172.21.15.16:0/2351771516 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f55d408f170 con 0x7f55ec100ad0 2024-04-02T04:45:23.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.941+0000 7f55ea59c700 1 --2- 172.21.15.16:0/2351771516 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f55d806dea0 0x7f55d8070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:23.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.942+0000 7f55ea59c700 1 --2- 172.21.15.16:0/2351771516 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f55d806dea0 0x7f55d8070360 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f55dc011490 tx=0x7f55dc009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:23.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:23.946+0000 7f55e3fff700 1 -- 172.21.15.16:0/2351771516 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f55d405c010 con 0x7f55ec100ad0 2024-04-02T04:45:24.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.171+0000 7f55f1180700 1 -- 172.21.15.16:0/2351771516 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f55ec02cd70 con 0x7f55d806dea0 2024-04-02T04:45:24.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.182+0000 7f55e3fff700 1 -- 172.21.15.16:0/2351771516 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f55ec02cd70 con 0x7f55d806dea0 2024-04-02T04:45:24.186 INFO:teuthology.orchestra.run.smithi016.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:45:24.186 INFO:teuthology.orchestra.run.smithi016.stdout:alertmanager.smithi016 smithi016 *:9093,9094 running (9m) 60s ago 11m 24.2M - 0.25.0 c8568f914cd2 b95d8354b016 2024-04-02T04:45:24.186 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi016 smithi016 running (11m) 60s ago 11m 7814k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 9de6c16a487f 2024-04-02T04:45:24.186 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi027 smithi027 running (10m) 34s ago 10m 7884k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 99956592471a 2024-04-02T04:45:24.186 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi016 smithi016 running (11m) 60s ago 11m 6660k - 18.2.2-1059-g4d05f4ca 909dce4fe53d bd73919b0711 2024-04-02T04:45:24.186 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi027 smithi027 running (10m) 34s ago 10m 6660k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 7a54c37f202e 2024-04-02T04:45:24.186 INFO:teuthology.orchestra.run.smithi016.stdout:grafana.smithi016 smithi016 *:3000 running (9m) 60s ago 11m 85.6M - 9.4.7 954c08fa6188 7c43d133569c 2024-04-02T04:45:24.186 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.fyzrnm smithi016 running (5m) 60s ago 5m 16.6M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 70f28b9572a6 2024-04-02T04:45:24.186 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.uhgkgg smithi016 running (5m) 60s ago 5m 14.9M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 0aa4d16562a6 2024-04-02T04:45:24.186 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.sdublg smithi027 running (5m) 34s ago 5m 18.8M - 18.2.2-1059-g4d05f4ca 909dce4fe53d a538451099db 2024-04-02T04:45:24.187 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.zwrfoo smithi027 running (5m) 34s ago 5m 180M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 4cb3fa428d71 2024-04-02T04:45:24.187 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi016.eqnebx smithi016 *:8443,9283,8765 running (2m) 60s ago 12m 471M - 19.0.0-1979-g6cec793d 78f830190839 d61ed91b4577 2024-04-02T04:45:24.187 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi027.kjzcsc smithi027 *:8443,9283,8765 running (101s) 34s ago 10m 448M - 19.0.0-1979-g6cec793d 78f830190839 5072a5ffc761 2024-04-02T04:45:24.187 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi016 smithi016 running (76s) 60s ago 13m 27.7M 2048M 19.0.0-1979-g6cec793d 78f830190839 eabfc84f1cdf 2024-04-02T04:45:24.187 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi027 smithi027 running (49s) 34s ago 10m 24.4M 2048M 19.0.0-1979-g6cec793d 78f830190839 fcd68b20dcc5 2024-04-02T04:45:24.187 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi016 smithi016 *:9100 running (11m) 60s ago 11m 15.0M - 1.5.0 0da6a335fe13 09f54c65b53c 2024-04-02T04:45:24.187 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi027 smithi027 *:9100 running (10m) 34s ago 10m 15.2M - 1.5.0 0da6a335fe13 e41a657c7443 2024-04-02T04:45:24.187 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 smithi016 running (9m) 60s ago 9m 193M 1217M 18.2.2-1059-g4d05f4ca 909dce4fe53d 3f40108debd6 2024-04-02T04:45:24.187 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 smithi016 running (9m) 60s ago 8m 206M 1217M 18.2.2-1059-g4d05f4ca 909dce4fe53d 7361abc9e1b6 2024-04-02T04:45:24.187 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 smithi016 running (8m) 60s ago 8m 169M 1217M 18.2.2-1059-g4d05f4ca 909dce4fe53d 52e592b4d1f2 2024-04-02T04:45:24.187 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 smithi027 running (8m) 34s ago 8m 232M 2240M 18.2.2-1059-g4d05f4ca 909dce4fe53d cf7c7fe022f0 2024-04-02T04:45:24.187 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 smithi027 running (7m) 34s ago 7m 207M 2240M 18.2.2-1059-g4d05f4ca 909dce4fe53d 8cbbedf8cac9 2024-04-02T04:45:24.187 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 smithi027 running (7m) 34s ago 7m 190M 2240M 18.2.2-1059-g4d05f4ca 909dce4fe53d ac6094415529 2024-04-02T04:45:24.187 INFO:teuthology.orchestra.run.smithi016.stdout:prometheus.smithi016 smithi016 *:9095 running (115s) 60s ago 10m 51.1M - 2.43.0 a07b618ecd1d 466f00d73ddd 2024-04-02T04:45:24.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.184+0000 7f55f1180700 1 -- 172.21.15.16:0/2351771516 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f55d806dea0 msgr2=0x7f55d8070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:24.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.184+0000 7f55f1180700 1 --2- 172.21.15.16:0/2351771516 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f55d806dea0 0x7f55d8070360 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f55dc011490 tx=0x7f55dc009260 comp rx=0 tx=0).stop 2024-04-02T04:45:24.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.184+0000 7f55f1180700 1 -- 172.21.15.16:0/2351771516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55ec100ad0 msgr2=0x7f55ec1964b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:24.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.184+0000 7f55f1180700 1 --2- 172.21.15.16:0/2351771516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55ec100ad0 0x7f55ec1964b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f55d4005f00 tx=0x7f55d400b5e0 comp rx=0 tx=0).stop 2024-04-02T04:45:24.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.184+0000 7f55f1180700 1 -- 172.21.15.16:0/2351771516 shutdown_connections 2024-04-02T04:45:24.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.185+0000 7f55f1180700 1 --2- 172.21.15.16:0/2351771516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55ec100ad0 0x7f55ec1964b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:24.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.185+0000 7f55f1180700 1 --2- 172.21.15.16:0/2351771516 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f55d806dea0 0x7f55d8070360 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:24.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.185+0000 7f55f1180700 1 --2- 172.21.15.16:0/2351771516 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55ec101cd0 0x7f55ec1969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:24.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.185+0000 7f55f1180700 1 -- 172.21.15.16:0/2351771516 >> 172.21.15.16:0/2351771516 conn(0x7f55ec0fc070 msgr2=0x7f55ec105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:24.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.185+0000 7f55f1180700 1 -- 172.21.15.16:0/2351771516 shutdown_connections 2024-04-02T04:45:24.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.185+0000 7f55f1180700 1 -- 172.21.15.16:0/2351771516 wait complete. 2024-04-02T04:45:24.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.291+0000 7fe4cd77c700 1 -- 172.21.15.16:0/157715091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4c8100ae0 msgr2=0x7fe4c8100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:24.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.291+0000 7fe4cd77c700 1 --2- 172.21.15.16:0/157715091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4c8100ae0 0x7fe4c8100f00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe4b800b0a0 tx=0x7fe4b800b3b0 comp rx=0 tx=0).stop 2024-04-02T04:45:24.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.291+0000 7fe4cd77c700 1 -- 172.21.15.16:0/157715091 shutdown_connections 2024-04-02T04:45:24.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.291+0000 7fe4cd77c700 1 --2- 172.21.15.16:0/157715091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4c8101ce0 0x7fe4c8102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:24.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.291+0000 7fe4cd77c700 1 --2- 172.21.15.16:0/157715091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4c8100ae0 0x7fe4c8100f00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:24.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.291+0000 7fe4cd77c700 1 -- 172.21.15.16:0/157715091 >> 172.21.15.16:0/157715091 conn(0x7fe4c80fc0a0 msgr2=0x7fe4c80fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:24.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.291+0000 7fe4cd77c700 1 -- 172.21.15.16:0/157715091 shutdown_connections 2024-04-02T04:45:24.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.291+0000 7fe4cd77c700 1 -- 172.21.15.16:0/157715091 wait complete. 2024-04-02T04:45:24.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.292+0000 7fe4cd77c700 1 Processor -- start 2024-04-02T04:45:24.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.292+0000 7fe4cd77c700 1 -- start start 2024-04-02T04:45:24.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.292+0000 7fe4cd77c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4c8100ae0 0x7fe4c81964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:24.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.292+0000 7fe4cd77c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4c8101ce0 0x7fe4c8196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:24.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.292+0000 7fe4cd77c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c8197030 con 0x7fe4c8101ce0 2024-04-02T04:45:24.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.292+0000 7fe4cd77c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c8197170 con 0x7fe4c8100ae0 2024-04-02T04:45:24.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.292+0000 7fe4c67fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4c8101ce0 0x7fe4c8196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:24.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.292+0000 7fe4c67fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4c8101ce0 0x7fe4c8196a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44676/0 (socket says 172.21.15.16:44676) 2024-04-02T04:45:24.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.292+0000 7fe4c67fc700 1 -- 172.21.15.16:0/3471282884 learned_addr learned my addr 172.21.15.16:0/3471282884 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:45:24.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.292+0000 7fe4c6ffd700 1 --2- 172.21.15.16:0/3471282884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4c8100ae0 0x7fe4c81964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:24.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.293+0000 7fe4c67fc700 1 -- 172.21.15.16:0/3471282884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4c8100ae0 msgr2=0x7fe4c81964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:24.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.293+0000 7fe4c67fc700 1 --2- 172.21.15.16:0/3471282884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4c8100ae0 0x7fe4c81964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:24.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.293+0000 7fe4c67fc700 1 -- 172.21.15.16:0/3471282884 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b8009d00 con 0x7fe4c8101ce0 2024-04-02T04:45:24.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.293+0000 7fe4c6ffd700 1 --2- 172.21.15.16:0/3471282884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4c8100ae0 0x7fe4c81964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T04:45:24.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.293+0000 7fe4c67fc700 1 --2- 172.21.15.16:0/3471282884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4c8101ce0 0x7fe4c8196a10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe4b000e920 tx=0x7fe4b000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:24.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.293+0000 7fe4bffff700 1 -- 172.21.15.16:0/3471282884 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4b000c9d0 con 0x7fe4c8101ce0 2024-04-02T04:45:24.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.293+0000 7fe4cd77c700 1 -- 172.21.15.16:0/3471282884 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4c819bbd0 con 0x7fe4c8101ce0 2024-04-02T04:45:24.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.293+0000 7fe4bffff700 1 -- 172.21.15.16:0/3471282884 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe4b0010430 con 0x7fe4c8101ce0 2024-04-02T04:45:24.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.293+0000 7fe4cd77c700 1 -- 172.21.15.16:0/3471282884 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4c819c1a0 con 0x7fe4c8101ce0 2024-04-02T04:45:24.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.293+0000 7fe4bffff700 1 -- 172.21.15.16:0/3471282884 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4b0018630 con 0x7fe4c8101ce0 2024-04-02T04:45:24.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.295+0000 7fe4cd77c700 1 -- 172.21.15.16:0/3471282884 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4a8005320 con 0x7fe4c8101ce0 2024-04-02T04:45:24.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.295+0000 7fe4bffff700 1 -- 172.21.15.16:0/3471282884 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 37) v1 ==== 91963+0+0 (secure 0 0 0) 0x7fe4b00188f0 con 0x7fe4c8101ce0 2024-04-02T04:45:24.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.295+0000 7fe4bffff700 1 --2- 172.21.15.16:0/3471282884 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fe4b406dea0 0x7fe4b4070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:24.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.296+0000 7fe4bffff700 1 -- 172.21.15.16:0/3471282884 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fe4b0014070 con 0x7fe4c8101ce0 2024-04-02T04:45:24.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.296+0000 7fe4c6ffd700 1 --2- 172.21.15.16:0/3471282884 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fe4b406dea0 0x7fe4b4070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:24.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.296+0000 7fe4c6ffd700 1 --2- 172.21.15.16:0/3471282884 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fe4b406dea0 0x7fe4b4070360 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8009fd0 tx=0x7fe4b8006000 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:24.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.300+0000 7fe4bffff700 1 -- 172.21.15.16:0/3471282884 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe4b0053280 con 0x7fe4c8101ce0 2024-04-02T04:45:24.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.593+0000 7fe4cd77c700 1 -- 172.21.15.16:0/3471282884 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe4a80058a0 con 0x7fe4c8101ce0 2024-04-02T04:45:24.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.593+0000 7fe4bffff700 1 -- 172.21.15.16:0/3471282884 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+733 (secure 0 0 0) 0x7fe4b005ac00 con 0x7fe4c8101ce0 2024-04-02T04:45:24.597 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:45:24.597 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": { 2024-04-02T04:45:24.597 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:45:24.597 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:45:24.597 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": { 2024-04-02T04:45:24.597 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:45:24.597 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:45:24.597 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": { 2024-04-02T04:45:24.597 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 6 2024-04-02T04:45:24.597 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:45:24.597 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-04-02T04:45:24.598 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 4 2024-04-02T04:45:24.598 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:45:24.598 INFO:teuthology.orchestra.run.smithi016.stdout: "overall": { 2024-04-02T04:45:24.598 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 10, 2024-04-02T04:45:24.598 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 4 2024-04-02T04:45:24.598 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-02T04:45:24.598 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:45:24.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.595+0000 7fe4cd77c700 1 -- 172.21.15.16:0/3471282884 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fe4b406dea0 msgr2=0x7fe4b4070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:24.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.595+0000 7fe4cd77c700 1 --2- 172.21.15.16:0/3471282884 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fe4b406dea0 0x7fe4b4070360 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8009fd0 tx=0x7fe4b8006000 comp rx=0 tx=0).stop 2024-04-02T04:45:24.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.595+0000 7fe4cd77c700 1 -- 172.21.15.16:0/3471282884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4c8101ce0 msgr2=0x7fe4c8196a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:24.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.595+0000 7fe4cd77c700 1 --2- 172.21.15.16:0/3471282884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4c8101ce0 0x7fe4c8196a10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe4b000e920 tx=0x7fe4b000ec30 comp rx=0 tx=0).stop 2024-04-02T04:45:24.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.596+0000 7fe4cd77c700 1 -- 172.21.15.16:0/3471282884 shutdown_connections 2024-04-02T04:45:24.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.596+0000 7fe4cd77c700 1 --2- 172.21.15.16:0/3471282884 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fe4b406dea0 0x7fe4b4070360 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:24.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.596+0000 7fe4cd77c700 1 --2- 172.21.15.16:0/3471282884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4c8100ae0 0x7fe4c81964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:24.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.596+0000 7fe4cd77c700 1 --2- 172.21.15.16:0/3471282884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4c8101ce0 0x7fe4c8196a10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:24.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.596+0000 7fe4cd77c700 1 -- 172.21.15.16:0/3471282884 >> 172.21.15.16:0/3471282884 conn(0x7fe4c80fc0a0 msgr2=0x7fe4c81052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:24.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.596+0000 7fe4cd77c700 1 -- 172.21.15.16:0/3471282884 shutdown_connections 2024-04-02T04:45:24.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.596+0000 7fe4cd77c700 1 -- 172.21.15.16:0/3471282884 wait complete. 2024-04-02T04:45:24.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.700+0000 7fd26a6c6700 1 -- 172.21.15.16:0/3970046406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2640ff9e0 msgr2=0x7fd2640ffe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:24.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.700+0000 7fd26a6c6700 1 --2- 172.21.15.16:0/3970046406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2640ff9e0 0x7fd2640ffe00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd254009910 tx=0x7fd254009c20 comp rx=0 tx=0).stop 2024-04-02T04:45:24.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.700+0000 7fd26a6c6700 1 -- 172.21.15.16:0/3970046406 shutdown_connections 2024-04-02T04:45:24.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.700+0000 7fd26a6c6700 1 --2- 172.21.15.16:0/3970046406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd264100be0 0x7fd264101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:24.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.700+0000 7fd26a6c6700 1 --2- 172.21.15.16:0/3970046406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2640ff9e0 0x7fd2640ffe00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:24.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.700+0000 7fd26a6c6700 1 -- 172.21.15.16:0/3970046406 >> 172.21.15.16:0/3970046406 conn(0x7fd2640faf80 msgr2=0x7fd2640fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:24.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.700+0000 7fd26a6c6700 1 -- 172.21.15.16:0/3970046406 shutdown_connections 2024-04-02T04:45:24.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.700+0000 7fd26a6c6700 1 -- 172.21.15.16:0/3970046406 wait complete. 2024-04-02T04:45:24.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.701+0000 7fd26a6c6700 1 Processor -- start 2024-04-02T04:45:24.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.701+0000 7fd26a6c6700 1 -- start start 2024-04-02T04:45:24.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.701+0000 7fd26a6c6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2640ff9e0 0x7fd264103d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:24.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.701+0000 7fd26a6c6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd264100be0 0x7fd2641023e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:24.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.701+0000 7fd26a6c6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd264102920 con 0x7fd264100be0 2024-04-02T04:45:24.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.701+0000 7fd26a6c6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd264102a60 con 0x7fd2640ff9e0 2024-04-02T04:45:24.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.701+0000 7fd2637fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd264100be0 0x7fd2641023e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:24.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.701+0000 7fd2637fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd264100be0 0x7fd2641023e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44686/0 (socket says 172.21.15.16:44686) 2024-04-02T04:45:24.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.701+0000 7fd2637fe700 1 -- 172.21.15.16:0/3056309798 learned_addr learned my addr 172.21.15.16:0/3056309798 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:45:24.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.701+0000 7fd263fff700 1 --2- 172.21.15.16:0/3056309798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2640ff9e0 0x7fd264103d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:24.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.702+0000 7fd2637fe700 1 -- 172.21.15.16:0/3056309798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2640ff9e0 msgr2=0x7fd264103d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:24.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.702+0000 7fd2637fe700 1 --2- 172.21.15.16:0/3056309798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2640ff9e0 0x7fd264103d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:24.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.702+0000 7fd2637fe700 1 -- 172.21.15.16:0/3056309798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2540095f0 con 0x7fd264100be0 2024-04-02T04:45:24.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.702+0000 7fd2637fe700 1 --2- 172.21.15.16:0/3056309798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd264100be0 0x7fd2641023e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd24c00e920 tx=0x7fd24c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:24.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.702+0000 7fd2617fa700 1 -- 172.21.15.16:0/3056309798 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd24c00c9d0 con 0x7fd264100be0 2024-04-02T04:45:24.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.702+0000 7fd2617fa700 1 -- 172.21.15.16:0/3056309798 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd24c010430 con 0x7fd264100be0 2024-04-02T04:45:24.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.702+0000 7fd26a6c6700 1 -- 172.21.15.16:0/3056309798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd264102d40 con 0x7fd264100be0 2024-04-02T04:45:24.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.702+0000 7fd2617fa700 1 -- 172.21.15.16:0/3056309798 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd24c018630 con 0x7fd264100be0 2024-04-02T04:45:24.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.702+0000 7fd26a6c6700 1 -- 172.21.15.16:0/3056309798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd26419e520 con 0x7fd264100be0 2024-04-02T04:45:24.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.703+0000 7fd26a6c6700 1 -- 172.21.15.16:0/3056309798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd26404ea80 con 0x7fd264100be0 2024-04-02T04:45:24.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.704+0000 7fd2617fa700 1 -- 172.21.15.16:0/3056309798 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 37) v1 ==== 91963+0+0 (secure 0 0 0) 0x7fd24c0105a0 con 0x7fd264100be0 2024-04-02T04:45:24.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.704+0000 7fd2617fa700 1 --2- 172.21.15.16:0/3056309798 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd25006dea0 0x7fd250070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:24.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.704+0000 7fd2617fa700 1 -- 172.21.15.16:0/3056309798 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fd24c014070 con 0x7fd264100be0 2024-04-02T04:45:24.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.704+0000 7fd263fff700 1 --2- 172.21.15.16:0/3056309798 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd25006dea0 0x7fd250070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:24.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.704+0000 7fd263fff700 1 --2- 172.21.15.16:0/3056309798 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd25006dea0 0x7fd250070360 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd25400b800 tx=0x7fd254009f70 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:24.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.709+0000 7fd2617fa700 1 -- 172.21.15.16:0/3056309798 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd24c053280 con 0x7fd264100be0 2024-04-02T04:45:24.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.956+0000 7fd26a6c6700 1 -- 172.21.15.16:0/3056309798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd264108540 con 0x7fd264100be0 2024-04-02T04:45:24.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.957+0000 7fd2617fa700 1 -- 172.21.15.16:0/3056309798 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1875 (secure 0 0 0) 0x7fd24c05aee0 con 0x7fd264100be0 2024-04-02T04:45:24.960 INFO:teuthology.orchestra.run.smithi016.stdout:e15 2024-04-02T04:45:24.960 INFO:teuthology.orchestra.run.smithi016.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:45:24.960 INFO:teuthology.orchestra.run.smithi016.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:45:24.961 INFO:teuthology.orchestra.run.smithi016.stdout:legacy client fscid: 1 2024-04-02T04:45:24.961 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:45:24.961 INFO:teuthology.orchestra.run.smithi016.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:45:24.961 INFO:teuthology.orchestra.run.smithi016.stdout:fs_name cephfs 2024-04-02T04:45:24.961 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 15 2024-04-02T04:45:24.961 INFO:teuthology.orchestra.run.smithi016.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:45:24.961 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-04-02T04:39:30.740168+0000 2024-04-02T04:45:24.961 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-04-02T04:39:49.875319+0000 2024-04-02T04:45:24.961 INFO:teuthology.orchestra.run.smithi016.stdout:tableserver 0 2024-04-02T04:45:24.961 INFO:teuthology.orchestra.run.smithi016.stdout:root 0 2024-04-02T04:45:24.961 INFO:teuthology.orchestra.run.smithi016.stdout:session_timeout 60 2024-04-02T04:45:24.961 INFO:teuthology.orchestra.run.smithi016.stdout:session_autoclose 300 2024-04-02T04:45:24.961 INFO:teuthology.orchestra.run.smithi016.stdout:max_file_size 1099511627776 2024-04-02T04:45:24.961 INFO:teuthology.orchestra.run.smithi016.stdout:max_xattr_size 65536 2024-04-02T04:45:24.962 INFO:teuthology.orchestra.run.smithi016.stdout:required_client_features {} 2024-04-02T04:45:24.962 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure 0 2024-04-02T04:45:24.962 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure_osd_epoch 46 2024-04-02T04:45:24.962 INFO:teuthology.orchestra.run.smithi016.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:45:24.962 INFO:teuthology.orchestra.run.smithi016.stdout:max_mds 1 2024-04-02T04:45:24.962 INFO:teuthology.orchestra.run.smithi016.stdout:in 0 2024-04-02T04:45:24.962 INFO:teuthology.orchestra.run.smithi016.stdout:up {0=24285} 2024-04-02T04:45:24.962 INFO:teuthology.orchestra.run.smithi016.stdout:failed 2024-04-02T04:45:24.962 INFO:teuthology.orchestra.run.smithi016.stdout:damaged 2024-04-02T04:45:24.962 INFO:teuthology.orchestra.run.smithi016.stdout:stopped 2024-04-02T04:45:24.962 INFO:teuthology.orchestra.run.smithi016.stdout:data_pools [3] 2024-04-02T04:45:24.962 INFO:teuthology.orchestra.run.smithi016.stdout:metadata_pool 2 2024-04-02T04:45:24.963 INFO:teuthology.orchestra.run.smithi016.stdout:inline_data enabled 2024-04-02T04:45:24.963 INFO:teuthology.orchestra.run.smithi016.stdout:balancer 2024-04-02T04:45:24.963 INFO:teuthology.orchestra.run.smithi016.stdout:bal_rank_mask -1 2024-04-02T04:45:24.963 INFO:teuthology.orchestra.run.smithi016.stdout:standby_count_wanted 1 2024-04-02T04:45:24.963 INFO:teuthology.orchestra.run.smithi016.stdout:qdb_cluster leader: 0 members: 2024-04-02T04:45:24.963 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.zwrfoo{0:24285} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.27:6824/1461455976,v1:172.21.15.27:6825/1461455976] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:45:24.963 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:45:24.963 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:45:24.963 INFO:teuthology.orchestra.run.smithi016.stdout:Standby daemons: 2024-04-02T04:45:24.963 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:45:24.963 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.fyzrnm{-1:14488} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.16:6828/2578719859,v1:172.21.15.16:6829/2578719859] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:45:24.963 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.uhgkgg{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.16:6826/601263579,v1:172.21.15.16:6827/601263579] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:45:24.963 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.sdublg{-1:24295} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.27:6826/4177192740,v1:172.21.15.27:6827/4177192740] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:45:24.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.960+0000 7fd26a6c6700 1 -- 172.21.15.16:0/3056309798 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd25006dea0 msgr2=0x7fd250070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:24.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.960+0000 7fd26a6c6700 1 --2- 172.21.15.16:0/3056309798 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd25006dea0 0x7fd250070360 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd25400b800 tx=0x7fd254009f70 comp rx=0 tx=0).stop 2024-04-02T04:45:24.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.960+0000 7fd26a6c6700 1 -- 172.21.15.16:0/3056309798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd264100be0 msgr2=0x7fd2641023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:24.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.960+0000 7fd26a6c6700 1 --2- 172.21.15.16:0/3056309798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd264100be0 0x7fd2641023e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd24c00e920 tx=0x7fd24c00ec30 comp rx=0 tx=0).stop 2024-04-02T04:45:24.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.960+0000 7fd26a6c6700 1 -- 172.21.15.16:0/3056309798 shutdown_connections 2024-04-02T04:45:24.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.960+0000 7fd26a6c6700 1 --2- 172.21.15.16:0/3056309798 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd25006dea0 0x7fd250070360 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:24.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.960+0000 7fd26a6c6700 1 --2- 172.21.15.16:0/3056309798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2640ff9e0 0x7fd264103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:24.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.960+0000 7fd26a6c6700 1 --2- 172.21.15.16:0/3056309798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd264100be0 0x7fd2641023e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:24.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.960+0000 7fd26a6c6700 1 -- 172.21.15.16:0/3056309798 >> 172.21.15.16:0/3056309798 conn(0x7fd2640faf80 msgr2=0x7fd264068ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:24.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.960+0000 7fd26a6c6700 1 -- 172.21.15.16:0/3056309798 shutdown_connections 2024-04-02T04:45:24.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:24.960+0000 7fd26a6c6700 1 -- 172.21.15.16:0/3056309798 wait complete. 2024-04-02T04:45:24.965 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 15 2024-04-02T04:45:25.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.068+0000 7f7866581700 1 -- 172.21.15.16:0/1823759392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78600ff9d0 msgr2=0x7f78600ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:25.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.068+0000 7f7866581700 1 --2- 172.21.15.16:0/1823759392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78600ff9d0 0x7f78600ffdf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7850009910 tx=0x7f7850009c20 comp rx=0 tx=0).stop 2024-04-02T04:45:25.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.068+0000 7f7866581700 1 -- 172.21.15.16:0/1823759392 shutdown_connections 2024-04-02T04:45:25.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.068+0000 7f7866581700 1 --2- 172.21.15.16:0/1823759392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7860100bd0 0x7f7860101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:25.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.068+0000 7f7866581700 1 --2- 172.21.15.16:0/1823759392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78600ff9d0 0x7f78600ffdf0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:25.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.068+0000 7f7866581700 1 -- 172.21.15.16:0/1823759392 >> 172.21.15.16:0/1823759392 conn(0x7f78600faf70 msgr2=0x7f78600fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:25.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.068+0000 7f7866581700 1 -- 172.21.15.16:0/1823759392 shutdown_connections 2024-04-02T04:45:25.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.068+0000 7f7866581700 1 -- 172.21.15.16:0/1823759392 wait complete. 2024-04-02T04:45:25.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.068+0000 7f7866581700 1 Processor -- start 2024-04-02T04:45:25.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.068+0000 7f7866581700 1 -- start start 2024-04-02T04:45:25.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.069+0000 7f7866581700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78600ff9d0 0x7f7860198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:25.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.069+0000 7f7866581700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7860100bd0 0x7f7860198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:25.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.069+0000 7f7866581700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7860199200 con 0x7f7860100bd0 2024-04-02T04:45:25.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.069+0000 7f7866581700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7860199340 con 0x7f78600ff9d0 2024-04-02T04:45:25.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.069+0000 7f785f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7860100bd0 0x7f7860198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:25.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.069+0000 7f785f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7860100bd0 0x7f7860198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44698/0 (socket says 172.21.15.16:44698) 2024-04-02T04:45:25.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.069+0000 7f785f7fe700 1 -- 172.21.15.16:0/1123285799 learned_addr learned my addr 172.21.15.16:0/1123285799 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:45:25.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.069+0000 7f785ffff700 1 --2- 172.21.15.16:0/1123285799 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78600ff9d0 0x7f7860198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:25.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.069+0000 7f785f7fe700 1 -- 172.21.15.16:0/1123285799 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78600ff9d0 msgr2=0x7f7860198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:25.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.069+0000 7f785f7fe700 1 --2- 172.21.15.16:0/1123285799 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78600ff9d0 0x7f7860198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:25.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.069+0000 7f785f7fe700 1 -- 172.21.15.16:0/1123285799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78500095f0 con 0x7f7860100bd0 2024-04-02T04:45:25.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.069+0000 7f785f7fe700 1 --2- 172.21.15.16:0/1123285799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7860100bd0 0x7f7860198c70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f784800c930 tx=0x7f784800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:25.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.070+0000 7f785d7fa700 1 -- 172.21.15.16:0/1123285799 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7848007ad0 con 0x7f7860100bd0 2024-04-02T04:45:25.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.070+0000 7f785d7fa700 1 -- 172.21.15.16:0/1123285799 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f784800ce80 con 0x7f7860100bd0 2024-04-02T04:45:25.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.070+0000 7f7866581700 1 -- 172.21.15.16:0/1123285799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f786019ddb0 con 0x7f7860100bd0 2024-04-02T04:45:25.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.070+0000 7f785d7fa700 1 -- 172.21.15.16:0/1123285799 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7848018630 con 0x7f7860100bd0 2024-04-02T04:45:25.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.070+0000 7f7866581700 1 -- 172.21.15.16:0/1123285799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f786019e350 con 0x7f7860100bd0 2024-04-02T04:45:25.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.071+0000 7f7866581700 1 -- 172.21.15.16:0/1123285799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f786004ea80 con 0x7f7860100bd0 2024-04-02T04:45:25.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.071+0000 7f785d7fa700 1 -- 172.21.15.16:0/1123285799 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 37) v1 ==== 91963+0+0 (secure 0 0 0) 0x7f7848007c30 con 0x7f7860100bd0 2024-04-02T04:45:25.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.071+0000 7f785d7fa700 1 --2- 172.21.15.16:0/1123285799 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f784c06de90 0x7f784c070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:25.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.072+0000 7f785d7fa700 1 -- 172.21.15.16:0/1123285799 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f784808ddc0 con 0x7f7860100bd0 2024-04-02T04:45:25.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.072+0000 7f785ffff700 1 --2- 172.21.15.16:0/1123285799 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f784c06de90 0x7f784c070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:25.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.072+0000 7f785ffff700 1 --2- 172.21.15.16:0/1123285799 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f784c06de90 0x7f784c070350 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7850005090 tx=0x7f7850005180 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:25.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.076+0000 7f785d7fa700 1 -- 172.21.15.16:0/1123285799 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f784805ac60 con 0x7f7860100bd0 2024-04-02T04:45:25.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.299+0000 7f7866581700 1 -- 172.21.15.16:0/1123285799 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7860068c00 con 0x7f784c06de90 2024-04-02T04:45:25.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.300+0000 7f785d7fa700 1 -- 172.21.15.16:0/1123285799 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f7860068c00 con 0x7f784c06de90 2024-04-02T04:45:25.303 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:45:25.303 INFO:teuthology.orchestra.run.smithi016.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:45:25.304 INFO:teuthology.orchestra.run.smithi016.stdout: "in_progress": true, 2024-04-02T04:45:25.304 INFO:teuthology.orchestra.run.smithi016.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-02T04:45:25.304 INFO:teuthology.orchestra.run.smithi016.stdout: "services_complete": [ 2024-04-02T04:45:25.304 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr", 2024-04-02T04:45:25.304 INFO:teuthology.orchestra.run.smithi016.stdout: "mon" 2024-04-02T04:45:25.304 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-04-02T04:45:25.304 INFO:teuthology.orchestra.run.smithi016.stdout: "progress": "4/23 daemons upgraded", 2024-04-02T04:45:25.304 INFO:teuthology.orchestra.run.smithi016.stdout: "message": "Currently upgrading mon daemons", 2024-04-02T04:45:25.304 INFO:teuthology.orchestra.run.smithi016.stdout: "is_paused": false 2024-04-02T04:45:25.304 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:45:25.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.302+0000 7f7866581700 1 -- 172.21.15.16:0/1123285799 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f784c06de90 msgr2=0x7f784c070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:25.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.302+0000 7f7866581700 1 --2- 172.21.15.16:0/1123285799 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f784c06de90 0x7f784c070350 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7850005090 tx=0x7f7850005180 comp rx=0 tx=0).stop 2024-04-02T04:45:25.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.302+0000 7f7866581700 1 -- 172.21.15.16:0/1123285799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7860100bd0 msgr2=0x7f7860198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:25.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.302+0000 7f7866581700 1 --2- 172.21.15.16:0/1123285799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7860100bd0 0x7f7860198c70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f784800c930 tx=0x7f784800cc40 comp rx=0 tx=0).stop 2024-04-02T04:45:25.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.302+0000 7f7866581700 1 -- 172.21.15.16:0/1123285799 shutdown_connections 2024-04-02T04:45:25.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.302+0000 7f7866581700 1 --2- 172.21.15.16:0/1123285799 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f784c06de90 0x7f784c070350 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.302+0000 7f7866581700 1 --2- 172.21.15.16:0/1123285799 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78600ff9d0 0x7f7860198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.302+0000 7f7866581700 1 --2- 172.21.15.16:0/1123285799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7860100bd0 0x7f7860198c70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.302+0000 7f7866581700 1 -- 172.21.15.16:0/1123285799 >> 172.21.15.16:0/1123285799 conn(0x7f78600faf70 msgr2=0x7f78600684f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.302+0000 7f7866581700 1 -- 172.21.15.16:0/1123285799 shutdown_connections 2024-04-02T04:45:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.302+0000 7f7866581700 1 -- 172.21.15.16:0/1123285799 wait complete. 2024-04-02T04:45:25.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.410+0000 7ff6dbe28700 1 -- 172.21.15.16:0/2273630591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6d4101d80 msgr2=0x7ff6d41021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:25.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.410+0000 7ff6dbe28700 1 --2- 172.21.15.16:0/2273630591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6d4101d80 0x7ff6d41021e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8009910 tx=0x7ff6c8009c20 comp rx=0 tx=0).stop 2024-04-02T04:45:25.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.410+0000 7ff6dbe28700 1 -- 172.21.15.16:0/2273630591 shutdown_connections 2024-04-02T04:45:25.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.410+0000 7ff6dbe28700 1 --2- 172.21.15.16:0/2273630591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6d4101d80 0x7ff6d41021e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:25.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.410+0000 7ff6dbe28700 1 --2- 172.21.15.16:0/2273630591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6d40feda0 0x7ff6d40ff1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:25.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.410+0000 7ff6dbe28700 1 -- 172.21.15.16:0/2273630591 >> 172.21.15.16:0/2273630591 conn(0x7ff6d40fa700 msgr2=0x7ff6d40fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:25.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.410+0000 7ff6dbe28700 1 -- 172.21.15.16:0/2273630591 shutdown_connections 2024-04-02T04:45:25.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.410+0000 7ff6dbe28700 1 -- 172.21.15.16:0/2273630591 wait complete. 2024-04-02T04:45:25.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.411+0000 7ff6dbe28700 1 Processor -- start 2024-04-02T04:45:25.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.411+0000 7ff6dbe28700 1 -- start start 2024-04-02T04:45:25.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.411+0000 7ff6dbe28700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6d40feda0 0x7ff6d4196590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:25.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.411+0000 7ff6dbe28700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6d4101d80 0x7ff6d4196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:25.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.411+0000 7ff6dbe28700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d4197060 con 0x7ff6d40feda0 2024-04-02T04:45:25.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.411+0000 7ff6dbe28700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d41971a0 con 0x7ff6d4101d80 2024-04-02T04:45:25.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.411+0000 7ff6d9bc4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6d40feda0 0x7ff6d4196590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:25.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.411+0000 7ff6d9bc4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6d40feda0 0x7ff6d4196590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44726/0 (socket says 172.21.15.16:44726) 2024-04-02T04:45:25.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.411+0000 7ff6d9bc4700 1 -- 172.21.15.16:0/3621869271 learned_addr learned my addr 172.21.15.16:0/3621869271 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:45:25.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.411+0000 7ff6d93c3700 1 --2- 172.21.15.16:0/3621869271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6d4101d80 0x7ff6d4196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:25.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.411+0000 7ff6d9bc4700 1 -- 172.21.15.16:0/3621869271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6d4101d80 msgr2=0x7ff6d4196ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:25.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.411+0000 7ff6d9bc4700 1 --2- 172.21.15.16:0/3621869271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6d4101d80 0x7ff6d4196ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:25.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.411+0000 7ff6d9bc4700 1 -- 172.21.15.16:0/3621869271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c80095f0 con 0x7ff6d40feda0 2024-04-02T04:45:25.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.412+0000 7ff6d9bc4700 1 --2- 172.21.15.16:0/3621869271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6d40feda0 0x7ff6d4196590 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc00c910 tx=0x7ff6cc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:25.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.412+0000 7ff6d2ffd700 1 -- 172.21.15.16:0/3621869271 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6cc0041f0 con 0x7ff6d40feda0 2024-04-02T04:45:25.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.412+0000 7ff6d2ffd700 1 -- 172.21.15.16:0/3621869271 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff6cc00ce60 con 0x7ff6d40feda0 2024-04-02T04:45:25.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.412+0000 7ff6d2ffd700 1 -- 172.21.15.16:0/3621869271 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6cc003d90 con 0x7ff6d40feda0 2024-04-02T04:45:25.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.412+0000 7ff6dbe28700 1 -- 172.21.15.16:0/3621869271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d419bbc0 con 0x7ff6d40feda0 2024-04-02T04:45:25.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.412+0000 7ff6dbe28700 1 -- 172.21.15.16:0/3621869271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d419c0e0 con 0x7ff6d40feda0 2024-04-02T04:45:25.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.413+0000 7ff6dbe28700 1 -- 172.21.15.16:0/3621869271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6d4190660 con 0x7ff6d40feda0 2024-04-02T04:45:25.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.413+0000 7ff6d2ffd700 1 -- 172.21.15.16:0/3621869271 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 37) v1 ==== 91963+0+0 (secure 0 0 0) 0x7ff6cc007900 con 0x7ff6d40feda0 2024-04-02T04:45:25.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.414+0000 7ff6d2ffd700 1 --2- 172.21.15.16:0/3621869271 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ff6bc06dee0 0x7ff6bc0703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:25.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.414+0000 7ff6d2ffd700 1 -- 172.21.15.16:0/3621869271 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6500+0+0 (secure 0 0 0) 0x7ff6cc08db40 con 0x7ff6d40feda0 2024-04-02T04:45:25.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.414+0000 7ff6d93c3700 1 --2- 172.21.15.16:0/3621869271 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ff6bc06dee0 0x7ff6bc0703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:25.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.414+0000 7ff6d93c3700 1 --2- 172.21.15.16:0/3621869271 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ff6bc06dee0 0x7ff6bc0703a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8005090 tx=0x7ff6c801a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:25.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.418+0000 7ff6d2ffd700 1 -- 172.21.15.16:0/3621869271 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff6cc05a940 con 0x7ff6d40feda0 2024-04-02T04:45:25.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:25 smithi016 ceph-mon[93230]: from='client.44127 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:45:25.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:25 smithi016 ceph-mon[93230]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:45:25.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:25 smithi016 ceph-mon[93230]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:45:25.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:25 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:25.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:25 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:25.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:25 smithi016 ceph-mon[93230]: Reconfiguring mds.cephfs.smithi027.sdublg (monmap changed)... 2024-04-02T04:45:25.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:25 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi027.sdublg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:45:25.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:25 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:25.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:25 smithi016 ceph-mon[93230]: Reconfiguring daemon mds.cephfs.smithi027.sdublg on smithi027 2024-04-02T04:45:25.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:25 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/3471282884' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:45:25.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:25 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/3056309798' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:45:25.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:25 smithi027 ceph-mon[83036]: from='client.44127 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:45:25.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:25 smithi027 ceph-mon[83036]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:45:25.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:25 smithi027 ceph-mon[83036]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:45:25.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:25 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:25.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:25 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:25.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:25 smithi027 ceph-mon[83036]: Reconfiguring mds.cephfs.smithi027.sdublg (monmap changed)... 2024-04-02T04:45:25.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:25 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi027.sdublg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:45:25.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:25 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:25.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:25 smithi027 ceph-mon[83036]: Reconfiguring daemon mds.cephfs.smithi027.sdublg on smithi027 2024-04-02T04:45:25.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:25 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/3471282884' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:45:25.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:25 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/3056309798' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:45:25.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.708+0000 7ff6dbe28700 1 -- 172.21.15.16:0/3621869271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff6d404ea80 con 0x7ff6d40feda0 2024-04-02T04:45:25.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.708+0000 7ff6d2ffd700 1 -- 172.21.15.16:0/3621869271 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7ff6cc05a4d0 con 0x7ff6d40feda0 2024-04-02T04:45:25.711 INFO:teuthology.orchestra.run.smithi016.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-02T04:45:25.711 INFO:teuthology.orchestra.run.smithi016.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-02T04:45:25.711 INFO:teuthology.orchestra.run.smithi016.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-02T04:45:25.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.710+0000 7ff6dbe28700 1 -- 172.21.15.16:0/3621869271 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ff6bc06dee0 msgr2=0x7ff6bc0703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:25.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.710+0000 7ff6dbe28700 1 --2- 172.21.15.16:0/3621869271 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ff6bc06dee0 0x7ff6bc0703a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8005090 tx=0x7ff6c801a040 comp rx=0 tx=0).stop 2024-04-02T04:45:25.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.710+0000 7ff6dbe28700 1 -- 172.21.15.16:0/3621869271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6d40feda0 msgr2=0x7ff6d4196590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:25.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.710+0000 7ff6dbe28700 1 --2- 172.21.15.16:0/3621869271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6d40feda0 0x7ff6d4196590 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc00c910 tx=0x7ff6cc00cc20 comp rx=0 tx=0).stop 2024-04-02T04:45:25.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.710+0000 7ff6dbe28700 1 -- 172.21.15.16:0/3621869271 shutdown_connections 2024-04-02T04:45:25.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.710+0000 7ff6dbe28700 1 --2- 172.21.15.16:0/3621869271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6d40feda0 0x7ff6d4196590 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:25.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.710+0000 7ff6dbe28700 1 --2- 172.21.15.16:0/3621869271 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ff6bc06dee0 0x7ff6bc0703a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:25.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.710+0000 7ff6dbe28700 1 --2- 172.21.15.16:0/3621869271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6d4101d80 0x7ff6d4196ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:25.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.710+0000 7ff6dbe28700 1 -- 172.21.15.16:0/3621869271 >> 172.21.15.16:0/3621869271 conn(0x7ff6d40fa700 msgr2=0x7ff6d41053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:25.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.710+0000 7ff6dbe28700 1 -- 172.21.15.16:0/3621869271 shutdown_connections 2024-04-02T04:45:25.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:25.710+0000 7ff6dbe28700 1 -- 172.21.15.16:0/3621869271 wait complete. 2024-04-02T04:45:26.542 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:26 smithi016 ceph-mon[93230]: pgmap v37: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:45:26.542 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:26 smithi016 ceph-mon[93230]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:45:26.542 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:26 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/3621869271' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:45:26.542 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:26 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:26.543 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:26 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:26.543 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:26 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:45:26.543 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:26 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:45:26.543 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:26 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:45:26.543 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:26 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:26.543 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:26 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi016"}]: dispatch 2024-04-02T04:45:26.543 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:26 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi016"}]': finished 2024-04-02T04:45:26.543 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:26 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi027"}]: dispatch 2024-04-02T04:45:26.543 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:26 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi027"}]': finished 2024-04-02T04:45:26.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:26 smithi027 ceph-mon[83036]: pgmap v37: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:45:26.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:26 smithi027 ceph-mon[83036]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:45:26.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:26 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/3621869271' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:45:26.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:26 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:26.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:26 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:26.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:26 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:45:26.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:26 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:45:26.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:26 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:45:26.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:26 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:26.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:26 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi016"}]: dispatch 2024-04-02T04:45:26.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:26 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi016"}]': finished 2024-04-02T04:45:26.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:26 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi027"}]: dispatch 2024-04-02T04:45:26.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:26 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi027"}]': finished 2024-04-02T04:45:27.490 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:27 smithi016 ceph-mon[93230]: Upgrade: Setting container_image for all mon 2024-04-02T04:45:27.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:27 smithi027 ceph-mon[83036]: Upgrade: Setting container_image for all mon 2024-04-02T04:45:28.427 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:28 smithi016 ceph-mon[93230]: pgmap v38: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:45:28.427 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:28 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:28.427 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:28 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi016", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-02T04:45:28.428 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:28 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:28.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:28 smithi027 ceph-mon[83036]: pgmap v38: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:45:28.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:28 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:28.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:28 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi016", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-02T04:45:28.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:28 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:29.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:29 smithi016 ceph-mon[93230]: Upgrade: Updating crash.smithi016 (1/2) 2024-04-02T04:45:29.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:29 smithi016 ceph-mon[93230]: Deploying daemon crash.smithi016 on smithi016 2024-04-02T04:45:29.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:29 smithi027 ceph-mon[83036]: Upgrade: Updating crash.smithi016 (1/2) 2024-04-02T04:45:29.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:29 smithi027 ceph-mon[83036]: Deploying daemon crash.smithi016 on smithi016 2024-04-02T04:45:30.363 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:30 smithi016 ceph-mon[93230]: pgmap v39: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:45:30.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:30 smithi027 ceph-mon[83036]: pgmap v39: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:45:32.356 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:32 smithi016 ceph-mon[93230]: pgmap v40: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:45:32.356 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:32 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:32.356 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:32 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:32.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:32 smithi027 ceph-mon[83036]: pgmap v40: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:45:32.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:32 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:32.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:32 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:33.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:33 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:45:33.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:33 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:33.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:33 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi027", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-02T04:45:33.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:33 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:33.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:33 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:45:33.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:33 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:33.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:33 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi027", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-02T04:45:33.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:33 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:34.299 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:34 smithi027 ceph-mon[83036]: Upgrade: Updating crash.smithi027 (2/2) 2024-04-02T04:45:34.300 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:34 smithi027 ceph-mon[83036]: Deploying daemon crash.smithi027 on smithi027 2024-04-02T04:45:34.300 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:34 smithi027 ceph-mon[83036]: pgmap v41: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:45:34.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:34 smithi016 ceph-mon[93230]: Upgrade: Updating crash.smithi027 (2/2) 2024-04-02T04:45:34.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:34 smithi016 ceph-mon[93230]: Deploying daemon crash.smithi027 on smithi027 2024-04-02T04:45:34.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:34 smithi016 ceph-mon[93230]: pgmap v41: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:45:36.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:36 smithi027 ceph-mon[83036]: pgmap v42: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:45:36.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:36 smithi016 ceph-mon[93230]: pgmap v42: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:45:37.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:37 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:37.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:37 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:37.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:37 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:45:37.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:37 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:37.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:37 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:37.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:37 smithi027 ceph-mon[83036]: pgmap v43: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:45:37.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:37 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:37.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:37 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:37.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:37 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:45:37.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:37 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:37.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:37 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:37.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:37 smithi016 ceph-mon[93230]: pgmap v43: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:45:40.356 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:40 smithi016 ceph-mon[93230]: pgmap v44: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:45:40.356 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:40 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:40.356 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:40 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:40.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:40 smithi027 ceph-mon[83036]: pgmap v44: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:45:40.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:40 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:40.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:40 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:42.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:42 smithi027 ceph-mon[83036]: pgmap v45: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:45:42.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:42 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:42.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:42 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:42.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:42 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:42.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:42 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:42.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:42 smithi016 ceph-mon[93230]: pgmap v45: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:45:42.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:42 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:42.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:42 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:42.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:42 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:42.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:42 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:43.809 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:43 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:43.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:43 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:43.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:43 smithi016 ceph-mon[93230]: pgmap v46: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:45:43.951 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:43 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:43.951 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:43 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:43.951 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:43 smithi027 ceph-mon[83036]: pgmap v46: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:45:45.777 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:45 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:45.777 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:45 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:45.778 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:45 smithi016 ceph-mon[93230]: pgmap v47: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:45:46.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:45 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:46.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:45 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:46.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:45 smithi027 ceph-mon[83036]: pgmap v47: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:45:47.465 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:47 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:47.466 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:47 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:47.466 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:47 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:47.466 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:47 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:45:47.466 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:47 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:47.466 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:47 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:45:47.466 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:47 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:45:47.466 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:47 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:45:47.467 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:47 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:45:47.467 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:47 smithi016 ceph-mon[93230]: Upgrade: Setting container_image for all crash 2024-04-02T04:45:47.467 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:47 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:47.467 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:47 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi016"}]: dispatch 2024-04-02T04:45:47.467 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:47 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi016"}]': finished 2024-04-02T04:45:47.467 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:47 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi027"}]: dispatch 2024-04-02T04:45:47.467 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:47 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi027"}]': finished 2024-04-02T04:45:47.467 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:47 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-02T04:45:47.468 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:47 smithi016 ceph-mon[93230]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-02T04:45:47.468 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:47 smithi016 ceph-mon[93230]: Upgrade: osd.0 is safe to restart 2024-04-02T04:45:47.468 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:47 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:45:47.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:47 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:47.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:47 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:47.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:47 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:47.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:47 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:45:47.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:47 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:47.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:47 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:45:47.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:47 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:45:47.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:47 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:45:47.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:47 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:45:47.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:47 smithi027 ceph-mon[83036]: Upgrade: Setting container_image for all crash 2024-04-02T04:45:47.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:47 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:47.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:47 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi016"}]: dispatch 2024-04-02T04:45:47.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:47 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi016"}]': finished 2024-04-02T04:45:47.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:47 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi027"}]: dispatch 2024-04-02T04:45:47.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:47 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi027"}]': finished 2024-04-02T04:45:47.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:47 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-02T04:45:47.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:47 smithi027 ceph-mon[83036]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-02T04:45:47.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:47 smithi027 ceph-mon[83036]: Upgrade: osd.0 is safe to restart 2024-04-02T04:45:47.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:47 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:45:48.528 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:48 smithi016 ceph-mon[93230]: pgmap v48: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:45:48.528 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:48 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:48.529 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:48 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-02T04:45:48.529 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:48 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:48.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:48 smithi027 ceph-mon[83036]: pgmap v48: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:45:48.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:48 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:48.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:48 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-02T04:45:48.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:48 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:45:49.625 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:49 smithi016 ceph-mon[93230]: Upgrade: Updating osd.0 2024-04-02T04:45:49.625 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:49 smithi016 ceph-mon[93230]: Deploying daemon osd.0 on smithi016 2024-04-02T04:45:49.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:49 smithi027 ceph-mon[83036]: Upgrade: Updating osd.0 2024-04-02T04:45:49.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:49 smithi027 ceph-mon[83036]: Deploying daemon osd.0 on smithi016 2024-04-02T04:45:50.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:50 smithi016 ceph-mon[93230]: pgmap v49: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-02T04:45:50.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:50 smithi016 ceph-mon[93230]: mgrmap e38: smithi016.eqnebx(active, since 92s), standbys: smithi027.kjzcsc 2024-04-02T04:45:50.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:50 smithi027 ceph-mon[83036]: pgmap v49: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-02T04:45:50.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:50 smithi027 ceph-mon[83036]: mgrmap e38: smithi016.eqnebx(active, since 92s), standbys: smithi027.kjzcsc 2024-04-02T04:45:51.222 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:50 smithi016 systemd[1]: Stopping Ceph osd.0 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4... 2024-04-02T04:45:51.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:51 smithi016 ceph-mon[93230]: osd.0 marked itself down and dead 2024-04-02T04:45:51.560 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:51 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0[47723]: 2024-04-02T04:45:51.219+0000 7f5b46c29700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T04:45:51.560 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:51 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0[47723]: 2024-04-02T04:45:51.219+0000 7f5b46c29700 -1 osd.0 49 *** Got signal Terminated *** 2024-04-02T04:45:51.560 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:51 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0[47723]: 2024-04-02T04:45:51.219+0000 7f5b46c29700 -1 osd.0 49 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T04:45:51.639 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:51 smithi027 ceph-mon[83036]: osd.0 marked itself down and dead 2024-04-02T04:45:52.031 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:51 smithi016 podman[101740]: 2024-04-02 04:45:51.771306516 +0000 UTC m=+0.666956867 container died 3f40108debd61e1164adc04db1e2071e12fddddaebd3e2fb6d8db6ff75a640b0 (image=quay.ceph.io/ceph-ci/ceph@sha256:397d0c1d908c002c558906e3ea05982bc336d403ee4d900400cc3bdd9243aa2e, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=reef-4d05f4c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , ceph=True) 2024-04-02T04:45:52.310 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:52 smithi016 podman[101740]: 2024-04-02 04:45:52.119607376 +0000 UTC m=+1.015257717 container cleanup 3f40108debd61e1164adc04db1e2071e12fddddaebd3e2fb6d8db6ff75a640b0 (image=quay.ceph.io/ceph-ci/ceph@sha256:397d0c1d908c002c558906e3ea05982bc336d403ee4d900400cc3bdd9243aa2e, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=reef-4d05f4c, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-02T04:45:52.310 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:52 smithi016 bash[101740]: ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0 2024-04-02T04:45:52.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:52 smithi027 ceph-mon[83036]: pgmap v50: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-02T04:45:52.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:52 smithi027 ceph-mon[83036]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-02T04:45:52.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:52 smithi027 ceph-mon[83036]: failed to encode map e50 with expected crc 2024-04-02T04:45:52.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:52 smithi027 ceph-mon[83036]: osdmap e50: 6 total, 5 up, 6 in 2024-04-02T04:45:52.741 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:52 smithi016 podman[101757]: 2024-04-02 04:45:52.412729495 +0000 UTC m=+0.638993928 container remove 3f40108debd61e1164adc04db1e2071e12fddddaebd3e2fb6d8db6ff75a640b0 (image=quay.ceph.io/ceph-ci/ceph@sha256:397d0c1d908c002c558906e3ea05982bc336d403ee4d900400cc3bdd9243aa2e, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=reef-4d05f4c, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-02T04:45:52.741 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:52 smithi016 ceph-mon[93230]: pgmap v50: 65 pgs: 65 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-02T04:45:52.741 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:52 smithi016 ceph-mon[93230]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-02T04:45:52.741 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:52 smithi016 ceph-mon[93230]: failed to encode map e50 with expected crc 2024-04-02T04:45:52.741 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:52 smithi016 ceph-mon[93230]: osdmap e50: 6 total, 5 up, 6 in 2024-04-02T04:45:53.043 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:52 smithi016 podman[101848]: 2024-04-02 04:45:52.641199386 +0000 UTC m=+0.018008971 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:45:53.044 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:52 smithi016 podman[101848]: 2024-04-02 04:45:52.755290164 +0000 UTC m=+0.132099737 container create 526907add990337f83c3ee767b7ce5dce563cdfb83ec2b037654b83f5e54281d (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0-deactivate, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-6cec793, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T04:45:53.044 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:52 smithi016 podman[101848]: 2024-04-02 04:45:52.966914439 +0000 UTC m=+0.343724024 container init 526907add990337f83c3ee767b7ce5dce563cdfb83ec2b037654b83f5e54281d (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-6cec793, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-02T04:45:53.044 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:52 smithi016 podman[101848]: 2024-04-02 04:45:52.970397966 +0000 UTC m=+0.347207547 container start 526907add990337f83c3ee767b7ce5dce563cdfb83ec2b037654b83f5e54281d (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-6cec793, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-02T04:45:53.044 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:53 smithi016 podman[101848]: 2024-04-02 04:45:53.041294146 +0000 UTC m=+0.418103731 container attach 526907add990337f83c3ee767b7ce5dce563cdfb83ec2b037654b83f5e54281d (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, RELEASE=squid-6cec793, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-02T04:45:53.310 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:53 smithi016 podman[101848]: 2024-04-02 04:45:53.281334526 +0000 UTC m=+0.658144092 container died 526907add990337f83c3ee767b7ce5dce563cdfb83ec2b037654b83f5e54281d (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-6cec793, org.label-schema.license=GPLv2) 2024-04-02T04:45:53.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:53 smithi027 ceph-mon[83036]: failed to encode map e51 with expected crc 2024-04-02T04:45:53.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:53 smithi027 ceph-mon[83036]: osdmap e51: 6 total, 5 up, 6 in 2024-04-02T04:45:53.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:53 smithi027 ceph-mon[83036]: failed to encode map e51 with expected crc 2024-04-02T04:45:53.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:53 smithi027 ceph-mon[83036]: failed to encode map e51 with expected crc 2024-04-02T04:45:53.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:53 smithi027 ceph-mon[83036]: failed to encode map e51 with expected crc 2024-04-02T04:45:53.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:53 smithi027 ceph-mon[83036]: failed to encode map e51 with expected crc 2024-04-02T04:45:53.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:53 smithi027 ceph-mon[83036]: failed to encode map e51 with expected crc 2024-04-02T04:45:53.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:53 smithi027 ceph-mon[83036]: pgmap v53: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 383 B/s wr, 0 op/s; 51/267 objects degraded (19.101%) 2024-04-02T04:45:53.761 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:53 smithi016 ceph-mon[93230]: failed to encode map e51 with expected crc 2024-04-02T04:45:53.761 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:53 smithi016 ceph-mon[93230]: osdmap e51: 6 total, 5 up, 6 in 2024-04-02T04:45:53.761 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:53 smithi016 ceph-mon[93230]: failed to encode map e51 with expected crc 2024-04-02T04:45:53.761 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:53 smithi016 ceph-mon[93230]: failed to encode map e51 with expected crc 2024-04-02T04:45:53.761 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:53 smithi016 ceph-mon[93230]: failed to encode map e51 with expected crc 2024-04-02T04:45:53.761 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:53 smithi016 ceph-mon[93230]: failed to encode map e51 with expected crc 2024-04-02T04:45:53.761 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:53 smithi016 ceph-mon[93230]: failed to encode map e51 with expected crc 2024-04-02T04:45:53.761 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:53 smithi016 ceph-mon[93230]: pgmap v53: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 383 B/s wr, 0 op/s; 51/267 objects degraded (19.101%) 2024-04-02T04:45:54.060 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:53 smithi016 podman[101874]: 2024-04-02 04:45:53.937851555 +0000 UTC m=+0.648384090 container remove 526907add990337f83c3ee767b7ce5dce563cdfb83ec2b037654b83f5e54281d (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0-deactivate, RELEASE=squid-6cec793, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS) 2024-04-02T04:45:54.060 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:53 smithi016 systemd[1]: ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@osd.0.service: Deactivated successfully. 2024-04-02T04:45:54.060 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:53 smithi016 systemd[1]: Stopped Ceph osd.0 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4. 2024-04-02T04:45:54.060 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:53 smithi016 systemd[1]: ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@osd.0.service: Consumed 50.932s CPU time. 2024-04-02T04:45:54.522 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:54 smithi016 ceph-mon[93230]: Health check failed: Degraded data redundancy: 51/267 objects degraded (19.101%), 19 pgs degraded (PG_DEGRADED) 2024-04-02T04:45:54.523 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:54 smithi016 systemd[1]: Starting Ceph osd.0 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4... 2024-04-02T04:45:54.523 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:54 smithi016 podman[101978]: 2024-04-02 04:45:54.287759174 +0000 UTC m=+0.017859325 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:45:54.523 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:54 smithi016 podman[101978]: 2024-04-02 04:45:54.388858921 +0000 UTC m=+0.118959061 container create e31b0389cd6c9c79be51039736325e9a84918ac6d1d9008a37c484641abc3f2b (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0-activate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-6cec793, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-02T04:45:54.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:54 smithi027 ceph-mon[83036]: Health check failed: Degraded data redundancy: 51/267 objects degraded (19.101%), 19 pgs degraded (PG_DEGRADED) 2024-04-02T04:45:54.810 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:54 smithi016 podman[101978]: 2024-04-02 04:45:54.599659997 +0000 UTC m=+0.329760123 container init e31b0389cd6c9c79be51039736325e9a84918ac6d1d9008a37c484641abc3f2b (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0-activate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-6cec793, GIT_BRANCH=HEAD) 2024-04-02T04:45:54.810 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:54 smithi016 podman[101978]: 2024-04-02 04:45:54.602818636 +0000 UTC m=+0.332918762 container start e31b0389cd6c9c79be51039736325e9a84918ac6d1d9008a37c484641abc3f2b (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0-activate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-6cec793, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True) 2024-04-02T04:45:54.810 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:54 smithi016 podman[101978]: 2024-04-02 04:45:54.67463258 +0000 UTC m=+0.404732721 container attach e31b0389cd6c9c79be51039736325e9a84918ac6d1d9008a37c484641abc3f2b (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0-activate, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-6cec793, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-02T04:45:55.310 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:55 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0-activate[101992]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-02T04:45:55.310 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:55 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0-activate[101992]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-02T04:45:55.310 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:55 smithi016 bash[101978]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-02T04:45:55.310 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:55 smithi016 bash[101978]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-02T04:45:55.310 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:55 smithi016 bash[101978]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-02T04:45:55.310 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:55 smithi016 bash[101978]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-02T04:45:55.310 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:55 smithi016 bash[101978]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-04-02T04:45:55.310 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:55 smithi016 bash[101978]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-02T04:45:55.311 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:55 smithi016 bash[101978]: --> ceph-volume raw activate successful for osd ID: 0 2024-04-02T04:45:55.311 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:55 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0-activate[101992]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-02T04:45:55.311 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:55 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0-activate[101992]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-02T04:45:55.311 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:55 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0-activate[101992]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-04-02T04:45:55.311 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:55 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0-activate[101992]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-02T04:45:55.311 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:55 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0-activate[101992]: --> ceph-volume raw activate successful for osd ID: 0 2024-04-02T04:45:55.311 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:55 smithi016 podman[101978]: 2024-04-02 04:45:55.077892607 +0000 UTC m=+0.807992751 container died e31b0389cd6c9c79be51039736325e9a84918ac6d1d9008a37c484641abc3f2b (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0-activate, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-6cec793, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-02T04:45:55.626 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:55 smithi016 ceph-mon[93230]: pgmap v54: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 383 B/s wr, 0 op/s; 51/267 objects degraded (19.101%) 2024-04-02T04:45:55.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:55 smithi027 ceph-mon[83036]: pgmap v54: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 383 B/s wr, 0 op/s; 51/267 objects degraded (19.101%) 2024-04-02T04:45:55.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:55.826+0000 7f5d18843700 1 -- 172.21.15.16:0/772429662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d14103e90 msgr2=0x7f5d14106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:55.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:55.826+0000 7f5d18843700 1 --2- 172.21.15.16:0/772429662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d14103e90 0x7f5d14106280 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5d08009910 tx=0x7f5d08009c20 comp rx=0 tx=0).stop 2024-04-02T04:45:55.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:55.826+0000 7f5d18843700 1 -- 172.21.15.16:0/772429662 shutdown_connections 2024-04-02T04:45:55.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:55.826+0000 7f5d18843700 1 --2- 172.21.15.16:0/772429662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d14103e90 0x7f5d14106280 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:55.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:55.826+0000 7f5d18843700 1 --2- 172.21.15.16:0/772429662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d14101560 0x7f5d14103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:55.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:55.826+0000 7f5d18843700 1 -- 172.21.15.16:0/772429662 >> 172.21.15.16:0/772429662 conn(0x7f5d140faf70 msgr2=0x7f5d140fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:55.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:55.826+0000 7f5d18843700 1 -- 172.21.15.16:0/772429662 shutdown_connections 2024-04-02T04:45:55.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:55.826+0000 7f5d18843700 1 -- 172.21.15.16:0/772429662 wait complete. 2024-04-02T04:45:55.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:55.827+0000 7f5d18843700 1 Processor -- start 2024-04-02T04:45:55.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:55.827+0000 7f5d18843700 1 -- start start 2024-04-02T04:45:55.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:55.827+0000 7f5d18843700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d14101560 0x7f5d141987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:55.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:55.827+0000 7f5d18843700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d14103e90 0x7f5d14198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:55.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:55.827+0000 7f5d18843700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d14199310 con 0x7f5d14101560 2024-04-02T04:45:55.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:55.827+0000 7f5d18843700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d14199450 con 0x7f5d14103e90 2024-04-02T04:45:55.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:55.827+0000 7f5d1259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d14101560 0x7f5d141987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:55.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:55.827+0000 7f5d1259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d14101560 0x7f5d141987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56882/0 (socket says 172.21.15.16:56882) 2024-04-02T04:45:55.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:55.827+0000 7f5d1259c700 1 -- 172.21.15.16:0/3973884150 learned_addr learned my addr 172.21.15.16:0/3973884150 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:45:55.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:55.828+0000 7f5d11d9b700 1 --2- 172.21.15.16:0/3973884150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d14103e90 0x7f5d14198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:55.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:55.828+0000 7f5d1259c700 1 -- 172.21.15.16:0/3973884150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d14103e90 msgr2=0x7f5d14198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:55.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:55.828+0000 7f5d1259c700 1 --2- 172.21.15.16:0/3973884150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d14103e90 0x7f5d14198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:55.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:55.828+0000 7f5d1259c700 1 -- 172.21.15.16:0/3973884150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d080095f0 con 0x7f5d14101560 2024-04-02T04:45:55.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:55.828+0000 7f5d1259c700 1 --2- 172.21.15.16:0/3973884150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d14101560 0x7f5d141987b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400b4b0 tx=0x7f5d0400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:55.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:55.828+0000 7f5d037fe700 1 -- 172.21.15.16:0/3973884150 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d04011800 con 0x7f5d14101560 2024-04-02T04:45:55.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:55.828+0000 7f5d037fe700 1 -- 172.21.15.16:0/3973884150 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d04011e20 con 0x7f5d14101560 2024-04-02T04:45:55.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:55.828+0000 7f5d18843700 1 -- 172.21.15.16:0/3973884150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d1419deb0 con 0x7f5d14101560 2024-04-02T04:45:55.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:55.828+0000 7f5d037fe700 1 -- 172.21.15.16:0/3973884150 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d0400f2e0 con 0x7f5d14101560 2024-04-02T04:45:55.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:55.828+0000 7f5d18843700 1 -- 172.21.15.16:0/3973884150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d1419e3f0 con 0x7f5d14101560 2024-04-02T04:45:55.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:55.829+0000 7f5d18843700 1 -- 172.21.15.16:0/3973884150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d1404ea80 con 0x7f5d14101560 2024-04-02T04:45:55.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:55.830+0000 7f5d037fe700 1 -- 172.21.15.16:0/3973884150 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f5d04010380 con 0x7f5d14101560 2024-04-02T04:45:55.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:55.830+0000 7f5d037fe700 1 --2- 172.21.15.16:0/3973884150 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5cfc06dd60 0x7f5cfc070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:55.830+0000 7f5d037fe700 1 -- 172.21.15.16:0/3973884150 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f5d0408d720 con 0x7f5d14101560 2024-04-02T04:45:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:55.830+0000 7f5d11d9b700 1 --2- 172.21.15.16:0/3973884150 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5cfc06dd60 0x7f5cfc070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:55.831+0000 7f5d11d9b700 1 --2- 172.21.15.16:0/3973884150 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5cfc06dd60 0x7f5cfc070220 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5d08005090 tx=0x7f5d08005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:55.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:55.835+0000 7f5d037fe700 1 -- 172.21.15.16:0/3973884150 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5d0405a740 con 0x7f5d14101560 2024-04-02T04:45:55.899 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:55 smithi016 podman[102132]: 2024-04-02 04:45:55.653814041 +0000 UTC m=+0.570289927 container remove e31b0389cd6c9c79be51039736325e9a84918ac6d1d9008a37c484641abc3f2b (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0-activate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-6cec793, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-02T04:45:55.899 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:55 smithi016 podman[102202]: 2024-04-02 04:45:55.770653754 +0000 UTC m=+0.020627219 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:45:55.899 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:55 smithi016 podman[102202]: 2024-04-02 04:45:55.896533944 +0000 UTC m=+0.146507392 container create 421d9e1cc751685a0849fcb4c1d0592e6671836c7a718a8bc6731ce954bdaaef (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-6cec793, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-02T04:45:56.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.062+0000 7f5d18843700 1 -- 172.21.15.16:0/3973884150 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d140ffd80 con 0x7f5cfc06dd60 2024-04-02T04:45:56.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.063+0000 7f5d037fe700 1 -- 172.21.15.16:0/3973884150 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f5d140ffd80 con 0x7f5cfc06dd60 2024-04-02T04:45:56.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.065+0000 7f5d18843700 1 -- 172.21.15.16:0/3973884150 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5cfc06dd60 msgr2=0x7f5cfc070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:56.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.065+0000 7f5d18843700 1 --2- 172.21.15.16:0/3973884150 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5cfc06dd60 0x7f5cfc070220 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5d08005090 tx=0x7f5d08005960 comp rx=0 tx=0).stop 2024-04-02T04:45:56.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.065+0000 7f5d18843700 1 -- 172.21.15.16:0/3973884150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d14101560 msgr2=0x7f5d141987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:56.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.065+0000 7f5d18843700 1 --2- 172.21.15.16:0/3973884150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d14101560 0x7f5d141987b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400b4b0 tx=0x7f5d0400b7c0 comp rx=0 tx=0).stop 2024-04-02T04:45:56.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.065+0000 7f5d18843700 1 -- 172.21.15.16:0/3973884150 shutdown_connections 2024-04-02T04:45:56.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.066+0000 7f5d18843700 1 --2- 172.21.15.16:0/3973884150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d14101560 0x7f5d141987b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:56.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.066+0000 7f5d18843700 1 --2- 172.21.15.16:0/3973884150 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5cfc06dd60 0x7f5cfc070220 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:56.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.066+0000 7f5d18843700 1 --2- 172.21.15.16:0/3973884150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d14103e90 0x7f5d14198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:56.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.066+0000 7f5d18843700 1 -- 172.21.15.16:0/3973884150 >> 172.21.15.16:0/3973884150 conn(0x7f5d140faf70 msgr2=0x7f5d140ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:56.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.066+0000 7f5d18843700 1 -- 172.21.15.16:0/3973884150 shutdown_connections 2024-04-02T04:45:56.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.066+0000 7f5d18843700 1 -- 172.21.15.16:0/3973884150 wait complete. 2024-04-02T04:45:56.080 INFO:teuthology.orchestra.run.smithi016.stdout:true 2024-04-02T04:45:56.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.174+0000 7fefc7419700 1 -- 172.21.15.16:0/2727552408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefc00ff350 msgr2=0x7fefc00ff770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:56.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.174+0000 7fefc7419700 1 --2- 172.21.15.16:0/2727552408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefc00ff350 0x7fefc00ff770 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fefbc009910 tx=0x7fefbc009c20 comp rx=0 tx=0).stop 2024-04-02T04:45:56.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.174+0000 7fefc7419700 1 -- 172.21.15.16:0/2727552408 shutdown_connections 2024-04-02T04:45:56.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.174+0000 7fefc7419700 1 --2- 172.21.15.16:0/2727552408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefc00ffcb0 0x7fefc0100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:56.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.174+0000 7fefc7419700 1 --2- 172.21.15.16:0/2727552408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefc00ff350 0x7fefc00ff770 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:56.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.174+0000 7fefc7419700 1 -- 172.21.15.16:0/2727552408 >> 172.21.15.16:0/2727552408 conn(0x7fefc00faf70 msgr2=0x7fefc00fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:56.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.175+0000 7fefc7419700 1 -- 172.21.15.16:0/2727552408 shutdown_connections 2024-04-02T04:45:56.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.175+0000 7fefc7419700 1 -- 172.21.15.16:0/2727552408 wait complete. 2024-04-02T04:45:56.179 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:56 smithi016 podman[102202]: 2024-04-02 04:45:56.107602004 +0000 UTC m=+0.357575450 container init 421d9e1cc751685a0849fcb4c1d0592e6671836c7a718a8bc6731ce954bdaaef (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, RELEASE=squid-6cec793, maintainer=Guillaume Abrioux ) 2024-04-02T04:45:56.179 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:56 smithi016 podman[102202]: 2024-04-02 04:45:56.11073757 +0000 UTC m=+0.360711018 container start 421d9e1cc751685a0849fcb4c1d0592e6671836c7a718a8bc6731ce954bdaaef (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-6cec793, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-02T04:45:56.179 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:56 smithi016 ceph-osd[102246]: -- 172.21.15.16:0/4148847915 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558e16197680 con 0x558e16161000 2024-04-02T04:45:56.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.175+0000 7fefc7419700 1 Processor -- start 2024-04-02T04:45:56.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.175+0000 7fefc7419700 1 -- start start 2024-04-02T04:45:56.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.175+0000 7fefc7419700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefc00ff350 0x7fefc0198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:56.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.175+0000 7fefc7419700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefc00ffcb0 0x7fefc0198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:56.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.175+0000 7fefc7419700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc0199270 con 0x7fefc00ffcb0 2024-04-02T04:45:56.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.175+0000 7fefc7419700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc01993b0 con 0x7fefc00ff350 2024-04-02T04:45:56.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.176+0000 7fefc49b4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefc00ffcb0 0x7fefc0198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:56.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.176+0000 7fefc49b4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefc00ffcb0 0x7fefc0198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56900/0 (socket says 172.21.15.16:56900) 2024-04-02T04:45:56.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.176+0000 7fefc49b4700 1 -- 172.21.15.16:0/1587033767 learned_addr learned my addr 172.21.15.16:0/1587033767 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:45:56.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.176+0000 7fefc51b5700 1 --2- 172.21.15.16:0/1587033767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefc00ff350 0x7fefc0198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:56.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.176+0000 7fefc49b4700 1 -- 172.21.15.16:0/1587033767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefc00ff350 msgr2=0x7fefc0198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:56.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.176+0000 7fefc49b4700 1 --2- 172.21.15.16:0/1587033767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefc00ff350 0x7fefc0198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:56.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.176+0000 7fefc49b4700 1 -- 172.21.15.16:0/1587033767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefbc0095f0 con 0x7fefc00ffcb0 2024-04-02T04:45:56.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.176+0000 7fefc49b4700 1 --2- 172.21.15.16:0/1587033767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefc00ffcb0 0x7fefc0198c50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fefb000d8d0 tx=0x7fefb000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:56.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.176+0000 7fefb67fc700 1 -- 172.21.15.16:0/1587033767 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefb00098a0 con 0x7fefc00ffcb0 2024-04-02T04:45:56.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.176+0000 7fefc7419700 1 -- 172.21.15.16:0/1587033767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefc019ddc0 con 0x7fefc00ffcb0 2024-04-02T04:45:56.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.176+0000 7fefb67fc700 1 -- 172.21.15.16:0/1587033767 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fefb0010460 con 0x7fefc00ffcb0 2024-04-02T04:45:56.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.176+0000 7fefb67fc700 1 -- 172.21.15.16:0/1587033767 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefb000f5f0 con 0x7fefc00ffcb0 2024-04-02T04:45:56.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.176+0000 7fefc7419700 1 -- 172.21.15.16:0/1587033767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefc019e360 con 0x7fefc00ffcb0 2024-04-02T04:45:56.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.177+0000 7fefc7419700 1 -- 172.21.15.16:0/1587033767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefc0192840 con 0x7fefc00ffcb0 2024-04-02T04:45:56.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.178+0000 7fefb67fc700 1 -- 172.21.15.16:0/1587033767 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fefb0009a00 con 0x7fefc00ffcb0 2024-04-02T04:45:56.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.178+0000 7fefb67fc700 1 --2- 172.21.15.16:0/1587033767 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fefac06dd60 0x7fefac070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:56.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.178+0000 7fefb67fc700 1 -- 172.21.15.16:0/1587033767 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fefb008da20 con 0x7fefc00ffcb0 2024-04-02T04:45:56.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.178+0000 7fefc51b5700 1 --2- 172.21.15.16:0/1587033767 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fefac06dd60 0x7fefac070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:56.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.179+0000 7fefc51b5700 1 --2- 172.21.15.16:0/1587033767 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fefac06dd60 0x7fefac070220 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fefbc005090 tx=0x7fefbc005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:56.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.183+0000 7fefb67fc700 1 -- 172.21.15.16:0/1587033767 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fefb005aae0 con 0x7fefc00ffcb0 2024-04-02T04:45:56.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.412+0000 7fefc7419700 1 -- 172.21.15.16:0/1587033767 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fefc002cdc0 con 0x7fefac06dd60 2024-04-02T04:45:56.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.413+0000 7fefb67fc700 1 -- 172.21.15.16:0/1587033767 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fefc002cdc0 con 0x7fefac06dd60 2024-04-02T04:45:56.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.415+0000 7fefc7419700 1 -- 172.21.15.16:0/1587033767 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fefac06dd60 msgr2=0x7fefac070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:56.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.415+0000 7fefc7419700 1 --2- 172.21.15.16:0/1587033767 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fefac06dd60 0x7fefac070220 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fefbc005090 tx=0x7fefbc005960 comp rx=0 tx=0).stop 2024-04-02T04:45:56.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.415+0000 7fefc7419700 1 -- 172.21.15.16:0/1587033767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefc00ffcb0 msgr2=0x7fefc0198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:56.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.415+0000 7fefc7419700 1 --2- 172.21.15.16:0/1587033767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefc00ffcb0 0x7fefc0198c50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fefb000d8d0 tx=0x7fefb000dbe0 comp rx=0 tx=0).stop 2024-04-02T04:45:56.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.415+0000 7fefc7419700 1 -- 172.21.15.16:0/1587033767 shutdown_connections 2024-04-02T04:45:56.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.415+0000 7fefc7419700 1 --2- 172.21.15.16:0/1587033767 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fefac06dd60 0x7fefac070220 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:56.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.415+0000 7fefc7419700 1 --2- 172.21.15.16:0/1587033767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefc00ff350 0x7fefc0198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:56.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.415+0000 7fefc7419700 1 --2- 172.21.15.16:0/1587033767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefc00ffcb0 0x7fefc0198c50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:56.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.415+0000 7fefc7419700 1 -- 172.21.15.16:0/1587033767 >> 172.21.15.16:0/1587033767 conn(0x7fefc00faf70 msgr2=0x7fefc0107d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:56.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.416+0000 7fefc7419700 1 -- 172.21.15.16:0/1587033767 shutdown_connections 2024-04-02T04:45:56.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.416+0000 7fefc7419700 1 -- 172.21.15.16:0/1587033767 wait complete. 2024-04-02T04:45:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.524+0000 7f46e4c7b700 1 -- 172.21.15.16:0/2663874503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46e0100bd0 msgr2=0x7f46e0101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:56.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.524+0000 7f46e4c7b700 1 --2- 172.21.15.16:0/2663874503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46e0100bd0 0x7f46e0101030 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f46d0009910 tx=0x7f46d0009c20 comp rx=0 tx=0).stop 2024-04-02T04:45:56.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.524+0000 7f46e4c7b700 1 -- 172.21.15.16:0/2663874503 shutdown_connections 2024-04-02T04:45:56.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.524+0000 7f46e4c7b700 1 --2- 172.21.15.16:0/2663874503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46e0100bd0 0x7f46e0101030 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:56.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.524+0000 7f46e4c7b700 1 --2- 172.21.15.16:0/2663874503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46e00ff9d0 0x7f46e00ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:56.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.524+0000 7f46e4c7b700 1 -- 172.21.15.16:0/2663874503 >> 172.21.15.16:0/2663874503 conn(0x7f46e00faf70 msgr2=0x7f46e00fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:56.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.524+0000 7f46e4c7b700 1 -- 172.21.15.16:0/2663874503 shutdown_connections 2024-04-02T04:45:56.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.524+0000 7f46e4c7b700 1 -- 172.21.15.16:0/2663874503 wait complete. 2024-04-02T04:45:56.528 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:56 smithi016 bash[102202]: 421d9e1cc751685a0849fcb4c1d0592e6671836c7a718a8bc6731ce954bdaaef 2024-04-02T04:45:56.528 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:56 smithi016 systemd[1]: Started Ceph osd.0 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4. 2024-04-02T04:45:56.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.525+0000 7f46e4c7b700 1 Processor -- start 2024-04-02T04:45:56.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.525+0000 7f46e4c7b700 1 -- start start 2024-04-02T04:45:56.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.525+0000 7f46e4c7b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46e00ff9d0 0x7f46e0103910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:56.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.525+0000 7f46e4c7b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46e0100bd0 0x7f46e0101fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:56.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.525+0000 7f46e4c7b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e01024f0 con 0x7f46e0100bd0 2024-04-02T04:45:56.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.525+0000 7f46e4c7b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e0102630 con 0x7f46e00ff9d0 2024-04-02T04:45:56.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.525+0000 7f46ddd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46e0100bd0 0x7f46e0101fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:56.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.525+0000 7f46ddd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46e0100bd0 0x7f46e0101fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56930/0 (socket says 172.21.15.16:56930) 2024-04-02T04:45:56.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.525+0000 7f46de59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46e00ff9d0 0x7f46e0103910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:56.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.525+0000 7f46ddd9b700 1 -- 172.21.15.16:0/1846047903 learned_addr learned my addr 172.21.15.16:0/1846047903 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:45:56.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.526+0000 7f46ddd9b700 1 -- 172.21.15.16:0/1846047903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46e00ff9d0 msgr2=0x7f46e0103910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:56.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.526+0000 7f46ddd9b700 1 --2- 172.21.15.16:0/1846047903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46e00ff9d0 0x7f46e0103910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:56.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.526+0000 7f46ddd9b700 1 -- 172.21.15.16:0/1846047903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d00095f0 con 0x7f46e0100bd0 2024-04-02T04:45:56.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.526+0000 7f46ddd9b700 1 --2- 172.21.15.16:0/1846047903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46e0100bd0 0x7f46e0101fb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f46d00059d0 tx=0x7f46d000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:56.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.526+0000 7f46d77fe700 1 -- 172.21.15.16:0/1846047903 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46d001d070 con 0x7f46e0100bd0 2024-04-02T04:45:56.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.526+0000 7f46d77fe700 1 -- 172.21.15.16:0/1846047903 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f46d0005dc0 con 0x7f46e0100bd0 2024-04-02T04:45:56.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.526+0000 7f46e4c7b700 1 -- 172.21.15.16:0/1846047903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46e01028b0 con 0x7f46e0100bd0 2024-04-02T04:45:56.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.526+0000 7f46d77fe700 1 -- 172.21.15.16:0/1846047903 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46d0021410 con 0x7f46e0100bd0 2024-04-02T04:45:56.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.526+0000 7f46e4c7b700 1 -- 172.21.15.16:0/1846047903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46e0102da0 con 0x7f46e0100bd0 2024-04-02T04:45:56.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.527+0000 7f46e4c7b700 1 -- 172.21.15.16:0/1846047903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46c00052f0 con 0x7f46e0100bd0 2024-04-02T04:45:56.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.528+0000 7f46d77fe700 1 -- 172.21.15.16:0/1846047903 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f46d0003a60 con 0x7f46e0100bd0 2024-04-02T04:45:56.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.528+0000 7f46d77fe700 1 --2- 172.21.15.16:0/1846047903 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f46cc06dd60 0x7f46cc070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:56.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.528+0000 7f46d77fe700 1 -- 172.21.15.16:0/1846047903 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f46d008ebf0 con 0x7f46e0100bd0 2024-04-02T04:45:56.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.528+0000 7f46de59c700 1 --2- 172.21.15.16:0/1846047903 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f46cc06dd60 0x7f46cc070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:56.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.529+0000 7f46de59c700 1 --2- 172.21.15.16:0/1846047903 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f46cc06dd60 0x7f46cc070220 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f46e0100a30 tx=0x7f46c8007400 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:56.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.533+0000 7f46d77fe700 1 -- 172.21.15.16:0/1846047903 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f46d005bcb0 con 0x7f46e0100bd0 2024-04-02T04:45:56.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.756+0000 7f46e4c7b700 1 -- 172.21.15.16:0/1846047903 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f46c0000bc0 con 0x7f46cc06dd60 2024-04-02T04:45:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.772+0000 7f46d77fe700 1 -- 172.21.15.16:0/1846047903 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f46c0000bc0 con 0x7f46cc06dd60 2024-04-02T04:45:56.776 INFO:teuthology.orchestra.run.smithi016.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:45:56.776 INFO:teuthology.orchestra.run.smithi016.stdout:alertmanager.smithi016 smithi016 *:9093,9094 running (10m) 15s ago 11m 24.2M - 0.25.0 c8568f914cd2 b95d8354b016 2024-04-02T04:45:56.776 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi016 smithi016 running (12m) 15s ago 12m 7922k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 9de6c16a487f 2024-04-02T04:45:56.776 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi027 smithi027 running (11m) 16s ago 11m 7964k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 99956592471a 2024-04-02T04:45:56.776 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi016 smithi016 running (26s) 15s ago 12m 6660k - 19.0.0-1979-g6cec793d 78f830190839 dc63ce0eef58 2024-04-02T04:45:56.776 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi027 smithi027 running (20s) 16s ago 10m 6665k - 19.0.0-1979-g6cec793d 78f830190839 2beb077018e4 2024-04-02T04:45:56.776 INFO:teuthology.orchestra.run.smithi016.stdout:grafana.smithi016 smithi016 *:3000 running (10m) 15s ago 11m 86.2M - 9.4.7 954c08fa6188 7c43d133569c 2024-04-02T04:45:56.776 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.fyzrnm smithi016 running (6m) 15s ago 6m 17.2M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 70f28b9572a6 2024-04-02T04:45:56.776 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.uhgkgg smithi016 running (6m) 15s ago 6m 15.5M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 0aa4d16562a6 2024-04-02T04:45:56.776 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.sdublg smithi027 running (6m) 16s ago 6m 19.3M - 18.2.2-1059-g4d05f4ca 909dce4fe53d a538451099db 2024-04-02T04:45:56.776 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.zwrfoo smithi027 running (6m) 16s ago 6m 158M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 4cb3fa428d71 2024-04-02T04:45:56.776 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi016.eqnebx smithi016 *:8443,9283,8765 running (2m) 15s ago 13m 495M - 19.0.0-1979-g6cec793d 78f830190839 d61ed91b4577 2024-04-02T04:45:56.776 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi027.kjzcsc smithi027 *:8443,9283,8765 running (2m) 16s ago 10m 449M - 19.0.0-1979-g6cec793d 78f830190839 5072a5ffc761 2024-04-02T04:45:56.776 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi016 smithi016 running (109s) 15s ago 13m 42.5M 2048M 19.0.0-1979-g6cec793d 78f830190839 eabfc84f1cdf 2024-04-02T04:45:56.777 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi027 smithi027 running (81s) 16s ago 10m 36.8M 2048M 19.0.0-1979-g6cec793d 78f830190839 fcd68b20dcc5 2024-04-02T04:45:56.777 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi016 smithi016 *:9100 running (12m) 15s ago 12m 14.8M - 1.5.0 0da6a335fe13 09f54c65b53c 2024-04-02T04:45:56.777 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi027 smithi027 *:9100 running (10m) 16s ago 10m 15.2M - 1.5.0 0da6a335fe13 e41a657c7443 2024-04-02T04:45:56.777 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 smithi016 starting - - - 1217M 2024-04-02T04:45:56.777 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 smithi016 running (9m) 15s ago 9m 222M 1217M 18.2.2-1059-g4d05f4ca 909dce4fe53d 7361abc9e1b6 2024-04-02T04:45:56.777 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 smithi016 running (9m) 15s ago 9m 179M 1217M 18.2.2-1059-g4d05f4ca 909dce4fe53d 52e592b4d1f2 2024-04-02T04:45:56.777 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 smithi027 running (8m) 16s ago 8m 236M 2240M 18.2.2-1059-g4d05f4ca 909dce4fe53d cf7c7fe022f0 2024-04-02T04:45:56.777 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 smithi027 running (8m) 16s ago 8m 211M 2240M 18.2.2-1059-g4d05f4ca 909dce4fe53d 8cbbedf8cac9 2024-04-02T04:45:56.777 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 smithi027 running (7m) 16s ago 7m 194M 2240M 18.2.2-1059-g4d05f4ca 909dce4fe53d ac6094415529 2024-04-02T04:45:56.777 INFO:teuthology.orchestra.run.smithi016.stdout:prometheus.smithi016 smithi016 *:9095 running (2m) 15s ago 11m 56.0M - 2.43.0 a07b618ecd1d 466f00d73ddd 2024-04-02T04:45:56.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.774+0000 7f46e4c7b700 1 -- 172.21.15.16:0/1846047903 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f46cc06dd60 msgr2=0x7f46cc070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:56.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.774+0000 7f46e4c7b700 1 --2- 172.21.15.16:0/1846047903 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f46cc06dd60 0x7f46cc070220 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f46e0100a30 tx=0x7f46c8007400 comp rx=0 tx=0).stop 2024-04-02T04:45:56.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.774+0000 7f46e4c7b700 1 -- 172.21.15.16:0/1846047903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46e0100bd0 msgr2=0x7f46e0101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:56.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.774+0000 7f46e4c7b700 1 --2- 172.21.15.16:0/1846047903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46e0100bd0 0x7f46e0101fb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f46d00059d0 tx=0x7f46d000bd70 comp rx=0 tx=0).stop 2024-04-02T04:45:56.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.774+0000 7f46e4c7b700 1 -- 172.21.15.16:0/1846047903 shutdown_connections 2024-04-02T04:45:56.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.774+0000 7f46e4c7b700 1 --2- 172.21.15.16:0/1846047903 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f46cc06dd60 0x7f46cc070220 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:56.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.774+0000 7f46e4c7b700 1 --2- 172.21.15.16:0/1846047903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46e00ff9d0 0x7f46e0103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:56.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.774+0000 7f46e4c7b700 1 --2- 172.21.15.16:0/1846047903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46e0100bd0 0x7f46e0101fb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:56.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.774+0000 7f46e4c7b700 1 -- 172.21.15.16:0/1846047903 >> 172.21.15.16:0/1846047903 conn(0x7f46e00faf70 msgr2=0x7f46e0068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:56.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.774+0000 7f46e4c7b700 1 -- 172.21.15.16:0/1846047903 shutdown_connections 2024-04-02T04:45:56.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.775+0000 7f46e4c7b700 1 -- 172.21.15.16:0/1846047903 wait complete. 2024-04-02T04:45:56.779 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:45:56 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0[102237]: 2024-04-02T04:45:56.663+0000 7f839ed207c0 -1 Falling back to public interface 2024-04-02T04:45:56.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.888+0000 7f664b0ba700 1 -- 172.21.15.16:0/1378467327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66440ff9e0 msgr2=0x7f66440ffe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:56.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.888+0000 7f664b0ba700 1 --2- 172.21.15.16:0/1378467327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66440ff9e0 0x7f66440ffe00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6638009910 tx=0x7f6638009c20 comp rx=0 tx=0).stop 2024-04-02T04:45:56.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.888+0000 7f664b0ba700 1 -- 172.21.15.16:0/1378467327 shutdown_connections 2024-04-02T04:45:56.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.888+0000 7f664b0ba700 1 --2- 172.21.15.16:0/1378467327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6644100be0 0x7f6644101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:56.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.888+0000 7f664b0ba700 1 --2- 172.21.15.16:0/1378467327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66440ff9e0 0x7f66440ffe00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:56.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.888+0000 7f664b0ba700 1 -- 172.21.15.16:0/1378467327 >> 172.21.15.16:0/1378467327 conn(0x7f66440faf80 msgr2=0x7f66440fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:56.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.888+0000 7f664b0ba700 1 -- 172.21.15.16:0/1378467327 shutdown_connections 2024-04-02T04:45:56.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.888+0000 7f664b0ba700 1 -- 172.21.15.16:0/1378467327 wait complete. 2024-04-02T04:45:56.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.889+0000 7f664b0ba700 1 Processor -- start 2024-04-02T04:45:56.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.889+0000 7f664b0ba700 1 -- start start 2024-04-02T04:45:56.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.890+0000 7f664b0ba700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66440ff9e0 0x7f6644103920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:56.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.890+0000 7f664b0ba700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6644100be0 0x7f6644101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:56.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.890+0000 7f664b0ba700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664419a300 con 0x7f6644100be0 2024-04-02T04:45:56.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.890+0000 7f664b0ba700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664419a440 con 0x7f66440ff9e0 2024-04-02T04:45:56.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.890+0000 7f6643fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6644100be0 0x7f6644101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:56.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.890+0000 7f6648e56700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66440ff9e0 0x7f6644103920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:56.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.890+0000 7f6643fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6644100be0 0x7f6644101fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56944/0 (socket says 172.21.15.16:56944) 2024-04-02T04:45:56.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.890+0000 7f6643fff700 1 -- 172.21.15.16:0/2618919215 learned_addr learned my addr 172.21.15.16:0/2618919215 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:45:56.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.890+0000 7f6643fff700 1 -- 172.21.15.16:0/2618919215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66440ff9e0 msgr2=0x7f6644103920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:56.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.890+0000 7f6643fff700 1 --2- 172.21.15.16:0/2618919215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66440ff9e0 0x7f6644103920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:56.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.890+0000 7f6643fff700 1 -- 172.21.15.16:0/2618919215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66380095f0 con 0x7f6644100be0 2024-04-02T04:45:56.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.890+0000 7f6643fff700 1 --2- 172.21.15.16:0/2618919215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6644100be0 0x7f6644101fc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f663000b4b0 tx=0x7f663000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:56.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.891+0000 7f6641ffb700 1 -- 172.21.15.16:0/2618919215 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6630011800 con 0x7f6644100be0 2024-04-02T04:45:56.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.891+0000 7f664b0ba700 1 -- 172.21.15.16:0/2618919215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6644102500 con 0x7f6644100be0 2024-04-02T04:45:56.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.891+0000 7f664b0ba700 1 -- 172.21.15.16:0/2618919215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6644102aa0 con 0x7f6644100be0 2024-04-02T04:45:56.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.892+0000 7f6641ffb700 1 -- 172.21.15.16:0/2618919215 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6630011e20 con 0x7f6644100be0 2024-04-02T04:45:56.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.892+0000 7f6641ffb700 1 -- 172.21.15.16:0/2618919215 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f663000f2e0 con 0x7f6644100be0 2024-04-02T04:45:56.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.893+0000 7f664b0ba700 1 -- 172.21.15.16:0/2618919215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66280052f0 con 0x7f6644100be0 2024-04-02T04:45:56.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.894+0000 7f6641ffb700 1 -- 172.21.15.16:0/2618919215 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f6630011960 con 0x7f6644100be0 2024-04-02T04:45:56.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.895+0000 7f6641ffb700 1 --2- 172.21.15.16:0/2618919215 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f663406dd60 0x7f6634070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:56.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.895+0000 7f6641ffb700 1 -- 172.21.15.16:0/2618919215 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f663008d940 con 0x7f6644100be0 2024-04-02T04:45:56.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.895+0000 7f6648e56700 1 --2- 172.21.15.16:0/2618919215 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f663406dd60 0x7f6634070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:56.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.895+0000 7f6648e56700 1 --2- 172.21.15.16:0/2618919215 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f663406dd60 0x7f6634070220 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6638009fd0 tx=0x7f6638005090 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:56.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:56.899+0000 7f6641ffb700 1 -- 172.21.15.16:0/2618919215 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f663005aa80 con 0x7f6644100be0 2024-04-02T04:45:57.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.198+0000 7f664b0ba700 1 -- 172.21.15.16:0/2618919215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6628005870 con 0x7f6644100be0 2024-04-02T04:45:57.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.199+0000 7f6641ffb700 1 -- 172.21.15.16:0/2618919215 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+732 (secure 0 0 0) 0x7f663005a610 con 0x7f6644100be0 2024-04-02T04:45:57.202 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:45:57.203 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": { 2024-04-02T04:45:57.203 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:45:57.203 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:45:57.203 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": { 2024-04-02T04:45:57.203 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:45:57.203 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:45:57.203 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": { 2024-04-02T04:45:57.203 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 5 2024-04-02T04:45:57.203 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:45:57.203 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-04-02T04:45:57.203 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 4 2024-04-02T04:45:57.204 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:45:57.204 INFO:teuthology.orchestra.run.smithi016.stdout: "overall": { 2024-04-02T04:45:57.204 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 9, 2024-04-02T04:45:57.204 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 4 2024-04-02T04:45:57.204 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-02T04:45:57.204 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:45:57.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.202+0000 7f664b0ba700 1 -- 172.21.15.16:0/2618919215 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f663406dd60 msgr2=0x7f6634070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:57.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.202+0000 7f664b0ba700 1 --2- 172.21.15.16:0/2618919215 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f663406dd60 0x7f6634070220 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6638009fd0 tx=0x7f6638005090 comp rx=0 tx=0).stop 2024-04-02T04:45:57.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.202+0000 7f664b0ba700 1 -- 172.21.15.16:0/2618919215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6644100be0 msgr2=0x7f6644101fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:57.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.202+0000 7f664b0ba700 1 --2- 172.21.15.16:0/2618919215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6644100be0 0x7f6644101fc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f663000b4b0 tx=0x7f663000b7c0 comp rx=0 tx=0).stop 2024-04-02T04:45:57.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.202+0000 7f664b0ba700 1 -- 172.21.15.16:0/2618919215 shutdown_connections 2024-04-02T04:45:57.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.202+0000 7f664b0ba700 1 --2- 172.21.15.16:0/2618919215 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f663406dd60 0x7f6634070220 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:57.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.202+0000 7f664b0ba700 1 --2- 172.21.15.16:0/2618919215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66440ff9e0 0x7f6644103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:57.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.202+0000 7f664b0ba700 1 --2- 172.21.15.16:0/2618919215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6644100be0 0x7f6644101fc0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:57.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.202+0000 7f664b0ba700 1 -- 172.21.15.16:0/2618919215 >> 172.21.15.16:0/2618919215 conn(0x7f66440faf80 msgr2=0x7f6644069060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:57.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.202+0000 7f664b0ba700 1 -- 172.21.15.16:0/2618919215 shutdown_connections 2024-04-02T04:45:57.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.202+0000 7f664b0ba700 1 -- 172.21.15.16:0/2618919215 wait complete. 2024-04-02T04:45:57.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:57 smithi016 ceph-mon[93230]: from='client.34216 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:45:57.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:57 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:57.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:57 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:57.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:57 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:45:57.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:57 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/2618919215' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:45:57.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.314+0000 7f34c80d3700 1 -- 172.21.15.16:0/3398894520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34c00fedb0 msgr2=0x7f34c00ff1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:57.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.314+0000 7f34c80d3700 1 --2- 172.21.15.16:0/3398894520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34c00fedb0 0x7f34c00ff1d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f34b000b0a0 tx=0x7f34b000b3b0 comp rx=0 tx=0).stop 2024-04-02T04:45:57.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.314+0000 7f34c80d3700 1 -- 172.21.15.16:0/3398894520 shutdown_connections 2024-04-02T04:45:57.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.314+0000 7f34c80d3700 1 --2- 172.21.15.16:0/3398894520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34c0101d90 0x7f34c01021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:57.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.314+0000 7f34c80d3700 1 --2- 172.21.15.16:0/3398894520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34c00fedb0 0x7f34c00ff1d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:57.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.314+0000 7f34c80d3700 1 -- 172.21.15.16:0/3398894520 >> 172.21.15.16:0/3398894520 conn(0x7f34c00fa730 msgr2=0x7f34c00fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:57.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.314+0000 7f34c80d3700 1 -- 172.21.15.16:0/3398894520 shutdown_connections 2024-04-02T04:45:57.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.314+0000 7f34c80d3700 1 -- 172.21.15.16:0/3398894520 wait complete. 2024-04-02T04:45:57.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.315+0000 7f34c80d3700 1 Processor -- start 2024-04-02T04:45:57.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.315+0000 7f34c80d3700 1 -- start start 2024-04-02T04:45:57.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.315+0000 7f34c80d3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34c00fedb0 0x7f34c01964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:57.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.315+0000 7f34c80d3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34c0101d90 0x7f34c0196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:57.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.315+0000 7f34c80d3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c0197030 con 0x7f34c0101d90 2024-04-02T04:45:57.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.315+0000 7f34c80d3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c0197170 con 0x7f34c00fedb0 2024-04-02T04:45:57.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.315+0000 7f34c566e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34c0101d90 0x7f34c0196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:57.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.315+0000 7f34c566e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34c0101d90 0x7f34c0196a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56958/0 (socket says 172.21.15.16:56958) 2024-04-02T04:45:57.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.315+0000 7f34c566e700 1 -- 172.21.15.16:0/2916069932 learned_addr learned my addr 172.21.15.16:0/2916069932 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:45:57.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.315+0000 7f34c5e6f700 1 --2- 172.21.15.16:0/2916069932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34c00fedb0 0x7f34c01964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:57.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.315+0000 7f34c566e700 1 -- 172.21.15.16:0/2916069932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34c00fedb0 msgr2=0x7f34c01964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:57.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.315+0000 7f34c566e700 1 --2- 172.21.15.16:0/2916069932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34c00fedb0 0x7f34c01964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:57.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.315+0000 7f34c566e700 1 -- 172.21.15.16:0/2916069932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b0009d00 con 0x7f34c0101d90 2024-04-02T04:45:57.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.316+0000 7f34c5e6f700 1 --2- 172.21.15.16:0/2916069932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34c00fedb0 0x7f34c01964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T04:45:57.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.316+0000 7f34c566e700 1 --2- 172.21.15.16:0/2916069932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34c0101d90 0x7f34c0196a10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f34b400d8d0 tx=0x7f34b400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:57.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.316+0000 7f34beffd700 1 -- 172.21.15.16:0/2916069932 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34b4009940 con 0x7f34c0101d90 2024-04-02T04:45:57.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.316+0000 7f34beffd700 1 -- 172.21.15.16:0/2916069932 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f34b4010460 con 0x7f34c0101d90 2024-04-02T04:45:57.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.316+0000 7f34c80d3700 1 -- 172.21.15.16:0/2916069932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34c019bbd0 con 0x7f34c0101d90 2024-04-02T04:45:57.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.316+0000 7f34beffd700 1 -- 172.21.15.16:0/2916069932 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34b400f5f0 con 0x7f34c0101d90 2024-04-02T04:45:57.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.316+0000 7f34c80d3700 1 -- 172.21.15.16:0/2916069932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34c019c1a0 con 0x7f34c0101d90 2024-04-02T04:45:57.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.317+0000 7f34c80d3700 1 -- 172.21.15.16:0/2916069932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34c0190670 con 0x7f34c0101d90 2024-04-02T04:45:57.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.317+0000 7f34beffd700 1 -- 172.21.15.16:0/2916069932 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f34b40105d0 con 0x7f34c0101d90 2024-04-02T04:45:57.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.318+0000 7f34beffd700 1 --2- 172.21.15.16:0/2916069932 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f34a806ddb0 0x7f34a8070270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:57.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.318+0000 7f34beffd700 1 -- 172.21.15.16:0/2916069932 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f34b408dbd0 con 0x7f34c0101d90 2024-04-02T04:45:57.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.318+0000 7f34c5e6f700 1 --2- 172.21.15.16:0/2916069932 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f34a806ddb0 0x7f34a8070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:57.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.318+0000 7f34c5e6f700 1 --2- 172.21.15.16:0/2916069932 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f34a806ddb0 0x7f34a8070270 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f34b0000f80 tx=0x7f34b000b7d0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:57.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.323+0000 7f34beffd700 1 -- 172.21.15.16:0/2916069932 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f34b405abf0 con 0x7f34c0101d90 2024-04-02T04:45:57.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.576+0000 7f34c80d3700 1 -- 172.21.15.16:0/2916069932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f34c004ea80 con 0x7f34c0101d90 2024-04-02T04:45:57.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.576+0000 7f34beffd700 1 -- 172.21.15.16:0/2916069932 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1875 (secure 0 0 0) 0x7f34b4016770 con 0x7f34c0101d90 2024-04-02T04:45:57.579 INFO:teuthology.orchestra.run.smithi016.stdout:e15 2024-04-02T04:45:57.579 INFO:teuthology.orchestra.run.smithi016.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:45:57.579 INFO:teuthology.orchestra.run.smithi016.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:45:57.580 INFO:teuthology.orchestra.run.smithi016.stdout:legacy client fscid: 1 2024-04-02T04:45:57.580 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:45:57.580 INFO:teuthology.orchestra.run.smithi016.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:45:57.580 INFO:teuthology.orchestra.run.smithi016.stdout:fs_name cephfs 2024-04-02T04:45:57.580 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 15 2024-04-02T04:45:57.580 INFO:teuthology.orchestra.run.smithi016.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:45:57.580 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-04-02T04:39:30.740168+0000 2024-04-02T04:45:57.580 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-04-02T04:39:49.875319+0000 2024-04-02T04:45:57.580 INFO:teuthology.orchestra.run.smithi016.stdout:tableserver 0 2024-04-02T04:45:57.580 INFO:teuthology.orchestra.run.smithi016.stdout:root 0 2024-04-02T04:45:57.580 INFO:teuthology.orchestra.run.smithi016.stdout:session_timeout 60 2024-04-02T04:45:57.580 INFO:teuthology.orchestra.run.smithi016.stdout:session_autoclose 300 2024-04-02T04:45:57.580 INFO:teuthology.orchestra.run.smithi016.stdout:max_file_size 1099511627776 2024-04-02T04:45:57.580 INFO:teuthology.orchestra.run.smithi016.stdout:max_xattr_size 65536 2024-04-02T04:45:57.581 INFO:teuthology.orchestra.run.smithi016.stdout:required_client_features {} 2024-04-02T04:45:57.581 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure 0 2024-04-02T04:45:57.581 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure_osd_epoch 46 2024-04-02T04:45:57.581 INFO:teuthology.orchestra.run.smithi016.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:45:57.581 INFO:teuthology.orchestra.run.smithi016.stdout:max_mds 1 2024-04-02T04:45:57.581 INFO:teuthology.orchestra.run.smithi016.stdout:in 0 2024-04-02T04:45:57.581 INFO:teuthology.orchestra.run.smithi016.stdout:up {0=24285} 2024-04-02T04:45:57.581 INFO:teuthology.orchestra.run.smithi016.stdout:failed 2024-04-02T04:45:57.581 INFO:teuthology.orchestra.run.smithi016.stdout:damaged 2024-04-02T04:45:57.581 INFO:teuthology.orchestra.run.smithi016.stdout:stopped 2024-04-02T04:45:57.581 INFO:teuthology.orchestra.run.smithi016.stdout:data_pools [3] 2024-04-02T04:45:57.581 INFO:teuthology.orchestra.run.smithi016.stdout:metadata_pool 2 2024-04-02T04:45:57.581 INFO:teuthology.orchestra.run.smithi016.stdout:inline_data enabled 2024-04-02T04:45:57.581 INFO:teuthology.orchestra.run.smithi016.stdout:balancer 2024-04-02T04:45:57.581 INFO:teuthology.orchestra.run.smithi016.stdout:bal_rank_mask -1 2024-04-02T04:45:57.582 INFO:teuthology.orchestra.run.smithi016.stdout:standby_count_wanted 1 2024-04-02T04:45:57.582 INFO:teuthology.orchestra.run.smithi016.stdout:qdb_cluster leader: 0 members: 2024-04-02T04:45:57.582 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.zwrfoo{0:24285} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.27:6824/1461455976,v1:172.21.15.27:6825/1461455976] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:45:57.582 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:45:57.582 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:45:57.582 INFO:teuthology.orchestra.run.smithi016.stdout:Standby daemons: 2024-04-02T04:45:57.582 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:45:57.582 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.fyzrnm{-1:14488} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.16:6828/2578719859,v1:172.21.15.16:6829/2578719859] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:45:57.582 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.uhgkgg{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.16:6826/601263579,v1:172.21.15.16:6827/601263579] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:45:57.582 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.sdublg{-1:24295} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.27:6826/4177192740,v1:172.21.15.27:6827/4177192740] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:45:57.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.579+0000 7f34c80d3700 1 -- 172.21.15.16:0/2916069932 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f34a806ddb0 msgr2=0x7f34a8070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:57.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.579+0000 7f34c80d3700 1 --2- 172.21.15.16:0/2916069932 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f34a806ddb0 0x7f34a8070270 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f34b0000f80 tx=0x7f34b000b7d0 comp rx=0 tx=0).stop 2024-04-02T04:45:57.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.579+0000 7f34c80d3700 1 -- 172.21.15.16:0/2916069932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34c0101d90 msgr2=0x7f34c0196a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:57.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.579+0000 7f34c80d3700 1 --2- 172.21.15.16:0/2916069932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34c0101d90 0x7f34c0196a10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f34b400d8d0 tx=0x7f34b400dc90 comp rx=0 tx=0).stop 2024-04-02T04:45:57.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.579+0000 7f34c80d3700 1 -- 172.21.15.16:0/2916069932 shutdown_connections 2024-04-02T04:45:57.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.579+0000 7f34c80d3700 1 --2- 172.21.15.16:0/2916069932 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f34a806ddb0 0x7f34a8070270 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:57.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.579+0000 7f34c80d3700 1 --2- 172.21.15.16:0/2916069932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34c00fedb0 0x7f34c01964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:57.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.579+0000 7f34c80d3700 1 --2- 172.21.15.16:0/2916069932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34c0101d90 0x7f34c0196a10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:57.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.579+0000 7f34c80d3700 1 -- 172.21.15.16:0/2916069932 >> 172.21.15.16:0/2916069932 conn(0x7f34c00fa730 msgr2=0x7f34c0105330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:57.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.579+0000 7f34c80d3700 1 -- 172.21.15.16:0/2916069932 shutdown_connections 2024-04-02T04:45:57.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.579+0000 7f34c80d3700 1 -- 172.21.15.16:0/2916069932 wait complete. 2024-04-02T04:45:57.584 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 15 2024-04-02T04:45:57.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:57 smithi027 ceph-mon[83036]: from='client.34216 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:45:57.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:57 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:57.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:57 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:45:57.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:57 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:45:57.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:57 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/2618919215' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:45:57.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.698+0000 7f01242bc700 1 -- 172.21.15.16:0/3233805743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f011c100b00 msgr2=0x7f011c100f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:57.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.698+0000 7f01242bc700 1 --2- 172.21.15.16:0/3233805743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f011c100b00 0x7f011c100f20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0118009910 tx=0x7f0118009c20 comp rx=0 tx=0).stop 2024-04-02T04:45:57.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.698+0000 7f01242bc700 1 -- 172.21.15.16:0/3233805743 shutdown_connections 2024-04-02T04:45:57.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.698+0000 7f01242bc700 1 --2- 172.21.15.16:0/3233805743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f011c101d00 0x7f011c102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:57.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.698+0000 7f01242bc700 1 --2- 172.21.15.16:0/3233805743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f011c100b00 0x7f011c100f20 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:57.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.698+0000 7f01242bc700 1 -- 172.21.15.16:0/3233805743 >> 172.21.15.16:0/3233805743 conn(0x7f011c0fc0a0 msgr2=0x7f011c0fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:57.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.698+0000 7f01242bc700 1 -- 172.21.15.16:0/3233805743 shutdown_connections 2024-04-02T04:45:57.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.698+0000 7f01242bc700 1 -- 172.21.15.16:0/3233805743 wait complete. 2024-04-02T04:45:57.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.698+0000 7f01242bc700 1 Processor -- start 2024-04-02T04:45:57.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.698+0000 7f01242bc700 1 -- start start 2024-04-02T04:45:57.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.699+0000 7f01242bc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f011c100b00 0x7f011c198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:57.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.699+0000 7f01242bc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f011c101d00 0x7f011c198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.699+0000 7f01242bc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011c199360 con 0x7f011c100b00 2024-04-02T04:45:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.699+0000 7f01242bc700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011c1994a0 con 0x7f011c101d00 2024-04-02T04:45:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.699+0000 7f0122058700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f011c100b00 0x7f011c198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.699+0000 7f0122058700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f011c100b00 0x7f011c198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56968/0 (socket says 172.21.15.16:56968) 2024-04-02T04:45:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.699+0000 7f0122058700 1 -- 172.21.15.16:0/190799306 learned_addr learned my addr 172.21.15.16:0/190799306 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:45:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.699+0000 7f0121857700 1 --2- 172.21.15.16:0/190799306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f011c101d00 0x7f011c198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.699+0000 7f0122058700 1 -- 172.21.15.16:0/190799306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f011c101d00 msgr2=0x7f011c198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.699+0000 7f0122058700 1 --2- 172.21.15.16:0/190799306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f011c101d00 0x7f011c198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.699+0000 7f0122058700 1 -- 172.21.15.16:0/190799306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01180095f0 con 0x7f011c100b00 2024-04-02T04:45:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.699+0000 7f0122058700 1 --2- 172.21.15.16:0/190799306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f011c100b00 0x7f011c198800 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f01180059d0 tx=0x7f011800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.699+0000 7f01137fe700 1 -- 172.21.15.16:0/190799306 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f011801d070 con 0x7f011c100b00 2024-04-02T04:45:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.700+0000 7f01137fe700 1 -- 172.21.15.16:0/190799306 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f011800f460 con 0x7f011c100b00 2024-04-02T04:45:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.700+0000 7f01137fe700 1 -- 172.21.15.16:0/190799306 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0118021620 con 0x7f011c100b00 2024-04-02T04:45:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.700+0000 7f01242bc700 1 -- 172.21.15.16:0/190799306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f011c19dea0 con 0x7f011c100b00 2024-04-02T04:45:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.700+0000 7f01242bc700 1 -- 172.21.15.16:0/190799306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f011c19e2e0 con 0x7f011c100b00 2024-04-02T04:45:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.701+0000 7f01242bc700 1 -- 172.21.15.16:0/190799306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f011c1066d0 con 0x7f011c100b00 2024-04-02T04:45:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.701+0000 7f01137fe700 1 -- 172.21.15.16:0/190799306 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f011800fa70 con 0x7f011c100b00 2024-04-02T04:45:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.702+0000 7f01137fe700 1 --2- 172.21.15.16:0/190799306 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f010806dd60 0x7f0108070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:57.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.702+0000 7f01137fe700 1 -- 172.21.15.16:0/190799306 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f011808ef10 con 0x7f011c100b00 2024-04-02T04:45:57.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.702+0000 7f0121857700 1 --2- 172.21.15.16:0/190799306 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f010806dd60 0x7f0108070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:57.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.702+0000 7f0121857700 1 --2- 172.21.15.16:0/190799306 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f010806dd60 0x7f0108070220 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f010c006fd0 tx=0x7f010c008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:57.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.707+0000 7f01137fe700 1 -- 172.21.15.16:0/190799306 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f011805bfd0 con 0x7f011c100b00 2024-04-02T04:45:57.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.937+0000 7f01242bc700 1 -- 172.21.15.16:0/190799306 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f011c02ce10 con 0x7f010806dd60 2024-04-02T04:45:57.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.938+0000 7f01137fe700 1 -- 172.21.15.16:0/190799306 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f011c02ce10 con 0x7f010806dd60 2024-04-02T04:45:57.941 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:45:57.941 INFO:teuthology.orchestra.run.smithi016.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:45:57.941 INFO:teuthology.orchestra.run.smithi016.stdout: "in_progress": true, 2024-04-02T04:45:57.942 INFO:teuthology.orchestra.run.smithi016.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-02T04:45:57.942 INFO:teuthology.orchestra.run.smithi016.stdout: "services_complete": [ 2024-04-02T04:45:57.942 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr", 2024-04-02T04:45:57.942 INFO:teuthology.orchestra.run.smithi016.stdout: "mon", 2024-04-02T04:45:57.942 INFO:teuthology.orchestra.run.smithi016.stdout: "crash" 2024-04-02T04:45:57.942 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-04-02T04:45:57.942 INFO:teuthology.orchestra.run.smithi016.stdout: "progress": "6/23 daemons upgraded", 2024-04-02T04:45:57.942 INFO:teuthology.orchestra.run.smithi016.stdout: "message": "Currently upgrading osd daemons", 2024-04-02T04:45:57.942 INFO:teuthology.orchestra.run.smithi016.stdout: "is_paused": false 2024-04-02T04:45:57.942 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:45:57.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.942+0000 7f01242bc700 1 -- 172.21.15.16:0/190799306 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f010806dd60 msgr2=0x7f0108070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:57.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.942+0000 7f01242bc700 1 --2- 172.21.15.16:0/190799306 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f010806dd60 0x7f0108070220 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f010c006fd0 tx=0x7f010c008040 comp rx=0 tx=0).stop 2024-04-02T04:45:57.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.942+0000 7f01242bc700 1 -- 172.21.15.16:0/190799306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f011c100b00 msgr2=0x7f011c198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:57.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.942+0000 7f01242bc700 1 --2- 172.21.15.16:0/190799306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f011c100b00 0x7f011c198800 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f01180059d0 tx=0x7f011800be20 comp rx=0 tx=0).stop 2024-04-02T04:45:57.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.942+0000 7f01242bc700 1 -- 172.21.15.16:0/190799306 shutdown_connections 2024-04-02T04:45:57.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.942+0000 7f01242bc700 1 --2- 172.21.15.16:0/190799306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f011c100b00 0x7f011c198800 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:57.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.942+0000 7f01242bc700 1 --2- 172.21.15.16:0/190799306 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f010806dd60 0x7f0108070220 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:57.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.942+0000 7f01242bc700 1 --2- 172.21.15.16:0/190799306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f011c101d00 0x7f011c198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:57.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.942+0000 7f01242bc700 1 -- 172.21.15.16:0/190799306 >> 172.21.15.16:0/190799306 conn(0x7f011c0fc0a0 msgr2=0x7f011c073400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:57.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.942+0000 7f01242bc700 1 -- 172.21.15.16:0/190799306 shutdown_connections 2024-04-02T04:45:57.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:57.942+0000 7f01242bc700 1 -- 172.21.15.16:0/190799306 wait complete. 2024-04-02T04:45:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.052+0000 7f2e7d2d3700 1 -- 172.21.15.16:0/384442713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e780ff9d0 msgr2=0x7f2e780ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.052+0000 7f2e7d2d3700 1 --2- 172.21.15.16:0/384442713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e780ff9d0 0x7f2e780ffdf0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2e60009910 tx=0x7f2e60009c20 comp rx=0 tx=0).stop 2024-04-02T04:45:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.052+0000 7f2e7d2d3700 1 -- 172.21.15.16:0/384442713 shutdown_connections 2024-04-02T04:45:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.052+0000 7f2e7d2d3700 1 --2- 172.21.15.16:0/384442713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e78100bd0 0x7f2e78101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:58.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.052+0000 7f2e7d2d3700 1 --2- 172.21.15.16:0/384442713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e780ff9d0 0x7f2e780ffdf0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:58.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.052+0000 7f2e7d2d3700 1 -- 172.21.15.16:0/384442713 >> 172.21.15.16:0/384442713 conn(0x7f2e780faf70 msgr2=0x7f2e780fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:58.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.052+0000 7f2e7d2d3700 1 -- 172.21.15.16:0/384442713 shutdown_connections 2024-04-02T04:45:58.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.052+0000 7f2e7d2d3700 1 -- 172.21.15.16:0/384442713 wait complete. 2024-04-02T04:45:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.053+0000 7f2e7d2d3700 1 Processor -- start 2024-04-02T04:45:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.053+0000 7f2e7d2d3700 1 -- start start 2024-04-02T04:45:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.053+0000 7f2e7d2d3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e780ff9d0 0x7f2e78195440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.053+0000 7f2e7d2d3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e78100bd0 0x7f2e78195980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.053+0000 7f2e7d2d3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e78195fa0 con 0x7f2e780ff9d0 2024-04-02T04:45:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.053+0000 7f2e7d2d3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e781960e0 con 0x7f2e78100bd0 2024-04-02T04:45:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.053+0000 7f2e76ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e780ff9d0 0x7f2e78195440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.053+0000 7f2e76ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e780ff9d0 0x7f2e78195440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56990/0 (socket says 172.21.15.16:56990) 2024-04-02T04:45:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.053+0000 7f2e76ffd700 1 -- 172.21.15.16:0/571858509 learned_addr learned my addr 172.21.15.16:0/571858509 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:45:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.053+0000 7f2e76ffd700 1 -- 172.21.15.16:0/571858509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e78100bd0 msgr2=0x7f2e78195980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.053+0000 7f2e76ffd700 1 --2- 172.21.15.16:0/571858509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e78100bd0 0x7f2e78195980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.053+0000 7f2e76ffd700 1 -- 172.21.15.16:0/571858509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e600095f0 con 0x7f2e780ff9d0 2024-04-02T04:45:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.053+0000 7f2e76ffd700 1 --2- 172.21.15.16:0/571858509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e780ff9d0 0x7f2e78195440 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2e600059d0 tx=0x7f2e6000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.054+0000 7f2e6ffff700 1 -- 172.21.15.16:0/571858509 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e6001f070 con 0x7f2e780ff9d0 2024-04-02T04:45:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.054+0000 7f2e6ffff700 1 -- 172.21.15.16:0/571858509 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2e60005dc0 con 0x7f2e780ff9d0 2024-04-02T04:45:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.054+0000 7f2e7d2d3700 1 -- 172.21.15.16:0/571858509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e78068480 con 0x7f2e780ff9d0 2024-04-02T04:45:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.054+0000 7f2e6ffff700 1 -- 172.21.15.16:0/571858509 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e6000f690 con 0x7f2e780ff9d0 2024-04-02T04:45:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.054+0000 7f2e7d2d3700 1 -- 172.21.15.16:0/571858509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e780689f0 con 0x7f2e780ff9d0 2024-04-02T04:45:58.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.055+0000 7f2e7d2d3700 1 -- 172.21.15.16:0/571858509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e7804ea80 con 0x7f2e780ff9d0 2024-04-02T04:45:58.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.055+0000 7f2e6ffff700 1 -- 172.21.15.16:0/571858509 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f2e600199f0 con 0x7f2e780ff9d0 2024-04-02T04:45:58.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.056+0000 7f2e6ffff700 1 --2- 172.21.15.16:0/571858509 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f2e6406dd60 0x7f2e64070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:45:58.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.056+0000 7f2e6ffff700 1 -- 172.21.15.16:0/571858509 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f2e60090ef0 con 0x7f2e780ff9d0 2024-04-02T04:45:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.056+0000 7f2e767fc700 1 --2- 172.21.15.16:0/571858509 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f2e6406dd60 0x7f2e64070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:45:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.056+0000 7f2e767fc700 1 --2- 172.21.15.16:0/571858509 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f2e6406dd60 0x7f2e64070220 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2e68005950 tx=0x7f2e680058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:45:58.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.060+0000 7f2e6ffff700 1 -- 172.21.15.16:0/571858509 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2e6005e030 con 0x7f2e780ff9d0 2024-04-02T04:45:58.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.357+0000 7f2e7d2d3700 1 -- 172.21.15.16:0/571858509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2e78108530 con 0x7f2e780ff9d0 2024-04-02T04:45:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.357+0000 7f2e6ffff700 1 -- 172.21.15.16:0/571858509 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1517 (secure 0 0 0) 0x7f2e60028600 con 0x7f2e780ff9d0 2024-04-02T04:45:58.361 INFO:teuthology.orchestra.run.smithi016.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Degraded data redundancy: 51/267 objects degraded (19.101%), 19 pgs degraded 2024-04-02T04:45:58.361 INFO:teuthology.orchestra.run.smithi016.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-02T04:45:58.361 INFO:teuthology.orchestra.run.smithi016.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-02T04:45:58.361 INFO:teuthology.orchestra.run.smithi016.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-02T04:45:58.362 INFO:teuthology.orchestra.run.smithi016.stdout: osd.0 (root=default,host=smithi016) is down 2024-04-02T04:45:58.362 INFO:teuthology.orchestra.run.smithi016.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 51/267 objects degraded (19.101%), 19 pgs degraded 2024-04-02T04:45:58.362 INFO:teuthology.orchestra.run.smithi016.stdout: pg 1.0 is active+undersized+degraded, acting [3,1] 2024-04-02T04:45:58.362 INFO:teuthology.orchestra.run.smithi016.stdout: pg 2.0 is active+undersized+degraded, acting [3,1] 2024-04-02T04:45:58.362 INFO:teuthology.orchestra.run.smithi016.stdout: pg 2.1 is active+undersized+degraded, acting [2,1] 2024-04-02T04:45:58.362 INFO:teuthology.orchestra.run.smithi016.stdout: pg 2.2 is active+undersized+degraded, acting [5,1] 2024-04-02T04:45:58.362 INFO:teuthology.orchestra.run.smithi016.stdout: pg 2.4 is active+undersized+degraded, acting [1,4] 2024-04-02T04:45:58.362 INFO:teuthology.orchestra.run.smithi016.stdout: pg 2.5 is active+undersized+degraded, acting [3,4] 2024-04-02T04:45:58.362 INFO:teuthology.orchestra.run.smithi016.stdout: pg 2.8 is active+undersized+degraded, acting [3,5] 2024-04-02T04:45:58.362 INFO:teuthology.orchestra.run.smithi016.stdout: pg 2.9 is active+undersized+degraded, acting [1,4] 2024-04-02T04:45:58.362 INFO:teuthology.orchestra.run.smithi016.stdout: pg 2.c is active+undersized+degraded, acting [2,3] 2024-04-02T04:45:58.362 INFO:teuthology.orchestra.run.smithi016.stdout: pg 2.e is active+undersized+degraded, acting [2,3] 2024-04-02T04:45:58.363 INFO:teuthology.orchestra.run.smithi016.stdout: pg 2.f is active+undersized+degraded, acting [4,5] 2024-04-02T04:45:58.363 INFO:teuthology.orchestra.run.smithi016.stdout: pg 2.10 is active+undersized+degraded, acting [2,1] 2024-04-02T04:45:58.363 INFO:teuthology.orchestra.run.smithi016.stdout: pg 2.13 is active+undersized+degraded, acting [4,2] 2024-04-02T04:45:58.363 INFO:teuthology.orchestra.run.smithi016.stdout: pg 2.15 is active+undersized+degraded, acting [1,3] 2024-04-02T04:45:58.363 INFO:teuthology.orchestra.run.smithi016.stdout: pg 2.19 is active+undersized+degraded, acting [4,2] 2024-04-02T04:45:58.363 INFO:teuthology.orchestra.run.smithi016.stdout: pg 2.1b is active+undersized+degraded, acting [1,5] 2024-04-02T04:45:58.363 INFO:teuthology.orchestra.run.smithi016.stdout: pg 2.1d is active+undersized+degraded, acting [3,5] 2024-04-02T04:45:58.363 INFO:teuthology.orchestra.run.smithi016.stdout: pg 2.1e is active+undersized+degraded, acting [2,5] 2024-04-02T04:45:58.363 INFO:teuthology.orchestra.run.smithi016.stdout: pg 2.1f is active+undersized+degraded, acting [3,4] 2024-04-02T04:45:58.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.360+0000 7f2e7d2d3700 1 -- 172.21.15.16:0/571858509 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f2e6406dd60 msgr2=0x7f2e64070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:58.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.360+0000 7f2e7d2d3700 1 --2- 172.21.15.16:0/571858509 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f2e6406dd60 0x7f2e64070220 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2e68005950 tx=0x7f2e680058e0 comp rx=0 tx=0).stop 2024-04-02T04:45:58.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.360+0000 7f2e7d2d3700 1 -- 172.21.15.16:0/571858509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e780ff9d0 msgr2=0x7f2e78195440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:45:58.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.360+0000 7f2e7d2d3700 1 --2- 172.21.15.16:0/571858509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e780ff9d0 0x7f2e78195440 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2e600059d0 tx=0x7f2e6000be20 comp rx=0 tx=0).stop 2024-04-02T04:45:58.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.360+0000 7f2e7d2d3700 1 -- 172.21.15.16:0/571858509 shutdown_connections 2024-04-02T04:45:58.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.360+0000 7f2e7d2d3700 1 --2- 172.21.15.16:0/571858509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e780ff9d0 0x7f2e78195440 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:58.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.360+0000 7f2e7d2d3700 1 --2- 172.21.15.16:0/571858509 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f2e6406dd60 0x7f2e64070220 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:58.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.360+0000 7f2e7d2d3700 1 --2- 172.21.15.16:0/571858509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e78100bd0 0x7f2e78195980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:45:58.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.360+0000 7f2e7d2d3700 1 -- 172.21.15.16:0/571858509 >> 172.21.15.16:0/571858509 conn(0x7f2e780faf70 msgr2=0x7f2e78104220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:45:58.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.360+0000 7f2e7d2d3700 1 -- 172.21.15.16:0/571858509 shutdown_connections 2024-04-02T04:45:58.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:45:58.360+0000 7f2e7d2d3700 1 -- 172.21.15.16:0/571858509 wait complete. 2024-04-02T04:45:58.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:58 smithi016 ceph-mon[93230]: from='client.34222 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:45:58.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:58 smithi016 ceph-mon[93230]: from='client.34226 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:45:58.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:58 smithi016 ceph-mon[93230]: pgmap v55: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 51/267 objects degraded (19.101%) 2024-04-02T04:45:58.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:58 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/2916069932' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:45:58.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:58 smithi027 ceph-mon[83036]: from='client.34222 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:45:58.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:58 smithi027 ceph-mon[83036]: from='client.34226 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:45:58.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:58 smithi027 ceph-mon[83036]: pgmap v55: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 51/267 objects degraded (19.101%) 2024-04-02T04:45:58.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:58 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/2916069932' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:45:59.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:59 smithi016 ceph-mon[93230]: from='client.34238 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:45:59.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:45:59 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/571858509' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:45:59.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:59 smithi027 ceph-mon[83036]: from='client.34238 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:45:59.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:45:59 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/571858509' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:46:00.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:00 smithi016 ceph-mon[93230]: pgmap v56: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 51/267 objects degraded (19.101%) 2024-04-02T04:46:00.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:00 smithi027 ceph-mon[83036]: pgmap v56: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 51/267 objects degraded (19.101%) 2024-04-02T04:46:01.265 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:46:01 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0[102237]: 2024-04-02T04:46:01.214+0000 7f839ed207c0 -1 osd.0 49 log_to_monitors true 2024-04-02T04:46:01.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:01 smithi016 ceph-mon[93230]: from='osd.0 [v2:172.21.15.16:6802/1136760904,v1:172.21.15.16:6803/1136760904]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-02T04:46:01.560 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:46:01 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0[102237]: 2024-04-02T04:46:01.281+0000 7f8390178700 -1 osd.0 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T04:46:01.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:01 smithi027 ceph-mon[83036]: from='osd.0 [v2:172.21.15.16:6802/1136760904,v1:172.21.15.16:6803/1136760904]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-02T04:46:02.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:02 smithi016 ceph-mon[93230]: pgmap v57: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 51/267 objects degraded (19.101%) 2024-04-02T04:46:02.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:02 smithi016 ceph-mon[93230]: from='osd.0 [v2:172.21.15.16:6802/1136760904,v1:172.21.15.16:6803/1136760904]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-02T04:46:02.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:02 smithi016 ceph-mon[93230]: failed to encode map e52 with expected crc 2024-04-02T04:46:02.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:02 smithi016 ceph-mon[93230]: osdmap e52: 6 total, 5 up, 6 in 2024-04-02T04:46:02.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:02 smithi016 ceph-mon[93230]: from='osd.0 [v2:172.21.15.16:6802/1136760904,v1:172.21.15.16:6803/1136760904]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-04-02T04:46:02.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:02 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:02.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:02 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:02.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:02 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:02.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:02 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:46:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:02 smithi027 ceph-mon[83036]: pgmap v57: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 260 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 51/267 objects degraded (19.101%) 2024-04-02T04:46:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:02 smithi027 ceph-mon[83036]: from='osd.0 [v2:172.21.15.16:6802/1136760904,v1:172.21.15.16:6803/1136760904]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-02T04:46:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:02 smithi027 ceph-mon[83036]: failed to encode map e52 with expected crc 2024-04-02T04:46:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:02 smithi027 ceph-mon[83036]: osdmap e52: 6 total, 5 up, 6 in 2024-04-02T04:46:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:02 smithi027 ceph-mon[83036]: from='osd.0 [v2:172.21.15.16:6802/1136760904,v1:172.21.15.16:6803/1136760904]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-04-02T04:46:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:02 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:02.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:02 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:02.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:02 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:02.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:02 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:46:03.550 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:03 smithi016 ceph-mon[93230]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-02T04:46:03.550 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:03 smithi016 ceph-mon[93230]: failed to encode map e52 with expected crc 2024-04-02T04:46:03.550 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:03 smithi016 ceph-mon[93230]: osd.0 [v2:172.21.15.16:6802/1136760904,v1:172.21.15.16:6803/1136760904] boot 2024-04-02T04:46:03.550 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:03 smithi016 ceph-mon[93230]: osdmap e53: 6 total, 6 up, 6 in 2024-04-02T04:46:03.550 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:03 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T04:46:03.550 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:03 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:03.550 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:03 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:03.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:03 smithi027 ceph-mon[83036]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-02T04:46:03.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:03 smithi027 ceph-mon[83036]: failed to encode map e52 with expected crc 2024-04-02T04:46:03.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:03 smithi027 ceph-mon[83036]: osd.0 [v2:172.21.15.16:6802/1136760904,v1:172.21.15.16:6803/1136760904] boot 2024-04-02T04:46:03.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:03 smithi027 ceph-mon[83036]: osdmap e53: 6 total, 6 up, 6 in 2024-04-02T04:46:03.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:03 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T04:46:03.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:03 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:03.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:03 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:04.435 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:04 smithi016 ceph-mon[93230]: pgmap v60: 65 pgs: 25 peering, 3 active+undersized+degraded, 6 active+undersized, 31 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 8/267 objects degraded (2.996%) 2024-04-02T04:46:04.435 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:04 smithi016 ceph-mon[93230]: failed to encode map e54 with expected crc 2024-04-02T04:46:04.436 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:04 smithi016 ceph-mon[93230]: osdmap e54: 6 total, 6 up, 6 in 2024-04-02T04:46:04.436 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:04 smithi016 ceph-mon[93230]: failed to encode map e54 with expected crc 2024-04-02T04:46:04.436 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:04 smithi016 ceph-mon[93230]: failed to encode map e54 with expected crc 2024-04-02T04:46:04.436 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:04 smithi016 ceph-mon[93230]: failed to encode map e54 with expected crc 2024-04-02T04:46:04.436 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:04 smithi016 ceph-mon[93230]: failed to encode map e54 with expected crc 2024-04-02T04:46:04.436 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:04 smithi016 ceph-mon[93230]: failed to encode map e54 with expected crc 2024-04-02T04:46:04.436 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:04 smithi016 ceph-mon[93230]: failed to encode map e54 with expected crc 2024-04-02T04:46:04.437 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:04 smithi016 ceph-mon[93230]: failed to encode map e54 with expected crc 2024-04-02T04:46:04.437 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:04 smithi016 ceph-mon[93230]: Health check update: Degraded data redundancy: 8/267 objects degraded (2.996%), 3 pgs degraded (PG_DEGRADED) 2024-04-02T04:46:04.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:04 smithi027 ceph-mon[83036]: pgmap v60: 65 pgs: 25 peering, 3 active+undersized+degraded, 6 active+undersized, 31 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 8/267 objects degraded (2.996%) 2024-04-02T04:46:04.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:04 smithi027 ceph-mon[83036]: failed to encode map e54 with expected crc 2024-04-02T04:46:04.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:04 smithi027 ceph-mon[83036]: osdmap e54: 6 total, 6 up, 6 in 2024-04-02T04:46:04.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:04 smithi027 ceph-mon[83036]: failed to encode map e54 with expected crc 2024-04-02T04:46:04.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:04 smithi027 ceph-mon[83036]: failed to encode map e54 with expected crc 2024-04-02T04:46:04.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:04 smithi027 ceph-mon[83036]: failed to encode map e54 with expected crc 2024-04-02T04:46:04.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:04 smithi027 ceph-mon[83036]: failed to encode map e54 with expected crc 2024-04-02T04:46:04.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:04 smithi027 ceph-mon[83036]: failed to encode map e54 with expected crc 2024-04-02T04:46:04.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:04 smithi027 ceph-mon[83036]: failed to encode map e54 with expected crc 2024-04-02T04:46:04.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:04 smithi027 ceph-mon[83036]: failed to encode map e54 with expected crc 2024-04-02T04:46:04.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:04 smithi027 ceph-mon[83036]: Health check update: Degraded data redundancy: 8/267 objects degraded (2.996%), 3 pgs degraded (PG_DEGRADED) 2024-04-02T04:46:06.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:06 smithi016 ceph-mon[93230]: pgmap v62: 65 pgs: 25 peering, 3 active+undersized+degraded, 6 active+undersized, 31 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 8/267 objects degraded (2.996%) 2024-04-02T04:46:06.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:06.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:06.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:46:06.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:46:06.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:06.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:46:06.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:46:06.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:46:06.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:46:06.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-02T04:46:06.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:06 smithi027 ceph-mon[83036]: pgmap v62: 65 pgs: 25 peering, 3 active+undersized+degraded, 6 active+undersized, 31 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 8/267 objects degraded (2.996%) 2024-04-02T04:46:06.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:06.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:06.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:46:06.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:46:06.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:06.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:46:06.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:46:06.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:46:06.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:46:06.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-02T04:46:07.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:07 smithi016 ceph-mon[93230]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-02T04:46:07.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:07 smithi016 ceph-mon[93230]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-04-02T04:46:07.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:07 smithi027 ceph-mon[83036]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-02T04:46:07.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:07 smithi027 ceph-mon[83036]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-04-02T04:46:08.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:08 smithi016 ceph-mon[93230]: pgmap v63: 65 pgs: 25 peering, 40 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:08.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:08 smithi016 ceph-mon[93230]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 8/267 objects degraded (2.996%), 3 pgs degraded) 2024-04-02T04:46:08.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:08 smithi027 ceph-mon[83036]: pgmap v63: 65 pgs: 25 peering, 40 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:08.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:08 smithi027 ceph-mon[83036]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 8/267 objects degraded (2.996%), 3 pgs degraded) 2024-04-02T04:46:10.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:10 smithi016 ceph-mon[93230]: pgmap v64: 65 pgs: 65 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:10.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:10 smithi027 ceph-mon[83036]: pgmap v64: 65 pgs: 65 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:12.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:12 smithi016 ceph-mon[93230]: pgmap v65: 65 pgs: 65 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:12.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:12 smithi027 ceph-mon[83036]: pgmap v65: 65 pgs: 65 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:14.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:14 smithi016 ceph-mon[93230]: pgmap v66: 65 pgs: 65 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:14.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:14 smithi027 ceph-mon[83036]: pgmap v66: 65 pgs: 65 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:16.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:16 smithi016 ceph-mon[93230]: pgmap v67: 65 pgs: 65 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:16.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:16 smithi027 ceph-mon[83036]: pgmap v67: 65 pgs: 65 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:18.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:18 smithi027 ceph-mon[83036]: pgmap v68: 65 pgs: 65 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:18.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:18 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:18.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:18 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:46:18.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:18 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:18.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:18 smithi016 ceph-mon[93230]: pgmap v68: 65 pgs: 65 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:18.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:18 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:18.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:18 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:46:18.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:18 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:20.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:20 smithi027 ceph-mon[83036]: pgmap v69: 65 pgs: 65 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:20.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:20 smithi016 ceph-mon[93230]: pgmap v69: 65 pgs: 65 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:22.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:22 smithi027 ceph-mon[83036]: pgmap v70: 65 pgs: 65 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:22.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:22 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-02T04:46:22.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:22 smithi027 ceph-mon[83036]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-02T04:46:22.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:22 smithi027 ceph-mon[83036]: Upgrade: osd.1 is safe to restart 2024-04-02T04:46:22.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:22 smithi016 ceph-mon[93230]: pgmap v70: 65 pgs: 65 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:22.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:22 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-02T04:46:22.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:22 smithi016 ceph-mon[93230]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-02T04:46:22.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:22 smithi016 ceph-mon[93230]: Upgrade: osd.1 is safe to restart 2024-04-02T04:46:24.058 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:23 smithi016 ceph-mon[93230]: Upgrade: Updating osd.1 2024-04-02T04:46:24.058 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:23 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:24.058 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:23 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-02T04:46:24.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:23 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:46:24.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:23 smithi016 ceph-mon[93230]: Deploying daemon osd.1 on smithi016 2024-04-02T04:46:24.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:23 smithi016 ceph-mon[93230]: pgmap v71: 65 pgs: 65 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:24.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:23 smithi027 ceph-mon[83036]: Upgrade: Updating osd.1 2024-04-02T04:46:24.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:23 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:24.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:23 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-02T04:46:24.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:23 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:46:24.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:23 smithi027 ceph-mon[83036]: Deploying daemon osd.1 on smithi016 2024-04-02T04:46:24.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:23 smithi027 ceph-mon[83036]: pgmap v71: 65 pgs: 65 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:26.279 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:26 smithi016 ceph-mon[93230]: pgmap v72: 65 pgs: 65 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:26.280 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:26 smithi016 systemd[1]: Stopping Ceph osd.1 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4... 2024-04-02T04:46:26.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:26 smithi027 ceph-mon[83036]: pgmap v72: 65 pgs: 65 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:26.559 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:26 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1[55673]: 2024-04-02T04:46:26.274+0000 7fdef71bd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T04:46:26.560 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:26 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1[55673]: 2024-04-02T04:46:26.274+0000 7fdef71bd700 -1 osd.1 54 *** Got signal Terminated *** 2024-04-02T04:46:26.560 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:26 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1[55673]: 2024-04-02T04:46:26.274+0000 7fdef71bd700 -1 osd.1 54 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T04:46:27.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:27 smithi027 ceph-mon[83036]: osd.1 marked itself down and dead 2024-04-02T04:46:27.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:27 smithi016 ceph-mon[93230]: osd.1 marked itself down and dead 2024-04-02T04:46:27.939 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:27 smithi016 podman[107620]: 2024-04-02 04:46:27.654760524 +0000 UTC m=+1.491020609 container died 7361abc9e1b64de5caab0c5c2ae78eacb4051bc88ef61f9cd8f06f5d23616993 (image=quay.ceph.io/ceph-ci/ceph@sha256:397d0c1d908c002c558906e3ea05982bc336d403ee4d900400cc3bdd9243aa2e, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=reef-4d05f4c, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326) 2024-04-02T04:46:28.222 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:28 smithi016 podman[107620]: 2024-04-02 04:46:28.026126348 +0000 UTC m=+1.862386419 container cleanup 7361abc9e1b64de5caab0c5c2ae78eacb4051bc88ef61f9cd8f06f5d23616993 (image=quay.ceph.io/ceph-ci/ceph@sha256:397d0c1d908c002c558906e3ea05982bc336d403ee4d900400cc3bdd9243aa2e, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=reef-4d05f4c, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240326) 2024-04-02T04:46:28.222 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:28 smithi016 bash[107620]: ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1 2024-04-02T04:46:28.223 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:28 smithi016 ceph-mon[93230]: pgmap v73: 65 pgs: 65 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:28.223 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:28 smithi016 ceph-mon[93230]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-02T04:46:28.223 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:28 smithi016 ceph-mon[93230]: failed to encode map e55 with expected crc 2024-04-02T04:46:28.223 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:28 smithi016 ceph-mon[93230]: failed to encode map e55 with expected crc 2024-04-02T04:46:28.223 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:28 smithi016 ceph-mon[93230]: failed to encode map e55 with expected crc 2024-04-02T04:46:28.223 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:28 smithi016 ceph-mon[93230]: osdmap e55: 6 total, 5 up, 6 in 2024-04-02T04:46:28.223 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:28 smithi016 ceph-mon[93230]: failed to encode map e55 with expected crc 2024-04-02T04:46:28.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:28 smithi027 ceph-mon[83036]: pgmap v73: 65 pgs: 65 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:28.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:28 smithi027 ceph-mon[83036]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-02T04:46:28.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:28 smithi027 ceph-mon[83036]: failed to encode map e55 with expected crc 2024-04-02T04:46:28.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:28 smithi027 ceph-mon[83036]: failed to encode map e55 with expected crc 2024-04-02T04:46:28.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:28 smithi027 ceph-mon[83036]: failed to encode map e55 with expected crc 2024-04-02T04:46:28.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:28 smithi027 ceph-mon[83036]: osdmap e55: 6 total, 5 up, 6 in 2024-04-02T04:46:28.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:28 smithi027 ceph-mon[83036]: failed to encode map e55 with expected crc 2024-04-02T04:46:28.503 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:28 smithi016 podman[107639]: 2024-04-02 04:46:28.219982217 +0000 UTC m=+0.556269568 container remove 7361abc9e1b64de5caab0c5c2ae78eacb4051bc88ef61f9cd8f06f5d23616993 (image=quay.ceph.io/ceph-ci/ceph@sha256:397d0c1d908c002c558906e3ea05982bc336d403ee4d900400cc3bdd9243aa2e, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, RELEASE=reef-4d05f4c) 2024-04-02T04:46:28.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.497+0000 7f03a6818700 1 -- 172.21.15.16:0/736884369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03a0100bd0 msgr2=0x7f03a0101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:28.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.497+0000 7f03a6818700 1 --2- 172.21.15.16:0/736884369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03a0100bd0 0x7f03a0101030 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0394009910 tx=0x7f0394009c20 comp rx=0 tx=0).stop 2024-04-02T04:46:28.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.498+0000 7f03a6818700 1 -- 172.21.15.16:0/736884369 shutdown_connections 2024-04-02T04:46:28.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.498+0000 7f03a6818700 1 --2- 172.21.15.16:0/736884369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03a0100bd0 0x7f03a0101030 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:28.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.498+0000 7f03a6818700 1 --2- 172.21.15.16:0/736884369 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03a00ff9d0 0x7f03a00ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:28.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.498+0000 7f03a6818700 1 -- 172.21.15.16:0/736884369 >> 172.21.15.16:0/736884369 conn(0x7f03a00faf70 msgr2=0x7f03a00fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:46:28.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.498+0000 7f03a6818700 1 -- 172.21.15.16:0/736884369 shutdown_connections 2024-04-02T04:46:28.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.498+0000 7f03a6818700 1 -- 172.21.15.16:0/736884369 wait complete. 2024-04-02T04:46:28.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.498+0000 7f03a6818700 1 Processor -- start 2024-04-02T04:46:28.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.498+0000 7f03a6818700 1 -- start start 2024-04-02T04:46:28.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.499+0000 7f03a6818700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03a00ff9d0 0x7f03a01964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:46:28.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.499+0000 7f03a6818700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03a0100bd0 0x7f03a01969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:46:28.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.499+0000 7f03a6818700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a0197010 con 0x7f03a0100bd0 2024-04-02T04:46:28.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.499+0000 7f03a6818700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a0197150 con 0x7f03a00ff9d0 2024-04-02T04:46:28.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.499+0000 7f039f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03a0100bd0 0x7f03a01969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:46:28.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.499+0000 7f039f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03a0100bd0 0x7f03a01969f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59614/0 (socket says 172.21.15.16:59614) 2024-04-02T04:46:28.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.499+0000 7f039f7fe700 1 -- 172.21.15.16:0/3518362956 learned_addr learned my addr 172.21.15.16:0/3518362956 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:46:28.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.499+0000 7f039ffff700 1 --2- 172.21.15.16:0/3518362956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03a00ff9d0 0x7f03a01964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:46:28.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.499+0000 7f039f7fe700 1 -- 172.21.15.16:0/3518362956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03a00ff9d0 msgr2=0x7f03a01964b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:28.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.499+0000 7f039f7fe700 1 --2- 172.21.15.16:0/3518362956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03a00ff9d0 0x7f03a01964b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:28.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.499+0000 7f039f7fe700 1 -- 172.21.15.16:0/3518362956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03940095f0 con 0x7f03a0100bd0 2024-04-02T04:46:28.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.500+0000 7f039f7fe700 1 --2- 172.21.15.16:0/3518362956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03a0100bd0 0x7f03a01969f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f03940092e0 tx=0x7f039400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:46:28.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.500+0000 7f039d7fa700 1 -- 172.21.15.16:0/3518362956 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f039401d070 con 0x7f03a0100bd0 2024-04-02T04:46:28.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.500+0000 7f03a6818700 1 -- 172.21.15.16:0/3518362956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a019bb50 con 0x7f03a0100bd0 2024-04-02T04:46:28.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.500+0000 7f039d7fa700 1 -- 172.21.15.16:0/3518362956 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f039400f460 con 0x7f03a0100bd0 2024-04-02T04:46:28.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.500+0000 7f03a6818700 1 -- 172.21.15.16:0/3518362956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a019c0c0 con 0x7f03a0100bd0 2024-04-02T04:46:28.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.500+0000 7f039d7fa700 1 -- 172.21.15.16:0/3518362956 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0394021640 con 0x7f03a0100bd0 2024-04-02T04:46:28.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.501+0000 7f03a6818700 1 -- 172.21.15.16:0/3518362956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03a004ea80 con 0x7f03a0100bd0 2024-04-02T04:46:28.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.502+0000 7f039d7fa700 1 -- 172.21.15.16:0/3518362956 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f039400fa90 con 0x7f03a0100bd0 2024-04-02T04:46:28.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.502+0000 7f039d7fa700 1 --2- 172.21.15.16:0/3518362956 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f038806ddb0 0x7f0388070270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:46:28.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.502+0000 7f039d7fa700 1 -- 172.21.15.16:0/3518362956 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f039408ec90 con 0x7f03a0100bd0 2024-04-02T04:46:28.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.502+0000 7f039ffff700 1 --2- 172.21.15.16:0/3518362956 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f038806ddb0 0x7f0388070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:46:28.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.503+0000 7f039ffff700 1 --2- 172.21.15.16:0/3518362956 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f038806ddb0 0x7f0388070270 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f03a0100a30 tx=0x7f039000b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:46:28.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.507+0000 7f039d7fa700 1 -- 172.21.15.16:0/3518362956 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f039405bd60 con 0x7f03a0100bd0 2024-04-02T04:46:28.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.744+0000 7f03a6818700 1 -- 172.21.15.16:0/3518362956 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f03a0069420 con 0x7f038806ddb0 2024-04-02T04:46:28.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.745+0000 7f039d7fa700 1 -- 172.21.15.16:0/3518362956 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f03a0069420 con 0x7f038806ddb0 2024-04-02T04:46:28.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.747+0000 7f03a6818700 1 -- 172.21.15.16:0/3518362956 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f038806ddb0 msgr2=0x7f0388070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:28.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.747+0000 7f03a6818700 1 --2- 172.21.15.16:0/3518362956 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f038806ddb0 0x7f0388070270 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f03a0100a30 tx=0x7f039000b410 comp rx=0 tx=0).stop 2024-04-02T04:46:28.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.747+0000 7f03a6818700 1 -- 172.21.15.16:0/3518362956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03a0100bd0 msgr2=0x7f03a01969f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:28.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.747+0000 7f03a6818700 1 --2- 172.21.15.16:0/3518362956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03a0100bd0 0x7f03a01969f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f03940092e0 tx=0x7f039400bd70 comp rx=0 tx=0).stop 2024-04-02T04:46:28.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.747+0000 7f03a6818700 1 -- 172.21.15.16:0/3518362956 shutdown_connections 2024-04-02T04:46:28.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.747+0000 7f03a6818700 1 --2- 172.21.15.16:0/3518362956 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f038806ddb0 0x7f0388070270 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:28.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.747+0000 7f03a6818700 1 --2- 172.21.15.16:0/3518362956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03a00ff9d0 0x7f03a01964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:28.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.747+0000 7f03a6818700 1 --2- 172.21.15.16:0/3518362956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03a0100bd0 0x7f03a01969f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:28.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.747+0000 7f03a6818700 1 -- 172.21.15.16:0/3518362956 >> 172.21.15.16:0/3518362956 conn(0x7f03a00faf70 msgr2=0x7f03a0068d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:46:28.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.747+0000 7f03a6818700 1 -- 172.21.15.16:0/3518362956 shutdown_connections 2024-04-02T04:46:28.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.747+0000 7f03a6818700 1 -- 172.21.15.16:0/3518362956 wait complete. 2024-04-02T04:46:28.762 INFO:teuthology.orchestra.run.smithi016.stdout:true 2024-04-02T04:46:28.762 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:28 smithi016 podman[107752]: 2024-04-02 04:46:28.582313111 +0000 UTC m=+0.018573147 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:46:28.762 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:28 smithi016 podman[107752]: 2024-04-02 04:46:28.687485207 +0000 UTC m=+0.123745217 container create 04eb71e5814d32e6ddb641c5557fa3dc9304f4b60b94b6e850b5b7a924baf714 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-6cec793, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T04:46:28.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.857+0000 7f9bba9b2700 1 -- 172.21.15.16:0/2855043698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bb4103f40 msgr2=0x7f9bb4106330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:28.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.857+0000 7f9bba9b2700 1 --2- 172.21.15.16:0/2855043698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bb4103f40 0x7f9bb4106330 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4009910 tx=0x7f9ba4009c20 comp rx=0 tx=0).stop 2024-04-02T04:46:28.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.857+0000 7f9bba9b2700 1 -- 172.21.15.16:0/2855043698 shutdown_connections 2024-04-02T04:46:28.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.857+0000 7f9bba9b2700 1 --2- 172.21.15.16:0/2855043698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bb4103f40 0x7f9bb4106330 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:28.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.857+0000 7f9bba9b2700 1 --2- 172.21.15.16:0/2855043698 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bb4101610 0x7f9bb4103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:28.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.857+0000 7f9bba9b2700 1 -- 172.21.15.16:0/2855043698 >> 172.21.15.16:0/2855043698 conn(0x7f9bb40fafc0 msgr2=0x7f9bb40fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:46:28.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.857+0000 7f9bba9b2700 1 -- 172.21.15.16:0/2855043698 shutdown_connections 2024-04-02T04:46:28.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.857+0000 7f9bba9b2700 1 -- 172.21.15.16:0/2855043698 wait complete. 2024-04-02T04:46:28.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.857+0000 7f9bba9b2700 1 Processor -- start 2024-04-02T04:46:28.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.857+0000 7f9bba9b2700 1 -- start start 2024-04-02T04:46:28.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.858+0000 7f9bba9b2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bb4101610 0x7f9bb4100f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:46:28.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.858+0000 7f9bba9b2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bb4103f40 0x7f9bb40ff5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:46:28.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.858+0000 7f9bba9b2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb40ffb00 con 0x7f9bb4101610 2024-04-02T04:46:28.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.858+0000 7f9bba9b2700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb40ffc40 con 0x7f9bb4103f40 2024-04-02T04:46:28.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.858+0000 7f9bb3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bb4101610 0x7f9bb4100f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:46:28.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.858+0000 7f9bb3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bb4101610 0x7f9bb4100f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59626/0 (socket says 172.21.15.16:59626) 2024-04-02T04:46:28.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.858+0000 7f9bb3fff700 1 -- 172.21.15.16:0/40698868 learned_addr learned my addr 172.21.15.16:0/40698868 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:46:28.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.858+0000 7f9bb37fe700 1 --2- 172.21.15.16:0/40698868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bb4103f40 0x7f9bb40ff5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:46:28.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.858+0000 7f9bb3fff700 1 -- 172.21.15.16:0/40698868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bb4103f40 msgr2=0x7f9bb40ff5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:28.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.858+0000 7f9bb3fff700 1 --2- 172.21.15.16:0/40698868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bb4103f40 0x7f9bb40ff5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:28.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.858+0000 7f9bb3fff700 1 -- 172.21.15.16:0/40698868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba40095f0 con 0x7f9bb4101610 2024-04-02T04:46:28.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.858+0000 7f9bb3fff700 1 --2- 172.21.15.16:0/40698868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bb4101610 0x7f9bb4100f20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c00e920 tx=0x7f9b9c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:46:28.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.859+0000 7f9bb17fa700 1 -- 172.21.15.16:0/40698868 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b9c00c9d0 con 0x7f9bb4101610 2024-04-02T04:46:28.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.859+0000 7f9bb17fa700 1 -- 172.21.15.16:0/40698868 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b9c010430 con 0x7f9bb4101610 2024-04-02T04:46:28.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.859+0000 7f9bba9b2700 1 -- 172.21.15.16:0/40698868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bb40fff20 con 0x7f9bb4101610 2024-04-02T04:46:28.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.859+0000 7f9bb17fa700 1 -- 172.21.15.16:0/40698868 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b9c018630 con 0x7f9bb4101610 2024-04-02T04:46:28.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.859+0000 7f9bba9b2700 1 -- 172.21.15.16:0/40698868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bb419e500 con 0x7f9bb4101610 2024-04-02T04:46:28.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.860+0000 7f9bba9b2700 1 -- 172.21.15.16:0/40698868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bb404ea80 con 0x7f9bb4101610 2024-04-02T04:46:28.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.860+0000 7f9bb17fa700 1 -- 172.21.15.16:0/40698868 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f9b9c0105a0 con 0x7f9bb4101610 2024-04-02T04:46:28.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.861+0000 7f9bb17fa700 1 --2- 172.21.15.16:0/40698868 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f9ba006dd60 0x7f9ba0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:46:28.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.861+0000 7f9bb17fa700 1 -- 172.21.15.16:0/40698868 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f9b9c014070 con 0x7f9bb4101610 2024-04-02T04:46:28.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.861+0000 7f9bb37fe700 1 --2- 172.21.15.16:0/40698868 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f9ba006dd60 0x7f9ba0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:46:28.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.861+0000 7f9bb37fe700 1 --2- 172.21.15.16:0/40698868 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f9ba006dd60 0x7f9ba0070220 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9ba400b800 tx=0x7f9ba4009f70 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:46:28.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:28.865+0000 7f9bb17fa700 1 -- 172.21.15.16:0/40698868 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9b9c053280 con 0x7f9bb4101610 2024-04-02T04:46:29.060 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:28 smithi016 podman[107752]: 2024-04-02 04:46:28.907018391 +0000 UTC m=+0.343278422 container init 04eb71e5814d32e6ddb641c5557fa3dc9304f4b60b94b6e850b5b7a924baf714 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-6cec793, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS) 2024-04-02T04:46:29.060 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:28 smithi016 podman[107752]: 2024-04-02 04:46:28.910262605 +0000 UTC m=+0.346522617 container start 04eb71e5814d32e6ddb641c5557fa3dc9304f4b60b94b6e850b5b7a924baf714 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, RELEASE=squid-6cec793, ceph=True, CEPH_POINT_RELEASE=) 2024-04-02T04:46:29.060 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:29 smithi016 podman[107752]: 2024-04-02 04:46:29.006930692 +0000 UTC m=+0.443190709 container attach 04eb71e5814d32e6ddb641c5557fa3dc9304f4b60b94b6e850b5b7a924baf714 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, RELEASE=squid-6cec793, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-02T04:46:29.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.093+0000 7f9bba9b2700 1 -- 172.21.15.16:0/40698868 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9bb4105e20 con 0x7f9ba006dd60 2024-04-02T04:46:29.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.094+0000 7f9bb17fa700 1 -- 172.21.15.16:0/40698868 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f9bb4105e20 con 0x7f9ba006dd60 2024-04-02T04:46:29.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.096+0000 7f9bba9b2700 1 -- 172.21.15.16:0/40698868 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f9ba006dd60 msgr2=0x7f9ba0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:29.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.096+0000 7f9bba9b2700 1 --2- 172.21.15.16:0/40698868 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f9ba006dd60 0x7f9ba0070220 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9ba400b800 tx=0x7f9ba4009f70 comp rx=0 tx=0).stop 2024-04-02T04:46:29.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.096+0000 7f9bba9b2700 1 -- 172.21.15.16:0/40698868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bb4101610 msgr2=0x7f9bb4100f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:29.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.096+0000 7f9bba9b2700 1 --2- 172.21.15.16:0/40698868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bb4101610 0x7f9bb4100f20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c00e920 tx=0x7f9b9c00ec30 comp rx=0 tx=0).stop 2024-04-02T04:46:29.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.096+0000 7f9bba9b2700 1 -- 172.21.15.16:0/40698868 shutdown_connections 2024-04-02T04:46:29.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.096+0000 7f9bba9b2700 1 --2- 172.21.15.16:0/40698868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bb4101610 0x7f9bb4100f20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:29.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.096+0000 7f9bba9b2700 1 --2- 172.21.15.16:0/40698868 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f9ba006dd60 0x7f9ba0070220 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:29.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.096+0000 7f9bba9b2700 1 --2- 172.21.15.16:0/40698868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bb4103f40 0x7f9bb40ff5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:29.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.096+0000 7f9bba9b2700 1 -- 172.21.15.16:0/40698868 >> 172.21.15.16:0/40698868 conn(0x7f9bb40fafc0 msgr2=0x7f9bb40fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:46:29.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.096+0000 7f9bba9b2700 1 -- 172.21.15.16:0/40698868 shutdown_connections 2024-04-02T04:46:29.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.096+0000 7f9bba9b2700 1 -- 172.21.15.16:0/40698868 wait complete. 2024-04-02T04:46:29.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.206+0000 7f69c21de700 1 -- 172.21.15.16:0/1022634477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69bc103ed0 msgr2=0x7f69bc1062c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:29.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.206+0000 7f69c21de700 1 --2- 172.21.15.16:0/1022634477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69bc103ed0 0x7f69bc1062c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f69ac009910 tx=0x7f69ac009c20 comp rx=0 tx=0).stop 2024-04-02T04:46:29.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.206+0000 7f69c21de700 1 -- 172.21.15.16:0/1022634477 shutdown_connections 2024-04-02T04:46:29.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.206+0000 7f69c21de700 1 --2- 172.21.15.16:0/1022634477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69bc103ed0 0x7f69bc1062c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:29.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.206+0000 7f69c21de700 1 --2- 172.21.15.16:0/1022634477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69bc1015a0 0x7f69bc103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:29.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.206+0000 7f69c21de700 1 -- 172.21.15.16:0/1022634477 >> 172.21.15.16:0/1022634477 conn(0x7f69bc0fafb0 msgr2=0x7f69bc0fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:46:29.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.206+0000 7f69c21de700 1 -- 172.21.15.16:0/1022634477 shutdown_connections 2024-04-02T04:46:29.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.206+0000 7f69c21de700 1 -- 172.21.15.16:0/1022634477 wait complete. 2024-04-02T04:46:29.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.207+0000 7f69c21de700 1 Processor -- start 2024-04-02T04:46:29.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.207+0000 7f69c21de700 1 -- start start 2024-04-02T04:46:29.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.207+0000 7f69c21de700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69bc1015a0 0x7f69bc100eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:46:29.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.207+0000 7f69c21de700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69bc103ed0 0x7f69bc0ff550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:46:29.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.207+0000 7f69c21de700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69bc101440 con 0x7f69bc1015a0 2024-04-02T04:46:29.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.207+0000 7f69c21de700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69bc0ffa90 con 0x7f69bc103ed0 2024-04-02T04:46:29.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.207+0000 7f69bb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69bc1015a0 0x7f69bc100eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:46:29.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.207+0000 7f69bb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69bc1015a0 0x7f69bc100eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59638/0 (socket says 172.21.15.16:59638) 2024-04-02T04:46:29.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.207+0000 7f69bb7fe700 1 -- 172.21.15.16:0/2083324680 learned_addr learned my addr 172.21.15.16:0/2083324680 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:46:29.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.207+0000 7f69baffd700 1 --2- 172.21.15.16:0/2083324680 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69bc103ed0 0x7f69bc0ff550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:46:29.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.207+0000 7f69bb7fe700 1 -- 172.21.15.16:0/2083324680 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69bc103ed0 msgr2=0x7f69bc0ff550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:29.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.207+0000 7f69bb7fe700 1 --2- 172.21.15.16:0/2083324680 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69bc103ed0 0x7f69bc0ff550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:29.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.207+0000 7f69bb7fe700 1 -- 172.21.15.16:0/2083324680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69ac0095f0 con 0x7f69bc1015a0 2024-04-02T04:46:29.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.208+0000 7f69bb7fe700 1 --2- 172.21.15.16:0/2083324680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69bc1015a0 0x7f69bc100eb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f69b000c930 tx=0x7f69b000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:46:29.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.208+0000 7f69b8ff9700 1 -- 172.21.15.16:0/2083324680 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69b0007ad0 con 0x7f69bc1015a0 2024-04-02T04:46:29.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.208+0000 7f69b8ff9700 1 -- 172.21.15.16:0/2083324680 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f69b000ce80 con 0x7f69bc1015a0 2024-04-02T04:46:29.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.208+0000 7f69c21de700 1 -- 172.21.15.16:0/2083324680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69bc0ffd70 con 0x7f69bc1015a0 2024-04-02T04:46:29.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.208+0000 7f69b8ff9700 1 -- 172.21.15.16:0/2083324680 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69b0018630 con 0x7f69bc1015a0 2024-04-02T04:46:29.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.208+0000 7f69c21de700 1 -- 172.21.15.16:0/2083324680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69bc100340 con 0x7f69bc1015a0 2024-04-02T04:46:29.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.209+0000 7f69c21de700 1 -- 172.21.15.16:0/2083324680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69bc04ea80 con 0x7f69bc1015a0 2024-04-02T04:46:29.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.210+0000 7f69b8ff9700 1 -- 172.21.15.16:0/2083324680 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f69b000f450 con 0x7f69bc1015a0 2024-04-02T04:46:29.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.210+0000 7f69b8ff9700 1 --2- 172.21.15.16:0/2083324680 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f69a406dd60 0x7f69a4070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:46:29.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.210+0000 7f69b8ff9700 1 -- 172.21.15.16:0/2083324680 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f69b008db10 con 0x7f69bc1015a0 2024-04-02T04:46:29.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.210+0000 7f69baffd700 1 --2- 172.21.15.16:0/2083324680 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f69a406dd60 0x7f69a4070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:46:29.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.211+0000 7f69baffd700 1 --2- 172.21.15.16:0/2083324680 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f69a406dd60 0x7f69a4070220 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f69ac0051c0 tx=0x7f69ac01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:46:29.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.215+0000 7f69b8ff9700 1 -- 172.21.15.16:0/2083324680 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f69b005abd0 con 0x7f69bc1015a0 2024-04-02T04:46:29.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:29 smithi027 ceph-mon[83036]: failed to encode map e56 with expected crc 2024-04-02T04:46:29.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:29 smithi027 ceph-mon[83036]: osdmap e56: 6 total, 5 up, 6 in 2024-04-02T04:46:29.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:29 smithi027 ceph-mon[83036]: failed to encode map e56 with expected crc 2024-04-02T04:46:29.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:29 smithi027 ceph-mon[83036]: failed to encode map e56 with expected crc 2024-04-02T04:46:29.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:29 smithi027 ceph-mon[83036]: failed to encode map e56 with expected crc 2024-04-02T04:46:29.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:29 smithi027 ceph-mon[83036]: failed to encode map e56 with expected crc 2024-04-02T04:46:29.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:29 smithi027 ceph-mon[83036]: failed to encode map e56 with expected crc 2024-04-02T04:46:29.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:29 smithi027 ceph-mon[83036]: failed to encode map e56 with expected crc 2024-04-02T04:46:29.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:29 smithi027 ceph-mon[83036]: failed to encode map e56 with expected crc 2024-04-02T04:46:29.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.428+0000 7f69c21de700 1 -- 172.21.15.16:0/2083324680 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f69bc1048d0 con 0x7f69a406dd60 2024-04-02T04:46:29.433 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:29 smithi016 podman[107752]: 2024-04-02 04:46:29.217090414 +0000 UTC m=+0.653350447 container died 04eb71e5814d32e6ddb641c5557fa3dc9304f4b60b94b6e850b5b7a924baf714 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1-deactivate, RELEASE=squid-6cec793, org.label-schema.build-date=20240326, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-02T04:46:29.433 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:29 smithi016 ceph-mon[93230]: failed to encode map e56 with expected crc 2024-04-02T04:46:29.433 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:29 smithi016 ceph-mon[93230]: osdmap e56: 6 total, 5 up, 6 in 2024-04-02T04:46:29.433 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:29 smithi016 ceph-mon[93230]: failed to encode map e56 with expected crc 2024-04-02T04:46:29.433 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:29 smithi016 ceph-mon[93230]: failed to encode map e56 with expected crc 2024-04-02T04:46:29.434 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:29 smithi016 ceph-mon[93230]: failed to encode map e56 with expected crc 2024-04-02T04:46:29.434 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:29 smithi016 ceph-mon[93230]: failed to encode map e56 with expected crc 2024-04-02T04:46:29.434 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:29 smithi016 ceph-mon[93230]: failed to encode map e56 with expected crc 2024-04-02T04:46:29.434 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:29 smithi016 ceph-mon[93230]: failed to encode map e56 with expected crc 2024-04-02T04:46:29.434 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:29 smithi016 ceph-mon[93230]: failed to encode map e56 with expected crc 2024-04-02T04:46:29.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.441+0000 7f69b8ff9700 1 -- 172.21.15.16:0/2083324680 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f69bc1048d0 con 0x7f69a406dd60 2024-04-02T04:46:29.444 INFO:teuthology.orchestra.run.smithi016.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:46:29.444 INFO:teuthology.orchestra.run.smithi016.stdout:alertmanager.smithi016 smithi016 *:9093,9094 running (10m) 27s ago 12m 24.1M - 0.25.0 c8568f914cd2 b95d8354b016 2024-04-02T04:46:29.444 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi016 smithi016 running (12m) 27s ago 12m 7948k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 9de6c16a487f 2024-04-02T04:46:29.444 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi027 smithi027 running (11m) 49s ago 11m 7964k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 99956592471a 2024-04-02T04:46:29.444 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi016 smithi016 running (58s) 27s ago 12m 6660k - 19.0.0-1979-g6cec793d 78f830190839 dc63ce0eef58 2024-04-02T04:46:29.444 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi027 smithi027 running (53s) 49s ago 11m 6665k - 19.0.0-1979-g6cec793d 78f830190839 2beb077018e4 2024-04-02T04:46:29.444 INFO:teuthology.orchestra.run.smithi016.stdout:grafana.smithi016 smithi016 *:3000 running (10m) 27s ago 12m 86.5M - 9.4.7 954c08fa6188 7c43d133569c 2024-04-02T04:46:29.445 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.fyzrnm smithi016 running (6m) 27s ago 6m 17.4M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 70f28b9572a6 2024-04-02T04:46:29.445 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.uhgkgg smithi016 running (6m) 27s ago 6m 15.6M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 0aa4d16562a6 2024-04-02T04:46:29.445 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.sdublg smithi027 running (6m) 49s ago 6m 19.3M - 18.2.2-1059-g4d05f4ca 909dce4fe53d a538451099db 2024-04-02T04:46:29.445 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.zwrfoo smithi027 running (6m) 49s ago 6m 158M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 4cb3fa428d71 2024-04-02T04:46:29.445 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi016.eqnebx smithi016 *:8443,9283,8765 running (3m) 27s ago 13m 498M - 19.0.0-1979-g6cec793d 78f830190839 d61ed91b4577 2024-04-02T04:46:29.445 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi027.kjzcsc smithi027 *:8443,9283,8765 running (2m) 49s ago 11m 449M - 19.0.0-1979-g6cec793d 78f830190839 5072a5ffc761 2024-04-02T04:46:29.445 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi016 smithi016 running (2m) 27s ago 14m 44.7M 2048M 19.0.0-1979-g6cec793d 78f830190839 eabfc84f1cdf 2024-04-02T04:46:29.445 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi027 smithi027 running (114s) 49s ago 11m 36.8M 2048M 19.0.0-1979-g6cec793d 78f830190839 fcd68b20dcc5 2024-04-02T04:46:29.445 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi016 smithi016 *:9100 running (12m) 27s ago 12m 14.7M - 1.5.0 0da6a335fe13 09f54c65b53c 2024-04-02T04:46:29.445 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi027 smithi027 *:9100 running (11m) 49s ago 11m 15.2M - 1.5.0 0da6a335fe13 e41a657c7443 2024-04-02T04:46:29.445 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 smithi016 running (33s) 27s ago 10m 12.7M 1217M 19.0.0-1979-g6cec793d 78f830190839 421d9e1cc751 2024-04-02T04:46:29.445 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 smithi016 running (10m) 27s ago 10m 223M 1217M 18.2.2-1059-g4d05f4ca 909dce4fe53d 7361abc9e1b6 2024-04-02T04:46:29.445 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 smithi016 running (9m) 27s ago 9m 179M 1217M 18.2.2-1059-g4d05f4ca 909dce4fe53d 52e592b4d1f2 2024-04-02T04:46:29.445 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 smithi027 running (9m) 49s ago 9m 236M 2240M 18.2.2-1059-g4d05f4ca 909dce4fe53d cf7c7fe022f0 2024-04-02T04:46:29.445 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 smithi027 running (8m) 49s ago 8m 211M 2240M 18.2.2-1059-g4d05f4ca 909dce4fe53d 8cbbedf8cac9 2024-04-02T04:46:29.446 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 smithi027 running (8m) 49s ago 8m 194M 2240M 18.2.2-1059-g4d05f4ca 909dce4fe53d ac6094415529 2024-04-02T04:46:29.446 INFO:teuthology.orchestra.run.smithi016.stdout:prometheus.smithi016 smithi016 *:9095 running (3m) 27s ago 11m 56.1M - 2.43.0 a07b618ecd1d 466f00d73ddd 2024-04-02T04:46:29.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.442+0000 7f69c21de700 1 -- 172.21.15.16:0/2083324680 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f69a406dd60 msgr2=0x7f69a4070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:29.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.442+0000 7f69c21de700 1 --2- 172.21.15.16:0/2083324680 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f69a406dd60 0x7f69a4070220 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f69ac0051c0 tx=0x7f69ac01a040 comp rx=0 tx=0).stop 2024-04-02T04:46:29.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.442+0000 7f69c21de700 1 -- 172.21.15.16:0/2083324680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69bc1015a0 msgr2=0x7f69bc100eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:29.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.442+0000 7f69c21de700 1 --2- 172.21.15.16:0/2083324680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69bc1015a0 0x7f69bc100eb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f69b000c930 tx=0x7f69b000cc40 comp rx=0 tx=0).stop 2024-04-02T04:46:29.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.442+0000 7f69c21de700 1 -- 172.21.15.16:0/2083324680 shutdown_connections 2024-04-02T04:46:29.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.442+0000 7f69c21de700 1 --2- 172.21.15.16:0/2083324680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69bc1015a0 0x7f69bc100eb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:29.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.442+0000 7f69c21de700 1 --2- 172.21.15.16:0/2083324680 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f69a406dd60 0x7f69a4070220 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:29.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.442+0000 7f69c21de700 1 --2- 172.21.15.16:0/2083324680 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69bc103ed0 0x7f69bc0ff550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:29.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.442+0000 7f69c21de700 1 -- 172.21.15.16:0/2083324680 >> 172.21.15.16:0/2083324680 conn(0x7f69bc0fafb0 msgr2=0x7f69bc0fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:46:29.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.442+0000 7f69c21de700 1 -- 172.21.15.16:0/2083324680 shutdown_connections 2024-04-02T04:46:29.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.442+0000 7f69c21de700 1 -- 172.21.15.16:0/2083324680 wait complete. 2024-04-02T04:46:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.547+0000 7fd3318f9700 1 -- 172.21.15.16:0/4152056892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd32c101ce0 msgr2=0x7fd32c102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.547+0000 7fd3318f9700 1 --2- 172.21.15.16:0/4152056892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd32c101ce0 0x7fd32c102140 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd314009910 tx=0x7fd314009c20 comp rx=0 tx=0).stop 2024-04-02T04:46:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.547+0000 7fd3318f9700 1 -- 172.21.15.16:0/4152056892 shutdown_connections 2024-04-02T04:46:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.547+0000 7fd3318f9700 1 --2- 172.21.15.16:0/4152056892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd32c101ce0 0x7fd32c102140 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.547+0000 7fd3318f9700 1 --2- 172.21.15.16:0/4152056892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd32c100ae0 0x7fd32c100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.547+0000 7fd3318f9700 1 -- 172.21.15.16:0/4152056892 >> 172.21.15.16:0/4152056892 conn(0x7fd32c0fc0a0 msgr2=0x7fd32c0fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:46:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.547+0000 7fd3318f9700 1 -- 172.21.15.16:0/4152056892 shutdown_connections 2024-04-02T04:46:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.547+0000 7fd3318f9700 1 -- 172.21.15.16:0/4152056892 wait complete. 2024-04-02T04:46:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.548+0000 7fd3318f9700 1 Processor -- start 2024-04-02T04:46:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.548+0000 7fd3318f9700 1 -- start start 2024-04-02T04:46:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.548+0000 7fd3318f9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd32c100ae0 0x7fd32c1964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:46:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.548+0000 7fd3318f9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd32c101ce0 0x7fd32c196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:46:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.548+0000 7fd3318f9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32c197020 con 0x7fd32c100ae0 2024-04-02T04:46:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.548+0000 7fd3318f9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32c197160 con 0x7fd32c101ce0 2024-04-02T04:46:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.548+0000 7fd32affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd32c100ae0 0x7fd32c1964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:46:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.548+0000 7fd32affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd32c100ae0 0x7fd32c1964c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59654/0 (socket says 172.21.15.16:59654) 2024-04-02T04:46:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.548+0000 7fd32affd700 1 -- 172.21.15.16:0/1255213670 learned_addr learned my addr 172.21.15.16:0/1255213670 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:46:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.548+0000 7fd32a7fc700 1 --2- 172.21.15.16:0/1255213670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd32c101ce0 0x7fd32c196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:46:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.549+0000 7fd32affd700 1 -- 172.21.15.16:0/1255213670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd32c101ce0 msgr2=0x7fd32c196a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.549+0000 7fd32affd700 1 --2- 172.21.15.16:0/1255213670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd32c101ce0 0x7fd32c196a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.549+0000 7fd32affd700 1 -- 172.21.15.16:0/1255213670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3140095f0 con 0x7fd32c100ae0 2024-04-02T04:46:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.549+0000 7fd32affd700 1 --2- 172.21.15.16:0/1255213670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd32c100ae0 0x7fd32c1964c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd31c007e90 tx=0x7fd31c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:46:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.549+0000 7fd3308f7700 1 -- 172.21.15.16:0/1255213670 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd31c017070 con 0x7fd32c100ae0 2024-04-02T04:46:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.549+0000 7fd3318f9700 1 -- 172.21.15.16:0/1255213670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd32c19bbc0 con 0x7fd32c100ae0 2024-04-02T04:46:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.550+0000 7fd3318f9700 1 -- 172.21.15.16:0/1255213670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd32c19c110 con 0x7fd32c100ae0 2024-04-02T04:46:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.550+0000 7fd3308f7700 1 -- 172.21.15.16:0/1255213670 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd31c00f040 con 0x7fd32c100ae0 2024-04-02T04:46:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.550+0000 7fd3308f7700 1 -- 172.21.15.16:0/1255213670 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd31c004790 con 0x7fd32c100ae0 2024-04-02T04:46:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.551+0000 7fd3318f9700 1 -- 172.21.15.16:0/1255213670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd30c005320 con 0x7fd32c100ae0 2024-04-02T04:46:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.552+0000 7fd3308f7700 1 -- 172.21.15.16:0/1255213670 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fd31c004020 con 0x7fd32c100ae0 2024-04-02T04:46:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.552+0000 7fd3308f7700 1 --2- 172.21.15.16:0/1255213670 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd31806dd60 0x7fd318070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:46:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.552+0000 7fd3308f7700 1 -- 172.21.15.16:0/1255213670 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fd31c08d450 con 0x7fd32c100ae0 2024-04-02T04:46:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.552+0000 7fd32a7fc700 1 --2- 172.21.15.16:0/1255213670 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd31806dd60 0x7fd318070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:46:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.553+0000 7fd32a7fc700 1 --2- 172.21.15.16:0/1255213670 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd31806dd60 0x7fd318070220 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd314009fd0 tx=0x7fd314005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:46:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.558+0000 7fd3308f7700 1 -- 172.21.15.16:0/1255213670 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd31c05a510 con 0x7fd32c100ae0 2024-04-02T04:46:29.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.849+0000 7fd3318f9700 1 -- 172.21.15.16:0/1255213670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd30c005de0 con 0x7fd32c100ae0 2024-04-02T04:46:29.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.849+0000 7fd3308f7700 1 -- 172.21.15.16:0/1255213670 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+836 (secure 0 0 0) 0x7fd31c05a0a0 con 0x7fd32c100ae0 2024-04-02T04:46:29.853 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:46:29.853 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": { 2024-04-02T04:46:29.854 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:46:29.854 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:46:29.854 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": { 2024-04-02T04:46:29.854 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:46:29.854 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:46:29.854 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": { 2024-04-02T04:46:29.854 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 4, 2024-04-02T04:46:29.854 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 1 2024-04-02T04:46:29.854 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:46:29.854 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-04-02T04:46:29.854 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 4 2024-04-02T04:46:29.854 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:46:29.854 INFO:teuthology.orchestra.run.smithi016.stdout: "overall": { 2024-04-02T04:46:29.855 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 8, 2024-04-02T04:46:29.855 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 5 2024-04-02T04:46:29.855 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-02T04:46:29.855 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:46:29.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.852+0000 7fd3318f9700 1 -- 172.21.15.16:0/1255213670 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd31806dd60 msgr2=0x7fd318070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:29.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.852+0000 7fd3318f9700 1 --2- 172.21.15.16:0/1255213670 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd31806dd60 0x7fd318070220 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd314009fd0 tx=0x7fd314005960 comp rx=0 tx=0).stop 2024-04-02T04:46:29.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.852+0000 7fd3318f9700 1 -- 172.21.15.16:0/1255213670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd32c100ae0 msgr2=0x7fd32c1964c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:29.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.852+0000 7fd3318f9700 1 --2- 172.21.15.16:0/1255213670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd32c100ae0 0x7fd32c1964c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd31c007e90 tx=0x7fd31c00d3b0 comp rx=0 tx=0).stop 2024-04-02T04:46:29.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.852+0000 7fd3318f9700 1 -- 172.21.15.16:0/1255213670 shutdown_connections 2024-04-02T04:46:29.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.852+0000 7fd3318f9700 1 --2- 172.21.15.16:0/1255213670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd32c100ae0 0x7fd32c1964c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:29.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.852+0000 7fd3318f9700 1 --2- 172.21.15.16:0/1255213670 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd31806dd60 0x7fd318070220 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:29.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.852+0000 7fd3318f9700 1 --2- 172.21.15.16:0/1255213670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd32c101ce0 0x7fd32c196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:29.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.852+0000 7fd3318f9700 1 -- 172.21.15.16:0/1255213670 >> 172.21.15.16:0/1255213670 conn(0x7fd32c0fc0a0 msgr2=0x7fd32c105280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:46:29.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.852+0000 7fd3318f9700 1 -- 172.21.15.16:0/1255213670 shutdown_connections 2024-04-02T04:46:29.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.852+0000 7fd3318f9700 1 -- 172.21.15.16:0/1255213670 wait complete. 2024-04-02T04:46:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.966+0000 7fa0b184b700 1 -- 172.21.15.16:0/3271050372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0ac101d00 msgr2=0x7fa0ac102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.966+0000 7fa0b184b700 1 --2- 172.21.15.16:0/3271050372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0ac101d00 0x7fa0ac102160 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0009910 tx=0x7fa0a0009c20 comp rx=0 tx=0).stop 2024-04-02T04:46:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.966+0000 7fa0b184b700 1 -- 172.21.15.16:0/3271050372 shutdown_connections 2024-04-02T04:46:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.966+0000 7fa0b184b700 1 --2- 172.21.15.16:0/3271050372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0ac101d00 0x7fa0ac102160 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.966+0000 7fa0b184b700 1 --2- 172.21.15.16:0/3271050372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0ac100b00 0x7fa0ac100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.966+0000 7fa0b184b700 1 -- 172.21.15.16:0/3271050372 >> 172.21.15.16:0/3271050372 conn(0x7fa0ac0fc0a0 msgr2=0x7fa0ac0fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:46:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.966+0000 7fa0b184b700 1 -- 172.21.15.16:0/3271050372 shutdown_connections 2024-04-02T04:46:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.966+0000 7fa0b184b700 1 -- 172.21.15.16:0/3271050372 wait complete. 2024-04-02T04:46:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.967+0000 7fa0b184b700 1 Processor -- start 2024-04-02T04:46:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.967+0000 7fa0b184b700 1 -- start start 2024-04-02T04:46:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.967+0000 7fa0b184b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0ac100b00 0x7fa0ac1964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:46:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.967+0000 7fa0b184b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0ac101d00 0x7fa0ac1969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:46:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.967+0000 7fa0b184b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac197000 con 0x7fa0ac100b00 2024-04-02T04:46:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.967+0000 7fa0b184b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac197140 con 0x7fa0ac101d00 2024-04-02T04:46:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.967+0000 7fa0aaffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0ac100b00 0x7fa0ac1964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:46:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.967+0000 7fa0aaffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0ac100b00 0x7fa0ac1964a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59672/0 (socket says 172.21.15.16:59672) 2024-04-02T04:46:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.967+0000 7fa0aaffd700 1 -- 172.21.15.16:0/4261525251 learned_addr learned my addr 172.21.15.16:0/4261525251 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:46:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.967+0000 7fa0aa7fc700 1 --2- 172.21.15.16:0/4261525251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0ac101d00 0x7fa0ac1969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:46:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.967+0000 7fa0aaffd700 1 -- 172.21.15.16:0/4261525251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0ac101d00 msgr2=0x7fa0ac1969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.967+0000 7fa0aaffd700 1 --2- 172.21.15.16:0/4261525251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0ac101d00 0x7fa0ac1969e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.967+0000 7fa0aaffd700 1 -- 172.21.15.16:0/4261525251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a00095f0 con 0x7fa0ac100b00 2024-04-02T04:46:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.967+0000 7fa0aa7fc700 1 --2- 172.21.15.16:0/4261525251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0ac101d00 0x7fa0ac1969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T04:46:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.968+0000 7fa0aaffd700 1 --2- 172.21.15.16:0/4261525251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0ac100b00 0x7fa0ac1964a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa09c00d900 tx=0x7fa09c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:46:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.968+0000 7fa0b0849700 1 -- 172.21.15.16:0/4261525251 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa09c009900 con 0x7fa0ac100b00 2024-04-02T04:46:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.968+0000 7fa0b0849700 1 -- 172.21.15.16:0/4261525251 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa09c010460 con 0x7fa0ac100b00 2024-04-02T04:46:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.968+0000 7fa0b184b700 1 -- 172.21.15.16:0/4261525251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0ac19bba0 con 0x7fa0ac100b00 2024-04-02T04:46:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.968+0000 7fa0b0849700 1 -- 172.21.15.16:0/4261525251 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa09c00f5b0 con 0x7fa0ac100b00 2024-04-02T04:46:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.968+0000 7fa0b184b700 1 -- 172.21.15.16:0/4261525251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0ac19c0f0 con 0x7fa0ac100b00 2024-04-02T04:46:29.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.969+0000 7fa0b184b700 1 -- 172.21.15.16:0/4261525251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0ac1906a0 con 0x7fa0ac100b00 2024-04-02T04:46:29.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.970+0000 7fa0b0849700 1 -- 172.21.15.16:0/4261525251 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fa09c0105d0 con 0x7fa0ac100b00 2024-04-02T04:46:29.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.970+0000 7fa0b0849700 1 --2- 172.21.15.16:0/4261525251 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fa09406dd70 0x7fa094070230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:46:29.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.970+0000 7fa0b0849700 1 -- 172.21.15.16:0/4261525251 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fa09c08d6b0 con 0x7fa0ac100b00 2024-04-02T04:46:29.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.970+0000 7fa0aa7fc700 1 --2- 172.21.15.16:0/4261525251 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fa09406dd70 0x7fa094070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:46:29.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.971+0000 7fa0aa7fc700 1 --2- 172.21.15.16:0/4261525251 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fa09406dd70 0x7fa094070230 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa0a00051c0 tx=0x7fa0a000b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:46:29.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:29.975+0000 7fa0b0849700 1 -- 172.21.15.16:0/4261525251 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa09c05a7f0 con 0x7fa0ac100b00 2024-04-02T04:46:30.119 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:29 smithi016 podman[107825]: 2024-04-02 04:46:29.838157228 +0000 UTC m=+0.615633408 container remove 04eb71e5814d32e6ddb641c5557fa3dc9304f4b60b94b6e850b5b7a924baf714 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1-deactivate, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=squid-6cec793, CEPH_POINT_RELEASE=) 2024-04-02T04:46:30.119 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:29 smithi016 systemd[1]: ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@osd.1.service: Deactivated successfully. 2024-04-02T04:46:30.119 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:29 smithi016 systemd[1]: Stopped Ceph osd.1 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4. 2024-04-02T04:46:30.120 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:29 smithi016 systemd[1]: ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@osd.1.service: Consumed 59.316s CPU time. 2024-04-02T04:46:30.120 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:30 smithi016 systemd[1]: Starting Ceph osd.1 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4... 2024-04-02T04:46:30.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.237+0000 7fa0b184b700 1 -- 172.21.15.16:0/4261525251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa0ac04ea80 con 0x7fa0ac100b00 2024-04-02T04:46:30.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.237+0000 7fa0b0849700 1 -- 172.21.15.16:0/4261525251 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1875 (secure 0 0 0) 0x7fa09c017770 con 0x7fa0ac100b00 2024-04-02T04:46:30.241 INFO:teuthology.orchestra.run.smithi016.stdout:e15 2024-04-02T04:46:30.241 INFO:teuthology.orchestra.run.smithi016.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:46:30.241 INFO:teuthology.orchestra.run.smithi016.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:46:30.241 INFO:teuthology.orchestra.run.smithi016.stdout:legacy client fscid: 1 2024-04-02T04:46:30.241 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:46:30.241 INFO:teuthology.orchestra.run.smithi016.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:46:30.241 INFO:teuthology.orchestra.run.smithi016.stdout:fs_name cephfs 2024-04-02T04:46:30.242 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 15 2024-04-02T04:46:30.242 INFO:teuthology.orchestra.run.smithi016.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:46:30.242 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-04-02T04:39:30.740168+0000 2024-04-02T04:46:30.242 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-04-02T04:39:49.875319+0000 2024-04-02T04:46:30.242 INFO:teuthology.orchestra.run.smithi016.stdout:tableserver 0 2024-04-02T04:46:30.242 INFO:teuthology.orchestra.run.smithi016.stdout:root 0 2024-04-02T04:46:30.242 INFO:teuthology.orchestra.run.smithi016.stdout:session_timeout 60 2024-04-02T04:46:30.242 INFO:teuthology.orchestra.run.smithi016.stdout:session_autoclose 300 2024-04-02T04:46:30.242 INFO:teuthology.orchestra.run.smithi016.stdout:max_file_size 1099511627776 2024-04-02T04:46:30.242 INFO:teuthology.orchestra.run.smithi016.stdout:max_xattr_size 65536 2024-04-02T04:46:30.242 INFO:teuthology.orchestra.run.smithi016.stdout:required_client_features {} 2024-04-02T04:46:30.242 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure 0 2024-04-02T04:46:30.242 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure_osd_epoch 46 2024-04-02T04:46:30.242 INFO:teuthology.orchestra.run.smithi016.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:46:30.243 INFO:teuthology.orchestra.run.smithi016.stdout:max_mds 1 2024-04-02T04:46:30.243 INFO:teuthology.orchestra.run.smithi016.stdout:in 0 2024-04-02T04:46:30.243 INFO:teuthology.orchestra.run.smithi016.stdout:up {0=24285} 2024-04-02T04:46:30.243 INFO:teuthology.orchestra.run.smithi016.stdout:failed 2024-04-02T04:46:30.243 INFO:teuthology.orchestra.run.smithi016.stdout:damaged 2024-04-02T04:46:30.243 INFO:teuthology.orchestra.run.smithi016.stdout:stopped 2024-04-02T04:46:30.243 INFO:teuthology.orchestra.run.smithi016.stdout:data_pools [3] 2024-04-02T04:46:30.243 INFO:teuthology.orchestra.run.smithi016.stdout:metadata_pool 2 2024-04-02T04:46:30.243 INFO:teuthology.orchestra.run.smithi016.stdout:inline_data enabled 2024-04-02T04:46:30.243 INFO:teuthology.orchestra.run.smithi016.stdout:balancer 2024-04-02T04:46:30.243 INFO:teuthology.orchestra.run.smithi016.stdout:bal_rank_mask -1 2024-04-02T04:46:30.243 INFO:teuthology.orchestra.run.smithi016.stdout:standby_count_wanted 1 2024-04-02T04:46:30.243 INFO:teuthology.orchestra.run.smithi016.stdout:qdb_cluster leader: 0 members: 2024-04-02T04:46:30.243 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.zwrfoo{0:24285} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.27:6824/1461455976,v1:172.21.15.27:6825/1461455976] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:46:30.243 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:46:30.244 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:46:30.244 INFO:teuthology.orchestra.run.smithi016.stdout:Standby daemons: 2024-04-02T04:46:30.244 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:46:30.244 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.fyzrnm{-1:14488} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.16:6828/2578719859,v1:172.21.15.16:6829/2578719859] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:46:30.244 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.uhgkgg{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.16:6826/601263579,v1:172.21.15.16:6827/601263579] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:46:30.244 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.sdublg{-1:24295} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.27:6826/4177192740,v1:172.21.15.27:6827/4177192740] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:46:30.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.239+0000 7fa0b184b700 1 -- 172.21.15.16:0/4261525251 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fa09406dd70 msgr2=0x7fa094070230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:30.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.239+0000 7fa0b184b700 1 --2- 172.21.15.16:0/4261525251 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fa09406dd70 0x7fa094070230 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa0a00051c0 tx=0x7fa0a000b540 comp rx=0 tx=0).stop 2024-04-02T04:46:30.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.239+0000 7fa0b184b700 1 -- 172.21.15.16:0/4261525251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0ac100b00 msgr2=0x7fa0ac1964a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:30.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.239+0000 7fa0b184b700 1 --2- 172.21.15.16:0/4261525251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0ac100b00 0x7fa0ac1964a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa09c00d900 tx=0x7fa09c00dcc0 comp rx=0 tx=0).stop 2024-04-02T04:46:30.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.239+0000 7fa0b184b700 1 -- 172.21.15.16:0/4261525251 shutdown_connections 2024-04-02T04:46:30.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.240+0000 7fa0b184b700 1 --2- 172.21.15.16:0/4261525251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0ac100b00 0x7fa0ac1964a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:30.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.240+0000 7fa0b184b700 1 --2- 172.21.15.16:0/4261525251 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fa09406dd70 0x7fa094070230 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:30.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.240+0000 7fa0b184b700 1 --2- 172.21.15.16:0/4261525251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0ac101d00 0x7fa0ac1969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:30.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.240+0000 7fa0b184b700 1 -- 172.21.15.16:0/4261525251 >> 172.21.15.16:0/4261525251 conn(0x7fa0ac0fc0a0 msgr2=0x7fa0ac1052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:46:30.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.240+0000 7fa0b184b700 1 -- 172.21.15.16:0/4261525251 shutdown_connections 2024-04-02T04:46:30.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.240+0000 7fa0b184b700 1 -- 172.21.15.16:0/4261525251 wait complete. 2024-04-02T04:46:30.245 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 15 2024-04-02T04:46:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.347+0000 7f53af897700 1 -- 172.21.15.16:0/3130753851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53a8101610 msgr2=0x7f53a8103a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.347+0000 7f53af897700 1 --2- 172.21.15.16:0/3130753851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53a8101610 0x7f53a8103a00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f53a4009910 tx=0x7f53a4009c20 comp rx=0 tx=0).stop 2024-04-02T04:46:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.348+0000 7f53af897700 1 -- 172.21.15.16:0/3130753851 shutdown_connections 2024-04-02T04:46:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.348+0000 7f53af897700 1 --2- 172.21.15.16:0/3130753851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53a8103f40 0x7f53a8106330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.348+0000 7f53af897700 1 --2- 172.21.15.16:0/3130753851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53a8101610 0x7f53a8103a00 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.348+0000 7f53af897700 1 -- 172.21.15.16:0/3130753851 >> 172.21.15.16:0/3130753851 conn(0x7f53a80faf80 msgr2=0x7f53a80fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:46:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.348+0000 7f53af897700 1 -- 172.21.15.16:0/3130753851 shutdown_connections 2024-04-02T04:46:30.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.348+0000 7f53af897700 1 -- 172.21.15.16:0/3130753851 wait complete. 2024-04-02T04:46:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.348+0000 7f53af897700 1 Processor -- start 2024-04-02T04:46:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.348+0000 7f53af897700 1 -- start start 2024-04-02T04:46:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.348+0000 7f53af897700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53a8101610 0x7f53a8100f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:46:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.348+0000 7f53af897700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53a8103f40 0x7f53a80ff5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:46:30.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.348+0000 7f53af897700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a80ffb00 con 0x7f53a8103f40 2024-04-02T04:46:30.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.348+0000 7f53af897700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a80ffc40 con 0x7f53a8101610 2024-04-02T04:46:30.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.349+0000 7f53ace32700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53a8103f40 0x7f53a80ff5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:46:30.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.349+0000 7f53ace32700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53a8103f40 0x7f53a80ff5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59696/0 (socket says 172.21.15.16:59696) 2024-04-02T04:46:30.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.349+0000 7f53ace32700 1 -- 172.21.15.16:0/3189892398 learned_addr learned my addr 172.21.15.16:0/3189892398 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:46:30.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.349+0000 7f53ad633700 1 --2- 172.21.15.16:0/3189892398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53a8101610 0x7f53a8100f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:46:30.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.349+0000 7f53ace32700 1 -- 172.21.15.16:0/3189892398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53a8101610 msgr2=0x7f53a8100f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:30.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.349+0000 7f53ace32700 1 --2- 172.21.15.16:0/3189892398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53a8101610 0x7f53a8100f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:30.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.349+0000 7f53ace32700 1 -- 172.21.15.16:0/3189892398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a40095f0 con 0x7f53a8103f40 2024-04-02T04:46:30.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.349+0000 7f53ace32700 1 --2- 172.21.15.16:0/3189892398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53a8103f40 0x7f53a80ff5c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f539800d900 tx=0x7f539800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:46:30.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.349+0000 7f539e7fc700 1 -- 172.21.15.16:0/3189892398 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53980041f0 con 0x7f53a8103f40 2024-04-02T04:46:30.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.349+0000 7f539e7fc700 1 -- 172.21.15.16:0/3189892398 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5398004cf0 con 0x7f53a8103f40 2024-04-02T04:46:30.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.349+0000 7f539e7fc700 1 -- 172.21.15.16:0/3189892398 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f539800b750 con 0x7f53a8103f40 2024-04-02T04:46:30.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.349+0000 7f53af897700 1 -- 172.21.15.16:0/3189892398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53a80fff20 con 0x7f53a8103f40 2024-04-02T04:46:30.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.349+0000 7f53af897700 1 -- 172.21.15.16:0/3189892398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53a819e510 con 0x7f53a8103f40 2024-04-02T04:46:30.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.350+0000 7f53af897700 1 -- 172.21.15.16:0/3189892398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53a804ea80 con 0x7f53a8103f40 2024-04-02T04:46:30.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.351+0000 7f539e7fc700 1 -- 172.21.15.16:0/3189892398 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f5398004350 con 0x7f53a8103f40 2024-04-02T04:46:30.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.351+0000 7f539e7fc700 1 --2- 172.21.15.16:0/3189892398 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f539406dd60 0x7f5394070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:46:30.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.352+0000 7f539e7fc700 1 -- 172.21.15.16:0/3189892398 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f539808d9f0 con 0x7f53a8103f40 2024-04-02T04:46:30.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.352+0000 7f53ad633700 1 --2- 172.21.15.16:0/3189892398 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f539406dd60 0x7f5394070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:46:30.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.352+0000 7f53ad633700 1 --2- 172.21.15.16:0/3189892398 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f539406dd60 0x7f5394070220 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f53a4006010 tx=0x7f53a400b580 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:46:30.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.356+0000 7f539e7fc700 1 -- 172.21.15.16:0/3189892398 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5398093020 con 0x7f53a8103f40 2024-04-02T04:46:30.385 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:30 smithi016 ceph-mon[93230]: from='client.34248 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:46:30.385 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:30 smithi016 ceph-mon[93230]: pgmap v76: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:30.385 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:30 smithi016 ceph-mon[93230]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:46:30.385 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:30 smithi016 ceph-mon[93230]: Health check failed: Reduced data availability: 9 pgs peering (PG_AVAILABILITY) 2024-04-02T04:46:30.386 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:30 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/1255213670' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:46:30.386 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:30 smithi016 podman[107976]: 2024-04-02 04:46:30.287212202 +0000 UTC m=+0.105751917 container create aa39fa6a25c22adc6bbf2bc07efab30f4298ed272aa20ac10c4620f1f03afcb9 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1-activate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-6cec793) 2024-04-02T04:46:30.386 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:30 smithi016 podman[107976]: 2024-04-02 04:46:30.199909888 +0000 UTC m=+0.018449630 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:46:30.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:30 smithi027 ceph-mon[83036]: from='client.34248 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:46:30.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:30 smithi027 ceph-mon[83036]: pgmap v76: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:30.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:30 smithi027 ceph-mon[83036]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:46:30.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:30 smithi027 ceph-mon[83036]: Health check failed: Reduced data availability: 9 pgs peering (PG_AVAILABILITY) 2024-04-02T04:46:30.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:30 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/1255213670' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:46:30.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.582+0000 7f53af897700 1 -- 172.21.15.16:0/3189892398 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f53a8104900 con 0x7f539406dd60 2024-04-02T04:46:30.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.583+0000 7f539e7fc700 1 -- 172.21.15.16:0/3189892398 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f53a8104900 con 0x7f539406dd60 2024-04-02T04:46:30.586 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:46:30.586 INFO:teuthology.orchestra.run.smithi016.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:46:30.587 INFO:teuthology.orchestra.run.smithi016.stdout: "in_progress": true, 2024-04-02T04:46:30.587 INFO:teuthology.orchestra.run.smithi016.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-02T04:46:30.587 INFO:teuthology.orchestra.run.smithi016.stdout: "services_complete": [ 2024-04-02T04:46:30.587 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr", 2024-04-02T04:46:30.587 INFO:teuthology.orchestra.run.smithi016.stdout: "mon", 2024-04-02T04:46:30.587 INFO:teuthology.orchestra.run.smithi016.stdout: "crash" 2024-04-02T04:46:30.587 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-04-02T04:46:30.587 INFO:teuthology.orchestra.run.smithi016.stdout: "progress": "7/23 daemons upgraded", 2024-04-02T04:46:30.587 INFO:teuthology.orchestra.run.smithi016.stdout: "message": "Currently upgrading osd daemons", 2024-04-02T04:46:30.587 INFO:teuthology.orchestra.run.smithi016.stdout: "is_paused": false 2024-04-02T04:46:30.587 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:46:30.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.586+0000 7f53af897700 1 -- 172.21.15.16:0/3189892398 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f539406dd60 msgr2=0x7f5394070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:30.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.586+0000 7f53af897700 1 --2- 172.21.15.16:0/3189892398 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f539406dd60 0x7f5394070220 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f53a4006010 tx=0x7f53a400b580 comp rx=0 tx=0).stop 2024-04-02T04:46:30.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.586+0000 7f53af897700 1 -- 172.21.15.16:0/3189892398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53a8103f40 msgr2=0x7f53a80ff5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:30.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.586+0000 7f53af897700 1 --2- 172.21.15.16:0/3189892398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53a8103f40 0x7f53a80ff5c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f539800d900 tx=0x7f539800dcc0 comp rx=0 tx=0).stop 2024-04-02T04:46:30.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.586+0000 7f53af897700 1 -- 172.21.15.16:0/3189892398 shutdown_connections 2024-04-02T04:46:30.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.586+0000 7f53af897700 1 --2- 172.21.15.16:0/3189892398 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f539406dd60 0x7f5394070220 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:30.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.586+0000 7f53af897700 1 --2- 172.21.15.16:0/3189892398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53a8101610 0x7f53a8100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:30.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.586+0000 7f53af897700 1 --2- 172.21.15.16:0/3189892398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53a8103f40 0x7f53a80ff5c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:30.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.586+0000 7f53af897700 1 -- 172.21.15.16:0/3189892398 >> 172.21.15.16:0/3189892398 conn(0x7f53a80faf80 msgr2=0x7f53a80fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:46:30.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.586+0000 7f53af897700 1 -- 172.21.15.16:0/3189892398 shutdown_connections 2024-04-02T04:46:30.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.586+0000 7f53af897700 1 -- 172.21.15.16:0/3189892398 wait complete. 2024-04-02T04:46:30.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.692+0000 7f8604784700 1 -- 172.21.15.16:0/3962683335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85fc101560 msgr2=0x7f85fc103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:30.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.692+0000 7f8604784700 1 --2- 172.21.15.16:0/3962683335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85fc101560 0x7f85fc103950 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f85f8009910 tx=0x7f85f8009c20 comp rx=0 tx=0).stop 2024-04-02T04:46:30.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.692+0000 7f8604784700 1 -- 172.21.15.16:0/3962683335 shutdown_connections 2024-04-02T04:46:30.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.692+0000 7f8604784700 1 --2- 172.21.15.16:0/3962683335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85fc103e90 0x7f85fc106280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:30.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.692+0000 7f8604784700 1 --2- 172.21.15.16:0/3962683335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85fc101560 0x7f85fc103950 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:30.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.693+0000 7f8604784700 1 -- 172.21.15.16:0/3962683335 >> 172.21.15.16:0/3962683335 conn(0x7f85fc0faf70 msgr2=0x7f85fc0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:46:30.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.693+0000 7f8604784700 1 -- 172.21.15.16:0/3962683335 shutdown_connections 2024-04-02T04:46:30.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.693+0000 7f8604784700 1 -- 172.21.15.16:0/3962683335 wait complete. 2024-04-02T04:46:30.697 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:30 smithi016 podman[107976]: 2024-04-02 04:46:30.490295389 +0000 UTC m=+0.308835117 container init aa39fa6a25c22adc6bbf2bc07efab30f4298ed272aa20ac10c4620f1f03afcb9 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1-activate, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-6cec793, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-02T04:46:30.697 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:30 smithi016 podman[107976]: 2024-04-02 04:46:30.495018418 +0000 UTC m=+0.313558140 container start aa39fa6a25c22adc6bbf2bc07efab30f4298ed272aa20ac10c4620f1f03afcb9 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1-activate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-6cec793, org.label-schema.schema-version=1.0) 2024-04-02T04:46:30.697 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:30 smithi016 podman[107976]: 2024-04-02 04:46:30.573102489 +0000 UTC m=+0.391642217 container attach aa39fa6a25c22adc6bbf2bc07efab30f4298ed272aa20ac10c4620f1f03afcb9 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1-activate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-6cec793, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T04:46:30.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.693+0000 7f8604784700 1 Processor -- start 2024-04-02T04:46:30.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.693+0000 7f8604784700 1 -- start start 2024-04-02T04:46:30.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.693+0000 7f8604784700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85fc101560 0x7f85fc100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:46:30.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.693+0000 7f8604784700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85fc103e90 0x7f85fc0ff510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:46:30.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.693+0000 7f8604784700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc0ffa50 con 0x7f85fc101560 2024-04-02T04:46:30.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.693+0000 7f8604784700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc0ffb90 con 0x7f85fc103e90 2024-04-02T04:46:30.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.694+0000 7f8602520700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85fc101560 0x7f85fc100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:46:30.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.694+0000 7f8602520700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85fc101560 0x7f85fc100e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59712/0 (socket says 172.21.15.16:59712) 2024-04-02T04:46:30.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.694+0000 7f8602520700 1 -- 172.21.15.16:0/291376895 learned_addr learned my addr 172.21.15.16:0/291376895 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:46:30.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.694+0000 7f8601d1f700 1 --2- 172.21.15.16:0/291376895 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85fc103e90 0x7f85fc0ff510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:46:30.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.694+0000 7f8602520700 1 -- 172.21.15.16:0/291376895 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85fc103e90 msgr2=0x7f85fc0ff510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:30.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.694+0000 7f8602520700 1 --2- 172.21.15.16:0/291376895 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85fc103e90 0x7f85fc0ff510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:30.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.694+0000 7f8602520700 1 -- 172.21.15.16:0/291376895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f80095f0 con 0x7f85fc101560 2024-04-02T04:46:30.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.694+0000 7f8602520700 1 --2- 172.21.15.16:0/291376895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85fc101560 0x7f85fc100e20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f85f80092e0 tx=0x7f85f800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:46:30.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.694+0000 7f85f37fe700 1 -- 172.21.15.16:0/291376895 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85f801d070 con 0x7f85fc101560 2024-04-02T04:46:30.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.694+0000 7f85f37fe700 1 -- 172.21.15.16:0/291376895 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f85f800f460 con 0x7f85fc101560 2024-04-02T04:46:30.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.694+0000 7f85f37fe700 1 -- 172.21.15.16:0/291376895 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85f8021640 con 0x7f85fc101560 2024-04-02T04:46:30.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.695+0000 7f8604784700 1 -- 172.21.15.16:0/291376895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85fc0ffe10 con 0x7f85fc101560 2024-04-02T04:46:30.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.695+0000 7f8604784700 1 -- 172.21.15.16:0/291376895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85fc100330 con 0x7f85fc101560 2024-04-02T04:46:30.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.696+0000 7f8604784700 1 -- 172.21.15.16:0/291376895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85fc04ea80 con 0x7f85fc101560 2024-04-02T04:46:30.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.697+0000 7f85f37fe700 1 -- 172.21.15.16:0/291376895 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f85f800f5d0 con 0x7f85fc101560 2024-04-02T04:46:30.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.697+0000 7f85f37fe700 1 --2- 172.21.15.16:0/291376895 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f85e806dd60 0x7f85e8070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:46:30.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.697+0000 7f85f37fe700 1 -- 172.21.15.16:0/291376895 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f85f808ef10 con 0x7f85fc101560 2024-04-02T04:46:30.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.697+0000 7f8601d1f700 1 --2- 172.21.15.16:0/291376895 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f85e806dd60 0x7f85e8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:46:30.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.698+0000 7f8601d1f700 1 --2- 172.21.15.16:0/291376895 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f85e806dd60 0x7f85e8070220 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f85ec006fd0 tx=0x7f85ec008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:46:30.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:30.702+0000 7f85f37fe700 1 -- 172.21.15.16:0/291376895 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f85f805bfd0 con 0x7f85fc101560 2024-04-02T04:46:31.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:31.003+0000 7f8604784700 1 -- 172.21.15.16:0/291376895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f85fc1096b0 con 0x7f85fc101560 2024-04-02T04:46:31.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:31.003+0000 7f85f37fe700 1 -- 172.21.15.16:0/291376895 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1103 (secure 0 0 0) 0x7f85f805bb60 con 0x7f85fc101560 2024-04-02T04:46:31.007 INFO:teuthology.orchestra.run.smithi016.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Reduced data availability: 9 pgs peering 2024-04-02T04:46:31.007 INFO:teuthology.orchestra.run.smithi016.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-02T04:46:31.007 INFO:teuthology.orchestra.run.smithi016.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-02T04:46:31.007 INFO:teuthology.orchestra.run.smithi016.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-02T04:46:31.007 INFO:teuthology.orchestra.run.smithi016.stdout: osd.1 (root=default,host=smithi016) is down 2024-04-02T04:46:31.007 INFO:teuthology.orchestra.run.smithi016.stdout:[WRN] PG_AVAILABILITY: Reduced data availability: 9 pgs peering 2024-04-02T04:46:31.007 INFO:teuthology.orchestra.run.smithi016.stdout: pg 2.6 is stuck peering for 100s, current state peering, last acting [3,4] 2024-04-02T04:46:31.007 INFO:teuthology.orchestra.run.smithi016.stdout: pg 2.9 is stuck peering for 2m, current state peering, last acting [4,0] 2024-04-02T04:46:31.007 INFO:teuthology.orchestra.run.smithi016.stdout: pg 2.a is stuck peering for 2m, current state peering, last acting [4,3] 2024-04-02T04:46:31.007 INFO:teuthology.orchestra.run.smithi016.stdout: pg 2.d is stuck peering for 2m, current state peering, last acting [3,2] 2024-04-02T04:46:31.007 INFO:teuthology.orchestra.run.smithi016.stdout: pg 2.15 is stuck peering for 102s, current state peering, last acting [3,0] 2024-04-02T04:46:31.008 INFO:teuthology.orchestra.run.smithi016.stdout: pg 3.0 is stuck peering for 104s, current state peering, last acting [2,4] 2024-04-02T04:46:31.008 INFO:teuthology.orchestra.run.smithi016.stdout: pg 3.2 is stuck peering for 104s, current state peering, last acting [5,3] 2024-04-02T04:46:31.008 INFO:teuthology.orchestra.run.smithi016.stdout: pg 3.4 is stuck peering for 103s, current state peering, last acting [2,3] 2024-04-02T04:46:31.008 INFO:teuthology.orchestra.run.smithi016.stdout: pg 3.19 is stuck peering for 103s, current state peering, last acting [4,3] 2024-04-02T04:46:31.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:31.006+0000 7f8604784700 1 -- 172.21.15.16:0/291376895 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f85e806dd60 msgr2=0x7f85e8070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:31.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:31.006+0000 7f8604784700 1 --2- 172.21.15.16:0/291376895 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f85e806dd60 0x7f85e8070220 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f85ec006fd0 tx=0x7f85ec008040 comp rx=0 tx=0).stop 2024-04-02T04:46:31.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:31.006+0000 7f8604784700 1 -- 172.21.15.16:0/291376895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85fc101560 msgr2=0x7f85fc100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:46:31.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:31.006+0000 7f8604784700 1 --2- 172.21.15.16:0/291376895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85fc101560 0x7f85fc100e20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f85f80092e0 tx=0x7f85f800be20 comp rx=0 tx=0).stop 2024-04-02T04:46:31.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:31.006+0000 7f8604784700 1 -- 172.21.15.16:0/291376895 shutdown_connections 2024-04-02T04:46:31.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:31.006+0000 7f8604784700 1 --2- 172.21.15.16:0/291376895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85fc101560 0x7f85fc100e20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:31.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:31.006+0000 7f8604784700 1 --2- 172.21.15.16:0/291376895 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f85e806dd60 0x7f85e8070220 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:31.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:31.006+0000 7f8604784700 1 --2- 172.21.15.16:0/291376895 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85fc103e90 0x7f85fc0ff510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:46:31.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:31.006+0000 7f8604784700 1 -- 172.21.15.16:0/291376895 >> 172.21.15.16:0/291376895 conn(0x7f85fc0faf70 msgr2=0x7f85fc0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:46:31.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:31.006+0000 7f8604784700 1 -- 172.21.15.16:0/291376895 shutdown_connections 2024-04-02T04:46:31.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:46:31.006+0000 7f8604784700 1 -- 172.21.15.16:0/291376895 wait complete. 2024-04-02T04:46:31.293 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:31 smithi016 ceph-mon[93230]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:46:31.293 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:31 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/4261525251' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:46:31.293 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:31 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/291376895' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:46:31.293 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:30 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1-activate[108015]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-02T04:46:31.294 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:30 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1-activate[108015]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-02T04:46:31.294 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:30 smithi016 bash[107976]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-02T04:46:31.294 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:30 smithi016 bash[107976]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-02T04:46:31.294 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:30 smithi016 bash[107976]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-02T04:46:31.294 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:30 smithi016 bash[107976]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-02T04:46:31.294 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:30 smithi016 bash[107976]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-04-02T04:46:31.294 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:30 smithi016 bash[107976]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-02T04:46:31.294 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:30 smithi016 bash[107976]: --> ceph-volume raw activate successful for osd ID: 1 2024-04-02T04:46:31.294 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:30 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1-activate[108015]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-02T04:46:31.294 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:30 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1-activate[108015]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-02T04:46:31.294 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:30 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1-activate[108015]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-04-02T04:46:31.294 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:30 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1-activate[108015]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-02T04:46:31.294 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:30 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1-activate[108015]: --> ceph-volume raw activate successful for osd ID: 1 2024-04-02T04:46:31.295 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:30 smithi016 podman[107976]: 2024-04-02 04:46:30.97648 +0000 UTC m=+0.795019719 container died aa39fa6a25c22adc6bbf2bc07efab30f4298ed272aa20ac10c4620f1f03afcb9 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1-activate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-6cec793) 2024-04-02T04:46:31.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:31 smithi027 ceph-mon[83036]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:46:31.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:31 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/4261525251' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:46:31.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:31 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/291376895' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:46:31.810 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:31 smithi016 podman[108162]: 2024-04-02 04:46:31.544258904 +0000 UTC m=+0.559319598 container remove aa39fa6a25c22adc6bbf2bc07efab30f4298ed272aa20ac10c4620f1f03afcb9 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-6cec793, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-02T04:46:31.810 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:31 smithi016 podman[108229]: 2024-04-02 04:46:31.66053504 +0000 UTC m=+0.018119235 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:46:31.810 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:31 smithi016 podman[108229]: 2024-04-02 04:46:31.761833099 +0000 UTC m=+0.119417358 container create 3612a56236a3d6e0438927443f396a9af27ea390e7218efc062ed7d9105b9828 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=squid-6cec793, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-02T04:46:32.125 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:31 smithi016 podman[108229]: 2024-04-02 04:46:31.931734834 +0000 UTC m=+0.289319014 container init 3612a56236a3d6e0438927443f396a9af27ea390e7218efc062ed7d9105b9828 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, RELEASE=squid-6cec793, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T04:46:32.125 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:31 smithi016 podman[108229]: 2024-04-02 04:46:31.934914449 +0000 UTC m=+0.292498634 container start 3612a56236a3d6e0438927443f396a9af27ea390e7218efc062ed7d9105b9828 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=squid-6cec793, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-02T04:46:32.125 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:31 smithi016 ceph-osd[108248]: -- 172.21.15.16:0/3603091738 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562e9522d680 con 0x562e951f6c00 2024-04-02T04:46:32.125 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:32 smithi016 bash[108229]: 3612a56236a3d6e0438927443f396a9af27ea390e7218efc062ed7d9105b9828 2024-04-02T04:46:32.125 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:32 smithi016 systemd[1]: Started Ceph osd.1 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4. 2024-04-02T04:46:32.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:32 smithi027 ceph-mon[83036]: from='client.34268 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:46:32.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:32 smithi027 ceph-mon[83036]: pgmap v77: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:32.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:32 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:32.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:32 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:32.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:32 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:32.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:32 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:46:32.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:32 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:46:32.484 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:32 smithi016 ceph-mon[93230]: from='client.34268 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:46:32.484 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:32 smithi016 ceph-mon[93230]: pgmap v77: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:32.485 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:32 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:32.485 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:32 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:32.485 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:32 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:32.485 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:32 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:46:32.485 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:32 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:46:32.810 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:32 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1[108244]: 2024-04-02T04:46:32.481+0000 7ff4a2b477c0 -1 Falling back to public interface 2024-04-02T04:46:33.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:33 smithi016 ceph-mon[93230]: Health check failed: Degraded data redundancy: 34/267 objects degraded (12.734%), 13 pgs degraded (PG_DEGRADED) 2024-04-02T04:46:33.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:33 smithi016 ceph-mon[93230]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs peering) 2024-04-02T04:46:33.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:33 smithi027 ceph-mon[83036]: Health check failed: Degraded data redundancy: 34/267 objects degraded (12.734%), 13 pgs degraded (PG_DEGRADED) 2024-04-02T04:46:33.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:33 smithi027 ceph-mon[83036]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs peering) 2024-04-02T04:46:34.524 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:34 smithi016 ceph-mon[93230]: pgmap v78: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-04-02T04:46:34.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:34 smithi027 ceph-mon[83036]: pgmap v78: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-04-02T04:46:36.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:36 smithi016 ceph-mon[93230]: pgmap v79: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-04-02T04:46:36.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:36 smithi027 ceph-mon[83036]: pgmap v79: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-04-02T04:46:37.809 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:37 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1[108244]: 2024-04-02T04:46:37.397+0000 7ff4a2b477c0 -1 osd.1 54 log_to_monitors true 2024-04-02T04:46:38.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:38 smithi027 ceph-mon[83036]: pgmap v80: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-04-02T04:46:38.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:38 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:38.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:38 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:38.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:38 smithi027 ceph-mon[83036]: from='osd.1 [v2:172.21.15.16:6810/2744685333,v1:172.21.15.16:6811/2744685333]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-02T04:46:38.560 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:46:38 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1[108244]: 2024-04-02T04:46:38.195+0000 7ff493f9f700 -1 osd.1 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T04:46:38.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:38 smithi016 ceph-mon[93230]: pgmap v80: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 260 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-04-02T04:46:38.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:38 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:38.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:38 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:38.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:38 smithi016 ceph-mon[93230]: from='osd.1 [v2:172.21.15.16:6810/2744685333,v1:172.21.15.16:6811/2744685333]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-02T04:46:39.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:39 smithi016 ceph-mon[93230]: from='osd.1 [v2:172.21.15.16:6810/2744685333,v1:172.21.15.16:6811/2744685333]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-02T04:46:39.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:39 smithi016 ceph-mon[93230]: osdmap e57: 6 total, 5 up, 6 in 2024-04-02T04:46:39.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:39 smithi016 ceph-mon[93230]: from='osd.1 [v2:172.21.15.16:6810/2744685333,v1:172.21.15.16:6811/2744685333]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-04-02T04:46:39.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:39 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:39.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:39 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:39.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:39 smithi016 ceph-mon[93230]: failed to encode map e57 with expected crc 2024-04-02T04:46:39.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:39 smithi027 ceph-mon[83036]: from='osd.1 [v2:172.21.15.16:6810/2744685333,v1:172.21.15.16:6811/2744685333]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-02T04:46:39.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:39 smithi027 ceph-mon[83036]: osdmap e57: 6 total, 5 up, 6 in 2024-04-02T04:46:39.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:39 smithi027 ceph-mon[83036]: from='osd.1 [v2:172.21.15.16:6810/2744685333,v1:172.21.15.16:6811/2744685333]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-04-02T04:46:39.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:39 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:39.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:39 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:39.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:39 smithi027 ceph-mon[83036]: failed to encode map e57 with expected crc 2024-04-02T04:46:40.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:40 smithi016 ceph-mon[93230]: pgmap v82: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-04-02T04:46:40.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:40 smithi016 ceph-mon[93230]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-02T04:46:40.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:40 smithi016 ceph-mon[93230]: failed to encode map e57 with expected crc 2024-04-02T04:46:40.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:40 smithi016 ceph-mon[93230]: osd.1 [v2:172.21.15.16:6810/2744685333,v1:172.21.15.16:6811/2744685333] boot 2024-04-02T04:46:40.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:40 smithi016 ceph-mon[93230]: osdmap e58: 6 total, 6 up, 6 in 2024-04-02T04:46:40.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:40 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T04:46:40.547 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:40 smithi027 ceph-mon[83036]: pgmap v82: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-04-02T04:46:40.547 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:40 smithi027 ceph-mon[83036]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-02T04:46:40.548 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:40 smithi027 ceph-mon[83036]: failed to encode map e57 with expected crc 2024-04-02T04:46:40.548 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:40 smithi027 ceph-mon[83036]: osd.1 [v2:172.21.15.16:6810/2744685333,v1:172.21.15.16:6811/2744685333] boot 2024-04-02T04:46:40.548 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:40 smithi027 ceph-mon[83036]: osdmap e58: 6 total, 6 up, 6 in 2024-04-02T04:46:40.548 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:40 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T04:46:41.479 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:41 smithi016 ceph-mon[93230]: osdmap e59: 6 total, 6 up, 6 in 2024-04-02T04:46:41.480 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:41 smithi016 ceph-mon[93230]: failed to encode map e59 with expected crc 2024-04-02T04:46:41.480 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:41 smithi016 ceph-mon[93230]: failed to encode map e59 with expected crc 2024-04-02T04:46:41.480 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:41 smithi016 ceph-mon[93230]: failed to encode map e59 with expected crc 2024-04-02T04:46:41.480 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:41 smithi016 ceph-mon[93230]: failed to encode map e59 with expected crc 2024-04-02T04:46:41.480 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:41 smithi016 ceph-mon[93230]: failed to encode map e59 with expected crc 2024-04-02T04:46:41.480 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:41 smithi016 ceph-mon[93230]: failed to encode map e59 with expected crc 2024-04-02T04:46:41.480 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:41 smithi016 ceph-mon[93230]: failed to encode map e59 with expected crc 2024-04-02T04:46:41.480 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:41 smithi016 ceph-mon[93230]: failed to encode map e59 with expected crc 2024-04-02T04:46:41.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:41 smithi027 ceph-mon[83036]: osdmap e59: 6 total, 6 up, 6 in 2024-04-02T04:46:41.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:41 smithi027 ceph-mon[83036]: failed to encode map e59 with expected crc 2024-04-02T04:46:41.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:41 smithi027 ceph-mon[83036]: failed to encode map e59 with expected crc 2024-04-02T04:46:41.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:41 smithi027 ceph-mon[83036]: failed to encode map e59 with expected crc 2024-04-02T04:46:41.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:41 smithi027 ceph-mon[83036]: failed to encode map e59 with expected crc 2024-04-02T04:46:41.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:41 smithi027 ceph-mon[83036]: failed to encode map e59 with expected crc 2024-04-02T04:46:41.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:41 smithi027 ceph-mon[83036]: failed to encode map e59 with expected crc 2024-04-02T04:46:41.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:41 smithi027 ceph-mon[83036]: failed to encode map e59 with expected crc 2024-04-02T04:46:41.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:41 smithi027 ceph-mon[83036]: failed to encode map e59 with expected crc 2024-04-02T04:46:42.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:42 smithi027 ceph-mon[83036]: pgmap v85: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-04-02T04:46:42.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:42 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:42.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:42 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:42.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:42 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:46:42.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:42 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:46:42.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:42 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:42.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:42 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:46:42.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:42 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:46:42.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:42 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:46:42.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:42 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:46:42.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:42 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-02T04:46:42.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:42 smithi016 ceph-mon[93230]: pgmap v85: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-04-02T04:46:42.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:42 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:42.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:42 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:42.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:42 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:46:42.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:42 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:46:42.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:42 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:42.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:42 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:46:42.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:42 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:46:42.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:42 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:46:42.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:42 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:46:42.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:42 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-02T04:46:43.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:43 smithi016 ceph-mon[93230]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-02T04:46:43.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:43 smithi016 ceph-mon[93230]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-04-02T04:46:43.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:43 smithi027 ceph-mon[83036]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-02T04:46:43.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:43 smithi027 ceph-mon[83036]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-04-02T04:46:44.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:44 smithi016 ceph-mon[93230]: pgmap v86: 65 pgs: 7 active+undersized+degraded, 5 active+undersized, 53 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 18/267 objects degraded (6.742%) 2024-04-02T04:46:44.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:44 smithi016 ceph-mon[93230]: Health check update: Degraded data redundancy: 18/267 objects degraded (6.742%), 7 pgs degraded (PG_DEGRADED) 2024-04-02T04:46:44.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:44 smithi027 ceph-mon[83036]: pgmap v86: 65 pgs: 7 active+undersized+degraded, 5 active+undersized, 53 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 18/267 objects degraded (6.742%) 2024-04-02T04:46:44.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:44 smithi027 ceph-mon[83036]: Health check update: Degraded data redundancy: 18/267 objects degraded (6.742%), 7 pgs degraded (PG_DEGRADED) 2024-04-02T04:46:45.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:45 smithi016 ceph-mon[93230]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 18/267 objects degraded (6.742%), 7 pgs degraded) 2024-04-02T04:46:45.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:45 smithi027 ceph-mon[83036]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 18/267 objects degraded (6.742%), 7 pgs degraded) 2024-04-02T04:46:46.461 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:46 smithi016 ceph-mon[93230]: pgmap v87: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:46.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:46 smithi027 ceph-mon[83036]: pgmap v87: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:48.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:48 smithi027 ceph-mon[83036]: pgmap v88: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:48.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:48 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:48.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:48 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:46:48.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:48 smithi016 ceph-mon[93230]: pgmap v88: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:48.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:48 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:48.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:48 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:46:50.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:50 smithi027 ceph-mon[83036]: pgmap v89: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:50.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:50 smithi016 ceph-mon[93230]: pgmap v89: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:52.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:52 smithi027 ceph-mon[83036]: pgmap v90: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:52.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:52 smithi016 ceph-mon[93230]: pgmap v90: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:53.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:53 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:53.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:53 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:54.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:54 smithi027 ceph-mon[83036]: pgmap v91: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:54.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:54 smithi016 ceph-mon[93230]: pgmap v91: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:56.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:56 smithi027 ceph-mon[83036]: pgmap v92: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:56.513 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:56 smithi016 ceph-mon[93230]: pgmap v92: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:57.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:57 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-02T04:46:57.435 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:57 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-02T04:46:58.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:58 smithi027 ceph-mon[83036]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-02T04:46:58.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:58 smithi027 ceph-mon[83036]: Upgrade: osd.2 is safe to restart 2024-04-02T04:46:58.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:58 smithi027 ceph-mon[83036]: pgmap v93: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:58.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:58 smithi016 ceph-mon[93230]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-02T04:46:58.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:58 smithi016 ceph-mon[93230]: Upgrade: osd.2 is safe to restart 2024-04-02T04:46:58.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:58 smithi016 ceph-mon[93230]: pgmap v93: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:46:59.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:59 smithi016 ceph-mon[93230]: Upgrade: Updating osd.2 2024-04-02T04:46:59.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:59 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:59.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:59 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-02T04:46:59.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:46:59 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:46:59.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:59 smithi027 ceph-mon[83036]: Upgrade: Updating osd.2 2024-04-02T04:46:59.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:59 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:46:59.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:59 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-02T04:46:59.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:46:59 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:47:00.496 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:00 smithi016 ceph-mon[93230]: Deploying daemon osd.2 on smithi016 2024-04-02T04:47:00.496 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:00 smithi016 ceph-mon[93230]: pgmap v94: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:47:00.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:00 smithi027 ceph-mon[83036]: Deploying daemon osd.2 on smithi016 2024-04-02T04:47:00.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:00 smithi027 ceph-mon[83036]: pgmap v94: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:47:01.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.118+0000 7ff3fd62b700 1 -- 172.21.15.16:0/2106984430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3f80fedd0 msgr2=0x7ff3f80ff1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:01.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.118+0000 7ff3fd62b700 1 --2- 172.21.15.16:0/2106984430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3f80fedd0 0x7ff3f80ff1f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8009910 tx=0x7ff3e8009c20 comp rx=0 tx=0).stop 2024-04-02T04:47:01.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.118+0000 7ff3fd62b700 1 -- 172.21.15.16:0/2106984430 shutdown_connections 2024-04-02T04:47:01.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.118+0000 7ff3fd62b700 1 --2- 172.21.15.16:0/2106984430 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3f8101db0 0x7ff3f8102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:01.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.118+0000 7ff3fd62b700 1 --2- 172.21.15.16:0/2106984430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3f80fedd0 0x7ff3f80ff1f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:01.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.118+0000 7ff3fd62b700 1 -- 172.21.15.16:0/2106984430 >> 172.21.15.16:0/2106984430 conn(0x7ff3f80fa730 msgr2=0x7ff3f80fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:01.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.118+0000 7ff3fd62b700 1 -- 172.21.15.16:0/2106984430 shutdown_connections 2024-04-02T04:47:01.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.118+0000 7ff3fd62b700 1 -- 172.21.15.16:0/2106984430 wait complete. 2024-04-02T04:47:01.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.119+0000 7ff3fd62b700 1 Processor -- start 2024-04-02T04:47:01.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.119+0000 7ff3fd62b700 1 -- start start 2024-04-02T04:47:01.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.119+0000 7ff3fd62b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3f80fedd0 0x7ff3f81964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:01.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.119+0000 7ff3fd62b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3f8101db0 0x7ff3f8196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:01.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.119+0000 7ff3fd62b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f8197030 con 0x7ff3f80fedd0 2024-04-02T04:47:01.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.119+0000 7ff3fd62b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f8197170 con 0x7ff3f8101db0 2024-04-02T04:47:01.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.119+0000 7ff3f6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3f80fedd0 0x7ff3f81964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:01.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.119+0000 7ff3f6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3f80fedd0 0x7ff3f81964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44586/0 (socket says 172.21.15.16:44586) 2024-04-02T04:47:01.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.119+0000 7ff3f6ffd700 1 -- 172.21.15.16:0/3915404868 learned_addr learned my addr 172.21.15.16:0/3915404868 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:47:01.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.119+0000 7ff3f67fc700 1 --2- 172.21.15.16:0/3915404868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3f8101db0 0x7ff3f8196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:01.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.120+0000 7ff3f6ffd700 1 -- 172.21.15.16:0/3915404868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3f8101db0 msgr2=0x7ff3f8196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:01.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.120+0000 7ff3f6ffd700 1 --2- 172.21.15.16:0/3915404868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3f8101db0 0x7ff3f8196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:01.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.120+0000 7ff3f6ffd700 1 -- 172.21.15.16:0/3915404868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e80095f0 con 0x7ff3f80fedd0 2024-04-02T04:47:01.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.120+0000 7ff3f6ffd700 1 --2- 172.21.15.16:0/3915404868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3f80fedd0 0x7ff3f81964d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff3e80059d0 tx=0x7ff3e800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:01.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.120+0000 7ff3dffff700 1 -- 172.21.15.16:0/3915404868 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3e801d070 con 0x7ff3f80fedd0 2024-04-02T04:47:01.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.120+0000 7ff3dffff700 1 -- 172.21.15.16:0/3915404868 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff3e800f460 con 0x7ff3f80fedd0 2024-04-02T04:47:01.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.120+0000 7ff3fd62b700 1 -- 172.21.15.16:0/3915404868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3f819bb70 con 0x7ff3f80fedd0 2024-04-02T04:47:01.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.120+0000 7ff3dffff700 1 -- 172.21.15.16:0/3915404868 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3e8021640 con 0x7ff3f80fedd0 2024-04-02T04:47:01.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.120+0000 7ff3fd62b700 1 -- 172.21.15.16:0/3915404868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3f819c060 con 0x7ff3f80fedd0 2024-04-02T04:47:01.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.121+0000 7ff3fd62b700 1 -- 172.21.15.16:0/3915404868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3f8190690 con 0x7ff3f80fedd0 2024-04-02T04:47:01.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.122+0000 7ff3dffff700 1 -- 172.21.15.16:0/3915404868 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7ff3e800f5d0 con 0x7ff3f80fedd0 2024-04-02T04:47:01.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.122+0000 7ff3dffff700 1 --2- 172.21.15.16:0/3915404868 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ff3e006dd60 0x7ff3e0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:01.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.122+0000 7ff3dffff700 1 -- 172.21.15.16:0/3915404868 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6500+0+0 (secure 0 0 0) 0x7ff3e808ecd0 con 0x7ff3f80fedd0 2024-04-02T04:47:01.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.122+0000 7ff3f67fc700 1 --2- 172.21.15.16:0/3915404868 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ff3e006dd60 0x7ff3e0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:01.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.123+0000 7ff3f67fc700 1 --2- 172.21.15.16:0/3915404868 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ff3e006dd60 0x7ff3e0070220 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec007c60 tx=0x7ff3ec0073d0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:01.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.127+0000 7ff3dffff700 1 -- 172.21.15.16:0/3915404868 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff3e805be10 con 0x7ff3f80fedd0 2024-04-02T04:47:01.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.393+0000 7ff3fd62b700 1 -- 172.21.15.16:0/3915404868 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff3f802cdc0 con 0x7ff3e006dd60 2024-04-02T04:47:01.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.394+0000 7ff3dffff700 1 -- 172.21.15.16:0/3915404868 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7ff3f802cdc0 con 0x7ff3e006dd60 2024-04-02T04:47:01.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.396+0000 7ff3fd62b700 1 -- 172.21.15.16:0/3915404868 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ff3e006dd60 msgr2=0x7ff3e0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:01.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.396+0000 7ff3fd62b700 1 --2- 172.21.15.16:0/3915404868 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ff3e006dd60 0x7ff3e0070220 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec007c60 tx=0x7ff3ec0073d0 comp rx=0 tx=0).stop 2024-04-02T04:47:01.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.396+0000 7ff3fd62b700 1 -- 172.21.15.16:0/3915404868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3f80fedd0 msgr2=0x7ff3f81964d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:01.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.396+0000 7ff3fd62b700 1 --2- 172.21.15.16:0/3915404868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3f80fedd0 0x7ff3f81964d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff3e80059d0 tx=0x7ff3e800bd70 comp rx=0 tx=0).stop 2024-04-02T04:47:01.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.396+0000 7ff3fd62b700 1 -- 172.21.15.16:0/3915404868 shutdown_connections 2024-04-02T04:47:01.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.396+0000 7ff3fd62b700 1 --2- 172.21.15.16:0/3915404868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3f80fedd0 0x7ff3f81964d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:01.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.396+0000 7ff3fd62b700 1 --2- 172.21.15.16:0/3915404868 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ff3e006dd60 0x7ff3e0070220 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:01.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.396+0000 7ff3fd62b700 1 --2- 172.21.15.16:0/3915404868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3f8101db0 0x7ff3f8196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:01.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.396+0000 7ff3fd62b700 1 -- 172.21.15.16:0/3915404868 >> 172.21.15.16:0/3915404868 conn(0x7ff3f80fa730 msgr2=0x7ff3f8105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:01.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.396+0000 7ff3fd62b700 1 -- 172.21.15.16:0/3915404868 shutdown_connections 2024-04-02T04:47:01.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.396+0000 7ff3fd62b700 1 -- 172.21.15.16:0/3915404868 wait complete. 2024-04-02T04:47:01.410 INFO:teuthology.orchestra.run.smithi016.stdout:true 2024-04-02T04:47:01.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.506+0000 7fd5797c3700 1 -- 172.21.15.16:0/2606256364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd574103e90 msgr2=0x7fd574106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:01.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.506+0000 7fd5797c3700 1 --2- 172.21.15.16:0/2606256364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd574103e90 0x7fd574106280 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd568009910 tx=0x7fd568009c20 comp rx=0 tx=0).stop 2024-04-02T04:47:01.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.506+0000 7fd5797c3700 1 -- 172.21.15.16:0/2606256364 shutdown_connections 2024-04-02T04:47:01.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.506+0000 7fd5797c3700 1 --2- 172.21.15.16:0/2606256364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd574103e90 0x7fd574106280 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:01.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.506+0000 7fd5797c3700 1 --2- 172.21.15.16:0/2606256364 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd574101560 0x7fd574103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:01.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.506+0000 7fd5797c3700 1 -- 172.21.15.16:0/2606256364 >> 172.21.15.16:0/2606256364 conn(0x7fd5740faf70 msgr2=0x7fd5740fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:01.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.506+0000 7fd5797c3700 1 -- 172.21.15.16:0/2606256364 shutdown_connections 2024-04-02T04:47:01.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.506+0000 7fd5797c3700 1 -- 172.21.15.16:0/2606256364 wait complete. 2024-04-02T04:47:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.507+0000 7fd5797c3700 1 Processor -- start 2024-04-02T04:47:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.507+0000 7fd5797c3700 1 -- start start 2024-04-02T04:47:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.507+0000 7fd5797c3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd574101560 0x7fd574196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.508+0000 7fd5797c3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd574103e90 0x7fd5741969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.508+0000 7fd5797c3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd574196ff0 con 0x7fd574101560 2024-04-02T04:47:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.508+0000 7fd5797c3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd574197130 con 0x7fd574103e90 2024-04-02T04:47:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.508+0000 7fd572ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd574101560 0x7fd574196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.508+0000 7fd572ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd574101560 0x7fd574196490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44606/0 (socket says 172.21.15.16:44606) 2024-04-02T04:47:01.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.508+0000 7fd572ffd700 1 -- 172.21.15.16:0/2286213857 learned_addr learned my addr 172.21.15.16:0/2286213857 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:47:01.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.508+0000 7fd5727fc700 1 --2- 172.21.15.16:0/2286213857 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd574103e90 0x7fd5741969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:01.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.508+0000 7fd572ffd700 1 -- 172.21.15.16:0/2286213857 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd574103e90 msgr2=0x7fd5741969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:01.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.508+0000 7fd572ffd700 1 --2- 172.21.15.16:0/2286213857 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd574103e90 0x7fd5741969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:01.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.508+0000 7fd572ffd700 1 -- 172.21.15.16:0/2286213857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5680095f0 con 0x7fd574101560 2024-04-02T04:47:01.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.508+0000 7fd572ffd700 1 --2- 172.21.15.16:0/2286213857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd574101560 0x7fd574196490 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd56400d8d0 tx=0x7fd56400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:01.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.508+0000 7fd55bfff700 1 -- 172.21.15.16:0/2286213857 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5640098a0 con 0x7fd574101560 2024-04-02T04:47:01.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.508+0000 7fd5797c3700 1 -- 172.21.15.16:0/2286213857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd57419bb90 con 0x7fd574101560 2024-04-02T04:47:01.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.508+0000 7fd55bfff700 1 -- 172.21.15.16:0/2286213857 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd564010460 con 0x7fd574101560 2024-04-02T04:47:01.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.509+0000 7fd55bfff700 1 -- 172.21.15.16:0/2286213857 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd56400f5f0 con 0x7fd574101560 2024-04-02T04:47:01.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.509+0000 7fd5797c3700 1 -- 172.21.15.16:0/2286213857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd57419c0e0 con 0x7fd574101560 2024-04-02T04:47:01.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.510+0000 7fd5797c3700 1 -- 172.21.15.16:0/2286213857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd57404ea80 con 0x7fd574101560 2024-04-02T04:47:01.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.510+0000 7fd55bfff700 1 -- 172.21.15.16:0/2286213857 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fd564009a00 con 0x7fd574101560 2024-04-02T04:47:01.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.510+0000 7fd55bfff700 1 --2- 172.21.15.16:0/2286213857 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd55c06dd60 0x7fd55c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:01.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.511+0000 7fd55bfff700 1 -- 172.21.15.16:0/2286213857 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fd56408d920 con 0x7fd574101560 2024-04-02T04:47:01.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.511+0000 7fd5727fc700 1 --2- 172.21.15.16:0/2286213857 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd55c06dd60 0x7fd55c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:01.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.511+0000 7fd5727fc700 1 --2- 172.21.15.16:0/2286213857 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd55c06dd60 0x7fd55c070220 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd568005090 tx=0x7fd568005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:01.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.516+0000 7fd55bfff700 1 -- 172.21.15.16:0/2286213857 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd56405aa60 con 0x7fd574101560 2024-04-02T04:47:01.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:01 smithi027 ceph-mon[83036]: pgmap v95: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:47:01.739 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:01 smithi016 ceph-mon[93230]: pgmap v95: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:47:01.739 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:01 smithi016 systemd[1]: Stopping Ceph osd.2 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4... 2024-04-02T04:47:01.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.753+0000 7fd5797c3700 1 -- 172.21.15.16:0/2286213857 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd574105d70 con 0x7fd55c06dd60 2024-04-02T04:47:01.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.755+0000 7fd55bfff700 1 -- 172.21.15.16:0/2286213857 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fd574105d70 con 0x7fd55c06dd60 2024-04-02T04:47:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.756+0000 7fd5797c3700 1 -- 172.21.15.16:0/2286213857 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd55c06dd60 msgr2=0x7fd55c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.756+0000 7fd5797c3700 1 --2- 172.21.15.16:0/2286213857 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd55c06dd60 0x7fd55c070220 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd568005090 tx=0x7fd568005960 comp rx=0 tx=0).stop 2024-04-02T04:47:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.756+0000 7fd5797c3700 1 -- 172.21.15.16:0/2286213857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd574101560 msgr2=0x7fd574196490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.756+0000 7fd5797c3700 1 --2- 172.21.15.16:0/2286213857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd574101560 0x7fd574196490 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd56400d8d0 tx=0x7fd56400dbe0 comp rx=0 tx=0).stop 2024-04-02T04:47:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.757+0000 7fd5797c3700 1 -- 172.21.15.16:0/2286213857 shutdown_connections 2024-04-02T04:47:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.757+0000 7fd5797c3700 1 --2- 172.21.15.16:0/2286213857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd574101560 0x7fd574196490 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.757+0000 7fd5797c3700 1 --2- 172.21.15.16:0/2286213857 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd55c06dd60 0x7fd55c070220 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.757+0000 7fd5797c3700 1 --2- 172.21.15.16:0/2286213857 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd574103e90 0x7fd5741969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.757+0000 7fd5797c3700 1 -- 172.21.15.16:0/2286213857 >> 172.21.15.16:0/2286213857 conn(0x7fd5740faf70 msgr2=0x7fd5740fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.757+0000 7fd5797c3700 1 -- 172.21.15.16:0/2286213857 shutdown_connections 2024-04-02T04:47:01.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.757+0000 7fd5797c3700 1 -- 172.21.15.16:0/2286213857 wait complete. 2024-04-02T04:47:01.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.862+0000 7f2e823d9700 1 -- 172.21.15.16:0/4097973863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e7c100ae0 msgr2=0x7f2e7c100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:01.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.862+0000 7f2e823d9700 1 --2- 172.21.15.16:0/4097973863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e7c100ae0 0x7f2e7c100f00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c009910 tx=0x7f2e6c009c20 comp rx=0 tx=0).stop 2024-04-02T04:47:01.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.862+0000 7f2e823d9700 1 -- 172.21.15.16:0/4097973863 shutdown_connections 2024-04-02T04:47:01.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.862+0000 7f2e823d9700 1 --2- 172.21.15.16:0/4097973863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e7c101ce0 0x7f2e7c102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:01.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.862+0000 7f2e823d9700 1 --2- 172.21.15.16:0/4097973863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e7c100ae0 0x7f2e7c100f00 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:01.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.862+0000 7f2e823d9700 1 -- 172.21.15.16:0/4097973863 >> 172.21.15.16:0/4097973863 conn(0x7f2e7c0fc0a0 msgr2=0x7f2e7c0fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:01.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.862+0000 7f2e823d9700 1 -- 172.21.15.16:0/4097973863 shutdown_connections 2024-04-02T04:47:01.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.862+0000 7f2e823d9700 1 -- 172.21.15.16:0/4097973863 wait complete. 2024-04-02T04:47:01.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.863+0000 7f2e823d9700 1 Processor -- start 2024-04-02T04:47:01.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.863+0000 7f2e823d9700 1 -- start start 2024-04-02T04:47:01.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.863+0000 7f2e823d9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e7c100ae0 0x7f2e7c1964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:01.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.863+0000 7f2e823d9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e7c101ce0 0x7f2e7c196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:01.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.863+0000 7f2e823d9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7c197020 con 0x7f2e7c100ae0 2024-04-02T04:47:01.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.863+0000 7f2e823d9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7c197160 con 0x7f2e7c101ce0 2024-04-02T04:47:01.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.863+0000 7f2e7bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e7c100ae0 0x7f2e7c1964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:01.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.863+0000 7f2e7bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e7c100ae0 0x7f2e7c1964c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59582/0 (socket says 172.21.15.16:59582) 2024-04-02T04:47:01.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.863+0000 7f2e7bfff700 1 -- 172.21.15.16:0/1818309360 learned_addr learned my addr 172.21.15.16:0/1818309360 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:47:01.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.863+0000 7f2e7b7fe700 1 --2- 172.21.15.16:0/1818309360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e7c101ce0 0x7f2e7c196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:01.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.863+0000 7f2e7bfff700 1 -- 172.21.15.16:0/1818309360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e7c101ce0 msgr2=0x7f2e7c196a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:01.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.863+0000 7f2e7bfff700 1 --2- 172.21.15.16:0/1818309360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e7c101ce0 0x7f2e7c196a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:01.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.863+0000 7f2e7bfff700 1 -- 172.21.15.16:0/1818309360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e6c0095f0 con 0x7f2e7c100ae0 2024-04-02T04:47:01.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.864+0000 7f2e7bfff700 1 --2- 172.21.15.16:0/1818309360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e7c100ae0 0x7f2e7c1964c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c00b620 tx=0x7f2e6c00f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:01.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.864+0000 7f2e797fa700 1 -- 172.21.15.16:0/1818309360 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e6c01d070 con 0x7f2e7c100ae0 2024-04-02T04:47:01.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.864+0000 7f2e797fa700 1 -- 172.21.15.16:0/1818309360 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2e6c0052a0 con 0x7f2e7c100ae0 2024-04-02T04:47:01.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.864+0000 7f2e823d9700 1 -- 172.21.15.16:0/1818309360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e7c19bb60 con 0x7f2e7c100ae0 2024-04-02T04:47:01.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.864+0000 7f2e797fa700 1 -- 172.21.15.16:0/1818309360 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e6c017400 con 0x7f2e7c100ae0 2024-04-02T04:47:01.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.864+0000 7f2e823d9700 1 -- 172.21.15.16:0/1818309360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e7c19c050 con 0x7f2e7c100ae0 2024-04-02T04:47:01.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.865+0000 7f2e823d9700 1 -- 172.21.15.16:0/1818309360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e7c190660 con 0x7f2e7c100ae0 2024-04-02T04:47:01.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.865+0000 7f2e797fa700 1 -- 172.21.15.16:0/1818309360 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f2e6c003810 con 0x7f2e7c100ae0 2024-04-02T04:47:01.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.866+0000 7f2e797fa700 1 --2- 172.21.15.16:0/1818309360 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f2e6406dd70 0x7f2e64070230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:01.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.866+0000 7f2e797fa700 1 -- 172.21.15.16:0/1818309360 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f2e6c08eec0 con 0x7f2e7c100ae0 2024-04-02T04:47:01.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.866+0000 7f2e7b7fe700 1 --2- 172.21.15.16:0/1818309360 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f2e6406dd70 0x7f2e64070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:01.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.866+0000 7f2e7b7fe700 1 --2- 172.21.15.16:0/1818309360 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f2e6406dd70 0x7f2e64070230 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2e70009bf0 tx=0x7f2e70009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:01.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:01.870+0000 7f2e797fa700 1 -- 172.21.15.16:0/1818309360 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2e6c05c000 con 0x7f2e7c100ae0 2024-04-02T04:47:02.059 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:01 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2[63084]: 2024-04-02T04:47:01.735+0000 7f4e2408e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T04:47:02.060 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:01 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2[63084]: 2024-04-02T04:47:01.735+0000 7f4e2408e700 -1 osd.2 59 *** Got signal Terminated *** 2024-04-02T04:47:02.060 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:01 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2[63084]: 2024-04-02T04:47:01.735+0000 7f4e2408e700 -1 osd.2 59 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T04:47:02.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.085+0000 7f2e823d9700 1 -- 172.21.15.16:0/1818309360 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2e7c02ce60 con 0x7f2e6406dd70 2024-04-02T04:47:02.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.097+0000 7f2e797fa700 1 -- 172.21.15.16:0/1818309360 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f2e7c02ce60 con 0x7f2e6406dd70 2024-04-02T04:47:02.101 INFO:teuthology.orchestra.run.smithi016.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:47:02.101 INFO:teuthology.orchestra.run.smithi016.stdout:alertmanager.smithi016 smithi016 *:9093,9094 running (11m) 25s ago 12m 24.1M - 0.25.0 c8568f914cd2 b95d8354b016 2024-04-02T04:47:02.101 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi016 smithi016 running (13m) 25s ago 13m 8052k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 9de6c16a487f 2024-04-02T04:47:02.101 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi027 smithi027 running (12m) 82s ago 12m 7964k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 99956592471a 2024-04-02T04:47:02.101 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi016 smithi016 running (91s) 25s ago 13m 6660k - 19.0.0-1979-g6cec793d 78f830190839 dc63ce0eef58 2024-04-02T04:47:02.101 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi027 smithi027 running (86s) 82s ago 12m 6665k - 19.0.0-1979-g6cec793d 78f830190839 2beb077018e4 2024-04-02T04:47:02.101 INFO:teuthology.orchestra.run.smithi016.stdout:grafana.smithi016 smithi016 *:3000 running (11m) 25s ago 12m 85.1M - 9.4.7 954c08fa6188 7c43d133569c 2024-04-02T04:47:02.101 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.fyzrnm smithi016 running (7m) 25s ago 7m 17.7M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 70f28b9572a6 2024-04-02T04:47:02.101 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.uhgkgg smithi016 running (7m) 25s ago 7m 15.9M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 0aa4d16562a6 2024-04-02T04:47:02.101 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.sdublg smithi027 running (7m) 82s ago 7m 19.3M - 18.2.2-1059-g4d05f4ca 909dce4fe53d a538451099db 2024-04-02T04:47:02.101 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.zwrfoo smithi027 running (7m) 82s ago 7m 158M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 4cb3fa428d71 2024-04-02T04:47:02.101 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi016.eqnebx smithi016 *:8443,9283,8765 running (4m) 25s ago 14m 499M - 19.0.0-1979-g6cec793d 78f830190839 d61ed91b4577 2024-04-02T04:47:02.101 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi027.kjzcsc smithi027 *:8443,9283,8765 running (3m) 82s ago 11m 449M - 19.0.0-1979-g6cec793d 78f830190839 5072a5ffc761 2024-04-02T04:47:02.101 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi016 smithi016 running (2m) 25s ago 14m 45.9M 2048M 19.0.0-1979-g6cec793d 78f830190839 eabfc84f1cdf 2024-04-02T04:47:02.102 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi027 smithi027 running (2m) 82s ago 11m 36.8M 2048M 19.0.0-1979-g6cec793d 78f830190839 fcd68b20dcc5 2024-04-02T04:47:02.102 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi016 smithi016 *:9100 running (13m) 25s ago 13m 14.8M - 1.5.0 0da6a335fe13 09f54c65b53c 2024-04-02T04:47:02.102 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi027 smithi027 *:9100 running (11m) 82s ago 11m 15.2M - 1.5.0 0da6a335fe13 e41a657c7443 2024-04-02T04:47:02.102 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 smithi016 running (66s) 25s ago 11m 81.0M 1217M 19.0.0-1979-g6cec793d 78f830190839 421d9e1cc751 2024-04-02T04:47:02.102 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 smithi016 running (30s) 25s ago 10m 15.1M 1217M 19.0.0-1979-g6cec793d 78f830190839 3612a56236a3 2024-04-02T04:47:02.102 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 smithi016 running (10m) 25s ago 10m 180M 1217M 18.2.2-1059-g4d05f4ca 909dce4fe53d 52e592b4d1f2 2024-04-02T04:47:02.102 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 smithi027 running (9m) 82s ago 9m 236M 2240M 18.2.2-1059-g4d05f4ca 909dce4fe53d cf7c7fe022f0 2024-04-02T04:47:02.102 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 smithi027 running (9m) 82s ago 9m 211M 2240M 18.2.2-1059-g4d05f4ca 909dce4fe53d 8cbbedf8cac9 2024-04-02T04:47:02.102 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 smithi027 running (8m) 82s ago 8m 194M 2240M 18.2.2-1059-g4d05f4ca 909dce4fe53d ac6094415529 2024-04-02T04:47:02.102 INFO:teuthology.orchestra.run.smithi016.stdout:prometheus.smithi016 smithi016 *:9095 running (3m) 25s ago 12m 58.4M - 2.43.0 a07b618ecd1d 466f00d73ddd 2024-04-02T04:47:02.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.099+0000 7f2e823d9700 1 -- 172.21.15.16:0/1818309360 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f2e6406dd70 msgr2=0x7f2e64070230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:02.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.099+0000 7f2e823d9700 1 --2- 172.21.15.16:0/1818309360 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f2e6406dd70 0x7f2e64070230 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2e70009bf0 tx=0x7f2e70009260 comp rx=0 tx=0).stop 2024-04-02T04:47:02.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.099+0000 7f2e823d9700 1 -- 172.21.15.16:0/1818309360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e7c100ae0 msgr2=0x7f2e7c1964c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:02.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.099+0000 7f2e823d9700 1 --2- 172.21.15.16:0/1818309360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e7c100ae0 0x7f2e7c1964c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c00b620 tx=0x7f2e6c00f510 comp rx=0 tx=0).stop 2024-04-02T04:47:02.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.099+0000 7f2e823d9700 1 -- 172.21.15.16:0/1818309360 shutdown_connections 2024-04-02T04:47:02.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.100+0000 7f2e823d9700 1 --2- 172.21.15.16:0/1818309360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e7c100ae0 0x7f2e7c1964c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:02.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.100+0000 7f2e823d9700 1 --2- 172.21.15.16:0/1818309360 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f2e6406dd70 0x7f2e64070230 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:02.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.100+0000 7f2e823d9700 1 --2- 172.21.15.16:0/1818309360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e7c101ce0 0x7f2e7c196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:02.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.100+0000 7f2e823d9700 1 -- 172.21.15.16:0/1818309360 >> 172.21.15.16:0/1818309360 conn(0x7f2e7c0fc0a0 msgr2=0x7f2e7c1052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:02.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.100+0000 7f2e823d9700 1 -- 172.21.15.16:0/1818309360 shutdown_connections 2024-04-02T04:47:02.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.100+0000 7f2e823d9700 1 -- 172.21.15.16:0/1818309360 wait complete. 2024-04-02T04:47:02.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.204+0000 7f835da97700 1 -- 172.21.15.16:0/508230807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83580fedf0 msgr2=0x7f83580ff210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:02.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.204+0000 7f835da97700 1 --2- 172.21.15.16:0/508230807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83580fedf0 0x7f83580ff210 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8348009910 tx=0x7f8348009c20 comp rx=0 tx=0).stop 2024-04-02T04:47:02.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.204+0000 7f835da97700 1 -- 172.21.15.16:0/508230807 shutdown_connections 2024-04-02T04:47:02.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.204+0000 7f835da97700 1 --2- 172.21.15.16:0/508230807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8358101e70 0x7f83581022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:02.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.204+0000 7f835da97700 1 --2- 172.21.15.16:0/508230807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83580fedf0 0x7f83580ff210 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:02.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.204+0000 7f835da97700 1 -- 172.21.15.16:0/508230807 >> 172.21.15.16:0/508230807 conn(0x7f83580fa730 msgr2=0x7f83580fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:02.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.205+0000 7f835da97700 1 -- 172.21.15.16:0/508230807 shutdown_connections 2024-04-02T04:47:02.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.205+0000 7f835da97700 1 -- 172.21.15.16:0/508230807 wait complete. 2024-04-02T04:47:02.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.205+0000 7f835da97700 1 Processor -- start 2024-04-02T04:47:02.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.205+0000 7f835da97700 1 -- start start 2024-04-02T04:47:02.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.205+0000 7f835da97700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83580fedf0 0x7f8358198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:02.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.205+0000 7f835da97700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8358101e70 0x7f8358198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:02.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.205+0000 7f835da97700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8358199460 con 0x7f8358101e70 2024-04-02T04:47:02.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.205+0000 7f835da97700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83581995a0 con 0x7f83580fedf0 2024-04-02T04:47:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.206+0000 7f8356ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8358101e70 0x7f8358198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.206+0000 7f8356ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8358101e70 0x7f8358198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59594/0 (socket says 172.21.15.16:59594) 2024-04-02T04:47:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.206+0000 7f8356ffd700 1 -- 172.21.15.16:0/1969395893 learned_addr learned my addr 172.21.15.16:0/1969395893 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:47:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.206+0000 7f83577fe700 1 --2- 172.21.15.16:0/1969395893 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83580fedf0 0x7f8358198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.206+0000 7f8356ffd700 1 -- 172.21.15.16:0/1969395893 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83580fedf0 msgr2=0x7f8358198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.206+0000 7f8356ffd700 1 --2- 172.21.15.16:0/1969395893 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83580fedf0 0x7f8358198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.206+0000 7f8356ffd700 1 -- 172.21.15.16:0/1969395893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83480095f0 con 0x7f8358101e70 2024-04-02T04:47:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.206+0000 7f8356ffd700 1 --2- 172.21.15.16:0/1969395893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8358101e70 0x7f8358198e40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f834c00b4b0 tx=0x7f834c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.206+0000 7f8354ff9700 1 -- 172.21.15.16:0/1969395893 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f834c011800 con 0x7f8358101e70 2024-04-02T04:47:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.206+0000 7f8354ff9700 1 -- 172.21.15.16:0/1969395893 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f834c011e20 con 0x7f8358101e70 2024-04-02T04:47:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.206+0000 7f835da97700 1 -- 172.21.15.16:0/1969395893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f835819dfb0 con 0x7f8358101e70 2024-04-02T04:47:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.206+0000 7f8354ff9700 1 -- 172.21.15.16:0/1969395893 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f834c00f2e0 con 0x7f8358101e70 2024-04-02T04:47:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.206+0000 7f835da97700 1 -- 172.21.15.16:0/1969395893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f835819e550 con 0x7f8358101e70 2024-04-02T04:47:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.207+0000 7f835da97700 1 -- 172.21.15.16:0/1969395893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8358106840 con 0x7f8358101e70 2024-04-02T04:47:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.208+0000 7f8354ff9700 1 -- 172.21.15.16:0/1969395893 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f834c010380 con 0x7f8358101e70 2024-04-02T04:47:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.208+0000 7f8354ff9700 1 --2- 172.21.15.16:0/1969395893 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f834006dd60 0x7f8340070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.208+0000 7f8354ff9700 1 -- 172.21.15.16:0/1969395893 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f834c08d680 con 0x7f8358101e70 2024-04-02T04:47:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.208+0000 7f83577fe700 1 --2- 172.21.15.16:0/1969395893 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f834006dd60 0x7f8340070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.209+0000 7f83577fe700 1 --2- 172.21.15.16:0/1969395893 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f834006dd60 0x7f8340070220 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f83480051c0 tx=0x7f834801a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:02.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.213+0000 7f8354ff9700 1 -- 172.21.15.16:0/1969395893 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f834c05a740 con 0x7f8358101e70 2024-04-02T04:47:02.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.514+0000 7f835da97700 1 -- 172.21.15.16:0/1969395893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f835804ea80 con 0x7f8358101e70 2024-04-02T04:47:02.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.515+0000 7f8354ff9700 1 -- 172.21.15.16:0/1969395893 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+836 (secure 0 0 0) 0x7f834c05a2d0 con 0x7f8358101e70 2024-04-02T04:47:02.518 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:47:02.518 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": { 2024-04-02T04:47:02.519 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:47:02.519 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:47:02.519 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": { 2024-04-02T04:47:02.519 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:47:02.519 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:47:02.519 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": { 2024-04-02T04:47:02.519 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 3, 2024-04-02T04:47:02.519 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:47:02.519 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:47:02.519 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-04-02T04:47:02.519 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 4 2024-04-02T04:47:02.519 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:47:02.519 INFO:teuthology.orchestra.run.smithi016.stdout: "overall": { 2024-04-02T04:47:02.520 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 7, 2024-04-02T04:47:02.520 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 6 2024-04-02T04:47:02.520 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-02T04:47:02.520 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:47:02.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.517+0000 7f835da97700 1 -- 172.21.15.16:0/1969395893 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f834006dd60 msgr2=0x7f8340070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.517+0000 7f835da97700 1 --2- 172.21.15.16:0/1969395893 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f834006dd60 0x7f8340070220 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f83480051c0 tx=0x7f834801a040 comp rx=0 tx=0).stop 2024-04-02T04:47:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.517+0000 7f835da97700 1 -- 172.21.15.16:0/1969395893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8358101e70 msgr2=0x7f8358198e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.517+0000 7f835da97700 1 --2- 172.21.15.16:0/1969395893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8358101e70 0x7f8358198e40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f834c00b4b0 tx=0x7f834c00b7c0 comp rx=0 tx=0).stop 2024-04-02T04:47:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.517+0000 7f835da97700 1 -- 172.21.15.16:0/1969395893 shutdown_connections 2024-04-02T04:47:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.517+0000 7f835da97700 1 --2- 172.21.15.16:0/1969395893 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f834006dd60 0x7f8340070220 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.517+0000 7f835da97700 1 --2- 172.21.15.16:0/1969395893 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83580fedf0 0x7f8358198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.517+0000 7f835da97700 1 --2- 172.21.15.16:0/1969395893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8358101e70 0x7f8358198e40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.517+0000 7f835da97700 1 -- 172.21.15.16:0/1969395893 >> 172.21.15.16:0/1969395893 conn(0x7f83580fa730 msgr2=0x7f8358100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.517+0000 7f835da97700 1 -- 172.21.15.16:0/1969395893 shutdown_connections 2024-04-02T04:47:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.517+0000 7f835da97700 1 -- 172.21.15.16:0/1969395893 wait complete. 2024-04-02T04:47:02.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.623+0000 7fbf5fd4c700 1 -- 172.21.15.16:0/1663201873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf58103eb0 msgr2=0x7fbf581062a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:02.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.623+0000 7fbf5fd4c700 1 --2- 172.21.15.16:0/1663201873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf58103eb0 0x7fbf581062a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c009910 tx=0x7fbf4c009c20 comp rx=0 tx=0).stop 2024-04-02T04:47:02.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.623+0000 7fbf5fd4c700 1 -- 172.21.15.16:0/1663201873 shutdown_connections 2024-04-02T04:47:02.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.623+0000 7fbf5fd4c700 1 --2- 172.21.15.16:0/1663201873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf58103eb0 0x7fbf581062a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:02.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.623+0000 7fbf5fd4c700 1 --2- 172.21.15.16:0/1663201873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf58101580 0x7fbf58103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:02.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.623+0000 7fbf5fd4c700 1 -- 172.21.15.16:0/1663201873 >> 172.21.15.16:0/1663201873 conn(0x7fbf580faf90 msgr2=0x7fbf580fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:02.627 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:02 smithi016 ceph-mon[93230]: from='client.34280 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:47:02.627 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:02 smithi016 ceph-mon[93230]: osd.2 marked itself down and dead 2024-04-02T04:47:02.627 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:02 smithi016 ceph-mon[93230]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:47:02.627 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:02 smithi016 ceph-mon[93230]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:47:02.627 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:02 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:47:02.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.623+0000 7fbf5fd4c700 1 -- 172.21.15.16:0/1663201873 shutdown_connections 2024-04-02T04:47:02.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.623+0000 7fbf5fd4c700 1 -- 172.21.15.16:0/1663201873 wait complete. 2024-04-02T04:47:02.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.624+0000 7fbf5fd4c700 1 Processor -- start 2024-04-02T04:47:02.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.624+0000 7fbf5fd4c700 1 -- start start 2024-04-02T04:47:02.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.624+0000 7fbf5fd4c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf58101580 0x7fbf58100e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:02.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.624+0000 7fbf5fd4c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf58103eb0 0x7fbf580ff530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:02.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.624+0000 7fbf5fd4c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf580ffa70 con 0x7fbf58103eb0 2024-04-02T04:47:02.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.624+0000 7fbf5fd4c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf580ffbb0 con 0x7fbf58101580 2024-04-02T04:47:02.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.624+0000 7fbf5d2e7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf58103eb0 0x7fbf580ff530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:02.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.624+0000 7fbf5d2e7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf58103eb0 0x7fbf580ff530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59604/0 (socket says 172.21.15.16:59604) 2024-04-02T04:47:02.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.624+0000 7fbf5d2e7700 1 -- 172.21.15.16:0/1839817476 learned_addr learned my addr 172.21.15.16:0/1839817476 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:47:02.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.624+0000 7fbf5dae8700 1 --2- 172.21.15.16:0/1839817476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf58101580 0x7fbf58100e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:02.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.624+0000 7fbf5d2e7700 1 -- 172.21.15.16:0/1839817476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf58101580 msgr2=0x7fbf58100e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:02.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.624+0000 7fbf5d2e7700 1 --2- 172.21.15.16:0/1839817476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf58101580 0x7fbf58100e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:02.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.624+0000 7fbf5d2e7700 1 -- 172.21.15.16:0/1839817476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf4c0095f0 con 0x7fbf58103eb0 2024-04-02T04:47:02.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.624+0000 7fbf5d2e7700 1 --2- 172.21.15.16:0/1839817476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf58103eb0 0x7fbf580ff530 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c006010 tx=0x7fbf4c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:02.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.625+0000 7fbf56ffd700 1 -- 172.21.15.16:0/1839817476 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf4c01d070 con 0x7fbf58103eb0 2024-04-02T04:47:02.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.625+0000 7fbf56ffd700 1 -- 172.21.15.16:0/1839817476 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf4c00f460 con 0x7fbf58103eb0 2024-04-02T04:47:02.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.625+0000 7fbf5fd4c700 1 -- 172.21.15.16:0/1839817476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf580ffe30 con 0x7fbf58103eb0 2024-04-02T04:47:02.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.625+0000 7fbf56ffd700 1 -- 172.21.15.16:0/1839817476 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf4c017610 con 0x7fbf58103eb0 2024-04-02T04:47:02.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.625+0000 7fbf5fd4c700 1 -- 172.21.15.16:0/1839817476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf581003a0 con 0x7fbf58103eb0 2024-04-02T04:47:02.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.626+0000 7fbf5fd4c700 1 -- 172.21.15.16:0/1839817476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf5804ea80 con 0x7fbf58103eb0 2024-04-02T04:47:02.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.626+0000 7fbf56ffd700 1 -- 172.21.15.16:0/1839817476 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fbf4c00f5d0 con 0x7fbf58103eb0 2024-04-02T04:47:02.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.627+0000 7fbf56ffd700 1 --2- 172.21.15.16:0/1839817476 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fbf4006dd70 0x7fbf40070230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:02.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.627+0000 7fbf56ffd700 1 -- 172.21.15.16:0/1839817476 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fbf4c08ed00 con 0x7fbf58103eb0 2024-04-02T04:47:02.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.627+0000 7fbf5dae8700 1 --2- 172.21.15.16:0/1839817476 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fbf4006dd70 0x7fbf40070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:02.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.627+0000 7fbf5dae8700 1 --2- 172.21.15.16:0/1839817476 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fbf4006dd70 0x7fbf40070230 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbf48009550 tx=0x7fbf480091b0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:02.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.631+0000 7fbf56ffd700 1 -- 172.21.15.16:0/1839817476 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbf4c092030 con 0x7fbf58103eb0 2024-04-02T04:47:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:02 smithi027 ceph-mon[83036]: from='client.34280 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:47:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:02 smithi027 ceph-mon[83036]: osd.2 marked itself down and dead 2024-04-02T04:47:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:02 smithi027 ceph-mon[83036]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:47:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:02 smithi027 ceph-mon[83036]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:47:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:02 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:47:02.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.896+0000 7fbf5fd4c700 1 -- 172.21.15.16:0/1839817476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbf581003a0 con 0x7fbf58103eb0 2024-04-02T04:47:02.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.897+0000 7fbf56ffd700 1 -- 172.21.15.16:0/1839817476 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1875 (secure 0 0 0) 0x7fbf581003a0 con 0x7fbf58103eb0 2024-04-02T04:47:02.900 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:02 smithi016 podman[113075]: 2024-04-02 04:47:02.803139868 +0000 UTC m=+1.163680181 container died 52e592b4d1f20412b4b1a2f21b34acb418314149b728952f76b54ee4a01fece8 (image=quay.ceph.io/ceph-ci/ceph@sha256:397d0c1d908c002c558906e3ea05982bc336d403ee4d900400cc3bdd9243aa2e, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2, org.label-schema.license=GPLv2, RELEASE=reef-4d05f4c, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T04:47:02.900 INFO:teuthology.orchestra.run.smithi016.stdout:e15 2024-04-02T04:47:02.901 INFO:teuthology.orchestra.run.smithi016.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:47:02.901 INFO:teuthology.orchestra.run.smithi016.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:47:02.901 INFO:teuthology.orchestra.run.smithi016.stdout:legacy client fscid: 1 2024-04-02T04:47:02.901 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:47:02.901 INFO:teuthology.orchestra.run.smithi016.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:47:02.901 INFO:teuthology.orchestra.run.smithi016.stdout:fs_name cephfs 2024-04-02T04:47:02.901 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 15 2024-04-02T04:47:02.901 INFO:teuthology.orchestra.run.smithi016.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:47:02.901 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-04-02T04:39:30.740168+0000 2024-04-02T04:47:02.901 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-04-02T04:39:49.875319+0000 2024-04-02T04:47:02.901 INFO:teuthology.orchestra.run.smithi016.stdout:tableserver 0 2024-04-02T04:47:02.901 INFO:teuthology.orchestra.run.smithi016.stdout:root 0 2024-04-02T04:47:02.901 INFO:teuthology.orchestra.run.smithi016.stdout:session_timeout 60 2024-04-02T04:47:02.901 INFO:teuthology.orchestra.run.smithi016.stdout:session_autoclose 300 2024-04-02T04:47:02.902 INFO:teuthology.orchestra.run.smithi016.stdout:max_file_size 1099511627776 2024-04-02T04:47:02.902 INFO:teuthology.orchestra.run.smithi016.stdout:max_xattr_size 65536 2024-04-02T04:47:02.902 INFO:teuthology.orchestra.run.smithi016.stdout:required_client_features {} 2024-04-02T04:47:02.902 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure 0 2024-04-02T04:47:02.902 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure_osd_epoch 46 2024-04-02T04:47:02.902 INFO:teuthology.orchestra.run.smithi016.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:47:02.902 INFO:teuthology.orchestra.run.smithi016.stdout:max_mds 1 2024-04-02T04:47:02.902 INFO:teuthology.orchestra.run.smithi016.stdout:in 0 2024-04-02T04:47:02.902 INFO:teuthology.orchestra.run.smithi016.stdout:up {0=24285} 2024-04-02T04:47:02.902 INFO:teuthology.orchestra.run.smithi016.stdout:failed 2024-04-02T04:47:02.902 INFO:teuthology.orchestra.run.smithi016.stdout:damaged 2024-04-02T04:47:02.902 INFO:teuthology.orchestra.run.smithi016.stdout:stopped 2024-04-02T04:47:02.902 INFO:teuthology.orchestra.run.smithi016.stdout:data_pools [3] 2024-04-02T04:47:02.902 INFO:teuthology.orchestra.run.smithi016.stdout:metadata_pool 2 2024-04-02T04:47:02.903 INFO:teuthology.orchestra.run.smithi016.stdout:inline_data enabled 2024-04-02T04:47:02.903 INFO:teuthology.orchestra.run.smithi016.stdout:balancer 2024-04-02T04:47:02.903 INFO:teuthology.orchestra.run.smithi016.stdout:bal_rank_mask -1 2024-04-02T04:47:02.903 INFO:teuthology.orchestra.run.smithi016.stdout:standby_count_wanted 1 2024-04-02T04:47:02.903 INFO:teuthology.orchestra.run.smithi016.stdout:qdb_cluster leader: 0 members: 2024-04-02T04:47:02.903 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.zwrfoo{0:24285} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.27:6824/1461455976,v1:172.21.15.27:6825/1461455976] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:47:02.903 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:47:02.903 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:47:02.903 INFO:teuthology.orchestra.run.smithi016.stdout:Standby daemons: 2024-04-02T04:47:02.903 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:47:02.903 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.fyzrnm{-1:14488} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.16:6828/2578719859,v1:172.21.15.16:6829/2578719859] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:47:02.903 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.uhgkgg{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.16:6826/601263579,v1:172.21.15.16:6827/601263579] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:47:02.903 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.sdublg{-1:24295} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.27:6826/4177192740,v1:172.21.15.27:6827/4177192740] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:47:02.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.899+0000 7fbf5fd4c700 1 -- 172.21.15.16:0/1839817476 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fbf4006dd70 msgr2=0x7fbf40070230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:02.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.899+0000 7fbf5fd4c700 1 --2- 172.21.15.16:0/1839817476 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fbf4006dd70 0x7fbf40070230 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbf48009550 tx=0x7fbf480091b0 comp rx=0 tx=0).stop 2024-04-02T04:47:02.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.899+0000 7fbf5fd4c700 1 -- 172.21.15.16:0/1839817476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf58103eb0 msgr2=0x7fbf580ff530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:02.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.899+0000 7fbf5fd4c700 1 --2- 172.21.15.16:0/1839817476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf58103eb0 0x7fbf580ff530 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c006010 tx=0x7fbf4c00bd70 comp rx=0 tx=0).stop 2024-04-02T04:47:02.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.899+0000 7fbf5fd4c700 1 -- 172.21.15.16:0/1839817476 shutdown_connections 2024-04-02T04:47:02.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.899+0000 7fbf5fd4c700 1 --2- 172.21.15.16:0/1839817476 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fbf4006dd70 0x7fbf40070230 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:02.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.899+0000 7fbf5fd4c700 1 --2- 172.21.15.16:0/1839817476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf58101580 0x7fbf58100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:02.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.899+0000 7fbf5fd4c700 1 --2- 172.21.15.16:0/1839817476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf58103eb0 0x7fbf580ff530 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:02.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.899+0000 7fbf5fd4c700 1 -- 172.21.15.16:0/1839817476 >> 172.21.15.16:0/1839817476 conn(0x7fbf580faf90 msgr2=0x7fbf580fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:02.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.899+0000 7fbf5fd4c700 1 -- 172.21.15.16:0/1839817476 shutdown_connections 2024-04-02T04:47:02.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:02.899+0000 7fbf5fd4c700 1 -- 172.21.15.16:0/1839817476 wait complete. 2024-04-02T04:47:02.905 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 15 2024-04-02T04:47:03.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.006+0000 7f3bcf3b9700 1 -- 172.21.15.16:0/3932134476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bc8101d90 msgr2=0x7f3bc81021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:03.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.006+0000 7f3bcf3b9700 1 --2- 172.21.15.16:0/3932134476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bc8101d90 0x7f3bc81021f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc009910 tx=0x7f3bbc009c20 comp rx=0 tx=0).stop 2024-04-02T04:47:03.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.006+0000 7f3bcf3b9700 1 -- 172.21.15.16:0/3932134476 shutdown_connections 2024-04-02T04:47:03.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.006+0000 7f3bcf3b9700 1 --2- 172.21.15.16:0/3932134476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bc8101d90 0x7f3bc81021f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:03.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.006+0000 7f3bcf3b9700 1 --2- 172.21.15.16:0/3932134476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bc80fedb0 0x7f3bc80ff1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:03.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.006+0000 7f3bcf3b9700 1 -- 172.21.15.16:0/3932134476 >> 172.21.15.16:0/3932134476 conn(0x7f3bc80fa710 msgr2=0x7f3bc80fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:03.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.006+0000 7f3bcf3b9700 1 -- 172.21.15.16:0/3932134476 shutdown_connections 2024-04-02T04:47:03.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.006+0000 7f3bcf3b9700 1 -- 172.21.15.16:0/3932134476 wait complete. 2024-04-02T04:47:03.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.007+0000 7f3bcf3b9700 1 Processor -- start 2024-04-02T04:47:03.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.007+0000 7f3bcf3b9700 1 -- start start 2024-04-02T04:47:03.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.007+0000 7f3bcf3b9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bc80fedb0 0x7f3bc8196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:03.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.007+0000 7f3bcf3b9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bc8101d90 0x7f3bc8196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:03.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.007+0000 7f3bcf3b9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc81970c0 con 0x7f3bc80fedb0 2024-04-02T04:47:03.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.007+0000 7f3bcf3b9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc8197200 con 0x7f3bc8101d90 2024-04-02T04:47:03.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.007+0000 7f3bcd155700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bc80fedb0 0x7f3bc8196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:03.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.007+0000 7f3bcd155700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bc80fedb0 0x7f3bc8196560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59624/0 (socket says 172.21.15.16:59624) 2024-04-02T04:47:03.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.007+0000 7f3bcd155700 1 -- 172.21.15.16:0/2830594589 learned_addr learned my addr 172.21.15.16:0/2830594589 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:47:03.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.007+0000 7f3bcc954700 1 --2- 172.21.15.16:0/2830594589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bc8101d90 0x7f3bc8196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:03.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.007+0000 7f3bcd155700 1 -- 172.21.15.16:0/2830594589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bc8101d90 msgr2=0x7f3bc8196aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:03.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.007+0000 7f3bcd155700 1 --2- 172.21.15.16:0/2830594589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bc8101d90 0x7f3bc8196aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:03.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.007+0000 7f3bcd155700 1 -- 172.21.15.16:0/2830594589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bbc0095f0 con 0x7f3bc80fedb0 2024-04-02T04:47:03.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.008+0000 7f3bcd155700 1 --2- 172.21.15.16:0/2830594589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bc80fedb0 0x7f3bc8196560 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8007e90 tx=0x7f3bb800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:03.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.008+0000 7f3bc67fc700 1 -- 172.21.15.16:0/2830594589 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3bb8017070 con 0x7f3bc80fedb0 2024-04-02T04:47:03.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.008+0000 7f3bcf3b9700 1 -- 172.21.15.16:0/2830594589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bc819bc10 con 0x7f3bc80fedb0 2024-04-02T04:47:03.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.008+0000 7f3bc67fc700 1 -- 172.21.15.16:0/2830594589 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3bb800f040 con 0x7f3bc80fedb0 2024-04-02T04:47:03.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.008+0000 7f3bc67fc700 1 -- 172.21.15.16:0/2830594589 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3bb80048f0 con 0x7f3bc80fedb0 2024-04-02T04:47:03.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.008+0000 7f3bcf3b9700 1 -- 172.21.15.16:0/2830594589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc819c160 con 0x7f3bc80fedb0 2024-04-02T04:47:03.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.009+0000 7f3bcf3b9700 1 -- 172.21.15.16:0/2830594589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bc8190670 con 0x7f3bc80fedb0 2024-04-02T04:47:03.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.009+0000 7f3bc67fc700 1 -- 172.21.15.16:0/2830594589 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f3bb8004120 con 0x7f3bc80fedb0 2024-04-02T04:47:03.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.010+0000 7f3bc67fc700 1 --2- 172.21.15.16:0/2830594589 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f3bb006dd00 0x7f3bb00701c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:03.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.010+0000 7f3bc67fc700 1 -- 172.21.15.16:0/2830594589 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f3bb808d570 con 0x7f3bc80fedb0 2024-04-02T04:47:03.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.010+0000 7f3bcc954700 1 --2- 172.21.15.16:0/2830594589 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f3bb006dd00 0x7f3bb00701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:03.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.010+0000 7f3bcc954700 1 --2- 172.21.15.16:0/2830594589 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f3bb006dd00 0x7f3bb00701c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc005090 tx=0x7f3bbc01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:03.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.015+0000 7f3bc67fc700 1 -- 172.21.15.16:0/2830594589 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3bb805a630 con 0x7f3bc80fedb0 2024-04-02T04:47:03.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.245+0000 7f3bcf3b9700 1 -- 172.21.15.16:0/2830594589 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3bc802ce40 con 0x7f3bb006dd00 2024-04-02T04:47:03.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.246+0000 7f3bc67fc700 1 -- 172.21.15.16:0/2830594589 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f3bc802ce40 con 0x7f3bb006dd00 2024-04-02T04:47:03.249 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:47:03.250 INFO:teuthology.orchestra.run.smithi016.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:47:03.250 INFO:teuthology.orchestra.run.smithi016.stdout: "in_progress": true, 2024-04-02T04:47:03.250 INFO:teuthology.orchestra.run.smithi016.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-02T04:47:03.250 INFO:teuthology.orchestra.run.smithi016.stdout: "services_complete": [ 2024-04-02T04:47:03.250 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr", 2024-04-02T04:47:03.250 INFO:teuthology.orchestra.run.smithi016.stdout: "mon", 2024-04-02T04:47:03.250 INFO:teuthology.orchestra.run.smithi016.stdout: "crash" 2024-04-02T04:47:03.250 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-04-02T04:47:03.250 INFO:teuthology.orchestra.run.smithi016.stdout: "progress": "8/23 daemons upgraded", 2024-04-02T04:47:03.250 INFO:teuthology.orchestra.run.smithi016.stdout: "message": "Currently upgrading osd daemons", 2024-04-02T04:47:03.250 INFO:teuthology.orchestra.run.smithi016.stdout: "is_paused": false 2024-04-02T04:47:03.250 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:47:03.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.249+0000 7f3bcf3b9700 1 -- 172.21.15.16:0/2830594589 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f3bb006dd00 msgr2=0x7f3bb00701c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:03.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.249+0000 7f3bcf3b9700 1 --2- 172.21.15.16:0/2830594589 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f3bb006dd00 0x7f3bb00701c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc005090 tx=0x7f3bbc01a040 comp rx=0 tx=0).stop 2024-04-02T04:47:03.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.249+0000 7f3bcf3b9700 1 -- 172.21.15.16:0/2830594589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bc80fedb0 msgr2=0x7f3bc8196560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:03.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.249+0000 7f3bcf3b9700 1 --2- 172.21.15.16:0/2830594589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bc80fedb0 0x7f3bc8196560 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8007e90 tx=0x7f3bb800d3b0 comp rx=0 tx=0).stop 2024-04-02T04:47:03.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.249+0000 7f3bcf3b9700 1 -- 172.21.15.16:0/2830594589 shutdown_connections 2024-04-02T04:47:03.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.249+0000 7f3bcf3b9700 1 --2- 172.21.15.16:0/2830594589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bc80fedb0 0x7f3bc8196560 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:03.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.249+0000 7f3bcf3b9700 1 --2- 172.21.15.16:0/2830594589 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f3bb006dd00 0x7f3bb00701c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:03.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.249+0000 7f3bcf3b9700 1 --2- 172.21.15.16:0/2830594589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bc8101d90 0x7f3bc8196aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:03.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.249+0000 7f3bcf3b9700 1 -- 172.21.15.16:0/2830594589 >> 172.21.15.16:0/2830594589 conn(0x7f3bc80fa710 msgr2=0x7f3bc81053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:03.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.249+0000 7f3bcf3b9700 1 -- 172.21.15.16:0/2830594589 shutdown_connections 2024-04-02T04:47:03.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.249+0000 7f3bcf3b9700 1 -- 172.21.15.16:0/2830594589 wait complete. 2024-04-02T04:47:03.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.365+0000 7f902d220700 1 -- 172.21.15.16:0/134546392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9028100be0 msgr2=0x7f9028101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:03.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.365+0000 7f902d220700 1 --2- 172.21.15.16:0/134546392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9028100be0 0x7f9028101040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f901c009910 tx=0x7f901c009c20 comp rx=0 tx=0).stop 2024-04-02T04:47:03.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.365+0000 7f902d220700 1 -- 172.21.15.16:0/134546392 shutdown_connections 2024-04-02T04:47:03.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.365+0000 7f902d220700 1 --2- 172.21.15.16:0/134546392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9028100be0 0x7f9028101040 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:03.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.365+0000 7f902d220700 1 --2- 172.21.15.16:0/134546392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90280ff9e0 0x7f90280ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:03.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.365+0000 7f902d220700 1 -- 172.21.15.16:0/134546392 >> 172.21.15.16:0/134546392 conn(0x7f90280faf80 msgr2=0x7f90280fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:03.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.365+0000 7f902d220700 1 -- 172.21.15.16:0/134546392 shutdown_connections 2024-04-02T04:47:03.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.365+0000 7f902d220700 1 -- 172.21.15.16:0/134546392 wait complete. 2024-04-02T04:47:03.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.366+0000 7f902d220700 1 Processor -- start 2024-04-02T04:47:03.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.366+0000 7f902d220700 1 -- start start 2024-04-02T04:47:03.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.367+0000 7f902d220700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90280ff9e0 0x7f9028198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:03.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.367+0000 7f902d220700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9028100be0 0x7f9028198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:03.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.367+0000 7f902d220700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9028199090 con 0x7f90280ff9e0 2024-04-02T04:47:03.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.367+0000 7f902d220700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90281991d0 con 0x7f9028100be0 2024-04-02T04:47:03.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.367+0000 7f9026d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90280ff9e0 0x7f9028198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:03.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.367+0000 7f9026d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90280ff9e0 0x7f9028198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59652/0 (socket says 172.21.15.16:59652) 2024-04-02T04:47:03.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.367+0000 7f9026d9d700 1 -- 172.21.15.16:0/2837440122 learned_addr learned my addr 172.21.15.16:0/2837440122 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:47:03.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.367+0000 7f902659c700 1 --2- 172.21.15.16:0/2837440122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9028100be0 0x7f9028198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:03.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.367+0000 7f9026d9d700 1 -- 172.21.15.16:0/2837440122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9028100be0 msgr2=0x7f9028198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:03.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.367+0000 7f9026d9d700 1 --2- 172.21.15.16:0/2837440122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9028100be0 0x7f9028198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:03.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.367+0000 7f9026d9d700 1 -- 172.21.15.16:0/2837440122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f901c0095f0 con 0x7f90280ff9e0 2024-04-02T04:47:03.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.368+0000 7f9026d9d700 1 --2- 172.21.15.16:0/2837440122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90280ff9e0 0x7f9028198530 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f901800e8f0 tx=0x7f901800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:03.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.368+0000 7f900ffff700 1 -- 172.21.15.16:0/2837440122 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9018009790 con 0x7f90280ff9e0 2024-04-02T04:47:03.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.368+0000 7f902d220700 1 -- 172.21.15.16:0/2837440122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f902819dc30 con 0x7f90280ff9e0 2024-04-02T04:47:03.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.368+0000 7f900ffff700 1 -- 172.21.15.16:0/2837440122 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f90180044e0 con 0x7f90280ff9e0 2024-04-02T04:47:03.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.368+0000 7f900ffff700 1 -- 172.21.15.16:0/2837440122 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9018003e00 con 0x7f90280ff9e0 2024-04-02T04:47:03.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.368+0000 7f902d220700 1 -- 172.21.15.16:0/2837440122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f902819e180 con 0x7f90280ff9e0 2024-04-02T04:47:03.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.369+0000 7f902d220700 1 -- 172.21.15.16:0/2837440122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9028106eb0 con 0x7f90280ff9e0 2024-04-02T04:47:03.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.370+0000 7f900ffff700 1 -- 172.21.15.16:0/2837440122 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f901800cba0 con 0x7f90280ff9e0 2024-04-02T04:47:03.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.370+0000 7f900ffff700 1 --2- 172.21.15.16:0/2837440122 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f901006dd70 0x7f9010070230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:03.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.371+0000 7f900ffff700 1 -- 172.21.15.16:0/2837440122 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f9018014070 con 0x7f90280ff9e0 2024-04-02T04:47:03.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.371+0000 7f902659c700 1 --2- 172.21.15.16:0/2837440122 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f901006dd70 0x7f9010070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:03.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.371+0000 7f902659c700 1 --2- 172.21.15.16:0/2837440122 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f901006dd70 0x7f9010070230 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f901c00b760 tx=0x7f901c000bc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:03.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.380+0000 7f900ffff700 1 -- 172.21.15.16:0/2837440122 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f901805af90 con 0x7f90280ff9e0 2024-04-02T04:47:03.560 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:03 smithi016 podman[113075]: 2024-04-02 04:47:03.236675264 +0000 UTC m=+1.597215573 container cleanup 52e592b4d1f20412b4b1a2f21b34acb418314149b728952f76b54ee4a01fece8 (image=quay.ceph.io/ceph-ci/ceph@sha256:397d0c1d908c002c558906e3ea05982bc336d403ee4d900400cc3bdd9243aa2e, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, RELEASE=reef-4d05f4c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-02T04:47:03.560 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:03 smithi016 bash[113075]: ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2 2024-04-02T04:47:03.560 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:03 smithi016 podman[113161]: 2024-04-02 04:47:03.404843901 +0000 UTC m=+0.601971554 container remove 52e592b4d1f20412b4b1a2f21b34acb418314149b728952f76b54ee4a01fece8 (image=quay.ceph.io/ceph-ci/ceph@sha256:397d0c1d908c002c558906e3ea05982bc336d403ee4d900400cc3bdd9243aa2e, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=reef-4d05f4c, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-02T04:47:03.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:03 smithi016 ceph-mon[93230]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-02T04:47:03.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:03 smithi016 ceph-mon[93230]: osdmap e60: 6 total, 5 up, 6 in 2024-04-02T04:47:03.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:03 smithi016 ceph-mon[93230]: failed to encode map e60 with expected crc 2024-04-02T04:47:03.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:03 smithi016 ceph-mon[93230]: failed to encode map e60 with expected crc 2024-04-02T04:47:03.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:03 smithi016 ceph-mon[93230]: failed to encode map e60 with expected crc 2024-04-02T04:47:03.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:03 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/1969395893' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:47:03.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:03 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/1839817476' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:47:03.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:03 smithi016 ceph-mon[93230]: pgmap v97: 65 pgs: 12 peering, 7 stale+active+clean, 46 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:47:03.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:03 smithi016 ceph-mon[93230]: from='client.34300 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:47:03.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:03 smithi027 ceph-mon[83036]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-02T04:47:03.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:03 smithi027 ceph-mon[83036]: osdmap e60: 6 total, 5 up, 6 in 2024-04-02T04:47:03.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:03 smithi027 ceph-mon[83036]: failed to encode map e60 with expected crc 2024-04-02T04:47:03.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:03 smithi027 ceph-mon[83036]: failed to encode map e60 with expected crc 2024-04-02T04:47:03.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:03 smithi027 ceph-mon[83036]: failed to encode map e60 with expected crc 2024-04-02T04:47:03.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:03 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/1969395893' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:47:03.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:03 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/1839817476' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:47:03.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:03 smithi027 ceph-mon[83036]: pgmap v97: 65 pgs: 12 peering, 7 stale+active+clean, 46 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:47:03.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:03 smithi027 ceph-mon[83036]: from='client.34300 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:47:03.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.724+0000 7f902d220700 1 -- 172.21.15.16:0/2837440122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f9028109690 con 0x7f90280ff9e0 2024-04-02T04:47:03.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.724+0000 7f900ffff700 1 -- 172.21.15.16:0/2837440122 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+551 (secure 0 0 0) 0x7f901805ab20 con 0x7f90280ff9e0 2024-04-02T04:47:03.728 INFO:teuthology.orchestra.run.smithi016.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Reduced data availability: 2 pgs peering 2024-04-02T04:47:03.728 INFO:teuthology.orchestra.run.smithi016.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-02T04:47:03.728 INFO:teuthology.orchestra.run.smithi016.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-02T04:47:03.728 INFO:teuthology.orchestra.run.smithi016.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-02T04:47:03.728 INFO:teuthology.orchestra.run.smithi016.stdout: osd.2 (root=default,host=smithi016) is down 2024-04-02T04:47:03.728 INFO:teuthology.orchestra.run.smithi016.stdout:[WRN] PG_AVAILABILITY: Reduced data availability: 2 pgs peering 2024-04-02T04:47:03.728 INFO:teuthology.orchestra.run.smithi016.stdout: pg 2.c is stuck peering for 2m, current state peering, last acting [3,0] 2024-04-02T04:47:03.728 INFO:teuthology.orchestra.run.smithi016.stdout: pg 3.1e is stuck peering for 2m, current state peering, last acting [3,1] 2024-04-02T04:47:03.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.727+0000 7f902d220700 1 -- 172.21.15.16:0/2837440122 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f901006dd70 msgr2=0x7f9010070230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:03.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.727+0000 7f902d220700 1 --2- 172.21.15.16:0/2837440122 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f901006dd70 0x7f9010070230 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f901c00b760 tx=0x7f901c000bc0 comp rx=0 tx=0).stop 2024-04-02T04:47:03.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.727+0000 7f902d220700 1 -- 172.21.15.16:0/2837440122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90280ff9e0 msgr2=0x7f9028198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:03.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.727+0000 7f902d220700 1 --2- 172.21.15.16:0/2837440122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90280ff9e0 0x7f9028198530 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f901800e8f0 tx=0x7f901800ecb0 comp rx=0 tx=0).stop 2024-04-02T04:47:03.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.727+0000 7f902d220700 1 -- 172.21.15.16:0/2837440122 shutdown_connections 2024-04-02T04:47:03.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.727+0000 7f902d220700 1 --2- 172.21.15.16:0/2837440122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90280ff9e0 0x7f9028198530 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:03.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.727+0000 7f902d220700 1 --2- 172.21.15.16:0/2837440122 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f901006dd70 0x7f9010070230 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:03.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.727+0000 7f902d220700 1 --2- 172.21.15.16:0/2837440122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9028100be0 0x7f9028198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:03.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.727+0000 7f902d220700 1 -- 172.21.15.16:0/2837440122 >> 172.21.15.16:0/2837440122 conn(0x7f90280faf80 msgr2=0x7f9028103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:03.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.727+0000 7f902d220700 1 -- 172.21.15.16:0/2837440122 shutdown_connections 2024-04-02T04:47:03.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:03.727+0000 7f902d220700 1 -- 172.21.15.16:0/2837440122 wait complete. 2024-04-02T04:47:03.978 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:03 smithi016 podman[113303]: 2024-04-02 04:47:03.889323471 +0000 UTC m=+0.112877454 container create 8244cbf65a6c1c064f983f00fbde34fcf13cd21f9aa783cd7d4ec5804a049e42 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2-deactivate, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, RELEASE=squid-6cec793, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326) 2024-04-02T04:47:03.978 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:03 smithi016 podman[113303]: 2024-04-02 04:47:03.794573219 +0000 UTC m=+0.018127185 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:47:04.310 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:04 smithi016 podman[113303]: 2024-04-02 04:47:04.109146694 +0000 UTC m=+0.332700633 container init 8244cbf65a6c1c064f983f00fbde34fcf13cd21f9aa783cd7d4ec5804a049e42 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2-deactivate, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, RELEASE=squid-6cec793, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-02T04:47:04.310 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:04 smithi016 podman[113303]: 2024-04-02 04:47:04.112270562 +0000 UTC m=+0.335824500 container start 8244cbf65a6c1c064f983f00fbde34fcf13cd21f9aa783cd7d4ec5804a049e42 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-6cec793, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-02T04:47:04.310 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:04 smithi016 podman[113303]: 2024-04-02 04:47:04.18404782 +0000 UTC m=+0.407601763 container attach 8244cbf65a6c1c064f983f00fbde34fcf13cd21f9aa783cd7d4ec5804a049e42 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2-deactivate, RELEASE=squid-6cec793, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T04:47:04.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:04 smithi027 ceph-mon[83036]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-02T04:47:04.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:04 smithi027 ceph-mon[83036]: failed to encode map e61 with expected crc 2024-04-02T04:47:04.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:04 smithi027 ceph-mon[83036]: failed to encode map e61 with expected crc 2024-04-02T04:47:04.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:04 smithi027 ceph-mon[83036]: osdmap e61: 6 total, 5 up, 6 in 2024-04-02T04:47:04.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:04 smithi027 ceph-mon[83036]: failed to encode map e61 with expected crc 2024-04-02T04:47:04.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:04 smithi027 ceph-mon[83036]: failed to encode map e61 with expected crc 2024-04-02T04:47:04.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:04 smithi027 ceph-mon[83036]: failed to encode map e61 with expected crc 2024-04-02T04:47:04.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:04 smithi027 ceph-mon[83036]: failed to encode map e61 with expected crc 2024-04-02T04:47:04.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:04 smithi027 ceph-mon[83036]: failed to encode map e61 with expected crc 2024-04-02T04:47:04.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:04 smithi027 ceph-mon[83036]: failed to encode map e61 with expected crc 2024-04-02T04:47:04.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:04 smithi027 ceph-mon[83036]: failed to encode map e61 with expected crc 2024-04-02T04:47:04.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:04 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/2837440122' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:47:04.707 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:04 smithi016 podman[113303]: 2024-04-02 04:47:04.414015764 +0000 UTC m=+0.637569711 container died 8244cbf65a6c1c064f983f00fbde34fcf13cd21f9aa783cd7d4ec5804a049e42 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-6cec793, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, ceph=True) 2024-04-02T04:47:04.707 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:04 smithi016 ceph-mon[93230]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-02T04:47:04.707 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:04 smithi016 ceph-mon[93230]: failed to encode map e61 with expected crc 2024-04-02T04:47:04.707 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:04 smithi016 ceph-mon[93230]: failed to encode map e61 with expected crc 2024-04-02T04:47:04.707 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:04 smithi016 ceph-mon[93230]: osdmap e61: 6 total, 5 up, 6 in 2024-04-02T04:47:04.707 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:04 smithi016 ceph-mon[93230]: failed to encode map e61 with expected crc 2024-04-02T04:47:04.707 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:04 smithi016 ceph-mon[93230]: failed to encode map e61 with expected crc 2024-04-02T04:47:04.707 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:04 smithi016 ceph-mon[93230]: failed to encode map e61 with expected crc 2024-04-02T04:47:04.707 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:04 smithi016 ceph-mon[93230]: failed to encode map e61 with expected crc 2024-04-02T04:47:04.707 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:04 smithi016 ceph-mon[93230]: failed to encode map e61 with expected crc 2024-04-02T04:47:04.707 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:04 smithi016 ceph-mon[93230]: failed to encode map e61 with expected crc 2024-04-02T04:47:04.708 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:04 smithi016 ceph-mon[93230]: failed to encode map e61 with expected crc 2024-04-02T04:47:04.708 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:04 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/2837440122' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:47:05.310 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:04 smithi016 podman[113329]: 2024-04-02 04:47:04.987884701 +0000 UTC m=+0.568352737 container remove 8244cbf65a6c1c064f983f00fbde34fcf13cd21f9aa783cd7d4ec5804a049e42 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2-deactivate, maintainer=Guillaume Abrioux , RELEASE=squid-6cec793, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-02T04:47:05.310 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:05 smithi016 systemd[1]: ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@osd.2.service: Deactivated successfully. 2024-04-02T04:47:05.310 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:05 smithi016 systemd[1]: Stopped Ceph osd.2 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4. 2024-04-02T04:47:05.310 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:05 smithi016 systemd[1]: ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@osd.2.service: Consumed 47.471s CPU time. 2024-04-02T04:47:05.310 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:05 smithi016 systemd[1]: Starting Ceph osd.2 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4... 2024-04-02T04:47:05.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:05 smithi027 ceph-mon[83036]: pgmap v99: 65 pgs: 19 peering, 3 stale+active+clean, 43 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:47:05.694 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:05 smithi016 ceph-mon[93230]: pgmap v99: 65 pgs: 19 peering, 3 stale+active+clean, 43 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:47:05.694 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:05 smithi016 podman[113435]: 2024-04-02 04:47:05.343310548 +0000 UTC m=+0.018819835 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:47:05.694 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:05 smithi016 podman[113435]: 2024-04-02 04:47:05.455633672 +0000 UTC m=+0.131142942 container create 0295acf7d749decb29891d3d4faaff5137c82ced468c00064db01bce9af5b3fe (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2-activate, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-6cec793, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-02T04:47:05.694 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:05 smithi016 podman[113435]: 2024-04-02 04:47:05.691491427 +0000 UTC m=+0.367000708 container init 0295acf7d749decb29891d3d4faaff5137c82ced468c00064db01bce9af5b3fe (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2-activate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, RELEASE=squid-6cec793, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=) 2024-04-02T04:47:06.060 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:05 smithi016 podman[113435]: 2024-04-02 04:47:05.694931345 +0000 UTC m=+0.370440620 container start 0295acf7d749decb29891d3d4faaff5137c82ced468c00064db01bce9af5b3fe (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2-activate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-6cec793, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T04:47:06.060 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:05 smithi016 podman[113435]: 2024-04-02 04:47:05.775007039 +0000 UTC m=+0.450516327 container attach 0295acf7d749decb29891d3d4faaff5137c82ced468c00064db01bce9af5b3fe (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2-activate, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-6cec793, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-02T04:47:06.548 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:06 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2-activate[113450]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-02T04:47:06.548 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:06 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2-activate[113450]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-02T04:47:06.548 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:06 smithi016 bash[113435]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-02T04:47:06.548 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:06 smithi016 bash[113435]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-02T04:47:06.548 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:06 smithi016 bash[113435]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-02T04:47:06.548 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:06 smithi016 bash[113435]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-02T04:47:06.548 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:06 smithi016 bash[113435]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-04-02T04:47:06.548 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:06 smithi016 bash[113435]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-02T04:47:06.548 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:06 smithi016 bash[113435]: --> ceph-volume raw activate successful for osd ID: 2 2024-04-02T04:47:06.548 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:06 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2-activate[113450]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-02T04:47:06.549 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:06 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2-activate[113450]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-02T04:47:06.549 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:06 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2-activate[113450]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-04-02T04:47:06.549 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:06 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2-activate[113450]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-02T04:47:06.549 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:06 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2-activate[113450]: --> ceph-volume raw activate successful for osd ID: 2 2024-04-02T04:47:06.549 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:06 smithi016 podman[113435]: 2024-04-02 04:47:06.166333584 +0000 UTC m=+0.841842864 container died 0295acf7d749decb29891d3d4faaff5137c82ced468c00064db01bce9af5b3fe (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2-activate, RELEASE=squid-6cec793, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_CLEAN=True, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T04:47:07.161 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:06 smithi016 podman[113585]: 2024-04-02 04:47:06.837254835 +0000 UTC m=+0.664489171 container remove 0295acf7d749decb29891d3d4faaff5137c82ced468c00064db01bce9af5b3fe (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-6cec793, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0) 2024-04-02T04:47:07.162 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:07 smithi016 podman[113652]: 2024-04-02 04:47:07.0548741 +0000 UTC m=+0.114501070 container create a8e9b8d0303c9ac8dd63db1cc767a668e0668b8847e1fafecf767a56b17c42cd (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-6cec793, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, ceph=True) 2024-04-02T04:47:07.162 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:07 smithi016 podman[113652]: 2024-04-02 04:47:06.95885446 +0000 UTC m=+0.018481428 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:47:07.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:07 smithi016 ceph-mon[93230]: Health check failed: Degraded data redundancy: 11/267 objects degraded (4.120%), 4 pgs degraded (PG_DEGRADED) 2024-04-02T04:47:07.560 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:07 smithi016 podman[113652]: 2024-04-02 04:47:07.265697903 +0000 UTC m=+0.325324881 container init a8e9b8d0303c9ac8dd63db1cc767a668e0668b8847e1fafecf767a56b17c42cd (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-6cec793, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T04:47:07.560 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:07 smithi016 podman[113652]: 2024-04-02 04:47:07.268643456 +0000 UTC m=+0.328270426 container start a8e9b8d0303c9ac8dd63db1cc767a668e0668b8847e1fafecf767a56b17c42cd (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-6cec793, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T04:47:07.560 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:07 smithi016 ceph-osd[113671]: -- 172.21.15.16:0/3978489368 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5567021fd680 con 0x5567021c7000 2024-04-02T04:47:07.560 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:07 smithi016 bash[113652]: a8e9b8d0303c9ac8dd63db1cc767a668e0668b8847e1fafecf767a56b17c42cd 2024-04-02T04:47:07.560 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:07 smithi016 systemd[1]: Started Ceph osd.2 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4. 2024-04-02T04:47:07.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:07 smithi027 ceph-mon[83036]: Health check failed: Degraded data redundancy: 11/267 objects degraded (4.120%), 4 pgs degraded (PG_DEGRADED) 2024-04-02T04:47:08.163 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:07 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2[113667]: 2024-04-02T04:47:07.816+0000 7f02c75787c0 -1 Falling back to public interface 2024-04-02T04:47:08.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:08 smithi016 ceph-mon[93230]: pgmap v100: 65 pgs: 4 active+undersized+degraded, 19 peering, 4 active+undersized, 38 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 11/267 objects degraded (4.120%) 2024-04-02T04:47:08.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:08 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:08.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:08 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:08.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:08 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:47:08.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:08 smithi027 ceph-mon[83036]: pgmap v100: 65 pgs: 4 active+undersized+degraded, 19 peering, 4 active+undersized, 38 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 11/267 objects degraded (4.120%) 2024-04-02T04:47:08.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:08 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:08.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:08 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:08.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:08 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:47:09.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:09 smithi016 ceph-mon[93230]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 6 pgs peering) 2024-04-02T04:47:09.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:09 smithi027 ceph-mon[83036]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 6 pgs peering) 2024-04-02T04:47:10.482 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:10 smithi016 ceph-mon[93230]: pgmap v101: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 35/267 objects degraded (13.109%) 2024-04-02T04:47:10.547 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:10 smithi027 ceph-mon[83036]: pgmap v101: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 35/267 objects degraded (13.109%) 2024-04-02T04:47:12.222 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:11 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2[113667]: 2024-04-02T04:47:11.994+0000 7f02c75787c0 -1 osd.2 59 log_to_monitors true 2024-04-02T04:47:12.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:12 smithi016 ceph-mon[93230]: pgmap v102: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 35/267 objects degraded (13.109%) 2024-04-02T04:47:12.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:12 smithi016 ceph-mon[93230]: from='osd.2 [v2:172.21.15.16:6818/1004843818,v1:172.21.15.16:6819/1004843818]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-02T04:47:12.560 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:47:12 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2[113667]: 2024-04-02T04:47:12.236+0000 7f02b49c8700 -1 osd.2 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T04:47:12.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:12 smithi027 ceph-mon[83036]: pgmap v102: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 35/267 objects degraded (13.109%) 2024-04-02T04:47:12.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:12 smithi027 ceph-mon[83036]: from='osd.2 [v2:172.21.15.16:6818/1004843818,v1:172.21.15.16:6819/1004843818]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-02T04:47:13.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:13 smithi016 ceph-mon[93230]: from='osd.2 [v2:172.21.15.16:6818/1004843818,v1:172.21.15.16:6819/1004843818]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-02T04:47:13.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:13 smithi016 ceph-mon[93230]: osdmap e62: 6 total, 5 up, 6 in 2024-04-02T04:47:13.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:13 smithi016 ceph-mon[93230]: from='osd.2 [v2:172.21.15.16:6818/1004843818,v1:172.21.15.16:6819/1004843818]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-04-02T04:47:13.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:13 smithi016 ceph-mon[93230]: failed to encode map e62 with expected crc 2024-04-02T04:47:13.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:13 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:13.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:13 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:13.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:13 smithi016 ceph-mon[93230]: Health check update: Degraded data redundancy: 35/267 objects degraded (13.109%), 12 pgs degraded (PG_DEGRADED) 2024-04-02T04:47:13.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:13 smithi027 ceph-mon[83036]: from='osd.2 [v2:172.21.15.16:6818/1004843818,v1:172.21.15.16:6819/1004843818]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-02T04:47:13.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:13 smithi027 ceph-mon[83036]: osdmap e62: 6 total, 5 up, 6 in 2024-04-02T04:47:13.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:13 smithi027 ceph-mon[83036]: from='osd.2 [v2:172.21.15.16:6818/1004843818,v1:172.21.15.16:6819/1004843818]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-04-02T04:47:13.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:13 smithi027 ceph-mon[83036]: failed to encode map e62 with expected crc 2024-04-02T04:47:13.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:13 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:13.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:13 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:13.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:13 smithi027 ceph-mon[83036]: Health check update: Degraded data redundancy: 35/267 objects degraded (13.109%), 12 pgs degraded (PG_DEGRADED) 2024-04-02T04:47:14.553 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:14 smithi016 ceph-mon[93230]: failed to encode map e62 with expected crc 2024-04-02T04:47:14.554 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:14 smithi016 ceph-mon[93230]: pgmap v104: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 35/267 objects degraded (13.109%) 2024-04-02T04:47:14.554 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:14 smithi016 ceph-mon[93230]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-02T04:47:14.554 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:14 smithi016 ceph-mon[93230]: failed to encode map e62 with expected crc 2024-04-02T04:47:14.554 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:14 smithi016 ceph-mon[93230]: failed to encode map e63 with expected crc 2024-04-02T04:47:14.554 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:14 smithi016 ceph-mon[93230]: osd.2 [v2:172.21.15.16:6818/1004843818,v1:172.21.15.16:6819/1004843818] boot 2024-04-02T04:47:14.554 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:14 smithi016 ceph-mon[93230]: osdmap e63: 6 total, 6 up, 6 in 2024-04-02T04:47:14.554 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:14 smithi016 ceph-mon[93230]: failed to encode map e63 with expected crc 2024-04-02T04:47:14.554 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:14 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T04:47:14.554 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:14 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:14.554 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:14 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:14.554 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:14 smithi016 ceph-mon[93230]: osdmap e64: 6 total, 6 up, 6 in 2024-04-02T04:47:14.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:14 smithi027 ceph-mon[83036]: failed to encode map e62 with expected crc 2024-04-02T04:47:14.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:14 smithi027 ceph-mon[83036]: pgmap v104: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 35/267 objects degraded (13.109%) 2024-04-02T04:47:14.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:14 smithi027 ceph-mon[83036]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-02T04:47:14.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:14 smithi027 ceph-mon[83036]: failed to encode map e62 with expected crc 2024-04-02T04:47:14.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:14 smithi027 ceph-mon[83036]: failed to encode map e63 with expected crc 2024-04-02T04:47:14.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:14 smithi027 ceph-mon[83036]: osd.2 [v2:172.21.15.16:6818/1004843818,v1:172.21.15.16:6819/1004843818] boot 2024-04-02T04:47:14.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:14 smithi027 ceph-mon[83036]: osdmap e63: 6 total, 6 up, 6 in 2024-04-02T04:47:14.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:14 smithi027 ceph-mon[83036]: failed to encode map e63 with expected crc 2024-04-02T04:47:14.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:14 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T04:47:14.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:14 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:14.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:14 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:14.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:14 smithi027 ceph-mon[83036]: osdmap e64: 6 total, 6 up, 6 in 2024-04-02T04:47:15.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:15 smithi016 ceph-mon[93230]: failed to encode map e64 with expected crc 2024-04-02T04:47:15.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:15 smithi016 ceph-mon[93230]: failed to encode map e64 with expected crc 2024-04-02T04:47:15.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:15 smithi016 ceph-mon[93230]: failed to encode map e64 with expected crc 2024-04-02T04:47:15.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:15 smithi016 ceph-mon[93230]: failed to encode map e64 with expected crc 2024-04-02T04:47:15.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:15 smithi016 ceph-mon[93230]: failed to encode map e64 with expected crc 2024-04-02T04:47:15.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:15 smithi016 ceph-mon[93230]: failed to encode map e64 with expected crc 2024-04-02T04:47:15.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:15 smithi027 ceph-mon[83036]: failed to encode map e64 with expected crc 2024-04-02T04:47:15.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:15 smithi027 ceph-mon[83036]: failed to encode map e64 with expected crc 2024-04-02T04:47:15.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:15 smithi027 ceph-mon[83036]: failed to encode map e64 with expected crc 2024-04-02T04:47:15.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:15 smithi027 ceph-mon[83036]: failed to encode map e64 with expected crc 2024-04-02T04:47:15.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:15 smithi027 ceph-mon[83036]: failed to encode map e64 with expected crc 2024-04-02T04:47:15.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:15 smithi027 ceph-mon[83036]: failed to encode map e64 with expected crc 2024-04-02T04:47:16.386 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:16 smithi016 ceph-mon[93230]: pgmap v107: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 35/267 objects degraded (13.109%) 2024-04-02T04:47:16.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:16 smithi027 ceph-mon[83036]: pgmap v107: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 35/267 objects degraded (13.109%) 2024-04-02T04:47:18.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:18 smithi027 ceph-mon[83036]: pgmap v108: 65 pgs: 10 active+undersized+degraded, 12 active+undersized, 43 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 30/267 objects degraded (11.236%) 2024-04-02T04:47:18.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:18 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:18.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:18 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:47:18.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:18 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:18.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:18 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:18.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:18 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:47:18.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:18 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:47:18.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:18 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:18.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:18 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:47:18.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:18 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:47:18.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:18 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:47:18.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:18 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:47:18.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:18 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-02T04:47:18.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:18 smithi027 ceph-mon[83036]: Health check update: Degraded data redundancy: 30/267 objects degraded (11.236%), 10 pgs degraded (PG_DEGRADED) 2024-04-02T04:47:18.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:18 smithi016 ceph-mon[93230]: pgmap v108: 65 pgs: 10 active+undersized+degraded, 12 active+undersized, 43 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 30/267 objects degraded (11.236%) 2024-04-02T04:47:18.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:18 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:18.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:18 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:47:18.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:18 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:18.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:18 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:18.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:18 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:47:18.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:18 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:47:18.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:18 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:18.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:18 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:47:18.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:18 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:47:18.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:18 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:47:18.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:18 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:47:18.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:18 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-02T04:47:18.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:18 smithi016 ceph-mon[93230]: Health check update: Degraded data redundancy: 30/267 objects degraded (11.236%), 10 pgs degraded (PG_DEGRADED) 2024-04-02T04:47:19.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:19 smithi027 ceph-mon[83036]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-02T04:47:19.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:19 smithi027 ceph-mon[83036]: Upgrade: unsafe to stop osd(s) at this time (10 PGs are or would become offline) 2024-04-02T04:47:19.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:19 smithi016 ceph-mon[93230]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-02T04:47:19.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:19 smithi016 ceph-mon[93230]: Upgrade: unsafe to stop osd(s) at this time (10 PGs are or would become offline) 2024-04-02T04:47:20.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:20 smithi027 ceph-mon[83036]: pgmap v109: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:47:20.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:20 smithi027 ceph-mon[83036]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 30/267 objects degraded (11.236%), 10 pgs degraded) 2024-04-02T04:47:20.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:20 smithi016 ceph-mon[93230]: pgmap v109: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:47:20.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:20 smithi016 ceph-mon[93230]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 30/267 objects degraded (11.236%), 10 pgs degraded) 2024-04-02T04:47:22.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:22 smithi027 ceph-mon[83036]: pgmap v110: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:47:22.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:22 smithi016 ceph-mon[93230]: pgmap v110: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:47:24.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:24 smithi027 ceph-mon[83036]: pgmap v111: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:47:24.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:24 smithi016 ceph-mon[93230]: pgmap v111: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:47:26.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:26 smithi027 ceph-mon[83036]: pgmap v112: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:47:26.498 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:26 smithi016 ceph-mon[93230]: pgmap v112: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:47:28.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:28 smithi027 ceph-mon[83036]: pgmap v113: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:47:28.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:28 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:28.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:28 smithi016 ceph-mon[93230]: pgmap v113: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:47:28.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:28 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:30.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:30 smithi027 ceph-mon[83036]: pgmap v114: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:47:30.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:30 smithi016 ceph-mon[93230]: pgmap v114: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:47:32.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:32 smithi027 ceph-mon[83036]: pgmap v115: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:47:32.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:32 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:32.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:32 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:47:32.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:32 smithi016 ceph-mon[93230]: pgmap v115: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:47:32.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:32 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:32.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:32 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:47:33.344 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:33 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-02T04:47:33.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:33 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-02T04:47:33.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:33.848+0000 7fa277b88700 1 -- 172.21.15.16:0/3918111095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2700ff9e0 msgr2=0x7fa2700ffe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:33.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:33.848+0000 7fa277b88700 1 --2- 172.21.15.16:0/3918111095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2700ff9e0 0x7fa2700ffe00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa260009910 tx=0x7fa260009c20 comp rx=0 tx=0).stop 2024-04-02T04:47:33.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:33.848+0000 7fa277b88700 1 -- 172.21.15.16:0/3918111095 shutdown_connections 2024-04-02T04:47:33.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:33.848+0000 7fa277b88700 1 --2- 172.21.15.16:0/3918111095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa270100be0 0x7fa270101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:33.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:33.848+0000 7fa277b88700 1 --2- 172.21.15.16:0/3918111095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2700ff9e0 0x7fa2700ffe00 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:33.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:33.848+0000 7fa277b88700 1 -- 172.21.15.16:0/3918111095 >> 172.21.15.16:0/3918111095 conn(0x7fa2700faf80 msgr2=0x7fa2700fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:33.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:33.848+0000 7fa277b88700 1 -- 172.21.15.16:0/3918111095 shutdown_connections 2024-04-02T04:47:33.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:33.848+0000 7fa277b88700 1 -- 172.21.15.16:0/3918111095 wait complete. 2024-04-02T04:47:33.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:33.849+0000 7fa277b88700 1 Processor -- start 2024-04-02T04:47:33.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:33.849+0000 7fa277b88700 1 -- start start 2024-04-02T04:47:33.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:33.849+0000 7fa277b88700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2700ff9e0 0x7fa270103920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:33.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:33.849+0000 7fa277b88700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa270100be0 0x7fa270101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:33.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:33.849+0000 7fa277b88700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa270102500 con 0x7fa2700ff9e0 2024-04-02T04:47:33.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:33.849+0000 7fa277b88700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa270102640 con 0x7fa270100be0 2024-04-02T04:47:33.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:33.849+0000 7fa275924700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2700ff9e0 0x7fa270103920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:33.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:33.849+0000 7fa275924700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2700ff9e0 0x7fa270103920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40594/0 (socket says 172.21.15.16:40594) 2024-04-02T04:47:33.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:33.849+0000 7fa275924700 1 -- 172.21.15.16:0/2194193346 learned_addr learned my addr 172.21.15.16:0/2194193346 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:47:33.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:33.849+0000 7fa275123700 1 --2- 172.21.15.16:0/2194193346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa270100be0 0x7fa270101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:33.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:33.850+0000 7fa275924700 1 -- 172.21.15.16:0/2194193346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa270100be0 msgr2=0x7fa270101fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:33.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:33.850+0000 7fa275924700 1 --2- 172.21.15.16:0/2194193346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa270100be0 0x7fa270101fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:33.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:33.850+0000 7fa275924700 1 -- 172.21.15.16:0/2194193346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2600095f0 con 0x7fa2700ff9e0 2024-04-02T04:47:33.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:33.850+0000 7fa275924700 1 --2- 172.21.15.16:0/2194193346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2700ff9e0 0x7fa270103920 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa2600059d0 tx=0x7fa26000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:33.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:33.850+0000 7fa266ffd700 1 -- 172.21.15.16:0/2194193346 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa26001d070 con 0x7fa2700ff9e0 2024-04-02T04:47:33.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:33.850+0000 7fa266ffd700 1 -- 172.21.15.16:0/2194193346 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa260005dc0 con 0x7fa2700ff9e0 2024-04-02T04:47:33.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:33.850+0000 7fa277b88700 1 -- 172.21.15.16:0/2194193346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2701028c0 con 0x7fa2700ff9e0 2024-04-02T04:47:33.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:33.850+0000 7fa266ffd700 1 -- 172.21.15.16:0/2194193346 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa260021410 con 0x7fa2700ff9e0 2024-04-02T04:47:33.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:33.850+0000 7fa277b88700 1 -- 172.21.15.16:0/2194193346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa270102e30 con 0x7fa2700ff9e0 2024-04-02T04:47:33.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:33.851+0000 7fa277b88700 1 -- 172.21.15.16:0/2194193346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa27004ea80 con 0x7fa2700ff9e0 2024-04-02T04:47:33.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:33.852+0000 7fa266ffd700 1 -- 172.21.15.16:0/2194193346 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fa26000f840 con 0x7fa2700ff9e0 2024-04-02T04:47:33.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:33.852+0000 7fa266ffd700 1 --2- 172.21.15.16:0/2194193346 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fa25c06dd60 0x7fa25c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:33.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:33.852+0000 7fa266ffd700 1 -- 172.21.15.16:0/2194193346 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fa26008e9e0 con 0x7fa2700ff9e0 2024-04-02T04:47:33.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:33.852+0000 7fa275123700 1 --2- 172.21.15.16:0/2194193346 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fa25c06dd60 0x7fa25c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:33.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:33.853+0000 7fa275123700 1 --2- 172.21.15.16:0/2194193346 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fa25c06dd60 0x7fa25c070220 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa26c005950 tx=0x7fa26c0058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:33.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:33.857+0000 7fa266ffd700 1 -- 172.21.15.16:0/2194193346 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa26005bbd0 con 0x7fa2700ff9e0 2024-04-02T04:47:34.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.083+0000 7fa277b88700 1 -- 172.21.15.16:0/2194193346 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa270102e30 con 0x7fa25c06dd60 2024-04-02T04:47:34.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.085+0000 7fa266ffd700 1 -- 172.21.15.16:0/2194193346 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fa270102e30 con 0x7fa25c06dd60 2024-04-02T04:47:34.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.087+0000 7fa277b88700 1 -- 172.21.15.16:0/2194193346 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fa25c06dd60 msgr2=0x7fa25c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:34.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.087+0000 7fa277b88700 1 --2- 172.21.15.16:0/2194193346 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fa25c06dd60 0x7fa25c070220 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa26c005950 tx=0x7fa26c0058e0 comp rx=0 tx=0).stop 2024-04-02T04:47:34.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.087+0000 7fa277b88700 1 -- 172.21.15.16:0/2194193346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2700ff9e0 msgr2=0x7fa270103920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:34.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.087+0000 7fa277b88700 1 --2- 172.21.15.16:0/2194193346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2700ff9e0 0x7fa270103920 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa2600059d0 tx=0x7fa26000be20 comp rx=0 tx=0).stop 2024-04-02T04:47:34.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.087+0000 7fa277b88700 1 -- 172.21.15.16:0/2194193346 shutdown_connections 2024-04-02T04:47:34.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.087+0000 7fa277b88700 1 --2- 172.21.15.16:0/2194193346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2700ff9e0 0x7fa270103920 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:34.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.087+0000 7fa277b88700 1 --2- 172.21.15.16:0/2194193346 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fa25c06dd60 0x7fa25c070220 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:34.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.087+0000 7fa277b88700 1 --2- 172.21.15.16:0/2194193346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa270100be0 0x7fa270101fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:34.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.087+0000 7fa277b88700 1 -- 172.21.15.16:0/2194193346 >> 172.21.15.16:0/2194193346 conn(0x7fa2700faf80 msgr2=0x7fa270069040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:34.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.087+0000 7fa277b88700 1 -- 172.21.15.16:0/2194193346 shutdown_connections 2024-04-02T04:47:34.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.087+0000 7fa277b88700 1 -- 172.21.15.16:0/2194193346 wait complete. 2024-04-02T04:47:34.102 INFO:teuthology.orchestra.run.smithi016.stdout:true 2024-04-02T04:47:34.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.195+0000 7efc563b1700 1 -- 172.21.15.16:0/3729151920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc501015d0 msgr2=0x7efc501039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:34.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.195+0000 7efc563b1700 1 --2- 172.21.15.16:0/3729151920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc501015d0 0x7efc501039c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efc38009910 tx=0x7efc38009c20 comp rx=0 tx=0).stop 2024-04-02T04:47:34.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.195+0000 7efc563b1700 1 -- 172.21.15.16:0/3729151920 shutdown_connections 2024-04-02T04:47:34.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.195+0000 7efc563b1700 1 --2- 172.21.15.16:0/3729151920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc50103f00 0x7efc501062f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:34.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.195+0000 7efc563b1700 1 --2- 172.21.15.16:0/3729151920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc501015d0 0x7efc501039c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:34.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.195+0000 7efc563b1700 1 -- 172.21.15.16:0/3729151920 >> 172.21.15.16:0/3729151920 conn(0x7efc500faf80 msgr2=0x7efc500fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:34.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.196+0000 7efc563b1700 1 -- 172.21.15.16:0/3729151920 shutdown_connections 2024-04-02T04:47:34.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.196+0000 7efc563b1700 1 -- 172.21.15.16:0/3729151920 wait complete. 2024-04-02T04:47:34.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.196+0000 7efc563b1700 1 Processor -- start 2024-04-02T04:47:34.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.196+0000 7efc563b1700 1 -- start start 2024-04-02T04:47:34.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.197+0000 7efc563b1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc501015d0 0x7efc50196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:34.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.197+0000 7efc563b1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc50103f00 0x7efc501969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:34.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.197+0000 7efc563b1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc50196fe0 con 0x7efc501015d0 2024-04-02T04:47:34.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.197+0000 7efc563b1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc50197120 con 0x7efc50103f00 2024-04-02T04:47:34.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.197+0000 7efc4ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc501015d0 0x7efc50196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:34.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.197+0000 7efc4ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc501015d0 0x7efc50196480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40608/0 (socket says 172.21.15.16:40608) 2024-04-02T04:47:34.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.197+0000 7efc4ffff700 1 -- 172.21.15.16:0/3252613379 learned_addr learned my addr 172.21.15.16:0/3252613379 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:47:34.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.197+0000 7efc4f7fe700 1 --2- 172.21.15.16:0/3252613379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc50103f00 0x7efc501969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:34.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.197+0000 7efc4ffff700 1 -- 172.21.15.16:0/3252613379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc50103f00 msgr2=0x7efc501969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:34.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.197+0000 7efc4ffff700 1 --2- 172.21.15.16:0/3252613379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc50103f00 0x7efc501969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:34.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.197+0000 7efc4ffff700 1 -- 172.21.15.16:0/3252613379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc380095f0 con 0x7efc501015d0 2024-04-02T04:47:34.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.197+0000 7efc4ffff700 1 --2- 172.21.15.16:0/3252613379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc501015d0 0x7efc50196480 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7efc38006010 tx=0x7efc3800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:34.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.197+0000 7efc4d7fa700 1 -- 172.21.15.16:0/3252613379 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc3801d070 con 0x7efc501015d0 2024-04-02T04:47:34.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.197+0000 7efc4d7fa700 1 -- 172.21.15.16:0/3252613379 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc3800f460 con 0x7efc501015d0 2024-04-02T04:47:34.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.197+0000 7efc563b1700 1 -- 172.21.15.16:0/3252613379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc5019bb20 con 0x7efc501015d0 2024-04-02T04:47:34.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.197+0000 7efc4d7fa700 1 -- 172.21.15.16:0/3252613379 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc38021620 con 0x7efc501015d0 2024-04-02T04:47:34.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.197+0000 7efc563b1700 1 -- 172.21.15.16:0/3252613379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc5019c010 con 0x7efc501015d0 2024-04-02T04:47:34.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.198+0000 7efc563b1700 1 -- 172.21.15.16:0/3252613379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc5004ea80 con 0x7efc501015d0 2024-04-02T04:47:34.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.199+0000 7efc4d7fa700 1 -- 172.21.15.16:0/3252613379 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7efc3800f5d0 con 0x7efc501015d0 2024-04-02T04:47:34.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.200+0000 7efc4d7fa700 1 --2- 172.21.15.16:0/3252613379 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7efc3c06dd60 0x7efc3c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:34.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.200+0000 7efc4d7fa700 1 -- 172.21.15.16:0/3252613379 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6500+0+0 (secure 0 0 0) 0x7efc3808ecb0 con 0x7efc501015d0 2024-04-02T04:47:34.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.200+0000 7efc4f7fe700 1 --2- 172.21.15.16:0/3252613379 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7efc3c06dd60 0x7efc3c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:34.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.200+0000 7efc4f7fe700 1 --2- 172.21.15.16:0/3252613379 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7efc3c06dd60 0x7efc3c070220 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efc40007c60 tx=0x7efc400073d0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:34.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.204+0000 7efc4d7fa700 1 -- 172.21.15.16:0/3252613379 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7efc3805bdf0 con 0x7efc501015d0 2024-04-02T04:47:34.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:34 smithi027 ceph-mon[83036]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-02T04:47:34.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:34 smithi027 ceph-mon[83036]: Upgrade: osd.3 is safe to restart 2024-04-02T04:47:34.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:34 smithi027 ceph-mon[83036]: pgmap v116: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:47:34.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:34 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:34.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:34 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-02T04:47:34.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:34 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:47:34.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.429+0000 7efc563b1700 1 -- 172.21.15.16:0/3252613379 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efc50105dc0 con 0x7efc3c06dd60 2024-04-02T04:47:34.433 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:34 smithi016 ceph-mon[93230]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-02T04:47:34.433 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:34 smithi016 ceph-mon[93230]: Upgrade: osd.3 is safe to restart 2024-04-02T04:47:34.433 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:34 smithi016 ceph-mon[93230]: pgmap v116: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:47:34.433 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:34 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:34.433 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:34 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-02T04:47:34.433 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:34 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:47:34.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.430+0000 7efc4d7fa700 1 -- 172.21.15.16:0/3252613379 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7efc50105dc0 con 0x7efc3c06dd60 2024-04-02T04:47:34.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.433+0000 7efc563b1700 1 -- 172.21.15.16:0/3252613379 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7efc3c06dd60 msgr2=0x7efc3c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:34.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.433+0000 7efc563b1700 1 --2- 172.21.15.16:0/3252613379 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7efc3c06dd60 0x7efc3c070220 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efc40007c60 tx=0x7efc400073d0 comp rx=0 tx=0).stop 2024-04-02T04:47:34.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.433+0000 7efc563b1700 1 -- 172.21.15.16:0/3252613379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc501015d0 msgr2=0x7efc50196480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:34.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.433+0000 7efc563b1700 1 --2- 172.21.15.16:0/3252613379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc501015d0 0x7efc50196480 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7efc38006010 tx=0x7efc3800bd70 comp rx=0 tx=0).stop 2024-04-02T04:47:34.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.433+0000 7efc563b1700 1 -- 172.21.15.16:0/3252613379 shutdown_connections 2024-04-02T04:47:34.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.433+0000 7efc563b1700 1 --2- 172.21.15.16:0/3252613379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc501015d0 0x7efc50196480 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:34.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.433+0000 7efc563b1700 1 --2- 172.21.15.16:0/3252613379 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7efc3c06dd60 0x7efc3c070220 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:34.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.433+0000 7efc563b1700 1 --2- 172.21.15.16:0/3252613379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc50103f00 0x7efc501969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:34.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.433+0000 7efc563b1700 1 -- 172.21.15.16:0/3252613379 >> 172.21.15.16:0/3252613379 conn(0x7efc500faf80 msgr2=0x7efc500fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:34.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.433+0000 7efc563b1700 1 -- 172.21.15.16:0/3252613379 shutdown_connections 2024-04-02T04:47:34.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.433+0000 7efc563b1700 1 -- 172.21.15.16:0/3252613379 wait complete. 2024-04-02T04:47:34.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.538+0000 7fc024517700 1 -- 172.21.15.16:0/1892476737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc01c100390 msgr2=0x7fc01c1007f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:34.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.538+0000 7fc024517700 1 --2- 172.21.15.16:0/1892476737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc01c100390 0x7fc01c1007f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc00c009910 tx=0x7fc00c009c20 comp rx=0 tx=0).stop 2024-04-02T04:47:34.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.538+0000 7fc024517700 1 -- 172.21.15.16:0/1892476737 shutdown_connections 2024-04-02T04:47:34.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.538+0000 7fc024517700 1 --2- 172.21.15.16:0/1892476737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc01c100390 0x7fc01c1007f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:34.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.538+0000 7fc024517700 1 --2- 172.21.15.16:0/1892476737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc01c0ff190 0x7fc01c0ff5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:34.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.538+0000 7fc024517700 1 -- 172.21.15.16:0/1892476737 >> 172.21.15.16:0/1892476737 conn(0x7fc01c0fa730 msgr2=0x7fc01c0fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:34.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.538+0000 7fc024517700 1 -- 172.21.15.16:0/1892476737 shutdown_connections 2024-04-02T04:47:34.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.538+0000 7fc024517700 1 -- 172.21.15.16:0/1892476737 wait complete. 2024-04-02T04:47:34.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.538+0000 7fc024517700 1 Processor -- start 2024-04-02T04:47:34.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.538+0000 7fc024517700 1 -- start start 2024-04-02T04:47:34.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.539+0000 7fc024517700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc01c0ff190 0x7fc01c0fcb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:34.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.539+0000 7fc024517700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc01c100390 0x7fc01c19d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:34.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.539+0000 7fc024517700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01c1970e0 con 0x7fc01c100390 2024-04-02T04:47:34.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.539+0000 7fc024517700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01c19dbb0 con 0x7fc01c0ff190 2024-04-02T04:47:34.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.539+0000 7fc021ab2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc01c100390 0x7fc01c19d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:34.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.539+0000 7fc021ab2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc01c100390 0x7fc01c19d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40624/0 (socket says 172.21.15.16:40624) 2024-04-02T04:47:34.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.539+0000 7fc021ab2700 1 -- 172.21.15.16:0/62310645 learned_addr learned my addr 172.21.15.16:0/62310645 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:47:34.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.539+0000 7fc0222b3700 1 --2- 172.21.15.16:0/62310645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc01c0ff190 0x7fc01c0fcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:34.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.539+0000 7fc021ab2700 1 -- 172.21.15.16:0/62310645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc01c0ff190 msgr2=0x7fc01c0fcb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:34.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.539+0000 7fc021ab2700 1 --2- 172.21.15.16:0/62310645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc01c0ff190 0x7fc01c0fcb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:34.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.539+0000 7fc021ab2700 1 -- 172.21.15.16:0/62310645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc00c0095f0 con 0x7fc01c100390 2024-04-02T04:47:34.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.539+0000 7fc0222b3700 1 --2- 172.21.15.16:0/62310645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc01c0ff190 0x7fc01c0fcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T04:47:34.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.539+0000 7fc021ab2700 1 --2- 172.21.15.16:0/62310645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc01c100390 0x7fc01c19d670 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc00c005e30 tx=0x7fc00c005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:34.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.539+0000 7fc0137fe700 1 -- 172.21.15.16:0/62310645 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc00c01d070 con 0x7fc01c100390 2024-04-02T04:47:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.539+0000 7fc0137fe700 1 -- 172.21.15.16:0/62310645 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc00c0052a0 con 0x7fc01c100390 2024-04-02T04:47:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.539+0000 7fc0137fe700 1 -- 172.21.15.16:0/62310645 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc00c017400 con 0x7fc01c100390 2024-04-02T04:47:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.539+0000 7fc024517700 1 -- 172.21.15.16:0/62310645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc01c19ddb0 con 0x7fc01c100390 2024-04-02T04:47:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.540+0000 7fc024517700 1 -- 172.21.15.16:0/62310645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc01c19e320 con 0x7fc01c100390 2024-04-02T04:47:34.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.540+0000 7fc024517700 1 -- 172.21.15.16:0/62310645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc01c18ed10 con 0x7fc01c100390 2024-04-02T04:47:34.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.541+0000 7fc0137fe700 1 -- 172.21.15.16:0/62310645 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fc00c003710 con 0x7fc01c100390 2024-04-02T04:47:34.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.541+0000 7fc0137fe700 1 --2- 172.21.15.16:0/62310645 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc00806dd70 0x7fc008070230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:34.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.542+0000 7fc0222b3700 1 --2- 172.21.15.16:0/62310645 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc00806dd70 0x7fc008070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:34.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.542+0000 7fc0137fe700 1 -- 172.21.15.16:0/62310645 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fc00c08ef70 con 0x7fc01c100390 2024-04-02T04:47:34.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.542+0000 7fc0222b3700 1 --2- 172.21.15.16:0/62310645 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc00806dd70 0x7fc008070230 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc01c1001f0 tx=0x7fc018009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:34.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.546+0000 7fc0137fe700 1 -- 172.21.15.16:0/62310645 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc00c05bf90 con 0x7fc01c100390 2024-04-02T04:47:34.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.760+0000 7fc024517700 1 -- 172.21.15.16:0/62310645 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc01c02ce10 con 0x7fc00806dd70 2024-04-02T04:47:34.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.771+0000 7fc0137fe700 1 -- 172.21.15.16:0/62310645 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fc01c02ce10 con 0x7fc00806dd70 2024-04-02T04:47:34.775 INFO:teuthology.orchestra.run.smithi016.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:47:34.775 INFO:teuthology.orchestra.run.smithi016.stdout:alertmanager.smithi016 smithi016 *:9093,9094 running (11m) 22s ago 13m 24.1M - 0.25.0 c8568f914cd2 b95d8354b016 2024-04-02T04:47:34.775 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi016 smithi016 running (13m) 22s ago 13m 8300k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 9de6c16a487f 2024-04-02T04:47:34.775 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi027 smithi027 running (12m) 114s ago 12m 7964k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 99956592471a 2024-04-02T04:47:34.775 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi016 smithi016 running (2m) 22s ago 13m 6660k - 19.0.0-1979-g6cec793d 78f830190839 dc63ce0eef58 2024-04-02T04:47:34.775 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi027 smithi027 running (118s) 114s ago 12m 6665k - 19.0.0-1979-g6cec793d 78f830190839 2beb077018e4 2024-04-02T04:47:34.776 INFO:teuthology.orchestra.run.smithi016.stdout:grafana.smithi016 smithi016 *:3000 running (11m) 22s ago 13m 84.6M - 9.4.7 954c08fa6188 7c43d133569c 2024-04-02T04:47:34.776 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.fyzrnm smithi016 running (7m) 22s ago 7m 17.9M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 70f28b9572a6 2024-04-02T04:47:34.776 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.uhgkgg smithi016 running (7m) 22s ago 7m 16.1M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 0aa4d16562a6 2024-04-02T04:47:34.776 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.sdublg smithi027 running (7m) 114s ago 7m 19.3M - 18.2.2-1059-g4d05f4ca 909dce4fe53d a538451099db 2024-04-02T04:47:34.776 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.zwrfoo smithi027 running (7m) 114s ago 7m 158M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 4cb3fa428d71 2024-04-02T04:47:34.776 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi016.eqnebx smithi016 *:8443,9283,8765 running (4m) 22s ago 14m 501M - 19.0.0-1979-g6cec793d 78f830190839 d61ed91b4577 2024-04-02T04:47:34.776 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi027.kjzcsc smithi027 *:8443,9283,8765 running (3m) 114s ago 12m 449M - 19.0.0-1979-g6cec793d 78f830190839 5072a5ffc761 2024-04-02T04:47:34.776 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi016 smithi016 running (3m) 22s ago 15m 46.9M 2048M 19.0.0-1979-g6cec793d 78f830190839 eabfc84f1cdf 2024-04-02T04:47:34.776 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi027 smithi027 running (2m) 114s ago 12m 36.8M 2048M 19.0.0-1979-g6cec793d 78f830190839 fcd68b20dcc5 2024-04-02T04:47:34.776 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi016 smithi016 *:9100 running (13m) 22s ago 13m 15.2M - 1.5.0 0da6a335fe13 09f54c65b53c 2024-04-02T04:47:34.776 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi027 smithi027 *:9100 running (12m) 114s ago 12m 15.2M - 1.5.0 0da6a335fe13 e41a657c7443 2024-04-02T04:47:34.776 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 smithi016 running (98s) 22s ago 11m 81.5M 1217M 19.0.0-1979-g6cec793d 78f830190839 421d9e1cc751 2024-04-02T04:47:34.776 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 smithi016 running (63s) 22s ago 11m 87.0M 1217M 19.0.0-1979-g6cec793d 78f830190839 3612a56236a3 2024-04-02T04:47:34.776 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 smithi016 running (27s) 22s ago 10m 17.1M 1217M 19.0.0-1979-g6cec793d 78f830190839 a8e9b8d0303c 2024-04-02T04:47:34.776 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 smithi027 running (10m) 114s ago 10m 236M 2240M 18.2.2-1059-g4d05f4ca 909dce4fe53d cf7c7fe022f0 2024-04-02T04:47:34.776 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 smithi027 running (9m) 114s ago 9m 211M 2240M 18.2.2-1059-g4d05f4ca 909dce4fe53d 8cbbedf8cac9 2024-04-02T04:47:34.777 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 smithi027 running (9m) 114s ago 9m 194M 2240M 18.2.2-1059-g4d05f4ca 909dce4fe53d ac6094415529 2024-04-02T04:47:34.777 INFO:teuthology.orchestra.run.smithi016.stdout:prometheus.smithi016 smithi016 *:9095 running (4m) 22s ago 13m 58.5M - 2.43.0 a07b618ecd1d 466f00d73ddd 2024-04-02T04:47:34.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.774+0000 7fc024517700 1 -- 172.21.15.16:0/62310645 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc00806dd70 msgr2=0x7fc008070230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:34.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.774+0000 7fc024517700 1 --2- 172.21.15.16:0/62310645 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc00806dd70 0x7fc008070230 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc01c1001f0 tx=0x7fc018009260 comp rx=0 tx=0).stop 2024-04-02T04:47:34.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.774+0000 7fc024517700 1 -- 172.21.15.16:0/62310645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc01c100390 msgr2=0x7fc01c19d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:34.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.774+0000 7fc024517700 1 --2- 172.21.15.16:0/62310645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc01c100390 0x7fc01c19d670 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc00c005e30 tx=0x7fc00c005e60 comp rx=0 tx=0).stop 2024-04-02T04:47:34.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.774+0000 7fc024517700 1 -- 172.21.15.16:0/62310645 shutdown_connections 2024-04-02T04:47:34.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.774+0000 7fc024517700 1 --2- 172.21.15.16:0/62310645 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc00806dd70 0x7fc008070230 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:34.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.774+0000 7fc024517700 1 --2- 172.21.15.16:0/62310645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc01c0ff190 0x7fc01c0fcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:34.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.774+0000 7fc024517700 1 --2- 172.21.15.16:0/62310645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc01c100390 0x7fc01c19d670 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:34.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.774+0000 7fc024517700 1 -- 172.21.15.16:0/62310645 >> 172.21.15.16:0/62310645 conn(0x7fc01c0fa730 msgr2=0x7fc01c103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:34.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.774+0000 7fc024517700 1 -- 172.21.15.16:0/62310645 shutdown_connections 2024-04-02T04:47:34.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.774+0000 7fc024517700 1 -- 172.21.15.16:0/62310645 wait complete. 2024-04-02T04:47:34.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.877+0000 7f4d5ba74700 1 -- 172.21.15.16:0/2794617906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d541015f0 msgr2=0x7f4d541039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:34.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.877+0000 7f4d5ba74700 1 --2- 172.21.15.16:0/2794617906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d541015f0 0x7f4d541039e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4d5000b0a0 tx=0x7f4d5000b3b0 comp rx=0 tx=0).stop 2024-04-02T04:47:34.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.877+0000 7f4d5ba74700 1 -- 172.21.15.16:0/2794617906 shutdown_connections 2024-04-02T04:47:34.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.877+0000 7f4d5ba74700 1 --2- 172.21.15.16:0/2794617906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d54103f20 0x7f4d54106310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:34.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.877+0000 7f4d5ba74700 1 --2- 172.21.15.16:0/2794617906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d541015f0 0x7f4d541039e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:34.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.877+0000 7f4d5ba74700 1 -- 172.21.15.16:0/2794617906 >> 172.21.15.16:0/2794617906 conn(0x7f4d540faf80 msgr2=0x7f4d540fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:34.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.878+0000 7f4d5ba74700 1 -- 172.21.15.16:0/2794617906 shutdown_connections 2024-04-02T04:47:34.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.878+0000 7f4d5ba74700 1 -- 172.21.15.16:0/2794617906 wait complete. 2024-04-02T04:47:34.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.878+0000 7f4d5ba74700 1 Processor -- start 2024-04-02T04:47:34.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.878+0000 7f4d5ba74700 1 -- start start 2024-04-02T04:47:34.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.878+0000 7f4d5ba74700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d541015f0 0x7f4d54196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:34.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.878+0000 7f4d5ba74700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d54103f20 0x7f4d541969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:34.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.878+0000 7f4d5ba74700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d54196fe0 con 0x7f4d54103f20 2024-04-02T04:47:34.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.878+0000 7f4d5ba74700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d54197120 con 0x7f4d541015f0 2024-04-02T04:47:34.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.878+0000 7f4d5900f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d54103f20 0x7f4d541969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:34.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.878+0000 7f4d5900f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d54103f20 0x7f4d541969c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40636/0 (socket says 172.21.15.16:40636) 2024-04-02T04:47:34.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.878+0000 7f4d5900f700 1 -- 172.21.15.16:0/1592374488 learned_addr learned my addr 172.21.15.16:0/1592374488 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:47:34.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.878+0000 7f4d59810700 1 --2- 172.21.15.16:0/1592374488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d541015f0 0x7f4d54196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:34.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.879+0000 7f4d5900f700 1 -- 172.21.15.16:0/1592374488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d541015f0 msgr2=0x7f4d54196480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:34.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.879+0000 7f4d5900f700 1 --2- 172.21.15.16:0/1592374488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d541015f0 0x7f4d54196480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:34.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.879+0000 7f4d5900f700 1 -- 172.21.15.16:0/1592374488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d50009d00 con 0x7f4d54103f20 2024-04-02T04:47:34.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.879+0000 7f4d5900f700 1 --2- 172.21.15.16:0/1592374488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d54103f20 0x7f4d541969c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400d900 tx=0x7f4d4400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:34.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.879+0000 7f4d4affd700 1 -- 172.21.15.16:0/1592374488 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d440041f0 con 0x7f4d54103f20 2024-04-02T04:47:34.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.879+0000 7f4d4affd700 1 -- 172.21.15.16:0/1592374488 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4d44004cf0 con 0x7f4d54103f20 2024-04-02T04:47:34.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.879+0000 7f4d5ba74700 1 -- 172.21.15.16:0/1592374488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d5419bb80 con 0x7f4d54103f20 2024-04-02T04:47:34.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.879+0000 7f4d4affd700 1 -- 172.21.15.16:0/1592374488 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d4400b750 con 0x7f4d54103f20 2024-04-02T04:47:34.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.879+0000 7f4d5ba74700 1 -- 172.21.15.16:0/1592374488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d5419c0d0 con 0x7f4d54103f20 2024-04-02T04:47:34.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.880+0000 7f4d5ba74700 1 -- 172.21.15.16:0/1592374488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d5404ea80 con 0x7f4d54103f20 2024-04-02T04:47:34.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.881+0000 7f4d4affd700 1 -- 172.21.15.16:0/1592374488 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f4d44004750 con 0x7f4d54103f20 2024-04-02T04:47:34.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.881+0000 7f4d4affd700 1 --2- 172.21.15.16:0/1592374488 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f4d4006dd60 0x7f4d40070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:34.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.881+0000 7f4d4affd700 1 -- 172.21.15.16:0/1592374488 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f4d4408d9a0 con 0x7f4d54103f20 2024-04-02T04:47:34.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.881+0000 7f4d59810700 1 --2- 172.21.15.16:0/1592374488 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f4d4006dd60 0x7f4d40070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:34.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.882+0000 7f4d59810700 1 --2- 172.21.15.16:0/1592374488 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f4d4006dd60 0x7f4d40070220 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4d50007820 tx=0x7f4d50006000 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:34.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:34.887+0000 7f4d4affd700 1 -- 172.21.15.16:0/1592374488 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4d4405aae0 con 0x7f4d54103f20 2024-04-02T04:47:35.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.179+0000 7f4d5ba74700 1 -- 172.21.15.16:0/1592374488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4d5419bd10 con 0x7f4d54103f20 2024-04-02T04:47:35.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.179+0000 7f4d4affd700 1 -- 172.21.15.16:0/1592374488 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+836 (secure 0 0 0) 0x7f4d44016360 con 0x7f4d54103f20 2024-04-02T04:47:35.183 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:47:35.184 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": { 2024-04-02T04:47:35.184 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:47:35.184 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:47:35.184 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": { 2024-04-02T04:47:35.184 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:47:35.184 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:47:35.184 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": { 2024-04-02T04:47:35.184 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 3, 2024-04-02T04:47:35.184 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 3 2024-04-02T04:47:35.184 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:47:35.184 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-04-02T04:47:35.184 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 4 2024-04-02T04:47:35.184 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:47:35.184 INFO:teuthology.orchestra.run.smithi016.stdout: "overall": { 2024-04-02T04:47:35.184 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 7, 2024-04-02T04:47:35.185 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 7 2024-04-02T04:47:35.185 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-02T04:47:35.185 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:47:35.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.181+0000 7f4d5ba74700 1 -- 172.21.15.16:0/1592374488 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f4d4006dd60 msgr2=0x7f4d40070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:35.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.181+0000 7f4d5ba74700 1 --2- 172.21.15.16:0/1592374488 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f4d4006dd60 0x7f4d40070220 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4d50007820 tx=0x7f4d50006000 comp rx=0 tx=0).stop 2024-04-02T04:47:35.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.181+0000 7f4d5ba74700 1 -- 172.21.15.16:0/1592374488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d54103f20 msgr2=0x7f4d541969c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:35.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.181+0000 7f4d5ba74700 1 --2- 172.21.15.16:0/1592374488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d54103f20 0x7f4d541969c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400d900 tx=0x7f4d4400dc10 comp rx=0 tx=0).stop 2024-04-02T04:47:35.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.182+0000 7f4d5ba74700 1 -- 172.21.15.16:0/1592374488 shutdown_connections 2024-04-02T04:47:35.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.182+0000 7f4d5ba74700 1 --2- 172.21.15.16:0/1592374488 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f4d4006dd60 0x7f4d40070220 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:35.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.182+0000 7f4d5ba74700 1 --2- 172.21.15.16:0/1592374488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d541015f0 0x7f4d54196480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:35.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.182+0000 7f4d5ba74700 1 --2- 172.21.15.16:0/1592374488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d54103f20 0x7f4d541969c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:35.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.182+0000 7f4d5ba74700 1 -- 172.21.15.16:0/1592374488 >> 172.21.15.16:0/1592374488 conn(0x7f4d540faf80 msgr2=0x7f4d540fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:35.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.182+0000 7f4d5ba74700 1 -- 172.21.15.16:0/1592374488 shutdown_connections 2024-04-02T04:47:35.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.182+0000 7f4d5ba74700 1 -- 172.21.15.16:0/1592374488 wait complete. 2024-04-02T04:47:35.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.287+0000 7fa301b99700 1 -- 172.21.15.16:0/4074940099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2fc103f70 msgr2=0x7fa2fc106360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:35.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.287+0000 7fa301b99700 1 --2- 172.21.15.16:0/4074940099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2fc103f70 0x7fa2fc106360 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4009910 tx=0x7fa2e4009c20 comp rx=0 tx=0).stop 2024-04-02T04:47:35.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.287+0000 7fa301b99700 1 -- 172.21.15.16:0/4074940099 shutdown_connections 2024-04-02T04:47:35.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.287+0000 7fa301b99700 1 --2- 172.21.15.16:0/4074940099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2fc103f70 0x7fa2fc106360 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:35.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.287+0000 7fa301b99700 1 --2- 172.21.15.16:0/4074940099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2fc101640 0x7fa2fc103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:35.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.287+0000 7fa301b99700 1 -- 172.21.15.16:0/4074940099 >> 172.21.15.16:0/4074940099 conn(0x7fa2fc0faf70 msgr2=0x7fa2fc0fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:35.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.287+0000 7fa301b99700 1 -- 172.21.15.16:0/4074940099 shutdown_connections 2024-04-02T04:47:35.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.287+0000 7fa301b99700 1 -- 172.21.15.16:0/4074940099 wait complete. 2024-04-02T04:47:35.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.287+0000 7fa301b99700 1 Processor -- start 2024-04-02T04:47:35.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.288+0000 7fa301b99700 1 -- start start 2024-04-02T04:47:35.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.288+0000 7fa301b99700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2fc101640 0x7fa2fc196510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:35.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.288+0000 7fa301b99700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2fc103f70 0x7fa2fc196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:35.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.288+0000 7fa301b99700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2fc197070 con 0x7fa2fc103f70 2024-04-02T04:47:35.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.288+0000 7fa301b99700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2fc1971b0 con 0x7fa2fc101640 2024-04-02T04:47:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.288+0000 7fa2faffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2fc103f70 0x7fa2fc196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.288+0000 7fa2faffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2fc103f70 0x7fa2fc196a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40656/0 (socket says 172.21.15.16:40656) 2024-04-02T04:47:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.288+0000 7fa2faffd700 1 -- 172.21.15.16:0/3023515903 learned_addr learned my addr 172.21.15.16:0/3023515903 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:47:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.288+0000 7fa2fb7fe700 1 --2- 172.21.15.16:0/3023515903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2fc101640 0x7fa2fc196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.288+0000 7fa2faffd700 1 -- 172.21.15.16:0/3023515903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2fc101640 msgr2=0x7fa2fc196510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.288+0000 7fa2faffd700 1 --2- 172.21.15.16:0/3023515903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2fc101640 0x7fa2fc196510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.288+0000 7fa2faffd700 1 -- 172.21.15.16:0/3023515903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e40095f0 con 0x7fa2fc103f70 2024-04-02T04:47:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.288+0000 7fa2faffd700 1 --2- 172.21.15.16:0/3023515903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2fc103f70 0x7fa2fc196a50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4006010 tx=0x7fa2e400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.289+0000 7fa2f8ff9700 1 -- 172.21.15.16:0/3023515903 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2e401d070 con 0x7fa2fc103f70 2024-04-02T04:47:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.289+0000 7fa2f8ff9700 1 -- 172.21.15.16:0/3023515903 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa2e400f460 con 0x7fa2fc103f70 2024-04-02T04:47:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.289+0000 7fa301b99700 1 -- 172.21.15.16:0/3023515903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2fc19bbb0 con 0x7fa2fc103f70 2024-04-02T04:47:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.289+0000 7fa2f8ff9700 1 -- 172.21.15.16:0/3023515903 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2e4021640 con 0x7fa2fc103f70 2024-04-02T04:47:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.289+0000 7fa301b99700 1 -- 172.21.15.16:0/3023515903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2fc19c120 con 0x7fa2fc103f70 2024-04-02T04:47:35.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.290+0000 7fa301b99700 1 -- 172.21.15.16:0/3023515903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2fc04ea80 con 0x7fa2fc103f70 2024-04-02T04:47:35.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.290+0000 7fa2f8ff9700 1 -- 172.21.15.16:0/3023515903 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fa2e400f5d0 con 0x7fa2fc103f70 2024-04-02T04:47:35.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.291+0000 7fa2f8ff9700 1 --2- 172.21.15.16:0/3023515903 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fa2e806dd60 0x7fa2e8070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:35.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.291+0000 7fa2f8ff9700 1 -- 172.21.15.16:0/3023515903 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fa2e408f000 con 0x7fa2fc103f70 2024-04-02T04:47:35.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.291+0000 7fa2fb7fe700 1 --2- 172.21.15.16:0/3023515903 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fa2e806dd60 0x7fa2e8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:35.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.291+0000 7fa2fb7fe700 1 --2- 172.21.15.16:0/3023515903 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fa2e806dd60 0x7fa2e8070220 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec0060b0 tx=0x7fa2ec007b80 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:35.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.295+0000 7fa2f8ff9700 1 -- 172.21.15.16:0/3023515903 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa2e40c3280 con 0x7fa2fc103f70 2024-04-02T04:47:35.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:35 smithi027 ceph-mon[83036]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:47:35.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:35 smithi027 ceph-mon[83036]: Upgrade: Updating osd.3 2024-04-02T04:47:35.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:35 smithi027 ceph-mon[83036]: Deploying daemon osd.3 on smithi027 2024-04-02T04:47:35.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.546+0000 7fa301b99700 1 -- 172.21.15.16:0/3023515903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa2fc109680 con 0x7fa2fc103f70 2024-04-02T04:47:35.550 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:35 smithi016 ceph-mon[93230]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:47:35.550 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:35 smithi016 ceph-mon[93230]: Upgrade: Updating osd.3 2024-04-02T04:47:35.550 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:35 smithi016 ceph-mon[93230]: Deploying daemon osd.3 on smithi027 2024-04-02T04:47:35.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.546+0000 7fa2f8ff9700 1 -- 172.21.15.16:0/3023515903 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1875 (secure 0 0 0) 0x7fa2e405c020 con 0x7fa2fc103f70 2024-04-02T04:47:35.551 INFO:teuthology.orchestra.run.smithi016.stdout:e15 2024-04-02T04:47:35.551 INFO:teuthology.orchestra.run.smithi016.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:47:35.551 INFO:teuthology.orchestra.run.smithi016.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:47:35.551 INFO:teuthology.orchestra.run.smithi016.stdout:legacy client fscid: 1 2024-04-02T04:47:35.551 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:47:35.551 INFO:teuthology.orchestra.run.smithi016.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:47:35.551 INFO:teuthology.orchestra.run.smithi016.stdout:fs_name cephfs 2024-04-02T04:47:35.551 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 15 2024-04-02T04:47:35.551 INFO:teuthology.orchestra.run.smithi016.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:47:35.552 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-04-02T04:39:30.740168+0000 2024-04-02T04:47:35.552 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-04-02T04:39:49.875319+0000 2024-04-02T04:47:35.552 INFO:teuthology.orchestra.run.smithi016.stdout:tableserver 0 2024-04-02T04:47:35.552 INFO:teuthology.orchestra.run.smithi016.stdout:root 0 2024-04-02T04:47:35.552 INFO:teuthology.orchestra.run.smithi016.stdout:session_timeout 60 2024-04-02T04:47:35.552 INFO:teuthology.orchestra.run.smithi016.stdout:session_autoclose 300 2024-04-02T04:47:35.552 INFO:teuthology.orchestra.run.smithi016.stdout:max_file_size 1099511627776 2024-04-02T04:47:35.552 INFO:teuthology.orchestra.run.smithi016.stdout:max_xattr_size 65536 2024-04-02T04:47:35.552 INFO:teuthology.orchestra.run.smithi016.stdout:required_client_features {} 2024-04-02T04:47:35.552 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure 0 2024-04-02T04:47:35.552 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure_osd_epoch 46 2024-04-02T04:47:35.552 INFO:teuthology.orchestra.run.smithi016.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:47:35.552 INFO:teuthology.orchestra.run.smithi016.stdout:max_mds 1 2024-04-02T04:47:35.552 INFO:teuthology.orchestra.run.smithi016.stdout:in 0 2024-04-02T04:47:35.553 INFO:teuthology.orchestra.run.smithi016.stdout:up {0=24285} 2024-04-02T04:47:35.553 INFO:teuthology.orchestra.run.smithi016.stdout:failed 2024-04-02T04:47:35.553 INFO:teuthology.orchestra.run.smithi016.stdout:damaged 2024-04-02T04:47:35.553 INFO:teuthology.orchestra.run.smithi016.stdout:stopped 2024-04-02T04:47:35.553 INFO:teuthology.orchestra.run.smithi016.stdout:data_pools [3] 2024-04-02T04:47:35.553 INFO:teuthology.orchestra.run.smithi016.stdout:metadata_pool 2 2024-04-02T04:47:35.553 INFO:teuthology.orchestra.run.smithi016.stdout:inline_data enabled 2024-04-02T04:47:35.553 INFO:teuthology.orchestra.run.smithi016.stdout:balancer 2024-04-02T04:47:35.553 INFO:teuthology.orchestra.run.smithi016.stdout:bal_rank_mask -1 2024-04-02T04:47:35.553 INFO:teuthology.orchestra.run.smithi016.stdout:standby_count_wanted 1 2024-04-02T04:47:35.553 INFO:teuthology.orchestra.run.smithi016.stdout:qdb_cluster leader: 0 members: 2024-04-02T04:47:35.553 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.zwrfoo{0:24285} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.27:6824/1461455976,v1:172.21.15.27:6825/1461455976] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:47:35.553 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:47:35.553 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:47:35.553 INFO:teuthology.orchestra.run.smithi016.stdout:Standby daemons: 2024-04-02T04:47:35.554 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:47:35.554 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.fyzrnm{-1:14488} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.16:6828/2578719859,v1:172.21.15.16:6829/2578719859] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:47:35.554 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.uhgkgg{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.16:6826/601263579,v1:172.21.15.16:6827/601263579] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:47:35.554 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.sdublg{-1:24295} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.27:6826/4177192740,v1:172.21.15.27:6827/4177192740] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:47:35.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.548+0000 7fa301b99700 1 -- 172.21.15.16:0/3023515903 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fa2e806dd60 msgr2=0x7fa2e8070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:35.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.548+0000 7fa301b99700 1 --2- 172.21.15.16:0/3023515903 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fa2e806dd60 0x7fa2e8070220 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec0060b0 tx=0x7fa2ec007b80 comp rx=0 tx=0).stop 2024-04-02T04:47:35.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.548+0000 7fa301b99700 1 -- 172.21.15.16:0/3023515903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2fc103f70 msgr2=0x7fa2fc196a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:35.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.548+0000 7fa301b99700 1 --2- 172.21.15.16:0/3023515903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2fc103f70 0x7fa2fc196a50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4006010 tx=0x7fa2e400be20 comp rx=0 tx=0).stop 2024-04-02T04:47:35.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.548+0000 7fa301b99700 1 -- 172.21.15.16:0/3023515903 shutdown_connections 2024-04-02T04:47:35.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.548+0000 7fa301b99700 1 --2- 172.21.15.16:0/3023515903 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fa2e806dd60 0x7fa2e8070220 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:35.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.548+0000 7fa301b99700 1 --2- 172.21.15.16:0/3023515903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2fc101640 0x7fa2fc196510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:35.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.548+0000 7fa301b99700 1 --2- 172.21.15.16:0/3023515903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2fc103f70 0x7fa2fc196a50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:35.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.548+0000 7fa301b99700 1 -- 172.21.15.16:0/3023515903 >> 172.21.15.16:0/3023515903 conn(0x7fa2fc0faf70 msgr2=0x7fa2fc0fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:35.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.548+0000 7fa301b99700 1 -- 172.21.15.16:0/3023515903 shutdown_connections 2024-04-02T04:47:35.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.548+0000 7fa301b99700 1 -- 172.21.15.16:0/3023515903 wait complete. 2024-04-02T04:47:35.555 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 15 2024-04-02T04:47:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.655+0000 7f30517fd700 1 -- 172.21.15.16:0/3565256372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f304c102110 msgr2=0x7f304c102530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.655+0000 7f30517fd700 1 --2- 172.21.15.16:0/3565256372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f304c102110 0x7f304c102530 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f303c009910 tx=0x7f303c009c20 comp rx=0 tx=0).stop 2024-04-02T04:47:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.655+0000 7f30517fd700 1 -- 172.21.15.16:0/3565256372 shutdown_connections 2024-04-02T04:47:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.655+0000 7f30517fd700 1 --2- 172.21.15.16:0/3565256372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f304c0feb90 0x7f304c0ff010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.655+0000 7f30517fd700 1 --2- 172.21.15.16:0/3565256372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f304c102110 0x7f304c102530 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.655+0000 7f30517fd700 1 -- 172.21.15.16:0/3565256372 >> 172.21.15.16:0/3565256372 conn(0x7f304c0fa730 msgr2=0x7f304c0fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.655+0000 7f30517fd700 1 -- 172.21.15.16:0/3565256372 shutdown_connections 2024-04-02T04:47:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.655+0000 7f30517fd700 1 -- 172.21.15.16:0/3565256372 wait complete. 2024-04-02T04:47:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.656+0000 7f30517fd700 1 Processor -- start 2024-04-02T04:47:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.656+0000 7f30517fd700 1 -- start start 2024-04-02T04:47:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.656+0000 7f30517fd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f304c0feb90 0x7f304c196590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.656+0000 7f30517fd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f304c102110 0x7f304c196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.656+0000 7f30517fd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c1970f0 con 0x7f304c0feb90 2024-04-02T04:47:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.656+0000 7f30517fd700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c197230 con 0x7f304c102110 2024-04-02T04:47:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.656+0000 7f304affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f304c0feb90 0x7f304c196590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.656+0000 7f304affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f304c0feb90 0x7f304c196590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40668/0 (socket says 172.21.15.16:40668) 2024-04-02T04:47:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.657+0000 7f304affd700 1 -- 172.21.15.16:0/3019661107 learned_addr learned my addr 172.21.15.16:0/3019661107 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:47:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.657+0000 7f304a7fc700 1 --2- 172.21.15.16:0/3019661107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f304c102110 0x7f304c196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.657+0000 7f304affd700 1 -- 172.21.15.16:0/3019661107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f304c102110 msgr2=0x7f304c196ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.657+0000 7f304affd700 1 --2- 172.21.15.16:0/3019661107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f304c102110 0x7f304c196ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.657+0000 7f304affd700 1 -- 172.21.15.16:0/3019661107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f303c0095f0 con 0x7f304c0feb90 2024-04-02T04:47:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.657+0000 7f304affd700 1 --2- 172.21.15.16:0/3019661107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f304c0feb90 0x7f304c196590 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f303c005850 tx=0x7f303c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.657+0000 7f3033fff700 1 -- 172.21.15.16:0/3019661107 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f303c01d070 con 0x7f304c0feb90 2024-04-02T04:47:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.657+0000 7f30517fd700 1 -- 172.21.15.16:0/3019661107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f304c19bc30 con 0x7f304c0feb90 2024-04-02T04:47:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.657+0000 7f3033fff700 1 -- 172.21.15.16:0/3019661107 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f303c00f460 con 0x7f304c0feb90 2024-04-02T04:47:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.657+0000 7f3033fff700 1 -- 172.21.15.16:0/3019661107 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f303c017630 con 0x7f304c0feb90 2024-04-02T04:47:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.657+0000 7f30517fd700 1 -- 172.21.15.16:0/3019661107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f304c19c120 con 0x7f304c0feb90 2024-04-02T04:47:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.658+0000 7f30517fd700 1 -- 172.21.15.16:0/3019661107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f304c1906b0 con 0x7f304c0feb90 2024-04-02T04:47:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.659+0000 7f3033fff700 1 -- 172.21.15.16:0/3019661107 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f303c00fab0 con 0x7f304c0feb90 2024-04-02T04:47:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.659+0000 7f3033fff700 1 --2- 172.21.15.16:0/3019661107 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f303406dd60 0x7f3034070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.659+0000 7f3033fff700 1 -- 172.21.15.16:0/3019661107 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f303c08efa0 con 0x7f304c0feb90 2024-04-02T04:47:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.659+0000 7f304a7fc700 1 --2- 172.21.15.16:0/3019661107 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f303406dd60 0x7f3034070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.659+0000 7f304a7fc700 1 --2- 172.21.15.16:0/3019661107 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f303406dd60 0x7f3034070220 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f304c068f80 tx=0x7f304000a380 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:35.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.664+0000 7f3033fff700 1 -- 172.21.15.16:0/3019661107 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f303c05c060 con 0x7f304c0feb90 2024-04-02T04:47:35.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.888+0000 7f30517fd700 1 -- 172.21.15.16:0/3019661107 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f304c02ce70 con 0x7f303406dd60 2024-04-02T04:47:35.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.889+0000 7f3033fff700 1 -- 172.21.15.16:0/3019661107 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f304c02ce70 con 0x7f303406dd60 2024-04-02T04:47:35.892 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:47:35.892 INFO:teuthology.orchestra.run.smithi016.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:47:35.892 INFO:teuthology.orchestra.run.smithi016.stdout: "in_progress": true, 2024-04-02T04:47:35.892 INFO:teuthology.orchestra.run.smithi016.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-02T04:47:35.892 INFO:teuthology.orchestra.run.smithi016.stdout: "services_complete": [ 2024-04-02T04:47:35.892 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr", 2024-04-02T04:47:35.892 INFO:teuthology.orchestra.run.smithi016.stdout: "mon", 2024-04-02T04:47:35.893 INFO:teuthology.orchestra.run.smithi016.stdout: "crash" 2024-04-02T04:47:35.893 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-04-02T04:47:35.893 INFO:teuthology.orchestra.run.smithi016.stdout: "progress": "9/23 daemons upgraded", 2024-04-02T04:47:35.893 INFO:teuthology.orchestra.run.smithi016.stdout: "message": "Currently upgrading osd daemons", 2024-04-02T04:47:35.893 INFO:teuthology.orchestra.run.smithi016.stdout: "is_paused": false 2024-04-02T04:47:35.893 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:47:35.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.891+0000 7f30517fd700 1 -- 172.21.15.16:0/3019661107 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f303406dd60 msgr2=0x7f3034070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:35.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.891+0000 7f30517fd700 1 --2- 172.21.15.16:0/3019661107 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f303406dd60 0x7f3034070220 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f304c068f80 tx=0x7f304000a380 comp rx=0 tx=0).stop 2024-04-02T04:47:35.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.891+0000 7f30517fd700 1 -- 172.21.15.16:0/3019661107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f304c0feb90 msgr2=0x7f304c196590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:35.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.891+0000 7f30517fd700 1 --2- 172.21.15.16:0/3019661107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f304c0feb90 0x7f304c196590 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f303c005850 tx=0x7f303c00bd70 comp rx=0 tx=0).stop 2024-04-02T04:47:35.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.891+0000 7f30517fd700 1 -- 172.21.15.16:0/3019661107 shutdown_connections 2024-04-02T04:47:35.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.891+0000 7f30517fd700 1 --2- 172.21.15.16:0/3019661107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f304c0feb90 0x7f304c196590 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:35.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.891+0000 7f30517fd700 1 --2- 172.21.15.16:0/3019661107 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f303406dd60 0x7f3034070220 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:35.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.891+0000 7f30517fd700 1 --2- 172.21.15.16:0/3019661107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f304c102110 0x7f304c196ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:35.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.891+0000 7f30517fd700 1 -- 172.21.15.16:0/3019661107 >> 172.21.15.16:0/3019661107 conn(0x7f304c0fa730 msgr2=0x7f304c105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:35.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.891+0000 7f30517fd700 1 -- 172.21.15.16:0/3019661107 shutdown_connections 2024-04-02T04:47:35.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:35.891+0000 7f30517fd700 1 -- 172.21.15.16:0/3019661107 wait complete. 2024-04-02T04:47:36.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.003+0000 7f5b04b91700 1 -- 172.21.15.16:0/2296637175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b00100be0 msgr2=0x7f5b00101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:36.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.003+0000 7f5b04b91700 1 --2- 172.21.15.16:0/2296637175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b00100be0 0x7f5b00101040 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5af4009910 tx=0x7f5af4009c20 comp rx=0 tx=0).stop 2024-04-02T04:47:36.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.003+0000 7f5b04b91700 1 -- 172.21.15.16:0/2296637175 shutdown_connections 2024-04-02T04:47:36.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.004+0000 7f5b04b91700 1 --2- 172.21.15.16:0/2296637175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b00100be0 0x7f5b00101040 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:36.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.004+0000 7f5b04b91700 1 --2- 172.21.15.16:0/2296637175 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b000ff9e0 0x7f5b000ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:36.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.004+0000 7f5b04b91700 1 -- 172.21.15.16:0/2296637175 >> 172.21.15.16:0/2296637175 conn(0x7f5b000faf80 msgr2=0x7f5b000fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:36.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.004+0000 7f5b04b91700 1 -- 172.21.15.16:0/2296637175 shutdown_connections 2024-04-02T04:47:36.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.004+0000 7f5b04b91700 1 -- 172.21.15.16:0/2296637175 wait complete. 2024-04-02T04:47:36.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.004+0000 7f5b04b91700 1 Processor -- start 2024-04-02T04:47:36.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.004+0000 7f5b04b91700 1 -- start start 2024-04-02T04:47:36.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.004+0000 7f5b04b91700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b000ff9e0 0x7f5b001964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:36.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.004+0000 7f5b04b91700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b00100be0 0x7f5b00196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:36.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.004+0000 7f5b04b91700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b00197020 con 0x7f5b00100be0 2024-04-02T04:47:36.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.004+0000 7f5b04b91700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b00197160 con 0x7f5b000ff9e0 2024-04-02T04:47:36.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.005+0000 7f5afdd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b00100be0 0x7f5b00196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:36.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.005+0000 7f5afdd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b00100be0 0x7f5b00196a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40686/0 (socket says 172.21.15.16:40686) 2024-04-02T04:47:36.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.005+0000 7f5afdd9b700 1 -- 172.21.15.16:0/674631879 learned_addr learned my addr 172.21.15.16:0/674631879 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:47:36.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.005+0000 7f5afe59c700 1 --2- 172.21.15.16:0/674631879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b000ff9e0 0x7f5b001964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:36.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.005+0000 7f5afdd9b700 1 -- 172.21.15.16:0/674631879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b000ff9e0 msgr2=0x7f5b001964c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:36.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.005+0000 7f5afdd9b700 1 --2- 172.21.15.16:0/674631879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b000ff9e0 0x7f5b001964c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:36.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.005+0000 7f5afdd9b700 1 -- 172.21.15.16:0/674631879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af40095f0 con 0x7f5b00100be0 2024-04-02T04:47:36.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.005+0000 7f5afdd9b700 1 --2- 172.21.15.16:0/674631879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b00100be0 0x7f5b00196a00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5af4006010 tx=0x7f5af400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:36.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.005+0000 7f5aef7fe700 1 -- 172.21.15.16:0/674631879 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5af401d070 con 0x7f5b00100be0 2024-04-02T04:47:36.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.005+0000 7f5b04b91700 1 -- 172.21.15.16:0/674631879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b0019bb60 con 0x7f5b00100be0 2024-04-02T04:47:36.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.005+0000 7f5aef7fe700 1 -- 172.21.15.16:0/674631879 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5af400f460 con 0x7f5b00100be0 2024-04-02T04:47:36.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.005+0000 7f5aef7fe700 1 -- 172.21.15.16:0/674631879 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5af4021640 con 0x7f5b00100be0 2024-04-02T04:47:36.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.005+0000 7f5b04b91700 1 -- 172.21.15.16:0/674631879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b0019c050 con 0x7f5b00100be0 2024-04-02T04:47:36.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.006+0000 7f5b04b91700 1 -- 172.21.15.16:0/674631879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b0004ea80 con 0x7f5b00100be0 2024-04-02T04:47:36.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.007+0000 7f5aef7fe700 1 -- 172.21.15.16:0/674631879 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f5af400fab0 con 0x7f5b00100be0 2024-04-02T04:47:36.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.007+0000 7f5aef7fe700 1 --2- 172.21.15.16:0/674631879 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5ae806dd60 0x7f5ae8070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:47:36.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.007+0000 7f5aef7fe700 1 -- 172.21.15.16:0/674631879 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f5af408ed90 con 0x7f5b00100be0 2024-04-02T04:47:36.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.007+0000 7f5afe59c700 1 --2- 172.21.15.16:0/674631879 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5ae806dd60 0x7f5ae8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:47:36.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.008+0000 7f5afe59c700 1 --2- 172.21.15.16:0/674631879 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5ae806dd60 0x7f5ae8070220 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5af0005fd0 tx=0x7f5af0005f60 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:47:36.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.012+0000 7f5aef7fe700 1 -- 172.21.15.16:0/674631879 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5af405bed0 con 0x7f5b00100be0 2024-04-02T04:47:36.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.302+0000 7f5b04b91700 1 -- 172.21.15.16:0/674631879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5b0019c450 con 0x7f5b00100be0 2024-04-02T04:47:36.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.303+0000 7f5aef7fe700 1 -- 172.21.15.16:0/674631879 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f5af40260e0 con 0x7f5b00100be0 2024-04-02T04:47:36.306 INFO:teuthology.orchestra.run.smithi016.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-02T04:47:36.306 INFO:teuthology.orchestra.run.smithi016.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-02T04:47:36.306 INFO:teuthology.orchestra.run.smithi016.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-02T04:47:36.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.305+0000 7f5b04b91700 1 -- 172.21.15.16:0/674631879 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5ae806dd60 msgr2=0x7f5ae8070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:36.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.305+0000 7f5b04b91700 1 --2- 172.21.15.16:0/674631879 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5ae806dd60 0x7f5ae8070220 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5af0005fd0 tx=0x7f5af0005f60 comp rx=0 tx=0).stop 2024-04-02T04:47:36.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.305+0000 7f5b04b91700 1 -- 172.21.15.16:0/674631879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b00100be0 msgr2=0x7f5b00196a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:47:36.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.305+0000 7f5b04b91700 1 --2- 172.21.15.16:0/674631879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b00100be0 0x7f5b00196a00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5af4006010 tx=0x7f5af400be20 comp rx=0 tx=0).stop 2024-04-02T04:47:36.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.305+0000 7f5b04b91700 1 -- 172.21.15.16:0/674631879 shutdown_connections 2024-04-02T04:47:36.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.305+0000 7f5b04b91700 1 --2- 172.21.15.16:0/674631879 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5ae806dd60 0x7f5ae8070220 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:36.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.305+0000 7f5b04b91700 1 --2- 172.21.15.16:0/674631879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b000ff9e0 0x7f5b001964c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:36.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.305+0000 7f5b04b91700 1 --2- 172.21.15.16:0/674631879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b00100be0 0x7f5b00196a00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:47:36.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.305+0000 7f5b04b91700 1 -- 172.21.15.16:0/674631879 >> 172.21.15.16:0/674631879 conn(0x7f5b000faf80 msgr2=0x7f5b000690c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:47:36.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.305+0000 7f5b04b91700 1 -- 172.21.15.16:0/674631879 shutdown_connections 2024-04-02T04:47:36.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:47:36.305+0000 7f5b04b91700 1 -- 172.21.15.16:0/674631879 wait complete. 2024-04-02T04:47:36.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:36 smithi027 ceph-mon[83036]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:47:36.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:36 smithi027 ceph-mon[83036]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:47:36.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:36 smithi027 ceph-mon[83036]: pgmap v117: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:47:36.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:36 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/1592374488' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:47:36.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:36 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/3023515903' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:47:36.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:36 smithi016 ceph-mon[93230]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:47:36.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:36 smithi016 ceph-mon[93230]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:47:36.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:36 smithi016 ceph-mon[93230]: pgmap v117: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:47:36.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:36 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/1592374488' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:47:36.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:36 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/3023515903' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:47:37.279 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:37 smithi027 ceph-mon[83036]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:47:37.279 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:37 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/674631879' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:47:37.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:37 smithi016 ceph-mon[93230]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:47:37.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:37 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/674631879' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:47:37.896 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:37 smithi027 systemd[1]: Stopping Ceph osd.3 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4... 2024-04-02T04:47:38.163 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:37 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3[43496]: 2024-04-02T04:47:37.922+0000 7f717311b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T04:47:38.164 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:37 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3[43496]: 2024-04-02T04:47:37.922+0000 7f717311b700 -1 osd.3 64 *** Got signal Terminated *** 2024-04-02T04:47:38.164 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:37 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3[43496]: 2024-04-02T04:47:37.922+0000 7f717311b700 -1 osd.3 64 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T04:47:38.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:38 smithi016 ceph-mon[93230]: pgmap v118: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:47:38.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:38 smithi016 ceph-mon[93230]: osd.3 marked itself down and dead 2024-04-02T04:47:38.645 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:38 smithi027 ceph-mon[83036]: pgmap v118: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:47:38.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:38 smithi027 ceph-mon[83036]: osd.3 marked itself down and dead 2024-04-02T04:47:38.984 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:38 smithi027 podman[92736]: 2024-04-02 04:47:38.705478134 +0000 UTC m=+0.905632946 container died cf7c7fe022f06dfc0e3032a19e133ed4596942e4fe9944ec056d1c94f7f5e457 (image=quay.ceph.io/ceph-ci/ceph@sha256:397d0c1d908c002c558906e3ea05982bc336d403ee4d900400cc3bdd9243aa2e, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3, RELEASE=reef-4d05f4c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD) 2024-04-02T04:47:39.392 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:39 smithi027 ceph-mon[83036]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-02T04:47:39.392 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:39 smithi027 ceph-mon[83036]: failed to encode map e65 with expected crc 2024-04-02T04:47:39.392 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:39 smithi027 ceph-mon[83036]: osdmap e65: 6 total, 5 up, 6 in 2024-04-02T04:47:39.393 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:39 smithi027 podman[92736]: 2024-04-02 04:47:39.088796804 +0000 UTC m=+1.288951614 container cleanup cf7c7fe022f06dfc0e3032a19e133ed4596942e4fe9944ec056d1c94f7f5e457 (image=quay.ceph.io/ceph-ci/ceph@sha256:397d0c1d908c002c558906e3ea05982bc336d403ee4d900400cc3bdd9243aa2e, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, RELEASE=reef-4d05f4c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-02T04:47:39.393 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:39 smithi027 bash[92736]: ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3 2024-04-02T04:47:39.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:39 smithi016 ceph-mon[93230]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-02T04:47:39.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:39 smithi016 ceph-mon[93230]: failed to encode map e65 with expected crc 2024-04-02T04:47:39.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:39 smithi016 ceph-mon[93230]: osdmap e65: 6 total, 5 up, 6 in 2024-04-02T04:47:39.646 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:39 smithi027 podman[92765]: 2024-04-02 04:47:39.390256507 +0000 UTC m=+0.680677459 container remove cf7c7fe022f06dfc0e3032a19e133ed4596942e4fe9944ec056d1c94f7f5e457 (image=quay.ceph.io/ceph-ci/ceph@sha256:397d0c1d908c002c558906e3ea05982bc336d403ee4d900400cc3bdd9243aa2e, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=reef-4d05f4c, GIT_BRANCH=HEAD) 2024-04-02T04:47:40.104 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:39 smithi027 podman[92856]: 2024-04-02 04:47:39.618921804 +0000 UTC m=+0.018358433 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:47:40.105 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:39 smithi027 podman[92856]: 2024-04-02 04:47:39.741511305 +0000 UTC m=+0.140947970 container create 1de8791bfa911bfc3f371f59195b5af36552243a6b31b5ad2e4b0547bf0f685b (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-6cec793, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-02T04:47:40.105 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:40 smithi027 podman[92856]: 2024-04-02 04:47:40.102212712 +0000 UTC m=+0.501649346 container init 1de8791bfa911bfc3f371f59195b5af36552243a6b31b5ad2e4b0547bf0f685b (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3-deactivate, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=squid-6cec793, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240326) 2024-04-02T04:47:40.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:40 smithi027 ceph-mon[83036]: pgmap v120: 65 pgs: 9 peering, 14 stale+active+clean, 42 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:47:40.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:40 smithi027 ceph-mon[83036]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-02T04:47:40.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:40 smithi027 ceph-mon[83036]: failed to encode map e66 with expected crc 2024-04-02T04:47:40.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:40 smithi027 ceph-mon[83036]: failed to encode map e66 with expected crc 2024-04-02T04:47:40.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:40 smithi027 ceph-mon[83036]: osdmap e66: 6 total, 5 up, 6 in 2024-04-02T04:47:40.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:40 smithi027 ceph-mon[83036]: failed to encode map e66 with expected crc 2024-04-02T04:47:40.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:40 smithi027 ceph-mon[83036]: failed to encode map e66 with expected crc 2024-04-02T04:47:40.397 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:40 smithi027 podman[92856]: 2024-04-02 04:47:40.10566752 +0000 UTC m=+0.505104147 container start 1de8791bfa911bfc3f371f59195b5af36552243a6b31b5ad2e4b0547bf0f685b (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3-deactivate, RELEASE=squid-6cec793, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0) 2024-04-02T04:47:40.397 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:40 smithi027 podman[92856]: 2024-04-02 04:47:40.194083957 +0000 UTC m=+0.593520590 container attach 1de8791bfa911bfc3f371f59195b5af36552243a6b31b5ad2e4b0547bf0f685b (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3-deactivate, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-6cec793, io.buildah.version=1.33.5) 2024-04-02T04:47:40.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:40 smithi016 ceph-mon[93230]: pgmap v120: 65 pgs: 9 peering, 14 stale+active+clean, 42 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:47:40.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:40 smithi016 ceph-mon[93230]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-02T04:47:40.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:40 smithi016 ceph-mon[93230]: failed to encode map e66 with expected crc 2024-04-02T04:47:40.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:40 smithi016 ceph-mon[93230]: failed to encode map e66 with expected crc 2024-04-02T04:47:40.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:40 smithi016 ceph-mon[93230]: osdmap e66: 6 total, 5 up, 6 in 2024-04-02T04:47:40.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:40 smithi016 ceph-mon[93230]: failed to encode map e66 with expected crc 2024-04-02T04:47:40.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:40 smithi016 ceph-mon[93230]: failed to encode map e66 with expected crc 2024-04-02T04:47:40.699 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:40 smithi027 podman[92856]: 2024-04-02 04:47:40.416473643 +0000 UTC m=+0.815910268 container died 1de8791bfa911bfc3f371f59195b5af36552243a6b31b5ad2e4b0547bf0f685b (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-6cec793, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-02T04:47:41.396 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:40 smithi027 podman[92882]: 2024-04-02 04:47:40.955907087 +0000 UTC m=+0.532455613 container remove 1de8791bfa911bfc3f371f59195b5af36552243a6b31b5ad2e4b0547bf0f685b (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3-deactivate, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-6cec793, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-02T04:47:41.396 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:40 smithi027 systemd[1]: ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@osd.3.service: Deactivated successfully. 2024-04-02T04:47:41.396 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:40 smithi027 systemd[1]: Stopped Ceph osd.3 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4. 2024-04-02T04:47:41.396 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:40 smithi027 systemd[1]: ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@osd.3.service: Consumed 1min 4.845s CPU time. 2024-04-02T04:47:41.396 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:41 smithi027 systemd[1]: Starting Ceph osd.3 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4... 2024-04-02T04:47:41.695 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:41 smithi027 podman[92981]: 2024-04-02 04:47:41.302775744 +0000 UTC m=+0.018273782 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:47:41.695 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:41 smithi027 podman[92981]: 2024-04-02 04:47:41.423642771 +0000 UTC m=+0.139140811 container create 2b18507f40506a0ff38948fb1a0532c45e4e8c5004d3c8ed421610f4ea6a56d1 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3-activate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-6cec793, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T04:47:41.696 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:41 smithi027 podman[92981]: 2024-04-02 04:47:41.693268569 +0000 UTC m=+0.408766621 container init 2b18507f40506a0ff38948fb1a0532c45e4e8c5004d3c8ed421610f4ea6a56d1 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3-activate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-6cec793, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-02T04:47:42.146 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:41 smithi027 podman[92981]: 2024-04-02 04:47:41.696498311 +0000 UTC m=+0.411996355 container start 2b18507f40506a0ff38948fb1a0532c45e4e8c5004d3c8ed421610f4ea6a56d1 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , RELEASE=squid-6cec793, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-02T04:47:42.146 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:41 smithi027 podman[92981]: 2024-04-02 04:47:41.75974798 +0000 UTC m=+0.475246026 container attach 2b18507f40506a0ff38948fb1a0532c45e4e8c5004d3c8ed421610f4ea6a56d1 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3-activate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, RELEASE=squid-6cec793, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-02T04:47:42.432 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:42 smithi027 ceph-mon[83036]: pgmap v122: 65 pgs: 9 peering, 14 stale+active+clean, 42 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:47:42.433 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:42 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3-activate[92996]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-02T04:47:42.433 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:42 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3-activate[92996]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-02T04:47:42.433 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:42 smithi027 bash[92981]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-02T04:47:42.433 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:42 smithi027 bash[92981]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-02T04:47:42.433 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:42 smithi027 bash[92981]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-02T04:47:42.433 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:42 smithi027 bash[92981]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-02T04:47:42.433 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:42 smithi027 bash[92981]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-04-02T04:47:42.433 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:42 smithi027 bash[92981]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-02T04:47:42.433 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:42 smithi027 bash[92981]: --> ceph-volume raw activate successful for osd ID: 3 2024-04-02T04:47:42.433 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:42 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3-activate[92996]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-02T04:47:42.433 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:42 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3-activate[92996]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-02T04:47:42.434 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:42 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3-activate[92996]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-04-02T04:47:42.434 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:42 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3-activate[92996]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-02T04:47:42.434 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:42 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3-activate[92996]: --> ceph-volume raw activate successful for osd ID: 3 2024-04-02T04:47:42.434 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:42 smithi027 podman[92981]: 2024-04-02 04:47:42.185020683 +0000 UTC m=+0.900518721 container died 2b18507f40506a0ff38948fb1a0532c45e4e8c5004d3c8ed421610f4ea6a56d1 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-6cec793, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-02T04:47:42.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:42 smithi016 ceph-mon[93230]: pgmap v122: 65 pgs: 9 peering, 14 stale+active+clean, 42 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:47:42.778 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:42 smithi027 podman[93127]: 2024-04-02 04:47:42.671676109 +0000 UTC m=+0.476840616 container remove 2b18507f40506a0ff38948fb1a0532c45e4e8c5004d3c8ed421610f4ea6a56d1 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3-activate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-6cec793, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T04:47:43.050 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:42 smithi027 podman[93189]: 2024-04-02 04:47:42.794566685 +0000 UTC m=+0.021420985 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:47:43.050 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:42 smithi027 podman[93189]: 2024-04-02 04:47:42.93458958 +0000 UTC m=+0.161443899 container create 0dd13b5be8c048af76377f18225b037644f2f03f37e11ee29d6ceef0a43a66d3 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3, maintainer=Guillaume Abrioux , RELEASE=squid-6cec793, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-02T04:47:43.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:43 smithi016 ceph-mon[93230]: Health check failed: Degraded data redundancy: 44/267 objects degraded (16.479%), 13 pgs degraded (PG_DEGRADED) 2024-04-02T04:47:43.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:43 smithi027 ceph-mon[83036]: Health check failed: Degraded data redundancy: 44/267 objects degraded (16.479%), 13 pgs degraded (PG_DEGRADED) 2024-04-02T04:47:43.646 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:43 smithi027 podman[93189]: 2024-04-02 04:47:43.150449565 +0000 UTC m=+0.377303847 container init 0dd13b5be8c048af76377f18225b037644f2f03f37e11ee29d6ceef0a43a66d3 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-6cec793) 2024-04-02T04:47:43.646 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:43 smithi027 podman[93189]: 2024-04-02 04:47:43.153931238 +0000 UTC m=+0.380785512 container start 0dd13b5be8c048af76377f18225b037644f2f03f37e11ee29d6ceef0a43a66d3 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-6cec793, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-02T04:47:43.646 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:43 smithi027 bash[93189]: 0dd13b5be8c048af76377f18225b037644f2f03f37e11ee29d6ceef0a43a66d3 2024-04-02T04:47:43.647 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:43 smithi027 systemd[1]: Started Ceph osd.3 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4. 2024-04-02T04:47:44.116 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:43 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3[93204]: 2024-04-02T04:47:43.694+0000 7f8d5e5657c0 -1 Falling back to public interface 2024-04-02T04:47:44.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:44 smithi027 ceph-mon[83036]: pgmap v123: 65 pgs: 13 active+undersized+degraded, 9 peering, 16 active+undersized, 27 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 44/267 objects degraded (16.479%) 2024-04-02T04:47:44.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:44 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:44.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:44 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:44.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:44 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:47:44.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:44 smithi016 ceph-mon[93230]: pgmap v123: 65 pgs: 13 active+undersized+degraded, 9 peering, 16 active+undersized, 27 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 44/267 objects degraded (16.479%) 2024-04-02T04:47:44.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:44 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:44.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:44 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:44.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:44 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:47:45.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:45 smithi027 ceph-mon[83036]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-02T04:47:45.809 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:45 smithi016 ceph-mon[93230]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-02T04:47:46.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:46 smithi027 ceph-mon[83036]: pgmap v124: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 59/267 objects degraded (22.097%) 2024-04-02T04:47:46.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:46 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:46.710 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:46 smithi016 ceph-mon[93230]: pgmap v124: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 59/267 objects degraded (22.097%) 2024-04-02T04:47:46.710 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:46 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:47.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:47 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:47.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:47 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:47.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:47 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:47:47.809 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:47 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:47.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:47 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:47.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:47 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:47:48.475 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:48 smithi027 ceph-mon[83036]: pgmap v125: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 59/267 objects degraded (22.097%) 2024-04-02T04:47:48.475 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:48 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:48.475 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:48 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:48.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:48 smithi016 ceph-mon[93230]: pgmap v125: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 59/267 objects degraded (22.097%) 2024-04-02T04:47:48.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:48 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:48.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:48 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:48.896 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:48 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3[93204]: 2024-04-02T04:47:48.682+0000 7f8d5e5657c0 -1 osd.3 64 log_to_monitors true 2024-04-02T04:47:49.474 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:49 smithi027 ceph-mon[83036]: from='osd.3 [v2:172.21.15.27:6800/4092279016,v1:172.21.15.27:6801/4092279016]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-02T04:47:49.474 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:49 smithi027 ceph-mon[83036]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-02T04:47:49.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:49 smithi016 ceph-mon[93230]: from='osd.3 [v2:172.21.15.27:6800/4092279016,v1:172.21.15.27:6801/4092279016]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-02T04:47:49.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:49 smithi016 ceph-mon[93230]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-02T04:47:50.525 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:47:50 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3[93204]: 2024-04-02T04:47:50.270+0000 7f8d4b9b5700 -1 osd.3 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T04:47:50.526 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:50 smithi027 ceph-mon[83036]: pgmap v126: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 59/267 objects degraded (22.097%) 2024-04-02T04:47:50.526 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:50 smithi027 ceph-mon[83036]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-02T04:47:50.526 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:50 smithi027 ceph-mon[83036]: osdmap e67: 6 total, 5 up, 6 in 2024-04-02T04:47:50.526 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:50 smithi027 ceph-mon[83036]: from='osd.3 [v2:172.21.15.27:6800/4092279016,v1:172.21.15.27:6801/4092279016]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-02T04:47:50.526 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:50 smithi027 ceph-mon[83036]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-02T04:47:50.526 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:50 smithi027 ceph-mon[83036]: failed to encode map e67 with expected crc 2024-04-02T04:47:50.526 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:50 smithi027 ceph-mon[83036]: from='osd.3 ' entity='osd.3' 2024-04-02T04:47:50.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:50 smithi016 ceph-mon[93230]: pgmap v126: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 59/267 objects degraded (22.097%) 2024-04-02T04:47:50.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:50 smithi016 ceph-mon[93230]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-02T04:47:50.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:50 smithi016 ceph-mon[93230]: osdmap e67: 6 total, 5 up, 6 in 2024-04-02T04:47:50.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:50 smithi016 ceph-mon[93230]: from='osd.3 [v2:172.21.15.27:6800/4092279016,v1:172.21.15.27:6801/4092279016]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-02T04:47:50.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:50 smithi016 ceph-mon[93230]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-02T04:47:50.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:50 smithi016 ceph-mon[93230]: failed to encode map e67 with expected crc 2024-04-02T04:47:50.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:50 smithi016 ceph-mon[93230]: from='osd.3 ' entity='osd.3' 2024-04-02T04:47:52.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:52 smithi016 ceph-mon[93230]: pgmap v128: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 59/267 objects degraded (22.097%) 2024-04-02T04:47:52.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:52 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:52.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:52 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:52.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:52 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:47:52.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:52 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:47:52.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:52 smithi016 ceph-mon[93230]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-02T04:47:52.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:52 smithi016 ceph-mon[93230]: failed to encode map e68 with expected crc 2024-04-02T04:47:52.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:52 smithi016 ceph-mon[93230]: osd.3 [v2:172.21.15.27:6800/4092279016,v1:172.21.15.27:6801/4092279016] boot 2024-04-02T04:47:52.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:52 smithi016 ceph-mon[93230]: osdmap e68: 6 total, 6 up, 6 in 2024-04-02T04:47:52.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:52 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T04:47:52.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:52 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:52.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:52 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:47:52.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:52 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:47:52.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:52 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:47:52.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:52 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:47:52.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:52 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-02T04:47:52.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:52 smithi027 ceph-mon[83036]: pgmap v128: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 59/267 objects degraded (22.097%) 2024-04-02T04:47:52.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:52 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:52.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:52 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:52.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:52 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:47:52.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:52 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:47:52.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:52 smithi027 ceph-mon[83036]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-02T04:47:52.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:52 smithi027 ceph-mon[83036]: failed to encode map e68 with expected crc 2024-04-02T04:47:52.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:52 smithi027 ceph-mon[83036]: osd.3 [v2:172.21.15.27:6800/4092279016,v1:172.21.15.27:6801/4092279016] boot 2024-04-02T04:47:52.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:52 smithi027 ceph-mon[83036]: osdmap e68: 6 total, 6 up, 6 in 2024-04-02T04:47:52.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:52 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T04:47:52.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:52 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:47:52.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:52 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:47:52.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:52 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:47:52.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:52 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:47:52.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:52 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:47:52.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:52 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-02T04:47:53.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:53 smithi016 ceph-mon[93230]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-02T04:47:53.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:53 smithi016 ceph-mon[93230]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-04-02T04:47:53.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:53 smithi016 ceph-mon[93230]: failed to encode map e69 with expected crc 2024-04-02T04:47:53.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:53 smithi016 ceph-mon[93230]: osdmap e69: 6 total, 6 up, 6 in 2024-04-02T04:47:53.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:53 smithi016 ceph-mon[93230]: failed to encode map e69 with expected crc 2024-04-02T04:47:53.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:53 smithi016 ceph-mon[93230]: failed to encode map e69 with expected crc 2024-04-02T04:47:53.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:53 smithi016 ceph-mon[93230]: failed to encode map e69 with expected crc 2024-04-02T04:47:53.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:53 smithi016 ceph-mon[93230]: Health check update: Degraded data redundancy: 59/267 objects degraded (22.097%), 18 pgs degraded (PG_DEGRADED) 2024-04-02T04:47:53.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:53 smithi027 ceph-mon[83036]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-02T04:47:53.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:53 smithi027 ceph-mon[83036]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-04-02T04:47:53.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:53 smithi027 ceph-mon[83036]: failed to encode map e69 with expected crc 2024-04-02T04:47:53.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:53 smithi027 ceph-mon[83036]: osdmap e69: 6 total, 6 up, 6 in 2024-04-02T04:47:53.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:53 smithi027 ceph-mon[83036]: failed to encode map e69 with expected crc 2024-04-02T04:47:53.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:53 smithi027 ceph-mon[83036]: failed to encode map e69 with expected crc 2024-04-02T04:47:53.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:53 smithi027 ceph-mon[83036]: failed to encode map e69 with expected crc 2024-04-02T04:47:53.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:53 smithi027 ceph-mon[83036]: Health check update: Degraded data redundancy: 59/267 objects degraded (22.097%), 18 pgs degraded (PG_DEGRADED) 2024-04-02T04:47:54.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:54 smithi016 ceph-mon[93230]: pgmap v131: 65 pgs: 7 peering, 13 active+undersized+degraded, 10 active+undersized, 35 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 43/267 objects degraded (16.105%) 2024-04-02T04:47:54.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:54 smithi027 ceph-mon[83036]: pgmap v131: 65 pgs: 7 peering, 13 active+undersized+degraded, 10 active+undersized, 35 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 43/267 objects degraded (16.105%) 2024-04-02T04:47:55.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:55 smithi027 ceph-mon[83036]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 43/267 objects degraded (16.105%), 13 pgs degraded) 2024-04-02T04:47:55.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:55 smithi016 ceph-mon[93230]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 43/267 objects degraded (16.105%), 13 pgs degraded) 2024-04-02T04:47:56.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:56 smithi027 ceph-mon[83036]: pgmap v132: 65 pgs: 7 peering, 58 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:47:56.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:56 smithi016 ceph-mon[93230]: pgmap v132: 65 pgs: 7 peering, 58 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:47:58.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:47:58 smithi027 ceph-mon[83036]: pgmap v133: 65 pgs: 3 peering, 62 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:47:58.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:47:58 smithi016 ceph-mon[93230]: pgmap v133: 65 pgs: 3 peering, 62 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:00.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:00 smithi027 ceph-mon[83036]: pgmap v134: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:00.809 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:00 smithi016 ceph-mon[93230]: pgmap v134: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:02 smithi027 ceph-mon[83036]: pgmap v135: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:02 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:02 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:48:02.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:02 smithi016 ceph-mon[93230]: pgmap v135: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:02.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:02 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:02.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:02 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:48:04.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:04 smithi027 ceph-mon[83036]: pgmap v136: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:04.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:04 smithi016 ceph-mon[93230]: pgmap v136: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:06.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.425+0000 7fdb3e2dc700 1 -- 172.21.15.16:0/3361566324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb38101570 msgr2=0x7fdb38103960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:06.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.425+0000 7fdb3e2dc700 1 --2- 172.21.15.16:0/3361566324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb38101570 0x7fdb38103960 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdb28009910 tx=0x7fdb28009c20 comp rx=0 tx=0).stop 2024-04-02T04:48:06.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.425+0000 7fdb3e2dc700 1 -- 172.21.15.16:0/3361566324 shutdown_connections 2024-04-02T04:48:06.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.425+0000 7fdb3e2dc700 1 --2- 172.21.15.16:0/3361566324 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb38103ea0 0x7fdb38106290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:06.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.425+0000 7fdb3e2dc700 1 --2- 172.21.15.16:0/3361566324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb38101570 0x7fdb38103960 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:06.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.425+0000 7fdb3e2dc700 1 -- 172.21.15.16:0/3361566324 >> 172.21.15.16:0/3361566324 conn(0x7fdb380faf80 msgr2=0x7fdb380fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:06.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.425+0000 7fdb3e2dc700 1 -- 172.21.15.16:0/3361566324 shutdown_connections 2024-04-02T04:48:06.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.425+0000 7fdb3e2dc700 1 -- 172.21.15.16:0/3361566324 wait complete. 2024-04-02T04:48:06.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.426+0000 7fdb3e2dc700 1 Processor -- start 2024-04-02T04:48:06.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.426+0000 7fdb3e2dc700 1 -- start start 2024-04-02T04:48:06.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.426+0000 7fdb3e2dc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb38101570 0x7fdb381965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:06.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.426+0000 7fdb3e2dc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb38103ea0 0x7fdb38196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.426+0000 7fdb3e2dc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb38197140 con 0x7fdb38101570 2024-04-02T04:48:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.426+0000 7fdb3e2dc700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb38197280 con 0x7fdb38103ea0 2024-04-02T04:48:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.427+0000 7fdb37fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb38101570 0x7fdb381965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.427+0000 7fdb37fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb38101570 0x7fdb381965e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58866/0 (socket says 172.21.15.16:58866) 2024-04-02T04:48:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.427+0000 7fdb37fff700 1 -- 172.21.15.16:0/78933355 learned_addr learned my addr 172.21.15.16:0/78933355 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:48:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.427+0000 7fdb377fe700 1 --2- 172.21.15.16:0/78933355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb38103ea0 0x7fdb38196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.427+0000 7fdb37fff700 1 -- 172.21.15.16:0/78933355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb38103ea0 msgr2=0x7fdb38196b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.427+0000 7fdb37fff700 1 --2- 172.21.15.16:0/78933355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb38103ea0 0x7fdb38196b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.427+0000 7fdb37fff700 1 -- 172.21.15.16:0/78933355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb280095f0 con 0x7fdb38101570 2024-04-02T04:48:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.427+0000 7fdb37fff700 1 --2- 172.21.15.16:0/78933355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb38101570 0x7fdb381965e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdb28006010 tx=0x7fdb2800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.427+0000 7fdb357fa700 1 -- 172.21.15.16:0/78933355 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb2801d070 con 0x7fdb38101570 2024-04-02T04:48:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.427+0000 7fdb357fa700 1 -- 172.21.15.16:0/78933355 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb2800f460 con 0x7fdb38101570 2024-04-02T04:48:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.427+0000 7fdb3e2dc700 1 -- 172.21.15.16:0/78933355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb3819bc30 con 0x7fdb38101570 2024-04-02T04:48:06.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.427+0000 7fdb357fa700 1 -- 172.21.15.16:0/78933355 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb28021640 con 0x7fdb38101570 2024-04-02T04:48:06.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.427+0000 7fdb3e2dc700 1 -- 172.21.15.16:0/78933355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb3819c120 con 0x7fdb38101570 2024-04-02T04:48:06.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.429+0000 7fdb3e2dc700 1 -- 172.21.15.16:0/78933355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb3804ea80 con 0x7fdb38101570 2024-04-02T04:48:06.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.429+0000 7fdb357fa700 1 -- 172.21.15.16:0/78933355 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fdb2800fab0 con 0x7fdb38101570 2024-04-02T04:48:06.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.429+0000 7fdb357fa700 1 --2- 172.21.15.16:0/78933355 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fdb2406dd60 0x7fdb24070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:06.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.429+0000 7fdb357fa700 1 -- 172.21.15.16:0/78933355 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fdb2808ef30 con 0x7fdb38101570 2024-04-02T04:48:06.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.429+0000 7fdb377fe700 1 --2- 172.21.15.16:0/78933355 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fdb2406dd60 0x7fdb24070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:06.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.430+0000 7fdb377fe700 1 --2- 172.21.15.16:0/78933355 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fdb2406dd60 0x7fdb24070220 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdb20006fd0 tx=0x7fdb20008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:06.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.434+0000 7fdb357fa700 1 -- 172.21.15.16:0/78933355 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fdb280c3280 con 0x7fdb38101570 2024-04-02T04:48:06.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:06 smithi027 ceph-mon[83036]: pgmap v137: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:06.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.671+0000 7fdb3e2dc700 1 -- 172.21.15.16:0/78933355 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb381048c0 con 0x7fdb2406dd60 2024-04-02T04:48:06.674 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:06 smithi016 ceph-mon[93230]: pgmap v137: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:06.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.672+0000 7fdb357fa700 1 -- 172.21.15.16:0/78933355 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fdb381048c0 con 0x7fdb2406dd60 2024-04-02T04:48:06.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.674+0000 7fdb3e2dc700 1 -- 172.21.15.16:0/78933355 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fdb2406dd60 msgr2=0x7fdb24070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:06.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.674+0000 7fdb3e2dc700 1 --2- 172.21.15.16:0/78933355 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fdb2406dd60 0x7fdb24070220 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdb20006fd0 tx=0x7fdb20008040 comp rx=0 tx=0).stop 2024-04-02T04:48:06.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.674+0000 7fdb3e2dc700 1 -- 172.21.15.16:0/78933355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb38101570 msgr2=0x7fdb381965e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:06.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.674+0000 7fdb3e2dc700 1 --2- 172.21.15.16:0/78933355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb38101570 0x7fdb381965e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdb28006010 tx=0x7fdb2800be20 comp rx=0 tx=0).stop 2024-04-02T04:48:06.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.675+0000 7fdb3e2dc700 1 -- 172.21.15.16:0/78933355 shutdown_connections 2024-04-02T04:48:06.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.675+0000 7fdb3e2dc700 1 --2- 172.21.15.16:0/78933355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb38101570 0x7fdb381965e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:06.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.675+0000 7fdb3e2dc700 1 --2- 172.21.15.16:0/78933355 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fdb2406dd60 0x7fdb24070220 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:06.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.675+0000 7fdb3e2dc700 1 --2- 172.21.15.16:0/78933355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb38103ea0 0x7fdb38196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:06.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.675+0000 7fdb3e2dc700 1 -- 172.21.15.16:0/78933355 >> 172.21.15.16:0/78933355 conn(0x7fdb380faf80 msgr2=0x7fdb380fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:06.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.675+0000 7fdb3e2dc700 1 -- 172.21.15.16:0/78933355 shutdown_connections 2024-04-02T04:48:06.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.675+0000 7fdb3e2dc700 1 -- 172.21.15.16:0/78933355 wait complete. 2024-04-02T04:48:06.688 INFO:teuthology.orchestra.run.smithi016.stdout:true 2024-04-02T04:48:06.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.781+0000 7fed2cded700 1 -- 172.21.15.16:0/2159758057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed280fedf0 msgr2=0x7fed280ff210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:06.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.781+0000 7fed2cded700 1 --2- 172.21.15.16:0/2159758057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed280fedf0 0x7fed280ff210 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fed18009910 tx=0x7fed18009c20 comp rx=0 tx=0).stop 2024-04-02T04:48:06.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.781+0000 7fed2cded700 1 -- 172.21.15.16:0/2159758057 shutdown_connections 2024-04-02T04:48:06.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.781+0000 7fed2cded700 1 --2- 172.21.15.16:0/2159758057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed28101e70 0x7fed281022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:06.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.781+0000 7fed2cded700 1 --2- 172.21.15.16:0/2159758057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed280fedf0 0x7fed280ff210 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:06.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.781+0000 7fed2cded700 1 -- 172.21.15.16:0/2159758057 >> 172.21.15.16:0/2159758057 conn(0x7fed280fa730 msgr2=0x7fed280fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:06.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.781+0000 7fed2cded700 1 -- 172.21.15.16:0/2159758057 shutdown_connections 2024-04-02T04:48:06.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.781+0000 7fed2cded700 1 -- 172.21.15.16:0/2159758057 wait complete. 2024-04-02T04:48:06.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.782+0000 7fed2cded700 1 Processor -- start 2024-04-02T04:48:06.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.782+0000 7fed2cded700 1 -- start start 2024-04-02T04:48:06.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.782+0000 7fed2cded700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed280fedf0 0x7fed281965c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:06.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.782+0000 7fed2cded700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed28101e70 0x7fed28196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:06.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.782+0000 7fed2cded700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed28197120 con 0x7fed280fedf0 2024-04-02T04:48:06.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.782+0000 7fed2cded700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed28197260 con 0x7fed28101e70 2024-04-02T04:48:06.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.782+0000 7fed2659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed280fedf0 0x7fed281965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:06.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.782+0000 7fed2659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed280fedf0 0x7fed281965c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58884/0 (socket says 172.21.15.16:58884) 2024-04-02T04:48:06.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.782+0000 7fed2659c700 1 -- 172.21.15.16:0/3253557439 learned_addr learned my addr 172.21.15.16:0/3253557439 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:48:06.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.783+0000 7fed25d9b700 1 --2- 172.21.15.16:0/3253557439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed28101e70 0x7fed28196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:06.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.783+0000 7fed2659c700 1 -- 172.21.15.16:0/3253557439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed28101e70 msgr2=0x7fed28196b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:06.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.783+0000 7fed2659c700 1 --2- 172.21.15.16:0/3253557439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed28101e70 0x7fed28196b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:06.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.783+0000 7fed2659c700 1 -- 172.21.15.16:0/3253557439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed180095f0 con 0x7fed280fedf0 2024-04-02T04:48:06.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.783+0000 7fed2659c700 1 --2- 172.21.15.16:0/3253557439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed280fedf0 0x7fed281965c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fed18005850 tx=0x7fed1800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:06.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.783+0000 7fed177fe700 1 -- 172.21.15.16:0/3253557439 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed1801d070 con 0x7fed280fedf0 2024-04-02T04:48:06.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.783+0000 7fed177fe700 1 -- 172.21.15.16:0/3253557439 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed18005dc0 con 0x7fed280fedf0 2024-04-02T04:48:06.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.783+0000 7fed2cded700 1 -- 172.21.15.16:0/3253557439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed2819bc10 con 0x7fed280fedf0 2024-04-02T04:48:06.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.783+0000 7fed177fe700 1 -- 172.21.15.16:0/3253557439 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed18017400 con 0x7fed280fedf0 2024-04-02T04:48:06.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.783+0000 7fed2cded700 1 -- 172.21.15.16:0/3253557439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed2819c100 con 0x7fed280fedf0 2024-04-02T04:48:06.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.784+0000 7fed2cded700 1 -- 172.21.15.16:0/3253557439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed28190690 con 0x7fed280fedf0 2024-04-02T04:48:06.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.785+0000 7fed177fe700 1 -- 172.21.15.16:0/3253557439 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fed1800f6b0 con 0x7fed280fedf0 2024-04-02T04:48:06.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.785+0000 7fed177fe700 1 --2- 172.21.15.16:0/3253557439 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fed1006dd60 0x7fed10070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:06.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.785+0000 7fed25d9b700 1 --2- 172.21.15.16:0/3253557439 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fed1006dd60 0x7fed10070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:06.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.785+0000 7fed177fe700 1 -- 172.21.15.16:0/3253557439 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fed1808ef80 con 0x7fed280fedf0 2024-04-02T04:48:06.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.786+0000 7fed25d9b700 1 --2- 172.21.15.16:0/3253557439 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fed1006dd60 0x7fed10070220 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fed1c005950 tx=0x7fed1c00b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:06.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:06.790+0000 7fed177fe700 1 -- 172.21.15.16:0/3253557439 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fed1805c040 con 0x7fed280fedf0 2024-04-02T04:48:07.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.016+0000 7fed2cded700 1 -- 172.21.15.16:0/3253557439 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fed2802ce10 con 0x7fed1006dd60 2024-04-02T04:48:07.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.017+0000 7fed177fe700 1 -- 172.21.15.16:0/3253557439 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fed2802ce10 con 0x7fed1006dd60 2024-04-02T04:48:07.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.019+0000 7fed2cded700 1 -- 172.21.15.16:0/3253557439 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fed1006dd60 msgr2=0x7fed10070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:07.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.019+0000 7fed2cded700 1 --2- 172.21.15.16:0/3253557439 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fed1006dd60 0x7fed10070220 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fed1c005950 tx=0x7fed1c00b410 comp rx=0 tx=0).stop 2024-04-02T04:48:07.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.019+0000 7fed2cded700 1 -- 172.21.15.16:0/3253557439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed280fedf0 msgr2=0x7fed281965c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:07.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.019+0000 7fed2cded700 1 --2- 172.21.15.16:0/3253557439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed280fedf0 0x7fed281965c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fed18005850 tx=0x7fed1800bd70 comp rx=0 tx=0).stop 2024-04-02T04:48:07.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.019+0000 7fed2cded700 1 -- 172.21.15.16:0/3253557439 shutdown_connections 2024-04-02T04:48:07.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.019+0000 7fed2cded700 1 --2- 172.21.15.16:0/3253557439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed280fedf0 0x7fed281965c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:07.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.019+0000 7fed2cded700 1 --2- 172.21.15.16:0/3253557439 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fed1006dd60 0x7fed10070220 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:07.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.019+0000 7fed2cded700 1 --2- 172.21.15.16:0/3253557439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed28101e70 0x7fed28196b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:07.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.019+0000 7fed2cded700 1 -- 172.21.15.16:0/3253557439 >> 172.21.15.16:0/3253557439 conn(0x7fed280fa730 msgr2=0x7fed281054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:07.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.019+0000 7fed2cded700 1 -- 172.21.15.16:0/3253557439 shutdown_connections 2024-04-02T04:48:07.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.019+0000 7fed2cded700 1 -- 172.21.15.16:0/3253557439 wait complete. 2024-04-02T04:48:07.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.125+0000 7fdc5cda1700 1 -- 172.21.15.16:0/3068024275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc580ff9d0 msgr2=0x7fdc580ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:07.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.125+0000 7fdc5cda1700 1 --2- 172.21.15.16:0/3068024275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc580ff9d0 0x7fdc580ffdf0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdc48009910 tx=0x7fdc48009c20 comp rx=0 tx=0).stop 2024-04-02T04:48:07.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.125+0000 7fdc5cda1700 1 -- 172.21.15.16:0/3068024275 shutdown_connections 2024-04-02T04:48:07.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.125+0000 7fdc5cda1700 1 --2- 172.21.15.16:0/3068024275 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc58100bd0 0x7fdc58101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:07.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.126+0000 7fdc5cda1700 1 --2- 172.21.15.16:0/3068024275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc580ff9d0 0x7fdc580ffdf0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:07.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.126+0000 7fdc5cda1700 1 -- 172.21.15.16:0/3068024275 >> 172.21.15.16:0/3068024275 conn(0x7fdc580faf70 msgr2=0x7fdc580fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:07.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.126+0000 7fdc5cda1700 1 -- 172.21.15.16:0/3068024275 shutdown_connections 2024-04-02T04:48:07.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.126+0000 7fdc5cda1700 1 -- 172.21.15.16:0/3068024275 wait complete. 2024-04-02T04:48:07.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.126+0000 7fdc5cda1700 1 Processor -- start 2024-04-02T04:48:07.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.126+0000 7fdc5cda1700 1 -- start start 2024-04-02T04:48:07.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.127+0000 7fdc5cda1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc580ff9d0 0x7fdc58196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:07.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.127+0000 7fdc5cda1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc58100bd0 0x7fdc58196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:07.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.127+0000 7fdc5cda1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc58197060 con 0x7fdc58100bd0 2024-04-02T04:48:07.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.127+0000 7fdc5cda1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc581971a0 con 0x7fdc580ff9d0 2024-04-02T04:48:07.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.127+0000 7fdc55d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc58100bd0 0x7fdc58196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:07.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.127+0000 7fdc5659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc580ff9d0 0x7fdc58196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:07.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.127+0000 7fdc5659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc580ff9d0 0x7fdc58196500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.16:47792/0 (socket says 172.21.15.16:47792) 2024-04-02T04:48:07.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.127+0000 7fdc55d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc58100bd0 0x7fdc58196a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58892/0 (socket says 172.21.15.16:58892) 2024-04-02T04:48:07.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.127+0000 7fdc5659c700 1 -- 172.21.15.16:0/4246758392 learned_addr learned my addr 172.21.15.16:0/4246758392 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:48:07.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.127+0000 7fdc55d9b700 1 -- 172.21.15.16:0/4246758392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc580ff9d0 msgr2=0x7fdc58196500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:07.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.127+0000 7fdc55d9b700 1 --2- 172.21.15.16:0/4246758392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc580ff9d0 0x7fdc58196500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:07.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.127+0000 7fdc55d9b700 1 -- 172.21.15.16:0/4246758392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc480095f0 con 0x7fdc58100bd0 2024-04-02T04:48:07.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.127+0000 7fdc5659c700 1 --2- 172.21.15.16:0/4246758392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc580ff9d0 0x7fdc58196500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T04:48:07.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.127+0000 7fdc55d9b700 1 --2- 172.21.15.16:0/4246758392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc58100bd0 0x7fdc58196a40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c00d8d0 tx=0x7fdc4c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:07.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.127+0000 7fdc477fe700 1 -- 172.21.15.16:0/4246758392 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc4c00f860 con 0x7fdc58100bd0 2024-04-02T04:48:07.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.127+0000 7fdc477fe700 1 -- 172.21.15.16:0/4246758392 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc4c00fe80 con 0x7fdc58100bd0 2024-04-02T04:48:07.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.127+0000 7fdc5cda1700 1 -- 172.21.15.16:0/4246758392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc5819bbb0 con 0x7fdc58100bd0 2024-04-02T04:48:07.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.127+0000 7fdc477fe700 1 -- 172.21.15.16:0/4246758392 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc4c010ea0 con 0x7fdc58100bd0 2024-04-02T04:48:07.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.127+0000 7fdc5cda1700 1 -- 172.21.15.16:0/4246758392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc5819c180 con 0x7fdc58100bd0 2024-04-02T04:48:07.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.129+0000 7fdc5cda1700 1 -- 172.21.15.16:0/4246758392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc58190650 con 0x7fdc58100bd0 2024-04-02T04:48:07.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.129+0000 7fdc477fe700 1 -- 172.21.15.16:0/4246758392 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fdc4c00f9c0 con 0x7fdc58100bd0 2024-04-02T04:48:07.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.129+0000 7fdc477fe700 1 --2- 172.21.15.16:0/4246758392 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fdc4006dd60 0x7fdc40070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:07.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.129+0000 7fdc477fe700 1 -- 172.21.15.16:0/4246758392 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fdc4c08d720 con 0x7fdc58100bd0 2024-04-02T04:48:07.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.129+0000 7fdc5659c700 1 --2- 172.21.15.16:0/4246758392 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fdc4006dd60 0x7fdc40070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:07.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.130+0000 7fdc5659c700 1 --2- 172.21.15.16:0/4246758392 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fdc4006dd60 0x7fdc40070220 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdc480051c0 tx=0x7fdc48005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:07.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.134+0000 7fdc477fe700 1 -- 172.21.15.16:0/4246758392 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fdc4c05a7e0 con 0x7fdc58100bd0 2024-04-02T04:48:07.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.353+0000 7fdc5cda1700 1 -- 172.21.15.16:0/4246758392 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc5802ce10 con 0x7fdc4006dd60 2024-04-02T04:48:07.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.364+0000 7fdc477fe700 1 -- 172.21.15.16:0/4246758392 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fdc5802ce10 con 0x7fdc4006dd60 2024-04-02T04:48:07.367 INFO:teuthology.orchestra.run.smithi016.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:48:07.368 INFO:teuthology.orchestra.run.smithi016.stdout:alertmanager.smithi016 smithi016 *:9093,9094 running (12m) 54s ago 14m 24.1M - 0.25.0 c8568f914cd2 b95d8354b016 2024-04-02T04:48:07.368 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi016 smithi016 running (14m) 54s ago 14m 8300k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 9de6c16a487f 2024-04-02T04:48:07.368 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi027 smithi027 running (13m) 21s ago 13m 8128k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 99956592471a 2024-04-02T04:48:07.368 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi016 smithi016 running (2m) 54s ago 14m 6660k - 19.0.0-1979-g6cec793d 78f830190839 dc63ce0eef58 2024-04-02T04:48:07.368 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi027 smithi027 running (2m) 21s ago 13m 6665k - 19.0.0-1979-g6cec793d 78f830190839 2beb077018e4 2024-04-02T04:48:07.368 INFO:teuthology.orchestra.run.smithi016.stdout:grafana.smithi016 smithi016 *:3000 running (12m) 54s ago 13m 84.6M - 9.4.7 954c08fa6188 7c43d133569c 2024-04-02T04:48:07.368 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.fyzrnm smithi016 running (8m) 54s ago 8m 17.9M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 70f28b9572a6 2024-04-02T04:48:07.368 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.uhgkgg smithi016 running (8m) 54s ago 8m 16.1M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 0aa4d16562a6 2024-04-02T04:48:07.368 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.sdublg smithi027 running (8m) 21s ago 8m 20.2M - 18.2.2-1059-g4d05f4ca 909dce4fe53d a538451099db 2024-04-02T04:48:07.368 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.zwrfoo smithi027 running (8m) 21s ago 8m 157M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 4cb3fa428d71 2024-04-02T04:48:07.368 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi016.eqnebx smithi016 *:8443,9283,8765 running (5m) 54s ago 15m 501M - 19.0.0-1979-g6cec793d 78f830190839 d61ed91b4577 2024-04-02T04:48:07.368 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi027.kjzcsc smithi027 *:8443,9283,8765 running (4m) 21s ago 12m 450M - 19.0.0-1979-g6cec793d 78f830190839 5072a5ffc761 2024-04-02T04:48:07.368 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi016 smithi016 running (3m) 54s ago 15m 46.9M 2048M 19.0.0-1979-g6cec793d 78f830190839 eabfc84f1cdf 2024-04-02T04:48:07.368 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi027 smithi027 running (3m) 21s ago 12m 37.3M 2048M 19.0.0-1979-g6cec793d 78f830190839 fcd68b20dcc5 2024-04-02T04:48:07.368 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi016 smithi016 *:9100 running (14m) 54s ago 14m 15.2M - 1.5.0 0da6a335fe13 09f54c65b53c 2024-04-02T04:48:07.369 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi027 smithi027 *:9100 running (13m) 21s ago 13m 15.3M - 1.5.0 0da6a335fe13 e41a657c7443 2024-04-02T04:48:07.369 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 smithi016 running (2m) 54s ago 12m 81.5M 1217M 19.0.0-1979-g6cec793d 78f830190839 421d9e1cc751 2024-04-02T04:48:07.369 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 smithi016 running (95s) 54s ago 11m 87.0M 1217M 19.0.0-1979-g6cec793d 78f830190839 3612a56236a3 2024-04-02T04:48:07.369 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 smithi016 running (60s) 54s ago 11m 17.1M 1217M 19.0.0-1979-g6cec793d 78f830190839 a8e9b8d0303c 2024-04-02T04:48:07.369 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 smithi027 running (24s) 21s ago 10m 12.1M 2240M 19.0.0-1979-g6cec793d 78f830190839 0dd13b5be8c0 2024-04-02T04:48:07.369 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 smithi027 running (10m) 21s ago 10m 214M 2240M 18.2.2-1059-g4d05f4ca 909dce4fe53d 8cbbedf8cac9 2024-04-02T04:48:07.369 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 smithi027 running (9m) 21s ago 9m 196M 2240M 18.2.2-1059-g4d05f4ca 909dce4fe53d ac6094415529 2024-04-02T04:48:07.369 INFO:teuthology.orchestra.run.smithi016.stdout:prometheus.smithi016 smithi016 *:9095 running (4m) 54s ago 13m 58.5M - 2.43.0 a07b618ecd1d 466f00d73ddd 2024-04-02T04:48:07.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.366+0000 7fdc5cda1700 1 -- 172.21.15.16:0/4246758392 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fdc4006dd60 msgr2=0x7fdc40070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:07.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.366+0000 7fdc5cda1700 1 --2- 172.21.15.16:0/4246758392 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fdc4006dd60 0x7fdc40070220 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdc480051c0 tx=0x7fdc48005960 comp rx=0 tx=0).stop 2024-04-02T04:48:07.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.366+0000 7fdc5cda1700 1 -- 172.21.15.16:0/4246758392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc58100bd0 msgr2=0x7fdc58196a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:07.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.366+0000 7fdc5cda1700 1 --2- 172.21.15.16:0/4246758392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc58100bd0 0x7fdc58196a40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c00d8d0 tx=0x7fdc4c00dbe0 comp rx=0 tx=0).stop 2024-04-02T04:48:07.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.366+0000 7fdc5cda1700 1 -- 172.21.15.16:0/4246758392 shutdown_connections 2024-04-02T04:48:07.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.366+0000 7fdc5cda1700 1 --2- 172.21.15.16:0/4246758392 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fdc4006dd60 0x7fdc40070220 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:07.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.366+0000 7fdc5cda1700 1 --2- 172.21.15.16:0/4246758392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc580ff9d0 0x7fdc58196500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:07.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.366+0000 7fdc5cda1700 1 --2- 172.21.15.16:0/4246758392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc58100bd0 0x7fdc58196a40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:07.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.366+0000 7fdc5cda1700 1 -- 172.21.15.16:0/4246758392 >> 172.21.15.16:0/4246758392 conn(0x7fdc580faf70 msgr2=0x7fdc581041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:07.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.367+0000 7fdc5cda1700 1 -- 172.21.15.16:0/4246758392 shutdown_connections 2024-04-02T04:48:07.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.367+0000 7fdc5cda1700 1 -- 172.21.15.16:0/4246758392 wait complete. 2024-04-02T04:48:07.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.470+0000 7fc959983700 1 -- 172.21.15.16:0/39367852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc954100ae0 msgr2=0x7fc954100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:07.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.470+0000 7fc959983700 1 --2- 172.21.15.16:0/39367852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc954100ae0 0x7fc954100f00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc94400b0a0 tx=0x7fc94400b3b0 comp rx=0 tx=0).stop 2024-04-02T04:48:07.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.471+0000 7fc959983700 1 -- 172.21.15.16:0/39367852 shutdown_connections 2024-04-02T04:48:07.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.471+0000 7fc959983700 1 --2- 172.21.15.16:0/39367852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc954101ce0 0x7fc954102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:07.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.471+0000 7fc959983700 1 --2- 172.21.15.16:0/39367852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc954100ae0 0x7fc954100f00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:07.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.471+0000 7fc959983700 1 -- 172.21.15.16:0/39367852 >> 172.21.15.16:0/39367852 conn(0x7fc9540fc0a0 msgr2=0x7fc9540fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:07.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.471+0000 7fc959983700 1 -- 172.21.15.16:0/39367852 shutdown_connections 2024-04-02T04:48:07.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.471+0000 7fc959983700 1 -- 172.21.15.16:0/39367852 wait complete. 2024-04-02T04:48:07.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.471+0000 7fc959983700 1 Processor -- start 2024-04-02T04:48:07.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.471+0000 7fc959983700 1 -- start start 2024-04-02T04:48:07.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.471+0000 7fc959983700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc954100ae0 0x7fc954074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:07.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.471+0000 7fc959983700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc954101ce0 0x7fc9540731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:07.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.472+0000 7fc959983700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9540736e0 con 0x7fc954101ce0 2024-04-02T04:48:07.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.472+0000 7fc959983700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc954073820 con 0x7fc954100ae0 2024-04-02T04:48:07.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.472+0000 7fc9527fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc954101ce0 0x7fc9540731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:07.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.472+0000 7fc9527fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc954101ce0 0x7fc9540731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58902/0 (socket says 172.21.15.16:58902) 2024-04-02T04:48:07.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.472+0000 7fc9527fc700 1 -- 172.21.15.16:0/489133246 learned_addr learned my addr 172.21.15.16:0/489133246 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:48:07.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.472+0000 7fc952ffd700 1 --2- 172.21.15.16:0/489133246 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc954100ae0 0x7fc954074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:07.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.472+0000 7fc9527fc700 1 -- 172.21.15.16:0/489133246 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc954100ae0 msgr2=0x7fc954074b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:07.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.472+0000 7fc9527fc700 1 --2- 172.21.15.16:0/489133246 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc954100ae0 0x7fc954074b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:07.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.472+0000 7fc9527fc700 1 -- 172.21.15.16:0/489133246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc944009d00 con 0x7fc954101ce0 2024-04-02T04:48:07.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.472+0000 7fc9527fc700 1 --2- 172.21.15.16:0/489133246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc954101ce0 0x7fc9540731a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc93c00d8d0 tx=0x7fc93c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:07.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.472+0000 7fc958981700 1 -- 172.21.15.16:0/489133246 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc93c0098a0 con 0x7fc954101ce0 2024-04-02T04:48:07.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.472+0000 7fc958981700 1 -- 172.21.15.16:0/489133246 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc93c010460 con 0x7fc954101ce0 2024-04-02T04:48:07.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.472+0000 7fc959983700 1 -- 172.21.15.16:0/489133246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc954073b00 con 0x7fc954101ce0 2024-04-02T04:48:07.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.472+0000 7fc958981700 1 -- 172.21.15.16:0/489133246 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc93c00f5f0 con 0x7fc954101ce0 2024-04-02T04:48:07.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.472+0000 7fc959983700 1 -- 172.21.15.16:0/489133246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc95419e530 con 0x7fc954101ce0 2024-04-02T04:48:07.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.473+0000 7fc959983700 1 -- 172.21.15.16:0/489133246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc954190660 con 0x7fc954101ce0 2024-04-02T04:48:07.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.474+0000 7fc958981700 1 -- 172.21.15.16:0/489133246 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fc93c009a00 con 0x7fc954101ce0 2024-04-02T04:48:07.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.474+0000 7fc958981700 1 --2- 172.21.15.16:0/489133246 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc94006dd60 0x7fc940070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:07.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.474+0000 7fc958981700 1 -- 172.21.15.16:0/489133246 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fc93c08d9c0 con 0x7fc954101ce0 2024-04-02T04:48:07.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.474+0000 7fc952ffd700 1 --2- 172.21.15.16:0/489133246 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc94006dd60 0x7fc940070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:07.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.475+0000 7fc952ffd700 1 --2- 172.21.15.16:0/489133246 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc94006dd60 0x7fc940070220 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc944009fd0 tx=0x7fc944006000 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:07.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.479+0000 7fc958981700 1 -- 172.21.15.16:0/489133246 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc93c05aa60 con 0x7fc954101ce0 2024-04-02T04:48:07.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:07 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-02T04:48:07.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:07 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:07.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.770+0000 7fc959983700 1 -- 172.21.15.16:0/489133246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc954190040 con 0x7fc954101ce0 2024-04-02T04:48:07.774 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:07 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-02T04:48:07.774 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:07 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:07.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.771+0000 7fc958981700 1 -- 172.21.15.16:0/489133246 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+836 (secure 0 0 0) 0x7fc93c05a5f0 con 0x7fc954101ce0 2024-04-02T04:48:07.775 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:48:07.775 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": { 2024-04-02T04:48:07.775 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:48:07.775 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:48:07.775 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": { 2024-04-02T04:48:07.775 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:48:07.775 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:48:07.775 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": { 2024-04-02T04:48:07.775 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 2, 2024-04-02T04:48:07.775 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 4 2024-04-02T04:48:07.775 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:48:07.775 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-04-02T04:48:07.776 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 4 2024-04-02T04:48:07.776 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:48:07.776 INFO:teuthology.orchestra.run.smithi016.stdout: "overall": { 2024-04-02T04:48:07.776 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 6, 2024-04-02T04:48:07.776 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 8 2024-04-02T04:48:07.776 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-02T04:48:07.776 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:48:07.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.773+0000 7fc959983700 1 -- 172.21.15.16:0/489133246 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc94006dd60 msgr2=0x7fc940070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:07.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.773+0000 7fc959983700 1 --2- 172.21.15.16:0/489133246 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc94006dd60 0x7fc940070220 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc944009fd0 tx=0x7fc944006000 comp rx=0 tx=0).stop 2024-04-02T04:48:07.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.773+0000 7fc959983700 1 -- 172.21.15.16:0/489133246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc954101ce0 msgr2=0x7fc9540731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:07.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.773+0000 7fc959983700 1 --2- 172.21.15.16:0/489133246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc954101ce0 0x7fc9540731a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc93c00d8d0 tx=0x7fc93c00dbe0 comp rx=0 tx=0).stop 2024-04-02T04:48:07.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.773+0000 7fc959983700 1 -- 172.21.15.16:0/489133246 shutdown_connections 2024-04-02T04:48:07.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.773+0000 7fc959983700 1 --2- 172.21.15.16:0/489133246 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc94006dd60 0x7fc940070220 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:07.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.773+0000 7fc959983700 1 --2- 172.21.15.16:0/489133246 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc954100ae0 0x7fc954074b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:07.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.773+0000 7fc959983700 1 --2- 172.21.15.16:0/489133246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc954101ce0 0x7fc9540731a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:07.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.773+0000 7fc959983700 1 -- 172.21.15.16:0/489133246 >> 172.21.15.16:0/489133246 conn(0x7fc9540fc0a0 msgr2=0x7fc9541052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:07.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.774+0000 7fc959983700 1 -- 172.21.15.16:0/489133246 shutdown_connections 2024-04-02T04:48:07.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.774+0000 7fc959983700 1 -- 172.21.15.16:0/489133246 wait complete. 2024-04-02T04:48:07.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.879+0000 7f5341032700 1 -- 172.21.15.16:0/3008436871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f533c103f50 msgr2=0x7f533c106340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:07.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.879+0000 7f5341032700 1 --2- 172.21.15.16:0/3008436871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f533c103f50 0x7f533c106340 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5324009910 tx=0x7f5324009c20 comp rx=0 tx=0).stop 2024-04-02T04:48:07.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.879+0000 7f5341032700 1 -- 172.21.15.16:0/3008436871 shutdown_connections 2024-04-02T04:48:07.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.879+0000 7f5341032700 1 --2- 172.21.15.16:0/3008436871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f533c103f50 0x7f533c106340 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:07.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.879+0000 7f5341032700 1 --2- 172.21.15.16:0/3008436871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f533c101620 0x7f533c103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:07.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.879+0000 7f5341032700 1 -- 172.21.15.16:0/3008436871 >> 172.21.15.16:0/3008436871 conn(0x7f533c0faf70 msgr2=0x7f533c0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:07.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.879+0000 7f5341032700 1 -- 172.21.15.16:0/3008436871 shutdown_connections 2024-04-02T04:48:07.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.879+0000 7f5341032700 1 -- 172.21.15.16:0/3008436871 wait complete. 2024-04-02T04:48:07.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.879+0000 7f5341032700 1 Processor -- start 2024-04-02T04:48:07.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.880+0000 7f5341032700 1 -- start start 2024-04-02T04:48:07.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.880+0000 7f5341032700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f533c101620 0x7f533c100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:07.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.880+0000 7f5341032700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f533c103f50 0x7f533c0ff5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:07.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.880+0000 7f5341032700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533c0ffb10 con 0x7f533c101620 2024-04-02T04:48:07.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.880+0000 7f5341032700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533c0ffc50 con 0x7f533c103f50 2024-04-02T04:48:07.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.880+0000 7f533ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f533c101620 0x7f533c100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:07.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.880+0000 7f533ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f533c101620 0x7f533c100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58920/0 (socket says 172.21.15.16:58920) 2024-04-02T04:48:07.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.880+0000 7f533ad9d700 1 -- 172.21.15.16:0/3659659975 learned_addr learned my addr 172.21.15.16:0/3659659975 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:48:07.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.880+0000 7f533a59c700 1 --2- 172.21.15.16:0/3659659975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f533c103f50 0x7f533c0ff5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:07.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.880+0000 7f533ad9d700 1 -- 172.21.15.16:0/3659659975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f533c103f50 msgr2=0x7f533c0ff5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:07.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.880+0000 7f533ad9d700 1 --2- 172.21.15.16:0/3659659975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f533c103f50 0x7f533c0ff5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:07.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.880+0000 7f533ad9d700 1 -- 172.21.15.16:0/3659659975 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53240095f0 con 0x7f533c101620 2024-04-02T04:48:07.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.881+0000 7f533ad9d700 1 --2- 172.21.15.16:0/3659659975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f533c101620 0x7f533c100f30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f532c00ef10 tx=0x7f532c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:07.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.881+0000 7f5333fff700 1 -- 172.21.15.16:0/3659659975 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f532c00c9f0 con 0x7f533c101620 2024-04-02T04:48:07.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.881+0000 7f5341032700 1 -- 172.21.15.16:0/3659659975 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f533c0ffed0 con 0x7f533c101620 2024-04-02T04:48:07.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.881+0000 7f5333fff700 1 -- 172.21.15.16:0/3659659975 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f532c00f040 con 0x7f533c101620 2024-04-02T04:48:07.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.881+0000 7f5333fff700 1 -- 172.21.15.16:0/3659659975 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f532c0146b0 con 0x7f533c101620 2024-04-02T04:48:07.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.881+0000 7f5341032700 1 -- 172.21.15.16:0/3659659975 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f533c19e530 con 0x7f533c101620 2024-04-02T04:48:07.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.882+0000 7f5341032700 1 -- 172.21.15.16:0/3659659975 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f533c04ea80 con 0x7f533c101620 2024-04-02T04:48:07.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.882+0000 7f5333fff700 1 -- 172.21.15.16:0/3659659975 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f532c00cb50 con 0x7f533c101620 2024-04-02T04:48:07.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.883+0000 7f5333fff700 1 --2- 172.21.15.16:0/3659659975 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f532806dd60 0x7f5328070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:07.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.883+0000 7f5333fff700 1 -- 172.21.15.16:0/3659659975 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f532c08de80 con 0x7f533c101620 2024-04-02T04:48:07.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.883+0000 7f533a59c700 1 --2- 172.21.15.16:0/3659659975 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f532806dd60 0x7f5328070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:07.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.883+0000 7f533a59c700 1 --2- 172.21.15.16:0/3659659975 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f532806dd60 0x7f5328070220 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f532400b800 tx=0x7f5324009f70 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:07.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:07.888+0000 7f5333fff700 1 -- 172.21.15.16:0/3659659975 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f532c05af40 con 0x7f533c101620 2024-04-02T04:48:08.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.138+0000 7f5341032700 1 -- 172.21.15.16:0/3659659975 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f533c109870 con 0x7f533c101620 2024-04-02T04:48:08.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.138+0000 7f5333fff700 1 -- 172.21.15.16:0/3659659975 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1875 (secure 0 0 0) 0x7f532c007e60 con 0x7f533c101620 2024-04-02T04:48:08.141 INFO:teuthology.orchestra.run.smithi016.stdout:e15 2024-04-02T04:48:08.142 INFO:teuthology.orchestra.run.smithi016.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:48:08.142 INFO:teuthology.orchestra.run.smithi016.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:48:08.142 INFO:teuthology.orchestra.run.smithi016.stdout:legacy client fscid: 1 2024-04-02T04:48:08.142 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:48:08.142 INFO:teuthology.orchestra.run.smithi016.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:48:08.142 INFO:teuthology.orchestra.run.smithi016.stdout:fs_name cephfs 2024-04-02T04:48:08.142 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 15 2024-04-02T04:48:08.142 INFO:teuthology.orchestra.run.smithi016.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:48:08.142 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-04-02T04:39:30.740168+0000 2024-04-02T04:48:08.142 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-04-02T04:39:49.875319+0000 2024-04-02T04:48:08.142 INFO:teuthology.orchestra.run.smithi016.stdout:tableserver 0 2024-04-02T04:48:08.142 INFO:teuthology.orchestra.run.smithi016.stdout:root 0 2024-04-02T04:48:08.142 INFO:teuthology.orchestra.run.smithi016.stdout:session_timeout 60 2024-04-02T04:48:08.142 INFO:teuthology.orchestra.run.smithi016.stdout:session_autoclose 300 2024-04-02T04:48:08.143 INFO:teuthology.orchestra.run.smithi016.stdout:max_file_size 1099511627776 2024-04-02T04:48:08.143 INFO:teuthology.orchestra.run.smithi016.stdout:max_xattr_size 65536 2024-04-02T04:48:08.143 INFO:teuthology.orchestra.run.smithi016.stdout:required_client_features {} 2024-04-02T04:48:08.143 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure 0 2024-04-02T04:48:08.143 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure_osd_epoch 46 2024-04-02T04:48:08.143 INFO:teuthology.orchestra.run.smithi016.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:48:08.143 INFO:teuthology.orchestra.run.smithi016.stdout:max_mds 1 2024-04-02T04:48:08.143 INFO:teuthology.orchestra.run.smithi016.stdout:in 0 2024-04-02T04:48:08.143 INFO:teuthology.orchestra.run.smithi016.stdout:up {0=24285} 2024-04-02T04:48:08.143 INFO:teuthology.orchestra.run.smithi016.stdout:failed 2024-04-02T04:48:08.143 INFO:teuthology.orchestra.run.smithi016.stdout:damaged 2024-04-02T04:48:08.143 INFO:teuthology.orchestra.run.smithi016.stdout:stopped 2024-04-02T04:48:08.143 INFO:teuthology.orchestra.run.smithi016.stdout:data_pools [3] 2024-04-02T04:48:08.143 INFO:teuthology.orchestra.run.smithi016.stdout:metadata_pool 2 2024-04-02T04:48:08.144 INFO:teuthology.orchestra.run.smithi016.stdout:inline_data enabled 2024-04-02T04:48:08.144 INFO:teuthology.orchestra.run.smithi016.stdout:balancer 2024-04-02T04:48:08.144 INFO:teuthology.orchestra.run.smithi016.stdout:bal_rank_mask -1 2024-04-02T04:48:08.144 INFO:teuthology.orchestra.run.smithi016.stdout:standby_count_wanted 1 2024-04-02T04:48:08.144 INFO:teuthology.orchestra.run.smithi016.stdout:qdb_cluster leader: 0 members: 2024-04-02T04:48:08.144 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.zwrfoo{0:24285} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.27:6824/1461455976,v1:172.21.15.27:6825/1461455976] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:48:08.144 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:48:08.144 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:48:08.144 INFO:teuthology.orchestra.run.smithi016.stdout:Standby daemons: 2024-04-02T04:48:08.144 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:48:08.144 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.fyzrnm{-1:14488} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.16:6828/2578719859,v1:172.21.15.16:6829/2578719859] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:48:08.144 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.uhgkgg{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.16:6826/601263579,v1:172.21.15.16:6827/601263579] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:48:08.144 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.sdublg{-1:24295} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.27:6826/4177192740,v1:172.21.15.27:6827/4177192740] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:48:08.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.140+0000 7f5341032700 1 -- 172.21.15.16:0/3659659975 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f532806dd60 msgr2=0x7f5328070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:08.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.140+0000 7f5341032700 1 --2- 172.21.15.16:0/3659659975 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f532806dd60 0x7f5328070220 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f532400b800 tx=0x7f5324009f70 comp rx=0 tx=0).stop 2024-04-02T04:48:08.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.140+0000 7f5341032700 1 -- 172.21.15.16:0/3659659975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f533c101620 msgr2=0x7f533c100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:08.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.140+0000 7f5341032700 1 --2- 172.21.15.16:0/3659659975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f533c101620 0x7f533c100f30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f532c00ef10 tx=0x7f532c00c3a0 comp rx=0 tx=0).stop 2024-04-02T04:48:08.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.140+0000 7f5341032700 1 -- 172.21.15.16:0/3659659975 shutdown_connections 2024-04-02T04:48:08.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.140+0000 7f5341032700 1 --2- 172.21.15.16:0/3659659975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f533c101620 0x7f533c100f30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:08.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.140+0000 7f5341032700 1 --2- 172.21.15.16:0/3659659975 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f532806dd60 0x7f5328070220 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:08.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.140+0000 7f5341032700 1 --2- 172.21.15.16:0/3659659975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f533c103f50 0x7f533c0ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:08.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.140+0000 7f5341032700 1 -- 172.21.15.16:0/3659659975 >> 172.21.15.16:0/3659659975 conn(0x7f533c0faf70 msgr2=0x7f533c0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:08.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.140+0000 7f5341032700 1 -- 172.21.15.16:0/3659659975 shutdown_connections 2024-04-02T04:48:08.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.140+0000 7f5341032700 1 -- 172.21.15.16:0/3659659975 wait complete. 2024-04-02T04:48:08.146 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 15 2024-04-02T04:48:08.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.253+0000 7f453fead700 1 -- 172.21.15.16:0/965326885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4538100bf0 msgr2=0x7f4538101050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:08.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.253+0000 7f453fead700 1 --2- 172.21.15.16:0/965326885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4538100bf0 0x7f4538101050 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4528009910 tx=0x7f4528009c20 comp rx=0 tx=0).stop 2024-04-02T04:48:08.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.253+0000 7f453fead700 1 -- 172.21.15.16:0/965326885 shutdown_connections 2024-04-02T04:48:08.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.253+0000 7f453fead700 1 --2- 172.21.15.16:0/965326885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4538100bf0 0x7f4538101050 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:08.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.253+0000 7f453fead700 1 --2- 172.21.15.16:0/965326885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45380ff9f0 0x7f45380ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:08.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.253+0000 7f453fead700 1 -- 172.21.15.16:0/965326885 >> 172.21.15.16:0/965326885 conn(0x7f45380faf90 msgr2=0x7f45380fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:08.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.253+0000 7f453fead700 1 -- 172.21.15.16:0/965326885 shutdown_connections 2024-04-02T04:48:08.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.253+0000 7f453fead700 1 -- 172.21.15.16:0/965326885 wait complete. 2024-04-02T04:48:08.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.254+0000 7f453fead700 1 Processor -- start 2024-04-02T04:48:08.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.254+0000 7f453fead700 1 -- start start 2024-04-02T04:48:08.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.254+0000 7f453fead700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45380ff9f0 0x7f4538195490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:08.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.254+0000 7f453fead700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4538100bf0 0x7f45381959d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:08.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.254+0000 7f453fead700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4538195ff0 con 0x7f45380ff9f0 2024-04-02T04:48:08.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.254+0000 7f453fead700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4538196130 con 0x7f4538100bf0 2024-04-02T04:48:08.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.254+0000 7f453dc49700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45380ff9f0 0x7f4538195490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:08.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.254+0000 7f453dc49700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45380ff9f0 0x7f4538195490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58938/0 (socket says 172.21.15.16:58938) 2024-04-02T04:48:08.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.254+0000 7f453dc49700 1 -- 172.21.15.16:0/1497089934 learned_addr learned my addr 172.21.15.16:0/1497089934 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:48:08.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.254+0000 7f453d448700 1 --2- 172.21.15.16:0/1497089934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4538100bf0 0x7f45381959d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:08.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.254+0000 7f453dc49700 1 -- 172.21.15.16:0/1497089934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4538100bf0 msgr2=0x7f45381959d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:08.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.254+0000 7f453dc49700 1 --2- 172.21.15.16:0/1497089934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4538100bf0 0x7f45381959d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:08.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.254+0000 7f453dc49700 1 -- 172.21.15.16:0/1497089934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45280095f0 con 0x7f45380ff9f0 2024-04-02T04:48:08.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.255+0000 7f453dc49700 1 --2- 172.21.15.16:0/1497089934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45380ff9f0 0x7f4538195490 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f45340098f0 tx=0x7f4534009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:08.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.255+0000 7f452effd700 1 -- 172.21.15.16:0/1497089934 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4534012070 con 0x7f45380ff9f0 2024-04-02T04:48:08.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.255+0000 7f453fead700 1 -- 172.21.15.16:0/1497089934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4538069070 con 0x7f45380ff9f0 2024-04-02T04:48:08.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.255+0000 7f452effd700 1 -- 172.21.15.16:0/1497089934 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f45340044e0 con 0x7f45380ff9f0 2024-04-02T04:48:08.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.255+0000 7f452effd700 1 -- 172.21.15.16:0/1497089934 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4534019480 con 0x7f45380ff9f0 2024-04-02T04:48:08.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.255+0000 7f453fead700 1 -- 172.21.15.16:0/1497089934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4538069640 con 0x7f45380ff9f0 2024-04-02T04:48:08.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.256+0000 7f453fead700 1 -- 172.21.15.16:0/1497089934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f453804ea80 con 0x7f45380ff9f0 2024-04-02T04:48:08.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.256+0000 7f452effd700 1 -- 172.21.15.16:0/1497089934 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f4534016030 con 0x7f45380ff9f0 2024-04-02T04:48:08.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.257+0000 7f452effd700 1 --2- 172.21.15.16:0/1497089934 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f452406dd60 0x7f4524070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:08.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.257+0000 7f452effd700 1 -- 172.21.15.16:0/1497089934 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f453408efd0 con 0x7f45380ff9f0 2024-04-02T04:48:08.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.257+0000 7f453d448700 1 --2- 172.21.15.16:0/1497089934 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f452406dd60 0x7f4524070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:08.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.257+0000 7f453d448700 1 --2- 172.21.15.16:0/1497089934 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f452406dd60 0x7f4524070220 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f45280051c0 tx=0x7f452801a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:08.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.261+0000 7f452effd700 1 -- 172.21.15.16:0/1497089934 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f453405c090 con 0x7f45380ff9f0 2024-04-02T04:48:08.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.486+0000 7f453fead700 1 -- 172.21.15.16:0/1497089934 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f453818f150 con 0x7f452406dd60 2024-04-02T04:48:08.490 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:08 smithi016 ceph-mon[93230]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-02T04:48:08.490 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:08 smithi016 ceph-mon[93230]: Upgrade: osd.4 is safe to restart 2024-04-02T04:48:08.490 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:08 smithi016 ceph-mon[93230]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:48:08.490 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:08 smithi016 ceph-mon[93230]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:48:08.490 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:08 smithi016 ceph-mon[93230]: pgmap v138: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:08.490 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:08 smithi016 ceph-mon[93230]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:48:08.490 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:08 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/489133246' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:48:08.490 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:08 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/3659659975' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:48:08.490 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:08 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:08.490 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:08 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-02T04:48:08.491 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:08 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:48:08.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.487+0000 7f452effd700 1 -- 172.21.15.16:0/1497089934 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f453818f150 con 0x7f452406dd60 2024-04-02T04:48:08.491 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:48:08.491 INFO:teuthology.orchestra.run.smithi016.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:48:08.491 INFO:teuthology.orchestra.run.smithi016.stdout: "in_progress": true, 2024-04-02T04:48:08.491 INFO:teuthology.orchestra.run.smithi016.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-02T04:48:08.491 INFO:teuthology.orchestra.run.smithi016.stdout: "services_complete": [ 2024-04-02T04:48:08.491 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr", 2024-04-02T04:48:08.492 INFO:teuthology.orchestra.run.smithi016.stdout: "mon", 2024-04-02T04:48:08.492 INFO:teuthology.orchestra.run.smithi016.stdout: "crash" 2024-04-02T04:48:08.492 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-04-02T04:48:08.492 INFO:teuthology.orchestra.run.smithi016.stdout: "progress": "10/23 daemons upgraded", 2024-04-02T04:48:08.492 INFO:teuthology.orchestra.run.smithi016.stdout: "message": "Currently upgrading osd daemons", 2024-04-02T04:48:08.492 INFO:teuthology.orchestra.run.smithi016.stdout: "is_paused": false 2024-04-02T04:48:08.492 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:48:08.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.489+0000 7f453fead700 1 -- 172.21.15.16:0/1497089934 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f452406dd60 msgr2=0x7f4524070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:08.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.489+0000 7f453fead700 1 --2- 172.21.15.16:0/1497089934 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f452406dd60 0x7f4524070220 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f45280051c0 tx=0x7f452801a040 comp rx=0 tx=0).stop 2024-04-02T04:48:08.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.489+0000 7f453fead700 1 -- 172.21.15.16:0/1497089934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45380ff9f0 msgr2=0x7f4538195490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:08.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.489+0000 7f453fead700 1 --2- 172.21.15.16:0/1497089934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45380ff9f0 0x7f4538195490 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f45340098f0 tx=0x7f4534009c00 comp rx=0 tx=0).stop 2024-04-02T04:48:08.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.489+0000 7f453fead700 1 -- 172.21.15.16:0/1497089934 shutdown_connections 2024-04-02T04:48:08.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.489+0000 7f453fead700 1 --2- 172.21.15.16:0/1497089934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45380ff9f0 0x7f4538195490 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:08.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.489+0000 7f453fead700 1 --2- 172.21.15.16:0/1497089934 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f452406dd60 0x7f4524070220 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:08.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.489+0000 7f453fead700 1 --2- 172.21.15.16:0/1497089934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4538100bf0 0x7f45381959d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:08.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.489+0000 7f453fead700 1 -- 172.21.15.16:0/1497089934 >> 172.21.15.16:0/1497089934 conn(0x7f45380faf90 msgr2=0x7f4538104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:08.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.490+0000 7f453fead700 1 -- 172.21.15.16:0/1497089934 shutdown_connections 2024-04-02T04:48:08.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.490+0000 7f453fead700 1 -- 172.21.15.16:0/1497089934 wait complete. 2024-04-02T04:48:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.594+0000 7f5c178a3700 1 -- 172.21.15.16:0/4075575299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c10101d10 msgr2=0x7f5c10102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.594+0000 7f5c178a3700 1 --2- 172.21.15.16:0/4075575299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c10101d10 0x7f5c10102170 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5c04009910 tx=0x7f5c04009c20 comp rx=0 tx=0).stop 2024-04-02T04:48:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.594+0000 7f5c178a3700 1 -- 172.21.15.16:0/4075575299 shutdown_connections 2024-04-02T04:48:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.594+0000 7f5c178a3700 1 --2- 172.21.15.16:0/4075575299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c10101d10 0x7f5c10102170 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.594+0000 7f5c178a3700 1 --2- 172.21.15.16:0/4075575299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c10100b10 0x7f5c10100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.594+0000 7f5c178a3700 1 -- 172.21.15.16:0/4075575299 >> 172.21.15.16:0/4075575299 conn(0x7f5c100fc0d0 msgr2=0x7f5c100fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.594+0000 7f5c178a3700 1 -- 172.21.15.16:0/4075575299 shutdown_connections 2024-04-02T04:48:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.594+0000 7f5c178a3700 1 -- 172.21.15.16:0/4075575299 wait complete. 2024-04-02T04:48:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.595+0000 7f5c178a3700 1 Processor -- start 2024-04-02T04:48:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.595+0000 7f5c178a3700 1 -- start start 2024-04-02T04:48:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.595+0000 7f5c178a3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c10100b10 0x7f5c101964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.595+0000 7f5c178a3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c10101d10 0x7f5c10196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.595+0000 7f5c178a3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c10197030 con 0x7f5c10101d10 2024-04-02T04:48:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.595+0000 7f5c178a3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c10197170 con 0x7f5c10100b10 2024-04-02T04:48:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.595+0000 7f5c14e3e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c10101d10 0x7f5c10196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.595+0000 7f5c1563f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c10100b10 0x7f5c101964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:08.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.595+0000 7f5c14e3e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c10101d10 0x7f5c10196a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58956/0 (socket says 172.21.15.16:58956) 2024-04-02T04:48:08.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.595+0000 7f5c14e3e700 1 -- 172.21.15.16:0/1951187745 learned_addr learned my addr 172.21.15.16:0/1951187745 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:48:08.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.596+0000 7f5c14e3e700 1 -- 172.21.15.16:0/1951187745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c10100b10 msgr2=0x7f5c101964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:08.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.596+0000 7f5c14e3e700 1 --2- 172.21.15.16:0/1951187745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c10100b10 0x7f5c101964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:08.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.596+0000 7f5c14e3e700 1 -- 172.21.15.16:0/1951187745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c040095f0 con 0x7f5c10101d10 2024-04-02T04:48:08.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.596+0000 7f5c14e3e700 1 --2- 172.21.15.16:0/1951187745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c10101d10 0x7f5c10196a10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5c040092e0 tx=0x7f5c0400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:08.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.596+0000 7f5c027fc700 1 -- 172.21.15.16:0/1951187745 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c0401d070 con 0x7f5c10101d10 2024-04-02T04:48:08.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.596+0000 7f5c027fc700 1 -- 172.21.15.16:0/1951187745 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c0400f460 con 0x7f5c10101d10 2024-04-02T04:48:08.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.596+0000 7f5c178a3700 1 -- 172.21.15.16:0/1951187745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c1019bb20 con 0x7f5c10101d10 2024-04-02T04:48:08.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.596+0000 7f5c027fc700 1 -- 172.21.15.16:0/1951187745 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c04021620 con 0x7f5c10101d10 2024-04-02T04:48:08.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.596+0000 7f5c178a3700 1 -- 172.21.15.16:0/1951187745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c1019c090 con 0x7f5c10101d10 2024-04-02T04:48:08.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.597+0000 7f5c178a3700 1 -- 172.21.15.16:0/1951187745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c10190690 con 0x7f5c10101d10 2024-04-02T04:48:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.598+0000 7f5c027fc700 1 -- 172.21.15.16:0/1951187745 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f5c0400f5d0 con 0x7f5c10101d10 2024-04-02T04:48:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.598+0000 7f5c027fc700 1 --2- 172.21.15.16:0/1951187745 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5bfc06dd60 0x7f5bfc070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.598+0000 7f5c027fc700 1 -- 172.21.15.16:0/1951187745 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f5c0408ed20 con 0x7f5c10101d10 2024-04-02T04:48:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.598+0000 7f5c1563f700 1 --2- 172.21.15.16:0/1951187745 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5bfc06dd60 0x7f5bfc070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.598+0000 7f5c1563f700 1 --2- 172.21.15.16:0/1951187745 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5bfc06dd60 0x7f5bfc070220 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5c10101b70 tx=0x7f5c0c00b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:08.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.603+0000 7f5c027fc700 1 -- 172.21.15.16:0/1951187745 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5c0405bdc0 con 0x7f5c10101d10 2024-04-02T04:48:08.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:08 smithi027 ceph-mon[83036]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-02T04:48:08.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:08 smithi027 ceph-mon[83036]: Upgrade: osd.4 is safe to restart 2024-04-02T04:48:08.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:08 smithi027 ceph-mon[83036]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:48:08.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:08 smithi027 ceph-mon[83036]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:48:08.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:08 smithi027 ceph-mon[83036]: pgmap v138: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:08.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:08 smithi027 ceph-mon[83036]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:48:08.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:08 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/489133246' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:48:08.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:08 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/3659659975' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:48:08.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:08 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:08.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:08 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-02T04:48:08.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:08 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:48:08.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.894+0000 7f5c178a3700 1 -- 172.21.15.16:0/1951187745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5c1004ea80 con 0x7f5c10101d10 2024-04-02T04:48:08.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.894+0000 7f5c027fc700 1 -- 172.21.15.16:0/1951187745 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f5c0405b950 con 0x7f5c10101d10 2024-04-02T04:48:08.898 INFO:teuthology.orchestra.run.smithi016.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-02T04:48:08.898 INFO:teuthology.orchestra.run.smithi016.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-02T04:48:08.898 INFO:teuthology.orchestra.run.smithi016.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-02T04:48:08.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.896+0000 7f5c178a3700 1 -- 172.21.15.16:0/1951187745 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5bfc06dd60 msgr2=0x7f5bfc070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:08.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.896+0000 7f5c178a3700 1 --2- 172.21.15.16:0/1951187745 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5bfc06dd60 0x7f5bfc070220 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5c10101b70 tx=0x7f5c0c00b410 comp rx=0 tx=0).stop 2024-04-02T04:48:08.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.896+0000 7f5c178a3700 1 -- 172.21.15.16:0/1951187745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c10101d10 msgr2=0x7f5c10196a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:08.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.896+0000 7f5c178a3700 1 --2- 172.21.15.16:0/1951187745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c10101d10 0x7f5c10196a10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5c040092e0 tx=0x7f5c0400bd70 comp rx=0 tx=0).stop 2024-04-02T04:48:08.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.896+0000 7f5c178a3700 1 -- 172.21.15.16:0/1951187745 shutdown_connections 2024-04-02T04:48:08.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.896+0000 7f5c178a3700 1 --2- 172.21.15.16:0/1951187745 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5bfc06dd60 0x7f5bfc070220 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:08.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.896+0000 7f5c178a3700 1 --2- 172.21.15.16:0/1951187745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c10100b10 0x7f5c101964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:08.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.896+0000 7f5c178a3700 1 --2- 172.21.15.16:0/1951187745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c10101d10 0x7f5c10196a10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:08.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.896+0000 7f5c178a3700 1 -- 172.21.15.16:0/1951187745 >> 172.21.15.16:0/1951187745 conn(0x7f5c100fc0d0 msgr2=0x7f5c10105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:08.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.896+0000 7f5c178a3700 1 -- 172.21.15.16:0/1951187745 shutdown_connections 2024-04-02T04:48:08.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:08.896+0000 7f5c178a3700 1 -- 172.21.15.16:0/1951187745 wait complete. 2024-04-02T04:48:09.483 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:09 smithi027 ceph-mon[83036]: Upgrade: Updating osd.4 2024-04-02T04:48:09.483 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:09 smithi027 ceph-mon[83036]: Deploying daemon osd.4 on smithi027 2024-04-02T04:48:09.483 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:09 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/1951187745' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:48:09.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:09 smithi016 ceph-mon[93230]: Upgrade: Updating osd.4 2024-04-02T04:48:09.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:09 smithi016 ceph-mon[93230]: Deploying daemon osd.4 on smithi027 2024-04-02T04:48:09.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:09 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/1951187745' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:48:10.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:10 smithi027 ceph-mon[83036]: from='client.34364 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:48:10.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:10 smithi027 ceph-mon[83036]: pgmap v139: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:10.809 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:10 smithi016 ceph-mon[93230]: from='client.34364 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:48:10.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:10 smithi016 ceph-mon[93230]: pgmap v139: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:11.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:11 smithi027 ceph-mon[83036]: pgmap v140: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:11.646 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:11 smithi027 systemd[1]: Stopping Ceph osd.4 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4... 2024-04-02T04:48:11.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:11 smithi016 ceph-mon[93230]: pgmap v140: 65 pgs: 65 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:12.146 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:11 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4[50292]: 2024-04-02T04:48:11.725+0000 7f0571bce700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T04:48:12.146 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:11 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4[50292]: 2024-04-02T04:48:11.725+0000 7f0571bce700 -1 osd.4 69 *** Got signal Terminated *** 2024-04-02T04:48:12.146 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:11 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4[50292]: 2024-04-02T04:48:11.725+0000 7f0571bce700 -1 osd.4 69 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T04:48:12.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:12 smithi027 ceph-mon[83036]: osd.4 marked itself down and dead 2024-04-02T04:48:12.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:12 smithi016 ceph-mon[93230]: osd.4 marked itself down and dead 2024-04-02T04:48:13.081 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:12 smithi027 podman[97690]: 2024-04-02 04:48:12.802795852 +0000 UTC m=+1.197274260 container died 8cbbedf8cac91c93b646099955eac0eaeed0c5ac002e028e375ebf1b3db20dd9 (image=quay.ceph.io/ceph-ci/ceph@sha256:397d0c1d908c002c558906e3ea05982bc336d403ee4d900400cc3bdd9243aa2e, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4, GIT_BRANCH=HEAD, RELEASE=reef-4d05f4c, org.label-schema.build-date=20240326, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-02T04:48:13.369 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:13 smithi027 podman[97690]: 2024-04-02 04:48:13.233915504 +0000 UTC m=+1.628393912 container cleanup 8cbbedf8cac91c93b646099955eac0eaeed0c5ac002e028e375ebf1b3db20dd9 (image=quay.ceph.io/ceph-ci/ceph@sha256:397d0c1d908c002c558906e3ea05982bc336d403ee4d900400cc3bdd9243aa2e, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=reef-4d05f4c, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T04:48:13.369 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:13 smithi027 bash[97690]: ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4 2024-04-02T04:48:13.646 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:13 smithi027 podman[97718]: 2024-04-02 04:48:13.468669251 +0000 UTC m=+0.668417662 container remove 8cbbedf8cac91c93b646099955eac0eaeed0c5ac002e028e375ebf1b3db20dd9 (image=quay.ceph.io/ceph-ci/ceph@sha256:397d0c1d908c002c558906e3ea05982bc336d403ee4d900400cc3bdd9243aa2e, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=reef-4d05f4c, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T04:48:13.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:13 smithi027 ceph-mon[83036]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-02T04:48:13.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:13 smithi027 ceph-mon[83036]: failed to encode map e70 with expected crc 2024-04-02T04:48:13.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:13 smithi027 ceph-mon[83036]: osdmap e70: 6 total, 5 up, 6 in 2024-04-02T04:48:13.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:13 smithi027 ceph-mon[83036]: pgmap v142: 65 pgs: 5 peering, 5 stale+active+clean, 55 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:13.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:13 smithi016 ceph-mon[93230]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-02T04:48:13.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:13 smithi016 ceph-mon[93230]: failed to encode map e70 with expected crc 2024-04-02T04:48:13.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:13 smithi016 ceph-mon[93230]: osdmap e70: 6 total, 5 up, 6 in 2024-04-02T04:48:13.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:13 smithi016 ceph-mon[93230]: pgmap v142: 65 pgs: 5 peering, 5 stale+active+clean, 55 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:14.116 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:13 smithi027 podman[97809]: 2024-04-02 04:48:13.701170413 +0000 UTC m=+0.018274599 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:48:14.117 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:13 smithi027 podman[97809]: 2024-04-02 04:48:13.819525722 +0000 UTC m=+0.136629878 container create 70fb6fd2dc2843f03e0e818329803470be0e277f85e1b0dca41fddb348dda81a (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, RELEASE=squid-6cec793, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-02T04:48:14.117 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:14 smithi027 podman[97809]: 2024-04-02 04:48:14.030441366 +0000 UTC m=+0.347545534 container init 70fb6fd2dc2843f03e0e818329803470be0e277f85e1b0dca41fddb348dda81a (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-6cec793, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T04:48:14.117 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:14 smithi027 podman[97809]: 2024-04-02 04:48:14.033890487 +0000 UTC m=+0.350994645 container start 70fb6fd2dc2843f03e0e818329803470be0e277f85e1b0dca41fddb348dda81a (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, RELEASE=squid-6cec793, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T04:48:14.117 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:14 smithi027 podman[97809]: 2024-04-02 04:48:14.113525678 +0000 UTC m=+0.430629874 container attach 70fb6fd2dc2843f03e0e818329803470be0e277f85e1b0dca41fddb348dda81a (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-6cec793, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-02T04:48:14.371 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:14 smithi027 podman[97809]: 2024-04-02 04:48:14.339592357 +0000 UTC m=+0.656696526 container died 70fb6fd2dc2843f03e0e818329803470be0e277f85e1b0dca41fddb348dda81a (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, RELEASE=squid-6cec793, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T04:48:14.628 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:14 smithi027 ceph-mon[83036]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-02T04:48:14.628 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:14 smithi027 ceph-mon[83036]: osdmap e71: 6 total, 5 up, 6 in 2024-04-02T04:48:14.628 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:14 smithi027 ceph-mon[83036]: failed to encode map e71 with expected crc 2024-04-02T04:48:14.628 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:14 smithi027 ceph-mon[83036]: failed to encode map e71 with expected crc 2024-04-02T04:48:14.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:14 smithi016 ceph-mon[93230]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-02T04:48:14.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:14 smithi016 ceph-mon[93230]: osdmap e71: 6 total, 5 up, 6 in 2024-04-02T04:48:14.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:14 smithi016 ceph-mon[93230]: failed to encode map e71 with expected crc 2024-04-02T04:48:14.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:14 smithi016 ceph-mon[93230]: failed to encode map e71 with expected crc 2024-04-02T04:48:15.376 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:14 smithi027 podman[97836]: 2024-04-02 04:48:14.9342427 +0000 UTC m=+0.585777614 container remove 70fb6fd2dc2843f03e0e818329803470be0e277f85e1b0dca41fddb348dda81a (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-6cec793, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-02T04:48:15.376 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:14 smithi027 systemd[1]: ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@osd.4.service: Deactivated successfully. 2024-04-02T04:48:15.376 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:14 smithi027 systemd[1]: Stopped Ceph osd.4 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4. 2024-04-02T04:48:15.376 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:14 smithi027 systemd[1]: ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@osd.4.service: Consumed 54.934s CPU time. 2024-04-02T04:48:15.376 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:15 smithi027 systemd[1]: Starting Ceph osd.4 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4... 2024-04-02T04:48:15.632 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:15 smithi027 ceph-mon[83036]: pgmap v144: 65 pgs: 5 active+undersized+degraded, 13 peering, 3 stale+active+clean, 6 active+undersized, 38 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 22/267 objects degraded (8.240%) 2024-04-02T04:48:15.632 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:15 smithi027 podman[97936]: 2024-04-02 04:48:15.299149883 +0000 UTC m=+0.018145685 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:48:15.633 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:15 smithi027 podman[97936]: 2024-04-02 04:48:15.427518557 +0000 UTC m=+0.146514350 container create 1e7341401bdcb3d45a52d95032536c572dadf85bdfbce020add6a1e3e1e823c2 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4-activate, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, RELEASE=squid-6cec793, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-02T04:48:15.633 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:15 smithi027 podman[97936]: 2024-04-02 04:48:15.629930685 +0000 UTC m=+0.348926491 container init 1e7341401bdcb3d45a52d95032536c572dadf85bdfbce020add6a1e3e1e823c2 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4-activate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-6cec793, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T04:48:15.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:15 smithi016 ceph-mon[93230]: pgmap v144: 65 pgs: 5 active+undersized+degraded, 13 peering, 3 stale+active+clean, 6 active+undersized, 38 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 22/267 objects degraded (8.240%) 2024-04-02T04:48:15.896 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:15 smithi027 podman[97936]: 2024-04-02 04:48:15.633522649 +0000 UTC m=+0.352518443 container start 1e7341401bdcb3d45a52d95032536c572dadf85bdfbce020add6a1e3e1e823c2 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4-activate, org.label-schema.license=GPLv2, RELEASE=squid-6cec793, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T04:48:15.896 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:15 smithi027 podman[97936]: 2024-04-02 04:48:15.704691194 +0000 UTC m=+0.423686989 container attach 1e7341401bdcb3d45a52d95032536c572dadf85bdfbce020add6a1e3e1e823c2 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4-activate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-6cec793, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-02T04:48:16.378 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:16 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4-activate[97951]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-02T04:48:16.378 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:16 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4-activate[97951]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-02T04:48:16.378 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:16 smithi027 bash[97936]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-02T04:48:16.378 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:16 smithi027 bash[97936]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-02T04:48:16.378 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:16 smithi027 bash[97936]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-02T04:48:16.378 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:16 smithi027 bash[97936]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-02T04:48:16.378 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:16 smithi027 bash[97936]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-04-02T04:48:16.379 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:16 smithi027 bash[97936]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-02T04:48:16.379 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:16 smithi027 bash[97936]: --> ceph-volume raw activate successful for osd ID: 4 2024-04-02T04:48:16.379 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:16 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4-activate[97951]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-02T04:48:16.379 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:16 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4-activate[97951]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-02T04:48:16.379 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:16 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4-activate[97951]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-04-02T04:48:16.379 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:16 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4-activate[97951]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-02T04:48:16.379 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:16 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4-activate[97951]: --> ceph-volume raw activate successful for osd ID: 4 2024-04-02T04:48:16.379 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:16 smithi027 podman[97936]: 2024-04-02 04:48:16.118745317 +0000 UTC m=+0.837741111 container died 1e7341401bdcb3d45a52d95032536c572dadf85bdfbce020add6a1e3e1e823c2 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-6cec793, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326) 2024-04-02T04:48:16.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:16 smithi027 ceph-mon[83036]: Health check failed: Degraded data redundancy: 22/267 objects degraded (8.240%), 5 pgs degraded (PG_DEGRADED) 2024-04-02T04:48:16.711 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:16 smithi016 ceph-mon[93230]: Health check failed: Degraded data redundancy: 22/267 objects degraded (8.240%), 5 pgs degraded (PG_DEGRADED) 2024-04-02T04:48:17.091 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:16 smithi027 podman[98087]: 2024-04-02 04:48:16.750397753 +0000 UTC m=+0.626491901 container remove 1e7341401bdcb3d45a52d95032536c572dadf85bdfbce020add6a1e3e1e823c2 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4-activate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-6cec793, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T04:48:17.091 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:16 smithi027 podman[98150]: 2024-04-02 04:48:16.869392611 +0000 UTC m=+0.018142730 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:48:17.091 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:16 smithi027 podman[98150]: 2024-04-02 04:48:16.993196624 +0000 UTC m=+0.141946765 container create 9025eacf8145aaf2fee8b614698330e23898a09bd8fadebcb47768b3936a1fe7 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, RELEASE=squid-6cec793, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-02T04:48:17.646 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:17 smithi027 podman[98150]: 2024-04-02 04:48:17.211940579 +0000 UTC m=+0.360690690 container init 9025eacf8145aaf2fee8b614698330e23898a09bd8fadebcb47768b3936a1fe7 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=squid-6cec793, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326) 2024-04-02T04:48:17.646 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:17 smithi027 podman[98150]: 2024-04-02 04:48:17.215181336 +0000 UTC m=+0.363931437 container start 9025eacf8145aaf2fee8b614698330e23898a09bd8fadebcb47768b3936a1fe7 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-6cec793, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T04:48:17.646 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:17 smithi027 ceph-osd[98169]: -- 172.21.15.27:0/1089505154 <== mon.1 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556e51d93680 con 0x556e51d5cc00 2024-04-02T04:48:17.646 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:17 smithi027 bash[98150]: 9025eacf8145aaf2fee8b614698330e23898a09bd8fadebcb47768b3936a1fe7 2024-04-02T04:48:17.646 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:17 smithi027 systemd[1]: Started Ceph osd.4 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4. 2024-04-02T04:48:18.096 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:17 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4[98165]: 2024-04-02T04:48:17.755+0000 7f5e812f47c0 -1 Falling back to public interface 2024-04-02T04:48:18.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:18 smithi027 ceph-mon[83036]: pgmap v145: 65 pgs: 9 active+undersized+degraded, 13 peering, 1 stale+active+clean, 10 active+undersized, 32 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 33/267 objects degraded (12.360%) 2024-04-02T04:48:18.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:18 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:18.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:18 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:48:18.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:18 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:18.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:18 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:18.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:18 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:48:18.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:18 smithi016 ceph-mon[93230]: pgmap v145: 65 pgs: 9 active+undersized+degraded, 13 peering, 1 stale+active+clean, 10 active+undersized, 32 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 33/267 objects degraded (12.360%) 2024-04-02T04:48:18.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:18 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:18.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:18 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:48:18.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:18 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:18.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:18 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:18.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:18 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:48:19.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:19 smithi027 ceph-mon[83036]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-02T04:48:19.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:19 smithi016 ceph-mon[93230]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-02T04:48:20.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:20 smithi016 ceph-mon[93230]: pgmap v146: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 47/267 objects degraded (17.603%) 2024-04-02T04:48:20.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:20 smithi027 ceph-mon[83036]: pgmap v146: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 47/267 objects degraded (17.603%) 2024-04-02T04:48:21.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:21 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:21.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:21 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:21.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:21 smithi016 ceph-mon[93230]: pgmap v147: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 47/267 objects degraded (17.603%) 2024-04-02T04:48:21.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:21 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:21.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:21 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:21.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:21 smithi027 ceph-mon[83036]: pgmap v147: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 47/267 objects degraded (17.603%) 2024-04-02T04:48:22.748 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:22 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:22.748 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:22 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:22.748 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:22 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4[98165]: 2024-04-02T04:48:22.675+0000 7f5e812f47c0 -1 osd.4 69 log_to_monitors true 2024-04-02T04:48:22.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:22 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:22.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:22 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:23.803 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:48:23 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4[98165]: 2024-04-02T04:48:23.572+0000 7f5e6e744700 -1 osd.4 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T04:48:23.803 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:23 smithi027 ceph-mon[83036]: from='osd.4 [v2:172.21.15.27:6808/2931886568,v1:172.21.15.27:6809/2931886568]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T04:48:23.803 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:23 smithi027 ceph-mon[83036]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T04:48:23.803 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:23 smithi027 ceph-mon[83036]: Health check update: Degraded data redundancy: 47/267 objects degraded (17.603%), 15 pgs degraded (PG_DEGRADED) 2024-04-02T04:48:23.803 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:23 smithi027 ceph-mon[83036]: pgmap v148: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 47/267 objects degraded (17.603%) 2024-04-02T04:48:23.809 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:23 smithi016 ceph-mon[93230]: from='osd.4 [v2:172.21.15.27:6808/2931886568,v1:172.21.15.27:6809/2931886568]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T04:48:23.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:23 smithi016 ceph-mon[93230]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T04:48:23.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:23 smithi016 ceph-mon[93230]: Health check update: Degraded data redundancy: 47/267 objects degraded (17.603%), 15 pgs degraded (PG_DEGRADED) 2024-04-02T04:48:23.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:23 smithi016 ceph-mon[93230]: pgmap v148: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 260 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 47/267 objects degraded (17.603%) 2024-04-02T04:48:24.790 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:24 smithi027 ceph-mon[83036]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-02T04:48:24.790 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:24 smithi027 ceph-mon[83036]: failed to encode map e72 with expected crc 2024-04-02T04:48:24.790 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:24 smithi027 ceph-mon[83036]: osdmap e72: 6 total, 5 up, 6 in 2024-04-02T04:48:24.790 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:24 smithi027 ceph-mon[83036]: from='osd.4 [v2:172.21.15.27:6808/2931886568,v1:172.21.15.27:6809/2931886568]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-02T04:48:24.790 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:24 smithi027 ceph-mon[83036]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-02T04:48:24.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:24 smithi016 ceph-mon[93230]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-02T04:48:24.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:24 smithi016 ceph-mon[93230]: failed to encode map e72 with expected crc 2024-04-02T04:48:24.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:24 smithi016 ceph-mon[93230]: osdmap e72: 6 total, 5 up, 6 in 2024-04-02T04:48:24.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:24 smithi016 ceph-mon[93230]: from='osd.4 [v2:172.21.15.27:6808/2931886568,v1:172.21.15.27:6809/2931886568]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-02T04:48:24.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:24 smithi016 ceph-mon[93230]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-02T04:48:25.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:25 smithi016 ceph-mon[93230]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-02T04:48:25.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:25 smithi016 ceph-mon[93230]: failed to encode map e73 with expected crc 2024-04-02T04:48:25.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:25 smithi016 ceph-mon[93230]: osd.4 [v2:172.21.15.27:6808/2931886568,v1:172.21.15.27:6809/2931886568] boot 2024-04-02T04:48:25.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:25 smithi016 ceph-mon[93230]: osdmap e73: 6 total, 6 up, 6 in 2024-04-02T04:48:25.811 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:25 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T04:48:25.811 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:25 smithi016 ceph-mon[93230]: pgmap v151: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 260 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 47/267 objects degraded (17.603%) 2024-04-02T04:48:25.811 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:25 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:25.811 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:25 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:25.811 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:25 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:48:25.811 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:25 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:48:25.811 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:25 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:25.811 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:25 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:48:25.811 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:25 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:48:25.811 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:25 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:48:25.811 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:25 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:48:25.811 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:25 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-02T04:48:25.811 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:25 smithi016 ceph-mon[93230]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-02T04:48:25.811 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:25 smithi016 ceph-mon[93230]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-04-02T04:48:25.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:25 smithi027 ceph-mon[83036]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-02T04:48:25.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:25 smithi027 ceph-mon[83036]: failed to encode map e73 with expected crc 2024-04-02T04:48:25.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:25 smithi027 ceph-mon[83036]: osd.4 [v2:172.21.15.27:6808/2931886568,v1:172.21.15.27:6809/2931886568] boot 2024-04-02T04:48:25.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:25 smithi027 ceph-mon[83036]: osdmap e73: 6 total, 6 up, 6 in 2024-04-02T04:48:25.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:25 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T04:48:25.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:25 smithi027 ceph-mon[83036]: pgmap v151: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 260 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 47/267 objects degraded (17.603%) 2024-04-02T04:48:25.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:25 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:25.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:25 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:25.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:25 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:48:25.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:25 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:48:25.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:25 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:25.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:25 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:48:25.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:25 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:48:25.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:25 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:48:25.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:25 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:48:25.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:25 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-02T04:48:25.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:25 smithi027 ceph-mon[83036]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-02T04:48:25.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:25 smithi027 ceph-mon[83036]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-04-02T04:48:26.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:26 smithi016 ceph-mon[93230]: failed to encode map e74 with expected crc 2024-04-02T04:48:26.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:26 smithi016 ceph-mon[93230]: osdmap e74: 6 total, 6 up, 6 in 2024-04-02T04:48:26.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:26 smithi016 ceph-mon[93230]: failed to encode map e74 with expected crc 2024-04-02T04:48:26.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:26 smithi016 ceph-mon[93230]: failed to encode map e74 with expected crc 2024-04-02T04:48:26.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:26 smithi027 ceph-mon[83036]: failed to encode map e74 with expected crc 2024-04-02T04:48:26.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:26 smithi027 ceph-mon[83036]: osdmap e74: 6 total, 6 up, 6 in 2024-04-02T04:48:26.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:26 smithi027 ceph-mon[83036]: failed to encode map e74 with expected crc 2024-04-02T04:48:26.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:26 smithi027 ceph-mon[83036]: failed to encode map e74 with expected crc 2024-04-02T04:48:27.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:27 smithi027 ceph-mon[83036]: pgmap v153: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 260 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 39/267 objects degraded (14.607%) 2024-04-02T04:48:28.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:27 smithi016 ceph-mon[93230]: pgmap v153: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 260 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 39/267 objects degraded (14.607%) 2024-04-02T04:48:28.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:28 smithi027 ceph-mon[83036]: Health check update: Degraded data redundancy: 39/267 objects degraded (14.607%), 12 pgs degraded (PG_DEGRADED) 2024-04-02T04:48:29.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:28 smithi016 ceph-mon[93230]: Health check update: Degraded data redundancy: 39/267 objects degraded (14.607%), 12 pgs degraded (PG_DEGRADED) 2024-04-02T04:48:29.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:29 smithi027 ceph-mon[83036]: pgmap v154: 65 pgs: 5 active+undersized+degraded, 5 active+undersized, 55 active+clean; 260 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 22/267 objects degraded (8.240%) 2024-04-02T04:48:30.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:29 smithi016 ceph-mon[93230]: pgmap v154: 65 pgs: 5 active+undersized+degraded, 5 active+undersized, 55 active+clean; 260 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 22/267 objects degraded (8.240%) 2024-04-02T04:48:31.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:31 smithi016 ceph-mon[93230]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 22/267 objects degraded (8.240%), 5 pgs degraded) 2024-04-02T04:48:31.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:31 smithi027 ceph-mon[83036]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 22/267 objects degraded (8.240%), 5 pgs degraded) 2024-04-02T04:48:32.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:32 smithi016 ceph-mon[93230]: pgmap v155: 65 pgs: 65 active+clean; 260 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:32.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:32 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:32.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:32 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:48:32.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:32 smithi027 ceph-mon[83036]: pgmap v155: 65 pgs: 65 active+clean; 260 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:32.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:32 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:32.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:32 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:48:34.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:34 smithi016 ceph-mon[93230]: pgmap v156: 65 pgs: 65 active+clean; 260 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:34.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:34 smithi027 ceph-mon[83036]: pgmap v156: 65 pgs: 65 active+clean; 260 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:36.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:36 smithi016 ceph-mon[93230]: pgmap v157: 65 pgs: 65 active+clean; 260 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:36.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:36 smithi027 ceph-mon[83036]: pgmap v157: 65 pgs: 65 active+clean; 260 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:38.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:38 smithi016 ceph-mon[93230]: pgmap v158: 65 pgs: 65 active+clean; 260 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:38.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:38 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:38.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:38 smithi027 ceph-mon[83036]: pgmap v158: 65 pgs: 65 active+clean; 260 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:38.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:38 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:39.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.016+0000 7f6099691700 1 -- 172.21.15.16:0/526528192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6094074de0 msgr2=0x7f6094073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:39.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.016+0000 7f6099691700 1 --2- 172.21.15.16:0/526528192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6094074de0 0x7f6094073240 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f608400b0a0 tx=0x7f608400b3b0 comp rx=0 tx=0).stop 2024-04-02T04:48:39.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.016+0000 7f6099691700 1 -- 172.21.15.16:0/526528192 shutdown_connections 2024-04-02T04:48:39.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.016+0000 7f6099691700 1 --2- 172.21.15.16:0/526528192 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6094073810 0x7f6094073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:39.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.016+0000 7f6099691700 1 --2- 172.21.15.16:0/526528192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6094074de0 0x7f6094073240 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:39.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.016+0000 7f6099691700 1 -- 172.21.15.16:0/526528192 >> 172.21.15.16:0/526528192 conn(0x7f60940fc0d0 msgr2=0x7f60940fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:39.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.016+0000 7f6099691700 1 -- 172.21.15.16:0/526528192 shutdown_connections 2024-04-02T04:48:39.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.016+0000 7f6099691700 1 -- 172.21.15.16:0/526528192 wait complete. 2024-04-02T04:48:39.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.017+0000 7f6099691700 1 Processor -- start 2024-04-02T04:48:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.017+0000 7f6099691700 1 -- start start 2024-04-02T04:48:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.017+0000 7f6099691700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6094073810 0x7f6094198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.017+0000 7f6099691700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6094074de0 0x7f6094198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.017+0000 7f6099691700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60941992e0 con 0x7f6094074de0 2024-04-02T04:48:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.017+0000 7f6099691700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6094199420 con 0x7f6094073810 2024-04-02T04:48:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.017+0000 7f60927fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6094074de0 0x7f6094198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.017+0000 7f60927fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6094074de0 0x7f6094198cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38158/0 (socket says 172.21.15.16:38158) 2024-04-02T04:48:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.017+0000 7f60927fc700 1 -- 172.21.15.16:0/882668625 learned_addr learned my addr 172.21.15.16:0/882668625 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:48:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.017+0000 7f6092ffd700 1 --2- 172.21.15.16:0/882668625 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6094073810 0x7f6094198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.018+0000 7f60927fc700 1 -- 172.21.15.16:0/882668625 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6094073810 msgr2=0x7f6094198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.018+0000 7f60927fc700 1 --2- 172.21.15.16:0/882668625 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6094073810 0x7f6094198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.018+0000 7f60927fc700 1 -- 172.21.15.16:0/882668625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6084009d00 con 0x7f6094074de0 2024-04-02T04:48:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.018+0000 7f60927fc700 1 --2- 172.21.15.16:0/882668625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6094074de0 0x7f6094198cc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f607c00d900 tx=0x7f607c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.018+0000 7f608bfff700 1 -- 172.21.15.16:0/882668625 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f607c0041f0 con 0x7f6094074de0 2024-04-02T04:48:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.018+0000 7f608bfff700 1 -- 172.21.15.16:0/882668625 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f607c004350 con 0x7f6094074de0 2024-04-02T04:48:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.018+0000 7f6099691700 1 -- 172.21.15.16:0/882668625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f609419de80 con 0x7f6094074de0 2024-04-02T04:48:39.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.018+0000 7f608bfff700 1 -- 172.21.15.16:0/882668625 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f607c003cf0 con 0x7f6094074de0 2024-04-02T04:48:39.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.018+0000 7f6099691700 1 -- 172.21.15.16:0/882668625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f609419e420 con 0x7f6094074de0 2024-04-02T04:48:39.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.019+0000 7f6099691700 1 -- 172.21.15.16:0/882668625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60941928a0 con 0x7f6094074de0 2024-04-02T04:48:39.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.020+0000 7f608bfff700 1 -- 172.21.15.16:0/882668625 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f607c009730 con 0x7f6094074de0 2024-04-02T04:48:39.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.020+0000 7f608bfff700 1 --2- 172.21.15.16:0/882668625 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f608006dd70 0x7f6080070230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:39.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.020+0000 7f608bfff700 1 -- 172.21.15.16:0/882668625 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f607c08d9e0 con 0x7f6094074de0 2024-04-02T04:48:39.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.020+0000 7f6092ffd700 1 --2- 172.21.15.16:0/882668625 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f608006dd70 0x7f6080070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:39.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.021+0000 7f6092ffd700 1 --2- 172.21.15.16:0/882668625 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f608006dd70 0x7f6080070230 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6084009fd0 tx=0x7f6084007910 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:39.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.025+0000 7f608bfff700 1 -- 172.21.15.16:0/882668625 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f607c05aaa0 con 0x7f6094074de0 2024-04-02T04:48:39.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.250+0000 7f6099691700 1 -- 172.21.15.16:0/882668625 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f609402cdc0 con 0x7f608006dd70 2024-04-02T04:48:39.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.251+0000 7f608bfff700 1 -- 172.21.15.16:0/882668625 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f609402cdc0 con 0x7f608006dd70 2024-04-02T04:48:39.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.253+0000 7f6099691700 1 -- 172.21.15.16:0/882668625 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f608006dd70 msgr2=0x7f6080070230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.253+0000 7f6099691700 1 --2- 172.21.15.16:0/882668625 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f608006dd70 0x7f6080070230 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6084009fd0 tx=0x7f6084007910 comp rx=0 tx=0).stop 2024-04-02T04:48:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.253+0000 7f6099691700 1 -- 172.21.15.16:0/882668625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6094074de0 msgr2=0x7f6094198cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.253+0000 7f6099691700 1 --2- 172.21.15.16:0/882668625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6094074de0 0x7f6094198cc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f607c00d900 tx=0x7f607c00dc10 comp rx=0 tx=0).stop 2024-04-02T04:48:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.253+0000 7f6099691700 1 -- 172.21.15.16:0/882668625 shutdown_connections 2024-04-02T04:48:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.253+0000 7f6099691700 1 --2- 172.21.15.16:0/882668625 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f608006dd70 0x7f6080070230 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.253+0000 7f6099691700 1 --2- 172.21.15.16:0/882668625 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6094073810 0x7f6094198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.253+0000 7f6099691700 1 --2- 172.21.15.16:0/882668625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6094074de0 0x7f6094198cc0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.253+0000 7f6099691700 1 -- 172.21.15.16:0/882668625 >> 172.21.15.16:0/882668625 conn(0x7f60940fc0d0 msgr2=0x7f6094106d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.253+0000 7f6099691700 1 -- 172.21.15.16:0/882668625 shutdown_connections 2024-04-02T04:48:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.253+0000 7f6099691700 1 -- 172.21.15.16:0/882668625 wait complete. 2024-04-02T04:48:39.268 INFO:teuthology.orchestra.run.smithi016.stdout:true 2024-04-02T04:48:39.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.359+0000 7fed10b0b700 1 -- 172.21.15.16:0/1269924554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed0c100b00 msgr2=0x7fed0c100f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:39.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.359+0000 7fed10b0b700 1 --2- 172.21.15.16:0/1269924554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed0c100b00 0x7fed0c100f20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fecfc00b0a0 tx=0x7fecfc00b3b0 comp rx=0 tx=0).stop 2024-04-02T04:48:39.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.359+0000 7fed10b0b700 1 -- 172.21.15.16:0/1269924554 shutdown_connections 2024-04-02T04:48:39.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.359+0000 7fed10b0b700 1 --2- 172.21.15.16:0/1269924554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed0c101d00 0x7fed0c102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:39.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.359+0000 7fed10b0b700 1 --2- 172.21.15.16:0/1269924554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed0c100b00 0x7fed0c100f20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:39.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.359+0000 7fed10b0b700 1 -- 172.21.15.16:0/1269924554 >> 172.21.15.16:0/1269924554 conn(0x7fed0c0fc0a0 msgr2=0x7fed0c0fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:39.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.359+0000 7fed10b0b700 1 -- 172.21.15.16:0/1269924554 shutdown_connections 2024-04-02T04:48:39.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.359+0000 7fed10b0b700 1 -- 172.21.15.16:0/1269924554 wait complete. 2024-04-02T04:48:39.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.360+0000 7fed10b0b700 1 Processor -- start 2024-04-02T04:48:39.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.360+0000 7fed10b0b700 1 -- start start 2024-04-02T04:48:39.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.360+0000 7fed10b0b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed0c100b00 0x7fed0c1964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:39.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.360+0000 7fed10b0b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed0c101d00 0x7fed0c196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:39.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.360+0000 7fed10b0b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0c196f90 con 0x7fed0c100b00 2024-04-02T04:48:39.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.360+0000 7fed10b0b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0c1970d0 con 0x7fed0c101d00 2024-04-02T04:48:39.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.360+0000 7fed0a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed0c100b00 0x7fed0c1964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:39.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.360+0000 7fed0a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed0c100b00 0x7fed0c1964c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38176/0 (socket says 172.21.15.16:38176) 2024-04-02T04:48:39.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.360+0000 7fed0a59c700 1 -- 172.21.15.16:0/3171608572 learned_addr learned my addr 172.21.15.16:0/3171608572 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:48:39.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.360+0000 7fed09d9b700 1 --2- 172.21.15.16:0/3171608572 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed0c101d00 0x7fed0c196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:39.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.361+0000 7fed0a59c700 1 -- 172.21.15.16:0/3171608572 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed0c101d00 msgr2=0x7fed0c196a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:39.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.361+0000 7fed0a59c700 1 --2- 172.21.15.16:0/3171608572 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed0c101d00 0x7fed0c196a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:39.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.361+0000 7fed0a59c700 1 -- 172.21.15.16:0/3171608572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecfc009d00 con 0x7fed0c100b00 2024-04-02T04:48:39.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.361+0000 7fed0a59c700 1 --2- 172.21.15.16:0/3171608572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed0c100b00 0x7fed0c1964c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fecfc015040 tx=0x7fecfc003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:39.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.361+0000 7fed037fe700 1 -- 172.21.15.16:0/3171608572 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecfc021070 con 0x7fed0c100b00 2024-04-02T04:48:39.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.361+0000 7fed10b0b700 1 -- 172.21.15.16:0/3171608572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed0c19bae0 con 0x7fed0c100b00 2024-04-02T04:48:39.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.361+0000 7fed037fe700 1 -- 172.21.15.16:0/3171608572 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fecfc0075c0 con 0x7fed0c100b00 2024-04-02T04:48:39.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.361+0000 7fed037fe700 1 -- 172.21.15.16:0/3171608572 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecfc00e040 con 0x7fed0c100b00 2024-04-02T04:48:39.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.361+0000 7fed10b0b700 1 -- 172.21.15.16:0/3171608572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed0c19c020 con 0x7fed0c100b00 2024-04-02T04:48:39.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.362+0000 7fed10b0b700 1 -- 172.21.15.16:0/3171608572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed0c190680 con 0x7fed0c100b00 2024-04-02T04:48:39.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.363+0000 7fed037fe700 1 -- 172.21.15.16:0/3171608572 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fecfc004250 con 0x7fed0c100b00 2024-04-02T04:48:39.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.363+0000 7fed037fe700 1 --2- 172.21.15.16:0/3171608572 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fecf806dd60 0x7fecf8070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:39.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.363+0000 7fed037fe700 1 -- 172.21.15.16:0/3171608572 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fecfc08eda0 con 0x7fed0c100b00 2024-04-02T04:48:39.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.363+0000 7fed09d9b700 1 --2- 172.21.15.16:0/3171608572 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fecf806dd60 0x7fecf8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:39.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.364+0000 7fed09d9b700 1 --2- 172.21.15.16:0/3171608572 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fecf806dd60 0x7fecf8070220 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fecf4005ea0 tx=0x7fecf4005e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:39.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.368+0000 7fed037fe700 1 -- 172.21.15.16:0/3171608572 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fecfc05bf10 con 0x7fed0c100b00 2024-04-02T04:48:39.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.591+0000 7fed10b0b700 1 -- 172.21.15.16:0/3171608572 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fed0c02cd50 con 0x7fecf806dd60 2024-04-02T04:48:39.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.592+0000 7fed037fe700 1 -- 172.21.15.16:0/3171608572 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fed0c02cd50 con 0x7fecf806dd60 2024-04-02T04:48:39.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.594+0000 7fed10b0b700 1 -- 172.21.15.16:0/3171608572 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fecf806dd60 msgr2=0x7fecf8070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:39.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.594+0000 7fed10b0b700 1 --2- 172.21.15.16:0/3171608572 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fecf806dd60 0x7fecf8070220 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fecf4005ea0 tx=0x7fecf4005e30 comp rx=0 tx=0).stop 2024-04-02T04:48:39.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.594+0000 7fed10b0b700 1 -- 172.21.15.16:0/3171608572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed0c100b00 msgr2=0x7fed0c1964c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:39.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.594+0000 7fed10b0b700 1 --2- 172.21.15.16:0/3171608572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed0c100b00 0x7fed0c1964c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fecfc015040 tx=0x7fecfc003ce0 comp rx=0 tx=0).stop 2024-04-02T04:48:39.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.594+0000 7fed10b0b700 1 -- 172.21.15.16:0/3171608572 shutdown_connections 2024-04-02T04:48:39.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.594+0000 7fed10b0b700 1 --2- 172.21.15.16:0/3171608572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed0c100b00 0x7fed0c1964c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:39.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.594+0000 7fed10b0b700 1 --2- 172.21.15.16:0/3171608572 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fecf806dd60 0x7fecf8070220 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:39.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.594+0000 7fed10b0b700 1 --2- 172.21.15.16:0/3171608572 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed0c101d00 0x7fed0c196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:39.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.594+0000 7fed10b0b700 1 -- 172.21.15.16:0/3171608572 >> 172.21.15.16:0/3171608572 conn(0x7fed0c0fc0a0 msgr2=0x7fed0c1052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:39.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.594+0000 7fed10b0b700 1 -- 172.21.15.16:0/3171608572 shutdown_connections 2024-04-02T04:48:39.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.594+0000 7fed10b0b700 1 -- 172.21.15.16:0/3171608572 wait complete. 2024-04-02T04:48:39.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.700+0000 7f5247393700 1 -- 172.21.15.16:0/1295141599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5240101560 msgr2=0x7f5240103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:39.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.700+0000 7f5247393700 1 --2- 172.21.15.16:0/1295141599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5240101560 0x7f5240103950 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f523000b0a0 tx=0x7f523000b3b0 comp rx=0 tx=0).stop 2024-04-02T04:48:39.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.700+0000 7f5247393700 1 -- 172.21.15.16:0/1295141599 shutdown_connections 2024-04-02T04:48:39.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.700+0000 7f5247393700 1 --2- 172.21.15.16:0/1295141599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5240103e90 0x7f5240106280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:39.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.700+0000 7f5247393700 1 --2- 172.21.15.16:0/1295141599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5240101560 0x7f5240103950 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:39.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.700+0000 7f5247393700 1 -- 172.21.15.16:0/1295141599 >> 172.21.15.16:0/1295141599 conn(0x7f52400faf70 msgr2=0x7f52400fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:39.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.700+0000 7f5247393700 1 -- 172.21.15.16:0/1295141599 shutdown_connections 2024-04-02T04:48:39.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.700+0000 7f5247393700 1 -- 172.21.15.16:0/1295141599 wait complete. 2024-04-02T04:48:39.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.701+0000 7f5247393700 1 Processor -- start 2024-04-02T04:48:39.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.701+0000 7f5247393700 1 -- start start 2024-04-02T04:48:39.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.701+0000 7f5247393700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5240101560 0x7f5240196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:39.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.701+0000 7f5247393700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5240103e90 0x7f52401969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:39.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.701+0000 7f5247393700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5240196ff0 con 0x7f5240101560 2024-04-02T04:48:39.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.701+0000 7f5247393700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5240197130 con 0x7f5240103e90 2024-04-02T04:48:39.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.701+0000 7f524512f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5240101560 0x7f5240196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:39.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.701+0000 7f524512f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5240101560 0x7f5240196490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38194/0 (socket says 172.21.15.16:38194) 2024-04-02T04:48:39.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.701+0000 7f524512f700 1 -- 172.21.15.16:0/1860806811 learned_addr learned my addr 172.21.15.16:0/1860806811 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:48:39.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.701+0000 7f524492e700 1 --2- 172.21.15.16:0/1860806811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5240103e90 0x7f52401969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:39.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.701+0000 7f524512f700 1 -- 172.21.15.16:0/1860806811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5240103e90 msgr2=0x7f52401969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:39.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.701+0000 7f524512f700 1 --2- 172.21.15.16:0/1860806811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5240103e90 0x7f52401969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:39.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.701+0000 7f524512f700 1 -- 172.21.15.16:0/1860806811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5230009d00 con 0x7f5240101560 2024-04-02T04:48:39.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.702+0000 7f524512f700 1 --2- 172.21.15.16:0/1860806811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5240101560 0x7f5240196490 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5230006130 tx=0x7f5230003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:39.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.702+0000 7f523e7fc700 1 -- 172.21.15.16:0/1860806811 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5230021070 con 0x7f5240101560 2024-04-02T04:48:39.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.702+0000 7f523e7fc700 1 -- 172.21.15.16:0/1860806811 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f523000ce60 con 0x7f5240101560 2024-04-02T04:48:39.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.702+0000 7f523e7fc700 1 -- 172.21.15.16:0/1860806811 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f523000e040 con 0x7f5240101560 2024-04-02T04:48:39.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.702+0000 7f5247393700 1 -- 172.21.15.16:0/1860806811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f524019bb30 con 0x7f5240101560 2024-04-02T04:48:39.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.702+0000 7f5247393700 1 -- 172.21.15.16:0/1860806811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f524019c050 con 0x7f5240101560 2024-04-02T04:48:39.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.703+0000 7f5247393700 1 -- 172.21.15.16:0/1860806811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f524004ea80 con 0x7f5240101560 2024-04-02T04:48:39.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.703+0000 7f523e7fc700 1 -- 172.21.15.16:0/1860806811 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f5230004250 con 0x7f5240101560 2024-04-02T04:48:39.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.704+0000 7f523e7fc700 1 --2- 172.21.15.16:0/1860806811 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f522806dd60 0x7f5228070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:39.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.704+0000 7f524492e700 1 --2- 172.21.15.16:0/1860806811 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f522806dd60 0x7f5228070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:39.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.704+0000 7f523e7fc700 1 -- 172.21.15.16:0/1860806811 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f523008ef70 con 0x7f5240101560 2024-04-02T04:48:39.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.705+0000 7f524492e700 1 --2- 172.21.15.16:0/1860806811 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f522806dd60 0x7f5228070220 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5234005950 tx=0x7f523400b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:39.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.709+0000 7f523e7fc700 1 -- 172.21.15.16:0/1860806811 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f52300c3280 con 0x7f5240101560 2024-04-02T04:48:39.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.922+0000 7f5247393700 1 -- 172.21.15.16:0/1860806811 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5240105d70 con 0x7f522806dd60 2024-04-02T04:48:39.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.933+0000 7f523e7fc700 1 -- 172.21.15.16:0/1860806811 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f5240105d70 con 0x7f522806dd60 2024-04-02T04:48:39.938 INFO:teuthology.orchestra.run.smithi016.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:48:39.938 INFO:teuthology.orchestra.run.smithi016.stdout:alertmanager.smithi016 smithi016 *:9093,9094 running (13m) 87s ago 14m 24.1M - 0.25.0 c8568f914cd2 b95d8354b016 2024-04-02T04:48:39.938 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi016 smithi016 running (14m) 87s ago 14m 8300k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 9de6c16a487f 2024-04-02T04:48:39.938 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi027 smithi027 running (13m) 19s ago 13m 8392k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 99956592471a 2024-04-02T04:48:39.938 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi016 smithi016 running (3m) 87s ago 14m 6660k - 19.0.0-1979-g6cec793d 78f830190839 dc63ce0eef58 2024-04-02T04:48:39.938 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi027 smithi027 running (3m) 19s ago 13m 6665k - 19.0.0-1979-g6cec793d 78f830190839 2beb077018e4 2024-04-02T04:48:39.938 INFO:teuthology.orchestra.run.smithi016.stdout:grafana.smithi016 smithi016 *:3000 running (12m) 87s ago 14m 84.6M - 9.4.7 954c08fa6188 7c43d133569c 2024-04-02T04:48:39.938 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.fyzrnm smithi016 running (8m) 87s ago 8m 17.9M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 70f28b9572a6 2024-04-02T04:48:39.938 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.uhgkgg smithi016 running (9m) 87s ago 9m 16.1M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 0aa4d16562a6 2024-04-02T04:48:39.939 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.sdublg smithi027 running (8m) 19s ago 8m 20.4M - 18.2.2-1059-g4d05f4ca 909dce4fe53d a538451099db 2024-04-02T04:48:39.939 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.zwrfoo smithi027 running (9m) 19s ago 9m 157M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 4cb3fa428d71 2024-04-02T04:48:39.939 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi016.eqnebx smithi016 *:8443,9283,8765 running (5m) 87s ago 16m 501M - 19.0.0-1979-g6cec793d 78f830190839 d61ed91b4577 2024-04-02T04:48:39.939 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi027.kjzcsc smithi027 *:8443,9283,8765 running (4m) 19s ago 13m 450M - 19.0.0-1979-g6cec793d 78f830190839 5072a5ffc761 2024-04-02T04:48:39.939 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi016 smithi016 running (4m) 87s ago 16m 46.9M 2048M 19.0.0-1979-g6cec793d 78f830190839 eabfc84f1cdf 2024-04-02T04:48:39.939 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi027 smithi027 running (4m) 19s ago 13m 38.9M 2048M 19.0.0-1979-g6cec793d 78f830190839 fcd68b20dcc5 2024-04-02T04:48:39.939 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi016 smithi016 *:9100 running (14m) 87s ago 14m 15.2M - 1.5.0 0da6a335fe13 09f54c65b53c 2024-04-02T04:48:39.939 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi027 smithi027 *:9100 running (13m) 19s ago 13m 14.9M - 1.5.0 0da6a335fe13 e41a657c7443 2024-04-02T04:48:39.939 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 smithi016 running (2m) 87s ago 12m 81.5M 1217M 19.0.0-1979-g6cec793d 78f830190839 421d9e1cc751 2024-04-02T04:48:39.939 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 smithi016 running (2m) 87s ago 12m 87.0M 1217M 19.0.0-1979-g6cec793d 78f830190839 3612a56236a3 2024-04-02T04:48:39.939 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 smithi016 running (92s) 87s ago 11m 17.1M 1217M 19.0.0-1979-g6cec793d 78f830190839 a8e9b8d0303c 2024-04-02T04:48:39.939 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 smithi027 running (57s) 19s ago 11m 100M 2240M 19.0.0-1979-g6cec793d 78f830190839 0dd13b5be8c0 2024-04-02T04:48:39.939 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 smithi027 running (23s) 19s ago 10m 12.7M 2240M 19.0.0-1979-g6cec793d 78f830190839 9025eacf8145 2024-04-02T04:48:39.939 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 smithi027 running (10m) 19s ago 10m 196M 2240M 18.2.2-1059-g4d05f4ca 909dce4fe53d ac6094415529 2024-04-02T04:48:39.939 INFO:teuthology.orchestra.run.smithi016.stdout:prometheus.smithi016 smithi016 *:9095 running (5m) 87s ago 14m 58.5M - 2.43.0 a07b618ecd1d 466f00d73ddd 2024-04-02T04:48:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.935+0000 7f5247393700 1 -- 172.21.15.16:0/1860806811 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f522806dd60 msgr2=0x7f5228070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.935+0000 7f5247393700 1 --2- 172.21.15.16:0/1860806811 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f522806dd60 0x7f5228070220 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5234005950 tx=0x7f523400b410 comp rx=0 tx=0).stop 2024-04-02T04:48:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.935+0000 7f5247393700 1 -- 172.21.15.16:0/1860806811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5240101560 msgr2=0x7f5240196490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.935+0000 7f5247393700 1 --2- 172.21.15.16:0/1860806811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5240101560 0x7f5240196490 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5230006130 tx=0x7f5230003c30 comp rx=0 tx=0).stop 2024-04-02T04:48:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.935+0000 7f5247393700 1 -- 172.21.15.16:0/1860806811 shutdown_connections 2024-04-02T04:48:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.935+0000 7f5247393700 1 --2- 172.21.15.16:0/1860806811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5240101560 0x7f5240196490 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.935+0000 7f5247393700 1 --2- 172.21.15.16:0/1860806811 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f522806dd60 0x7f5228070220 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.935+0000 7f5247393700 1 --2- 172.21.15.16:0/1860806811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5240103e90 0x7f52401969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.935+0000 7f5247393700 1 -- 172.21.15.16:0/1860806811 >> 172.21.15.16:0/1860806811 conn(0x7f52400faf70 msgr2=0x7f52400fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.936+0000 7f5247393700 1 -- 172.21.15.16:0/1860806811 shutdown_connections 2024-04-02T04:48:39.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:39.936+0000 7f5247393700 1 -- 172.21.15.16:0/1860806811 wait complete. 2024-04-02T04:48:40.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.039+0000 7ff06323f700 1 -- 172.21.15.16:0/3897388453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff05c1015f0 msgr2=0x7ff05c1039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:40.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.039+0000 7ff06323f700 1 --2- 172.21.15.16:0/3897388453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff05c1015f0 0x7ff05c1039e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff05400b0a0 tx=0x7ff05400b3b0 comp rx=0 tx=0).stop 2024-04-02T04:48:40.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.039+0000 7ff06323f700 1 -- 172.21.15.16:0/3897388453 shutdown_connections 2024-04-02T04:48:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.039+0000 7ff06323f700 1 --2- 172.21.15.16:0/3897388453 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff05c103f20 0x7ff05c106310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.039+0000 7ff06323f700 1 --2- 172.21.15.16:0/3897388453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff05c1015f0 0x7ff05c1039e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.039+0000 7ff06323f700 1 -- 172.21.15.16:0/3897388453 >> 172.21.15.16:0/3897388453 conn(0x7ff05c0fafc0 msgr2=0x7ff05c0fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.039+0000 7ff06323f700 1 -- 172.21.15.16:0/3897388453 shutdown_connections 2024-04-02T04:48:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.039+0000 7ff06323f700 1 -- 172.21.15.16:0/3897388453 wait complete. 2024-04-02T04:48:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.040+0000 7ff06323f700 1 Processor -- start 2024-04-02T04:48:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.040+0000 7ff06323f700 1 -- start start 2024-04-02T04:48:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.040+0000 7ff06323f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff05c1015f0 0x7ff05c100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.040+0000 7ff06323f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff05c103f20 0x7ff05c0ff5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.040+0000 7ff06323f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05c0ffae0 con 0x7ff05c1015f0 2024-04-02T04:48:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.040+0000 7ff06323f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05c0ffc20 con 0x7ff05c103f20 2024-04-02T04:48:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.040+0000 7ff060fdb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff05c1015f0 0x7ff05c100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.040+0000 7ff060fdb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff05c1015f0 0x7ff05c100f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38208/0 (socket says 172.21.15.16:38208) 2024-04-02T04:48:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.040+0000 7ff060fdb700 1 -- 172.21.15.16:0/3036944070 learned_addr learned my addr 172.21.15.16:0/3036944070 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:48:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.040+0000 7ff05bfff700 1 --2- 172.21.15.16:0/3036944070 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff05c103f20 0x7ff05c0ff5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.041+0000 7ff060fdb700 1 -- 172.21.15.16:0/3036944070 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff05c103f20 msgr2=0x7ff05c0ff5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.041+0000 7ff060fdb700 1 --2- 172.21.15.16:0/3036944070 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff05c103f20 0x7ff05c0ff5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.041+0000 7ff060fdb700 1 -- 172.21.15.16:0/3036944070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff054009d00 con 0x7ff05c1015f0 2024-04-02T04:48:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.041+0000 7ff060fdb700 1 --2- 172.21.15.16:0/3036944070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff05c1015f0 0x7ff05c100f00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff054006130 tx=0x7ff054003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.041+0000 7ff059ffb700 1 -- 172.21.15.16:0/3036944070 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff054021070 con 0x7ff05c1015f0 2024-04-02T04:48:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.041+0000 7ff059ffb700 1 -- 172.21.15.16:0/3036944070 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff05400ce60 con 0x7ff05c1015f0 2024-04-02T04:48:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.041+0000 7ff06323f700 1 -- 172.21.15.16:0/3036944070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff05c0ffea0 con 0x7ff05c1015f0 2024-04-02T04:48:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.041+0000 7ff059ffb700 1 -- 172.21.15.16:0/3036944070 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff05400e040 con 0x7ff05c1015f0 2024-04-02T04:48:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.041+0000 7ff06323f700 1 -- 172.21.15.16:0/3036944070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff05c100410 con 0x7ff05c1015f0 2024-04-02T04:48:40.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.042+0000 7ff06323f700 1 -- 172.21.15.16:0/3036944070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff05c04ea80 con 0x7ff05c1015f0 2024-04-02T04:48:40.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.043+0000 7ff059ffb700 1 -- 172.21.15.16:0/3036944070 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7ff054004250 con 0x7ff05c1015f0 2024-04-02T04:48:40.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.043+0000 7ff059ffb700 1 --2- 172.21.15.16:0/3036944070 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ff04406dd60 0x7ff044070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:40.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.044+0000 7ff059ffb700 1 -- 172.21.15.16:0/3036944070 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6500+0+0 (secure 0 0 0) 0x7ff05408ed00 con 0x7ff05c1015f0 2024-04-02T04:48:40.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.044+0000 7ff05bfff700 1 --2- 172.21.15.16:0/3036944070 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ff04406dd60 0x7ff044070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:40.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.044+0000 7ff05bfff700 1 --2- 172.21.15.16:0/3036944070 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ff04406dd60 0x7ff044070220 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff050005950 tx=0x7ff05000b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:40.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.048+0000 7ff059ffb700 1 -- 172.21.15.16:0/3036944070 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff054092030 con 0x7ff05c1015f0 2024-04-02T04:48:40.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:40 smithi016 ceph-mon[93230]: pgmap v159: 65 pgs: 65 active+clean; 260 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:40.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:40 smithi016 ceph-mon[93230]: from='client.34374 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:48:40.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.340+0000 7ff06323f700 1 -- 172.21.15.16:0/3036944070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff05c100410 con 0x7ff05c1015f0 2024-04-02T04:48:40.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.340+0000 7ff059ffb700 1 -- 172.21.15.16:0/3036944070 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+836 (secure 0 0 0) 0x7ff054017360 con 0x7ff05c1015f0 2024-04-02T04:48:40.345 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:48:40.345 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": { 2024-04-02T04:48:40.345 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:48:40.345 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:48:40.345 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": { 2024-04-02T04:48:40.345 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:48:40.345 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:48:40.345 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": { 2024-04-02T04:48:40.345 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 1, 2024-04-02T04:48:40.345 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 5 2024-04-02T04:48:40.345 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:48:40.345 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-04-02T04:48:40.346 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 4 2024-04-02T04:48:40.346 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:48:40.346 INFO:teuthology.orchestra.run.smithi016.stdout: "overall": { 2024-04-02T04:48:40.346 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 5, 2024-04-02T04:48:40.346 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 9 2024-04-02T04:48:40.346 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-02T04:48:40.346 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:48:40.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.343+0000 7ff06323f700 1 -- 172.21.15.16:0/3036944070 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ff04406dd60 msgr2=0x7ff044070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:40.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.343+0000 7ff06323f700 1 --2- 172.21.15.16:0/3036944070 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ff04406dd60 0x7ff044070220 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff050005950 tx=0x7ff05000b410 comp rx=0 tx=0).stop 2024-04-02T04:48:40.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.343+0000 7ff06323f700 1 -- 172.21.15.16:0/3036944070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff05c1015f0 msgr2=0x7ff05c100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:40.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.343+0000 7ff06323f700 1 --2- 172.21.15.16:0/3036944070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff05c1015f0 0x7ff05c100f00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff054006130 tx=0x7ff054003c30 comp rx=0 tx=0).stop 2024-04-02T04:48:40.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.343+0000 7ff06323f700 1 -- 172.21.15.16:0/3036944070 shutdown_connections 2024-04-02T04:48:40.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.343+0000 7ff06323f700 1 --2- 172.21.15.16:0/3036944070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff05c1015f0 0x7ff05c100f00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:40.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.343+0000 7ff06323f700 1 --2- 172.21.15.16:0/3036944070 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ff04406dd60 0x7ff044070220 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:40.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.343+0000 7ff06323f700 1 --2- 172.21.15.16:0/3036944070 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff05c103f20 0x7ff05c0ff5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:40.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.343+0000 7ff06323f700 1 -- 172.21.15.16:0/3036944070 >> 172.21.15.16:0/3036944070 conn(0x7ff05c0fafc0 msgr2=0x7ff05c0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:40.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.343+0000 7ff06323f700 1 -- 172.21.15.16:0/3036944070 shutdown_connections 2024-04-02T04:48:40.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.343+0000 7ff06323f700 1 -- 172.21.15.16:0/3036944070 wait complete. 2024-04-02T04:48:40.448 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:40 smithi027 ceph-mon[83036]: pgmap v159: 65 pgs: 65 active+clean; 260 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:40.448 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:40 smithi027 ceph-mon[83036]: from='client.34374 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:48:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.448+0000 7f9c0fafa700 1 -- 172.21.15.16:0/4287775841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c08102130 msgr2=0x7f9c08102550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.448+0000 7f9c0fafa700 1 --2- 172.21.15.16:0/4287775841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c08102130 0x7f9c08102550 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9c04009910 tx=0x7f9c04009c20 comp rx=0 tx=0).stop 2024-04-02T04:48:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.448+0000 7f9c0fafa700 1 -- 172.21.15.16:0/4287775841 shutdown_connections 2024-04-02T04:48:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.448+0000 7f9c0fafa700 1 --2- 172.21.15.16:0/4287775841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c080febb0 0x7f9c080ff010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.448+0000 7f9c0fafa700 1 --2- 172.21.15.16:0/4287775841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c08102130 0x7f9c08102550 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.448+0000 7f9c0fafa700 1 -- 172.21.15.16:0/4287775841 >> 172.21.15.16:0/4287775841 conn(0x7f9c080fa730 msgr2=0x7f9c080fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.448+0000 7f9c0fafa700 1 -- 172.21.15.16:0/4287775841 shutdown_connections 2024-04-02T04:48:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.448+0000 7f9c0fafa700 1 -- 172.21.15.16:0/4287775841 wait complete. 2024-04-02T04:48:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.449+0000 7f9c0fafa700 1 Processor -- start 2024-04-02T04:48:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.449+0000 7f9c0fafa700 1 -- start start 2024-04-02T04:48:40.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.449+0000 7f9c0fafa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c080febb0 0x7f9c081017e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:40.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.449+0000 7f9c0fafa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c08102130 0x7f9c080ffe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:40.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.449+0000 7f9c0fafa700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c081003c0 con 0x7f9c080febb0 2024-04-02T04:48:40.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.449+0000 7f9c0fafa700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c08100500 con 0x7f9c08102130 2024-04-02T04:48:40.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.449+0000 7f9c0d896700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c080febb0 0x7f9c081017e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:40.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.449+0000 7f9c0d896700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c080febb0 0x7f9c081017e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38230/0 (socket says 172.21.15.16:38230) 2024-04-02T04:48:40.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.449+0000 7f9c0d896700 1 -- 172.21.15.16:0/1084059902 learned_addr learned my addr 172.21.15.16:0/1084059902 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:48:40.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.449+0000 7f9c0d095700 1 --2- 172.21.15.16:0/1084059902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c08102130 0x7f9c080ffe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:40.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.450+0000 7f9c0d896700 1 -- 172.21.15.16:0/1084059902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c08102130 msgr2=0x7f9c080ffe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:40.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.450+0000 7f9c0d896700 1 --2- 172.21.15.16:0/1084059902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c08102130 0x7f9c080ffe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:40.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.450+0000 7f9c0d896700 1 -- 172.21.15.16:0/1084059902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c040095f0 con 0x7f9c080febb0 2024-04-02T04:48:40.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.450+0000 7f9c0d896700 1 --2- 172.21.15.16:0/1084059902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c080febb0 0x7f9c081017e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9c040092e0 tx=0x7f9c0400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:40.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.450+0000 7f9bfeffd700 1 -- 172.21.15.16:0/1084059902 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c0401d070 con 0x7f9c080febb0 2024-04-02T04:48:40.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.450+0000 7f9bfeffd700 1 -- 172.21.15.16:0/1084059902 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9c04005dc0 con 0x7f9c080febb0 2024-04-02T04:48:40.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.450+0000 7f9bfeffd700 1 -- 172.21.15.16:0/1084059902 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c04017400 con 0x7f9c080febb0 2024-04-02T04:48:40.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.450+0000 7f9c0fafa700 1 -- 172.21.15.16:0/1084059902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c08100780 con 0x7f9c080febb0 2024-04-02T04:48:40.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.450+0000 7f9c0fafa700 1 -- 172.21.15.16:0/1084059902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c08100cf0 con 0x7f9c080febb0 2024-04-02T04:48:40.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.451+0000 7f9c0fafa700 1 -- 172.21.15.16:0/1084059902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c08190690 con 0x7f9c080febb0 2024-04-02T04:48:40.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.452+0000 7f9bfeffd700 1 -- 172.21.15.16:0/1084059902 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f9c0400f6f0 con 0x7f9c080febb0 2024-04-02T04:48:40.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.452+0000 7f9bfeffd700 1 --2- 172.21.15.16:0/1084059902 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f9bf406dd60 0x7f9bf4070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:40.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.452+0000 7f9bfeffd700 1 -- 172.21.15.16:0/1084059902 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f9c0408ed50 con 0x7f9c080febb0 2024-04-02T04:48:40.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.452+0000 7f9c0d095700 1 --2- 172.21.15.16:0/1084059902 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f9bf406dd60 0x7f9bf4070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:40.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.453+0000 7f9c0d095700 1 --2- 172.21.15.16:0/1084059902 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f9bf406dd60 0x7f9bf4070220 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8005950 tx=0x7f9bf800b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:40.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.457+0000 7f9bfeffd700 1 -- 172.21.15.16:0/1084059902 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9c0405be10 con 0x7f9c080febb0 2024-04-02T04:48:40.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.704+0000 7f9c0fafa700 1 -- 172.21.15.16:0/1084059902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9c08100cf0 con 0x7f9c080febb0 2024-04-02T04:48:40.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.704+0000 7f9bfeffd700 1 -- 172.21.15.16:0/1084059902 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1875 (secure 0 0 0) 0x7f9c08100cf0 con 0x7f9c080febb0 2024-04-02T04:48:40.709 INFO:teuthology.orchestra.run.smithi016.stdout:e15 2024-04-02T04:48:40.709 INFO:teuthology.orchestra.run.smithi016.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:48:40.709 INFO:teuthology.orchestra.run.smithi016.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:48:40.709 INFO:teuthology.orchestra.run.smithi016.stdout:legacy client fscid: 1 2024-04-02T04:48:40.709 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:48:40.709 INFO:teuthology.orchestra.run.smithi016.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:48:40.709 INFO:teuthology.orchestra.run.smithi016.stdout:fs_name cephfs 2024-04-02T04:48:40.709 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 15 2024-04-02T04:48:40.709 INFO:teuthology.orchestra.run.smithi016.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:48:40.709 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-04-02T04:39:30.740168+0000 2024-04-02T04:48:40.710 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-04-02T04:39:49.875319+0000 2024-04-02T04:48:40.710 INFO:teuthology.orchestra.run.smithi016.stdout:tableserver 0 2024-04-02T04:48:40.710 INFO:teuthology.orchestra.run.smithi016.stdout:root 0 2024-04-02T04:48:40.710 INFO:teuthology.orchestra.run.smithi016.stdout:session_timeout 60 2024-04-02T04:48:40.710 INFO:teuthology.orchestra.run.smithi016.stdout:session_autoclose 300 2024-04-02T04:48:40.710 INFO:teuthology.orchestra.run.smithi016.stdout:max_file_size 1099511627776 2024-04-02T04:48:40.710 INFO:teuthology.orchestra.run.smithi016.stdout:max_xattr_size 65536 2024-04-02T04:48:40.710 INFO:teuthology.orchestra.run.smithi016.stdout:required_client_features {} 2024-04-02T04:48:40.710 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure 0 2024-04-02T04:48:40.710 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure_osd_epoch 46 2024-04-02T04:48:40.710 INFO:teuthology.orchestra.run.smithi016.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:48:40.710 INFO:teuthology.orchestra.run.smithi016.stdout:max_mds 1 2024-04-02T04:48:40.710 INFO:teuthology.orchestra.run.smithi016.stdout:in 0 2024-04-02T04:48:40.710 INFO:teuthology.orchestra.run.smithi016.stdout:up {0=24285} 2024-04-02T04:48:40.711 INFO:teuthology.orchestra.run.smithi016.stdout:failed 2024-04-02T04:48:40.711 INFO:teuthology.orchestra.run.smithi016.stdout:damaged 2024-04-02T04:48:40.711 INFO:teuthology.orchestra.run.smithi016.stdout:stopped 2024-04-02T04:48:40.711 INFO:teuthology.orchestra.run.smithi016.stdout:data_pools [3] 2024-04-02T04:48:40.711 INFO:teuthology.orchestra.run.smithi016.stdout:metadata_pool 2 2024-04-02T04:48:40.711 INFO:teuthology.orchestra.run.smithi016.stdout:inline_data enabled 2024-04-02T04:48:40.711 INFO:teuthology.orchestra.run.smithi016.stdout:balancer 2024-04-02T04:48:40.711 INFO:teuthology.orchestra.run.smithi016.stdout:bal_rank_mask -1 2024-04-02T04:48:40.711 INFO:teuthology.orchestra.run.smithi016.stdout:standby_count_wanted 1 2024-04-02T04:48:40.711 INFO:teuthology.orchestra.run.smithi016.stdout:qdb_cluster leader: 0 members: 2024-04-02T04:48:40.711 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.zwrfoo{0:24285} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.27:6824/1461455976,v1:172.21.15.27:6825/1461455976] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:48:40.711 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:48:40.711 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:48:40.711 INFO:teuthology.orchestra.run.smithi016.stdout:Standby daemons: 2024-04-02T04:48:40.711 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:48:40.711 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.fyzrnm{-1:14488} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.16:6828/2578719859,v1:172.21.15.16:6829/2578719859] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:48:40.712 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.uhgkgg{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.16:6826/601263579,v1:172.21.15.16:6827/601263579] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:48:40.712 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.sdublg{-1:24295} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.27:6826/4177192740,v1:172.21.15.27:6827/4177192740] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:48:40.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.708+0000 7f9c0fafa700 1 -- 172.21.15.16:0/1084059902 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f9bf406dd60 msgr2=0x7f9bf4070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:40.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.708+0000 7f9c0fafa700 1 --2- 172.21.15.16:0/1084059902 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f9bf406dd60 0x7f9bf4070220 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8005950 tx=0x7f9bf800b410 comp rx=0 tx=0).stop 2024-04-02T04:48:40.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.708+0000 7f9c0fafa700 1 -- 172.21.15.16:0/1084059902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c080febb0 msgr2=0x7f9c081017e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:40.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.708+0000 7f9c0fafa700 1 --2- 172.21.15.16:0/1084059902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c080febb0 0x7f9c081017e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9c040092e0 tx=0x7f9c0400bd70 comp rx=0 tx=0).stop 2024-04-02T04:48:40.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.708+0000 7f9c0fafa700 1 -- 172.21.15.16:0/1084059902 shutdown_connections 2024-04-02T04:48:40.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.708+0000 7f9c0fafa700 1 --2- 172.21.15.16:0/1084059902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c080febb0 0x7f9c081017e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:40.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.708+0000 7f9c0fafa700 1 --2- 172.21.15.16:0/1084059902 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f9bf406dd60 0x7f9bf4070220 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:40.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.708+0000 7f9c0fafa700 1 --2- 172.21.15.16:0/1084059902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c08102130 0x7f9c080ffe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:40.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.708+0000 7f9c0fafa700 1 -- 172.21.15.16:0/1084059902 >> 172.21.15.16:0/1084059902 conn(0x7f9c080fa730 msgr2=0x7f9c081054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:40.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.708+0000 7f9c0fafa700 1 -- 172.21.15.16:0/1084059902 shutdown_connections 2024-04-02T04:48:40.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.708+0000 7f9c0fafa700 1 -- 172.21.15.16:0/1084059902 wait complete. 2024-04-02T04:48:40.713 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 15 2024-04-02T04:48:40.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.814+0000 7fd86beae700 1 -- 172.21.15.16:0/1323221111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8640ff9d0 msgr2=0x7fd8640ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:40.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.814+0000 7fd86beae700 1 --2- 172.21.15.16:0/1323221111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8640ff9d0 0x7fd8640ffdf0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd854009910 tx=0x7fd854009c20 comp rx=0 tx=0).stop 2024-04-02T04:48:40.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.814+0000 7fd86beae700 1 -- 172.21.15.16:0/1323221111 shutdown_connections 2024-04-02T04:48:40.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.814+0000 7fd86beae700 1 --2- 172.21.15.16:0/1323221111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd864100bd0 0x7fd864101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:40.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.814+0000 7fd86beae700 1 --2- 172.21.15.16:0/1323221111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8640ff9d0 0x7fd8640ffdf0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:40.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.814+0000 7fd86beae700 1 -- 172.21.15.16:0/1323221111 >> 172.21.15.16:0/1323221111 conn(0x7fd8640faf70 msgr2=0x7fd8640fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:40.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.814+0000 7fd86beae700 1 -- 172.21.15.16:0/1323221111 shutdown_connections 2024-04-02T04:48:40.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.814+0000 7fd86beae700 1 -- 172.21.15.16:0/1323221111 wait complete. 2024-04-02T04:48:40.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.815+0000 7fd86beae700 1 Processor -- start 2024-04-02T04:48:40.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.815+0000 7fd86beae700 1 -- start start 2024-04-02T04:48:40.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.815+0000 7fd86beae700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8640ff9d0 0x7fd864068f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:40.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.815+0000 7fd86beae700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd864100bd0 0x7fd86419b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:40.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.815+0000 7fd86beae700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86419b7e0 con 0x7fd8640ff9d0 2024-04-02T04:48:40.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.815+0000 7fd86beae700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86419b920 con 0x7fd864100bd0 2024-04-02T04:48:40.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.815+0000 7fd869c4a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8640ff9d0 0x7fd864068f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:40.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.815+0000 7fd869c4a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8640ff9d0 0x7fd864068f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38256/0 (socket says 172.21.15.16:38256) 2024-04-02T04:48:40.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.815+0000 7fd869c4a700 1 -- 172.21.15.16:0/1338285126 learned_addr learned my addr 172.21.15.16:0/1338285126 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:48:40.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.815+0000 7fd869449700 1 --2- 172.21.15.16:0/1338285126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd864100bd0 0x7fd86419b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:40.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.815+0000 7fd869c4a700 1 -- 172.21.15.16:0/1338285126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd864100bd0 msgr2=0x7fd86419b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:40.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.815+0000 7fd869c4a700 1 --2- 172.21.15.16:0/1338285126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd864100bd0 0x7fd86419b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:40.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.815+0000 7fd869c4a700 1 -- 172.21.15.16:0/1338285126 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8540095f0 con 0x7fd8640ff9d0 2024-04-02T04:48:40.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.815+0000 7fd869c4a700 1 --2- 172.21.15.16:0/1338285126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8640ff9d0 0x7fd864068f40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd854005f00 tx=0x7fd85400b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:40.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.816+0000 7fd85affd700 1 -- 172.21.15.16:0/1338285126 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd85401d070 con 0x7fd8640ff9d0 2024-04-02T04:48:40.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.816+0000 7fd85affd700 1 -- 172.21.15.16:0/1338285126 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd8540052a0 con 0x7fd8640ff9d0 2024-04-02T04:48:40.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.816+0000 7fd86beae700 1 -- 172.21.15.16:0/1338285126 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd86419bb70 con 0x7fd8640ff9d0 2024-04-02T04:48:40.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.816+0000 7fd85affd700 1 -- 172.21.15.16:0/1338285126 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd854017400 con 0x7fd8640ff9d0 2024-04-02T04:48:40.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.816+0000 7fd86beae700 1 -- 172.21.15.16:0/1338285126 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd86419c060 con 0x7fd8640ff9d0 2024-04-02T04:48:40.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.817+0000 7fd86beae700 1 -- 172.21.15.16:0/1338285126 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd86418f550 con 0x7fd8640ff9d0 2024-04-02T04:48:40.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.817+0000 7fd85affd700 1 -- 172.21.15.16:0/1338285126 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fd854003680 con 0x7fd8640ff9d0 2024-04-02T04:48:40.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.818+0000 7fd85affd700 1 --2- 172.21.15.16:0/1338285126 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd85006dd60 0x7fd850070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:40.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.818+0000 7fd85affd700 1 -- 172.21.15.16:0/1338285126 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fd85408ef00 con 0x7fd8640ff9d0 2024-04-02T04:48:40.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.818+0000 7fd869449700 1 --2- 172.21.15.16:0/1338285126 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd85006dd60 0x7fd850070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:40.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.818+0000 7fd869449700 1 --2- 172.21.15.16:0/1338285126 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd85006dd60 0x7fd850070220 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd860009c60 tx=0x7fd860009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:40.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:40.823+0000 7fd85affd700 1 -- 172.21.15.16:0/1338285126 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd85405bfc0 con 0x7fd8640ff9d0 2024-04-02T04:48:41.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.044+0000 7fd86beae700 1 -- 172.21.15.16:0/1338285126 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd86402ce10 con 0x7fd85006dd60 2024-04-02T04:48:41.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.046+0000 7fd85affd700 1 -- 172.21.15.16:0/1338285126 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fd86402ce10 con 0x7fd85006dd60 2024-04-02T04:48:41.050 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:48:41.050 INFO:teuthology.orchestra.run.smithi016.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:48:41.050 INFO:teuthology.orchestra.run.smithi016.stdout: "in_progress": true, 2024-04-02T04:48:41.050 INFO:teuthology.orchestra.run.smithi016.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-02T04:48:41.050 INFO:teuthology.orchestra.run.smithi016.stdout: "services_complete": [ 2024-04-02T04:48:41.050 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr", 2024-04-02T04:48:41.050 INFO:teuthology.orchestra.run.smithi016.stdout: "mon", 2024-04-02T04:48:41.050 INFO:teuthology.orchestra.run.smithi016.stdout: "crash" 2024-04-02T04:48:41.050 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-04-02T04:48:41.051 INFO:teuthology.orchestra.run.smithi016.stdout: "progress": "11/23 daemons upgraded", 2024-04-02T04:48:41.051 INFO:teuthology.orchestra.run.smithi016.stdout: "message": "Currently upgrading osd daemons", 2024-04-02T04:48:41.051 INFO:teuthology.orchestra.run.smithi016.stdout: "is_paused": false 2024-04-02T04:48:41.051 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:48:41.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.048+0000 7fd86beae700 1 -- 172.21.15.16:0/1338285126 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd85006dd60 msgr2=0x7fd850070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:41.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.048+0000 7fd86beae700 1 --2- 172.21.15.16:0/1338285126 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd85006dd60 0x7fd850070220 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd860009c60 tx=0x7fd860009260 comp rx=0 tx=0).stop 2024-04-02T04:48:41.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.048+0000 7fd86beae700 1 -- 172.21.15.16:0/1338285126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8640ff9d0 msgr2=0x7fd864068f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:41.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.048+0000 7fd86beae700 1 --2- 172.21.15.16:0/1338285126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8640ff9d0 0x7fd864068f40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd854005f00 tx=0x7fd85400b5e0 comp rx=0 tx=0).stop 2024-04-02T04:48:41.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.048+0000 7fd86beae700 1 -- 172.21.15.16:0/1338285126 shutdown_connections 2024-04-02T04:48:41.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.048+0000 7fd86beae700 1 --2- 172.21.15.16:0/1338285126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8640ff9d0 0x7fd864068f40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:41.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.048+0000 7fd86beae700 1 --2- 172.21.15.16:0/1338285126 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd85006dd60 0x7fd850070220 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:41.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.048+0000 7fd86beae700 1 --2- 172.21.15.16:0/1338285126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd864100bd0 0x7fd86419b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:41.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.048+0000 7fd86beae700 1 -- 172.21.15.16:0/1338285126 >> 172.21.15.16:0/1338285126 conn(0x7fd8640faf70 msgr2=0x7fd864104220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:41.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.048+0000 7fd86beae700 1 -- 172.21.15.16:0/1338285126 shutdown_connections 2024-04-02T04:48:41.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.048+0000 7fd86beae700 1 -- 172.21.15.16:0/1338285126 wait complete. 2024-04-02T04:48:41.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.156+0000 7fca0d9a1700 1 -- 172.21.15.16:0/4015132901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca080ffaf0 msgr2=0x7fca080fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:41.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.156+0000 7fca0d9a1700 1 --2- 172.21.15.16:0/4015132901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca080ffaf0 0x7fca080fff70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0009910 tx=0x7fc9f0009c20 comp rx=0 tx=0).stop 2024-04-02T04:48:41.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.156+0000 7fca0d9a1700 1 -- 172.21.15.16:0/4015132901 shutdown_connections 2024-04-02T04:48:41.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.156+0000 7fca0d9a1700 1 --2- 172.21.15.16:0/4015132901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca080ffaf0 0x7fca080fff70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:41.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.156+0000 7fca0d9a1700 1 --2- 172.21.15.16:0/4015132901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca08100e40 0x7fca08101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:41.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.156+0000 7fca0d9a1700 1 -- 172.21.15.16:0/4015132901 >> 172.21.15.16:0/4015132901 conn(0x7fca080fa580 msgr2=0x7fca080fc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:41.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.156+0000 7fca0d9a1700 1 -- 172.21.15.16:0/4015132901 shutdown_connections 2024-04-02T04:48:41.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.156+0000 7fca0d9a1700 1 -- 172.21.15.16:0/4015132901 wait complete. 2024-04-02T04:48:41.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.157+0000 7fca0d9a1700 1 Processor -- start 2024-04-02T04:48:41.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.157+0000 7fca0d9a1700 1 -- start start 2024-04-02T04:48:41.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.157+0000 7fca0d9a1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca080ffaf0 0x7fca081964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:41.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.157+0000 7fca0d9a1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca08100e40 0x7fca08196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:41.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.157+0000 7fca0d9a1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca08197030 con 0x7fca080ffaf0 2024-04-02T04:48:41.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.157+0000 7fca0d9a1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca08197170 con 0x7fca08100e40 2024-04-02T04:48:41.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.157+0000 7fca0c99f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca080ffaf0 0x7fca081964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:41.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.157+0000 7fca0c99f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca080ffaf0 0x7fca081964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38270/0 (socket says 172.21.15.16:38270) 2024-04-02T04:48:41.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.157+0000 7fca0c99f700 1 -- 172.21.15.16:0/106810280 learned_addr learned my addr 172.21.15.16:0/106810280 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:48:41.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.157+0000 7fca07fff700 1 --2- 172.21.15.16:0/106810280 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca08100e40 0x7fca08196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:41.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.157+0000 7fca0c99f700 1 -- 172.21.15.16:0/106810280 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca08100e40 msgr2=0x7fca08196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:41.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.157+0000 7fca0c99f700 1 --2- 172.21.15.16:0/106810280 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca08100e40 0x7fca08196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:41.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.157+0000 7fca0c99f700 1 -- 172.21.15.16:0/106810280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f00095f0 con 0x7fca080ffaf0 2024-04-02T04:48:41.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.158+0000 7fca0c99f700 1 --2- 172.21.15.16:0/106810280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca080ffaf0 0x7fca081964d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc9f800b480 tx=0x7fc9f800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:41.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.159+0000 7fca05ffb700 1 -- 172.21.15.16:0/106810280 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9f8011800 con 0x7fca080ffaf0 2024-04-02T04:48:41.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.159+0000 7fca05ffb700 1 -- 172.21.15.16:0/106810280 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc9f8011e20 con 0x7fca080ffaf0 2024-04-02T04:48:41.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.159+0000 7fca0d9a1700 1 -- 172.21.15.16:0/106810280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca0819bbd0 con 0x7fca080ffaf0 2024-04-02T04:48:41.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.159+0000 7fca05ffb700 1 -- 172.21.15.16:0/106810280 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9f8010ea0 con 0x7fca080ffaf0 2024-04-02T04:48:41.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.159+0000 7fca0d9a1700 1 -- 172.21.15.16:0/106810280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca0819c120 con 0x7fca080ffaf0 2024-04-02T04:48:41.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.160+0000 7fca0d9a1700 1 -- 172.21.15.16:0/106810280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca0804f2d0 con 0x7fca080ffaf0 2024-04-02T04:48:41.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.160+0000 7fca05ffb700 1 -- 172.21.15.16:0/106810280 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fc9f8011960 con 0x7fca080ffaf0 2024-04-02T04:48:41.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.161+0000 7fca05ffb700 1 --2- 172.21.15.16:0/106810280 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc9f406dd60 0x7fc9f4070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:48:41.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.161+0000 7fca05ffb700 1 -- 172.21.15.16:0/106810280 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fc9f808d670 con 0x7fca080ffaf0 2024-04-02T04:48:41.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.161+0000 7fca07fff700 1 --2- 172.21.15.16:0/106810280 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc9f406dd60 0x7fc9f4070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:48:41.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.161+0000 7fca07fff700 1 --2- 172.21.15.16:0/106810280 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc9f406dd60 0x7fc9f4070220 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0000c00 tx=0x7fc9f0005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:48:41.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.165+0000 7fca05ffb700 1 -- 172.21.15.16:0/106810280 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc9f805a690 con 0x7fca080ffaf0 2024-04-02T04:48:41.257 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:41 smithi027 ceph-mon[83036]: from='client.34378 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:48:41.257 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:41 smithi027 ceph-mon[83036]: from='client.34382 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:48:41.257 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:41 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/3036944070' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:48:41.257 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:41 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-02T04:48:41.257 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:41 smithi027 ceph-mon[83036]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-02T04:48:41.258 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:41 smithi027 ceph-mon[83036]: Upgrade: osd.5 is safe to restart 2024-04-02T04:48:41.258 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:41 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/1084059902' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:48:41.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:41 smithi016 ceph-mon[93230]: from='client.34378 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:48:41.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:41 smithi016 ceph-mon[93230]: from='client.34382 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:48:41.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:41 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/3036944070' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:48:41.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:41 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-02T04:48:41.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:41 smithi016 ceph-mon[93230]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-02T04:48:41.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:41 smithi016 ceph-mon[93230]: Upgrade: osd.5 is safe to restart 2024-04-02T04:48:41.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:41 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/1084059902' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:48:41.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.453+0000 7fca0d9a1700 1 -- 172.21.15.16:0/106810280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fca08108680 con 0x7fca080ffaf0 2024-04-02T04:48:41.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.454+0000 7fca05ffb700 1 -- 172.21.15.16:0/106810280 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fc9f805a220 con 0x7fca080ffaf0 2024-04-02T04:48:41.457 INFO:teuthology.orchestra.run.smithi016.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-02T04:48:41.457 INFO:teuthology.orchestra.run.smithi016.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-02T04:48:41.457 INFO:teuthology.orchestra.run.smithi016.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-02T04:48:41.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.456+0000 7fca0d9a1700 1 -- 172.21.15.16:0/106810280 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc9f406dd60 msgr2=0x7fc9f4070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:41.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.456+0000 7fca0d9a1700 1 --2- 172.21.15.16:0/106810280 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc9f406dd60 0x7fc9f4070220 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0000c00 tx=0x7fc9f0005960 comp rx=0 tx=0).stop 2024-04-02T04:48:41.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.456+0000 7fca0d9a1700 1 -- 172.21.15.16:0/106810280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca080ffaf0 msgr2=0x7fca081964d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:48:41.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.456+0000 7fca0d9a1700 1 --2- 172.21.15.16:0/106810280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca080ffaf0 0x7fca081964d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc9f800b480 tx=0x7fc9f800b790 comp rx=0 tx=0).stop 2024-04-02T04:48:41.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.456+0000 7fca0d9a1700 1 -- 172.21.15.16:0/106810280 shutdown_connections 2024-04-02T04:48:41.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.456+0000 7fca0d9a1700 1 --2- 172.21.15.16:0/106810280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca080ffaf0 0x7fca081964d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:41.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.456+0000 7fca0d9a1700 1 --2- 172.21.15.16:0/106810280 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc9f406dd60 0x7fc9f4070220 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:41.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.456+0000 7fca0d9a1700 1 --2- 172.21.15.16:0/106810280 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca08100e40 0x7fca08196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:48:41.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.456+0000 7fca0d9a1700 1 -- 172.21.15.16:0/106810280 >> 172.21.15.16:0/106810280 conn(0x7fca080fa580 msgr2=0x7fca080fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:48:41.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.456+0000 7fca0d9a1700 1 -- 172.21.15.16:0/106810280 shutdown_connections 2024-04-02T04:48:41.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:48:41.456+0000 7fca0d9a1700 1 -- 172.21.15.16:0/106810280 wait complete. 2024-04-02T04:48:42.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:42 smithi027 ceph-mon[83036]: from='client.34394 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:48:42.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:42 smithi027 ceph-mon[83036]: pgmap v160: 65 pgs: 65 active+clean; 260 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:42.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:42 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/106810280' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:48:42.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:42 smithi016 ceph-mon[93230]: from='client.34394 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:48:42.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:42 smithi016 ceph-mon[93230]: pgmap v160: 65 pgs: 65 active+clean; 260 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:42.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:42 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/106810280' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:48:43.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:43 smithi016 ceph-mon[93230]: Upgrade: Updating osd.5 2024-04-02T04:48:43.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:43 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:43.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:43 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-02T04:48:43.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:43 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:48:43.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:43 smithi016 ceph-mon[93230]: Deploying daemon osd.5 on smithi027 2024-04-02T04:48:43.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:43 smithi027 ceph-mon[83036]: Upgrade: Updating osd.5 2024-04-02T04:48:43.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:43 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:43.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:43 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-02T04:48:43.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:43 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:48:43.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:43 smithi027 ceph-mon[83036]: Deploying daemon osd.5 on smithi027 2024-04-02T04:48:44.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:44 smithi016 ceph-mon[93230]: pgmap v161: 65 pgs: 65 active+clean; 260 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:44.645 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:44 smithi027 ceph-mon[83036]: pgmap v161: 65 pgs: 65 active+clean; 260 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:46.146 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:45 smithi027 systemd[1]: Stopping Ceph osd.5 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4... 2024-04-02T04:48:46.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:46 smithi016 ceph-mon[93230]: pgmap v162: 65 pgs: 65 active+clean; 260 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:46.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:46 smithi016 ceph-mon[93230]: osd.5 marked itself down and dead 2024-04-02T04:48:46.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:46 smithi027 ceph-mon[83036]: pgmap v162: 65 pgs: 65 active+clean; 260 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:46.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:46 smithi027 ceph-mon[83036]: osd.5 marked itself down and dead 2024-04-02T04:48:46.646 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:46 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[56824]: 2024-04-02T04:48:46.158+0000 7fa42a536700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T04:48:46.647 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:46 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[56824]: 2024-04-02T04:48:46.158+0000 7fa42a536700 -1 osd.5 74 *** Got signal Terminated *** 2024-04-02T04:48:46.647 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:46 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[56824]: 2024-04-02T04:48:46.158+0000 7fa42a536700 -1 osd.5 74 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T04:48:46.987 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:46 smithi027 podman[102648]: 2024-04-02 04:48:46.709804684 +0000 UTC m=+0.656829628 container died ac6094415529b3021d36f353b5456b9d1a315bd3de2746ca893ff1022279ce2f (image=quay.ceph.io/ceph-ci/ceph@sha256:397d0c1d908c002c558906e3ea05982bc336d403ee4d900400cc3bdd9243aa2e, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5, GIT_CLEAN=True, RELEASE=reef-4d05f4c, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-02T04:48:47.289 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:47 smithi027 podman[102648]: 2024-04-02 04:48:47.083279604 +0000 UTC m=+1.030304548 container cleanup ac6094415529b3021d36f353b5456b9d1a315bd3de2746ca893ff1022279ce2f (image=quay.ceph.io/ceph-ci/ceph@sha256:397d0c1d908c002c558906e3ea05982bc336d403ee4d900400cc3bdd9243aa2e, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, RELEASE=reef-4d05f4c, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-02T04:48:47.290 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:47 smithi027 bash[102648]: ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5 2024-04-02T04:48:47.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:47 smithi016 ceph-mon[93230]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-02T04:48:47.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:47 smithi016 ceph-mon[93230]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-04-02T04:48:47.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:47 smithi016 ceph-mon[93230]: osdmap e75: 6 total, 5 up, 6 in 2024-04-02T04:48:47.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:47 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:47.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:47 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:48:47.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:47 smithi016 ceph-mon[93230]: osdmap e76: 6 total, 5 up, 6 in 2024-04-02T04:48:47.646 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:47 smithi027 podman[102667]: 2024-04-02 04:48:47.376352559 +0000 UTC m=+0.660979164 container remove ac6094415529b3021d36f353b5456b9d1a315bd3de2746ca893ff1022279ce2f (image=quay.ceph.io/ceph-ci/ceph@sha256:397d0c1d908c002c558906e3ea05982bc336d403ee4d900400cc3bdd9243aa2e, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5, org.label-schema.build-date=20240326, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=reef-4d05f4c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, ceph=True) 2024-04-02T04:48:47.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:47 smithi027 ceph-mon[83036]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-02T04:48:47.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:47 smithi027 ceph-mon[83036]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-04-02T04:48:47.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:47 smithi027 ceph-mon[83036]: osdmap e75: 6 total, 5 up, 6 in 2024-04-02T04:48:47.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:47 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:47.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:47 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:48:47.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:47 smithi027 ceph-mon[83036]: osdmap e76: 6 total, 5 up, 6 in 2024-04-02T04:48:48.016 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:47 smithi027 podman[102756]: 2024-04-02 04:48:47.611612828 +0000 UTC m=+0.018791023 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:48:48.016 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:47 smithi027 podman[102756]: 2024-04-02 04:48:47.752539621 +0000 UTC m=+0.159717798 container create 68ba5de1fe76d0b527df9efbcc4fe06398707b969f9bc013c2f93ff23ea08f9e (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5-deactivate, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=squid-6cec793, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-02T04:48:48.291 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:48 smithi027 podman[102756]: 2024-04-02 04:48:48.013883661 +0000 UTC m=+0.421061848 container init 68ba5de1fe76d0b527df9efbcc4fe06398707b969f9bc013c2f93ff23ea08f9e (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=squid-6cec793, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True) 2024-04-02T04:48:48.291 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:48 smithi027 podman[102756]: 2024-04-02 04:48:48.017420801 +0000 UTC m=+0.424598983 container start 68ba5de1fe76d0b527df9efbcc4fe06398707b969f9bc013c2f93ff23ea08f9e (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, RELEASE=squid-6cec793, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-02T04:48:48.291 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:48 smithi027 podman[102756]: 2024-04-02 04:48:48.113451771 +0000 UTC m=+0.520629971 container attach 68ba5de1fe76d0b527df9efbcc4fe06398707b969f9bc013c2f93ff23ea08f9e (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-6cec793, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T04:48:48.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:48 smithi016 ceph-mon[93230]: pgmap v164: 65 pgs: 3 peering, 11 stale+active+clean, 51 active+clean; 260 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:48.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:48 smithi016 ceph-mon[93230]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-02T04:48:48.595 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:48 smithi027 ceph-mon[83036]: pgmap v164: 65 pgs: 3 peering, 11 stale+active+clean, 51 active+clean; 260 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-02T04:48:48.596 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:48 smithi027 ceph-mon[83036]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-02T04:48:48.596 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:48 smithi027 podman[102756]: 2024-04-02 04:48:48.323465731 +0000 UTC m=+0.730643899 container died 68ba5de1fe76d0b527df9efbcc4fe06398707b969f9bc013c2f93ff23ea08f9e (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-6cec793, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, ceph=True) 2024-04-02T04:48:49.146 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:48 smithi027 podman[102782]: 2024-04-02 04:48:48.867469432 +0000 UTC m=+0.534753630 container remove 68ba5de1fe76d0b527df9efbcc4fe06398707b969f9bc013c2f93ff23ea08f9e (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-6cec793, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T04:48:49.146 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:48 smithi027 systemd[1]: ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@osd.5.service: Deactivated successfully. 2024-04-02T04:48:49.146 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:48 smithi027 systemd[1]: Stopped Ceph osd.5 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4. 2024-04-02T04:48:49.146 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:48 smithi027 systemd[1]: ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@osd.5.service: Consumed 54.214s CPU time. 2024-04-02T04:48:49.147 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:49 smithi027 systemd[1]: Starting Ceph osd.5 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4... 2024-04-02T04:48:49.582 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:49 smithi027 ceph-mon[83036]: Health check failed: Degraded data redundancy: 13/267 objects degraded (4.869%), 5 pgs degraded (PG_DEGRADED) 2024-04-02T04:48:49.582 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:49 smithi027 podman[102888]: 2024-04-02 04:48:49.20991792 +0000 UTC m=+0.018255181 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:48:49.582 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:49 smithi027 podman[102888]: 2024-04-02 04:48:49.343581466 +0000 UTC m=+0.151918691 container create 7f592f5e80bd0ce1ee2ecc7a2062d77f061bad5cff2aba6a9aed48d2a58a0d2c (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5-activate, RELEASE=squid-6cec793, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-02T04:48:49.582 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:49 smithi027 podman[102888]: 2024-04-02 04:48:49.579647161 +0000 UTC m=+0.387984396 container init 7f592f5e80bd0ce1ee2ecc7a2062d77f061bad5cff2aba6a9aed48d2a58a0d2c (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5-activate, RELEASE=squid-6cec793, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-02T04:48:49.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:49 smithi016 ceph-mon[93230]: Health check failed: Degraded data redundancy: 13/267 objects degraded (4.869%), 5 pgs degraded (PG_DEGRADED) 2024-04-02T04:48:49.896 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:49 smithi027 podman[102888]: 2024-04-02 04:48:49.583463189 +0000 UTC m=+0.391800425 container start 7f592f5e80bd0ce1ee2ecc7a2062d77f061bad5cff2aba6a9aed48d2a58a0d2c (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5-activate, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=squid-6cec793) 2024-04-02T04:48:49.896 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:49 smithi027 podman[102888]: 2024-04-02 04:48:49.737750218 +0000 UTC m=+0.546087459 container attach 7f592f5e80bd0ce1ee2ecc7a2062d77f061bad5cff2aba6a9aed48d2a58a0d2c (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5-activate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-6cec793) 2024-04-02T04:48:50.345 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:50 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5-activate[102903]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-02T04:48:50.345 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:50 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5-activate[102903]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-02T04:48:50.345 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:50 smithi027 bash[102888]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-02T04:48:50.345 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:50 smithi027 bash[102888]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-02T04:48:50.345 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:50 smithi027 bash[102888]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-02T04:48:50.345 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:50 smithi027 bash[102888]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-02T04:48:50.345 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:50 smithi027 bash[102888]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-04-02T04:48:50.345 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:50 smithi027 bash[102888]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-02T04:48:50.345 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:50 smithi027 bash[102888]: --> ceph-volume raw activate successful for osd ID: 5 2024-04-02T04:48:50.345 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:50 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5-activate[102903]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-02T04:48:50.345 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:50 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5-activate[102903]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-02T04:48:50.346 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:50 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5-activate[102903]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-04-02T04:48:50.346 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:50 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5-activate[102903]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-02T04:48:50.346 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:50 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5-activate[102903]: --> ceph-volume raw activate successful for osd ID: 5 2024-04-02T04:48:50.346 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:50 smithi027 podman[102888]: 2024-04-02 04:48:50.067388818 +0000 UTC m=+0.875726054 container died 7f592f5e80bd0ce1ee2ecc7a2062d77f061bad5cff2aba6a9aed48d2a58a0d2c (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5-activate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-6cec793, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-02T04:48:50.611 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:50 smithi027 ceph-mon[83036]: pgmap v166: 65 pgs: 5 active+undersized+degraded, 8 peering, 3 stale+active+clean, 5 active+undersized, 44 active+clean; 260 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 13/267 objects degraded (4.869%) 2024-04-02T04:48:50.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:50 smithi016 ceph-mon[93230]: pgmap v166: 65 pgs: 5 active+undersized+degraded, 8 peering, 3 stale+active+clean, 5 active+undersized, 44 active+clean; 260 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 13/267 objects degraded (4.869%) 2024-04-02T04:48:50.896 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:50 smithi027 podman[103023]: 2024-04-02 04:48:50.608530433 +0000 UTC m=+0.535368167 container remove 7f592f5e80bd0ce1ee2ecc7a2062d77f061bad5cff2aba6a9aed48d2a58a0d2c (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5-activate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.build-date=20240326, RELEASE=squid-6cec793, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-02T04:48:50.896 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:50 smithi027 podman[103086]: 2024-04-02 04:48:50.726061851 +0000 UTC m=+0.018494023 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:48:50.896 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:50 smithi027 podman[103086]: 2024-04-02 04:48:50.834482699 +0000 UTC m=+0.126914885 container create c2787f15737767bbcdb2ea71c8b69b5549819535c8237f3ad2c2b677738519ba (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-6cec793, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-02T04:48:51.396 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:51 smithi027 podman[103086]: 2024-04-02 04:48:51.120574979 +0000 UTC m=+0.413007126 container init c2787f15737767bbcdb2ea71c8b69b5549819535c8237f3ad2c2b677738519ba (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-6cec793, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-02T04:48:51.396 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:51 smithi027 podman[103086]: 2024-04-02 04:48:51.123983975 +0000 UTC m=+0.416416128 container start c2787f15737767bbcdb2ea71c8b69b5549819535c8237f3ad2c2b677738519ba (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5, RELEASE=squid-6cec793, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-02T04:48:51.396 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:51 smithi027 ceph-osd[103105]: -- 172.21.15.27:0/2705526332 <== mon.1 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563d43769680 con 0x563d43733000 2024-04-02T04:48:51.396 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:51 smithi027 bash[103086]: c2787f15737767bbcdb2ea71c8b69b5549819535c8237f3ad2c2b677738519ba 2024-04-02T04:48:51.396 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:51 smithi027 systemd[1]: Started Ceph osd.5 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4. 2024-04-02T04:48:52.053 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:51 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[103101]: 2024-04-02T04:48:51.662+0000 7fd124f777c0 -1 Falling back to public interface 2024-04-02T04:48:52.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:52 smithi027 ceph-mon[83036]: pgmap v167: 65 pgs: 11 active+undersized+degraded, 8 peering, 9 active+undersized, 37 active+clean; 260 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 37/267 objects degraded (13.858%) 2024-04-02T04:48:52.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:52 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:52.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:52 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:52.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:52 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:48:52.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:52 smithi016 ceph-mon[93230]: pgmap v167: 65 pgs: 11 active+undersized+degraded, 8 peering, 9 active+undersized, 37 active+clean; 260 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 37/267 objects degraded (13.858%) 2024-04-02T04:48:52.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:52 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:52.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:52 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:52.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:52 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:48:53.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:53 smithi016 ceph-mon[93230]: Health check update: Reduced data availability: 5 pgs peering (PG_AVAILABILITY) 2024-04-02T04:48:53.575 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:53 smithi027 ceph-mon[83036]: Health check update: Reduced data availability: 5 pgs peering (PG_AVAILABILITY) 2024-04-02T04:48:54.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:54 smithi016 ceph-mon[93230]: pgmap v168: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 260 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 41/267 objects degraded (15.356%) 2024-04-02T04:48:54.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:54 smithi016 ceph-mon[93230]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 5 pgs peering) 2024-04-02T04:48:54.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:54 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:54.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:54 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:54.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:54 smithi027 ceph-mon[83036]: pgmap v168: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 260 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 41/267 objects degraded (15.356%) 2024-04-02T04:48:54.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:54 smithi027 ceph-mon[83036]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 5 pgs peering) 2024-04-02T04:48:54.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:54 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:54.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:54 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:56.201 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:56 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[103101]: 2024-04-02T04:48:56.118+0000 7fd124f777c0 -1 osd.5 74 log_to_monitors true 2024-04-02T04:48:56.463 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:48:56 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[103101]: 2024-04-02T04:48:56.401+0000 7fd1163cf700 -1 osd.5 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T04:48:56.463 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:56 smithi027 ceph-mon[83036]: pgmap v169: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 260 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 41/267 objects degraded (15.356%) 2024-04-02T04:48:56.463 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:56 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:56.463 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:56 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:56.463 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:56 smithi027 ceph-mon[83036]: from='osd.5 [v2:172.21.15.27:6816/401102788,v1:172.21.15.27:6817/401102788]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T04:48:56.463 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:56 smithi027 ceph-mon[83036]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T04:48:56.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:56 smithi016 ceph-mon[93230]: pgmap v169: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 260 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 41/267 objects degraded (15.356%) 2024-04-02T04:48:56.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:56 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:56.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:56 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:48:56.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:56 smithi016 ceph-mon[93230]: from='osd.5 [v2:172.21.15.27:6816/401102788,v1:172.21.15.27:6817/401102788]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T04:48:56.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:56 smithi016 ceph-mon[93230]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T04:48:57.620 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:57 smithi027 ceph-mon[83036]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-02T04:48:57.620 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:57 smithi027 ceph-mon[83036]: osdmap e77: 6 total, 5 up, 6 in 2024-04-02T04:48:57.621 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:57 smithi027 ceph-mon[83036]: from='osd.5 [v2:172.21.15.27:6816/401102788,v1:172.21.15.27:6817/401102788]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-02T04:48:57.621 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:57 smithi027 ceph-mon[83036]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-02T04:48:57.621 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:57 smithi027 ceph-mon[83036]: pgmap v171: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 260 MiB data, 1005 MiB used, 535 GiB / 536 GiB avail; 41/267 objects degraded (15.356%) 2024-04-02T04:48:57.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:57 smithi016 ceph-mon[93230]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-02T04:48:57.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:57 smithi016 ceph-mon[93230]: osdmap e77: 6 total, 5 up, 6 in 2024-04-02T04:48:57.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:57 smithi016 ceph-mon[93230]: from='osd.5 [v2:172.21.15.27:6816/401102788,v1:172.21.15.27:6817/401102788]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-02T04:48:57.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:57 smithi016 ceph-mon[93230]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-04-02T04:48:57.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:57 smithi016 ceph-mon[93230]: pgmap v171: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 260 MiB data, 1005 MiB used, 535 GiB / 536 GiB avail; 41/267 objects degraded (15.356%) 2024-04-02T04:48:58.765 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:58 smithi027 ceph-mon[83036]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-02T04:48:58.766 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:58 smithi027 ceph-mon[83036]: osd.5 [v2:172.21.15.27:6816/401102788,v1:172.21.15.27:6817/401102788] boot 2024-04-02T04:48:58.766 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:58 smithi027 ceph-mon[83036]: osdmap e78: 6 total, 6 up, 6 in 2024-04-02T04:48:58.766 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:58 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T04:48:58.766 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:58 smithi027 ceph-mon[83036]: Health check update: Degraded data redundancy: 41/267 objects degraded (15.356%), 13 pgs degraded (PG_DEGRADED) 2024-04-02T04:48:58.766 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:48:58 smithi027 ceph-mon[83036]: osdmap e79: 6 total, 6 up, 6 in 2024-04-02T04:48:58.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:58 smithi016 ceph-mon[93230]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-02T04:48:58.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:58 smithi016 ceph-mon[93230]: osd.5 [v2:172.21.15.27:6816/401102788,v1:172.21.15.27:6817/401102788] boot 2024-04-02T04:48:58.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:58 smithi016 ceph-mon[93230]: osdmap e78: 6 total, 6 up, 6 in 2024-04-02T04:48:58.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:58 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T04:48:58.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:58 smithi016 ceph-mon[93230]: Health check update: Degraded data redundancy: 41/267 objects degraded (15.356%), 13 pgs degraded (PG_DEGRADED) 2024-04-02T04:48:58.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:48:58 smithi016 ceph-mon[93230]: osdmap e79: 6 total, 6 up, 6 in 2024-04-02T04:49:00.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:00 smithi016 ceph-mon[93230]: pgmap v174: 65 pgs: 2 peering, 10 active+undersized+degraded, 13 active+undersized, 40 active+clean; 260 MiB data, 1005 MiB used, 535 GiB / 536 GiB avail; 35/267 objects degraded (13.109%) 2024-04-02T04:49:00.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:00 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:00.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:00 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:00.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:00 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:49:00.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:00 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:49:00.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:00 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:00.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:00 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:49:00.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:00 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:00.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:00 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:00.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:00 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:00.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:00 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:00.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:00 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:00.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:00 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-04-02T04:49:00.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:00 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-04-02T04:49:00.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:00 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-04-02T04:49:00.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:00 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-04-02T04:49:00.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:00 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-04-02T04:49:00.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:00 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-04-02T04:49:00.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:00 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-04-02T04:49:00.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:00 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-04-02T04:49:00.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:00 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-04-02T04:49:00.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:00 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-04-02T04:49:00.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:00 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-04-02T04:49:00.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:00 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-04-02T04:49:00.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:00 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-04-02T04:49:00.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:00 smithi027 ceph-mon[83036]: pgmap v174: 65 pgs: 2 peering, 10 active+undersized+degraded, 13 active+undersized, 40 active+clean; 260 MiB data, 1005 MiB used, 535 GiB / 536 GiB avail; 35/267 objects degraded (13.109%) 2024-04-02T04:49:00.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:00 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:00.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:00 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:00.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:00 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:49:00.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:00 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:49:00.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:00 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:00.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:00 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:49:00.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:00 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:00.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:00 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:00.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:00 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:00.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:00 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:00.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:00 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:00.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:00 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-04-02T04:49:00.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:00 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-04-02T04:49:00.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:00 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-04-02T04:49:00.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:00 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-04-02T04:49:00.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:00 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-04-02T04:49:00.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:00 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-04-02T04:49:00.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:00 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-04-02T04:49:00.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:00 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-04-02T04:49:00.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:00 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-04-02T04:49:00.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:00 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-04-02T04:49:00.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:00 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-04-02T04:49:00.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:00 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-04-02T04:49:00.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:00 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-04-02T04:49:01.300 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:01 smithi016 ceph-mon[93230]: Upgrade: Setting container_image for all osd 2024-04-02T04:49:01.300 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:01 smithi016 ceph-mon[93230]: Upgrade: Setting require_osd_release to 19 squid 2024-04-02T04:49:01.300 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:01 smithi016 ceph-mon[93230]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-04-02T04:49:01.300 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:01 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-04-02T04:49:01.300 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:01 smithi016 ceph-mon[93230]: osdmap e80: 6 total, 6 up, 6 in 2024-04-02T04:49:01.301 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:01 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi016.uhgkgg"]}]: dispatch 2024-04-02T04:49:01.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:01 smithi027 ceph-mon[83036]: Upgrade: Setting container_image for all osd 2024-04-02T04:49:01.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:01 smithi027 ceph-mon[83036]: Upgrade: Setting require_osd_release to 19 squid 2024-04-02T04:49:01.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:01 smithi027 ceph-mon[83036]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-04-02T04:49:01.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:01 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-04-02T04:49:01.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:01 smithi027 ceph-mon[83036]: osdmap e80: 6 total, 6 up, 6 in 2024-04-02T04:49:01.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:01 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi016.uhgkgg"]}]: dispatch 2024-04-02T04:49:02.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:02 smithi016 ceph-mon[93230]: Upgrade: It appears safe to stop mds.cephfs.smithi016.uhgkgg 2024-04-02T04:49:02.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:02 smithi016 ceph-mon[93230]: pgmap v176: 65 pgs: 2 peering, 5 active+undersized+degraded, 11 active+undersized, 47 active+clean; 260 MiB data, 1005 MiB used, 535 GiB / 536 GiB avail; 15/267 objects degraded (5.618%) 2024-04-02T04:49:02.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:02 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:02.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:02 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:49:02.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:02 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:02.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:02 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi016.uhgkgg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:49:02.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:02 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:49:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:02 smithi027 ceph-mon[83036]: Upgrade: It appears safe to stop mds.cephfs.smithi016.uhgkgg 2024-04-02T04:49:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:02 smithi027 ceph-mon[83036]: pgmap v176: 65 pgs: 2 peering, 5 active+undersized+degraded, 11 active+undersized, 47 active+clean; 260 MiB data, 1005 MiB used, 535 GiB / 536 GiB avail; 15/267 objects degraded (5.618%) 2024-04-02T04:49:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:02 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:02 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:49:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:02 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:02 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi016.uhgkgg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:49:02.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:02 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:49:03.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:03 smithi016 ceph-mon[93230]: Upgrade: Updating mds.cephfs.smithi016.uhgkgg 2024-04-02T04:49:03.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:03 smithi016 ceph-mon[93230]: Deploying daemon mds.cephfs.smithi016.uhgkgg on smithi016 2024-04-02T04:49:03.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:03 smithi016 ceph-mon[93230]: Health check update: Degraded data redundancy: 15/267 objects degraded (5.618%), 5 pgs degraded (PG_DEGRADED) 2024-04-02T04:49:03.598 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:03 smithi027 ceph-mon[83036]: Upgrade: Updating mds.cephfs.smithi016.uhgkgg 2024-04-02T04:49:03.598 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:03 smithi027 ceph-mon[83036]: Deploying daemon mds.cephfs.smithi016.uhgkgg on smithi016 2024-04-02T04:49:03.598 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:03 smithi027 ceph-mon[83036]: Health check update: Degraded data redundancy: 15/267 objects degraded (5.618%), 5 pgs degraded (PG_DEGRADED) 2024-04-02T04:49:04.255 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:04 smithi016 ceph-mon[93230]: pgmap v177: 65 pgs: 2 peering, 63 active+clean; 260 MiB data, 1005 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:04.256 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:04 smithi016 ceph-mon[93230]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 15/267 objects degraded (5.618%), 5 pgs degraded) 2024-04-02T04:49:04.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:04 smithi027 ceph-mon[83036]: pgmap v177: 65 pgs: 2 peering, 63 active+clean; 260 MiB data, 1005 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:04.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:04 smithi027 ceph-mon[83036]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 15/267 objects degraded (5.618%), 5 pgs degraded) 2024-04-02T04:49:06.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:06 smithi016 ceph-mon[93230]: pgmap v178: 65 pgs: 65 active+clean; 260 MiB data, 1005 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:06.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:06 smithi016 ceph-mon[93230]: osdmap e81: 6 total, 6 up, 6 in 2024-04-02T04:49:06.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:06 smithi016 ceph-mon[93230]: fsmap cephfs:1 {0=cephfs.smithi027.zwrfoo=up:active} 2 up:standby 2024-04-02T04:49:06.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:06 smithi027 ceph-mon[83036]: pgmap v178: 65 pgs: 65 active+clean; 260 MiB data, 1005 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:06.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:06 smithi027 ceph-mon[83036]: osdmap e81: 6 total, 6 up, 6 in 2024-04-02T04:49:06.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:06 smithi027 ceph-mon[83036]: fsmap cephfs:1 {0=cephfs.smithi027.zwrfoo=up:active} 2 up:standby 2024-04-02T04:49:08.475 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:08 smithi016 ceph-mon[93230]: pgmap v180: 65 pgs: 65 active+clean; 260 MiB data, 1005 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:08.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:08 smithi027 ceph-mon[83036]: pgmap v180: 65 pgs: 65 active+clean; 260 MiB data, 1005 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:09.789 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:09 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:09.789 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:09 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:09.789 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:09 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:49:09.789 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:09 smithi016 ceph-mon[93230]: pgmap v181: 65 pgs: 65 active+clean; 260 MiB data, 1005 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:09.789 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:09 smithi016 ceph-mon[93230]: mds.? [v2:172.21.15.16:6826/81744708,v1:172.21.15.16:6827/81744708] up:boot 2024-04-02T04:49:09.790 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:09 smithi016 ceph-mon[93230]: fsmap cephfs:1 {0=cephfs.smithi027.zwrfoo=up:active} 3 up:standby 2024-04-02T04:49:09.790 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:09 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi016.uhgkgg"}]: dispatch 2024-04-02T04:49:10.048 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:09 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:10.048 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:09 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:10.048 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:09 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:49:10.048 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:09 smithi027 ceph-mon[83036]: pgmap v181: 65 pgs: 65 active+clean; 260 MiB data, 1005 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:10.048 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:09 smithi027 ceph-mon[83036]: mds.? [v2:172.21.15.16:6826/81744708,v1:172.21.15.16:6827/81744708] up:boot 2024-04-02T04:49:10.049 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:09 smithi027 ceph-mon[83036]: fsmap cephfs:1 {0=cephfs.smithi027.zwrfoo=up:active} 3 up:standby 2024-04-02T04:49:10.049 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:09 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi016.uhgkgg"}]: dispatch 2024-04-02T04:49:11.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.575+0000 7f65cb8d1700 1 -- 172.21.15.16:0/3052710836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c4073810 msgr2=0x7f65c4073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:11.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.575+0000 7f65cb8d1700 1 --2- 172.21.15.16:0/3052710836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c4073810 0x7f65c4073c90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f65b4009910 tx=0x7f65b4009c20 comp rx=0 tx=0).stop 2024-04-02T04:49:11.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.575+0000 7f65cb8d1700 1 -- 172.21.15.16:0/3052710836 shutdown_connections 2024-04-02T04:49:11.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.575+0000 7f65cb8d1700 1 --2- 172.21.15.16:0/3052710836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c4073810 0x7f65c4073c90 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:11.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.575+0000 7f65cb8d1700 1 --2- 172.21.15.16:0/3052710836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65c4074de0 0x7f65c4073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:11.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.575+0000 7f65cb8d1700 1 -- 172.21.15.16:0/3052710836 >> 172.21.15.16:0/3052710836 conn(0x7f65c40fc090 msgr2=0x7f65c40fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:11.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.576+0000 7f65cb8d1700 1 -- 172.21.15.16:0/3052710836 shutdown_connections 2024-04-02T04:49:11.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.576+0000 7f65cb8d1700 1 -- 172.21.15.16:0/3052710836 wait complete. 2024-04-02T04:49:11.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.576+0000 7f65cb8d1700 1 Processor -- start 2024-04-02T04:49:11.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.576+0000 7f65cb8d1700 1 -- start start 2024-04-02T04:49:11.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.577+0000 7f65cb8d1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c4073810 0x7f65c419cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:11.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.577+0000 7f65cb8d1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65c4074de0 0x7f65c419d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:11.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.577+0000 7f65cb8d1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c419d630 con 0x7f65c4073810 2024-04-02T04:49:11.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.577+0000 7f65cb8d1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c419d770 con 0x7f65c4074de0 2024-04-02T04:49:11.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.577+0000 7f65c966d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c4073810 0x7f65c419cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:11.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.577+0000 7f65c966d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c4073810 0x7f65c419cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34092/0 (socket says 172.21.15.16:34092) 2024-04-02T04:49:11.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.577+0000 7f65c966d700 1 -- 172.21.15.16:0/1534085200 learned_addr learned my addr 172.21.15.16:0/1534085200 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:49:11.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.577+0000 7f65c8e6c700 1 --2- 172.21.15.16:0/1534085200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65c4074de0 0x7f65c419d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:11.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.577+0000 7f65c966d700 1 -- 172.21.15.16:0/1534085200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65c4074de0 msgr2=0x7f65c419d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:11.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.577+0000 7f65c966d700 1 --2- 172.21.15.16:0/1534085200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65c4074de0 0x7f65c419d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:11.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.577+0000 7f65c966d700 1 -- 172.21.15.16:0/1534085200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65b40095f0 con 0x7f65c4073810 2024-04-02T04:49:11.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.577+0000 7f65c8e6c700 1 --2- 172.21.15.16:0/1534085200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65c4074de0 0x7f65c419d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T04:49:11.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.577+0000 7f65c966d700 1 --2- 172.21.15.16:0/1534085200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c4073810 0x7f65c419cad0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f65c000d900 tx=0x7f65c000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:11.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.578+0000 7f65ba7fc700 1 -- 172.21.15.16:0/1534085200 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65c0009900 con 0x7f65c4073810 2024-04-02T04:49:11.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.578+0000 7f65cb8d1700 1 -- 172.21.15.16:0/1534085200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65c41a21d0 con 0x7f65c4073810 2024-04-02T04:49:11.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.578+0000 7f65ba7fc700 1 -- 172.21.15.16:0/1534085200 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f65c0010460 con 0x7f65c4073810 2024-04-02T04:49:11.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.578+0000 7f65ba7fc700 1 -- 172.21.15.16:0/1534085200 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65c000f5f0 con 0x7f65c4073810 2024-04-02T04:49:11.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.578+0000 7f65cb8d1700 1 -- 172.21.15.16:0/1534085200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65c41a27a0 con 0x7f65c4073810 2024-04-02T04:49:11.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.579+0000 7f65ba7fc700 1 -- 172.21.15.16:0/1534085200 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f65c000f870 con 0x7f65c4073810 2024-04-02T04:49:11.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.580+0000 7f65cb8d1700 1 -- 172.21.15.16:0/1534085200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65c404ea80 con 0x7f65c4073810 2024-04-02T04:49:11.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.580+0000 7f65ba7fc700 1 --2- 172.21.15.16:0/1534085200 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f65b006dd60 0x7f65b0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:11.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.580+0000 7f65ba7fc700 1 -- 172.21.15.16:0/1534085200 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f65c008cc50 con 0x7f65c4073810 2024-04-02T04:49:11.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.580+0000 7f65c8e6c700 1 --2- 172.21.15.16:0/1534085200 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f65b006dd60 0x7f65b0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:11.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.580+0000 7f65c8e6c700 1 --2- 172.21.15.16:0/1534085200 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f65b006dd60 0x7f65b0070220 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f65b4000c00 tx=0x7f65b4005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:11.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.585+0000 7f65ba7fc700 1 -- 172.21.15.16:0/1534085200 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f65c0059c80 con 0x7f65c4073810 2024-04-02T04:49:11.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.818+0000 7f65cb8d1700 1 -- 172.21.15.16:0/1534085200 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f65c4103ad0 con 0x7f65b006dd60 2024-04-02T04:49:11.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.820+0000 7f65ba7fc700 1 -- 172.21.15.16:0/1534085200 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f65c4103ad0 con 0x7f65b006dd60 2024-04-02T04:49:11.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.822+0000 7f65cb8d1700 1 -- 172.21.15.16:0/1534085200 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f65b006dd60 msgr2=0x7f65b0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:11.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.822+0000 7f65cb8d1700 1 --2- 172.21.15.16:0/1534085200 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f65b006dd60 0x7f65b0070220 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f65b4000c00 tx=0x7f65b4005960 comp rx=0 tx=0).stop 2024-04-02T04:49:11.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.822+0000 7f65cb8d1700 1 -- 172.21.15.16:0/1534085200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c4073810 msgr2=0x7f65c419cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:11.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.822+0000 7f65cb8d1700 1 --2- 172.21.15.16:0/1534085200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c4073810 0x7f65c419cad0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f65c000d900 tx=0x7f65c000dc10 comp rx=0 tx=0).stop 2024-04-02T04:49:11.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.822+0000 7f65cb8d1700 1 -- 172.21.15.16:0/1534085200 shutdown_connections 2024-04-02T04:49:11.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.822+0000 7f65cb8d1700 1 --2- 172.21.15.16:0/1534085200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c4073810 0x7f65c419cad0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:11.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.822+0000 7f65cb8d1700 1 --2- 172.21.15.16:0/1534085200 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f65b006dd60 0x7f65b0070220 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:11.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.822+0000 7f65cb8d1700 1 --2- 172.21.15.16:0/1534085200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65c4074de0 0x7f65c419d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:11.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.822+0000 7f65cb8d1700 1 -- 172.21.15.16:0/1534085200 >> 172.21.15.16:0/1534085200 conn(0x7f65c40fc090 msgr2=0x7f65c41023b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:11.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.822+0000 7f65cb8d1700 1 -- 172.21.15.16:0/1534085200 shutdown_connections 2024-04-02T04:49:11.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.822+0000 7f65cb8d1700 1 -- 172.21.15.16:0/1534085200 wait complete. 2024-04-02T04:49:11.836 INFO:teuthology.orchestra.run.smithi016.stdout:true 2024-04-02T04:49:11.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.931+0000 7f0400687700 1 -- 172.21.15.16:0/482993135 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03f8100be0 msgr2=0x7f03f8101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:11.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.931+0000 7f0400687700 1 --2- 172.21.15.16:0/482993135 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03f8100be0 0x7f03f8101040 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f03e8009910 tx=0x7f03e8009c20 comp rx=0 tx=0).stop 2024-04-02T04:49:11.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.931+0000 7f0400687700 1 -- 172.21.15.16:0/482993135 shutdown_connections 2024-04-02T04:49:11.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.931+0000 7f0400687700 1 --2- 172.21.15.16:0/482993135 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03f8100be0 0x7f03f8101040 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:11.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.931+0000 7f0400687700 1 --2- 172.21.15.16:0/482993135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03f80ff9e0 0x7f03f80ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:11.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.931+0000 7f0400687700 1 -- 172.21.15.16:0/482993135 >> 172.21.15.16:0/482993135 conn(0x7f03f80faf80 msgr2=0x7f03f80fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:11.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.931+0000 7f0400687700 1 -- 172.21.15.16:0/482993135 shutdown_connections 2024-04-02T04:49:11.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.931+0000 7f0400687700 1 -- 172.21.15.16:0/482993135 wait complete. 2024-04-02T04:49:11.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.932+0000 7f0400687700 1 Processor -- start 2024-04-02T04:49:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.932+0000 7f0400687700 1 -- start start 2024-04-02T04:49:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.932+0000 7f0400687700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03f80ff9e0 0x7f03f8195430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.932+0000 7f0400687700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03f8100be0 0x7f03f8195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.932+0000 7f0400687700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f8195f90 con 0x7f03f8100be0 2024-04-02T04:49:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.932+0000 7f0400687700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f81960d0 con 0x7f03f80ff9e0 2024-04-02T04:49:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.932+0000 7f03fdc22700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03f8100be0 0x7f03f8195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.932+0000 7f03fdc22700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03f8100be0 0x7f03f8195970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57614/0 (socket says 172.21.15.16:57614) 2024-04-02T04:49:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.932+0000 7f03fdc22700 1 -- 172.21.15.16:0/2266850134 learned_addr learned my addr 172.21.15.16:0/2266850134 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:49:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.932+0000 7f03fe423700 1 --2- 172.21.15.16:0/2266850134 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03f80ff9e0 0x7f03f8195430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.933+0000 7f03fdc22700 1 -- 172.21.15.16:0/2266850134 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03f80ff9e0 msgr2=0x7f03f8195430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.933+0000 7f03fdc22700 1 --2- 172.21.15.16:0/2266850134 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03f80ff9e0 0x7f03f8195430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.933+0000 7f03fdc22700 1 -- 172.21.15.16:0/2266850134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e80095f0 con 0x7f03f8100be0 2024-04-02T04:49:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.933+0000 7f03fdc22700 1 --2- 172.21.15.16:0/2266850134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03f8100be0 0x7f03f8195970 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f03e800b800 tx=0x7f03e8005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.933+0000 7f03ef7fe700 1 -- 172.21.15.16:0/2266850134 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03e801d070 con 0x7f03f8100be0 2024-04-02T04:49:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.933+0000 7f0400687700 1 -- 172.21.15.16:0/2266850134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03f819aa80 con 0x7f03f8100be0 2024-04-02T04:49:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.933+0000 7f03ef7fe700 1 -- 172.21.15.16:0/2266850134 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f03e80052a0 con 0x7f03f8100be0 2024-04-02T04:49:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.933+0000 7f03ef7fe700 1 -- 172.21.15.16:0/2266850134 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03e8017400 con 0x7f03f8100be0 2024-04-02T04:49:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.933+0000 7f0400687700 1 -- 172.21.15.16:0/2266850134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03f819aff0 con 0x7f03f8100be0 2024-04-02T04:49:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.934+0000 7f0400687700 1 -- 172.21.15.16:0/2266850134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03f818f560 con 0x7f03f8100be0 2024-04-02T04:49:11.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.935+0000 7f03ef7fe700 1 -- 172.21.15.16:0/2266850134 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f03e8003780 con 0x7f03f8100be0 2024-04-02T04:49:11.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.935+0000 7f03ef7fe700 1 --2- 172.21.15.16:0/2266850134 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f03e406dd60 0x7f03e4070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:11.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.935+0000 7f03ef7fe700 1 -- 172.21.15.16:0/2266850134 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f03e808ef50 con 0x7f03f8100be0 2024-04-02T04:49:11.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.935+0000 7f03fe423700 1 --2- 172.21.15.16:0/2266850134 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f03e406dd60 0x7f03e4070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:11.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.936+0000 7f03fe423700 1 --2- 172.21.15.16:0/2266850134 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f03e406dd60 0x7f03e4070220 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f03f8100a40 tx=0x7f03f4009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:11.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:11.944+0000 7f03ef7fe700 1 -- 172.21.15.16:0/2266850134 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f03e80c3280 con 0x7f03f8100be0 2024-04-02T04:49:12.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.171+0000 7f0400687700 1 -- 172.21.15.16:0/2266850134 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f03f802cdc0 con 0x7f03e406dd60 2024-04-02T04:49:12.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.172+0000 7f03ef7fe700 1 -- 172.21.15.16:0/2266850134 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f03f802cdc0 con 0x7f03e406dd60 2024-04-02T04:49:12.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.174+0000 7f0400687700 1 -- 172.21.15.16:0/2266850134 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f03e406dd60 msgr2=0x7f03e4070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:12.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.174+0000 7f0400687700 1 --2- 172.21.15.16:0/2266850134 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f03e406dd60 0x7f03e4070220 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f03f8100a40 tx=0x7f03f4009260 comp rx=0 tx=0).stop 2024-04-02T04:49:12.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.174+0000 7f0400687700 1 -- 172.21.15.16:0/2266850134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03f8100be0 msgr2=0x7f03f8195970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:12.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.174+0000 7f0400687700 1 --2- 172.21.15.16:0/2266850134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03f8100be0 0x7f03f8195970 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f03e800b800 tx=0x7f03e8005f00 comp rx=0 tx=0).stop 2024-04-02T04:49:12.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.174+0000 7f0400687700 1 -- 172.21.15.16:0/2266850134 shutdown_connections 2024-04-02T04:49:12.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.174+0000 7f0400687700 1 --2- 172.21.15.16:0/2266850134 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f03e406dd60 0x7f03e4070220 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:12.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.174+0000 7f0400687700 1 --2- 172.21.15.16:0/2266850134 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03f80ff9e0 0x7f03f8195430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:12.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.174+0000 7f0400687700 1 --2- 172.21.15.16:0/2266850134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03f8100be0 0x7f03f8195970 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:12.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.174+0000 7f0400687700 1 -- 172.21.15.16:0/2266850134 >> 172.21.15.16:0/2266850134 conn(0x7f03f80faf80 msgr2=0x7f03f8104200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:12.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.174+0000 7f0400687700 1 -- 172.21.15.16:0/2266850134 shutdown_connections 2024-04-02T04:49:12.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.174+0000 7f0400687700 1 -- 172.21.15.16:0/2266850134 wait complete. 2024-04-02T04:49:12.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.301+0000 7fc6726a0700 1 -- 172.21.15.16:0/2127560439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc66c100b00 msgr2=0x7fc66c100f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:12.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.301+0000 7fc6726a0700 1 --2- 172.21.15.16:0/2127560439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc66c100b00 0x7fc66c100f20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc65c009910 tx=0x7fc65c009c20 comp rx=0 tx=0).stop 2024-04-02T04:49:12.305 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:12 smithi016 ceph-mon[93230]: pgmap v182: 65 pgs: 65 active+clean; 260 MiB data, 1005 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:12.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.301+0000 7fc6726a0700 1 -- 172.21.15.16:0/2127560439 shutdown_connections 2024-04-02T04:49:12.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.301+0000 7fc6726a0700 1 --2- 172.21.15.16:0/2127560439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc66c101d00 0x7fc66c102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:12.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.301+0000 7fc6726a0700 1 --2- 172.21.15.16:0/2127560439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc66c100b00 0x7fc66c100f20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:12.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.301+0000 7fc6726a0700 1 -- 172.21.15.16:0/2127560439 >> 172.21.15.16:0/2127560439 conn(0x7fc66c0fc0a0 msgr2=0x7fc66c0fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:12.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.301+0000 7fc6726a0700 1 -- 172.21.15.16:0/2127560439 shutdown_connections 2024-04-02T04:49:12.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.301+0000 7fc6726a0700 1 -- 172.21.15.16:0/2127560439 wait complete. 2024-04-02T04:49:12.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.302+0000 7fc6726a0700 1 Processor -- start 2024-04-02T04:49:12.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.302+0000 7fc6726a0700 1 -- start start 2024-04-02T04:49:12.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.302+0000 7fc6726a0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc66c100b00 0x7fc66c198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:12.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.302+0000 7fc6726a0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc66c101d00 0x7fc66c198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:12.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.302+0000 7fc6726a0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c199360 con 0x7fc66c100b00 2024-04-02T04:49:12.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.302+0000 7fc6726a0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c1994a0 con 0x7fc66c101d00 2024-04-02T04:49:12.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.302+0000 7fc66bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc66c100b00 0x7fc66c198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:12.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.303+0000 7fc66bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc66c100b00 0x7fc66c198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57626/0 (socket says 172.21.15.16:57626) 2024-04-02T04:49:12.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.303+0000 7fc66bfff700 1 -- 172.21.15.16:0/497763929 learned_addr learned my addr 172.21.15.16:0/497763929 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:49:12.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.303+0000 7fc66b7fe700 1 --2- 172.21.15.16:0/497763929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc66c101d00 0x7fc66c198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:12.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.303+0000 7fc66bfff700 1 -- 172.21.15.16:0/497763929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc66c101d00 msgr2=0x7fc66c198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:12.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.303+0000 7fc66bfff700 1 --2- 172.21.15.16:0/497763929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc66c101d00 0x7fc66c198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:12.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.303+0000 7fc66bfff700 1 -- 172.21.15.16:0/497763929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc65c0095f0 con 0x7fc66c100b00 2024-04-02T04:49:12.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.303+0000 7fc66bfff700 1 --2- 172.21.15.16:0/497763929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc66c100b00 0x7fc66c198800 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc65c00bd70 tx=0x7fc65c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:12.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.303+0000 7fc6697fa700 1 -- 172.21.15.16:0/497763929 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc65c01d070 con 0x7fc66c100b00 2024-04-02T04:49:12.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.303+0000 7fc6726a0700 1 -- 172.21.15.16:0/497763929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc66c19dea0 con 0x7fc66c100b00 2024-04-02T04:49:12.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.303+0000 7fc6697fa700 1 -- 172.21.15.16:0/497763929 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc65c00f460 con 0x7fc66c100b00 2024-04-02T04:49:12.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.303+0000 7fc6726a0700 1 -- 172.21.15.16:0/497763929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc66c19e390 con 0x7fc66c100b00 2024-04-02T04:49:12.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.303+0000 7fc6697fa700 1 -- 172.21.15.16:0/497763929 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc65c021640 con 0x7fc66c100b00 2024-04-02T04:49:12.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.305+0000 7fc6726a0700 1 -- 172.21.15.16:0/497763929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc66c1066d0 con 0x7fc66c100b00 2024-04-02T04:49:12.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.305+0000 7fc6697fa700 1 -- 172.21.15.16:0/497763929 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fc65c00fab0 con 0x7fc66c100b00 2024-04-02T04:49:12.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.305+0000 7fc6697fa700 1 --2- 172.21.15.16:0/497763929 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc65406dd70 0x7fc654070230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:12.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.306+0000 7fc6697fa700 1 -- 172.21.15.16:0/497763929 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fc65c08ee60 con 0x7fc66c100b00 2024-04-02T04:49:12.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.306+0000 7fc66b7fe700 1 --2- 172.21.15.16:0/497763929 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc65406dd70 0x7fc654070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:12.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.306+0000 7fc66b7fe700 1 --2- 172.21.15.16:0/497763929 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc65406dd70 0x7fc654070230 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc660009a10 tx=0x7fc660009190 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:12.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.310+0000 7fc6697fa700 1 -- 172.21.15.16:0/497763929 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc65c05bf20 con 0x7fc66c100b00 2024-04-02T04:49:12.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:12 smithi027 ceph-mon[83036]: pgmap v182: 65 pgs: 65 active+clean; 260 MiB data, 1005 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:12.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.532+0000 7fc6726a0700 1 -- 172.21.15.16:0/497763929 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc66c02cdc0 con 0x7fc65406dd70 2024-04-02T04:49:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.543+0000 7fc6697fa700 1 -- 172.21.15.16:0/497763929 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fc66c02cdc0 con 0x7fc65406dd70 2024-04-02T04:49:12.546 INFO:teuthology.orchestra.run.smithi016.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:49:12.547 INFO:teuthology.orchestra.run.smithi016.stdout:alertmanager.smithi016 smithi016 *:9093,9094 running (13m) 119s ago 15m 24.1M - 0.25.0 c8568f914cd2 b95d8354b016 2024-04-02T04:49:12.547 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi016 smithi016 running (15m) 119s ago 15m 8300k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 9de6c16a487f 2024-04-02T04:49:12.547 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi027 smithi027 running (14m) 18s ago 14m 8577k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 99956592471a 2024-04-02T04:49:12.547 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi016 smithi016 running (3m) 119s ago 15m 6660k - 19.0.0-1979-g6cec793d 78f830190839 dc63ce0eef58 2024-04-02T04:49:12.547 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi027 smithi027 running (3m) 18s ago 14m 6665k - 19.0.0-1979-g6cec793d 78f830190839 2beb077018e4 2024-04-02T04:49:12.547 INFO:teuthology.orchestra.run.smithi016.stdout:grafana.smithi016 smithi016 *:3000 running (13m) 119s ago 14m 84.6M - 9.4.7 954c08fa6188 7c43d133569c 2024-04-02T04:49:12.547 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.fyzrnm smithi016 running (9m) 119s ago 9m 17.9M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 70f28b9572a6 2024-04-02T04:49:12.547 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.uhgkgg smithi016 starting - - - - 2024-04-02T04:49:12.547 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.sdublg smithi027 running (9m) 18s ago 9m 20.6M - 18.2.2-1059-g4d05f4ca 909dce4fe53d a538451099db 2024-04-02T04:49:12.547 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.zwrfoo smithi027 running (9m) 18s ago 9m 156M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 4cb3fa428d71 2024-04-02T04:49:12.547 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi016.eqnebx smithi016 *:8443,9283,8765 running (6m) 119s ago 16m 501M - 19.0.0-1979-g6cec793d 78f830190839 d61ed91b4577 2024-04-02T04:49:12.547 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi027.kjzcsc smithi027 *:8443,9283,8765 running (5m) 18s ago 14m 450M - 19.0.0-1979-g6cec793d 78f830190839 5072a5ffc761 2024-04-02T04:49:12.547 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi016 smithi016 running (5m) 119s ago 16m 46.9M 2048M 19.0.0-1979-g6cec793d 78f830190839 eabfc84f1cdf 2024-04-02T04:49:12.548 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi027 smithi027 running (4m) 18s ago 13m 40.1M 2048M 19.0.0-1979-g6cec793d 78f830190839 fcd68b20dcc5 2024-04-02T04:49:12.548 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi016 smithi016 *:9100 running (15m) 119s ago 15m 15.2M - 1.5.0 0da6a335fe13 09f54c65b53c 2024-04-02T04:49:12.548 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi027 smithi027 *:9100 running (14m) 18s ago 14m 15.5M - 1.5.0 0da6a335fe13 e41a657c7443 2024-04-02T04:49:12.548 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 smithi016 running (3m) 119s ago 13m 81.5M 1217M 19.0.0-1979-g6cec793d 78f830190839 421d9e1cc751 2024-04-02T04:49:12.548 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 smithi016 running (2m) 119s ago 12m 87.0M 1217M 19.0.0-1979-g6cec793d 78f830190839 3612a56236a3 2024-04-02T04:49:12.548 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 smithi016 running (2m) 119s ago 12m 17.1M 1217M 19.0.0-1979-g6cec793d 78f830190839 a8e9b8d0303c 2024-04-02T04:49:12.548 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 smithi027 running (89s) 18s ago 11m 102M 2240M 19.0.0-1979-g6cec793d 78f830190839 0dd13b5be8c0 2024-04-02T04:49:12.548 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 smithi027 running (55s) 18s ago 11m 84.1M 2240M 19.0.0-1979-g6cec793d 78f830190839 9025eacf8145 2024-04-02T04:49:12.548 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 smithi027 running (21s) 18s ago 11m 14.8M 2240M 19.0.0-1979-g6cec793d 78f830190839 c2787f157377 2024-04-02T04:49:12.548 INFO:teuthology.orchestra.run.smithi016.stdout:prometheus.smithi016 smithi016 *:9095 running (5m) 119s ago 14m 58.5M - 2.43.0 a07b618ecd1d 466f00d73ddd 2024-04-02T04:49:12.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.545+0000 7fc6726a0700 1 -- 172.21.15.16:0/497763929 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc65406dd70 msgr2=0x7fc654070230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:12.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.545+0000 7fc6726a0700 1 --2- 172.21.15.16:0/497763929 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc65406dd70 0x7fc654070230 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc660009a10 tx=0x7fc660009190 comp rx=0 tx=0).stop 2024-04-02T04:49:12.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.545+0000 7fc6726a0700 1 -- 172.21.15.16:0/497763929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc66c100b00 msgr2=0x7fc66c198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:12.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.545+0000 7fc6726a0700 1 --2- 172.21.15.16:0/497763929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc66c100b00 0x7fc66c198800 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc65c00bd70 tx=0x7fc65c00bda0 comp rx=0 tx=0).stop 2024-04-02T04:49:12.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.546+0000 7fc6726a0700 1 -- 172.21.15.16:0/497763929 shutdown_connections 2024-04-02T04:49:12.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.546+0000 7fc6726a0700 1 --2- 172.21.15.16:0/497763929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc66c100b00 0x7fc66c198800 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:12.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.546+0000 7fc6726a0700 1 --2- 172.21.15.16:0/497763929 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc65406dd70 0x7fc654070230 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:12.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.546+0000 7fc6726a0700 1 --2- 172.21.15.16:0/497763929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc66c101d00 0x7fc66c198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:12.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.546+0000 7fc6726a0700 1 -- 172.21.15.16:0/497763929 >> 172.21.15.16:0/497763929 conn(0x7fc66c0fc0a0 msgr2=0x7fc66c073400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:12.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.546+0000 7fc6726a0700 1 -- 172.21.15.16:0/497763929 shutdown_connections 2024-04-02T04:49:12.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.546+0000 7fc6726a0700 1 -- 172.21.15.16:0/497763929 wait complete. 2024-04-02T04:49:12.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.657+0000 7fa0193bf700 1 -- 172.21.15.16:0/3212664212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa014103ea0 msgr2=0x7fa014106290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:12.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.657+0000 7fa0193bf700 1 --2- 172.21.15.16:0/3212664212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa014103ea0 0x7fa014106290 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa008009910 tx=0x7fa008009c20 comp rx=0 tx=0).stop 2024-04-02T04:49:12.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.657+0000 7fa0193bf700 1 -- 172.21.15.16:0/3212664212 shutdown_connections 2024-04-02T04:49:12.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.657+0000 7fa0193bf700 1 --2- 172.21.15.16:0/3212664212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa014103ea0 0x7fa014106290 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:12.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.657+0000 7fa0193bf700 1 --2- 172.21.15.16:0/3212664212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa014101570 0x7fa014103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:12.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.657+0000 7fa0193bf700 1 -- 172.21.15.16:0/3212664212 >> 172.21.15.16:0/3212664212 conn(0x7fa0140faf80 msgr2=0x7fa0140fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:12.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.657+0000 7fa0193bf700 1 -- 172.21.15.16:0/3212664212 shutdown_connections 2024-04-02T04:49:12.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.658+0000 7fa0193bf700 1 -- 172.21.15.16:0/3212664212 wait complete. 2024-04-02T04:49:12.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.658+0000 7fa0193bf700 1 Processor -- start 2024-04-02T04:49:12.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.658+0000 7fa0193bf700 1 -- start start 2024-04-02T04:49:12.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.658+0000 7fa0193bf700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa014101570 0x7fa014100de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:12.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.658+0000 7fa0193bf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa014103ea0 0x7fa0140ff520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:12.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.658+0000 7fa0193bf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa014101400 con 0x7fa014103ea0 2024-04-02T04:49:12.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.658+0000 7fa0193bf700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0140ffa60 con 0x7fa014101570 2024-04-02T04:49:12.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.659+0000 7fa0127fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa014103ea0 0x7fa0140ff520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:12.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.659+0000 7fa0127fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa014103ea0 0x7fa0140ff520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57642/0 (socket says 172.21.15.16:57642) 2024-04-02T04:49:12.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.659+0000 7fa0127fc700 1 -- 172.21.15.16:0/4274117908 learned_addr learned my addr 172.21.15.16:0/4274117908 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:49:12.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.659+0000 7fa012ffd700 1 --2- 172.21.15.16:0/4274117908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa014101570 0x7fa014100de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:12.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.659+0000 7fa0127fc700 1 -- 172.21.15.16:0/4274117908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa014101570 msgr2=0x7fa014100de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:12.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.659+0000 7fa0127fc700 1 --2- 172.21.15.16:0/4274117908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa014101570 0x7fa014100de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:12.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.659+0000 7fa0127fc700 1 -- 172.21.15.16:0/4274117908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0080095f0 con 0x7fa014103ea0 2024-04-02T04:49:12.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.659+0000 7fa0127fc700 1 --2- 172.21.15.16:0/4274117908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa014103ea0 0x7fa0140ff520 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa0080092e0 tx=0x7fa00800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:12.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.661+0000 7f9ffbfff700 1 -- 172.21.15.16:0/4274117908 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa00801d070 con 0x7fa014103ea0 2024-04-02T04:49:12.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.661+0000 7f9ffbfff700 1 -- 172.21.15.16:0/4274117908 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa00800f460 con 0x7fa014103ea0 2024-04-02T04:49:12.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.661+0000 7f9ffbfff700 1 -- 172.21.15.16:0/4274117908 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa008021640 con 0x7fa014103ea0 2024-04-02T04:49:12.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.661+0000 7fa0193bf700 1 -- 172.21.15.16:0/4274117908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0140ffcb0 con 0x7fa014103ea0 2024-04-02T04:49:12.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.661+0000 7fa0193bf700 1 -- 172.21.15.16:0/4274117908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa014100220 con 0x7fa014103ea0 2024-04-02T04:49:12.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.662+0000 7fa0193bf700 1 -- 172.21.15.16:0/4274117908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa01404ea80 con 0x7fa014103ea0 2024-04-02T04:49:12.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.663+0000 7f9ffbfff700 1 -- 172.21.15.16:0/4274117908 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fa00800f5d0 con 0x7fa014103ea0 2024-04-02T04:49:12.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.664+0000 7f9ffbfff700 1 --2- 172.21.15.16:0/4274117908 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f9ffc06ddc0 0x7f9ffc070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:12.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.664+0000 7f9ffbfff700 1 -- 172.21.15.16:0/4274117908 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fa00808ed30 con 0x7fa014103ea0 2024-04-02T04:49:12.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.664+0000 7fa012ffd700 1 --2- 172.21.15.16:0/4274117908 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f9ffc06ddc0 0x7f9ffc070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:12.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.664+0000 7fa012ffd700 1 --2- 172.21.15.16:0/4274117908 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f9ffc06ddc0 0x7f9ffc070280 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa004009a10 tx=0x7fa004009190 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:12.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.671+0000 7f9ffbfff700 1 -- 172.21.15.16:0/4274117908 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa00805bf10 con 0x7fa014103ea0 2024-04-02T04:49:12.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.965+0000 7fa0193bf700 1 -- 172.21.15.16:0/4274117908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa0141098e0 con 0x7fa014103ea0 2024-04-02T04:49:12.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.966+0000 7f9ffbfff700 1 -- 172.21.15.16:0/4274117908 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7fa00805baa0 con 0x7fa014103ea0 2024-04-02T04:49:12.969 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:49:12.969 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": { 2024-04-02T04:49:12.969 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:49:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:49:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": { 2024-04-02T04:49:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:49:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:49:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": { 2024-04-02T04:49:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 6 2024-04-02T04:49:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:49:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-04-02T04:49:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 3, 2024-04-02T04:49:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 1 2024-04-02T04:49:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:49:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: "overall": { 2024-04-02T04:49:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 3, 2024-04-02T04:49:12.971 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 11 2024-04-02T04:49:12.971 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-02T04:49:12.971 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:49:12.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.968+0000 7fa0193bf700 1 -- 172.21.15.16:0/4274117908 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f9ffc06ddc0 msgr2=0x7f9ffc070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:12.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.968+0000 7fa0193bf700 1 --2- 172.21.15.16:0/4274117908 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f9ffc06ddc0 0x7f9ffc070280 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa004009a10 tx=0x7fa004009190 comp rx=0 tx=0).stop 2024-04-02T04:49:12.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.968+0000 7fa0193bf700 1 -- 172.21.15.16:0/4274117908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa014103ea0 msgr2=0x7fa0140ff520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:12.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.968+0000 7fa0193bf700 1 --2- 172.21.15.16:0/4274117908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa014103ea0 0x7fa0140ff520 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa0080092e0 tx=0x7fa00800bd70 comp rx=0 tx=0).stop 2024-04-02T04:49:12.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.969+0000 7fa0193bf700 1 -- 172.21.15.16:0/4274117908 shutdown_connections 2024-04-02T04:49:12.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.969+0000 7fa0193bf700 1 --2- 172.21.15.16:0/4274117908 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f9ffc06ddc0 0x7f9ffc070280 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:12.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.969+0000 7fa0193bf700 1 --2- 172.21.15.16:0/4274117908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa014101570 0x7fa014100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:12.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.969+0000 7fa0193bf700 1 --2- 172.21.15.16:0/4274117908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa014103ea0 0x7fa0140ff520 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:12.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.969+0000 7fa0193bf700 1 -- 172.21.15.16:0/4274117908 >> 172.21.15.16:0/4274117908 conn(0x7fa0140faf80 msgr2=0x7fa0140fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:12.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.969+0000 7fa0193bf700 1 -- 172.21.15.16:0/4274117908 shutdown_connections 2024-04-02T04:49:12.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:12.969+0000 7fa0193bf700 1 -- 172.21.15.16:0/4274117908 wait complete. 2024-04-02T04:49:13.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.088+0000 7f98a6312700 1 -- 172.21.15.16:0/1143851907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98a0074d60 msgr2=0x7f98a00731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:13.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.088+0000 7f98a6312700 1 --2- 172.21.15.16:0/1143851907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98a0074d60 0x7f98a00731c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9890009910 tx=0x7f9890009c20 comp rx=0 tx=0).stop 2024-04-02T04:49:13.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.088+0000 7f98a6312700 1 -- 172.21.15.16:0/1143851907 shutdown_connections 2024-04-02T04:49:13.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.088+0000 7f98a6312700 1 --2- 172.21.15.16:0/1143851907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98a0073790 0x7f98a0073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:13.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.088+0000 7f98a6312700 1 --2- 172.21.15.16:0/1143851907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98a0074d60 0x7f98a00731c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:13.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.088+0000 7f98a6312700 1 -- 172.21.15.16:0/1143851907 >> 172.21.15.16:0/1143851907 conn(0x7f98a00fc0a0 msgr2=0x7f98a00fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:13.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.088+0000 7f98a6312700 1 -- 172.21.15.16:0/1143851907 shutdown_connections 2024-04-02T04:49:13.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.088+0000 7f98a6312700 1 -- 172.21.15.16:0/1143851907 wait complete. 2024-04-02T04:49:13.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.089+0000 7f98a6312700 1 Processor -- start 2024-04-02T04:49:13.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.089+0000 7f98a6312700 1 -- start start 2024-04-02T04:49:13.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.089+0000 7f98a6312700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98a0073790 0x7f98a01986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:13.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.089+0000 7f98a6312700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98a0074d60 0x7f98a0198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:13.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.089+0000 7f98a6312700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a0199230 con 0x7f98a0073790 2024-04-02T04:49:13.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.089+0000 7f98a6312700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a0199370 con 0x7f98a0074d60 2024-04-02T04:49:13.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.089+0000 7f989ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98a0073790 0x7f98a01986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:13.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.089+0000 7f989ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98a0073790 0x7f98a01986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57664/0 (socket says 172.21.15.16:57664) 2024-04-02T04:49:13.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.089+0000 7f989ffff700 1 -- 172.21.15.16:0/3082706411 learned_addr learned my addr 172.21.15.16:0/3082706411 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:49:13.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.089+0000 7f989f7fe700 1 --2- 172.21.15.16:0/3082706411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98a0074d60 0x7f98a0198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:13.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.090+0000 7f989ffff700 1 -- 172.21.15.16:0/3082706411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98a0074d60 msgr2=0x7f98a0198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:13.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.090+0000 7f989ffff700 1 --2- 172.21.15.16:0/3082706411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98a0074d60 0x7f98a0198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:13.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.090+0000 7f989ffff700 1 -- 172.21.15.16:0/3082706411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98900095f0 con 0x7f98a0073790 2024-04-02T04:49:13.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.090+0000 7f989ffff700 1 --2- 172.21.15.16:0/3082706411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98a0073790 0x7f98a01986d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f98900092e0 tx=0x7f989000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:13.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.090+0000 7f989d7fa700 1 -- 172.21.15.16:0/3082706411 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f989001d070 con 0x7f98a0073790 2024-04-02T04:49:13.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.090+0000 7f989d7fa700 1 -- 172.21.15.16:0/3082706411 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f989000f460 con 0x7f98a0073790 2024-04-02T04:49:13.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.090+0000 7f98a6312700 1 -- 172.21.15.16:0/3082706411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a019dd20 con 0x7f98a0073790 2024-04-02T04:49:13.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.090+0000 7f989d7fa700 1 -- 172.21.15.16:0/3082706411 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9890021620 con 0x7f98a0073790 2024-04-02T04:49:13.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.090+0000 7f98a6312700 1 -- 172.21.15.16:0/3082706411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a019e240 con 0x7f98a0073790 2024-04-02T04:49:13.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.091+0000 7f98a6312700 1 -- 172.21.15.16:0/3082706411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98a0192870 con 0x7f98a0073790 2024-04-02T04:49:13.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.092+0000 7f989d7fa700 1 -- 172.21.15.16:0/3082706411 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f989000f5d0 con 0x7f98a0073790 2024-04-02T04:49:13.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.092+0000 7f989d7fa700 1 --2- 172.21.15.16:0/3082706411 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f988806dd60 0x7f9888070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:13.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.092+0000 7f989d7fa700 1 -- 172.21.15.16:0/3082706411 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f989008ee20 con 0x7f98a0073790 2024-04-02T04:49:13.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.092+0000 7f989f7fe700 1 --2- 172.21.15.16:0/3082706411 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f988806dd60 0x7f9888070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:13.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.093+0000 7f989f7fe700 1 --2- 172.21.15.16:0/3082706411 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f988806dd60 0x7f9888070220 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9894005950 tx=0x7f989400b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:13.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.097+0000 7f989d7fa700 1 -- 172.21.15.16:0/3082706411 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f989005be40 con 0x7f98a0073790 2024-04-02T04:49:13.282 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:13 smithi016 ceph-mon[93230]: from='client.34408 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:49:13.300 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:13 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:13.300 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:13 smithi016 ceph-mon[93230]: from='client.34412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:49:13.300 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:13 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/4274117908' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:13.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.349+0000 7f98a6312700 1 -- 172.21.15.16:0/3082706411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f98a004ea80 con 0x7f98a0073790 2024-04-02T04:49:13.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.350+0000 7f989d7fa700 1 -- 172.21.15.16:0/3082706411 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1874 (secure 0 0 0) 0x7f989005b9d0 con 0x7f98a0073790 2024-04-02T04:49:13.354 INFO:teuthology.orchestra.run.smithi016.stdout:e17 2024-04-02T04:49:13.354 INFO:teuthology.orchestra.run.smithi016.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:49:13.354 INFO:teuthology.orchestra.run.smithi016.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:49:13.354 INFO:teuthology.orchestra.run.smithi016.stdout:legacy client fscid: 1 2024-04-02T04:49:13.354 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:49:13.354 INFO:teuthology.orchestra.run.smithi016.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:49:13.354 INFO:teuthology.orchestra.run.smithi016.stdout:fs_name cephfs 2024-04-02T04:49:13.354 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 15 2024-04-02T04:49:13.354 INFO:teuthology.orchestra.run.smithi016.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:49:13.354 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-04-02T04:39:30.740168+0000 2024-04-02T04:49:13.354 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-04-02T04:39:49.875319+0000 2024-04-02T04:49:13.354 INFO:teuthology.orchestra.run.smithi016.stdout:tableserver 0 2024-04-02T04:49:13.355 INFO:teuthology.orchestra.run.smithi016.stdout:root 0 2024-04-02T04:49:13.355 INFO:teuthology.orchestra.run.smithi016.stdout:session_timeout 60 2024-04-02T04:49:13.355 INFO:teuthology.orchestra.run.smithi016.stdout:session_autoclose 300 2024-04-02T04:49:13.355 INFO:teuthology.orchestra.run.smithi016.stdout:max_file_size 1099511627776 2024-04-02T04:49:13.355 INFO:teuthology.orchestra.run.smithi016.stdout:max_xattr_size 65536 2024-04-02T04:49:13.355 INFO:teuthology.orchestra.run.smithi016.stdout:required_client_features {} 2024-04-02T04:49:13.355 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure 0 2024-04-02T04:49:13.355 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure_osd_epoch 46 2024-04-02T04:49:13.355 INFO:teuthology.orchestra.run.smithi016.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:49:13.355 INFO:teuthology.orchestra.run.smithi016.stdout:max_mds 1 2024-04-02T04:49:13.355 INFO:teuthology.orchestra.run.smithi016.stdout:in 0 2024-04-02T04:49:13.355 INFO:teuthology.orchestra.run.smithi016.stdout:up {0=24285} 2024-04-02T04:49:13.355 INFO:teuthology.orchestra.run.smithi016.stdout:failed 2024-04-02T04:49:13.355 INFO:teuthology.orchestra.run.smithi016.stdout:damaged 2024-04-02T04:49:13.355 INFO:teuthology.orchestra.run.smithi016.stdout:stopped 2024-04-02T04:49:13.356 INFO:teuthology.orchestra.run.smithi016.stdout:data_pools [3] 2024-04-02T04:49:13.356 INFO:teuthology.orchestra.run.smithi016.stdout:metadata_pool 2 2024-04-02T04:49:13.356 INFO:teuthology.orchestra.run.smithi016.stdout:inline_data enabled 2024-04-02T04:49:13.356 INFO:teuthology.orchestra.run.smithi016.stdout:balancer 2024-04-02T04:49:13.356 INFO:teuthology.orchestra.run.smithi016.stdout:bal_rank_mask -1 2024-04-02T04:49:13.356 INFO:teuthology.orchestra.run.smithi016.stdout:standby_count_wanted 1 2024-04-02T04:49:13.356 INFO:teuthology.orchestra.run.smithi016.stdout:qdb_cluster leader: 0 members: 2024-04-02T04:49:13.356 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.zwrfoo{0:24285} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.27:6824/1461455976,v1:172.21.15.27:6825/1461455976] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:49:13.356 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:49:13.356 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:49:13.356 INFO:teuthology.orchestra.run.smithi016.stdout:Standby daemons: 2024-04-02T04:49:13.356 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:49:13.356 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.fyzrnm{-1:14488} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.16:6828/2578719859,v1:172.21.15.16:6829/2578719859] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:49:13.356 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.sdublg{-1:24295} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.27:6826/4177192740,v1:172.21.15.27:6827/4177192740] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:49:13.357 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.uhgkgg{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.16:6826/81744708,v1:172.21.15.16:6827/81744708] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T04:49:13.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.353+0000 7f98a6312700 1 -- 172.21.15.16:0/3082706411 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f988806dd60 msgr2=0x7f9888070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:13.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.353+0000 7f98a6312700 1 --2- 172.21.15.16:0/3082706411 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f988806dd60 0x7f9888070220 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9894005950 tx=0x7f989400b410 comp rx=0 tx=0).stop 2024-04-02T04:49:13.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.353+0000 7f98a6312700 1 -- 172.21.15.16:0/3082706411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98a0073790 msgr2=0x7f98a01986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:13.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.353+0000 7f98a6312700 1 --2- 172.21.15.16:0/3082706411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98a0073790 0x7f98a01986d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f98900092e0 tx=0x7f989000bd70 comp rx=0 tx=0).stop 2024-04-02T04:49:13.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.353+0000 7f98a6312700 1 -- 172.21.15.16:0/3082706411 shutdown_connections 2024-04-02T04:49:13.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.353+0000 7f98a6312700 1 --2- 172.21.15.16:0/3082706411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98a0073790 0x7f98a01986d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:13.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.353+0000 7f98a6312700 1 --2- 172.21.15.16:0/3082706411 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f988806dd60 0x7f9888070220 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:13.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.353+0000 7f98a6312700 1 --2- 172.21.15.16:0/3082706411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98a0074d60 0x7f98a0198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:13.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.353+0000 7f98a6312700 1 -- 172.21.15.16:0/3082706411 >> 172.21.15.16:0/3082706411 conn(0x7f98a00fc0a0 msgr2=0x7f98a0106d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:13.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.353+0000 7f98a6312700 1 -- 172.21.15.16:0/3082706411 shutdown_connections 2024-04-02T04:49:13.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.353+0000 7f98a6312700 1 -- 172.21.15.16:0/3082706411 wait complete. 2024-04-02T04:49:13.358 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 17 2024-04-02T04:49:13.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.460+0000 7f073f225700 1 -- 172.21.15.16:0/3375407003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0738100f80 msgr2=0x7f07381013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:13.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.460+0000 7f073f225700 1 --2- 172.21.15.16:0/3375407003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0738100f80 0x7f07381013a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0728009910 tx=0x7f0728009c20 comp rx=0 tx=0).stop 2024-04-02T04:49:13.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.460+0000 7f073f225700 1 -- 172.21.15.16:0/3375407003 shutdown_connections 2024-04-02T04:49:13.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.460+0000 7f073f225700 1 --2- 172.21.15.16:0/3375407003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07380ffcc0 0x7f0738100120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:13.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.460+0000 7f073f225700 1 --2- 172.21.15.16:0/3375407003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0738100f80 0x7f07381013a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:13.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.460+0000 7f073f225700 1 -- 172.21.15.16:0/3375407003 >> 172.21.15.16:0/3375407003 conn(0x7f07380fa730 msgr2=0x7f07380fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:13.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.460+0000 7f073f225700 1 -- 172.21.15.16:0/3375407003 shutdown_connections 2024-04-02T04:49:13.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.460+0000 7f073f225700 1 -- 172.21.15.16:0/3375407003 wait complete. 2024-04-02T04:49:13.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.460+0000 7f073f225700 1 Processor -- start 2024-04-02T04:49:13.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.460+0000 7f073f225700 1 -- start start 2024-04-02T04:49:13.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.460+0000 7f073f225700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07380ffcc0 0x7f0738103a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:13.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.460+0000 7f073f225700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0738100f80 0x7f07381020b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:13.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.461+0000 7f073f225700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07381025f0 con 0x7f07380ffcc0 2024-04-02T04:49:13.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.461+0000 7f073f225700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0738102730 con 0x7f0738100f80 2024-04-02T04:49:13.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.461+0000 7f073cfc1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07380ffcc0 0x7f0738103a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:13.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.461+0000 7f073cfc1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07380ffcc0 0x7f0738103a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57682/0 (socket says 172.21.15.16:57682) 2024-04-02T04:49:13.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.461+0000 7f073cfc1700 1 -- 172.21.15.16:0/3665983639 learned_addr learned my addr 172.21.15.16:0/3665983639 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:49:13.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.461+0000 7f0737fff700 1 --2- 172.21.15.16:0/3665983639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0738100f80 0x7f07381020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:13.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.461+0000 7f073cfc1700 1 -- 172.21.15.16:0/3665983639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0738100f80 msgr2=0x7f07381020b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:13.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.461+0000 7f073cfc1700 1 --2- 172.21.15.16:0/3665983639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0738100f80 0x7f07381020b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:13.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.461+0000 7f073cfc1700 1 -- 172.21.15.16:0/3665983639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07280095f0 con 0x7f07380ffcc0 2024-04-02T04:49:13.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.461+0000 7f073cfc1700 1 --2- 172.21.15.16:0/3665983639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07380ffcc0 0x7f0738103a10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0728005f00 tx=0x7f072800f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:13.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.461+0000 7f0735ffb700 1 -- 172.21.15.16:0/3665983639 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f072801d070 con 0x7f07380ffcc0 2024-04-02T04:49:13.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.461+0000 7f0735ffb700 1 -- 172.21.15.16:0/3665983639 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f072800fe30 con 0x7f07380ffcc0 2024-04-02T04:49:13.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.461+0000 7f073f225700 1 -- 172.21.15.16:0/3665983639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07381029b0 con 0x7f07380ffcc0 2024-04-02T04:49:13.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.461+0000 7f0735ffb700 1 -- 172.21.15.16:0/3665983639 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0728017400 con 0x7f07380ffcc0 2024-04-02T04:49:13.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.461+0000 7f073f225700 1 -- 172.21.15.16:0/3665983639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0738102f20 con 0x7f07380ffcc0 2024-04-02T04:49:13.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.462+0000 7f073f225700 1 -- 172.21.15.16:0/3665983639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f073804ea80 con 0x7f07380ffcc0 2024-04-02T04:49:13.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.463+0000 7f0735ffb700 1 -- 172.21.15.16:0/3665983639 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f07280038a0 con 0x7f07380ffcc0 2024-04-02T04:49:13.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.463+0000 7f0735ffb700 1 --2- 172.21.15.16:0/3665983639 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f072406dd60 0x7f0724070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:13.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.464+0000 7f0735ffb700 1 -- 172.21.15.16:0/3665983639 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f072808ee70 con 0x7f07380ffcc0 2024-04-02T04:49:13.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.464+0000 7f0737fff700 1 --2- 172.21.15.16:0/3665983639 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f072406dd60 0x7f0724070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:13.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.464+0000 7f0737fff700 1 --2- 172.21.15.16:0/3665983639 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f072406dd60 0x7f0724070220 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0720009b30 tx=0x7f0720009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:13.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.468+0000 7f0735ffb700 1 -- 172.21.15.16:0/3665983639 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f072805bf30 con 0x7f07380ffcc0 2024-04-02T04:49:13.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:13 smithi027 ceph-mon[83036]: from='client.34408 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:49:13.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:13 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:13.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:13 smithi027 ceph-mon[83036]: from='client.34412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:49:13.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:13 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/4274117908' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:13.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.695+0000 7f073f225700 1 -- 172.21.15.16:0/3665983639 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0738102f20 con 0x7f072406dd60 2024-04-02T04:49:13.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.697+0000 7f0735ffb700 1 -- 172.21.15.16:0/3665983639 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0738102f20 con 0x7f072406dd60 2024-04-02T04:49:13.700 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:49:13.700 INFO:teuthology.orchestra.run.smithi016.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:49:13.700 INFO:teuthology.orchestra.run.smithi016.stdout: "in_progress": true, 2024-04-02T04:49:13.700 INFO:teuthology.orchestra.run.smithi016.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-02T04:49:13.700 INFO:teuthology.orchestra.run.smithi016.stdout: "services_complete": [ 2024-04-02T04:49:13.700 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr", 2024-04-02T04:49:13.701 INFO:teuthology.orchestra.run.smithi016.stdout: "osd", 2024-04-02T04:49:13.701 INFO:teuthology.orchestra.run.smithi016.stdout: "mon", 2024-04-02T04:49:13.701 INFO:teuthology.orchestra.run.smithi016.stdout: "crash" 2024-04-02T04:49:13.701 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-04-02T04:49:13.701 INFO:teuthology.orchestra.run.smithi016.stdout: "progress": "12/23 daemons upgraded", 2024-04-02T04:49:13.701 INFO:teuthology.orchestra.run.smithi016.stdout: "message": "Currently upgrading mds daemons", 2024-04-02T04:49:13.701 INFO:teuthology.orchestra.run.smithi016.stdout: "is_paused": false 2024-04-02T04:49:13.701 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:49:13.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.699+0000 7f073f225700 1 -- 172.21.15.16:0/3665983639 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f072406dd60 msgr2=0x7f0724070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:13.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.699+0000 7f073f225700 1 --2- 172.21.15.16:0/3665983639 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f072406dd60 0x7f0724070220 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0720009b30 tx=0x7f0720009260 comp rx=0 tx=0).stop 2024-04-02T04:49:13.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.699+0000 7f073f225700 1 -- 172.21.15.16:0/3665983639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07380ffcc0 msgr2=0x7f0738103a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:13.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.699+0000 7f073f225700 1 --2- 172.21.15.16:0/3665983639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07380ffcc0 0x7f0738103a10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0728005f00 tx=0x7f072800f510 comp rx=0 tx=0).stop 2024-04-02T04:49:13.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.699+0000 7f073f225700 1 -- 172.21.15.16:0/3665983639 shutdown_connections 2024-04-02T04:49:13.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.699+0000 7f073f225700 1 --2- 172.21.15.16:0/3665983639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07380ffcc0 0x7f0738103a10 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:13.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.699+0000 7f073f225700 1 --2- 172.21.15.16:0/3665983639 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f072406dd60 0x7f0724070220 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:13.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.699+0000 7f073f225700 1 --2- 172.21.15.16:0/3665983639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0738100f80 0x7f07381020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:13.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.699+0000 7f073f225700 1 -- 172.21.15.16:0/3665983639 >> 172.21.15.16:0/3665983639 conn(0x7f07380fa730 msgr2=0x7f07380fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:13.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.699+0000 7f073f225700 1 -- 172.21.15.16:0/3665983639 shutdown_connections 2024-04-02T04:49:13.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.699+0000 7f073f225700 1 -- 172.21.15.16:0/3665983639 wait complete. 2024-04-02T04:49:13.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.805+0000 7f5ab1273700 1 -- 172.21.15.16:0/3751114634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aac0ff9e0 msgr2=0x7f5aac0ffe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:13.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.805+0000 7f5ab1273700 1 --2- 172.21.15.16:0/3751114634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aac0ff9e0 0x7f5aac0ffe00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c00b0a0 tx=0x7f5a9c00b3b0 comp rx=0 tx=0).stop 2024-04-02T04:49:13.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.805+0000 7f5ab1273700 1 -- 172.21.15.16:0/3751114634 shutdown_connections 2024-04-02T04:49:13.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.805+0000 7f5ab1273700 1 --2- 172.21.15.16:0/3751114634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5aac100be0 0x7f5aac101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:13.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.805+0000 7f5ab1273700 1 --2- 172.21.15.16:0/3751114634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aac0ff9e0 0x7f5aac0ffe00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:13.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.805+0000 7f5ab1273700 1 -- 172.21.15.16:0/3751114634 >> 172.21.15.16:0/3751114634 conn(0x7f5aac0faf80 msgr2=0x7f5aac0fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:13.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.805+0000 7f5ab1273700 1 -- 172.21.15.16:0/3751114634 shutdown_connections 2024-04-02T04:49:13.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.805+0000 7f5ab1273700 1 -- 172.21.15.16:0/3751114634 wait complete. 2024-04-02T04:49:13.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.805+0000 7f5ab1273700 1 Processor -- start 2024-04-02T04:49:13.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.805+0000 7f5ab1273700 1 -- start start 2024-04-02T04:49:13.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.806+0000 7f5ab1273700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5aac0ff9e0 0x7f5aac103920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:13.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.806+0000 7f5ab1273700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aac100be0 0x7f5aac101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:13.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.806+0000 7f5ab1273700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aac19a120 con 0x7f5aac100be0 2024-04-02T04:49:13.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.806+0000 7f5ab1273700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aac19a260 con 0x7f5aac0ff9e0 2024-04-02T04:49:13.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.806+0000 7f5aaa7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aac100be0 0x7f5aac101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:13.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.806+0000 7f5aaa7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aac100be0 0x7f5aac101fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57708/0 (socket says 172.21.15.16:57708) 2024-04-02T04:49:13.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.806+0000 7f5aaa7fc700 1 -- 172.21.15.16:0/2845065561 learned_addr learned my addr 172.21.15.16:0/2845065561 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:49:13.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.806+0000 7f5aaaffd700 1 --2- 172.21.15.16:0/2845065561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5aac0ff9e0 0x7f5aac103920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:13.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.806+0000 7f5aaa7fc700 1 -- 172.21.15.16:0/2845065561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5aac0ff9e0 msgr2=0x7f5aac103920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:13.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.806+0000 7f5aaa7fc700 1 --2- 172.21.15.16:0/2845065561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5aac0ff9e0 0x7f5aac103920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:13.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.806+0000 7f5aaa7fc700 1 -- 172.21.15.16:0/2845065561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a9c009d00 con 0x7f5aac100be0 2024-04-02T04:49:13.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.806+0000 7f5aaa7fc700 1 --2- 172.21.15.16:0/2845065561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aac100be0 0x7f5aac101fc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5a9400c8a0 tx=0x7f5a9400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:13.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.806+0000 7f5aa3fff700 1 -- 172.21.15.16:0/2845065561 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a940078e0 con 0x7f5aac100be0 2024-04-02T04:49:13.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.806+0000 7f5aa3fff700 1 -- 172.21.15.16:0/2845065561 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a9400f450 con 0x7f5aac100be0 2024-04-02T04:49:13.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.806+0000 7f5aa3fff700 1 -- 172.21.15.16:0/2845065561 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a9400e5e0 con 0x7f5aac100be0 2024-04-02T04:49:13.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.806+0000 7f5ab1273700 1 -- 172.21.15.16:0/2845065561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aac102500 con 0x7f5aac100be0 2024-04-02T04:49:13.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.807+0000 7f5ab1273700 1 -- 172.21.15.16:0/2845065561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aac102aa0 con 0x7f5aac100be0 2024-04-02T04:49:13.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.808+0000 7f5ab1273700 1 -- 172.21.15.16:0/2845065561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aac04ea80 con 0x7f5aac100be0 2024-04-02T04:49:13.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.808+0000 7f5aa3fff700 1 -- 172.21.15.16:0/2845065561 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f5a9400f5c0 con 0x7f5aac100be0 2024-04-02T04:49:13.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.808+0000 7f5aa3fff700 1 --2- 172.21.15.16:0/2845065561 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5a9806dd60 0x7f5a98070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:13.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.809+0000 7f5aaaffd700 1 --2- 172.21.15.16:0/2845065561 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5a9806dd60 0x7f5a98070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:13.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.809+0000 7f5aa3fff700 1 -- 172.21.15.16:0/2845065561 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f5a9408dae0 con 0x7f5aac100be0 2024-04-02T04:49:13.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.809+0000 7f5aaaffd700 1 --2- 172.21.15.16:0/2845065561 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5a9806dd60 0x7f5a98070220 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c009fd0 tx=0x7f5a9c006000 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:13.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:13.813+0000 7f5aa3fff700 1 -- 172.21.15.16:0/2845065561 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5a9405aba0 con 0x7f5aac100be0 2024-04-02T04:49:14.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:14.132+0000 7f5ab1273700 1 -- 172.21.15.16:0/2845065561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5aac102d20 con 0x7f5aac100be0 2024-04-02T04:49:14.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:14.133+0000 7f5aa3fff700 1 -- 172.21.15.16:0/2845065561 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f5a9405a730 con 0x7f5aac100be0 2024-04-02T04:49:14.136 INFO:teuthology.orchestra.run.smithi016.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-02T04:49:14.136 INFO:teuthology.orchestra.run.smithi016.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-02T04:49:14.136 INFO:teuthology.orchestra.run.smithi016.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-02T04:49:14.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:14.135+0000 7f5ab1273700 1 -- 172.21.15.16:0/2845065561 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5a9806dd60 msgr2=0x7f5a98070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:14.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:14.135+0000 7f5ab1273700 1 --2- 172.21.15.16:0/2845065561 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5a9806dd60 0x7f5a98070220 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c009fd0 tx=0x7f5a9c006000 comp rx=0 tx=0).stop 2024-04-02T04:49:14.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:14.135+0000 7f5ab1273700 1 -- 172.21.15.16:0/2845065561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aac100be0 msgr2=0x7f5aac101fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:14.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:14.135+0000 7f5ab1273700 1 --2- 172.21.15.16:0/2845065561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aac100be0 0x7f5aac101fc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5a9400c8a0 tx=0x7f5a9400cbb0 comp rx=0 tx=0).stop 2024-04-02T04:49:14.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:14.135+0000 7f5ab1273700 1 -- 172.21.15.16:0/2845065561 shutdown_connections 2024-04-02T04:49:14.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:14.135+0000 7f5ab1273700 1 --2- 172.21.15.16:0/2845065561 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5a9806dd60 0x7f5a98070220 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:14.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:14.135+0000 7f5ab1273700 1 --2- 172.21.15.16:0/2845065561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5aac0ff9e0 0x7f5aac103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:14.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:14.135+0000 7f5ab1273700 1 --2- 172.21.15.16:0/2845065561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aac100be0 0x7f5aac101fc0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:14.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:14.135+0000 7f5ab1273700 1 -- 172.21.15.16:0/2845065561 >> 172.21.15.16:0/2845065561 conn(0x7f5aac0faf80 msgr2=0x7f5aac069020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:14.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:14.135+0000 7f5ab1273700 1 -- 172.21.15.16:0/2845065561 shutdown_connections 2024-04-02T04:49:14.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:14.135+0000 7f5ab1273700 1 -- 172.21.15.16:0/2845065561 wait complete. 2024-04-02T04:49:14.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:14 smithi016 ceph-mon[93230]: from='client.34416 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:49:14.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:14 smithi016 ceph-mon[93230]: pgmap v183: 65 pgs: 65 active+clean; 260 MiB data, 1005 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:14.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:14 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/3082706411' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:49:14.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:14 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:14.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:14 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/2845065561' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:49:14.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:14 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:14.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:14 smithi027 ceph-mon[83036]: from='client.34416 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:49:14.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:14 smithi027 ceph-mon[83036]: pgmap v183: 65 pgs: 65 active+clean; 260 MiB data, 1005 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:14.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:14 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/3082706411' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:49:14.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:14 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:14.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:14 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/2845065561' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:49:14.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:14 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:15.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:15 smithi016 ceph-mon[93230]: from='client.34428 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:49:15.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:15 smithi027 ceph-mon[83036]: from='client.34428 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:49:16.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:16 smithi016 ceph-mon[93230]: pgmap v184: 65 pgs: 65 active+clean; 260 MiB data, 1005 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:16.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:16 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:16.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:16 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:16.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:16 smithi027 ceph-mon[83036]: pgmap v184: 65 pgs: 65 active+clean; 260 MiB data, 1005 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:16.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:16 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:16.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:16 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:18.300 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:18 smithi016 ceph-mon[93230]: pgmap v185: 65 pgs: 65 active+clean; 260 MiB data, 1005 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:18.300 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:18 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:18.301 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:18 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:49:18.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:18 smithi027 ceph-mon[83036]: pgmap v185: 65 pgs: 65 active+clean; 260 MiB data, 1005 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:18.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:18 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:18.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:18 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:49:19.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:19 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:19.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:19 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:19.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:19 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:49:19.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:19 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:49:19.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:19 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:19.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:19 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:49:19.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:19 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:19.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:19 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:19.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:19 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:19.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:19 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:19.811 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:19 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi016.fyzrnm"]}]: dispatch 2024-04-02T04:49:19.811 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:19 smithi016 ceph-mon[93230]: Upgrade: It appears safe to stop mds.cephfs.smithi016.fyzrnm 2024-04-02T04:49:19.811 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:19 smithi016 ceph-mon[93230]: pgmap v186: 65 pgs: 65 active+clean; 260 MiB data, 1005 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:19.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:19 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:19.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:19 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:19.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:19 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:49:19.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:19 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:49:19.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:19 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:19.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:19 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:49:19.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:19 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:19.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:19 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:19.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:19 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:19.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:19 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:19.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:19 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi016.fyzrnm"]}]: dispatch 2024-04-02T04:49:19.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:19 smithi027 ceph-mon[83036]: Upgrade: It appears safe to stop mds.cephfs.smithi016.fyzrnm 2024-04-02T04:49:19.897 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:19 smithi027 ceph-mon[83036]: pgmap v186: 65 pgs: 65 active+clean; 260 MiB data, 1005 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:21.394 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:21 smithi016 ceph-mon[93230]: Upgrade: Updating mds.cephfs.smithi016.fyzrnm 2024-04-02T04:49:21.394 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:21 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:21.394 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:21 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi016.fyzrnm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:49:21.394 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:21 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:49:21.394 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:21 smithi016 ceph-mon[93230]: Deploying daemon mds.cephfs.smithi016.fyzrnm on smithi016 2024-04-02T04:49:21.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:21 smithi027 ceph-mon[83036]: Upgrade: Updating mds.cephfs.smithi016.fyzrnm 2024-04-02T04:49:21.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:21 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:21.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:21 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi016.fyzrnm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:49:21.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:21 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:49:21.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:21 smithi027 ceph-mon[83036]: Deploying daemon mds.cephfs.smithi016.fyzrnm on smithi016 2024-04-02T04:49:22.213 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:22 smithi016 ceph-mon[93230]: pgmap v187: 65 pgs: 65 active+clean; 260 MiB data, 1005 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:22.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:22 smithi027 ceph-mon[83036]: pgmap v187: 65 pgs: 65 active+clean; 260 MiB data, 1005 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:24.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:24 smithi016 ceph-mon[93230]: pgmap v188: 65 pgs: 65 active+clean; 260 MiB data, 1005 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:24.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:24 smithi016 ceph-mon[93230]: osdmap e82: 6 total, 6 up, 6 in 2024-04-02T04:49:24.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:24 smithi016 ceph-mon[93230]: fsmap cephfs:1 {0=cephfs.smithi027.zwrfoo=up:active} 2 up:standby 2024-04-02T04:49:24.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:24 smithi027 ceph-mon[83036]: pgmap v188: 65 pgs: 65 active+clean; 260 MiB data, 1005 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:24.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:24 smithi027 ceph-mon[83036]: osdmap e82: 6 total, 6 up, 6 in 2024-04-02T04:49:24.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:24 smithi027 ceph-mon[83036]: fsmap cephfs:1 {0=cephfs.smithi027.zwrfoo=up:active} 2 up:standby 2024-04-02T04:49:26.497 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:26 smithi016 ceph-mon[93230]: pgmap v190: 65 pgs: 65 active+clean; 260 MiB data, 1005 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:26.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:26 smithi027 ceph-mon[83036]: pgmap v190: 65 pgs: 65 active+clean; 260 MiB data, 1005 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:28.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:28 smithi016 ceph-mon[93230]: pgmap v191: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:28.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:28 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:28.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:28 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:28.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:28 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:49:28.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:28 smithi027 ceph-mon[83036]: pgmap v191: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:28.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:28 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:28.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:28 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:28.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:28 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:49:29.479 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:29 smithi016 ceph-mon[93230]: mds.? [v2:172.21.15.16:6828/782430688,v1:172.21.15.16:6829/782430688] up:boot 2024-04-02T04:49:29.479 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:29 smithi016 ceph-mon[93230]: fsmap cephfs:1 {0=cephfs.smithi027.zwrfoo=up:active} 3 up:standby 2024-04-02T04:49:29.479 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:29 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi016.fyzrnm"}]: dispatch 2024-04-02T04:49:29.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:29 smithi027 ceph-mon[83036]: mds.? [v2:172.21.15.16:6828/782430688,v1:172.21.15.16:6829/782430688] up:boot 2024-04-02T04:49:29.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:29 smithi027 ceph-mon[83036]: fsmap cephfs:1 {0=cephfs.smithi027.zwrfoo=up:active} 3 up:standby 2024-04-02T04:49:29.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:29 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi016.fyzrnm"}]: dispatch 2024-04-02T04:49:30.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:30 smithi016 ceph-mon[93230]: pgmap v192: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:30.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:30 smithi027 ceph-mon[83036]: pgmap v192: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:32.277 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:32 smithi016 ceph-mon[93230]: pgmap v193: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:32.277 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:32 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:49:32.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:32 smithi027 ceph-mon[83036]: pgmap v193: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:32.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:32 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:49:34.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:33 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:34.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:33 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:34.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:33 smithi016 ceph-mon[93230]: pgmap v194: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:34.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:33 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:34.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:33 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:34.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:33 smithi027 ceph-mon[83036]: pgmap v194: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:35.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:34 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:35.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:34 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:35.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:34 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:35.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:34 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:36.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:35 smithi016 ceph-mon[93230]: pgmap v195: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:36.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:35 smithi027 ceph-mon[83036]: pgmap v195: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:38.245 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:38 smithi027 ceph-mon[83036]: pgmap v196: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:38.245 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:38 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:38.245 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:38 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:38.246 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:38 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:49:38.246 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:38 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:49:38.246 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:38 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:38.246 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:38 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:49:38.246 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:38 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:38.246 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:38 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:38.246 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:38 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:38.246 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:38 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:38.246 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:38 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi027.zwrfoo"]}]: dispatch 2024-04-02T04:49:38.246 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:38 smithi027 ceph-mon[83036]: Upgrade: It appears safe to stop mds.cephfs.smithi027.zwrfoo 2024-04-02T04:49:38.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:38 smithi016 ceph-mon[93230]: pgmap v196: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:38.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:38 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:38.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:38 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:38.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:38 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:49:38.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:38 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:49:38.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:38 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:38.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:38 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:49:38.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:38 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:38.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:38 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:38.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:38 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:38.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:38 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:38.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:38 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi027.zwrfoo"]}]: dispatch 2024-04-02T04:49:38.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:38 smithi016 ceph-mon[93230]: Upgrade: It appears safe to stop mds.cephfs.smithi027.zwrfoo 2024-04-02T04:49:40.358 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:40 smithi027 ceph-mon[83036]: pgmap v197: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:40.359 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:40 smithi027 ceph-mon[83036]: Upgrade: Updating mds.cephfs.smithi027.zwrfoo 2024-04-02T04:49:40.359 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:40 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:40.359 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:40 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi027.zwrfoo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:49:40.359 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:40 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:49:40.359 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:40 smithi027 ceph-mon[83036]: Deploying daemon mds.cephfs.smithi027.zwrfoo on smithi027 2024-04-02T04:49:40.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:40 smithi016 ceph-mon[93230]: pgmap v197: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:40.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:40 smithi016 ceph-mon[93230]: Upgrade: Updating mds.cephfs.smithi027.zwrfoo 2024-04-02T04:49:40.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:40 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:40.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:40 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi027.zwrfoo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:49:40.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:40 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:49:40.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:40 smithi016 ceph-mon[93230]: Deploying daemon mds.cephfs.smithi027.zwrfoo on smithi027 2024-04-02T04:49:41.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:41 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi016[93226]: 2024-04-02T04:49:41.367+0000 7f131edaa700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T04:49:42.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:42 smithi016 ceph-mon[93230]: pgmap v198: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:42.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:42 smithi016 ceph-mon[93230]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-02T04:49:42.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:42 smithi016 ceph-mon[93230]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T04:49:42.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:42 smithi016 ceph-mon[93230]: osdmap e83: 6 total, 6 up, 6 in 2024-04-02T04:49:42.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:42 smithi016 ceph-mon[93230]: Standby daemon mds.cephfs.smithi027.sdublg assigned to filesystem cephfs as rank 0 2024-04-02T04:49:42.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:42 smithi016 ceph-mon[93230]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-02T04:49:42.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:42 smithi016 ceph-mon[93230]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-02T04:49:42.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:42 smithi016 ceph-mon[93230]: fsmap cephfs:1/1 {0=cephfs.smithi027.sdublg=up:replay} 2 up:standby 2024-04-02T04:49:42.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:42 smithi027 ceph-mon[83036]: pgmap v198: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:49:42.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:42 smithi027 ceph-mon[83036]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-02T04:49:42.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:42 smithi027 ceph-mon[83036]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T04:49:42.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:42 smithi027 ceph-mon[83036]: osdmap e83: 6 total, 6 up, 6 in 2024-04-02T04:49:42.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:42 smithi027 ceph-mon[83036]: Standby daemon mds.cephfs.smithi027.sdublg assigned to filesystem cephfs as rank 0 2024-04-02T04:49:42.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:42 smithi027 ceph-mon[83036]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-02T04:49:42.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:42 smithi027 ceph-mon[83036]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-02T04:49:42.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:42 smithi027 ceph-mon[83036]: fsmap cephfs:1/1 {0=cephfs.smithi027.sdublg=up:replay} 2 up:standby 2024-04-02T04:49:43.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:43 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.zwrfoo"}]: dispatch 2024-04-02T04:49:43.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:43 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.zwrfoo"}]: dispatch 2024-04-02T04:49:44.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.254+0000 7f5f59a81700 1 -- 172.21.15.16:0/1616414425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f54100be0 msgr2=0x7f5f54101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:44.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.254+0000 7f5f59a81700 1 --2- 172.21.15.16:0/1616414425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f54100be0 0x7f5f54101040 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c009910 tx=0x7f5f3c009c20 comp rx=0 tx=0).stop 2024-04-02T04:49:44.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.254+0000 7f5f59a81700 1 -- 172.21.15.16:0/1616414425 shutdown_connections 2024-04-02T04:49:44.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.254+0000 7f5f59a81700 1 --2- 172.21.15.16:0/1616414425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f54100be0 0x7f5f54101040 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:44.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.255+0000 7f5f59a81700 1 --2- 172.21.15.16:0/1616414425 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f540ff9e0 0x7f5f540ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:44.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.255+0000 7f5f59a81700 1 -- 172.21.15.16:0/1616414425 >> 172.21.15.16:0/1616414425 conn(0x7f5f540faf80 msgr2=0x7f5f540fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:44.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.255+0000 7f5f59a81700 1 -- 172.21.15.16:0/1616414425 shutdown_connections 2024-04-02T04:49:44.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.255+0000 7f5f59a81700 1 -- 172.21.15.16:0/1616414425 wait complete. 2024-04-02T04:49:44.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.255+0000 7f5f59a81700 1 Processor -- start 2024-04-02T04:49:44.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.255+0000 7f5f59a81700 1 -- start start 2024-04-02T04:49:44.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.256+0000 7f5f59a81700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f540ff9e0 0x7f5f541953d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:44.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.256+0000 7f5f59a81700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f54100be0 0x7f5f54195910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:44.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.256+0000 7f5f59a81700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f54195f30 con 0x7f5f54100be0 2024-04-02T04:49:44.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.256+0000 7f5f59a81700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f54196070 con 0x7f5f540ff9e0 2024-04-02T04:49:44.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.256+0000 7f5f4affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f54100be0 0x7f5f54195910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:44.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.256+0000 7f5f4affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f54100be0 0x7f5f54195910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48014/0 (socket says 172.21.15.16:48014) 2024-04-02T04:49:44.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.256+0000 7f5f4affd700 1 -- 172.21.15.16:0/243926489 learned_addr learned my addr 172.21.15.16:0/243926489 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:49:44.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.256+0000 7f5f537fe700 1 --2- 172.21.15.16:0/243926489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f540ff9e0 0x7f5f541953d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:44.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.256+0000 7f5f4affd700 1 -- 172.21.15.16:0/243926489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f540ff9e0 msgr2=0x7f5f541953d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:44.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.256+0000 7f5f4affd700 1 --2- 172.21.15.16:0/243926489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f540ff9e0 0x7f5f541953d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:44.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.256+0000 7f5f4affd700 1 -- 172.21.15.16:0/243926489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f3c0095f0 con 0x7f5f54100be0 2024-04-02T04:49:44.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.257+0000 7f5f4affd700 1 --2- 172.21.15.16:0/243926489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f54100be0 0x7f5f54195910 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c005b40 tx=0x7f5f3c01a850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:44.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.257+0000 7f5f517fa700 1 -- 172.21.15.16:0/243926489 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f3c01f070 con 0x7f5f54100be0 2024-04-02T04:49:44.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.257+0000 7f5f517fa700 1 -- 172.21.15.16:0/243926489 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f3c0050d0 con 0x7f5f54100be0 2024-04-02T04:49:44.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.257+0000 7f5f59a81700 1 -- 172.21.15.16:0/243926489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f54069030 con 0x7f5f54100be0 2024-04-02T04:49:44.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.257+0000 7f5f517fa700 1 -- 172.21.15.16:0/243926489 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f3c00f460 con 0x7f5f54100be0 2024-04-02T04:49:44.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.257+0000 7f5f59a81700 1 -- 172.21.15.16:0/243926489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f540695a0 con 0x7f5f54100be0 2024-04-02T04:49:44.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.258+0000 7f5f59a81700 1 -- 172.21.15.16:0/243926489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f5404ea80 con 0x7f5f54100be0 2024-04-02T04:49:44.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.258+0000 7f5f517fa700 1 -- 172.21.15.16:0/243926489 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f5f3c023070 con 0x7f5f54100be0 2024-04-02T04:49:44.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.259+0000 7f5f517fa700 1 --2- 172.21.15.16:0/243926489 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5f4006dd70 0x7f5f40070230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:44.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.259+0000 7f5f517fa700 1 -- 172.21.15.16:0/243926489 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5f3c090ed0 con 0x7f5f54100be0 2024-04-02T04:49:44.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.259+0000 7f5f537fe700 1 --2- 172.21.15.16:0/243926489 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5f4006dd70 0x7f5f40070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:44.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.259+0000 7f5f537fe700 1 --2- 172.21.15.16:0/243926489 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5f4006dd70 0x7f5f40070230 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5f54100a40 tx=0x7f5f44009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:44.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.264+0000 7f5f517fa700 1 -- 172.21.15.16:0/243926489 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5f3c05df40 con 0x7f5f54100be0 2024-04-02T04:49:44.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:44 smithi027 ceph-mon[83036]: pgmap v200: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 0 op/s 2024-04-02T04:49:44.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.490+0000 7f5f59a81700 1 -- 172.21.15.16:0/243926489 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f5418f0b0 con 0x7f5f4006dd70 2024-04-02T04:49:44.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.491+0000 7f5f517fa700 1 -- 172.21.15.16:0/243926489 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f5f5418f0b0 con 0x7f5f4006dd70 2024-04-02T04:49:44.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.494+0000 7f5f59a81700 1 -- 172.21.15.16:0/243926489 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5f4006dd70 msgr2=0x7f5f40070230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:44.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.494+0000 7f5f59a81700 1 --2- 172.21.15.16:0/243926489 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5f4006dd70 0x7f5f40070230 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5f54100a40 tx=0x7f5f44009260 comp rx=0 tx=0).stop 2024-04-02T04:49:44.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.494+0000 7f5f59a81700 1 -- 172.21.15.16:0/243926489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f54100be0 msgr2=0x7f5f54195910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:44.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.494+0000 7f5f59a81700 1 --2- 172.21.15.16:0/243926489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f54100be0 0x7f5f54195910 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c005b40 tx=0x7f5f3c01a850 comp rx=0 tx=0).stop 2024-04-02T04:49:44.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.494+0000 7f5f59a81700 1 -- 172.21.15.16:0/243926489 shutdown_connections 2024-04-02T04:49:44.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.494+0000 7f5f59a81700 1 --2- 172.21.15.16:0/243926489 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5f4006dd70 0x7f5f40070230 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:44.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.494+0000 7f5f59a81700 1 --2- 172.21.15.16:0/243926489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f540ff9e0 0x7f5f541953d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:44.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.494+0000 7f5f59a81700 1 --2- 172.21.15.16:0/243926489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f54100be0 0x7f5f54195910 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:44.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.494+0000 7f5f59a81700 1 -- 172.21.15.16:0/243926489 >> 172.21.15.16:0/243926489 conn(0x7f5f540faf80 msgr2=0x7f5f54104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:44.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.494+0000 7f5f59a81700 1 -- 172.21.15.16:0/243926489 shutdown_connections 2024-04-02T04:49:44.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.494+0000 7f5f59a81700 1 -- 172.21.15.16:0/243926489 wait complete. 2024-04-02T04:49:44.509 INFO:teuthology.orchestra.run.smithi016.stdout:true 2024-04-02T04:49:44.509 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:44 smithi016 ceph-mon[93230]: pgmap v200: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 0 op/s 2024-04-02T04:49:44.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.600+0000 7fcba6a96700 1 -- 172.21.15.16:0/1434951790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba0101e50 msgr2=0x7fcba01022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:44.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.600+0000 7fcba6a96700 1 --2- 172.21.15.16:0/1434951790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba0101e50 0x7fcba01022b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c009910 tx=0x7fcb8c009c20 comp rx=0 tx=0).stop 2024-04-02T04:49:44.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.600+0000 7fcba6a96700 1 -- 172.21.15.16:0/1434951790 shutdown_connections 2024-04-02T04:49:44.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.600+0000 7fcba6a96700 1 --2- 172.21.15.16:0/1434951790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba0101e50 0x7fcba01022b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:44.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.600+0000 7fcba6a96700 1 --2- 172.21.15.16:0/1434951790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcba00fedd0 0x7fcba00ff1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:44.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.600+0000 7fcba6a96700 1 -- 172.21.15.16:0/1434951790 >> 172.21.15.16:0/1434951790 conn(0x7fcba00fa730 msgr2=0x7fcba00fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:44.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.600+0000 7fcba6a96700 1 -- 172.21.15.16:0/1434951790 shutdown_connections 2024-04-02T04:49:44.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.600+0000 7fcba6a96700 1 -- 172.21.15.16:0/1434951790 wait complete. 2024-04-02T04:49:44.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.601+0000 7fcba6a96700 1 Processor -- start 2024-04-02T04:49:44.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.601+0000 7fcba6a96700 1 -- start start 2024-04-02T04:49:44.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.601+0000 7fcba6a96700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba00fedd0 0x7fcba01964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:44.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.601+0000 7fcba6a96700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcba0101e50 0x7fcba0196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:44.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.601+0000 7fcba6a96700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba0197030 con 0x7fcba00fedd0 2024-04-02T04:49:44.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.601+0000 7fcba6a96700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba0197170 con 0x7fcba0101e50 2024-04-02T04:49:44.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.601+0000 7fcba4832700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba00fedd0 0x7fcba01964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:44.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.601+0000 7fcba4832700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba00fedd0 0x7fcba01964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48030/0 (socket says 172.21.15.16:48030) 2024-04-02T04:49:44.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.601+0000 7fcba4832700 1 -- 172.21.15.16:0/2278987900 learned_addr learned my addr 172.21.15.16:0/2278987900 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:49:44.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.601+0000 7fcb9ffff700 1 --2- 172.21.15.16:0/2278987900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcba0101e50 0x7fcba0196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:44.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.602+0000 7fcba4832700 1 -- 172.21.15.16:0/2278987900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcba0101e50 msgr2=0x7fcba0196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:44.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.602+0000 7fcba4832700 1 --2- 172.21.15.16:0/2278987900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcba0101e50 0x7fcba0196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:44.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.602+0000 7fcba4832700 1 -- 172.21.15.16:0/2278987900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8c0095f0 con 0x7fcba00fedd0 2024-04-02T04:49:44.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.602+0000 7fcba4832700 1 --2- 172.21.15.16:0/2278987900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba00fedd0 0x7fcba01964d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcb9400d900 tx=0x7fcb9400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:44.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.602+0000 7fcb9dffb700 1 -- 172.21.15.16:0/2278987900 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb940041f0 con 0x7fcba00fedd0 2024-04-02T04:49:44.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.602+0000 7fcb9dffb700 1 -- 172.21.15.16:0/2278987900 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcb94004350 con 0x7fcba00fedd0 2024-04-02T04:49:44.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.602+0000 7fcb9dffb700 1 -- 172.21.15.16:0/2278987900 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb94003cf0 con 0x7fcba00fedd0 2024-04-02T04:49:44.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.602+0000 7fcba6a96700 1 -- 172.21.15.16:0/2278987900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcba019bbd0 con 0x7fcba00fedd0 2024-04-02T04:49:44.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.602+0000 7fcba6a96700 1 -- 172.21.15.16:0/2278987900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba019c120 con 0x7fcba00fedd0 2024-04-02T04:49:44.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.603+0000 7fcba6a96700 1 -- 172.21.15.16:0/2278987900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcba0190670 con 0x7fcba00fedd0 2024-04-02T04:49:44.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.604+0000 7fcb9dffb700 1 -- 172.21.15.16:0/2278987900 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fcb94009730 con 0x7fcba00fedd0 2024-04-02T04:49:44.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.604+0000 7fcb9dffb700 1 --2- 172.21.15.16:0/2278987900 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fcb9006dd60 0x7fcb90070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:44.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.604+0000 7fcb9dffb700 1 -- 172.21.15.16:0/2278987900 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fcb9408d960 con 0x7fcba00fedd0 2024-04-02T04:49:44.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.604+0000 7fcb9ffff700 1 --2- 172.21.15.16:0/2278987900 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fcb9006dd60 0x7fcb90070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:44.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.604+0000 7fcb9ffff700 1 --2- 172.21.15.16:0/2278987900 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fcb9006dd60 0x7fcb90070220 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c009fd0 tx=0x7fcb8c005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:44.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.609+0000 7fcb9dffb700 1 -- 172.21.15.16:0/2278987900 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fcb9405aa50 con 0x7fcba00fedd0 2024-04-02T04:49:44.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.834+0000 7fcba6a96700 1 -- 172.21.15.16:0/2278987900 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcba002cdf0 con 0x7fcb9006dd60 2024-04-02T04:49:44.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.835+0000 7fcb9dffb700 1 -- 172.21.15.16:0/2278987900 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fcba002cdf0 con 0x7fcb9006dd60 2024-04-02T04:49:44.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.837+0000 7fcba6a96700 1 -- 172.21.15.16:0/2278987900 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fcb9006dd60 msgr2=0x7fcb90070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:44.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.838+0000 7fcba6a96700 1 --2- 172.21.15.16:0/2278987900 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fcb9006dd60 0x7fcb90070220 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c009fd0 tx=0x7fcb8c005960 comp rx=0 tx=0).stop 2024-04-02T04:49:44.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.838+0000 7fcba6a96700 1 -- 172.21.15.16:0/2278987900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba00fedd0 msgr2=0x7fcba01964d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:44.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.838+0000 7fcba6a96700 1 --2- 172.21.15.16:0/2278987900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba00fedd0 0x7fcba01964d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcb9400d900 tx=0x7fcb9400dc10 comp rx=0 tx=0).stop 2024-04-02T04:49:44.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.838+0000 7fcba6a96700 1 -- 172.21.15.16:0/2278987900 shutdown_connections 2024-04-02T04:49:44.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.838+0000 7fcba6a96700 1 --2- 172.21.15.16:0/2278987900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba00fedd0 0x7fcba01964d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:44.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.838+0000 7fcba6a96700 1 --2- 172.21.15.16:0/2278987900 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fcb9006dd60 0x7fcb90070220 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:44.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.838+0000 7fcba6a96700 1 --2- 172.21.15.16:0/2278987900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcba0101e50 0x7fcba0196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:44.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.838+0000 7fcba6a96700 1 -- 172.21.15.16:0/2278987900 >> 172.21.15.16:0/2278987900 conn(0x7fcba00fa730 msgr2=0x7fcba01053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:44.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.838+0000 7fcba6a96700 1 -- 172.21.15.16:0/2278987900 shutdown_connections 2024-04-02T04:49:44.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.838+0000 7fcba6a96700 1 -- 172.21.15.16:0/2278987900 wait complete. 2024-04-02T04:49:44.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.943+0000 7f5ef97cb700 1 -- 172.21.15.16:0/2854723677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ef4103f30 msgr2=0x7f5ef4106320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:44.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.943+0000 7f5ef97cb700 1 --2- 172.21.15.16:0/2854723677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ef4103f30 0x7f5ef4106320 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4009910 tx=0x7f5ee4009c20 comp rx=0 tx=0).stop 2024-04-02T04:49:44.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.943+0000 7f5ef97cb700 1 -- 172.21.15.16:0/2854723677 shutdown_connections 2024-04-02T04:49:44.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.943+0000 7f5ef97cb700 1 --2- 172.21.15.16:0/2854723677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ef4103f30 0x7f5ef4106320 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:44.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.943+0000 7f5ef97cb700 1 --2- 172.21.15.16:0/2854723677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ef4101600 0x7f5ef41039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:44.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.943+0000 7f5ef97cb700 1 -- 172.21.15.16:0/2854723677 >> 172.21.15.16:0/2854723677 conn(0x7f5ef40faf70 msgr2=0x7f5ef40fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:44.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.943+0000 7f5ef97cb700 1 -- 172.21.15.16:0/2854723677 shutdown_connections 2024-04-02T04:49:44.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.943+0000 7f5ef97cb700 1 -- 172.21.15.16:0/2854723677 wait complete. 2024-04-02T04:49:44.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.943+0000 7f5ef97cb700 1 Processor -- start 2024-04-02T04:49:44.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.944+0000 7f5ef97cb700 1 -- start start 2024-04-02T04:49:44.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.944+0000 7f5ef97cb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ef4101600 0x7f5ef4196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:44.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.944+0000 7f5ef97cb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ef4103f30 0x7f5ef4196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:44.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.944+0000 7f5ef97cb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef4197080 con 0x7f5ef4101600 2024-04-02T04:49:44.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.944+0000 7f5ef97cb700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef41971c0 con 0x7f5ef4103f30 2024-04-02T04:49:44.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.944+0000 7f5ef2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ef4101600 0x7f5ef4196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:44.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.944+0000 7f5ef2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ef4101600 0x7f5ef4196520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48048/0 (socket says 172.21.15.16:48048) 2024-04-02T04:49:44.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.944+0000 7f5ef2ffd700 1 -- 172.21.15.16:0/3109852061 learned_addr learned my addr 172.21.15.16:0/3109852061 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:49:44.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.944+0000 7f5ef27fc700 1 --2- 172.21.15.16:0/3109852061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ef4103f30 0x7f5ef4196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:44.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.944+0000 7f5ef2ffd700 1 -- 172.21.15.16:0/3109852061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ef4103f30 msgr2=0x7f5ef4196a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:44.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.944+0000 7f5ef2ffd700 1 --2- 172.21.15.16:0/3109852061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ef4103f30 0x7f5ef4196a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:44.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.944+0000 7f5ef2ffd700 1 -- 172.21.15.16:0/3109852061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee40095f0 con 0x7f5ef4101600 2024-04-02T04:49:44.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.944+0000 7f5ef2ffd700 1 --2- 172.21.15.16:0/3109852061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ef4101600 0x7f5ef4196520 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800e940 tx=0x7f5ee800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:44.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.944+0000 7f5edbfff700 1 -- 172.21.15.16:0/3109852061 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ee800cba0 con 0x7f5ef4101600 2024-04-02T04:49:44.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.945+0000 7f5ef97cb700 1 -- 172.21.15.16:0/3109852061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ef419bc20 con 0x7f5ef4101600 2024-04-02T04:49:44.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.945+0000 7f5edbfff700 1 -- 172.21.15.16:0/3109852061 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ee800ee90 con 0x7f5ef4101600 2024-04-02T04:49:44.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.945+0000 7f5edbfff700 1 -- 172.21.15.16:0/3109852061 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ee80055d0 con 0x7f5ef4101600 2024-04-02T04:49:44.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.945+0000 7f5ef97cb700 1 -- 172.21.15.16:0/3109852061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef419c170 con 0x7f5ef4101600 2024-04-02T04:49:44.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.945+0000 7f5ef97cb700 1 -- 172.21.15.16:0/3109852061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ef404ea80 con 0x7f5ef4101600 2024-04-02T04:49:44.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.946+0000 7f5edbfff700 1 -- 172.21.15.16:0/3109852061 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f5ee8004750 con 0x7f5ef4101600 2024-04-02T04:49:44.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.946+0000 7f5edbfff700 1 --2- 172.21.15.16:0/3109852061 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5edc06dd60 0x7f5edc070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:44.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.947+0000 7f5ef27fc700 1 --2- 172.21.15.16:0/3109852061 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5edc06dd60 0x7f5edc070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:44.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.947+0000 7f5edbfff700 1 -- 172.21.15.16:0/3109852061 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5ee8014070 con 0x7f5ef4101600 2024-04-02T04:49:44.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.947+0000 7f5ef27fc700 1 --2- 172.21.15.16:0/3109852061 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5edc06dd60 0x7f5edc070220 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5ee400b800 tx=0x7f5ee4009f70 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:44.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:44.951+0000 7f5edbfff700 1 -- 172.21.15.16:0/3109852061 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5ee805af90 con 0x7f5ef4101600 2024-04-02T04:49:45.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.170+0000 7f5ef97cb700 1 -- 172.21.15.16:0/3109852061 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ef4105e40 con 0x7f5edc06dd60 2024-04-02T04:49:45.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.181+0000 7f5edbfff700 1 -- 172.21.15.16:0/3109852061 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f5ef4105e40 con 0x7f5edc06dd60 2024-04-02T04:49:45.186 INFO:teuthology.orchestra.run.smithi016.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:49:45.186 INFO:teuthology.orchestra.run.smithi016.stdout:alertmanager.smithi016 smithi016 *:9093,9094 running (14m) 12s ago 15m 24.5M - 0.25.0 c8568f914cd2 b95d8354b016 2024-04-02T04:49:45.186 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi016 smithi016 running (16m) 12s ago 16m 8694k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 9de6c16a487f 2024-04-02T04:49:45.186 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi027 smithi027 running (14m) 50s ago 14m 8577k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 99956592471a 2024-04-02T04:49:45.186 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi016 smithi016 running (4m) 12s ago 15m 6660k - 19.0.0-1979-g6cec793d 78f830190839 dc63ce0eef58 2024-04-02T04:49:45.186 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi027 smithi027 running (4m) 50s ago 14m 6665k - 19.0.0-1979-g6cec793d 78f830190839 2beb077018e4 2024-04-02T04:49:45.186 INFO:teuthology.orchestra.run.smithi016.stdout:grafana.smithi016 smithi016 *:3000 running (13m) 12s ago 15m 86.9M - 9.4.7 954c08fa6188 7c43d133569c 2024-04-02T04:49:45.186 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.fyzrnm smithi016 running (17s) 12s ago 10m 14.9M - 19.0.0-1979-g6cec793d 78f830190839 98f0a83c6631 2024-04-02T04:49:45.186 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.uhgkgg smithi016 running (37s) 12s ago 10m 13.5M - 19.0.0-1979-g6cec793d 78f830190839 19be3989a243 2024-04-02T04:49:45.186 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.sdublg smithi027 running (9m) 50s ago 9m 20.6M - 18.2.2-1059-g4d05f4ca 909dce4fe53d a538451099db 2024-04-02T04:49:45.186 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.zwrfoo smithi027 running (10m) 50s ago 10m 156M - 18.2.2-1059-g4d05f4ca 909dce4fe53d 4cb3fa428d71 2024-04-02T04:49:45.186 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi016.eqnebx smithi016 *:8443,9283,8765 running (6m) 12s ago 17m 509M - 19.0.0-1979-g6cec793d 78f830190839 d61ed91b4577 2024-04-02T04:49:45.186 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi027.kjzcsc smithi027 *:8443,9283,8765 running (6m) 50s ago 14m 450M - 19.0.0-1979-g6cec793d 78f830190839 5072a5ffc761 2024-04-02T04:49:45.186 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi016 smithi016 running (5m) 12s ago 17m 50.3M 2048M 19.0.0-1979-g6cec793d 78f830190839 eabfc84f1cdf 2024-04-02T04:49:45.187 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi027 smithi027 running (5m) 50s ago 14m 40.1M 2048M 19.0.0-1979-g6cec793d 78f830190839 fcd68b20dcc5 2024-04-02T04:49:45.187 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi016 smithi016 *:9100 running (15m) 12s ago 15m 14.8M - 1.5.0 0da6a335fe13 09f54c65b53c 2024-04-02T04:49:45.187 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi027 smithi027 *:9100 running (14m) 50s ago 14m 15.5M - 1.5.0 0da6a335fe13 e41a657c7443 2024-04-02T04:49:45.187 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 smithi016 running (3m) 12s ago 13m 87.3M 1217M 19.0.0-1979-g6cec793d 78f830190839 421d9e1cc751 2024-04-02T04:49:45.187 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 smithi016 running (3m) 12s ago 13m 91.0M 1217M 19.0.0-1979-g6cec793d 78f830190839 3612a56236a3 2024-04-02T04:49:45.187 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 smithi016 running (2m) 12s ago 12m 81.7M 1217M 19.0.0-1979-g6cec793d 78f830190839 a8e9b8d0303c 2024-04-02T04:49:45.187 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 smithi027 running (2m) 50s ago 12m 102M 2240M 19.0.0-1979-g6cec793d 78f830190839 0dd13b5be8c0 2024-04-02T04:49:45.187 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 smithi027 running (88s) 50s ago 12m 84.1M 2240M 19.0.0-1979-g6cec793d 78f830190839 9025eacf8145 2024-04-02T04:49:45.187 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 smithi027 running (54s) 50s ago 11m 14.8M 2240M 19.0.0-1979-g6cec793d 78f830190839 c2787f157377 2024-04-02T04:49:45.187 INFO:teuthology.orchestra.run.smithi016.stdout:prometheus.smithi016 smithi016 *:9095 running (6m) 12s ago 15m 64.9M - 2.43.0 a07b618ecd1d 466f00d73ddd 2024-04-02T04:49:45.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.183+0000 7f5ef97cb700 1 -- 172.21.15.16:0/3109852061 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5edc06dd60 msgr2=0x7f5edc070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:45.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.183+0000 7f5ef97cb700 1 --2- 172.21.15.16:0/3109852061 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5edc06dd60 0x7f5edc070220 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5ee400b800 tx=0x7f5ee4009f70 comp rx=0 tx=0).stop 2024-04-02T04:49:45.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.183+0000 7f5ef97cb700 1 -- 172.21.15.16:0/3109852061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ef4101600 msgr2=0x7f5ef4196520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:45.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.183+0000 7f5ef97cb700 1 --2- 172.21.15.16:0/3109852061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ef4101600 0x7f5ef4196520 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800e940 tx=0x7f5ee800ec50 comp rx=0 tx=0).stop 2024-04-02T04:49:45.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.183+0000 7f5ef97cb700 1 -- 172.21.15.16:0/3109852061 shutdown_connections 2024-04-02T04:49:45.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.183+0000 7f5ef97cb700 1 --2- 172.21.15.16:0/3109852061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ef4101600 0x7f5ef4196520 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:45.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.183+0000 7f5ef97cb700 1 --2- 172.21.15.16:0/3109852061 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5edc06dd60 0x7f5edc070220 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:45.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.183+0000 7f5ef97cb700 1 --2- 172.21.15.16:0/3109852061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ef4103f30 0x7f5ef4196a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:45.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.183+0000 7f5ef97cb700 1 -- 172.21.15.16:0/3109852061 >> 172.21.15.16:0/3109852061 conn(0x7f5ef40faf70 msgr2=0x7f5ef40fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:45.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.183+0000 7f5ef97cb700 1 -- 172.21.15.16:0/3109852061 shutdown_connections 2024-04-02T04:49:45.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.183+0000 7f5ef97cb700 1 -- 172.21.15.16:0/3109852061 wait complete. 2024-04-02T04:49:45.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.287+0000 7fc61ce29700 1 -- 172.21.15.16:0/1485833305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc618100bf0 msgr2=0x7fc618101050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:45.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.287+0000 7fc61ce29700 1 --2- 172.21.15.16:0/1485833305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc618100bf0 0x7fc618101050 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc60c009910 tx=0x7fc60c009c20 comp rx=0 tx=0).stop 2024-04-02T04:49:45.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.287+0000 7fc61ce29700 1 -- 172.21.15.16:0/1485833305 shutdown_connections 2024-04-02T04:49:45.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.287+0000 7fc61ce29700 1 --2- 172.21.15.16:0/1485833305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc618100bf0 0x7fc618101050 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:45.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.287+0000 7fc61ce29700 1 --2- 172.21.15.16:0/1485833305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6180ff9f0 0x7fc6180ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:45.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.287+0000 7fc61ce29700 1 -- 172.21.15.16:0/1485833305 >> 172.21.15.16:0/1485833305 conn(0x7fc6180faf90 msgr2=0x7fc6180fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:45.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.287+0000 7fc61ce29700 1 -- 172.21.15.16:0/1485833305 shutdown_connections 2024-04-02T04:49:45.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.287+0000 7fc61ce29700 1 -- 172.21.15.16:0/1485833305 wait complete. 2024-04-02T04:49:45.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.288+0000 7fc61ce29700 1 Processor -- start 2024-04-02T04:49:45.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.288+0000 7fc61ce29700 1 -- start start 2024-04-02T04:49:45.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.288+0000 7fc61ce29700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6180ff9f0 0x7fc618196590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:45.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.288+0000 7fc61ce29700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc618100bf0 0x7fc618196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:45.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.288+0000 7fc61ce29700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6181970f0 con 0x7fc618100bf0 2024-04-02T04:49:45.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.288+0000 7fc61ce29700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc618197230 con 0x7fc6180ff9f0 2024-04-02T04:49:45.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.288+0000 7fc615d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc618100bf0 0x7fc618196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:45.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.288+0000 7fc615d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc618100bf0 0x7fc618196ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48056/0 (socket says 172.21.15.16:48056) 2024-04-02T04:49:45.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.288+0000 7fc615d9b700 1 -- 172.21.15.16:0/3380486219 learned_addr learned my addr 172.21.15.16:0/3380486219 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:49:45.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.288+0000 7fc61659c700 1 --2- 172.21.15.16:0/3380486219 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6180ff9f0 0x7fc618196590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:45.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.289+0000 7fc615d9b700 1 -- 172.21.15.16:0/3380486219 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6180ff9f0 msgr2=0x7fc618196590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:45.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.289+0000 7fc615d9b700 1 --2- 172.21.15.16:0/3380486219 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6180ff9f0 0x7fc618196590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:45.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.289+0000 7fc615d9b700 1 -- 172.21.15.16:0/3380486219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc60c0095f0 con 0x7fc618100bf0 2024-04-02T04:49:45.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.289+0000 7fc615d9b700 1 --2- 172.21.15.16:0/3380486219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc618100bf0 0x7fc618196ad0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc60c006010 tx=0x7fc60c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:45.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.289+0000 7fc6077fe700 1 -- 172.21.15.16:0/3380486219 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc60c01d070 con 0x7fc618100bf0 2024-04-02T04:49:45.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.289+0000 7fc6077fe700 1 -- 172.21.15.16:0/3380486219 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc60c00f460 con 0x7fc618100bf0 2024-04-02T04:49:45.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.289+0000 7fc61ce29700 1 -- 172.21.15.16:0/3380486219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc61819bc30 con 0x7fc618100bf0 2024-04-02T04:49:45.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.289+0000 7fc6077fe700 1 -- 172.21.15.16:0/3380486219 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc60c017610 con 0x7fc618100bf0 2024-04-02T04:49:45.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.290+0000 7fc61ce29700 1 -- 172.21.15.16:0/3380486219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc61819c1a0 con 0x7fc618100bf0 2024-04-02T04:49:45.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.291+0000 7fc61ce29700 1 -- 172.21.15.16:0/3380486219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc61804ea80 con 0x7fc618100bf0 2024-04-02T04:49:45.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.291+0000 7fc6077fe700 1 -- 172.21.15.16:0/3380486219 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fc60c00f5d0 con 0x7fc618100bf0 2024-04-02T04:49:45.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.291+0000 7fc6077fe700 1 --2- 172.21.15.16:0/3380486219 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc60006dd60 0x7fc600070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:45.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.291+0000 7fc6077fe700 1 -- 172.21.15.16:0/3380486219 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc60c08eff0 con 0x7fc618100bf0 2024-04-02T04:49:45.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.291+0000 7fc61659c700 1 --2- 172.21.15.16:0/3380486219 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc60006dd60 0x7fc600070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:45.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.292+0000 7fc61659c700 1 --2- 172.21.15.16:0/3380486219 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc60006dd60 0x7fc600070220 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc618100a50 tx=0x7fc60800b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:45.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.296+0000 7fc6077fe700 1 -- 172.21.15.16:0/3380486219 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc60c05c060 con 0x7fc618100bf0 2024-04-02T04:49:45.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.591+0000 7fc61ce29700 1 -- 172.21.15.16:0/3380486219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc618108750 con 0x7fc618100bf0 2024-04-02T04:49:45.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.591+0000 7fc6077fe700 1 -- 172.21.15.16:0/3380486219 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7fc60c05bbf0 con 0x7fc618100bf0 2024-04-02T04:49:45.596 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:49:45.596 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": { 2024-04-02T04:49:45.596 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:49:45.596 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:49:45.596 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": { 2024-04-02T04:49:45.596 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:49:45.596 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:49:45.596 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": { 2024-04-02T04:49:45.596 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 6 2024-04-02T04:49:45.596 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:49:45.596 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-04-02T04:49:45.596 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 1, 2024-04-02T04:49:45.597 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:49:45.597 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:49:45.597 INFO:teuthology.orchestra.run.smithi016.stdout: "overall": { 2024-04-02T04:49:45.597 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.2-1059-g4d05f4ca (4d05f4cad464de0fcdd11a2856643e533aea34e8) reef (stable)": 1, 2024-04-02T04:49:45.597 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 12 2024-04-02T04:49:45.597 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-02T04:49:45.597 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:49:45.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.594+0000 7fc61ce29700 1 -- 172.21.15.16:0/3380486219 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc60006dd60 msgr2=0x7fc600070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:45.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.594+0000 7fc61ce29700 1 --2- 172.21.15.16:0/3380486219 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc60006dd60 0x7fc600070220 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc618100a50 tx=0x7fc60800b410 comp rx=0 tx=0).stop 2024-04-02T04:49:45.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.594+0000 7fc61ce29700 1 -- 172.21.15.16:0/3380486219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc618100bf0 msgr2=0x7fc618196ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:45.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.594+0000 7fc61ce29700 1 --2- 172.21.15.16:0/3380486219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc618100bf0 0x7fc618196ad0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc60c006010 tx=0x7fc60c00bd70 comp rx=0 tx=0).stop 2024-04-02T04:49:45.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.594+0000 7fc61ce29700 1 -- 172.21.15.16:0/3380486219 shutdown_connections 2024-04-02T04:49:45.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.594+0000 7fc61ce29700 1 --2- 172.21.15.16:0/3380486219 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc60006dd60 0x7fc600070220 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:45.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.594+0000 7fc61ce29700 1 --2- 172.21.15.16:0/3380486219 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6180ff9f0 0x7fc618196590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:45.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.594+0000 7fc61ce29700 1 --2- 172.21.15.16:0/3380486219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc618100bf0 0x7fc618196ad0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:45.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.594+0000 7fc61ce29700 1 -- 172.21.15.16:0/3380486219 >> 172.21.15.16:0/3380486219 conn(0x7fc6180faf90 msgr2=0x7fc6180690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:45.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.594+0000 7fc61ce29700 1 -- 172.21.15.16:0/3380486219 shutdown_connections 2024-04-02T04:49:45.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.594+0000 7fc61ce29700 1 -- 172.21.15.16:0/3380486219 wait complete. 2024-04-02T04:49:45.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.699+0000 7fdbcd613700 1 -- 172.21.15.16:0/778180615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc8100c00 msgr2=0x7fdbc8101060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:45.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.699+0000 7fdbcd613700 1 --2- 172.21.15.16:0/778180615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc8100c00 0x7fdbc8101060 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc009910 tx=0x7fdbbc009c20 comp rx=0 tx=0).stop 2024-04-02T04:49:45.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.699+0000 7fdbcd613700 1 -- 172.21.15.16:0/778180615 shutdown_connections 2024-04-02T04:49:45.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.699+0000 7fdbcd613700 1 --2- 172.21.15.16:0/778180615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc8100c00 0x7fdbc8101060 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:45.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.699+0000 7fdbcd613700 1 --2- 172.21.15.16:0/778180615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbc80ffa00 0x7fdbc80ffe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:45.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.699+0000 7fdbcd613700 1 -- 172.21.15.16:0/778180615 >> 172.21.15.16:0/778180615 conn(0x7fdbc80fafe0 msgr2=0x7fdbc80fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:45.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.700+0000 7fdbcd613700 1 -- 172.21.15.16:0/778180615 shutdown_connections 2024-04-02T04:49:45.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.700+0000 7fdbcd613700 1 -- 172.21.15.16:0/778180615 wait complete. 2024-04-02T04:49:45.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.700+0000 7fdbcd613700 1 Processor -- start 2024-04-02T04:49:45.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.700+0000 7fdbcd613700 1 -- start start 2024-04-02T04:49:45.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.700+0000 7fdbcd613700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc80ffa00 0x7fdbc8196550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:45.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.700+0000 7fdbcd613700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbc8100c00 0x7fdbc8196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:45.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.700+0000 7fdbcd613700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc81970b0 con 0x7fdbc80ffa00 2024-04-02T04:49:45.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.700+0000 7fdbcd613700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc81971f0 con 0x7fdbc8100c00 2024-04-02T04:49:45.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.701+0000 7fdbc6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc80ffa00 0x7fdbc8196550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:45.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.701+0000 7fdbc6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc80ffa00 0x7fdbc8196550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48070/0 (socket says 172.21.15.16:48070) 2024-04-02T04:49:45.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.701+0000 7fdbc6ffd700 1 -- 172.21.15.16:0/1218479612 learned_addr learned my addr 172.21.15.16:0/1218479612 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:49:45.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.701+0000 7fdbc67fc700 1 --2- 172.21.15.16:0/1218479612 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbc8100c00 0x7fdbc8196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:45.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.701+0000 7fdbc6ffd700 1 -- 172.21.15.16:0/1218479612 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbc8100c00 msgr2=0x7fdbc8196a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:45.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.701+0000 7fdbc6ffd700 1 --2- 172.21.15.16:0/1218479612 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbc8100c00 0x7fdbc8196a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:45.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.701+0000 7fdbc6ffd700 1 -- 172.21.15.16:0/1218479612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbbc0095f0 con 0x7fdbc80ffa00 2024-04-02T04:49:45.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.701+0000 7fdbc6ffd700 1 --2- 172.21.15.16:0/1218479612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc80ffa00 0x7fdbc8196550 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdbb800d900 tx=0x7fdbb800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:45.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.701+0000 7fdbaffff700 1 -- 172.21.15.16:0/1218479612 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbb80041f0 con 0x7fdbc80ffa00 2024-04-02T04:49:45.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.701+0000 7fdbcd613700 1 -- 172.21.15.16:0/1218479612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbc819bc00 con 0x7fdbc80ffa00 2024-04-02T04:49:45.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.701+0000 7fdbaffff700 1 -- 172.21.15.16:0/1218479612 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdbb8004350 con 0x7fdbc80ffa00 2024-04-02T04:49:45.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.701+0000 7fdbaffff700 1 -- 172.21.15.16:0/1218479612 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbb8003cf0 con 0x7fdbc80ffa00 2024-04-02T04:49:45.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.701+0000 7fdbcd613700 1 -- 172.21.15.16:0/1218479612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbc819c150 con 0x7fdbc80ffa00 2024-04-02T04:49:45.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.702+0000 7fdbcd613700 1 -- 172.21.15.16:0/1218479612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbc804ea80 con 0x7fdbc80ffa00 2024-04-02T04:49:45.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.703+0000 7fdbaffff700 1 -- 172.21.15.16:0/1218479612 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fdbb8010460 con 0x7fdbc80ffa00 2024-04-02T04:49:45.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.703+0000 7fdbaffff700 1 --2- 172.21.15.16:0/1218479612 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fdbb006dd60 0x7fdbb0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:45.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.703+0000 7fdbaffff700 1 -- 172.21.15.16:0/1218479612 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdbb808da10 con 0x7fdbc80ffa00 2024-04-02T04:49:45.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.703+0000 7fdbc67fc700 1 --2- 172.21.15.16:0/1218479612 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fdbb006dd60 0x7fdbb0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:45.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.704+0000 7fdbc67fc700 1 --2- 172.21.15.16:0/1218479612 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fdbb006dd60 0x7fdbb0070220 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc009fd0 tx=0x7fdbbc01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:45.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.708+0000 7fdbaffff700 1 -- 172.21.15.16:0/1218479612 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fdbb805aaf0 con 0x7fdbc80ffa00 2024-04-02T04:49:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.954+0000 7fdbcd613700 1 -- 172.21.15.16:0/1218479612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fdbc819bd90 con 0x7fdbc80ffa00 2024-04-02T04:49:45.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.955+0000 7fdbaffff700 1 -- 172.21.15.16:0/1218479612 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1697 (secure 0 0 0) 0x7fdbc819bd90 con 0x7fdbc80ffa00 2024-04-02T04:49:45.959 INFO:teuthology.orchestra.run.smithi016.stdout:e21 2024-04-02T04:49:45.959 INFO:teuthology.orchestra.run.smithi016.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:49:45.959 INFO:teuthology.orchestra.run.smithi016.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:49:45.959 INFO:teuthology.orchestra.run.smithi016.stdout:legacy client fscid: 1 2024-04-02T04:49:45.960 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:49:45.960 INFO:teuthology.orchestra.run.smithi016.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:49:45.960 INFO:teuthology.orchestra.run.smithi016.stdout:fs_name cephfs 2024-04-02T04:49:45.960 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 21 2024-04-02T04:49:45.960 INFO:teuthology.orchestra.run.smithi016.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:49:45.960 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-04-02T04:39:30.740168+0000 2024-04-02T04:49:45.960 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-04-02T04:49:41.373238+0000 2024-04-02T04:49:45.960 INFO:teuthology.orchestra.run.smithi016.stdout:tableserver 0 2024-04-02T04:49:45.960 INFO:teuthology.orchestra.run.smithi016.stdout:root 0 2024-04-02T04:49:45.960 INFO:teuthology.orchestra.run.smithi016.stdout:session_timeout 60 2024-04-02T04:49:45.960 INFO:teuthology.orchestra.run.smithi016.stdout:session_autoclose 300 2024-04-02T04:49:45.960 INFO:teuthology.orchestra.run.smithi016.stdout:max_file_size 1099511627776 2024-04-02T04:49:45.960 INFO:teuthology.orchestra.run.smithi016.stdout:max_xattr_size 65536 2024-04-02T04:49:45.960 INFO:teuthology.orchestra.run.smithi016.stdout:required_client_features {} 2024-04-02T04:49:45.961 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure 0 2024-04-02T04:49:45.961 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure_osd_epoch 83 2024-04-02T04:49:45.961 INFO:teuthology.orchestra.run.smithi016.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:49:45.961 INFO:teuthology.orchestra.run.smithi016.stdout:max_mds 1 2024-04-02T04:49:45.961 INFO:teuthology.orchestra.run.smithi016.stdout:in 0 2024-04-02T04:49:45.961 INFO:teuthology.orchestra.run.smithi016.stdout:up {0=24295} 2024-04-02T04:49:45.961 INFO:teuthology.orchestra.run.smithi016.stdout:failed 2024-04-02T04:49:45.961 INFO:teuthology.orchestra.run.smithi016.stdout:damaged 2024-04-02T04:49:45.961 INFO:teuthology.orchestra.run.smithi016.stdout:stopped 2024-04-02T04:49:45.961 INFO:teuthology.orchestra.run.smithi016.stdout:data_pools [3] 2024-04-02T04:49:45.961 INFO:teuthology.orchestra.run.smithi016.stdout:metadata_pool 2 2024-04-02T04:49:45.961 INFO:teuthology.orchestra.run.smithi016.stdout:inline_data enabled 2024-04-02T04:49:45.961 INFO:teuthology.orchestra.run.smithi016.stdout:balancer 2024-04-02T04:49:45.961 INFO:teuthology.orchestra.run.smithi016.stdout:bal_rank_mask -1 2024-04-02T04:49:45.962 INFO:teuthology.orchestra.run.smithi016.stdout:standby_count_wanted 1 2024-04-02T04:49:45.962 INFO:teuthology.orchestra.run.smithi016.stdout:qdb_cluster leader: 0 members: 2024-04-02T04:49:45.962 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.sdublg{0:24295} state up:replay seq 2 join_fscid=1 addr [v2:172.21.15.27:6826/4177192740,v1:172.21.15.27:6827/4177192740] compat {c=[1],r=[1],i=[7ff]}] 2024-04-02T04:49:45.962 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:49:45.962 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:49:45.962 INFO:teuthology.orchestra.run.smithi016.stdout:Standby daemons: 2024-04-02T04:49:45.962 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:49:45.962 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.uhgkgg{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.16:6826/81744708,v1:172.21.15.16:6827/81744708] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T04:49:45.962 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.fyzrnm{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.16:6828/782430688,v1:172.21.15.16:6829/782430688] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T04:49:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.957+0000 7fdbcd613700 1 -- 172.21.15.16:0/1218479612 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fdbb006dd60 msgr2=0x7fdbb0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.957+0000 7fdbcd613700 1 --2- 172.21.15.16:0/1218479612 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fdbb006dd60 0x7fdbb0070220 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc009fd0 tx=0x7fdbbc01a040 comp rx=0 tx=0).stop 2024-04-02T04:49:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.957+0000 7fdbcd613700 1 -- 172.21.15.16:0/1218479612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc80ffa00 msgr2=0x7fdbc8196550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.957+0000 7fdbcd613700 1 --2- 172.21.15.16:0/1218479612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc80ffa00 0x7fdbc8196550 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdbb800d900 tx=0x7fdbb800dc10 comp rx=0 tx=0).stop 2024-04-02T04:49:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.957+0000 7fdbcd613700 1 -- 172.21.15.16:0/1218479612 shutdown_connections 2024-04-02T04:49:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.957+0000 7fdbcd613700 1 --2- 172.21.15.16:0/1218479612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc80ffa00 0x7fdbc8196550 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.957+0000 7fdbcd613700 1 --2- 172.21.15.16:0/1218479612 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fdbb006dd60 0x7fdbb0070220 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.957+0000 7fdbcd613700 1 --2- 172.21.15.16:0/1218479612 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbc8100c00 0x7fdbc8196a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.957+0000 7fdbcd613700 1 -- 172.21.15.16:0/1218479612 >> 172.21.15.16:0/1218479612 conn(0x7fdbc80fafe0 msgr2=0x7fdbc8069010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:45.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.957+0000 7fdbcd613700 1 -- 172.21.15.16:0/1218479612 shutdown_connections 2024-04-02T04:49:45.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:45.957+0000 7fdbcd613700 1 -- 172.21.15.16:0/1218479612 wait complete. 2024-04-02T04:49:45.964 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 21 2024-04-02T04:49:46.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.063+0000 7f11bdb23700 1 -- 172.21.15.16:0/3840832482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11b8100f80 msgr2=0x7f11b81013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:46.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.063+0000 7f11bdb23700 1 --2- 172.21.15.16:0/3840832482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11b8100f80 0x7f11b81013a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f11a8009910 tx=0x7f11a8009c20 comp rx=0 tx=0).stop 2024-04-02T04:49:46.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.063+0000 7f11bdb23700 1 -- 172.21.15.16:0/3840832482 shutdown_connections 2024-04-02T04:49:46.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.063+0000 7f11bdb23700 1 --2- 172.21.15.16:0/3840832482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11b80ffcc0 0x7f11b8100120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:46.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.063+0000 7f11bdb23700 1 --2- 172.21.15.16:0/3840832482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11b8100f80 0x7f11b81013a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:46.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.063+0000 7f11bdb23700 1 -- 172.21.15.16:0/3840832482 >> 172.21.15.16:0/3840832482 conn(0x7f11b80fa730 msgr2=0x7f11b80fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:46.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.063+0000 7f11bdb23700 1 -- 172.21.15.16:0/3840832482 shutdown_connections 2024-04-02T04:49:46.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.063+0000 7f11bdb23700 1 -- 172.21.15.16:0/3840832482 wait complete. 2024-04-02T04:49:46.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.064+0000 7f11bdb23700 1 Processor -- start 2024-04-02T04:49:46.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.064+0000 7f11bdb23700 1 -- start start 2024-04-02T04:49:46.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.064+0000 7f11bdb23700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11b80ffcc0 0x7f11b80ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:46.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.064+0000 7f11bdb23700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11b8100f80 0x7f11b819b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:46.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.064+0000 7f11bdb23700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b819b7f0 con 0x7f11b8100f80 2024-04-02T04:49:46.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.064+0000 7f11bdb23700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b819b930 con 0x7f11b80ffcc0 2024-04-02T04:49:46.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.064+0000 7f11aeffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11b8100f80 0x7f11b819b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:46.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.064+0000 7f11aeffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11b8100f80 0x7f11b819b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48090/0 (socket says 172.21.15.16:48090) 2024-04-02T04:49:46.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.064+0000 7f11b77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11b80ffcc0 0x7f11b80ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:46.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.064+0000 7f11aeffd700 1 -- 172.21.15.16:0/1556237002 learned_addr learned my addr 172.21.15.16:0/1556237002 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:49:46.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.065+0000 7f11aeffd700 1 -- 172.21.15.16:0/1556237002 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11b80ffcc0 msgr2=0x7f11b80ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:46.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.065+0000 7f11aeffd700 1 --2- 172.21.15.16:0/1556237002 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11b80ffcc0 0x7f11b80ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:46.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.065+0000 7f11aeffd700 1 -- 172.21.15.16:0/1556237002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a80095f0 con 0x7f11b8100f80 2024-04-02T04:49:46.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.065+0000 7f11aeffd700 1 --2- 172.21.15.16:0/1556237002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11b8100f80 0x7f11b819b2b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f11a000d900 tx=0x7f11a000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:46.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.065+0000 7f11b57fa700 1 -- 172.21.15.16:0/1556237002 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11a00041f0 con 0x7f11b8100f80 2024-04-02T04:49:46.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.065+0000 7f11bdb23700 1 -- 172.21.15.16:0/1556237002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11b819bbe0 con 0x7f11b8100f80 2024-04-02T04:49:46.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.065+0000 7f11b57fa700 1 -- 172.21.15.16:0/1556237002 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f11a0004cf0 con 0x7f11b8100f80 2024-04-02T04:49:46.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.065+0000 7f11b57fa700 1 -- 172.21.15.16:0/1556237002 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11a000b750 con 0x7f11b8100f80 2024-04-02T04:49:46.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.065+0000 7f11bdb23700 1 -- 172.21.15.16:0/1556237002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11b819c130 con 0x7f11b8100f80 2024-04-02T04:49:46.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.066+0000 7f11bdb23700 1 -- 172.21.15.16:0/1556237002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11b818f580 con 0x7f11b8100f80 2024-04-02T04:49:46.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.067+0000 7f11b57fa700 1 -- 172.21.15.16:0/1556237002 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f11a0004750 con 0x7f11b8100f80 2024-04-02T04:49:46.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.067+0000 7f11b57fa700 1 --2- 172.21.15.16:0/1556237002 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f11a406dd60 0x7f11a4070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:46.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.067+0000 7f11b57fa700 1 -- 172.21.15.16:0/1556237002 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f11a008d9d0 con 0x7f11b8100f80 2024-04-02T04:49:46.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.067+0000 7f11b77fe700 1 --2- 172.21.15.16:0/1556237002 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f11a406dd60 0x7f11a4070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:46.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.067+0000 7f11b77fe700 1 --2- 172.21.15.16:0/1556237002 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f11a406dd60 0x7f11a4070220 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f11a8000c00 tx=0x7f11a8005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:46.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.073+0000 7f11b57fa700 1 -- 172.21.15.16:0/1556237002 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f11a005aac0 con 0x7f11b8100f80 2024-04-02T04:49:46.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.294+0000 7f11bdb23700 1 -- 172.21.15.16:0/1556237002 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f11b802ce10 con 0x7f11a406dd60 2024-04-02T04:49:46.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.295+0000 7f11b57fa700 1 -- 172.21.15.16:0/1556237002 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f11b802ce10 con 0x7f11a406dd60 2024-04-02T04:49:46.300 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:49:46.300 INFO:teuthology.orchestra.run.smithi016.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:49:46.300 INFO:teuthology.orchestra.run.smithi016.stdout: "in_progress": true, 2024-04-02T04:49:46.300 INFO:teuthology.orchestra.run.smithi016.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-02T04:49:46.300 INFO:teuthology.orchestra.run.smithi016.stdout: "services_complete": [ 2024-04-02T04:49:46.300 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr", 2024-04-02T04:49:46.300 INFO:teuthology.orchestra.run.smithi016.stdout: "osd", 2024-04-02T04:49:46.300 INFO:teuthology.orchestra.run.smithi016.stdout: "mon", 2024-04-02T04:49:46.300 INFO:teuthology.orchestra.run.smithi016.stdout: "crash" 2024-04-02T04:49:46.300 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-04-02T04:49:46.301 INFO:teuthology.orchestra.run.smithi016.stdout: "progress": "14/23 daemons upgraded", 2024-04-02T04:49:46.301 INFO:teuthology.orchestra.run.smithi016.stdout: "message": "Currently upgrading mds daemons", 2024-04-02T04:49:46.301 INFO:teuthology.orchestra.run.smithi016.stdout: "is_paused": false 2024-04-02T04:49:46.301 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:49:46.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.297+0000 7f11bdb23700 1 -- 172.21.15.16:0/1556237002 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f11a406dd60 msgr2=0x7f11a4070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:46.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.297+0000 7f11bdb23700 1 --2- 172.21.15.16:0/1556237002 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f11a406dd60 0x7f11a4070220 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f11a8000c00 tx=0x7f11a8005960 comp rx=0 tx=0).stop 2024-04-02T04:49:46.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.299+0000 7f11bdb23700 1 -- 172.21.15.16:0/1556237002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11b8100f80 msgr2=0x7f11b819b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:46.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.299+0000 7f11bdb23700 1 --2- 172.21.15.16:0/1556237002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11b8100f80 0x7f11b819b2b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f11a000d900 tx=0x7f11a000dc10 comp rx=0 tx=0).stop 2024-04-02T04:49:46.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.299+0000 7f11bdb23700 1 -- 172.21.15.16:0/1556237002 shutdown_connections 2024-04-02T04:49:46.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.299+0000 7f11bdb23700 1 --2- 172.21.15.16:0/1556237002 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f11a406dd60 0x7f11a4070220 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:46.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.299+0000 7f11bdb23700 1 --2- 172.21.15.16:0/1556237002 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11b80ffcc0 0x7f11b80ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:46.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.299+0000 7f11bdb23700 1 --2- 172.21.15.16:0/1556237002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11b8100f80 0x7f11b819b2b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:46.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.299+0000 7f11bdb23700 1 -- 172.21.15.16:0/1556237002 >> 172.21.15.16:0/1556237002 conn(0x7f11b80fa730 msgr2=0x7f11b81042d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:46.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.299+0000 7f11bdb23700 1 -- 172.21.15.16:0/1556237002 shutdown_connections 2024-04-02T04:49:46.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.299+0000 7f11bdb23700 1 -- 172.21.15.16:0/1556237002 wait complete. 2024-04-02T04:49:46.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.406+0000 7f356ba20700 1 -- 172.21.15.16:0/2437738206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35640ffa20 msgr2=0x7f35640ffe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:46.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.406+0000 7f356ba20700 1 --2- 172.21.15.16:0/2437738206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35640ffa20 0x7f35640ffe40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3560009910 tx=0x7f3560009c20 comp rx=0 tx=0).stop 2024-04-02T04:49:46.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.406+0000 7f356ba20700 1 -- 172.21.15.16:0/2437738206 shutdown_connections 2024-04-02T04:49:46.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.406+0000 7f356ba20700 1 --2- 172.21.15.16:0/2437738206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3564100c20 0x7f3564101080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:46.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.406+0000 7f356ba20700 1 --2- 172.21.15.16:0/2437738206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35640ffa20 0x7f35640ffe40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:46.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.406+0000 7f356ba20700 1 -- 172.21.15.16:0/2437738206 >> 172.21.15.16:0/2437738206 conn(0x7f35640fafc0 msgr2=0x7f35640fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:46.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.406+0000 7f356ba20700 1 -- 172.21.15.16:0/2437738206 shutdown_connections 2024-04-02T04:49:46.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.406+0000 7f356ba20700 1 -- 172.21.15.16:0/2437738206 wait complete. 2024-04-02T04:49:46.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.406+0000 7f356ba20700 1 Processor -- start 2024-04-02T04:49:46.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.407+0000 7f356ba20700 1 -- start start 2024-04-02T04:49:46.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.407+0000 7f356ba20700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35640ffa20 0x7f3564195410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:46.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.407+0000 7f356ba20700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3564100c20 0x7f3564195950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:46.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.407+0000 7f356ba20700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3564195f70 con 0x7f35640ffa20 2024-04-02T04:49:46.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.407+0000 7f356ba20700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35641960b0 con 0x7f3564100c20 2024-04-02T04:49:46.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.407+0000 7f35697bc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35640ffa20 0x7f3564195410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:46.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.407+0000 7f35697bc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35640ffa20 0x7f3564195410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48108/0 (socket says 172.21.15.16:48108) 2024-04-02T04:49:46.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.407+0000 7f35697bc700 1 -- 172.21.15.16:0/558590007 learned_addr learned my addr 172.21.15.16:0/558590007 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:49:46.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.407+0000 7f3568fbb700 1 --2- 172.21.15.16:0/558590007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3564100c20 0x7f3564195950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:46.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.407+0000 7f35697bc700 1 -- 172.21.15.16:0/558590007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3564100c20 msgr2=0x7f3564195950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:46.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.407+0000 7f35697bc700 1 --2- 172.21.15.16:0/558590007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3564100c20 0x7f3564195950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:46.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.407+0000 7f35697bc700 1 -- 172.21.15.16:0/558590007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35600095f0 con 0x7f35640ffa20 2024-04-02T04:49:46.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.407+0000 7f35697bc700 1 --2- 172.21.15.16:0/558590007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35640ffa20 0x7f3564195410 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f356000b800 tx=0x7f3560005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:46.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.408+0000 7f355a7fc700 1 -- 172.21.15.16:0/558590007 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f356001f070 con 0x7f35640ffa20 2024-04-02T04:49:46.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.408+0000 7f355a7fc700 1 -- 172.21.15.16:0/558590007 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f35600052a0 con 0x7f35640ffa20 2024-04-02T04:49:46.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.408+0000 7f356ba20700 1 -- 172.21.15.16:0/558590007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3564068fb0 con 0x7f35640ffa20 2024-04-02T04:49:46.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.408+0000 7f355a7fc700 1 -- 172.21.15.16:0/558590007 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3560003e90 con 0x7f35640ffa20 2024-04-02T04:49:46.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.408+0000 7f356ba20700 1 -- 172.21.15.16:0/558590007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3564069520 con 0x7f35640ffa20 2024-04-02T04:49:46.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.409+0000 7f356ba20700 1 -- 172.21.15.16:0/558590007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f356404ea80 con 0x7f35640ffa20 2024-04-02T04:49:46.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.409+0000 7f355a7fc700 1 -- 172.21.15.16:0/558590007 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f3560023070 con 0x7f35640ffa20 2024-04-02T04:49:46.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.410+0000 7f355a7fc700 1 --2- 172.21.15.16:0/558590007 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f355006dd70 0x7f3550070230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:49:46.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.410+0000 7f355a7fc700 1 -- 172.21.15.16:0/558590007 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f3560091010 con 0x7f35640ffa20 2024-04-02T04:49:46.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.410+0000 7f3568fbb700 1 --2- 172.21.15.16:0/558590007 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f355006dd70 0x7f3550070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:49:46.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.410+0000 7f3568fbb700 1 --2- 172.21.15.16:0/558590007 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f355006dd70 0x7f3550070230 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3554009c60 tx=0x7f3554009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:49:46.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.414+0000 7f355a7fc700 1 -- 172.21.15.16:0/558590007 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f356005e080 con 0x7f35640ffa20 2024-04-02T04:49:46.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:46 smithi016 ceph-mon[93230]: from='client.34440 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:49:46.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:46 smithi016 ceph-mon[93230]: from='client.34444 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:49:46.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:46 smithi016 ceph-mon[93230]: pgmap v201: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 9.2 MiB/s rd, 3 op/s 2024-04-02T04:49:46.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:46 smithi016 ceph-mon[93230]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:49:46.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:46 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/3380486219' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:46.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:46 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/1218479612' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:49:46.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:46 smithi027 ceph-mon[83036]: from='client.34440 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:49:46.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:46 smithi027 ceph-mon[83036]: from='client.34444 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:49:46.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:46 smithi027 ceph-mon[83036]: pgmap v201: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 9.2 MiB/s rd, 3 op/s 2024-04-02T04:49:46.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:46 smithi027 ceph-mon[83036]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:49:46.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:46 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/3380486219' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:49:46.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:46 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/1218479612' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:49:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.705+0000 7f356ba20700 1 -- 172.21.15.16:0/558590007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3564108580 con 0x7f35640ffa20 2024-04-02T04:49:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.706+0000 7f355a7fc700 1 -- 172.21.15.16:0/558590007 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+297 (secure 0 0 0) 0x7f356005dc10 con 0x7f35640ffa20 2024-04-02T04:49:46.709 INFO:teuthology.orchestra.run.smithi016.stdout:HEALTH_WARN 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data 2024-04-02T04:49:46.709 INFO:teuthology.orchestra.run.smithi016.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-02T04:49:46.709 INFO:teuthology.orchestra.run.smithi016.stdout: fs cephfs is degraded 2024-04-02T04:49:46.710 INFO:teuthology.orchestra.run.smithi016.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-02T04:49:46.710 INFO:teuthology.orchestra.run.smithi016.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-02T04:49:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.708+0000 7f356ba20700 1 -- 172.21.15.16:0/558590007 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f355006dd70 msgr2=0x7f3550070230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.708+0000 7f356ba20700 1 --2- 172.21.15.16:0/558590007 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f355006dd70 0x7f3550070230 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3554009c60 tx=0x7f3554009260 comp rx=0 tx=0).stop 2024-04-02T04:49:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.708+0000 7f356ba20700 1 -- 172.21.15.16:0/558590007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35640ffa20 msgr2=0x7f3564195410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:49:46.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.708+0000 7f356ba20700 1 --2- 172.21.15.16:0/558590007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35640ffa20 0x7f3564195410 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f356000b800 tx=0x7f3560005e30 comp rx=0 tx=0).stop 2024-04-02T04:49:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.708+0000 7f356ba20700 1 -- 172.21.15.16:0/558590007 shutdown_connections 2024-04-02T04:49:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.708+0000 7f356ba20700 1 --2- 172.21.15.16:0/558590007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35640ffa20 0x7f3564195410 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.708+0000 7f356ba20700 1 --2- 172.21.15.16:0/558590007 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f355006dd70 0x7f3550070230 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.708+0000 7f356ba20700 1 --2- 172.21.15.16:0/558590007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3564100c20 0x7f3564195950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:49:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.708+0000 7f356ba20700 1 -- 172.21.15.16:0/558590007 >> 172.21.15.16:0/558590007 conn(0x7f35640fafc0 msgr2=0x7f35641041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:49:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.708+0000 7f356ba20700 1 -- 172.21.15.16:0/558590007 shutdown_connections 2024-04-02T04:49:46.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:49:46.708+0000 7f356ba20700 1 -- 172.21.15.16:0/558590007 wait complete. 2024-04-02T04:49:47.525 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:47 smithi016 ceph-mon[93230]: from='client.34460 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:49:47.525 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:47 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/558590007' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:49:47.525 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:47 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:47.525 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:47 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:49:47.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:47 smithi027 ceph-mon[83036]: from='client.34460 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:49:47.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:47 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/558590007' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:49:47.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:47 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:47.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:47 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:49:48.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:48 smithi016 ceph-mon[93230]: pgmap v202: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-04-02T04:49:48.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:48 smithi027 ceph-mon[83036]: pgmap v202: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-04-02T04:49:50.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:50 smithi027 ceph-mon[83036]: pgmap v203: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-04-02T04:49:50.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:50 smithi016 ceph-mon[93230]: pgmap v203: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-04-02T04:49:52.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:52 smithi016 ceph-mon[93230]: pgmap v204: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 7 op/s 2024-04-02T04:49:52.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:52 smithi016 ceph-mon[93230]: reconnect by client.24337 192.168.0.1:0/392542449 after 0.000999974 2024-04-02T04:49:52.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:52 smithi016 ceph-mon[93230]: reconnect by client.14540 192.168.0.1:0/2963519639 after 0.000999974 2024-04-02T04:49:52.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:52 smithi016 ceph-mon[93230]: mds.? [v2:172.21.15.27:6826/4177192740,v1:172.21.15.27:6827/4177192740] up:reconnect 2024-04-02T04:49:52.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:52 smithi016 ceph-mon[93230]: fsmap cephfs:1/1 {0=cephfs.smithi027.sdublg=up:reconnect} 2 up:standby 2024-04-02T04:49:52.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:52 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:52.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:52 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:52.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:52 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:49:52.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:52 smithi027 ceph-mon[83036]: pgmap v204: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 7 op/s 2024-04-02T04:49:52.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:52 smithi027 ceph-mon[83036]: reconnect by client.24337 192.168.0.1:0/392542449 after 0.000999974 2024-04-02T04:49:52.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:52 smithi027 ceph-mon[83036]: reconnect by client.14540 192.168.0.1:0/2963519639 after 0.000999974 2024-04-02T04:49:52.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:52 smithi027 ceph-mon[83036]: mds.? [v2:172.21.15.27:6826/4177192740,v1:172.21.15.27:6827/4177192740] up:reconnect 2024-04-02T04:49:52.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:52 smithi027 ceph-mon[83036]: fsmap cephfs:1/1 {0=cephfs.smithi027.sdublg=up:reconnect} 2 up:standby 2024-04-02T04:49:52.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:52 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:52.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:52 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:49:52.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:52 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:49:53.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:53 smithi016 ceph-mon[93230]: mds.? [v2:172.21.15.27:6826/4177192740,v1:172.21.15.27:6827/4177192740] up:rejoin 2024-04-02T04:49:53.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:53 smithi016 ceph-mon[93230]: mds.? [v2:172.21.15.27:6824/1016194753,v1:172.21.15.27:6825/1016194753] up:boot 2024-04-02T04:49:53.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:53 smithi016 ceph-mon[93230]: fsmap cephfs:1/1 {0=cephfs.smithi027.sdublg=up:rejoin} 3 up:standby 2024-04-02T04:49:53.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:53 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.zwrfoo"}]: dispatch 2024-04-02T04:49:53.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:53 smithi016 ceph-mon[93230]: daemon mds.cephfs.smithi027.sdublg is now active in filesystem cephfs as rank 0 2024-04-02T04:49:53.327 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:53 smithi027 ceph-mon[83036]: mds.? [v2:172.21.15.27:6826/4177192740,v1:172.21.15.27:6827/4177192740] up:rejoin 2024-04-02T04:49:53.327 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:53 smithi027 ceph-mon[83036]: mds.? [v2:172.21.15.27:6824/1016194753,v1:172.21.15.27:6825/1016194753] up:boot 2024-04-02T04:49:53.327 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:53 smithi027 ceph-mon[83036]: fsmap cephfs:1/1 {0=cephfs.smithi027.sdublg=up:rejoin} 3 up:standby 2024-04-02T04:49:53.327 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:53 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.zwrfoo"}]: dispatch 2024-04-02T04:49:53.327 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:53 smithi027 ceph-mon[83036]: daemon mds.cephfs.smithi027.sdublg is now active in filesystem cephfs as rank 0 2024-04-02T04:49:54.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:54 smithi016 ceph-mon[93230]: pgmap v205: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-04-02T04:49:54.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:54 smithi016 ceph-mon[93230]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-02T04:49:54.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:54 smithi016 ceph-mon[93230]: mds.? [v2:172.21.15.27:6826/4177192740,v1:172.21.15.27:6827/4177192740] up:active 2024-04-02T04:49:54.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:54 smithi016 ceph-mon[93230]: fsmap cephfs:1 {0=cephfs.smithi027.sdublg=up:active} 3 up:standby 2024-04-02T04:49:54.627 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:54 smithi027 ceph-mon[83036]: pgmap v205: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-04-02T04:49:54.627 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:54 smithi027 ceph-mon[83036]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-02T04:49:54.628 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:54 smithi027 ceph-mon[83036]: mds.? [v2:172.21.15.27:6826/4177192740,v1:172.21.15.27:6827/4177192740] up:active 2024-04-02T04:49:54.628 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:54 smithi027 ceph-mon[83036]: fsmap cephfs:1 {0=cephfs.smithi027.sdublg=up:active} 3 up:standby 2024-04-02T04:49:56.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:56 smithi016 ceph-mon[93230]: pgmap v206: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 85 B/s wr, 7 op/s 2024-04-02T04:49:56.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:56 smithi027 ceph-mon[83036]: pgmap v206: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 85 B/s wr, 7 op/s 2024-04-02T04:49:58.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:49:58 smithi016 ceph-mon[93230]: pgmap v207: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 13 MiB/s rd, 4.1 KiB/s wr, 7 op/s 2024-04-02T04:49:58.576 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:49:58 smithi027 ceph-mon[83036]: pgmap v207: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 13 MiB/s rd, 4.1 KiB/s wr, 7 op/s 2024-04-02T04:50:00.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:00 smithi016 ceph-mon[93230]: pgmap v208: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 10 MiB/s rd, 4.1 KiB/s wr, 7 op/s 2024-04-02T04:50:00.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:00 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:00.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:00 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:00.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:00 smithi016 ceph-mon[93230]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-02T04:50:00.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:00 smithi027 ceph-mon[83036]: pgmap v208: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 10 MiB/s rd, 4.1 KiB/s wr, 7 op/s 2024-04-02T04:50:00.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:00 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:00.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:00 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:00.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:00 smithi027 ceph-mon[83036]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-02T04:50:01.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:01 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:01.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:01 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:01.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:01 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:01.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:01 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:02.557 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:02 smithi027 ceph-mon[83036]: pgmap v209: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 4.7 MiB/s rd, 4.1 KiB/s wr, 6 op/s 2024-04-02T04:50:02.558 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:02 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:02.558 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:02 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:50:02.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:02 smithi016 ceph-mon[93230]: pgmap v209: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 4.7 MiB/s rd, 4.1 KiB/s wr, 6 op/s 2024-04-02T04:50:02.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:02 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:02.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:02 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:50:03.996 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:03 smithi027 ceph-mon[83036]: pgmap v210: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 4.1 KiB/s wr, 5 op/s 2024-04-02T04:50:04.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:03 smithi016 ceph-mon[93230]: pgmap v210: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 4.1 KiB/s wr, 5 op/s 2024-04-02T04:50:05.132 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:05 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:05.132 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:05 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:05.132 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:05 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:50:05.133 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:05 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:50:05.133 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:05 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:05.133 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:05 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:50:05.133 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:05 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:05.133 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:05 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:05.133 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:05 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:05.133 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:05 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:05.133 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:05 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi027.sdublg"]}]: dispatch 2024-04-02T04:50:05.133 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:05 smithi027 ceph-mon[83036]: Upgrade: It appears safe to stop mds.cephfs.smithi027.sdublg 2024-04-02T04:50:05.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:05 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:05.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:05 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:05.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:05 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:50:05.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:05 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:50:05.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:05 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:05.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:05 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:50:05.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:05 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:05.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:05 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:05.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:05 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:05.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:05 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:05.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:05 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi027.sdublg"]}]: dispatch 2024-04-02T04:50:05.311 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:05 smithi016 ceph-mon[93230]: Upgrade: It appears safe to stop mds.cephfs.smithi027.sdublg 2024-04-02T04:50:06.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:06 smithi016 ceph-mon[93230]: pgmap v211: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 345 KiB/s rd, 4.1 KiB/s wr, 5 op/s 2024-04-02T04:50:06.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:06.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi027.sdublg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:50:06.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:50:06.369 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:06 smithi027 ceph-mon[83036]: pgmap v211: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 345 KiB/s rd, 4.1 KiB/s wr, 5 op/s 2024-04-02T04:50:06.369 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:06.369 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi027.sdublg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T04:50:06.369 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:50:07.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:07 smithi016 ceph-mon[93230]: Upgrade: Updating mds.cephfs.smithi027.sdublg 2024-04-02T04:50:07.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:07 smithi016 ceph-mon[93230]: Deploying daemon mds.cephfs.smithi027.sdublg on smithi027 2024-04-02T04:50:07.334 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:07 smithi027 ceph-mon[83036]: Upgrade: Updating mds.cephfs.smithi027.sdublg 2024-04-02T04:50:07.334 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:07 smithi027 ceph-mon[83036]: Deploying daemon mds.cephfs.smithi027.sdublg on smithi027 2024-04-02T04:50:08.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:08 smithi027 ceph-mon[83036]: pgmap v212: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4.0 KiB/s wr, 4 op/s 2024-04-02T04:50:08.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:08 smithi027 ceph-mon[83036]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-02T04:50:08.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:08 smithi027 ceph-mon[83036]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T04:50:08.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:08 smithi027 ceph-mon[83036]: osdmap e84: 6 total, 6 up, 6 in 2024-04-02T04:50:08.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:08 smithi027 ceph-mon[83036]: Standby daemon mds.cephfs.smithi016.uhgkgg assigned to filesystem cephfs as rank 0 2024-04-02T04:50:08.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:08 smithi027 ceph-mon[83036]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-02T04:50:08.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:08 smithi027 ceph-mon[83036]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-02T04:50:08.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:08 smithi027 ceph-mon[83036]: fsmap cephfs:1/1 {0=cephfs.smithi016.uhgkgg=up:replay} 2 up:standby 2024-04-02T04:50:08.147 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:08 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.sdublg"}]: dispatch 2024-04-02T04:50:08.220 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:07 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi016[93226]: 2024-04-02T04:50:07.886+0000 7f131edaa700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T04:50:08.220 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:08 smithi016 ceph-mon[93230]: pgmap v212: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4.0 KiB/s wr, 4 op/s 2024-04-02T04:50:08.220 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:08 smithi016 ceph-mon[93230]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-02T04:50:08.220 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:08 smithi016 ceph-mon[93230]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T04:50:08.220 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:08 smithi016 ceph-mon[93230]: osdmap e84: 6 total, 6 up, 6 in 2024-04-02T04:50:08.220 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:08 smithi016 ceph-mon[93230]: Standby daemon mds.cephfs.smithi016.uhgkgg assigned to filesystem cephfs as rank 0 2024-04-02T04:50:08.220 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:08 smithi016 ceph-mon[93230]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-02T04:50:08.220 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:08 smithi016 ceph-mon[93230]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-02T04:50:08.220 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:08 smithi016 ceph-mon[93230]: fsmap cephfs:1/1 {0=cephfs.smithi016.uhgkgg=up:replay} 2 up:standby 2024-04-02T04:50:08.220 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:08 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.sdublg"}]: dispatch 2024-04-02T04:50:10.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:10 smithi016 ceph-mon[93230]: pgmap v214: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 0 op/s 2024-04-02T04:50:10.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:10 smithi027 ceph-mon[83036]: pgmap v214: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 0 op/s 2024-04-02T04:50:12.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:12 smithi016 ceph-mon[93230]: pgmap v215: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 5.2 MiB/s rd, 2 op/s 2024-04-02T04:50:12.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:12 smithi027 ceph-mon[83036]: pgmap v215: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 5.2 MiB/s rd, 2 op/s 2024-04-02T04:50:14.473 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:14 smithi027 ceph-mon[83036]: pgmap v216: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-04-02T04:50:14.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:14 smithi016 ceph-mon[93230]: pgmap v216: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-04-02T04:50:16.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:16 smithi016 ceph-mon[93230]: pgmap v217: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-04-02T04:50:16.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:16 smithi027 ceph-mon[83036]: pgmap v217: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-04-02T04:50:16.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:16.823+0000 7fc20424d700 1 -- 172.21.15.16:0/1023540052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1fc103f00 msgr2=0x7fc1fc1062f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:16.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:16.823+0000 7fc20424d700 1 --2- 172.21.15.16:0/1023540052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1fc103f00 0x7fc1fc1062f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec009910 tx=0x7fc1ec009c20 comp rx=0 tx=0).stop 2024-04-02T04:50:16.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:16.824+0000 7fc20424d700 1 -- 172.21.15.16:0/1023540052 shutdown_connections 2024-04-02T04:50:16.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:16.824+0000 7fc20424d700 1 --2- 172.21.15.16:0/1023540052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1fc103f00 0x7fc1fc1062f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:16.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:16.824+0000 7fc20424d700 1 --2- 172.21.15.16:0/1023540052 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1fc1015d0 0x7fc1fc1039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:16.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:16.824+0000 7fc20424d700 1 -- 172.21.15.16:0/1023540052 >> 172.21.15.16:0/1023540052 conn(0x7fc1fc0faf80 msgr2=0x7fc1fc0fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:16.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:16.824+0000 7fc20424d700 1 -- 172.21.15.16:0/1023540052 shutdown_connections 2024-04-02T04:50:16.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:16.824+0000 7fc20424d700 1 -- 172.21.15.16:0/1023540052 wait complete. 2024-04-02T04:50:16.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:16.824+0000 7fc20424d700 1 Processor -- start 2024-04-02T04:50:16.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:16.825+0000 7fc20424d700 1 -- start start 2024-04-02T04:50:16.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:16.825+0000 7fc20424d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1fc1015d0 0x7fc1fc1964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:16.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:16.825+0000 7fc20424d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1fc103f00 0x7fc1fc196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:16.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:16.825+0000 7fc20424d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc197040 con 0x7fc1fc103f00 2024-04-02T04:50:16.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:16.825+0000 7fc20424d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc197180 con 0x7fc1fc1015d0 2024-04-02T04:50:16.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:16.825+0000 7fc2017e8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1fc103f00 0x7fc1fc196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:16.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:16.825+0000 7fc2017e8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1fc103f00 0x7fc1fc196a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60676/0 (socket says 172.21.15.16:60676) 2024-04-02T04:50:16.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:16.825+0000 7fc2017e8700 1 -- 172.21.15.16:0/413404675 learned_addr learned my addr 172.21.15.16:0/413404675 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:50:16.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:16.825+0000 7fc201fe9700 1 --2- 172.21.15.16:0/413404675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1fc1015d0 0x7fc1fc1964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:16.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:16.825+0000 7fc2017e8700 1 -- 172.21.15.16:0/413404675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1fc1015d0 msgr2=0x7fc1fc1964e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:16.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:16.825+0000 7fc2017e8700 1 --2- 172.21.15.16:0/413404675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1fc1015d0 0x7fc1fc1964e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:16.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:16.825+0000 7fc2017e8700 1 -- 172.21.15.16:0/413404675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ec0095f0 con 0x7fc1fc103f00 2024-04-02T04:50:16.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:16.825+0000 7fc201fe9700 1 --2- 172.21.15.16:0/413404675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1fc1015d0 0x7fc1fc1964e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T04:50:16.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:16.826+0000 7fc2017e8700 1 --2- 172.21.15.16:0/413404675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1fc103f00 0x7fc1fc196a20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec0092e0 tx=0x7fc1ec00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:16.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:16.826+0000 7fc1f2ffd700 1 -- 172.21.15.16:0/413404675 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1ec01d070 con 0x7fc1fc103f00 2024-04-02T04:50:16.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:16.826+0000 7fc1f2ffd700 1 -- 172.21.15.16:0/413404675 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc1ec00f460 con 0x7fc1fc103f00 2024-04-02T04:50:16.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:16.826+0000 7fc1f2ffd700 1 -- 172.21.15.16:0/413404675 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1ec021640 con 0x7fc1fc103f00 2024-04-02T04:50:16.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:16.826+0000 7fc20424d700 1 -- 172.21.15.16:0/413404675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1fc19bb80 con 0x7fc1fc103f00 2024-04-02T04:50:16.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:16.826+0000 7fc20424d700 1 -- 172.21.15.16:0/413404675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1fc19c020 con 0x7fc1fc103f00 2024-04-02T04:50:16.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:16.827+0000 7fc20424d700 1 -- 172.21.15.16:0/413404675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1fc04ea80 con 0x7fc1fc103f00 2024-04-02T04:50:16.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:16.828+0000 7fc1f2ffd700 1 -- 172.21.15.16:0/413404675 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fc1ec00f5d0 con 0x7fc1fc103f00 2024-04-02T04:50:16.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:16.828+0000 7fc1f2ffd700 1 --2- 172.21.15.16:0/413404675 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc1e806dd60 0x7fc1e8070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:16.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:16.828+0000 7fc1f2ffd700 1 -- 172.21.15.16:0/413404675 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fc1ec08efa0 con 0x7fc1fc103f00 2024-04-02T04:50:16.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:16.828+0000 7fc201fe9700 1 --2- 172.21.15.16:0/413404675 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc1e806dd60 0x7fc1e8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:16.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:16.829+0000 7fc201fe9700 1 --2- 172.21.15.16:0/413404675 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc1e806dd60 0x7fc1e8070220 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8005950 tx=0x7fc1f8005f50 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:16.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:16.833+0000 7fc1f2ffd700 1 -- 172.21.15.16:0/413404675 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc1ec0c3280 con 0x7fc1fc103f00 2024-04-02T04:50:17.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.086+0000 7fc20424d700 1 -- 172.21.15.16:0/413404675 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1fc105e20 con 0x7fc1e806dd60 2024-04-02T04:50:17.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.087+0000 7fc1f2ffd700 1 -- 172.21.15.16:0/413404675 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc1fc105e20 con 0x7fc1e806dd60 2024-04-02T04:50:17.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.089+0000 7fc20424d700 1 -- 172.21.15.16:0/413404675 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc1e806dd60 msgr2=0x7fc1e8070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:17.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.089+0000 7fc20424d700 1 --2- 172.21.15.16:0/413404675 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc1e806dd60 0x7fc1e8070220 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8005950 tx=0x7fc1f8005f50 comp rx=0 tx=0).stop 2024-04-02T04:50:17.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.089+0000 7fc20424d700 1 -- 172.21.15.16:0/413404675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1fc103f00 msgr2=0x7fc1fc196a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:17.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.089+0000 7fc20424d700 1 --2- 172.21.15.16:0/413404675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1fc103f00 0x7fc1fc196a20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec0092e0 tx=0x7fc1ec00be20 comp rx=0 tx=0).stop 2024-04-02T04:50:17.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.090+0000 7fc20424d700 1 -- 172.21.15.16:0/413404675 shutdown_connections 2024-04-02T04:50:17.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.090+0000 7fc20424d700 1 --2- 172.21.15.16:0/413404675 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc1e806dd60 0x7fc1e8070220 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:17.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.090+0000 7fc20424d700 1 --2- 172.21.15.16:0/413404675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1fc1015d0 0x7fc1fc1964e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:17.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.090+0000 7fc20424d700 1 --2- 172.21.15.16:0/413404675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1fc103f00 0x7fc1fc196a20 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:17.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.090+0000 7fc20424d700 1 -- 172.21.15.16:0/413404675 >> 172.21.15.16:0/413404675 conn(0x7fc1fc0faf80 msgr2=0x7fc1fc0fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:17.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.090+0000 7fc20424d700 1 -- 172.21.15.16:0/413404675 shutdown_connections 2024-04-02T04:50:17.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.090+0000 7fc20424d700 1 -- 172.21.15.16:0/413404675 wait complete. 2024-04-02T04:50:17.108 INFO:teuthology.orchestra.run.smithi016.stdout:true 2024-04-02T04:50:17.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.205+0000 7f55337b6700 1 -- 172.21.15.16:0/1465057076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f552c103ef0 msgr2=0x7f552c1062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:17.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.205+0000 7f55337b6700 1 --2- 172.21.15.16:0/1465057076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f552c103ef0 0x7f552c1062e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f551c009910 tx=0x7f551c009c20 comp rx=0 tx=0).stop 2024-04-02T04:50:17.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.205+0000 7f55337b6700 1 -- 172.21.15.16:0/1465057076 shutdown_connections 2024-04-02T04:50:17.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.205+0000 7f55337b6700 1 --2- 172.21.15.16:0/1465057076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f552c103ef0 0x7f552c1062e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:17.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.205+0000 7f55337b6700 1 --2- 172.21.15.16:0/1465057076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f552c1015c0 0x7f552c1039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:17.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.205+0000 7f55337b6700 1 -- 172.21.15.16:0/1465057076 >> 172.21.15.16:0/1465057076 conn(0x7f552c0faf70 msgr2=0x7f552c0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:17.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.206+0000 7f55337b6700 1 -- 172.21.15.16:0/1465057076 shutdown_connections 2024-04-02T04:50:17.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.206+0000 7f55337b6700 1 -- 172.21.15.16:0/1465057076 wait complete. 2024-04-02T04:50:17.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.206+0000 7f55337b6700 1 Processor -- start 2024-04-02T04:50:17.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.206+0000 7f55337b6700 1 -- start start 2024-04-02T04:50:17.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.206+0000 7f55337b6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f552c1015c0 0x7f552c1965a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:17.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.206+0000 7f55337b6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f552c103ef0 0x7f552c196ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:17.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.206+0000 7f55337b6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552c197100 con 0x7f552c1015c0 2024-04-02T04:50:17.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.206+0000 7f55337b6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552c197240 con 0x7f552c103ef0 2024-04-02T04:50:17.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.207+0000 7f5531552700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f552c1015c0 0x7f552c1965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:17.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.207+0000 7f5531552700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f552c1015c0 0x7f552c1965a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60700/0 (socket says 172.21.15.16:60700) 2024-04-02T04:50:17.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.207+0000 7f5531552700 1 -- 172.21.15.16:0/1360290741 learned_addr learned my addr 172.21.15.16:0/1360290741 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:50:17.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.207+0000 7f5530d51700 1 --2- 172.21.15.16:0/1360290741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f552c103ef0 0x7f552c196ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:17.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.207+0000 7f5530d51700 1 -- 172.21.15.16:0/1360290741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f552c1015c0 msgr2=0x7f552c1965a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:17.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.207+0000 7f5530d51700 1 --2- 172.21.15.16:0/1360290741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f552c1015c0 0x7f552c1965a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:17.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.207+0000 7f5530d51700 1 -- 172.21.15.16:0/1360290741 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f551c0095f0 con 0x7f552c103ef0 2024-04-02T04:50:17.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.207+0000 7f5531552700 1 --2- 172.21.15.16:0/1360290741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f552c1015c0 0x7f552c1965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T04:50:17.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.208+0000 7f5530d51700 1 --2- 172.21.15.16:0/1360290741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f552c103ef0 0x7f552c196ae0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f551c0059d0 tx=0x7f551c00bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:17.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.208+0000 7f55227fc700 1 -- 172.21.15.16:0/1360290741 <== mon.1 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f551c01d070 con 0x7f552c103ef0 2024-04-02T04:50:17.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.208+0000 7f55227fc700 1 -- 172.21.15.16:0/1360290741 <== mon.1 v2:172.21.15.27:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f551c00f460 con 0x7f552c103ef0 2024-04-02T04:50:17.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.208+0000 7f55227fc700 1 -- 172.21.15.16:0/1360290741 <== mon.1 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f551c021640 con 0x7f552c103ef0 2024-04-02T04:50:17.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.208+0000 7f55337b6700 1 -- 172.21.15.16:0/1360290741 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f552c19bc40 con 0x7f552c103ef0 2024-04-02T04:50:17.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.208+0000 7f55337b6700 1 -- 172.21.15.16:0/1360290741 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f552c19c160 con 0x7f552c103ef0 2024-04-02T04:50:17.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.209+0000 7f55337b6700 1 -- 172.21.15.16:0/1360290741 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f552c04ea80 con 0x7f552c103ef0 2024-04-02T04:50:17.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.209+0000 7f55227fc700 1 -- 172.21.15.16:0/1360290741 <== mon.1 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f551c00f5d0 con 0x7f552c103ef0 2024-04-02T04:50:17.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.210+0000 7f55227fc700 1 --2- 172.21.15.16:0/1360290741 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f551806dd60 0x7f5518070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:17.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.210+0000 7f55227fc700 1 -- 172.21.15.16:0/1360290741 <== mon.1 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f551c08edc0 con 0x7f552c103ef0 2024-04-02T04:50:17.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.210+0000 7f5531552700 1 --2- 172.21.15.16:0/1360290741 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f551806dd60 0x7f5518070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:17.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.210+0000 7f5531552700 1 --2- 172.21.15.16:0/1360290741 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f551806dd60 0x7f5518070220 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5528009ea0 tx=0x7f5528009290 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:17.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.215+0000 7f55227fc700 1 -- 172.21.15.16:0/1360290741 <== mon.1 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f551c05bde0 con 0x7f552c103ef0 2024-04-02T04:50:17.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.491+0000 7f55337b6700 1 -- 172.21.15.16:0/1360290741 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f552c104930 con 0x7f551806dd60 2024-04-02T04:50:17.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.492+0000 7f55227fc700 1 -- 172.21.15.16:0/1360290741 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f552c104930 con 0x7f551806dd60 2024-04-02T04:50:17.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.494+0000 7f55337b6700 1 -- 172.21.15.16:0/1360290741 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f551806dd60 msgr2=0x7f5518070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:17.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.494+0000 7f55337b6700 1 --2- 172.21.15.16:0/1360290741 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f551806dd60 0x7f5518070220 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5528009ea0 tx=0x7f5528009290 comp rx=0 tx=0).stop 2024-04-02T04:50:17.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.494+0000 7f55337b6700 1 -- 172.21.15.16:0/1360290741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f552c103ef0 msgr2=0x7f552c196ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:17.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.494+0000 7f55337b6700 1 --2- 172.21.15.16:0/1360290741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f552c103ef0 0x7f552c196ae0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f551c0059d0 tx=0x7f551c00bd70 comp rx=0 tx=0).stop 2024-04-02T04:50:17.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.495+0000 7f55337b6700 1 -- 172.21.15.16:0/1360290741 shutdown_connections 2024-04-02T04:50:17.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.495+0000 7f55337b6700 1 --2- 172.21.15.16:0/1360290741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f552c1015c0 0x7f552c1965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:17.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.495+0000 7f55337b6700 1 --2- 172.21.15.16:0/1360290741 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f551806dd60 0x7f5518070220 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:17.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.495+0000 7f55337b6700 1 --2- 172.21.15.16:0/1360290741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f552c103ef0 0x7f552c196ae0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:17.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.495+0000 7f55337b6700 1 -- 172.21.15.16:0/1360290741 >> 172.21.15.16:0/1360290741 conn(0x7f552c0faf70 msgr2=0x7f552c0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:17.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.495+0000 7f55337b6700 1 -- 172.21.15.16:0/1360290741 shutdown_connections 2024-04-02T04:50:17.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.495+0000 7f55337b6700 1 -- 172.21.15.16:0/1360290741 wait complete. 2024-04-02T04:50:17.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.602+0000 7fbf6e984700 1 -- 172.21.15.16:0/3842964097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf68101cd0 msgr2=0x7fbf68102130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:17.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.602+0000 7fbf6e984700 1 --2- 172.21.15.16:0/3842964097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf68101cd0 0x7fbf68102130 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c009910 tx=0x7fbf5c009c20 comp rx=0 tx=0).stop 2024-04-02T04:50:17.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.602+0000 7fbf6e984700 1 -- 172.21.15.16:0/3842964097 shutdown_connections 2024-04-02T04:50:17.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.602+0000 7fbf6e984700 1 --2- 172.21.15.16:0/3842964097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf68101cd0 0x7fbf68102130 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:17.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.602+0000 7fbf6e984700 1 --2- 172.21.15.16:0/3842964097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf68100ad0 0x7fbf68100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:17.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.602+0000 7fbf6e984700 1 -- 172.21.15.16:0/3842964097 >> 172.21.15.16:0/3842964097 conn(0x7fbf680fc090 msgr2=0x7fbf680fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:17.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.602+0000 7fbf6e984700 1 -- 172.21.15.16:0/3842964097 shutdown_connections 2024-04-02T04:50:17.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.602+0000 7fbf6e984700 1 -- 172.21.15.16:0/3842964097 wait complete. 2024-04-02T04:50:17.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.603+0000 7fbf6e984700 1 Processor -- start 2024-04-02T04:50:17.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.603+0000 7fbf6e984700 1 -- start start 2024-04-02T04:50:17.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.603+0000 7fbf6e984700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf68100ad0 0x7fbf68074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:17.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.603+0000 7fbf6e984700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf68101cd0 0x7fbf680731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:17.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.603+0000 7fbf6e984700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf680736e0 con 0x7fbf68101cd0 2024-04-02T04:50:17.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.603+0000 7fbf6e984700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf68073820 con 0x7fbf68100ad0 2024-04-02T04:50:17.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.603+0000 7fbf677fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf68101cd0 0x7fbf680731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:17.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.603+0000 7fbf677fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf68101cd0 0x7fbf680731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60726/0 (socket says 172.21.15.16:60726) 2024-04-02T04:50:17.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.603+0000 7fbf677fe700 1 -- 172.21.15.16:0/1510692144 learned_addr learned my addr 172.21.15.16:0/1510692144 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:50:17.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.604+0000 7fbf67fff700 1 --2- 172.21.15.16:0/1510692144 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf68100ad0 0x7fbf68074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:17.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.604+0000 7fbf677fe700 1 -- 172.21.15.16:0/1510692144 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf68100ad0 msgr2=0x7fbf68074b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:17.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.604+0000 7fbf677fe700 1 --2- 172.21.15.16:0/1510692144 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf68100ad0 0x7fbf68074b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:17.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.604+0000 7fbf677fe700 1 -- 172.21.15.16:0/1510692144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5c0095f0 con 0x7fbf68101cd0 2024-04-02T04:50:17.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.604+0000 7fbf677fe700 1 --2- 172.21.15.16:0/1510692144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf68101cd0 0x7fbf680731a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c0092e0 tx=0x7fbf5c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:17.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.604+0000 7fbf657fa700 1 -- 172.21.15.16:0/1510692144 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf5c01d070 con 0x7fbf68101cd0 2024-04-02T04:50:17.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.604+0000 7fbf657fa700 1 -- 172.21.15.16:0/1510692144 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf5c00f460 con 0x7fbf68101cd0 2024-04-02T04:50:17.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.604+0000 7fbf6e984700 1 -- 172.21.15.16:0/1510692144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf68073aa0 con 0x7fbf68101cd0 2024-04-02T04:50:17.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.604+0000 7fbf657fa700 1 -- 172.21.15.16:0/1510692144 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf5c021640 con 0x7fbf68101cd0 2024-04-02T04:50:17.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.604+0000 7fbf6e984700 1 -- 172.21.15.16:0/1510692144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf68074010 con 0x7fbf68101cd0 2024-04-02T04:50:17.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.605+0000 7fbf6e984700 1 -- 172.21.15.16:0/1510692144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf68190650 con 0x7fbf68101cd0 2024-04-02T04:50:17.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.606+0000 7fbf657fa700 1 -- 172.21.15.16:0/1510692144 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fbf5c00f5d0 con 0x7fbf68101cd0 2024-04-02T04:50:17.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.606+0000 7fbf657fa700 1 --2- 172.21.15.16:0/1510692144 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fbf5006dd60 0x7fbf50070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:17.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.606+0000 7fbf657fa700 1 -- 172.21.15.16:0/1510692144 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fbf5c08ed90 con 0x7fbf68101cd0 2024-04-02T04:50:17.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.606+0000 7fbf67fff700 1 --2- 172.21.15.16:0/1510692144 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fbf5006dd60 0x7fbf50070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:17.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.607+0000 7fbf67fff700 1 --2- 172.21.15.16:0/1510692144 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fbf5006dd60 0x7fbf50070220 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbf68101b30 tx=0x7fbf5800a380 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:17.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.615+0000 7fbf657fa700 1 -- 172.21.15.16:0/1510692144 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbf5c05bdb0 con 0x7fbf68101cd0 2024-04-02T04:50:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.831+0000 7fbf6e984700 1 -- 172.21.15.16:0/1510692144 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf68074010 con 0x7fbf5006dd60 2024-04-02T04:50:17.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.842+0000 7fbf657fa700 1 -- 172.21.15.16:0/1510692144 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fbf68074010 con 0x7fbf5006dd60 2024-04-02T04:50:17.846 INFO:teuthology.orchestra.run.smithi016.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:50:17.846 INFO:teuthology.orchestra.run.smithi016.stdout:alertmanager.smithi016 smithi016 *:9093,9094 running (14m) 45s ago 16m 24.5M - 0.25.0 c8568f914cd2 b95d8354b016 2024-04-02T04:50:17.846 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi016 smithi016 running (16m) 45s ago 16m 8694k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 9de6c16a487f 2024-04-02T04:50:17.846 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi027 smithi027 running (15m) 18s ago 15m 8816k - 18.2.2-1059-g4d05f4ca 909dce4fe53d 99956592471a 2024-04-02T04:50:17.846 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi016 smithi016 running (4m) 45s ago 16m 6660k - 19.0.0-1979-g6cec793d 78f830190839 dc63ce0eef58 2024-04-02T04:50:17.846 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi027 smithi027 running (4m) 18s ago 15m 6665k - 19.0.0-1979-g6cec793d 78f830190839 2beb077018e4 2024-04-02T04:50:17.846 INFO:teuthology.orchestra.run.smithi016.stdout:grafana.smithi016 smithi016 *:3000 running (14m) 45s ago 16m 86.9M - 9.4.7 954c08fa6188 7c43d133569c 2024-04-02T04:50:17.846 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.fyzrnm smithi016 running (50s) 45s ago 10m 14.9M - 19.0.0-1979-g6cec793d 78f830190839 98f0a83c6631 2024-04-02T04:50:17.846 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.uhgkgg smithi016 running (69s) 45s ago 10m 13.5M - 19.0.0-1979-g6cec793d 78f830190839 19be3989a243 2024-04-02T04:50:17.846 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.sdublg smithi027 running (10m) 18s ago 10m 147M - 18.2.2-1059-g4d05f4ca 909dce4fe53d a538451099db 2024-04-02T04:50:17.846 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.zwrfoo smithi027 running (26s) 18s ago 10m 12.4M - 19.0.0-1979-g6cec793d 78f830190839 acc6430302ac 2024-04-02T04:50:17.846 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi016.eqnebx smithi016 *:8443,9283,8765 running (7m) 45s ago 17m 509M - 19.0.0-1979-g6cec793d 78f830190839 d61ed91b4577 2024-04-02T04:50:17.846 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi027.kjzcsc smithi027 *:8443,9283,8765 running (6m) 18s ago 15m 451M - 19.0.0-1979-g6cec793d 78f830190839 5072a5ffc761 2024-04-02T04:50:17.847 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi016 smithi016 running (6m) 45s ago 17m 50.3M 2048M 19.0.0-1979-g6cec793d 78f830190839 eabfc84f1cdf 2024-04-02T04:50:17.847 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi027 smithi027 running (5m) 18s ago 15m 46.3M 2048M 19.0.0-1979-g6cec793d 78f830190839 fcd68b20dcc5 2024-04-02T04:50:17.847 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi016 smithi016 *:9100 running (16m) 45s ago 16m 14.8M - 1.5.0 0da6a335fe13 09f54c65b53c 2024-04-02T04:50:17.847 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi027 smithi027 *:9100 running (15m) 18s ago 15m 15.2M - 1.5.0 0da6a335fe13 e41a657c7443 2024-04-02T04:50:17.847 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 smithi016 running (4m) 45s ago 14m 87.3M 1217M 19.0.0-1979-g6cec793d 78f830190839 421d9e1cc751 2024-04-02T04:50:17.847 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 smithi016 running (3m) 45s ago 13m 91.0M 1217M 19.0.0-1979-g6cec793d 78f830190839 3612a56236a3 2024-04-02T04:50:17.847 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 smithi016 running (3m) 45s ago 13m 81.7M 1217M 19.0.0-1979-g6cec793d 78f830190839 a8e9b8d0303c 2024-04-02T04:50:17.847 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 smithi027 running (2m) 18s ago 13m 110M 2240M 19.0.0-1979-g6cec793d 78f830190839 0dd13b5be8c0 2024-04-02T04:50:17.847 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 smithi027 running (2m) 18s ago 12m 88.6M 2240M 19.0.0-1979-g6cec793d 78f830190839 9025eacf8145 2024-04-02T04:50:17.847 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 smithi027 running (87s) 18s ago 12m 87.4M 2240M 19.0.0-1979-g6cec793d 78f830190839 c2787f157377 2024-04-02T04:50:17.847 INFO:teuthology.orchestra.run.smithi016.stdout:prometheus.smithi016 smithi016 *:9095 running (6m) 45s ago 15m 64.9M - 2.43.0 a07b618ecd1d 466f00d73ddd 2024-04-02T04:50:17.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.844+0000 7fbf6e984700 1 -- 172.21.15.16:0/1510692144 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fbf5006dd60 msgr2=0x7fbf50070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:17.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.844+0000 7fbf6e984700 1 --2- 172.21.15.16:0/1510692144 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fbf5006dd60 0x7fbf50070220 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbf68101b30 tx=0x7fbf5800a380 comp rx=0 tx=0).stop 2024-04-02T04:50:17.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.844+0000 7fbf6e984700 1 -- 172.21.15.16:0/1510692144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf68101cd0 msgr2=0x7fbf680731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:17.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.844+0000 7fbf6e984700 1 --2- 172.21.15.16:0/1510692144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf68101cd0 0x7fbf680731a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c0092e0 tx=0x7fbf5c00bd70 comp rx=0 tx=0).stop 2024-04-02T04:50:17.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.844+0000 7fbf6e984700 1 -- 172.21.15.16:0/1510692144 shutdown_connections 2024-04-02T04:50:17.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.844+0000 7fbf6e984700 1 --2- 172.21.15.16:0/1510692144 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fbf5006dd60 0x7fbf50070220 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:17.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.844+0000 7fbf6e984700 1 --2- 172.21.15.16:0/1510692144 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf68100ad0 0x7fbf68074b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:17.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.844+0000 7fbf6e984700 1 --2- 172.21.15.16:0/1510692144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf68101cd0 0x7fbf680731a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:17.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.844+0000 7fbf6e984700 1 -- 172.21.15.16:0/1510692144 >> 172.21.15.16:0/1510692144 conn(0x7fbf680fc090 msgr2=0x7fbf68105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:17.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.845+0000 7fbf6e984700 1 -- 172.21.15.16:0/1510692144 shutdown_connections 2024-04-02T04:50:17.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.845+0000 7fbf6e984700 1 -- 172.21.15.16:0/1510692144 wait complete. 2024-04-02T04:50:17.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.948+0000 7feaa6574700 1 -- 172.21.15.16:0/4143267330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaa0101d90 msgr2=0x7feaa01021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:17.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.948+0000 7feaa6574700 1 --2- 172.21.15.16:0/4143267330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaa0101d90 0x7feaa01021f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fea94009910 tx=0x7fea94009c20 comp rx=0 tx=0).stop 2024-04-02T04:50:17.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.948+0000 7feaa6574700 1 -- 172.21.15.16:0/4143267330 shutdown_connections 2024-04-02T04:50:17.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.948+0000 7feaa6574700 1 --2- 172.21.15.16:0/4143267330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaa0101d90 0x7feaa01021f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:17.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.948+0000 7feaa6574700 1 --2- 172.21.15.16:0/4143267330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feaa00fedb0 0x7feaa00ff1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:17.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.948+0000 7feaa6574700 1 -- 172.21.15.16:0/4143267330 >> 172.21.15.16:0/4143267330 conn(0x7feaa00fa710 msgr2=0x7feaa00fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:17.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.948+0000 7feaa6574700 1 -- 172.21.15.16:0/4143267330 shutdown_connections 2024-04-02T04:50:17.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.948+0000 7feaa6574700 1 -- 172.21.15.16:0/4143267330 wait complete. 2024-04-02T04:50:17.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.949+0000 7feaa6574700 1 Processor -- start 2024-04-02T04:50:17.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.949+0000 7feaa6574700 1 -- start start 2024-04-02T04:50:17.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.949+0000 7feaa6574700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feaa00fedb0 0x7feaa0196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:17.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.949+0000 7feaa6574700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaa0101d90 0x7feaa0196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:17.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.949+0000 7feaa6574700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa01970c0 con 0x7feaa0101d90 2024-04-02T04:50:17.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.949+0000 7feaa6574700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa0197200 con 0x7feaa00fedb0 2024-04-02T04:50:17.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.949+0000 7fea9f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaa0101d90 0x7feaa0196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:17.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.949+0000 7fea9f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaa0101d90 0x7feaa0196aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60750/0 (socket says 172.21.15.16:60750) 2024-04-02T04:50:17.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.949+0000 7fea9f7fe700 1 -- 172.21.15.16:0/1870431688 learned_addr learned my addr 172.21.15.16:0/1870431688 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:50:17.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.949+0000 7fea9ffff700 1 --2- 172.21.15.16:0/1870431688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feaa00fedb0 0x7feaa0196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:17.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.950+0000 7fea9f7fe700 1 -- 172.21.15.16:0/1870431688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feaa00fedb0 msgr2=0x7feaa0196560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:17.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.950+0000 7fea9f7fe700 1 --2- 172.21.15.16:0/1870431688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feaa00fedb0 0x7feaa0196560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:17.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.950+0000 7fea9f7fe700 1 -- 172.21.15.16:0/1870431688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea940095f0 con 0x7feaa0101d90 2024-04-02T04:50:17.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.950+0000 7fea9f7fe700 1 --2- 172.21.15.16:0/1870431688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaa0101d90 0x7feaa0196aa0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fea94005b40 tx=0x7fea9400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:17.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.950+0000 7fea9d7fa700 1 -- 172.21.15.16:0/1870431688 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea9401d070 con 0x7feaa0101d90 2024-04-02T04:50:17.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.950+0000 7fea9d7fa700 1 -- 172.21.15.16:0/1870431688 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fea94005dc0 con 0x7feaa0101d90 2024-04-02T04:50:17.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.950+0000 7feaa6574700 1 -- 172.21.15.16:0/1870431688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaa019bbb0 con 0x7feaa0101d90 2024-04-02T04:50:17.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.950+0000 7fea9d7fa700 1 -- 172.21.15.16:0/1870431688 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea94017400 con 0x7feaa0101d90 2024-04-02T04:50:17.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.950+0000 7feaa6574700 1 -- 172.21.15.16:0/1870431688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaa019c120 con 0x7feaa0101d90 2024-04-02T04:50:17.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.951+0000 7feaa6574700 1 -- 172.21.15.16:0/1870431688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaa0190670 con 0x7feaa0101d90 2024-04-02T04:50:17.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.952+0000 7fea9d7fa700 1 -- 172.21.15.16:0/1870431688 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fea9400f6d0 con 0x7feaa0101d90 2024-04-02T04:50:17.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.952+0000 7fea9d7fa700 1 --2- 172.21.15.16:0/1870431688 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fea8806dd70 0x7fea88070230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:17.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.952+0000 7fea9d7fa700 1 -- 172.21.15.16:0/1870431688 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fea9408f010 con 0x7feaa0101d90 2024-04-02T04:50:17.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.952+0000 7fea9ffff700 1 --2- 172.21.15.16:0/1870431688 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fea8806dd70 0x7fea88070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:17.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.953+0000 7fea9ffff700 1 --2- 172.21.15.16:0/1870431688 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fea8806dd70 0x7fea88070230 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7feaa0068e70 tx=0x7fea9000b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:17.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:17.957+0000 7fea9d7fa700 1 -- 172.21.15.16:0/1870431688 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fea9405c030 con 0x7feaa0101d90 2024-04-02T04:50:18.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:18 smithi027 ceph-mon[83036]: from='client.34472 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:50:18.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:18 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:18.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:18 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:50:18.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:18 smithi027 ceph-mon[83036]: pgmap v218: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-04-02T04:50:18.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.248+0000 7feaa6574700 1 -- 172.21.15.16:0/1870431688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7feaa004ea80 con 0x7feaa0101d90 2024-04-02T04:50:18.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.248+0000 7fea9d7fa700 1 -- 172.21.15.16:0/1870431688 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fea9400f460 con 0x7feaa0101d90 2024-04-02T04:50:18.252 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:50:18.252 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": { 2024-04-02T04:50:18.252 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:50:18.252 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:50:18.252 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": { 2024-04-02T04:50:18.252 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:50:18.252 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:50:18.252 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": { 2024-04-02T04:50:18.252 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 6 2024-04-02T04:50:18.253 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:50:18.253 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-04-02T04:50:18.253 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 3 2024-04-02T04:50:18.253 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:50:18.253 INFO:teuthology.orchestra.run.smithi016.stdout: "overall": { 2024-04-02T04:50:18.253 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 13 2024-04-02T04:50:18.253 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-02T04:50:18.253 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:50:18.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.250+0000 7feaa6574700 1 -- 172.21.15.16:0/1870431688 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fea8806dd70 msgr2=0x7fea88070230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:18.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.250+0000 7feaa6574700 1 --2- 172.21.15.16:0/1870431688 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fea8806dd70 0x7fea88070230 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7feaa0068e70 tx=0x7fea9000b410 comp rx=0 tx=0).stop 2024-04-02T04:50:18.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.250+0000 7feaa6574700 1 -- 172.21.15.16:0/1870431688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaa0101d90 msgr2=0x7feaa0196aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:18.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.250+0000 7feaa6574700 1 --2- 172.21.15.16:0/1870431688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaa0101d90 0x7feaa0196aa0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fea94005b40 tx=0x7fea9400bd70 comp rx=0 tx=0).stop 2024-04-02T04:50:18.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.250+0000 7feaa6574700 1 -- 172.21.15.16:0/1870431688 shutdown_connections 2024-04-02T04:50:18.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.250+0000 7feaa6574700 1 --2- 172.21.15.16:0/1870431688 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fea8806dd70 0x7fea88070230 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:18.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.250+0000 7feaa6574700 1 --2- 172.21.15.16:0/1870431688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feaa00fedb0 0x7feaa0196560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:18.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.250+0000 7feaa6574700 1 --2- 172.21.15.16:0/1870431688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaa0101d90 0x7feaa0196aa0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:18.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.251+0000 7feaa6574700 1 -- 172.21.15.16:0/1870431688 >> 172.21.15.16:0/1870431688 conn(0x7feaa00fa710 msgr2=0x7feaa01053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:18.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.251+0000 7feaa6574700 1 -- 172.21.15.16:0/1870431688 shutdown_connections 2024-04-02T04:50:18.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.251+0000 7feaa6574700 1 -- 172.21.15.16:0/1870431688 wait complete. 2024-04-02T04:50:18.358 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:18 smithi016 ceph-mon[93230]: from='client.34472 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:50:18.358 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:18 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:18.358 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:18 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:50:18.358 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:18 smithi016 ceph-mon[93230]: pgmap v218: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-04-02T04:50:18.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.356+0000 7fd623289700 1 -- 172.21.15.16:0/3672821236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61c101570 msgr2=0x7fd61c103960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:18.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.356+0000 7fd623289700 1 --2- 172.21.15.16:0/3672821236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61c101570 0x7fd61c103960 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd618009910 tx=0x7fd618009c20 comp rx=0 tx=0).stop 2024-04-02T04:50:18.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.356+0000 7fd623289700 1 -- 172.21.15.16:0/3672821236 shutdown_connections 2024-04-02T04:50:18.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.356+0000 7fd623289700 1 --2- 172.21.15.16:0/3672821236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd61c103ea0 0x7fd61c106290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:18.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.356+0000 7fd623289700 1 --2- 172.21.15.16:0/3672821236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61c101570 0x7fd61c103960 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:18.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.356+0000 7fd623289700 1 -- 172.21.15.16:0/3672821236 >> 172.21.15.16:0/3672821236 conn(0x7fd61c0faf80 msgr2=0x7fd61c0fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:18.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.356+0000 7fd623289700 1 -- 172.21.15.16:0/3672821236 shutdown_connections 2024-04-02T04:50:18.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.356+0000 7fd623289700 1 -- 172.21.15.16:0/3672821236 wait complete. 2024-04-02T04:50:18.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.356+0000 7fd623289700 1 Processor -- start 2024-04-02T04:50:18.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.356+0000 7fd623289700 1 -- start start 2024-04-02T04:50:18.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.357+0000 7fd623289700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd61c101570 0x7fd61c196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:18.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.357+0000 7fd623289700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61c103ea0 0x7fd61c1969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:18.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.357+0000 7fd623289700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61c196fe0 con 0x7fd61c103ea0 2024-04-02T04:50:18.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.357+0000 7fd623289700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61c197120 con 0x7fd61c101570 2024-04-02T04:50:18.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.357+0000 7fd620824700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61c103ea0 0x7fd61c1969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:18.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.357+0000 7fd620824700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61c103ea0 0x7fd61c1969c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60778/0 (socket says 172.21.15.16:60778) 2024-04-02T04:50:18.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.357+0000 7fd620824700 1 -- 172.21.15.16:0/1844491408 learned_addr learned my addr 172.21.15.16:0/1844491408 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:50:18.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.357+0000 7fd621025700 1 --2- 172.21.15.16:0/1844491408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd61c101570 0x7fd61c196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:18.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.357+0000 7fd620824700 1 -- 172.21.15.16:0/1844491408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd61c101570 msgr2=0x7fd61c196480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:18.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.357+0000 7fd620824700 1 --2- 172.21.15.16:0/1844491408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd61c101570 0x7fd61c196480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:18.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.357+0000 7fd620824700 1 -- 172.21.15.16:0/1844491408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6180095f0 con 0x7fd61c103ea0 2024-04-02T04:50:18.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.357+0000 7fd620824700 1 --2- 172.21.15.16:0/1844491408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61c103ea0 0x7fd61c1969c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00c8a0 tx=0x7fd60c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:18.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.358+0000 7fd6127fc700 1 -- 172.21.15.16:0/1844491408 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd60c0078e0 con 0x7fd61c103ea0 2024-04-02T04:50:18.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.358+0000 7fd623289700 1 -- 172.21.15.16:0/1844491408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd61c19bb80 con 0x7fd61c103ea0 2024-04-02T04:50:18.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.358+0000 7fd623289700 1 -- 172.21.15.16:0/1844491408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd61c19c150 con 0x7fd61c103ea0 2024-04-02T04:50:18.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.358+0000 7fd6127fc700 1 -- 172.21.15.16:0/1844491408 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd60c00f450 con 0x7fd61c103ea0 2024-04-02T04:50:18.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.358+0000 7fd6127fc700 1 -- 172.21.15.16:0/1844491408 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd60c00e5e0 con 0x7fd61c103ea0 2024-04-02T04:50:18.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.359+0000 7fd623289700 1 -- 172.21.15.16:0/1844491408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd600005320 con 0x7fd61c103ea0 2024-04-02T04:50:18.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.359+0000 7fd6127fc700 1 -- 172.21.15.16:0/1844491408 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fd60c00e740 con 0x7fd61c103ea0 2024-04-02T04:50:18.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.360+0000 7fd6127fc700 1 --2- 172.21.15.16:0/1844491408 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd60806dd70 0x7fd608070230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:18.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.360+0000 7fd6127fc700 1 -- 172.21.15.16:0/1844491408 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fd60c08d930 con 0x7fd61c103ea0 2024-04-02T04:50:18.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.360+0000 7fd621025700 1 --2- 172.21.15.16:0/1844491408 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd60806dd70 0x7fd608070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:18.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.360+0000 7fd621025700 1 --2- 172.21.15.16:0/1844491408 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd60806dd70 0x7fd608070230 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd618006010 tx=0x7fd61800b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:18.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.365+0000 7fd6127fc700 1 -- 172.21.15.16:0/1844491408 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd60c05a7b0 con 0x7fd61c103ea0 2024-04-02T04:50:18.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.619+0000 7fd623289700 1 -- 172.21.15.16:0/1844491408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd600005de0 con 0x7fd61c103ea0 2024-04-02T04:50:18.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.620+0000 7fd6127fc700 1 -- 172.21.15.16:0/1844491408 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) v1 ==== 76+0+1746 (secure 0 0 0) 0x7fd60c05a5d0 con 0x7fd61c103ea0 2024-04-02T04:50:18.623 INFO:teuthology.orchestra.run.smithi016.stdout:e27 2024-04-02T04:50:18.623 INFO:teuthology.orchestra.run.smithi016.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:50:18.623 INFO:teuthology.orchestra.run.smithi016.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:50:18.624 INFO:teuthology.orchestra.run.smithi016.stdout:legacy client fscid: 1 2024-04-02T04:50:18.624 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:50:18.624 INFO:teuthology.orchestra.run.smithi016.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:50:18.624 INFO:teuthology.orchestra.run.smithi016.stdout:fs_name cephfs 2024-04-02T04:50:18.624 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 27 2024-04-02T04:50:18.624 INFO:teuthology.orchestra.run.smithi016.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:50:18.624 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-04-02T04:39:30.740168+0000 2024-04-02T04:50:18.624 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-04-02T04:50:17.366507+0000 2024-04-02T04:50:18.624 INFO:teuthology.orchestra.run.smithi016.stdout:tableserver 0 2024-04-02T04:50:18.624 INFO:teuthology.orchestra.run.smithi016.stdout:root 0 2024-04-02T04:50:18.624 INFO:teuthology.orchestra.run.smithi016.stdout:session_timeout 60 2024-04-02T04:50:18.624 INFO:teuthology.orchestra.run.smithi016.stdout:session_autoclose 300 2024-04-02T04:50:18.624 INFO:teuthology.orchestra.run.smithi016.stdout:max_file_size 1099511627776 2024-04-02T04:50:18.624 INFO:teuthology.orchestra.run.smithi016.stdout:max_xattr_size 65536 2024-04-02T04:50:18.624 INFO:teuthology.orchestra.run.smithi016.stdout:required_client_features {} 2024-04-02T04:50:18.624 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure 0 2024-04-02T04:50:18.625 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure_osd_epoch 84 2024-04-02T04:50:18.625 INFO:teuthology.orchestra.run.smithi016.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-02T04:50:18.625 INFO:teuthology.orchestra.run.smithi016.stdout:max_mds 1 2024-04-02T04:50:18.625 INFO:teuthology.orchestra.run.smithi016.stdout:in 0 2024-04-02T04:50:18.625 INFO:teuthology.orchestra.run.smithi016.stdout:up {0=34404} 2024-04-02T04:50:18.625 INFO:teuthology.orchestra.run.smithi016.stdout:failed 2024-04-02T04:50:18.625 INFO:teuthology.orchestra.run.smithi016.stdout:damaged 2024-04-02T04:50:18.625 INFO:teuthology.orchestra.run.smithi016.stdout:stopped 2024-04-02T04:50:18.625 INFO:teuthology.orchestra.run.smithi016.stdout:data_pools [3] 2024-04-02T04:50:18.625 INFO:teuthology.orchestra.run.smithi016.stdout:metadata_pool 2 2024-04-02T04:50:18.625 INFO:teuthology.orchestra.run.smithi016.stdout:inline_data enabled 2024-04-02T04:50:18.625 INFO:teuthology.orchestra.run.smithi016.stdout:balancer 2024-04-02T04:50:18.625 INFO:teuthology.orchestra.run.smithi016.stdout:bal_rank_mask -1 2024-04-02T04:50:18.625 INFO:teuthology.orchestra.run.smithi016.stdout:standby_count_wanted 1 2024-04-02T04:50:18.625 INFO:teuthology.orchestra.run.smithi016.stdout:qdb_cluster leader: 0 members: 2024-04-02T04:50:18.625 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.uhgkgg{0:34404} state up:reconnect seq 19 join_fscid=1 addr [v2:172.21.15.16:6826/81744708,v1:172.21.15.16:6827/81744708] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T04:50:18.626 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:50:18.626 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:50:18.626 INFO:teuthology.orchestra.run.smithi016.stdout:Standby daemons: 2024-04-02T04:50:18.626 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:50:18.626 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.fyzrnm{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.16:6828/782430688,v1:172.21.15.16:6829/782430688] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T04:50:18.626 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.zwrfoo{-1:44359} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.27:6824/1016194753,v1:172.21.15.27:6825/1016194753] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T04:50:18.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.622+0000 7fd623289700 1 -- 172.21.15.16:0/1844491408 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd60806dd70 msgr2=0x7fd608070230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:18.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.622+0000 7fd623289700 1 --2- 172.21.15.16:0/1844491408 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd60806dd70 0x7fd608070230 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd618006010 tx=0x7fd61800b540 comp rx=0 tx=0).stop 2024-04-02T04:50:18.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.622+0000 7fd623289700 1 -- 172.21.15.16:0/1844491408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61c103ea0 msgr2=0x7fd61c1969c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:18.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.622+0000 7fd623289700 1 --2- 172.21.15.16:0/1844491408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61c103ea0 0x7fd61c1969c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00c8a0 tx=0x7fd60c00cbb0 comp rx=0 tx=0).stop 2024-04-02T04:50:18.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.622+0000 7fd623289700 1 -- 172.21.15.16:0/1844491408 shutdown_connections 2024-04-02T04:50:18.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.622+0000 7fd623289700 1 --2- 172.21.15.16:0/1844491408 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd60806dd70 0x7fd608070230 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:18.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.622+0000 7fd623289700 1 --2- 172.21.15.16:0/1844491408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd61c101570 0x7fd61c196480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:18.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.622+0000 7fd623289700 1 --2- 172.21.15.16:0/1844491408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61c103ea0 0x7fd61c1969c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:18.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.622+0000 7fd623289700 1 -- 172.21.15.16:0/1844491408 >> 172.21.15.16:0/1844491408 conn(0x7fd61c0faf80 msgr2=0x7fd61c0fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:18.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.622+0000 7fd623289700 1 -- 172.21.15.16:0/1844491408 shutdown_connections 2024-04-02T04:50:18.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.622+0000 7fd623289700 1 -- 172.21.15.16:0/1844491408 wait complete. 2024-04-02T04:50:18.628 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 27 2024-04-02T04:50:18.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.728+0000 7f743f18e700 1 -- 172.21.15.16:0/2285834511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7438100c10 msgr2=0x7f7438101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:18.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.728+0000 7f743f18e700 1 --2- 172.21.15.16:0/2285834511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7438100c10 0x7f7438101070 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7424009910 tx=0x7f7424009c20 comp rx=0 tx=0).stop 2024-04-02T04:50:18.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.728+0000 7f743f18e700 1 -- 172.21.15.16:0/2285834511 shutdown_connections 2024-04-02T04:50:18.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.728+0000 7f743f18e700 1 --2- 172.21.15.16:0/2285834511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7438100c10 0x7f7438101070 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:18.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.728+0000 7f743f18e700 1 --2- 172.21.15.16:0/2285834511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74380ffa10 0x7f74380ffe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:18.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.728+0000 7f743f18e700 1 -- 172.21.15.16:0/2285834511 >> 172.21.15.16:0/2285834511 conn(0x7f74380fafb0 msgr2=0x7f74380fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:18.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.728+0000 7f743f18e700 1 -- 172.21.15.16:0/2285834511 shutdown_connections 2024-04-02T04:50:18.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.728+0000 7f743f18e700 1 -- 172.21.15.16:0/2285834511 wait complete. 2024-04-02T04:50:18.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.729+0000 7f743f18e700 1 Processor -- start 2024-04-02T04:50:18.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.729+0000 7f743f18e700 1 -- start start 2024-04-02T04:50:18.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.729+0000 7f743f18e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74380ffa10 0x7f7438069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.729+0000 7f743f18e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7438100c10 0x7f743819b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.729+0000 7f743f18e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743819b810 con 0x7f74380ffa10 2024-04-02T04:50:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.729+0000 7f743f18e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743819b950 con 0x7f7438100c10 2024-04-02T04:50:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.729+0000 7f743cf2a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74380ffa10 0x7f7438069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.729+0000 7f743cf2a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74380ffa10 0x7f7438069ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60802/0 (socket says 172.21.15.16:60802) 2024-04-02T04:50:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.729+0000 7f743cf2a700 1 -- 172.21.15.16:0/1267459265 learned_addr learned my addr 172.21.15.16:0/1267459265 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:50:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.729+0000 7f7437fff700 1 --2- 172.21.15.16:0/1267459265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7438100c10 0x7f743819b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.730+0000 7f743cf2a700 1 -- 172.21.15.16:0/1267459265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7438100c10 msgr2=0x7f743819b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.730+0000 7f743cf2a700 1 --2- 172.21.15.16:0/1267459265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7438100c10 0x7f743819b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.730+0000 7f743cf2a700 1 -- 172.21.15.16:0/1267459265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74240095f0 con 0x7f74380ffa10 2024-04-02T04:50:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.730+0000 7f743cf2a700 1 --2- 172.21.15.16:0/1267459265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74380ffa10 0x7f7438069ad0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f742c00d900 tx=0x7f742c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.730+0000 7f7435ffb700 1 -- 172.21.15.16:0/1267459265 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f742c0041f0 con 0x7f74380ffa10 2024-04-02T04:50:18.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.730+0000 7f743f18e700 1 -- 172.21.15.16:0/1267459265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f743819bc00 con 0x7f74380ffa10 2024-04-02T04:50:18.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.730+0000 7f7435ffb700 1 -- 172.21.15.16:0/1267459265 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f742c004350 con 0x7f74380ffa10 2024-04-02T04:50:18.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.730+0000 7f7435ffb700 1 -- 172.21.15.16:0/1267459265 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f742c003cf0 con 0x7f74380ffa10 2024-04-02T04:50:18.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.730+0000 7f743f18e700 1 -- 172.21.15.16:0/1267459265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f743819c150 con 0x7f74380ffa10 2024-04-02T04:50:18.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.731+0000 7f743f18e700 1 -- 172.21.15.16:0/1267459265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f743818f590 con 0x7f74380ffa10 2024-04-02T04:50:18.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.732+0000 7f7435ffb700 1 -- 172.21.15.16:0/1267459265 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f742c009730 con 0x7f74380ffa10 2024-04-02T04:50:18.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.732+0000 7f7435ffb700 1 --2- 172.21.15.16:0/1267459265 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f742806dd60 0x7f7428070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:18.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.732+0000 7f7435ffb700 1 -- 172.21.15.16:0/1267459265 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f742c08da80 con 0x7f74380ffa10 2024-04-02T04:50:18.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.732+0000 7f7437fff700 1 --2- 172.21.15.16:0/1267459265 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f742806dd60 0x7f7428070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:18.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.733+0000 7f7437fff700 1 --2- 172.21.15.16:0/1267459265 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f742806dd60 0x7f7428070220 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f74240051c0 tx=0x7f742401a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:18.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.737+0000 7f7435ffb700 1 -- 172.21.15.16:0/1267459265 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f742c05aaa0 con 0x7f74380ffa10 2024-04-02T04:50:18.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.958+0000 7f743f18e700 1 -- 172.21.15.16:0/1267459265 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f743802ce20 con 0x7f742806dd60 2024-04-02T04:50:18.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.959+0000 7f7435ffb700 1 -- 172.21.15.16:0/1267459265 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f743802ce20 con 0x7f742806dd60 2024-04-02T04:50:18.963 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:50:18.963 INFO:teuthology.orchestra.run.smithi016.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:50:18.963 INFO:teuthology.orchestra.run.smithi016.stdout: "in_progress": true, 2024-04-02T04:50:18.963 INFO:teuthology.orchestra.run.smithi016.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-02T04:50:18.963 INFO:teuthology.orchestra.run.smithi016.stdout: "services_complete": [ 2024-04-02T04:50:18.963 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr", 2024-04-02T04:50:18.963 INFO:teuthology.orchestra.run.smithi016.stdout: "osd", 2024-04-02T04:50:18.963 INFO:teuthology.orchestra.run.smithi016.stdout: "mon", 2024-04-02T04:50:18.963 INFO:teuthology.orchestra.run.smithi016.stdout: "crash" 2024-04-02T04:50:18.963 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-04-02T04:50:18.963 INFO:teuthology.orchestra.run.smithi016.stdout: "progress": "15/23 daemons upgraded", 2024-04-02T04:50:18.963 INFO:teuthology.orchestra.run.smithi016.stdout: "message": "Currently upgrading mds daemons", 2024-04-02T04:50:18.963 INFO:teuthology.orchestra.run.smithi016.stdout: "is_paused": false 2024-04-02T04:50:18.963 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:50:18.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.962+0000 7f743f18e700 1 -- 172.21.15.16:0/1267459265 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f742806dd60 msgr2=0x7f7428070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:18.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.962+0000 7f743f18e700 1 --2- 172.21.15.16:0/1267459265 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f742806dd60 0x7f7428070220 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f74240051c0 tx=0x7f742401a040 comp rx=0 tx=0).stop 2024-04-02T04:50:18.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.962+0000 7f743f18e700 1 -- 172.21.15.16:0/1267459265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74380ffa10 msgr2=0x7f7438069ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:18.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.962+0000 7f743f18e700 1 --2- 172.21.15.16:0/1267459265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74380ffa10 0x7f7438069ad0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f742c00d900 tx=0x7f742c00dc10 comp rx=0 tx=0).stop 2024-04-02T04:50:18.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.962+0000 7f743f18e700 1 -- 172.21.15.16:0/1267459265 shutdown_connections 2024-04-02T04:50:18.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.962+0000 7f743f18e700 1 --2- 172.21.15.16:0/1267459265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74380ffa10 0x7f7438069ad0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:18.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.962+0000 7f743f18e700 1 --2- 172.21.15.16:0/1267459265 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f742806dd60 0x7f7428070220 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:18.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.962+0000 7f743f18e700 1 --2- 172.21.15.16:0/1267459265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7438100c10 0x7f743819b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:18.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.962+0000 7f743f18e700 1 -- 172.21.15.16:0/1267459265 >> 172.21.15.16:0/1267459265 conn(0x7f74380fafb0 msgr2=0x7f7438104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:18.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.962+0000 7f743f18e700 1 -- 172.21.15.16:0/1267459265 shutdown_connections 2024-04-02T04:50:18.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:18.962+0000 7f743f18e700 1 -- 172.21.15.16:0/1267459265 wait complete. 2024-04-02T04:50:19.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.067+0000 7f86237cb700 1 -- 172.21.15.16:0/10123495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f861c100ae0 msgr2=0x7f861c100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:19.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.067+0000 7f86237cb700 1 --2- 172.21.15.16:0/10123495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f861c100ae0 0x7f861c100f00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8618009910 tx=0x7f8618009c20 comp rx=0 tx=0).stop 2024-04-02T04:50:19.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.067+0000 7f86237cb700 1 -- 172.21.15.16:0/10123495 shutdown_connections 2024-04-02T04:50:19.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.067+0000 7f86237cb700 1 --2- 172.21.15.16:0/10123495 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f861c101ce0 0x7f861c102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:19.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.067+0000 7f86237cb700 1 --2- 172.21.15.16:0/10123495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f861c100ae0 0x7f861c100f00 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:19.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.067+0000 7f86237cb700 1 -- 172.21.15.16:0/10123495 >> 172.21.15.16:0/10123495 conn(0x7f861c0fc0a0 msgr2=0x7f861c0fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:19.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.068+0000 7f86237cb700 1 -- 172.21.15.16:0/10123495 shutdown_connections 2024-04-02T04:50:19.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.068+0000 7f86237cb700 1 -- 172.21.15.16:0/10123495 wait complete. 2024-04-02T04:50:19.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.068+0000 7f86237cb700 1 Processor -- start 2024-04-02T04:50:19.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.068+0000 7f86237cb700 1 -- start start 2024-04-02T04:50:19.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.068+0000 7f86237cb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f861c100ae0 0x7f861c1964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:19.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.068+0000 7f86237cb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f861c101ce0 0x7f861c196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:19.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.068+0000 7f86237cb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f861c197030 con 0x7f861c101ce0 2024-04-02T04:50:19.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.068+0000 7f86237cb700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f861c197170 con 0x7f861c100ae0 2024-04-02T04:50:19.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.069+0000 7f8620d66700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f861c101ce0 0x7f861c196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:19.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.069+0000 7f8620d66700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f861c101ce0 0x7f861c196a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60814/0 (socket says 172.21.15.16:60814) 2024-04-02T04:50:19.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.069+0000 7f8620d66700 1 -- 172.21.15.16:0/1437097728 learned_addr learned my addr 172.21.15.16:0/1437097728 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:50:19.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.069+0000 7f8621567700 1 --2- 172.21.15.16:0/1437097728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f861c100ae0 0x7f861c1964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:19.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.069+0000 7f8620d66700 1 -- 172.21.15.16:0/1437097728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f861c100ae0 msgr2=0x7f861c1964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:19.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.069+0000 7f8620d66700 1 --2- 172.21.15.16:0/1437097728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f861c100ae0 0x7f861c1964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:19.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.069+0000 7f8620d66700 1 -- 172.21.15.16:0/1437097728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86180095f0 con 0x7f861c101ce0 2024-04-02T04:50:19.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.069+0000 7f8620d66700 1 --2- 172.21.15.16:0/1437097728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f861c101ce0 0x7f861c196a10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f860c00d900 tx=0x7f860c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:19.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.069+0000 7f86127fc700 1 -- 172.21.15.16:0/1437097728 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f860c0041f0 con 0x7f861c101ce0 2024-04-02T04:50:19.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.069+0000 7f86127fc700 1 -- 172.21.15.16:0/1437097728 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f860c004cf0 con 0x7f861c101ce0 2024-04-02T04:50:19.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.069+0000 7f86127fc700 1 -- 172.21.15.16:0/1437097728 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f860c00b750 con 0x7f861c101ce0 2024-04-02T04:50:19.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.069+0000 7f86237cb700 1 -- 172.21.15.16:0/1437097728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f861c19bbd0 con 0x7f861c101ce0 2024-04-02T04:50:19.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.069+0000 7f86237cb700 1 -- 172.21.15.16:0/1437097728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f861c19c120 con 0x7f861c101ce0 2024-04-02T04:50:19.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.070+0000 7f86237cb700 1 -- 172.21.15.16:0/1437097728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f861c190660 con 0x7f861c101ce0 2024-04-02T04:50:19.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.071+0000 7f86127fc700 1 -- 172.21.15.16:0/1437097728 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f860c004750 con 0x7f861c101ce0 2024-04-02T04:50:19.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.071+0000 7f86127fc700 1 --2- 172.21.15.16:0/1437097728 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f860806dd60 0x7f8608070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:19.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.071+0000 7f8621567700 1 --2- 172.21.15.16:0/1437097728 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f860806dd60 0x7f8608070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:19.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.071+0000 7f86127fc700 1 -- 172.21.15.16:0/1437097728 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f860c08da20 con 0x7f861c101ce0 2024-04-02T04:50:19.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.072+0000 7f8621567700 1 --2- 172.21.15.16:0/1437097728 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f860806dd60 0x7f8608070220 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f861800b5c0 tx=0x7f8618005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:19.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.076+0000 7f86127fc700 1 -- 172.21.15.16:0/1437097728 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f860c05aac0 con 0x7f861c101ce0 2024-04-02T04:50:19.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:19 smithi016 ceph-mon[93230]: from='client.44363 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:50:19.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:19 smithi016 ceph-mon[93230]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:50:19.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:19 smithi016 ceph-mon[93230]: reconnect by client.14540 192.168.0.1:0/2963519639 after 0.000999985 2024-04-02T04:50:19.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:19 smithi016 ceph-mon[93230]: reconnect by client.24337 192.168.0.1:0/392542449 after 0.00199997 2024-04-02T04:50:19.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:19 smithi016 ceph-mon[93230]: mds.? [v2:172.21.15.16:6826/81744708,v1:172.21.15.16:6827/81744708] up:reconnect 2024-04-02T04:50:19.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:19 smithi016 ceph-mon[93230]: fsmap cephfs:1/1 {0=cephfs.smithi016.uhgkgg=up:reconnect} 2 up:standby 2024-04-02T04:50:19.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:19 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/1870431688' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:19.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:19 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/1844491408' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:50:19.372 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:19 smithi027 ceph-mon[83036]: from='client.44363 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:50:19.372 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:19 smithi027 ceph-mon[83036]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:50:19.372 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:19 smithi027 ceph-mon[83036]: reconnect by client.14540 192.168.0.1:0/2963519639 after 0.000999985 2024-04-02T04:50:19.372 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:19 smithi027 ceph-mon[83036]: reconnect by client.24337 192.168.0.1:0/392542449 after 0.00199997 2024-04-02T04:50:19.372 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:19 smithi027 ceph-mon[83036]: mds.? [v2:172.21.15.16:6826/81744708,v1:172.21.15.16:6827/81744708] up:reconnect 2024-04-02T04:50:19.372 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:19 smithi027 ceph-mon[83036]: fsmap cephfs:1/1 {0=cephfs.smithi016.uhgkgg=up:reconnect} 2 up:standby 2024-04-02T04:50:19.372 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:19 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/1870431688' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:19.372 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:19 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/1844491408' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:50:19.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.372+0000 7f86237cb700 1 -- 172.21.15.16:0/1437097728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f861c04ea80 con 0x7f861c101ce0 2024-04-02T04:50:19.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.372+0000 7f86127fc700 1 -- 172.21.15.16:0/1437097728 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+297 (secure 0 0 0) 0x7f860c05a650 con 0x7f861c101ce0 2024-04-02T04:50:19.375 INFO:teuthology.orchestra.run.smithi016.stdout:HEALTH_WARN 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data 2024-04-02T04:50:19.375 INFO:teuthology.orchestra.run.smithi016.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-02T04:50:19.375 INFO:teuthology.orchestra.run.smithi016.stdout: fs cephfs is degraded 2024-04-02T04:50:19.376 INFO:teuthology.orchestra.run.smithi016.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-02T04:50:19.376 INFO:teuthology.orchestra.run.smithi016.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-02T04:50:19.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.374+0000 7f86237cb700 1 -- 172.21.15.16:0/1437097728 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f860806dd60 msgr2=0x7f8608070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:19.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.374+0000 7f86237cb700 1 --2- 172.21.15.16:0/1437097728 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f860806dd60 0x7f8608070220 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f861800b5c0 tx=0x7f8618005960 comp rx=0 tx=0).stop 2024-04-02T04:50:19.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.374+0000 7f86237cb700 1 -- 172.21.15.16:0/1437097728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f861c101ce0 msgr2=0x7f861c196a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:19.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.374+0000 7f86237cb700 1 --2- 172.21.15.16:0/1437097728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f861c101ce0 0x7f861c196a10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f860c00d900 tx=0x7f860c00dc10 comp rx=0 tx=0).stop 2024-04-02T04:50:19.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.374+0000 7f86237cb700 1 -- 172.21.15.16:0/1437097728 shutdown_connections 2024-04-02T04:50:19.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.374+0000 7f86237cb700 1 --2- 172.21.15.16:0/1437097728 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f860806dd60 0x7f8608070220 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:19.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.374+0000 7f86237cb700 1 --2- 172.21.15.16:0/1437097728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f861c100ae0 0x7f861c1964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:19.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.374+0000 7f86237cb700 1 --2- 172.21.15.16:0/1437097728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f861c101ce0 0x7f861c196a10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:19.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.374+0000 7f86237cb700 1 -- 172.21.15.16:0/1437097728 >> 172.21.15.16:0/1437097728 conn(0x7f861c0fc0a0 msgr2=0x7f861c1052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:19.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.374+0000 7f86237cb700 1 -- 172.21.15.16:0/1437097728 shutdown_connections 2024-04-02T04:50:19.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:19.374+0000 7f86237cb700 1 -- 172.21.15.16:0/1437097728 wait complete. 2024-04-02T04:50:20.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:20 smithi027 ceph-mon[83036]: from='client.34492 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:50:20.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:20 smithi027 ceph-mon[83036]: pgmap v219: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 6 op/s 2024-04-02T04:50:20.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:20 smithi027 ceph-mon[83036]: mds.? [v2:172.21.15.16:6826/81744708,v1:172.21.15.16:6827/81744708] up:rejoin 2024-04-02T04:50:20.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:20 smithi027 ceph-mon[83036]: fsmap cephfs:1/1 {0=cephfs.smithi016.uhgkgg=up:rejoin} 2 up:standby 2024-04-02T04:50:20.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:20 smithi027 ceph-mon[83036]: daemon mds.cephfs.smithi016.uhgkgg is now active in filesystem cephfs as rank 0 2024-04-02T04:50:20.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:20 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/1437097728' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:50:20.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:20 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:20.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:20 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:20.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:20 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:50:20.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:20 smithi016 ceph-mon[93230]: from='client.34492 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:50:20.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:20 smithi016 ceph-mon[93230]: pgmap v219: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 6 op/s 2024-04-02T04:50:20.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:20 smithi016 ceph-mon[93230]: mds.? [v2:172.21.15.16:6826/81744708,v1:172.21.15.16:6827/81744708] up:rejoin 2024-04-02T04:50:20.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:20 smithi016 ceph-mon[93230]: fsmap cephfs:1/1 {0=cephfs.smithi016.uhgkgg=up:rejoin} 2 up:standby 2024-04-02T04:50:20.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:20 smithi016 ceph-mon[93230]: daemon mds.cephfs.smithi016.uhgkgg is now active in filesystem cephfs as rank 0 2024-04-02T04:50:20.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:20 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/1437097728' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:50:20.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:20 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:20.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:20 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:20.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:20 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:50:21.374 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:21 smithi027 ceph-mon[83036]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-02T04:50:21.374 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:21 smithi027 ceph-mon[83036]: mds.? [v2:172.21.15.16:6826/81744708,v1:172.21.15.16:6827/81744708] up:active 2024-04-02T04:50:21.374 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:21 smithi027 ceph-mon[83036]: mds.? [v2:172.21.15.27:6826/1507844210,v1:172.21.15.27:6827/1507844210] up:boot 2024-04-02T04:50:21.375 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:21 smithi027 ceph-mon[83036]: fsmap cephfs:1 {0=cephfs.smithi016.uhgkgg=up:active} 3 up:standby 2024-04-02T04:50:21.375 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:21 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.sdublg"}]: dispatch 2024-04-02T04:50:21.393 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:21 smithi016 ceph-mon[93230]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-02T04:50:21.393 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:21 smithi016 ceph-mon[93230]: mds.? [v2:172.21.15.16:6826/81744708,v1:172.21.15.16:6827/81744708] up:active 2024-04-02T04:50:21.393 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:21 smithi016 ceph-mon[93230]: mds.? [v2:172.21.15.27:6826/1507844210,v1:172.21.15.27:6827/1507844210] up:boot 2024-04-02T04:50:21.394 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:21 smithi016 ceph-mon[93230]: fsmap cephfs:1 {0=cephfs.smithi016.uhgkgg=up:active} 3 up:standby 2024-04-02T04:50:21.394 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:21 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.sdublg"}]: dispatch 2024-04-02T04:50:22.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:22 smithi027 ceph-mon[83036]: pgmap v220: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 85 B/s wr, 5 op/s 2024-04-02T04:50:22.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:22 smithi016 ceph-mon[93230]: pgmap v220: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 85 B/s wr, 5 op/s 2024-04-02T04:50:23.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:23 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:23.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:23 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:23.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:23 smithi016 ceph-mon[93230]: pgmap v221: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 170 B/s wr, 6 op/s 2024-04-02T04:50:23.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:23 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:23.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:23 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:23.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:23 smithi027 ceph-mon[83036]: pgmap v221: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 170 B/s wr, 6 op/s 2024-04-02T04:50:25.059 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:24 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:25.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:24 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:25.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:24 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:25.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:24 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:25.982 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:25 smithi027 ceph-mon[83036]: pgmap v222: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 170 B/s wr, 6 op/s 2024-04-02T04:50:26.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:25 smithi016 ceph-mon[93230]: pgmap v222: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 170 B/s wr, 6 op/s 2024-04-02T04:50:28.441 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:28 smithi016 ceph-mon[93230]: pgmap v223: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 5.0 MiB/s rd, 4.1 KiB/s wr, 6 op/s 2024-04-02T04:50:28.442 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:28 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:28.442 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:28 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:28.442 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:28 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:50:28.442 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:28 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:50:28.442 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:28 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:28.442 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:28 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:50:28.442 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:28 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:28.442 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:28 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:28.442 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:28 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:28.442 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:28 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:28.442 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:28 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:28.442 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:28 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:28.442 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:28 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi016.fyzrnm"}]: dispatch 2024-04-02T04:50:28.443 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:28 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi016.fyzrnm"}]': finished 2024-04-02T04:50:28.443 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:28 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi016.uhgkgg"}]: dispatch 2024-04-02T04:50:28.443 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:28 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi016.uhgkgg"}]': finished 2024-04-02T04:50:28.443 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:28 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi027.sdublg"}]: dispatch 2024-04-02T04:50:28.443 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:28 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi027.sdublg"}]': finished 2024-04-02T04:50:28.443 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:28 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi027.zwrfoo"}]: dispatch 2024-04-02T04:50:28.443 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:28 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi027.zwrfoo"}]': finished 2024-04-02T04:50:28.443 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:28 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:28.443 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:28 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:28.443 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:28 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:28.443 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:28 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:28.443 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:28 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:28.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:28 smithi027 ceph-mon[83036]: pgmap v223: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 5.0 MiB/s rd, 4.1 KiB/s wr, 6 op/s 2024-04-02T04:50:28.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:28 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:28.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:28 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:28.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:28 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:50:28.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:28 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:50:28.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:28 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:28.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:28 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:50:28.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:28 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:28.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:28 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:28.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:28 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:28.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:28 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:28.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:28 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:28.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:28 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:28.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:28 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi016.fyzrnm"}]: dispatch 2024-04-02T04:50:28.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:28 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi016.fyzrnm"}]': finished 2024-04-02T04:50:28.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:28 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi016.uhgkgg"}]: dispatch 2024-04-02T04:50:28.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:28 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi016.uhgkgg"}]': finished 2024-04-02T04:50:28.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:28 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi027.sdublg"}]: dispatch 2024-04-02T04:50:28.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:28 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi027.sdublg"}]': finished 2024-04-02T04:50:28.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:28 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi027.zwrfoo"}]: dispatch 2024-04-02T04:50:28.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:28 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi027.zwrfoo"}]': finished 2024-04-02T04:50:28.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:28 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:28.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:28 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:28.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:28 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:28.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:28 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:28.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:28 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:29.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:29 smithi016 ceph-mon[93230]: Upgrade: Setting container_image for all mds 2024-04-02T04:50:29.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:29 smithi016 ceph-mon[93230]: Upgrade: Setting container_image for all rgw 2024-04-02T04:50:29.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:29 smithi016 ceph-mon[93230]: Upgrade: Setting container_image for all rbd-mirror 2024-04-02T04:50:29.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:29 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:29.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:29 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi016", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-02T04:50:29.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:29 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:50:29.639 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:29 smithi027 ceph-mon[83036]: Upgrade: Setting container_image for all mds 2024-04-02T04:50:29.639 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:29 smithi027 ceph-mon[83036]: Upgrade: Setting container_image for all rgw 2024-04-02T04:50:29.639 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:29 smithi027 ceph-mon[83036]: Upgrade: Setting container_image for all rbd-mirror 2024-04-02T04:50:29.639 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:29 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:29.639 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:29 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi016", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-02T04:50:29.639 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:29 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:50:30.514 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:30 smithi016 ceph-mon[93230]: pgmap v224: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 3.7 MiB/s rd, 4.1 KiB/s wr, 6 op/s 2024-04-02T04:50:30.514 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:30 smithi016 ceph-mon[93230]: Upgrade: Updating ceph-exporter.smithi016 (1/2) 2024-04-02T04:50:30.514 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:30 smithi016 ceph-mon[93230]: Deploying daemon ceph-exporter.smithi016 on smithi016 2024-04-02T04:50:30.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:30 smithi027 ceph-mon[83036]: pgmap v224: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 3.7 MiB/s rd, 4.1 KiB/s wr, 6 op/s 2024-04-02T04:50:30.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:30 smithi027 ceph-mon[83036]: Upgrade: Updating ceph-exporter.smithi016 (1/2) 2024-04-02T04:50:30.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:30 smithi027 ceph-mon[83036]: Deploying daemon ceph-exporter.smithi016 on smithi016 2024-04-02T04:50:32.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:32 smithi016 ceph-mon[93230]: pgmap v225: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 345 KiB/s rd, 4.1 KiB/s wr, 5 op/s 2024-04-02T04:50:32.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:32 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:32.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:32 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:50:32.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:32 smithi027 ceph-mon[83036]: pgmap v225: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 345 KiB/s rd, 4.1 KiB/s wr, 5 op/s 2024-04-02T04:50:32.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:32 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:32.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:32 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:50:34.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:34 smithi027 ceph-mon[83036]: pgmap v226: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 3.4 KiB/s rd, 4.0 KiB/s wr, 5 op/s 2024-04-02T04:50:34.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:34 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:34.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:34 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:34.444 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:34 smithi016 ceph-mon[93230]: pgmap v226: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 3.4 KiB/s rd, 4.0 KiB/s wr, 5 op/s 2024-04-02T04:50:34.444 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:34 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:34.444 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:34 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:36.008 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:35 smithi027 ceph-mon[83036]: Upgrade: Updating ceph-exporter.smithi027 (2/2) 2024-04-02T04:50:36.008 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:35 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:36.008 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:35 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi027", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-02T04:50:36.008 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:35 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:50:36.008 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:35 smithi027 ceph-mon[83036]: Deploying daemon ceph-exporter.smithi027 on smithi027 2024-04-02T04:50:36.008 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:35 smithi027 ceph-mon[83036]: pgmap v227: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 3.9 KiB/s wr, 3 op/s 2024-04-02T04:50:36.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:35 smithi016 ceph-mon[93230]: Upgrade: Updating ceph-exporter.smithi027 (2/2) 2024-04-02T04:50:36.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:35 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:36.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:35 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi027", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-02T04:50:36.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:35 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:50:36.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:35 smithi016 ceph-mon[93230]: Deploying daemon ceph-exporter.smithi027 on smithi027 2024-04-02T04:50:36.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:35 smithi016 ceph-mon[93230]: pgmap v227: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 3.9 KiB/s wr, 3 op/s 2024-04-02T04:50:38.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:38 smithi016 ceph-mon[93230]: pgmap v228: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 3.9 KiB/s wr, 1 op/s 2024-04-02T04:50:38.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:38 smithi027 ceph-mon[83036]: pgmap v228: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail; 3.9 KiB/s wr, 1 op/s 2024-04-02T04:50:39.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:39 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:39.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:39 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:39.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:39 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:50:39.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:39 smithi016 ceph-mon[93230]: pgmap v229: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:39.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:39 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:39.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:39 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:39.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:39 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:50:39.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:39 smithi027 ceph-mon[83036]: pgmap v229: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:42.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:41 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:42.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:41 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:42.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:41 smithi016 ceph-mon[93230]: pgmap v230: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:42.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:41 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:42.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:41 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:42.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:41 smithi027 ceph-mon[83036]: pgmap v230: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:43.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:42 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:43.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:42 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:43.349 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:42 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:43.349 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:42 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:44.286 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:43 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:44.286 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:43 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:44.286 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:43 smithi027 ceph-mon[83036]: pgmap v231: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:44.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:43 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:44.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:43 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:44.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:43 smithi016 ceph-mon[93230]: pgmap v231: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:45.297 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:45 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:45.298 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:45 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:45.309 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:45 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:45.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:45 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:46.353 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:46 smithi016 ceph-mon[93230]: pgmap v232: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:46.354 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:46 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:46.354 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:46 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:46.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:46 smithi027 ceph-mon[83036]: pgmap v232: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:46.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:46 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:46.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:46 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:47.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:47 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:50:47.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:47 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:50:48.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:48 smithi027 ceph-mon[83036]: pgmap v233: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:48.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:48 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:48.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:48 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:48.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:48 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:50:48.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:48 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:50:48.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:48 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:48.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:48 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:50:48.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:48 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:48.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:48 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:48.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:48 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:48.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:48 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:48.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:48 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:48.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:48 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:48.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:48 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:48.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:48 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:48.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:48 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:48.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:48 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:48.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:48 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi016"}]: dispatch 2024-04-02T04:50:48.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:48 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi016"}]': finished 2024-04-02T04:50:48.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:48 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi027"}]: dispatch 2024-04-02T04:50:48.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:48 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi027"}]': finished 2024-04-02T04:50:48.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:48 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:48.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:48 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:48.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:48 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:48.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:48 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:48.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:48 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:48.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:48 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:48.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:48 smithi016 ceph-mon[93230]: pgmap v233: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:48.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:48 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:48.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:48 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:48.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:48 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:50:48.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:48 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:50:48.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:48 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:48.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:48 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:50:48.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:48 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:48.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:48 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:48.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:48 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:48.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:48 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:48.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:48 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:48.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:48 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:48.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:48 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:48.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:48 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:48.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:48 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:48.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:48 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:48.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:48 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi016"}]: dispatch 2024-04-02T04:50:48.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:48 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi016"}]': finished 2024-04-02T04:50:48.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:48 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi027"}]: dispatch 2024-04-02T04:50:48.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:48 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi027"}]': finished 2024-04-02T04:50:48.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:48 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:48.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:48 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:48.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:48 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:48.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:48 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:48.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:48 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:48.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:48 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:49.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:49 smithi016 ceph-mon[93230]: Upgrade: Setting container_image for all ceph-exporter 2024-04-02T04:50:49.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:49 smithi016 ceph-mon[93230]: Upgrade: Setting container_image for all iscsi 2024-04-02T04:50:49.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:49 smithi016 ceph-mon[93230]: Upgrade: Setting container_image for all nfs 2024-04-02T04:50:49.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:49 smithi016 ceph-mon[93230]: Upgrade: Setting container_image for all nvmeof 2024-04-02T04:50:49.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:49 smithi027 ceph-mon[83036]: Upgrade: Setting container_image for all ceph-exporter 2024-04-02T04:50:49.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:49 smithi027 ceph-mon[83036]: Upgrade: Setting container_image for all iscsi 2024-04-02T04:50:49.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:49 smithi027 ceph-mon[83036]: Upgrade: Setting container_image for all nfs 2024-04-02T04:50:49.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:49 smithi027 ceph-mon[83036]: Upgrade: Setting container_image for all nvmeof 2024-04-02T04:50:49.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.481+0000 7fded9983700 1 -- 172.21.15.16:0/3068745037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fded40fedd0 msgr2=0x7fded40ff1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:49.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.481+0000 7fded9983700 1 --2- 172.21.15.16:0/3068745037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fded40fedd0 0x7fded40ff1f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdec4009910 tx=0x7fdec4009c20 comp rx=0 tx=0).stop 2024-04-02T04:50:49.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.481+0000 7fded9983700 1 -- 172.21.15.16:0/3068745037 shutdown_connections 2024-04-02T04:50:49.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.481+0000 7fded9983700 1 --2- 172.21.15.16:0/3068745037 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fded4101db0 0x7fded4102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:49.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.481+0000 7fded9983700 1 --2- 172.21.15.16:0/3068745037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fded40fedd0 0x7fded40ff1f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:49.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.481+0000 7fded9983700 1 -- 172.21.15.16:0/3068745037 >> 172.21.15.16:0/3068745037 conn(0x7fded40fa730 msgr2=0x7fded40fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:49.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.481+0000 7fded9983700 1 -- 172.21.15.16:0/3068745037 shutdown_connections 2024-04-02T04:50:49.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.481+0000 7fded9983700 1 -- 172.21.15.16:0/3068745037 wait complete. 2024-04-02T04:50:49.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.482+0000 7fded9983700 1 Processor -- start 2024-04-02T04:50:49.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.482+0000 7fded9983700 1 -- start start 2024-04-02T04:50:49.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.482+0000 7fded9983700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fded40fedd0 0x7fded41016c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:49.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.482+0000 7fded9983700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fded4101db0 0x7fded40ffd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:49.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.482+0000 7fded9983700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded41002a0 con 0x7fded40fedd0 2024-04-02T04:50:49.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.482+0000 7fded9983700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded41003e0 con 0x7fded4101db0 2024-04-02T04:50:49.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.482+0000 7fded2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fded40fedd0 0x7fded41016c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:49.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.482+0000 7fded2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fded40fedd0 0x7fded41016c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36746/0 (socket says 172.21.15.16:36746) 2024-04-02T04:50:49.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.482+0000 7fded2ffd700 1 -- 172.21.15.16:0/273926381 learned_addr learned my addr 172.21.15.16:0/273926381 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:50:49.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.482+0000 7fded2ffd700 1 -- 172.21.15.16:0/273926381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fded4101db0 msgr2=0x7fded40ffd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:49.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.482+0000 7fded27fc700 1 --2- 172.21.15.16:0/273926381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fded4101db0 0x7fded40ffd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:49.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.482+0000 7fded2ffd700 1 --2- 172.21.15.16:0/273926381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fded4101db0 0x7fded40ffd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:49.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.482+0000 7fded2ffd700 1 -- 172.21.15.16:0/273926381 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec40095f0 con 0x7fded40fedd0 2024-04-02T04:50:49.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.483+0000 7fded27fc700 1 --2- 172.21.15.16:0/273926381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fded4101db0 0x7fded40ffd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T04:50:49.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.483+0000 7fded2ffd700 1 --2- 172.21.15.16:0/273926381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fded40fedd0 0x7fded41016c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdec40059d0 tx=0x7fdec400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:49.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.483+0000 7fded8981700 1 -- 172.21.15.16:0/273926381 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdec401d070 con 0x7fded40fedd0 2024-04-02T04:50:49.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.483+0000 7fded8981700 1 -- 172.21.15.16:0/273926381 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdec400f460 con 0x7fded40fedd0 2024-04-02T04:50:49.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.483+0000 7fded9983700 1 -- 172.21.15.16:0/273926381 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fded4100660 con 0x7fded40fedd0 2024-04-02T04:50:49.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.483+0000 7fded9983700 1 -- 172.21.15.16:0/273926381 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fded4100bd0 con 0x7fded40fedd0 2024-04-02T04:50:49.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.483+0000 7fded8981700 1 -- 172.21.15.16:0/273926381 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdec4021640 con 0x7fded40fedd0 2024-04-02T04:50:49.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.484+0000 7fded9983700 1 -- 172.21.15.16:0/273926381 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fded4190710 con 0x7fded40fedd0 2024-04-02T04:50:49.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.484+0000 7fded8981700 1 -- 172.21.15.16:0/273926381 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fdec400f5d0 con 0x7fded40fedd0 2024-04-02T04:50:49.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.485+0000 7fded8981700 1 --2- 172.21.15.16:0/273926381 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fdebc06dd60 0x7fdebc070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:49.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.485+0000 7fded8981700 1 -- 172.21.15.16:0/273926381 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fdec408ecc0 con 0x7fded40fedd0 2024-04-02T04:50:49.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.485+0000 7fded27fc700 1 --2- 172.21.15.16:0/273926381 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fdebc06dd60 0x7fdebc070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:49.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.486+0000 7fded27fc700 1 --2- 172.21.15.16:0/273926381 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fdebc06dd60 0x7fdebc070220 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdec8007900 tx=0x7fdec8008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:49.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.490+0000 7fded8981700 1 -- 172.21.15.16:0/273926381 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fdec405bd60 con 0x7fded40fedd0 2024-04-02T04:50:49.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.723+0000 7fded9983700 1 -- 172.21.15.16:0/273926381 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fded4100bd0 con 0x7fdebc06dd60 2024-04-02T04:50:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.724+0000 7fded8981700 1 -- 172.21.15.16:0/273926381 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+466 (secure 0 0 0) 0x7fded4100bd0 con 0x7fdebc06dd60 2024-04-02T04:50:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.726+0000 7fded9983700 1 -- 172.21.15.16:0/273926381 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fdebc06dd60 msgr2=0x7fdebc070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.726+0000 7fded9983700 1 --2- 172.21.15.16:0/273926381 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fdebc06dd60 0x7fdebc070220 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdec8007900 tx=0x7fdec8008040 comp rx=0 tx=0).stop 2024-04-02T04:50:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.726+0000 7fded9983700 1 -- 172.21.15.16:0/273926381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fded40fedd0 msgr2=0x7fded41016c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.726+0000 7fded9983700 1 --2- 172.21.15.16:0/273926381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fded40fedd0 0x7fded41016c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdec40059d0 tx=0x7fdec400be20 comp rx=0 tx=0).stop 2024-04-02T04:50:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.726+0000 7fded9983700 1 -- 172.21.15.16:0/273926381 shutdown_connections 2024-04-02T04:50:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.726+0000 7fded9983700 1 --2- 172.21.15.16:0/273926381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fded40fedd0 0x7fded41016c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.726+0000 7fded9983700 1 --2- 172.21.15.16:0/273926381 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fdebc06dd60 0x7fdebc070220 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.726+0000 7fded9983700 1 --2- 172.21.15.16:0/273926381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fded4101db0 0x7fded40ffd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.726+0000 7fded9983700 1 -- 172.21.15.16:0/273926381 >> 172.21.15.16:0/273926381 conn(0x7fded40fa730 msgr2=0x7fded4105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.726+0000 7fded9983700 1 -- 172.21.15.16:0/273926381 shutdown_connections 2024-04-02T04:50:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.726+0000 7fded9983700 1 -- 172.21.15.16:0/273926381 wait complete. 2024-04-02T04:50:49.741 INFO:teuthology.orchestra.run.smithi016.stdout:true 2024-04-02T04:50:49.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.836+0000 7fb8f5d0d700 1 -- 172.21.15.16:0/745730438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8f0103e90 msgr2=0x7fb8f0106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:49.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.836+0000 7fb8f5d0d700 1 --2- 172.21.15.16:0/745730438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8f0103e90 0x7fb8f0106280 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4009910 tx=0x7fb8e4009c20 comp rx=0 tx=0).stop 2024-04-02T04:50:49.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.836+0000 7fb8f5d0d700 1 -- 172.21.15.16:0/745730438 shutdown_connections 2024-04-02T04:50:49.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.836+0000 7fb8f5d0d700 1 --2- 172.21.15.16:0/745730438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8f0103e90 0x7fb8f0106280 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:49.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.836+0000 7fb8f5d0d700 1 --2- 172.21.15.16:0/745730438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8f0101560 0x7fb8f0103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:49.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.836+0000 7fb8f5d0d700 1 -- 172.21.15.16:0/745730438 >> 172.21.15.16:0/745730438 conn(0x7fb8f00faf70 msgr2=0x7fb8f00fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:49.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.836+0000 7fb8f5d0d700 1 -- 172.21.15.16:0/745730438 shutdown_connections 2024-04-02T04:50:49.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.836+0000 7fb8f5d0d700 1 -- 172.21.15.16:0/745730438 wait complete. 2024-04-02T04:50:49.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.837+0000 7fb8f5d0d700 1 Processor -- start 2024-04-02T04:50:49.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.837+0000 7fb8f5d0d700 1 -- start start 2024-04-02T04:50:49.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.837+0000 7fb8f5d0d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8f0101560 0x7fb8f01986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:49.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.837+0000 7fb8f5d0d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8f0103e90 0x7fb8f0198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:49.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.837+0000 7fb8f5d0d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f0199230 con 0x7fb8f0103e90 2024-04-02T04:50:49.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.837+0000 7fb8f5d0d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f0199370 con 0x7fb8f0101560 2024-04-02T04:50:49.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.837+0000 7fb8eeffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8f0103e90 0x7fb8f0198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:49.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.837+0000 7fb8eeffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8f0103e90 0x7fb8f0198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36756/0 (socket says 172.21.15.16:36756) 2024-04-02T04:50:49.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.837+0000 7fb8eeffd700 1 -- 172.21.15.16:0/2785627194 learned_addr learned my addr 172.21.15.16:0/2785627194 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:50:49.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.838+0000 7fb8ef7fe700 1 --2- 172.21.15.16:0/2785627194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8f0101560 0x7fb8f01986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:49.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.838+0000 7fb8eeffd700 1 -- 172.21.15.16:0/2785627194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8f0101560 msgr2=0x7fb8f01986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:49.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.838+0000 7fb8eeffd700 1 --2- 172.21.15.16:0/2785627194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8f0101560 0x7fb8f01986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:49.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.838+0000 7fb8eeffd700 1 -- 172.21.15.16:0/2785627194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e40095f0 con 0x7fb8f0103e90 2024-04-02T04:50:49.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.838+0000 7fb8eeffd700 1 --2- 172.21.15.16:0/2785627194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8f0103e90 0x7fb8f0198c10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4006010 tx=0x7fb8e400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:49.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.838+0000 7fb8ecff9700 1 -- 172.21.15.16:0/2785627194 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8e401d070 con 0x7fb8f0103e90 2024-04-02T04:50:49.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.838+0000 7fb8ecff9700 1 -- 172.21.15.16:0/2785627194 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb8e400f460 con 0x7fb8f0103e90 2024-04-02T04:50:49.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.838+0000 7fb8ecff9700 1 -- 172.21.15.16:0/2785627194 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8e4021620 con 0x7fb8f0103e90 2024-04-02T04:50:49.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.838+0000 7fb8f5d0d700 1 -- 172.21.15.16:0/2785627194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8f019dd70 con 0x7fb8f0103e90 2024-04-02T04:50:49.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.838+0000 7fb8f5d0d700 1 -- 172.21.15.16:0/2785627194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8f019e290 con 0x7fb8f0103e90 2024-04-02T04:50:49.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.839+0000 7fb8f5d0d700 1 -- 172.21.15.16:0/2785627194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8f004ea80 con 0x7fb8f0103e90 2024-04-02T04:50:49.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.840+0000 7fb8ecff9700 1 -- 172.21.15.16:0/2785627194 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fb8e400f5d0 con 0x7fb8f0103e90 2024-04-02T04:50:49.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.840+0000 7fb8ecff9700 1 --2- 172.21.15.16:0/2785627194 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fb8d806dd60 0x7fb8d8070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:49.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.840+0000 7fb8ecff9700 1 -- 172.21.15.16:0/2785627194 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fb8e408eee0 con 0x7fb8f0103e90 2024-04-02T04:50:49.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.841+0000 7fb8ef7fe700 1 --2- 172.21.15.16:0/2785627194 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fb8d806dd60 0x7fb8d8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:49.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.841+0000 7fb8ef7fe700 1 --2- 172.21.15.16:0/2785627194 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fb8d806dd60 0x7fb8d8070220 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0005950 tx=0x7fb8e000a300 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:49.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:49.849+0000 7fb8ecff9700 1 -- 172.21.15.16:0/2785627194 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb8e405bf00 con 0x7fb8f0103e90 2024-04-02T04:50:50.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.073+0000 7fb8f5d0d700 1 -- 172.21.15.16:0/2785627194 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8f00ffda0 con 0x7fb8d806dd60 2024-04-02T04:50:50.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.074+0000 7fb8ecff9700 1 -- 172.21.15.16:0/2785627194 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+466 (secure 0 0 0) 0x7fb8f00ffda0 con 0x7fb8d806dd60 2024-04-02T04:50:50.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.076+0000 7fb8f5d0d700 1 -- 172.21.15.16:0/2785627194 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fb8d806dd60 msgr2=0x7fb8d8070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:50.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.076+0000 7fb8f5d0d700 1 --2- 172.21.15.16:0/2785627194 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fb8d806dd60 0x7fb8d8070220 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0005950 tx=0x7fb8e000a300 comp rx=0 tx=0).stop 2024-04-02T04:50:50.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.076+0000 7fb8f5d0d700 1 -- 172.21.15.16:0/2785627194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8f0103e90 msgr2=0x7fb8f0198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:50.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.076+0000 7fb8f5d0d700 1 --2- 172.21.15.16:0/2785627194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8f0103e90 0x7fb8f0198c10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4006010 tx=0x7fb8e400bd70 comp rx=0 tx=0).stop 2024-04-02T04:50:50.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.076+0000 7fb8f5d0d700 1 -- 172.21.15.16:0/2785627194 shutdown_connections 2024-04-02T04:50:50.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.076+0000 7fb8f5d0d700 1 --2- 172.21.15.16:0/2785627194 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fb8d806dd60 0x7fb8d8070220 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:50.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.076+0000 7fb8f5d0d700 1 --2- 172.21.15.16:0/2785627194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8f0101560 0x7fb8f01986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:50.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.076+0000 7fb8f5d0d700 1 --2- 172.21.15.16:0/2785627194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8f0103e90 0x7fb8f0198c10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:50.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.076+0000 7fb8f5d0d700 1 -- 172.21.15.16:0/2785627194 >> 172.21.15.16:0/2785627194 conn(0x7fb8f00faf70 msgr2=0x7fb8f00ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:50.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.077+0000 7fb8f5d0d700 1 -- 172.21.15.16:0/2785627194 shutdown_connections 2024-04-02T04:50:50.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.077+0000 7fb8f5d0d700 1 -- 172.21.15.16:0/2785627194 wait complete. 2024-04-02T04:50:50.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.186+0000 7fbcc5d9e700 1 -- 172.21.15.16:0/2472987487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcc00ffa70 msgr2=0x7fbcc00ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:50.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.186+0000 7fbcc5d9e700 1 --2- 172.21.15.16:0/2472987487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcc00ffa70 0x7fbcc00ffef0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbcac009910 tx=0x7fbcac009c20 comp rx=0 tx=0).stop 2024-04-02T04:50:50.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.186+0000 7fbcc5d9e700 1 -- 172.21.15.16:0/2472987487 shutdown_connections 2024-04-02T04:50:50.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.186+0000 7fbcc5d9e700 1 --2- 172.21.15.16:0/2472987487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcc00ffa70 0x7fbcc00ffef0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:50.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.186+0000 7fbcc5d9e700 1 --2- 172.21.15.16:0/2472987487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcc0100e10 0x7fbcc0101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:50.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.186+0000 7fbcc5d9e700 1 -- 172.21.15.16:0/2472987487 >> 172.21.15.16:0/2472987487 conn(0x7fbcc00fa580 msgr2=0x7fbcc00fc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:50.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.186+0000 7fbcc5d9e700 1 -- 172.21.15.16:0/2472987487 shutdown_connections 2024-04-02T04:50:50.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.186+0000 7fbcc5d9e700 1 -- 172.21.15.16:0/2472987487 wait complete. 2024-04-02T04:50:50.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.187+0000 7fbcc5d9e700 1 Processor -- start 2024-04-02T04:50:50.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.187+0000 7fbcc5d9e700 1 -- start start 2024-04-02T04:50:50.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.187+0000 7fbcc5d9e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcc00ffa70 0x7fbcc01985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:50.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.187+0000 7fbcc5d9e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcc0100e10 0x7fbcc0198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:50.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.187+0000 7fbcc5d9e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc0199130 con 0x7fbcc0100e10 2024-04-02T04:50:50.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.187+0000 7fbcc5d9e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc0199270 con 0x7fbcc00ffa70 2024-04-02T04:50:50.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.187+0000 7fbcbffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcc0100e10 0x7fbcc0198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:50.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.187+0000 7fbcbffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcc0100e10 0x7fbcc0198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36782/0 (socket says 172.21.15.16:36782) 2024-04-02T04:50:50.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.187+0000 7fbcbffff700 1 -- 172.21.15.16:0/2883785621 learned_addr learned my addr 172.21.15.16:0/2883785621 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:50:50.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.187+0000 7fbcc4d9c700 1 --2- 172.21.15.16:0/2883785621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcc00ffa70 0x7fbcc01985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:50.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.188+0000 7fbcbffff700 1 -- 172.21.15.16:0/2883785621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcc00ffa70 msgr2=0x7fbcc01985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:50.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.188+0000 7fbcbffff700 1 --2- 172.21.15.16:0/2883785621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcc00ffa70 0x7fbcc01985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:50.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.188+0000 7fbcbffff700 1 -- 172.21.15.16:0/2883785621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcac0095f0 con 0x7fbcc0100e10 2024-04-02T04:50:50.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.188+0000 7fbcbffff700 1 --2- 172.21.15.16:0/2883785621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcc0100e10 0x7fbcc0198b10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbcac0059d0 tx=0x7fbcac00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:50.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.188+0000 7fbcbdffb700 1 -- 172.21.15.16:0/2883785621 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcac01d070 con 0x7fbcc0100e10 2024-04-02T04:50:50.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.188+0000 7fbcc5d9e700 1 -- 172.21.15.16:0/2883785621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcc019dc20 con 0x7fbcc0100e10 2024-04-02T04:50:50.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.188+0000 7fbcbdffb700 1 -- 172.21.15.16:0/2883785621 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbcac00f460 con 0x7fbcc0100e10 2024-04-02T04:50:50.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.188+0000 7fbcbdffb700 1 -- 172.21.15.16:0/2883785621 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcac021640 con 0x7fbcc0100e10 2024-04-02T04:50:50.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.188+0000 7fbcc5d9e700 1 -- 172.21.15.16:0/2883785621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcc019e190 con 0x7fbcc0100e10 2024-04-02T04:50:50.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.189+0000 7fbcc5d9e700 1 -- 172.21.15.16:0/2883785621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcc004f2d0 con 0x7fbcc0100e10 2024-04-02T04:50:50.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.190+0000 7fbcbdffb700 1 -- 172.21.15.16:0/2883785621 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fbcac00f5d0 con 0x7fbcc0100e10 2024-04-02T04:50:50.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.190+0000 7fbcbdffb700 1 --2- 172.21.15.16:0/2883785621 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fbcb006dd60 0x7fbcb0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:50.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.190+0000 7fbcbdffb700 1 -- 172.21.15.16:0/2883785621 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fbcac08f000 con 0x7fbcc0100e10 2024-04-02T04:50:50.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.190+0000 7fbcc4d9c700 1 --2- 172.21.15.16:0/2883785621 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fbcb006dd60 0x7fbcb0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:50.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.191+0000 7fbcc4d9c700 1 --2- 172.21.15.16:0/2883785621 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fbcb006dd60 0x7fbcb0070220 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4005fd0 tx=0x7fbcb4005dd0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:50.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.196+0000 7fbcbdffb700 1 -- 172.21.15.16:0/2883785621 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbcac0c3280 con 0x7fbcc0100e10 2024-04-02T04:50:50.358 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:50 smithi016 ceph-mon[93230]: pgmap v234: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:50.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:50 smithi027 ceph-mon[83036]: pgmap v234: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.425+0000 7fbcc5d9e700 1 -- 172.21.15.16:0/2883785621 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbcc00fc080 con 0x7fbcb006dd60 2024-04-02T04:50:50.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.437+0000 7fbcbdffb700 1 -- 172.21.15.16:0/2883785621 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fbcc00fc080 con 0x7fbcb006dd60 2024-04-02T04:50:50.441 INFO:teuthology.orchestra.run.smithi016.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:50:50.442 INFO:teuthology.orchestra.run.smithi016.stdout:alertmanager.smithi016 smithi016 *:9093,9094 running (15m) 7s ago 16m 24.6M - 0.25.0 c8568f914cd2 b95d8354b016 2024-04-02T04:50:50.442 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi016 smithi016 running (17s) 7s ago 17m 7302k - 19.0.0-1979-g6cec793d 78f830190839 1d3ba1f157b9 2024-04-02T04:50:50.442 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi027 smithi027 running (12s) 9s ago 15m 6966k - 19.0.0-1979-g6cec793d 78f830190839 3402997325e5 2024-04-02T04:50:50.442 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi016 smithi016 running (5m) 7s ago 17m 6660k - 19.0.0-1979-g6cec793d 78f830190839 dc63ce0eef58 2024-04-02T04:50:50.442 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi027 smithi027 running (5m) 9s ago 15m 6665k - 19.0.0-1979-g6cec793d 78f830190839 2beb077018e4 2024-04-02T04:50:50.442 INFO:teuthology.orchestra.run.smithi016.stdout:grafana.smithi016 smithi016 *:3000 running (15m) 7s ago 16m 87.8M - 9.4.7 954c08fa6188 7c43d133569c 2024-04-02T04:50:50.442 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.fyzrnm smithi016 running (83s) 7s ago 11m 16.0M - 19.0.0-1979-g6cec793d 78f830190839 98f0a83c6631 2024-04-02T04:50:50.442 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.uhgkgg smithi016 running (102s) 7s ago 11m 89.7M - 19.0.0-1979-g6cec793d 78f830190839 19be3989a243 2024-04-02T04:50:50.442 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.sdublg smithi027 running (31s) 9s ago 11m 12.7M - 19.0.0-1979-g6cec793d 78f830190839 7c307783baab 2024-04-02T04:50:50.442 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.zwrfoo smithi027 running (59s) 9s ago 11m 13.5M - 19.0.0-1979-g6cec793d 78f830190839 acc6430302ac 2024-04-02T04:50:50.442 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi016.eqnebx smithi016 *:8443,9283,8765 running (7m) 7s ago 18m 511M - 19.0.0-1979-g6cec793d 78f830190839 d61ed91b4577 2024-04-02T04:50:50.442 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi027.kjzcsc smithi027 *:8443,9283,8765 running (7m) 9s ago 15m 451M - 19.0.0-1979-g6cec793d 78f830190839 5072a5ffc761 2024-04-02T04:50:50.442 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi016 smithi016 running (6m) 7s ago 18m 53.8M 2048M 19.0.0-1979-g6cec793d 78f830190839 eabfc84f1cdf 2024-04-02T04:50:50.442 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi027 smithi027 running (6m) 9s ago 15m 43.7M 2048M 19.0.0-1979-g6cec793d 78f830190839 fcd68b20dcc5 2024-04-02T04:50:50.443 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi016 smithi016 *:9100 running (16m) 7s ago 16m 14.7M - 1.5.0 0da6a335fe13 09f54c65b53c 2024-04-02T04:50:50.443 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi027 smithi027 *:9100 running (15m) 9s ago 15m 15.1M - 1.5.0 0da6a335fe13 e41a657c7443 2024-04-02T04:50:50.443 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 smithi016 running (4m) 7s ago 14m 91.2M 1217M 19.0.0-1979-g6cec793d 78f830190839 421d9e1cc751 2024-04-02T04:50:50.443 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 smithi016 running (4m) 7s ago 14m 100M 1217M 19.0.0-1979-g6cec793d 78f830190839 3612a56236a3 2024-04-02T04:50:50.443 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 smithi016 running (3m) 7s ago 13m 91.3M 1217M 19.0.0-1979-g6cec793d 78f830190839 a8e9b8d0303c 2024-04-02T04:50:50.443 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 smithi027 running (3m) 9s ago 13m 112M 2240M 19.0.0-1979-g6cec793d 78f830190839 0dd13b5be8c0 2024-04-02T04:50:50.443 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 smithi027 running (2m) 9s ago 13m 89.5M 2240M 19.0.0-1979-g6cec793d 78f830190839 9025eacf8145 2024-04-02T04:50:50.443 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 smithi027 running (119s) 9s ago 12m 89.5M 2240M 19.0.0-1979-g6cec793d 78f830190839 c2787f157377 2024-04-02T04:50:50.443 INFO:teuthology.orchestra.run.smithi016.stdout:prometheus.smithi016 smithi016 *:9095 running (7m) 7s ago 16m 67.0M - 2.43.0 a07b618ecd1d 466f00d73ddd 2024-04-02T04:50:50.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.439+0000 7fbcc5d9e700 1 -- 172.21.15.16:0/2883785621 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fbcb006dd60 msgr2=0x7fbcb0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:50.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.439+0000 7fbcc5d9e700 1 --2- 172.21.15.16:0/2883785621 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fbcb006dd60 0x7fbcb0070220 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4005fd0 tx=0x7fbcb4005dd0 comp rx=0 tx=0).stop 2024-04-02T04:50:50.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.439+0000 7fbcc5d9e700 1 -- 172.21.15.16:0/2883785621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcc0100e10 msgr2=0x7fbcc0198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:50.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.439+0000 7fbcc5d9e700 1 --2- 172.21.15.16:0/2883785621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcc0100e10 0x7fbcc0198b10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbcac0059d0 tx=0x7fbcac00be20 comp rx=0 tx=0).stop 2024-04-02T04:50:50.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.439+0000 7fbcc5d9e700 1 -- 172.21.15.16:0/2883785621 shutdown_connections 2024-04-02T04:50:50.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.439+0000 7fbcc5d9e700 1 --2- 172.21.15.16:0/2883785621 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fbcb006dd60 0x7fbcb0070220 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:50.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.439+0000 7fbcc5d9e700 1 --2- 172.21.15.16:0/2883785621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcc00ffa70 0x7fbcc01985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:50.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.439+0000 7fbcc5d9e700 1 --2- 172.21.15.16:0/2883785621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcc0100e10 0x7fbcc0198b10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:50.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.439+0000 7fbcc5d9e700 1 -- 172.21.15.16:0/2883785621 >> 172.21.15.16:0/2883785621 conn(0x7fbcc00fa580 msgr2=0x7fbcc00fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:50.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.439+0000 7fbcc5d9e700 1 -- 172.21.15.16:0/2883785621 shutdown_connections 2024-04-02T04:50:50.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.439+0000 7fbcc5d9e700 1 -- 172.21.15.16:0/2883785621 wait complete. 2024-04-02T04:50:50.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.561+0000 7f2453c57700 1 -- 172.21.15.16:0/80011923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f244c100ad0 msgr2=0x7f244c100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:50.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.561+0000 7f2453c57700 1 --2- 172.21.15.16:0/80011923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f244c100ad0 0x7f244c100ef0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2440009910 tx=0x7f2440009c20 comp rx=0 tx=0).stop 2024-04-02T04:50:50.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.561+0000 7f2453c57700 1 -- 172.21.15.16:0/80011923 shutdown_connections 2024-04-02T04:50:50.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.561+0000 7f2453c57700 1 --2- 172.21.15.16:0/80011923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f244c101cd0 0x7f244c102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:50.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.561+0000 7f2453c57700 1 --2- 172.21.15.16:0/80011923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f244c100ad0 0x7f244c100ef0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:50.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.561+0000 7f2453c57700 1 -- 172.21.15.16:0/80011923 >> 172.21.15.16:0/80011923 conn(0x7f244c0fc090 msgr2=0x7f244c0fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:50.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.561+0000 7f2453c57700 1 -- 172.21.15.16:0/80011923 shutdown_connections 2024-04-02T04:50:50.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.561+0000 7f2453c57700 1 -- 172.21.15.16:0/80011923 wait complete. 2024-04-02T04:50:50.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.562+0000 7f2453c57700 1 Processor -- start 2024-04-02T04:50:50.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.562+0000 7f2453c57700 1 -- start start 2024-04-02T04:50:50.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.562+0000 7f2453c57700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f244c100ad0 0x7f244c1964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:50.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.562+0000 7f2453c57700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f244c101cd0 0x7f244c196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:50.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.562+0000 7f2453c57700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244c197050 con 0x7f244c100ad0 2024-04-02T04:50:50.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.562+0000 7f2453c57700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244c197190 con 0x7f244c101cd0 2024-04-02T04:50:50.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.562+0000 7f24519f3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f244c100ad0 0x7f244c1964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:50.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.562+0000 7f24519f3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f244c100ad0 0x7f244c1964f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36798/0 (socket says 172.21.15.16:36798) 2024-04-02T04:50:50.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.562+0000 7f24519f3700 1 -- 172.21.15.16:0/171439158 learned_addr learned my addr 172.21.15.16:0/171439158 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:50:50.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.563+0000 7f24511f2700 1 --2- 172.21.15.16:0/171439158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f244c101cd0 0x7f244c196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:50.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.563+0000 7f24519f3700 1 -- 172.21.15.16:0/171439158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f244c101cd0 msgr2=0x7f244c196a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:50.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.563+0000 7f24519f3700 1 --2- 172.21.15.16:0/171439158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f244c101cd0 0x7f244c196a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:50.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.563+0000 7f24519f3700 1 -- 172.21.15.16:0/171439158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24400095f0 con 0x7f244c100ad0 2024-04-02T04:50:50.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.563+0000 7f24511f2700 1 --2- 172.21.15.16:0/171439158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f244c101cd0 0x7f244c196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T04:50:50.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.563+0000 7f24519f3700 1 --2- 172.21.15.16:0/171439158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f244c100ad0 0x7f244c1964f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f24400059d0 tx=0x7f244000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:50.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.563+0000 7f243effd700 1 -- 172.21.15.16:0/171439158 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f244001d070 con 0x7f244c100ad0 2024-04-02T04:50:50.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.563+0000 7f243effd700 1 -- 172.21.15.16:0/171439158 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f244000f460 con 0x7f244c100ad0 2024-04-02T04:50:50.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.563+0000 7f2453c57700 1 -- 172.21.15.16:0/171439158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f244c19bb90 con 0x7f244c100ad0 2024-04-02T04:50:50.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.563+0000 7f243effd700 1 -- 172.21.15.16:0/171439158 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2440021640 con 0x7f244c100ad0 2024-04-02T04:50:50.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.563+0000 7f2453c57700 1 -- 172.21.15.16:0/171439158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f244c19c080 con 0x7f244c100ad0 2024-04-02T04:50:50.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.564+0000 7f2453c57700 1 -- 172.21.15.16:0/171439158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f244c190650 con 0x7f244c100ad0 2024-04-02T04:50:50.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.565+0000 7f243effd700 1 -- 172.21.15.16:0/171439158 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f244000f5d0 con 0x7f244c100ad0 2024-04-02T04:50:50.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.565+0000 7f243effd700 1 --2- 172.21.15.16:0/171439158 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f243806dd60 0x7f2438070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:50.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.565+0000 7f243effd700 1 -- 172.21.15.16:0/171439158 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f244008ee80 con 0x7f244c100ad0 2024-04-02T04:50:50.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.566+0000 7f24511f2700 1 --2- 172.21.15.16:0/171439158 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f243806dd60 0x7f2438070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:50.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.566+0000 7f24511f2700 1 --2- 172.21.15.16:0/171439158 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f243806dd60 0x7f2438070220 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2448005d90 tx=0x7f2448005d20 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:50.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.571+0000 7f243effd700 1 -- 172.21.15.16:0/171439158 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f244005bea0 con 0x7f244c100ad0 2024-04-02T04:50:50.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.927+0000 7f2453c57700 1 -- 172.21.15.16:0/171439158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f244c02cd80 con 0x7f244c100ad0 2024-04-02T04:50:50.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.928+0000 7f243effd700 1 -- 172.21.15.16:0/171439158 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f244005ba30 con 0x7f244c100ad0 2024-04-02T04:50:50.933 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:50:50.933 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": { 2024-04-02T04:50:50.933 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:50:50.933 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:50:50.933 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": { 2024-04-02T04:50:50.933 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:50:50.933 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:50:50.933 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": { 2024-04-02T04:50:50.933 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 6 2024-04-02T04:50:50.933 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:50:50.933 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-04-02T04:50:50.934 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 4 2024-04-02T04:50:50.934 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:50:50.934 INFO:teuthology.orchestra.run.smithi016.stdout: "overall": { 2024-04-02T04:50:50.934 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 14 2024-04-02T04:50:50.934 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-02T04:50:50.934 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:50:50.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.930+0000 7f2453c57700 1 -- 172.21.15.16:0/171439158 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f243806dd60 msgr2=0x7f2438070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:50.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.930+0000 7f2453c57700 1 --2- 172.21.15.16:0/171439158 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f243806dd60 0x7f2438070220 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2448005d90 tx=0x7f2448005d20 comp rx=0 tx=0).stop 2024-04-02T04:50:50.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.930+0000 7f2453c57700 1 -- 172.21.15.16:0/171439158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f244c100ad0 msgr2=0x7f244c1964f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:50.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.930+0000 7f2453c57700 1 --2- 172.21.15.16:0/171439158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f244c100ad0 0x7f244c1964f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f24400059d0 tx=0x7f244000be20 comp rx=0 tx=0).stop 2024-04-02T04:50:50.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.931+0000 7f2453c57700 1 -- 172.21.15.16:0/171439158 shutdown_connections 2024-04-02T04:50:50.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.931+0000 7f2453c57700 1 --2- 172.21.15.16:0/171439158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f244c100ad0 0x7f244c1964f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:50.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.931+0000 7f2453c57700 1 --2- 172.21.15.16:0/171439158 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f243806dd60 0x7f2438070220 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:50.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.931+0000 7f2453c57700 1 --2- 172.21.15.16:0/171439158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f244c101cd0 0x7f244c196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:50.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.931+0000 7f2453c57700 1 -- 172.21.15.16:0/171439158 >> 172.21.15.16:0/171439158 conn(0x7f244c0fc090 msgr2=0x7f244c1052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:50.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.931+0000 7f2453c57700 1 -- 172.21.15.16:0/171439158 shutdown_connections 2024-04-02T04:50:50.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:50.931+0000 7f2453c57700 1 -- 172.21.15.16:0/171439158 wait complete. 2024-04-02T04:50:51.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.063+0000 7fbf2df5a700 1 -- 172.21.15.16:0/487534673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf28103ea0 msgr2=0x7fbf28106290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:51.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.063+0000 7fbf2df5a700 1 --2- 172.21.15.16:0/487534673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf28103ea0 0x7fbf28106290 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbf10009910 tx=0x7fbf10009c20 comp rx=0 tx=0).stop 2024-04-02T04:50:51.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.063+0000 7fbf2df5a700 1 -- 172.21.15.16:0/487534673 shutdown_connections 2024-04-02T04:50:51.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.063+0000 7fbf2df5a700 1 --2- 172.21.15.16:0/487534673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf28103ea0 0x7fbf28106290 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:51.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.063+0000 7fbf2df5a700 1 --2- 172.21.15.16:0/487534673 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf28101570 0x7fbf28103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:51.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.063+0000 7fbf2df5a700 1 -- 172.21.15.16:0/487534673 >> 172.21.15.16:0/487534673 conn(0x7fbf280faf80 msgr2=0x7fbf280fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:51.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.063+0000 7fbf2df5a700 1 -- 172.21.15.16:0/487534673 shutdown_connections 2024-04-02T04:50:51.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.063+0000 7fbf2df5a700 1 -- 172.21.15.16:0/487534673 wait complete. 2024-04-02T04:50:51.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.064+0000 7fbf2df5a700 1 Processor -- start 2024-04-02T04:50:51.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.064+0000 7fbf2df5a700 1 -- start start 2024-04-02T04:50:51.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.064+0000 7fbf2df5a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf28101570 0x7fbf28196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:51.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.064+0000 7fbf2df5a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf28103ea0 0x7fbf28196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:51.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.064+0000 7fbf2df5a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf281970e0 con 0x7fbf28103ea0 2024-04-02T04:50:51.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.064+0000 7fbf2df5a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf28197220 con 0x7fbf28101570 2024-04-02T04:50:51.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.064+0000 7fbf26ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf28103ea0 0x7fbf28196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:51.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.064+0000 7fbf26ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf28103ea0 0x7fbf28196ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36814/0 (socket says 172.21.15.16:36814) 2024-04-02T04:50:51.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.064+0000 7fbf26ffd700 1 -- 172.21.15.16:0/1970981647 learned_addr learned my addr 172.21.15.16:0/1970981647 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:50:51.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.064+0000 7fbf277fe700 1 --2- 172.21.15.16:0/1970981647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf28101570 0x7fbf28196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:51.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.064+0000 7fbf26ffd700 1 -- 172.21.15.16:0/1970981647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf28101570 msgr2=0x7fbf28196580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:51.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.064+0000 7fbf26ffd700 1 --2- 172.21.15.16:0/1970981647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf28101570 0x7fbf28196580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:51.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.064+0000 7fbf26ffd700 1 -- 172.21.15.16:0/1970981647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf100095f0 con 0x7fbf28103ea0 2024-04-02T04:50:51.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.065+0000 7fbf26ffd700 1 --2- 172.21.15.16:0/1970981647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf28103ea0 0x7fbf28196ac0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbf10006010 tx=0x7fbf1000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:51.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.065+0000 7fbf24ff9700 1 -- 172.21.15.16:0/1970981647 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf1001d070 con 0x7fbf28103ea0 2024-04-02T04:50:51.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.065+0000 7fbf2df5a700 1 -- 172.21.15.16:0/1970981647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf2819bc20 con 0x7fbf28103ea0 2024-04-02T04:50:51.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.065+0000 7fbf24ff9700 1 -- 172.21.15.16:0/1970981647 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf1000f460 con 0x7fbf28103ea0 2024-04-02T04:50:51.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.065+0000 7fbf24ff9700 1 -- 172.21.15.16:0/1970981647 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf10021620 con 0x7fbf28103ea0 2024-04-02T04:50:51.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.065+0000 7fbf2df5a700 1 -- 172.21.15.16:0/1970981647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf2819c190 con 0x7fbf28103ea0 2024-04-02T04:50:51.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.066+0000 7fbf2df5a700 1 -- 172.21.15.16:0/1970981647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf2804ea80 con 0x7fbf28103ea0 2024-04-02T04:50:51.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.067+0000 7fbf24ff9700 1 -- 172.21.15.16:0/1970981647 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fbf1000f5d0 con 0x7fbf28103ea0 2024-04-02T04:50:51.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.067+0000 7fbf24ff9700 1 --2- 172.21.15.16:0/1970981647 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fbf1406dd70 0x7fbf14070230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:51.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.067+0000 7fbf24ff9700 1 -- 172.21.15.16:0/1970981647 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fbf1008ef10 con 0x7fbf28103ea0 2024-04-02T04:50:51.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.067+0000 7fbf277fe700 1 --2- 172.21.15.16:0/1970981647 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fbf1406dd70 0x7fbf14070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:51.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.068+0000 7fbf277fe700 1 --2- 172.21.15.16:0/1970981647 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fbf1406dd70 0x7fbf14070230 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbf1800a400 tx=0x7fbf1800d040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:51.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.072+0000 7fbf24ff9700 1 -- 172.21.15.16:0/1970981647 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbf1005bf30 con 0x7fbf28103ea0 2024-04-02T04:50:51.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.322+0000 7fbf2df5a700 1 -- 172.21.15.16:0/1970981647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbf281096c0 con 0x7fbf28103ea0 2024-04-02T04:50:51.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.323+0000 7fbf24ff9700 1 -- 172.21.15.16:0/1970981647 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1929 (secure 0 0 0) 0x7fbf1005bac0 con 0x7fbf28103ea0 2024-04-02T04:50:51.327 INFO:teuthology.orchestra.run.smithi016.stdout:e29 2024-04-02T04:50:51.328 INFO:teuthology.orchestra.run.smithi016.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:50:51.328 INFO:teuthology.orchestra.run.smithi016.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:50:51.328 INFO:teuthology.orchestra.run.smithi016.stdout:legacy client fscid: 1 2024-04-02T04:50:51.328 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:50:51.328 INFO:teuthology.orchestra.run.smithi016.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:50:51.328 INFO:teuthology.orchestra.run.smithi016.stdout:fs_name cephfs 2024-04-02T04:50:51.328 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 29 2024-04-02T04:50:51.328 INFO:teuthology.orchestra.run.smithi016.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:50:51.328 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-04-02T04:39:30.740168+0000 2024-04-02T04:50:51.328 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-04-02T04:50:20.104880+0000 2024-04-02T04:50:51.329 INFO:teuthology.orchestra.run.smithi016.stdout:tableserver 0 2024-04-02T04:50:51.329 INFO:teuthology.orchestra.run.smithi016.stdout:root 0 2024-04-02T04:50:51.329 INFO:teuthology.orchestra.run.smithi016.stdout:session_timeout 60 2024-04-02T04:50:51.329 INFO:teuthology.orchestra.run.smithi016.stdout:session_autoclose 300 2024-04-02T04:50:51.329 INFO:teuthology.orchestra.run.smithi016.stdout:max_file_size 1099511627776 2024-04-02T04:50:51.329 INFO:teuthology.orchestra.run.smithi016.stdout:max_xattr_size 65536 2024-04-02T04:50:51.329 INFO:teuthology.orchestra.run.smithi016.stdout:required_client_features {} 2024-04-02T04:50:51.329 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure 0 2024-04-02T04:50:51.329 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure_osd_epoch 84 2024-04-02T04:50:51.329 INFO:teuthology.orchestra.run.smithi016.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-02T04:50:51.329 INFO:teuthology.orchestra.run.smithi016.stdout:max_mds 1 2024-04-02T04:50:51.329 INFO:teuthology.orchestra.run.smithi016.stdout:in 0 2024-04-02T04:50:51.329 INFO:teuthology.orchestra.run.smithi016.stdout:up {0=34404} 2024-04-02T04:50:51.330 INFO:teuthology.orchestra.run.smithi016.stdout:failed 2024-04-02T04:50:51.330 INFO:teuthology.orchestra.run.smithi016.stdout:damaged 2024-04-02T04:50:51.330 INFO:teuthology.orchestra.run.smithi016.stdout:stopped 2024-04-02T04:50:51.330 INFO:teuthology.orchestra.run.smithi016.stdout:data_pools [3] 2024-04-02T04:50:51.330 INFO:teuthology.orchestra.run.smithi016.stdout:metadata_pool 2 2024-04-02T04:50:51.330 INFO:teuthology.orchestra.run.smithi016.stdout:inline_data enabled 2024-04-02T04:50:51.330 INFO:teuthology.orchestra.run.smithi016.stdout:balancer 2024-04-02T04:50:51.330 INFO:teuthology.orchestra.run.smithi016.stdout:bal_rank_mask -1 2024-04-02T04:50:51.330 INFO:teuthology.orchestra.run.smithi016.stdout:standby_count_wanted 1 2024-04-02T04:50:51.330 INFO:teuthology.orchestra.run.smithi016.stdout:qdb_cluster leader: 34404 members: 34404 2024-04-02T04:50:51.330 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.uhgkgg{0:34404} state up:active seq 21 join_fscid=1 addr [v2:172.21.15.16:6826/81744708,v1:172.21.15.16:6827/81744708] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T04:50:51.330 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:50:51.330 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:50:51.330 INFO:teuthology.orchestra.run.smithi016.stdout:Standby daemons: 2024-04-02T04:50:51.331 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:50:51.331 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.fyzrnm{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.16:6828/782430688,v1:172.21.15.16:6829/782430688] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T04:50:51.331 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.zwrfoo{-1:44359} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.27:6824/1016194753,v1:172.21.15.27:6825/1016194753] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T04:50:51.331 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.sdublg{-1:44383} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.27:6826/1507844210,v1:172.21.15.27:6827/1507844210] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T04:50:51.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.325+0000 7fbf2df5a700 1 -- 172.21.15.16:0/1970981647 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fbf1406dd70 msgr2=0x7fbf14070230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:51.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.325+0000 7fbf2df5a700 1 --2- 172.21.15.16:0/1970981647 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fbf1406dd70 0x7fbf14070230 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbf1800a400 tx=0x7fbf1800d040 comp rx=0 tx=0).stop 2024-04-02T04:50:51.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.325+0000 7fbf2df5a700 1 -- 172.21.15.16:0/1970981647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf28103ea0 msgr2=0x7fbf28196ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:51.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.325+0000 7fbf2df5a700 1 --2- 172.21.15.16:0/1970981647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf28103ea0 0x7fbf28196ac0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbf10006010 tx=0x7fbf1000bd70 comp rx=0 tx=0).stop 2024-04-02T04:50:51.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.325+0000 7fbf2df5a700 1 -- 172.21.15.16:0/1970981647 shutdown_connections 2024-04-02T04:50:51.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.325+0000 7fbf2df5a700 1 --2- 172.21.15.16:0/1970981647 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fbf1406dd70 0x7fbf14070230 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:51.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.325+0000 7fbf2df5a700 1 --2- 172.21.15.16:0/1970981647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf28101570 0x7fbf28196580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:51.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.325+0000 7fbf2df5a700 1 --2- 172.21.15.16:0/1970981647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf28103ea0 0x7fbf28196ac0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:51.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.325+0000 7fbf2df5a700 1 -- 172.21.15.16:0/1970981647 >> 172.21.15.16:0/1970981647 conn(0x7fbf280faf80 msgr2=0x7fbf280fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:51.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.325+0000 7fbf2df5a700 1 -- 172.21.15.16:0/1970981647 shutdown_connections 2024-04-02T04:50:51.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.325+0000 7fbf2df5a700 1 -- 172.21.15.16:0/1970981647 wait complete. 2024-04-02T04:50:51.333 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 29 2024-04-02T04:50:51.394 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:51 smithi016 ceph-mon[93230]: from='client.34508 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:50:51.394 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:51 smithi016 ceph-mon[93230]: Upgrade: Updating node-exporter.smithi016 (1/2) 2024-04-02T04:50:51.394 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:51 smithi016 ceph-mon[93230]: Deploying daemon node-exporter.smithi016 on smithi016 2024-04-02T04:50:51.394 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:51 smithi016 ceph-mon[93230]: from='client.34512 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:50:51.394 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:51 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/171439158' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:51.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.450+0000 7f41a7d74700 1 -- 172.21.15.16:0/4070080989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41a00ff9d0 msgr2=0x7f41a00ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:51.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.450+0000 7f41a7d74700 1 --2- 172.21.15.16:0/4070080989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41a00ff9d0 0x7f41a00ffdf0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f419c009910 tx=0x7f419c009c20 comp rx=0 tx=0).stop 2024-04-02T04:50:51.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.450+0000 7f41a7d74700 1 -- 172.21.15.16:0/4070080989 shutdown_connections 2024-04-02T04:50:51.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.450+0000 7f41a7d74700 1 --2- 172.21.15.16:0/4070080989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41a0100bd0 0x7f41a0101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:51.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.450+0000 7f41a7d74700 1 --2- 172.21.15.16:0/4070080989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41a00ff9d0 0x7f41a00ffdf0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:51.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.450+0000 7f41a7d74700 1 -- 172.21.15.16:0/4070080989 >> 172.21.15.16:0/4070080989 conn(0x7f41a00faf70 msgr2=0x7f41a00fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:51.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.450+0000 7f41a7d74700 1 -- 172.21.15.16:0/4070080989 shutdown_connections 2024-04-02T04:50:51.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.450+0000 7f41a7d74700 1 -- 172.21.15.16:0/4070080989 wait complete. 2024-04-02T04:50:51.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.451+0000 7f41a7d74700 1 Processor -- start 2024-04-02T04:50:51.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.451+0000 7f41a7d74700 1 -- start start 2024-04-02T04:50:51.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.451+0000 7f41a7d74700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41a00ff9d0 0x7f41a0068f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:51.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.451+0000 7f41a7d74700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41a0100bd0 0x7f41a019b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:51.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.451+0000 7f41a7d74700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a019b7e0 con 0x7f41a00ff9d0 2024-04-02T04:50:51.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.451+0000 7f41a7d74700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a019b920 con 0x7f41a0100bd0 2024-04-02T04:50:51.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.451+0000 7f41a5b10700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41a00ff9d0 0x7f41a0068f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:51.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.451+0000 7f41a5b10700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41a00ff9d0 0x7f41a0068f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36832/0 (socket says 172.21.15.16:36832) 2024-04-02T04:50:51.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.451+0000 7f41a5b10700 1 -- 172.21.15.16:0/1688799445 learned_addr learned my addr 172.21.15.16:0/1688799445 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:50:51.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.451+0000 7f41a530f700 1 --2- 172.21.15.16:0/1688799445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41a0100bd0 0x7f41a019b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:51.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.451+0000 7f41a5b10700 1 -- 172.21.15.16:0/1688799445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41a0100bd0 msgr2=0x7f41a019b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:51.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.451+0000 7f41a5b10700 1 --2- 172.21.15.16:0/1688799445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41a0100bd0 0x7f41a019b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:51.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.451+0000 7f41a5b10700 1 -- 172.21.15.16:0/1688799445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f419c0095f0 con 0x7f41a00ff9d0 2024-04-02T04:50:51.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.452+0000 7f41a5b10700 1 --2- 172.21.15.16:0/1688799445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41a00ff9d0 0x7f41a0068f40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f419c005f00 tx=0x7f419c00f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:51.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.452+0000 7f4196ffd700 1 -- 172.21.15.16:0/1688799445 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f419c01d070 con 0x7f41a00ff9d0 2024-04-02T04:50:51.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.452+0000 7f4196ffd700 1 -- 172.21.15.16:0/1688799445 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f419c00fe30 con 0x7f41a00ff9d0 2024-04-02T04:50:51.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.452+0000 7f41a7d74700 1 -- 172.21.15.16:0/1688799445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41a019bb70 con 0x7f41a00ff9d0 2024-04-02T04:50:51.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.452+0000 7f4196ffd700 1 -- 172.21.15.16:0/1688799445 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f419c017400 con 0x7f41a00ff9d0 2024-04-02T04:50:51.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.452+0000 7f41a7d74700 1 -- 172.21.15.16:0/1688799445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41a019c060 con 0x7f41a00ff9d0 2024-04-02T04:50:51.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.453+0000 7f41a7d74700 1 -- 172.21.15.16:0/1688799445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41a018f550 con 0x7f41a00ff9d0 2024-04-02T04:50:51.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.453+0000 7f4196ffd700 1 -- 172.21.15.16:0/1688799445 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f419c00f970 con 0x7f41a00ff9d0 2024-04-02T04:50:51.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.454+0000 7f4196ffd700 1 --2- 172.21.15.16:0/1688799445 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f418c06dd60 0x7f418c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:51.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.454+0000 7f4196ffd700 1 -- 172.21.15.16:0/1688799445 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f419c08f060 con 0x7f41a00ff9d0 2024-04-02T04:50:51.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.454+0000 7f41a530f700 1 --2- 172.21.15.16:0/1688799445 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f418c06dd60 0x7f418c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:51.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.454+0000 7f41a530f700 1 --2- 172.21.15.16:0/1688799445 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f418c06dd60 0x7f418c070220 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4190009bf0 tx=0x7f4190009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:51.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.460+0000 7f4196ffd700 1 -- 172.21.15.16:0/1688799445 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f419c05c080 con 0x7f41a00ff9d0 2024-04-02T04:50:51.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:51 smithi027 ceph-mon[83036]: from='client.34508 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:50:51.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:51 smithi027 ceph-mon[83036]: Upgrade: Updating node-exporter.smithi016 (1/2) 2024-04-02T04:50:51.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:51 smithi027 ceph-mon[83036]: Deploying daemon node-exporter.smithi016 on smithi016 2024-04-02T04:50:51.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:51 smithi027 ceph-mon[83036]: from='client.34512 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:50:51.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:51 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/171439158' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:50:51.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.691+0000 7f41a7d74700 1 -- 172.21.15.16:0/1688799445 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f41a002ce70 con 0x7f418c06dd60 2024-04-02T04:50:51.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.692+0000 7f4196ffd700 1 -- 172.21.15.16:0/1688799445 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+466 (secure 0 0 0) 0x7f41a002ce70 con 0x7f418c06dd60 2024-04-02T04:50:51.697 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:50:51.697 INFO:teuthology.orchestra.run.smithi016.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:50:51.697 INFO:teuthology.orchestra.run.smithi016.stdout: "in_progress": true, 2024-04-02T04:50:51.697 INFO:teuthology.orchestra.run.smithi016.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-02T04:50:51.697 INFO:teuthology.orchestra.run.smithi016.stdout: "services_complete": [ 2024-04-02T04:50:51.697 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr", 2024-04-02T04:50:51.697 INFO:teuthology.orchestra.run.smithi016.stdout: "osd", 2024-04-02T04:50:51.697 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-04-02T04:50:51.697 INFO:teuthology.orchestra.run.smithi016.stdout: "mon", 2024-04-02T04:50:51.697 INFO:teuthology.orchestra.run.smithi016.stdout: "mds", 2024-04-02T04:50:51.697 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph-exporter" 2024-04-02T04:50:51.698 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-04-02T04:50:51.698 INFO:teuthology.orchestra.run.smithi016.stdout: "progress": "18/23 daemons upgraded", 2024-04-02T04:50:51.698 INFO:teuthology.orchestra.run.smithi016.stdout: "message": "Currently upgrading node-exporter daemons", 2024-04-02T04:50:51.698 INFO:teuthology.orchestra.run.smithi016.stdout: "is_paused": false 2024-04-02T04:50:51.698 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:50:51.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.694+0000 7f41a7d74700 1 -- 172.21.15.16:0/1688799445 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f418c06dd60 msgr2=0x7f418c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:51.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.694+0000 7f41a7d74700 1 --2- 172.21.15.16:0/1688799445 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f418c06dd60 0x7f418c070220 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4190009bf0 tx=0x7f4190009260 comp rx=0 tx=0).stop 2024-04-02T04:50:51.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.694+0000 7f41a7d74700 1 -- 172.21.15.16:0/1688799445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41a00ff9d0 msgr2=0x7f41a0068f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:51.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.695+0000 7f41a7d74700 1 --2- 172.21.15.16:0/1688799445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41a00ff9d0 0x7f41a0068f40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f419c005f00 tx=0x7f419c00f510 comp rx=0 tx=0).stop 2024-04-02T04:50:51.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.695+0000 7f41a7d74700 1 -- 172.21.15.16:0/1688799445 shutdown_connections 2024-04-02T04:50:51.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.695+0000 7f41a7d74700 1 --2- 172.21.15.16:0/1688799445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41a00ff9d0 0x7f41a0068f40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:51.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.695+0000 7f41a7d74700 1 --2- 172.21.15.16:0/1688799445 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f418c06dd60 0x7f418c070220 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:51.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.695+0000 7f41a7d74700 1 --2- 172.21.15.16:0/1688799445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41a0100bd0 0x7f41a019b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:51.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.695+0000 7f41a7d74700 1 -- 172.21.15.16:0/1688799445 >> 172.21.15.16:0/1688799445 conn(0x7f41a00faf70 msgr2=0x7f41a0104220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:51.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.695+0000 7f41a7d74700 1 -- 172.21.15.16:0/1688799445 shutdown_connections 2024-04-02T04:50:51.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.695+0000 7f41a7d74700 1 -- 172.21.15.16:0/1688799445 wait complete. 2024-04-02T04:50:51.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.818+0000 7f200d295700 1 -- 172.21.15.16:0/931748913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2008103e90 msgr2=0x7f2008106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:51.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.818+0000 7f200d295700 1 --2- 172.21.15.16:0/931748913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2008103e90 0x7f2008106280 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc009910 tx=0x7f1ffc009c20 comp rx=0 tx=0).stop 2024-04-02T04:50:51.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.818+0000 7f200d295700 1 -- 172.21.15.16:0/931748913 shutdown_connections 2024-04-02T04:50:51.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.818+0000 7f200d295700 1 --2- 172.21.15.16:0/931748913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2008103e90 0x7f2008106280 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:51.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.818+0000 7f200d295700 1 --2- 172.21.15.16:0/931748913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2008101560 0x7f2008103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:51.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.818+0000 7f200d295700 1 -- 172.21.15.16:0/931748913 >> 172.21.15.16:0/931748913 conn(0x7f20080faf70 msgr2=0x7f20080fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:51.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.818+0000 7f200d295700 1 -- 172.21.15.16:0/931748913 shutdown_connections 2024-04-02T04:50:51.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.818+0000 7f200d295700 1 -- 172.21.15.16:0/931748913 wait complete. 2024-04-02T04:50:51.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.818+0000 7f200d295700 1 Processor -- start 2024-04-02T04:50:51.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.819+0000 7f200d295700 1 -- start start 2024-04-02T04:50:51.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.819+0000 7f200d295700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2008101560 0x7f20081964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:51.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.819+0000 7f200d295700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2008103e90 0x7f20081969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:51.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.819+0000 7f200d295700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2008197000 con 0x7f2008103e90 2024-04-02T04:50:51.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.819+0000 7f200d295700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2008197140 con 0x7f2008101560 2024-04-02T04:50:51.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.819+0000 7f20067fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2008103e90 0x7f20081969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:51.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.819+0000 7f20067fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2008103e90 0x7f20081969e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57952/0 (socket says 172.21.15.16:57952) 2024-04-02T04:50:51.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.819+0000 7f20067fc700 1 -- 172.21.15.16:0/3890787437 learned_addr learned my addr 172.21.15.16:0/3890787437 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:50:51.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.819+0000 7f2006ffd700 1 --2- 172.21.15.16:0/3890787437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2008101560 0x7f20081964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:51.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.819+0000 7f2006ffd700 1 -- 172.21.15.16:0/3890787437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2008103e90 msgr2=0x7f20081969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:51.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.820+0000 7f2006ffd700 1 --2- 172.21.15.16:0/3890787437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2008103e90 0x7f20081969e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:51.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.820+0000 7f2006ffd700 1 -- 172.21.15.16:0/3890787437 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ffc0095f0 con 0x7f2008101560 2024-04-02T04:50:51.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.820+0000 7f20067fc700 1 --2- 172.21.15.16:0/3890787437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2008103e90 0x7f20081969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T04:50:51.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.820+0000 7f2006ffd700 1 --2- 172.21.15.16:0/3890787437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2008101560 0x7f20081964a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1ff800e970 tx=0x7f1ff800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:51.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.820+0000 7f1feffff700 1 -- 172.21.15.16:0/3890787437 <== mon.1 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ff800c9d0 con 0x7f2008101560 2024-04-02T04:50:51.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.820+0000 7f200d295700 1 -- 172.21.15.16:0/3890787437 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f200819bba0 con 0x7f2008101560 2024-04-02T04:50:51.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.820+0000 7f1feffff700 1 -- 172.21.15.16:0/3890787437 <== mon.1 v2:172.21.15.27:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ff8010430 con 0x7f2008101560 2024-04-02T04:50:51.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.820+0000 7f200d295700 1 -- 172.21.15.16:0/3890787437 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f200819c0f0 con 0x7f2008101560 2024-04-02T04:50:51.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.820+0000 7f1feffff700 1 -- 172.21.15.16:0/3890787437 <== mon.1 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ff8018630 con 0x7f2008101560 2024-04-02T04:50:51.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.821+0000 7f200d295700 1 -- 172.21.15.16:0/3890787437 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ff4005320 con 0x7f2008101560 2024-04-02T04:50:51.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.822+0000 7f1feffff700 1 -- 172.21.15.16:0/3890787437 <== mon.1 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f1ff8010a80 con 0x7f2008101560 2024-04-02T04:50:51.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.822+0000 7f1feffff700 1 --2- 172.21.15.16:0/3890787437 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f1ff006dd60 0x7f1ff0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:50:51.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.822+0000 7f1feffff700 1 -- 172.21.15.16:0/3890787437 <== mon.1 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f1ff8014070 con 0x7f2008101560 2024-04-02T04:50:51.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.822+0000 7f20067fc700 1 --2- 172.21.15.16:0/3890787437 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f1ff006dd60 0x7f1ff0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:50:51.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.823+0000 7f20067fc700 1 --2- 172.21.15.16:0/3890787437 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f1ff006dd60 0x7f1ff0070220 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc009fd0 tx=0x7f1ffc005940 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:50:51.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:51.827+0000 7f1feffff700 1 -- 172.21.15.16:0/3890787437 <== mon.1 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1ff8053280 con 0x7f2008101560 2024-04-02T04:50:52.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:52.201+0000 7f200d295700 1 -- 172.21.15.16:0/3890787437 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1ff40055d0 con 0x7f2008101560 2024-04-02T04:50:52.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:52.203+0000 7f1feffff700 1 -- 172.21.15.16:0/3890787437 <== mon.1 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f1ff808de60 con 0x7f2008101560 2024-04-02T04:50:52.207 INFO:teuthology.orchestra.run.smithi016.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-02T04:50:52.207 INFO:teuthology.orchestra.run.smithi016.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-02T04:50:52.207 INFO:teuthology.orchestra.run.smithi016.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-02T04:50:52.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:52.205+0000 7f200d295700 1 -- 172.21.15.16:0/3890787437 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f1ff006dd60 msgr2=0x7f1ff0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:52.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:52.205+0000 7f200d295700 1 --2- 172.21.15.16:0/3890787437 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f1ff006dd60 0x7f1ff0070220 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc009fd0 tx=0x7f1ffc005940 comp rx=0 tx=0).stop 2024-04-02T04:50:52.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:52.205+0000 7f200d295700 1 -- 172.21.15.16:0/3890787437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2008101560 msgr2=0x7f20081964a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:50:52.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:52.205+0000 7f200d295700 1 --2- 172.21.15.16:0/3890787437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2008101560 0x7f20081964a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1ff800e970 tx=0x7f1ff800ec80 comp rx=0 tx=0).stop 2024-04-02T04:50:52.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:52.205+0000 7f200d295700 1 -- 172.21.15.16:0/3890787437 shutdown_connections 2024-04-02T04:50:52.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:52.205+0000 7f200d295700 1 --2- 172.21.15.16:0/3890787437 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f1ff006dd60 0x7f1ff0070220 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:52.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:52.205+0000 7f200d295700 1 --2- 172.21.15.16:0/3890787437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2008101560 0x7f20081964a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:52.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:52.205+0000 7f200d295700 1 --2- 172.21.15.16:0/3890787437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2008103e90 0x7f20081969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:50:52.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:52.205+0000 7f200d295700 1 -- 172.21.15.16:0/3890787437 >> 172.21.15.16:0/3890787437 conn(0x7f20080faf70 msgr2=0x7f20080fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:50:52.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:52.205+0000 7f200d295700 1 -- 172.21.15.16:0/3890787437 shutdown_connections 2024-04-02T04:50:52.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:50:52.205+0000 7f200d295700 1 -- 172.21.15.16:0/3890787437 wait complete. 2024-04-02T04:50:52.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:52 smithi016 ceph-mon[93230]: from='client.34516 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:50:52.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:52 smithi016 ceph-mon[93230]: pgmap v235: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:52.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:52 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/1970981647' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:50:52.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:52 smithi027 ceph-mon[83036]: from='client.34516 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:50:52.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:52 smithi027 ceph-mon[83036]: pgmap v235: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:52.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:52 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/1970981647' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:50:53.446 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:53 smithi027 ceph-mon[83036]: from='client.34528 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:50:53.447 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:53 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/3890787437' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:50:53.447 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:53 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:53.447 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:53 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:53.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:53 smithi016 ceph-mon[93230]: from='client.34528 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:50:53.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:53 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/3890787437' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:50:53.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:53 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:53.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:53 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:54.314 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:54 smithi027 ceph-mon[83036]: pgmap v236: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:54.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:54 smithi016 ceph-mon[93230]: pgmap v236: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:55.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:55 smithi016 ceph-mon[93230]: Upgrade: Updating node-exporter.smithi027 (2/2) 2024-04-02T04:50:55.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:55 smithi016 ceph-mon[93230]: Deploying daemon node-exporter.smithi027 on smithi027 2024-04-02T04:50:55.602 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:55 smithi027 ceph-mon[83036]: Upgrade: Updating node-exporter.smithi027 (2/2) 2024-04-02T04:50:55.602 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:55 smithi027 ceph-mon[83036]: Deploying daemon node-exporter.smithi027 on smithi027 2024-04-02T04:50:56.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:56 smithi027 ceph-mon[83036]: pgmap v237: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:56.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:56 smithi016 ceph-mon[93230]: pgmap v237: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:58.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:58 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:58.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:58 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:58.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:58 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:50:58.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:50:58 smithi016 ceph-mon[93230]: pgmap v238: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:50:58.111 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:58 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:58.111 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:58 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:50:58.111 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:58 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:50:58.112 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:50:58 smithi027 ceph-mon[83036]: pgmap v238: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:00.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:00 smithi016 ceph-mon[93230]: pgmap v239: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:00.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:00 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:00.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:00 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:00.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:00 smithi027 ceph-mon[83036]: pgmap v239: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:00.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:00 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:00.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:00 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:01.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:01 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:01.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:01 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:01.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:01 smithi016 ceph-mon[93230]: pgmap v240: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:01.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:01 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:01.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:01 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:01.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:01 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:01.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:01 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:01.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:01 smithi027 ceph-mon[83036]: pgmap v240: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:01.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:01 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:01.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:01 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:02.636 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:02 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:51:02.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:02 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:51:03.841 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:03 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:03.841 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:03 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:03.841 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:03 smithi027 ceph-mon[83036]: pgmap v241: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:04.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:03 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:04.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:03 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:04.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:03 smithi016 ceph-mon[93230]: pgmap v241: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:05.365 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:05 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:05.365 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:05 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:05.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:05 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:05.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:05 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:06.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:06 smithi016 ceph-mon[93230]: pgmap v242: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:06.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:06 smithi027 ceph-mon[83036]: pgmap v242: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:07.379 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:07 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:07.379 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:07 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:07.379 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:07 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:51:07.380 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:07 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:51:07.380 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:07 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:07.380 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:07 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:51:07.380 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:07 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:07.380 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:07 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:07.380 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:07 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:07.380 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:07 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:07.380 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:07 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:07.380 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:07 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:07.380 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:07 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:07.380 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:07 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:07.380 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:07 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:07.380 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:07 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:07.380 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:07 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:07.381 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:07 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:07.381 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:07 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:07.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:07 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:07.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:07 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:07.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:07 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:51:07.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:07 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:51:07.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:07 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:07.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:07 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:51:07.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:07 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:07.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:07 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:07.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:07 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:07.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:07 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:07.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:07 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:07.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:07 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:07.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:07 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:07.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:07 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:07.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:07 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:07.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:07 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:07.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:07 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:07.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:07 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:07.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:07 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:08.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:08 smithi016 ceph-mon[93230]: pgmap v243: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:08.599 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:08 smithi027 ceph-mon[83036]: pgmap v243: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:09.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:09 smithi016 ceph-mon[93230]: Upgrade: Updating prometheus.smithi016 2024-04-02T04:51:09.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:09 smithi016 ceph-mon[93230]: Deploying daemon prometheus.smithi016 on smithi016 2024-04-02T04:51:09.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:09 smithi027 ceph-mon[83036]: Upgrade: Updating prometheus.smithi016 2024-04-02T04:51:09.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:09 smithi027 ceph-mon[83036]: Deploying daemon prometheus.smithi016 on smithi016 2024-04-02T04:51:10.328 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:10 smithi016 ceph-mon[93230]: pgmap v244: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:10.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:10 smithi027 ceph-mon[83036]: pgmap v244: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:12.414 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:12 smithi016 ceph-mon[93230]: pgmap v245: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:12.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:12 smithi027 ceph-mon[83036]: pgmap v245: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:14.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:14 smithi027 ceph-mon[83036]: pgmap v246: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:14.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:14 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:14.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:14 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:14.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:14 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:51:14.718 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:14 smithi016 ceph-mon[93230]: pgmap v246: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:14.718 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:14 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:14.718 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:14 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:14.718 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:14 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:51:16.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:16 smithi016 ceph-mon[93230]: pgmap v247: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:16.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:16 smithi027 ceph-mon[83036]: pgmap v247: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:17.638 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:17 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:51:17.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:17 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:51:18.611 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:18 smithi016 ceph-mon[93230]: pgmap v248: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:18.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:18 smithi027 ceph-mon[83036]: pgmap v248: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:19.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:19 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:19.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:19 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:19.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:19 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:19.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:19 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:20.459 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:20 smithi016 ceph-mon[93230]: pgmap v249: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:20.459 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:20 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:20.459 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:20 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:20.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:20 smithi027 ceph-mon[83036]: pgmap v249: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:20.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:20 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:20.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:20 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:22.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.321+0000 7feb9bffe700 1 -- 172.21.15.16:0/898870440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb94100ad0 msgr2=0x7feb94100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:22.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.321+0000 7feb9bffe700 1 --2- 172.21.15.16:0/898870440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb94100ad0 0x7feb94100ef0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7feb90009910 tx=0x7feb90009c20 comp rx=0 tx=0).stop 2024-04-02T04:51:22.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.321+0000 7feb9bffe700 1 -- 172.21.15.16:0/898870440 shutdown_connections 2024-04-02T04:51:22.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.321+0000 7feb9bffe700 1 --2- 172.21.15.16:0/898870440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb94101cd0 0x7feb94102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:22.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.321+0000 7feb9bffe700 1 --2- 172.21.15.16:0/898870440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb94100ad0 0x7feb94100ef0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:22.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.321+0000 7feb9bffe700 1 -- 172.21.15.16:0/898870440 >> 172.21.15.16:0/898870440 conn(0x7feb940fc090 msgr2=0x7feb940fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:22.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.321+0000 7feb9bffe700 1 -- 172.21.15.16:0/898870440 shutdown_connections 2024-04-02T04:51:22.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.321+0000 7feb9bffe700 1 -- 172.21.15.16:0/898870440 wait complete. 2024-04-02T04:51:22.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.321+0000 7feb9bffe700 1 Processor -- start 2024-04-02T04:51:22.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.322+0000 7feb9bffe700 1 -- start start 2024-04-02T04:51:22.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.322+0000 7feb9bffe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb94100ad0 0x7feb941964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:22.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.322+0000 7feb9bffe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb94101cd0 0x7feb94196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:22.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.322+0000 7feb9bffe700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb94196fc0 con 0x7feb94100ad0 2024-04-02T04:51:22.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.322+0000 7feb9bffe700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb94197100 con 0x7feb94101cd0 2024-04-02T04:51:22.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.322+0000 7feb99d9a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb94100ad0 0x7feb941964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:22.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.322+0000 7feb99d9a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb94100ad0 0x7feb941964f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43792/0 (socket says 172.21.15.16:43792) 2024-04-02T04:51:22.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.322+0000 7feb99d9a700 1 -- 172.21.15.16:0/3446543959 learned_addr learned my addr 172.21.15.16:0/3446543959 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:51:22.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.322+0000 7feb99599700 1 --2- 172.21.15.16:0/3446543959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb94101cd0 0x7feb94196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:22.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.322+0000 7feb99d9a700 1 -- 172.21.15.16:0/3446543959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb94101cd0 msgr2=0x7feb94196a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:22.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.322+0000 7feb99d9a700 1 --2- 172.21.15.16:0/3446543959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb94101cd0 0x7feb94196a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:22.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.322+0000 7feb99d9a700 1 -- 172.21.15.16:0/3446543959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb900095f0 con 0x7feb94100ad0 2024-04-02T04:51:22.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.322+0000 7feb99d9a700 1 --2- 172.21.15.16:0/3446543959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb94100ad0 0x7feb941964f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7feb900059d0 tx=0x7feb9000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:22.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.322+0000 7feb8affd700 1 -- 172.21.15.16:0/3446543959 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb9001d070 con 0x7feb94100ad0 2024-04-02T04:51:22.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.323+0000 7feb9bffe700 1 -- 172.21.15.16:0/3446543959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb9419bb10 con 0x7feb94100ad0 2024-04-02T04:51:22.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.323+0000 7feb8affd700 1 -- 172.21.15.16:0/3446543959 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb9000f460 con 0x7feb94100ad0 2024-04-02T04:51:22.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.323+0000 7feb9bffe700 1 -- 172.21.15.16:0/3446543959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb9419c050 con 0x7feb94100ad0 2024-04-02T04:51:22.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.323+0000 7feb8affd700 1 -- 172.21.15.16:0/3446543959 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb900215e0 con 0x7feb94100ad0 2024-04-02T04:51:22.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.324+0000 7feb9bffe700 1 -- 172.21.15.16:0/3446543959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb94190650 con 0x7feb94100ad0 2024-04-02T04:51:22.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.324+0000 7feb8affd700 1 -- 172.21.15.16:0/3446543959 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7feb9000f5d0 con 0x7feb94100ad0 2024-04-02T04:51:22.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.324+0000 7feb8affd700 1 --2- 172.21.15.16:0/3446543959 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7feb8006dd60 0x7feb80070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:22.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.324+0000 7feb8affd700 1 -- 172.21.15.16:0/3446543959 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7feb9008edd0 con 0x7feb94100ad0 2024-04-02T04:51:22.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.324+0000 7feb99599700 1 --2- 172.21.15.16:0/3446543959 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7feb8006dd60 0x7feb80070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:22.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.325+0000 7feb99599700 1 --2- 172.21.15.16:0/3446543959 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7feb8006dd60 0x7feb80070220 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7feb8400ac50 tx=0x7feb8400a380 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:22.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.329+0000 7feb8affd700 1 -- 172.21.15.16:0/3446543959 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7feb9005bea0 con 0x7feb94100ad0 2024-04-02T04:51:22.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:22 smithi016 ceph-mon[93230]: pgmap v250: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:22.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.603+0000 7feb9bffe700 1 -- 172.21.15.16:0/3446543959 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feb9402cda0 con 0x7feb8006dd60 2024-04-02T04:51:22.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.604+0000 7feb8affd700 1 -- 172.21.15.16:0/3446543959 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+463 (secure 0 0 0) 0x7feb9402cda0 con 0x7feb8006dd60 2024-04-02T04:51:22.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.606+0000 7feb9bffe700 1 -- 172.21.15.16:0/3446543959 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7feb8006dd60 msgr2=0x7feb80070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:22.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.606+0000 7feb9bffe700 1 --2- 172.21.15.16:0/3446543959 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7feb8006dd60 0x7feb80070220 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7feb8400ac50 tx=0x7feb8400a380 comp rx=0 tx=0).stop 2024-04-02T04:51:22.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.606+0000 7feb9bffe700 1 -- 172.21.15.16:0/3446543959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb94100ad0 msgr2=0x7feb941964f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:22.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.606+0000 7feb9bffe700 1 --2- 172.21.15.16:0/3446543959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb94100ad0 0x7feb941964f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7feb900059d0 tx=0x7feb9000bd70 comp rx=0 tx=0).stop 2024-04-02T04:51:22.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.606+0000 7feb9bffe700 1 -- 172.21.15.16:0/3446543959 shutdown_connections 2024-04-02T04:51:22.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.606+0000 7feb9bffe700 1 --2- 172.21.15.16:0/3446543959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb94100ad0 0x7feb941964f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:22.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.606+0000 7feb9bffe700 1 --2- 172.21.15.16:0/3446543959 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7feb8006dd60 0x7feb80070220 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:22.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.606+0000 7feb9bffe700 1 --2- 172.21.15.16:0/3446543959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb94101cd0 0x7feb94196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:22.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.606+0000 7feb9bffe700 1 -- 172.21.15.16:0/3446543959 >> 172.21.15.16:0/3446543959 conn(0x7feb940fc090 msgr2=0x7feb941052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:22.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.606+0000 7feb9bffe700 1 -- 172.21.15.16:0/3446543959 shutdown_connections 2024-04-02T04:51:22.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.606+0000 7feb9bffe700 1 -- 172.21.15.16:0/3446543959 wait complete. 2024-04-02T04:51:22.620 INFO:teuthology.orchestra.run.smithi016.stdout:true 2024-04-02T04:51:22.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:22 smithi027 ceph-mon[83036]: pgmap v250: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:22.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.713+0000 7f2acfbce700 1 -- 172.21.15.16:0/575239439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac8103e90 msgr2=0x7f2ac8106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:22.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.713+0000 7f2acfbce700 1 --2- 172.21.15.16:0/575239439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac8103e90 0x7f2ac8106280 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8009910 tx=0x7f2ab8009c20 comp rx=0 tx=0).stop 2024-04-02T04:51:22.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.713+0000 7f2acfbce700 1 -- 172.21.15.16:0/575239439 shutdown_connections 2024-04-02T04:51:22.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.713+0000 7f2acfbce700 1 --2- 172.21.15.16:0/575239439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac8103e90 0x7f2ac8106280 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:22.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.713+0000 7f2acfbce700 1 --2- 172.21.15.16:0/575239439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ac8101560 0x7f2ac8103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:22.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.713+0000 7f2acfbce700 1 -- 172.21.15.16:0/575239439 >> 172.21.15.16:0/575239439 conn(0x7f2ac80faf70 msgr2=0x7f2ac80fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:22.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.713+0000 7f2acfbce700 1 -- 172.21.15.16:0/575239439 shutdown_connections 2024-04-02T04:51:22.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.713+0000 7f2acfbce700 1 -- 172.21.15.16:0/575239439 wait complete. 2024-04-02T04:51:22.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.714+0000 7f2acfbce700 1 Processor -- start 2024-04-02T04:51:22.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.714+0000 7f2acfbce700 1 -- start start 2024-04-02T04:51:22.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.714+0000 7f2acfbce700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac8101560 0x7f2ac8100e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:22.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.714+0000 7f2acfbce700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ac8103e90 0x7f2ac80ff510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:22.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.714+0000 7f2acfbce700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac8101400 con 0x7f2ac8101560 2024-04-02T04:51:22.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.714+0000 7f2acfbce700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac80ffa50 con 0x7f2ac8103e90 2024-04-02T04:51:22.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.714+0000 7f2acd96a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac8101560 0x7f2ac8100e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:22.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.714+0000 7f2acd96a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac8101560 0x7f2ac8100e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43810/0 (socket says 172.21.15.16:43810) 2024-04-02T04:51:22.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.714+0000 7f2acd96a700 1 -- 172.21.15.16:0/4009048217 learned_addr learned my addr 172.21.15.16:0/4009048217 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:51:22.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.715+0000 7f2acd169700 1 --2- 172.21.15.16:0/4009048217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ac8103e90 0x7f2ac80ff510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:22.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.715+0000 7f2acd96a700 1 -- 172.21.15.16:0/4009048217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ac8103e90 msgr2=0x7f2ac80ff510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:22.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.715+0000 7f2acd96a700 1 --2- 172.21.15.16:0/4009048217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ac8103e90 0x7f2ac80ff510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:22.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.715+0000 7f2acd96a700 1 -- 172.21.15.16:0/4009048217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab80095f0 con 0x7f2ac8101560 2024-04-02T04:51:22.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.715+0000 7f2acd169700 1 --2- 172.21.15.16:0/4009048217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ac8103e90 0x7f2ac80ff510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T04:51:22.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.715+0000 7f2acd96a700 1 --2- 172.21.15.16:0/4009048217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac8101560 0x7f2ac8100e70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2ac400c930 tx=0x7f2ac400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:22.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.715+0000 7f2abeffd700 1 -- 172.21.15.16:0/4009048217 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ac4007ad0 con 0x7f2ac8101560 2024-04-02T04:51:22.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.715+0000 7f2abeffd700 1 -- 172.21.15.16:0/4009048217 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2ac4007c30 con 0x7f2ac8101560 2024-04-02T04:51:22.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.715+0000 7f2acfbce700 1 -- 172.21.15.16:0/4009048217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ac80ffd30 con 0x7f2ac8101560 2024-04-02T04:51:22.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.715+0000 7f2abeffd700 1 -- 172.21.15.16:0/4009048217 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ac4018610 con 0x7f2ac8101560 2024-04-02T04:51:22.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.715+0000 7f2acfbce700 1 -- 172.21.15.16:0/4009048217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ac8100300 con 0x7f2ac8101560 2024-04-02T04:51:22.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.716+0000 7f2acfbce700 1 -- 172.21.15.16:0/4009048217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ac804ea80 con 0x7f2ac8101560 2024-04-02T04:51:22.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.717+0000 7f2abeffd700 1 -- 172.21.15.16:0/4009048217 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f2ac400f450 con 0x7f2ac8101560 2024-04-02T04:51:22.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.717+0000 7f2abeffd700 1 --2- 172.21.15.16:0/4009048217 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f2ab406dd70 0x7f2ab4070230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:22.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.717+0000 7f2abeffd700 1 -- 172.21.15.16:0/4009048217 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f2ac408ddf0 con 0x7f2ac8101560 2024-04-02T04:51:22.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.717+0000 7f2acd169700 1 --2- 172.21.15.16:0/4009048217 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f2ab406dd70 0x7f2ab4070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:22.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.718+0000 7f2acd169700 1 --2- 172.21.15.16:0/4009048217 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f2ab406dd70 0x7f2ab4070230 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2ab80051c0 tx=0x7f2ab800b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:22.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:22.722+0000 7f2abeffd700 1 -- 172.21.15.16:0/4009048217 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2ac405ae10 con 0x7f2ac8101560 2024-04-02T04:51:23.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.054+0000 7f2acfbce700 1 -- 172.21.15.16:0/4009048217 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ac8105d50 con 0x7f2ab406dd70 2024-04-02T04:51:23.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.056+0000 7f2abeffd700 1 -- 172.21.15.16:0/4009048217 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+463 (secure 0 0 0) 0x7f2ac8105d50 con 0x7f2ab406dd70 2024-04-02T04:51:23.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.058+0000 7f2acfbce700 1 -- 172.21.15.16:0/4009048217 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f2ab406dd70 msgr2=0x7f2ab4070230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:23.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.058+0000 7f2acfbce700 1 --2- 172.21.15.16:0/4009048217 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f2ab406dd70 0x7f2ab4070230 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2ab80051c0 tx=0x7f2ab800b540 comp rx=0 tx=0).stop 2024-04-02T04:51:23.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.058+0000 7f2acfbce700 1 -- 172.21.15.16:0/4009048217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac8101560 msgr2=0x7f2ac8100e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:23.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.058+0000 7f2acfbce700 1 --2- 172.21.15.16:0/4009048217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac8101560 0x7f2ac8100e70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2ac400c930 tx=0x7f2ac400ccf0 comp rx=0 tx=0).stop 2024-04-02T04:51:23.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.058+0000 7f2acfbce700 1 -- 172.21.15.16:0/4009048217 shutdown_connections 2024-04-02T04:51:23.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.058+0000 7f2acfbce700 1 --2- 172.21.15.16:0/4009048217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ac8101560 0x7f2ac8100e70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:23.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.058+0000 7f2acfbce700 1 --2- 172.21.15.16:0/4009048217 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f2ab406dd70 0x7f2ab4070230 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:23.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.058+0000 7f2acfbce700 1 --2- 172.21.15.16:0/4009048217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ac8103e90 0x7f2ac80ff510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:23.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.058+0000 7f2acfbce700 1 -- 172.21.15.16:0/4009048217 >> 172.21.15.16:0/4009048217 conn(0x7f2ac80faf70 msgr2=0x7f2ac80fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:23.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.058+0000 7f2acfbce700 1 -- 172.21.15.16:0/4009048217 shutdown_connections 2024-04-02T04:51:23.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.058+0000 7f2acfbce700 1 -- 172.21.15.16:0/4009048217 wait complete. 2024-04-02T04:51:23.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.170+0000 7fb9549e4700 1 -- 172.21.15.16:0/458115384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb950100bd0 msgr2=0x7fb950101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:23.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.170+0000 7fb9549e4700 1 --2- 172.21.15.16:0/458115384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb950100bd0 0x7fb950101030 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb944009910 tx=0x7fb944009c20 comp rx=0 tx=0).stop 2024-04-02T04:51:23.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.170+0000 7fb9549e4700 1 -- 172.21.15.16:0/458115384 shutdown_connections 2024-04-02T04:51:23.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.170+0000 7fb9549e4700 1 --2- 172.21.15.16:0/458115384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb950100bd0 0x7fb950101030 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:23.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.170+0000 7fb9549e4700 1 --2- 172.21.15.16:0/458115384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9500ff9d0 0x7fb9500ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:23.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.170+0000 7fb9549e4700 1 -- 172.21.15.16:0/458115384 >> 172.21.15.16:0/458115384 conn(0x7fb9500faf70 msgr2=0x7fb9500fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:23.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.170+0000 7fb9549e4700 1 -- 172.21.15.16:0/458115384 shutdown_connections 2024-04-02T04:51:23.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.170+0000 7fb9549e4700 1 -- 172.21.15.16:0/458115384 wait complete. 2024-04-02T04:51:23.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.171+0000 7fb9549e4700 1 Processor -- start 2024-04-02T04:51:23.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.171+0000 7fb9549e4700 1 -- start start 2024-04-02T04:51:23.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.171+0000 7fb9549e4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9500ff9d0 0x7fb9501038c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:23.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.171+0000 7fb9549e4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb950100bd0 0x7fb950101fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:23.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.171+0000 7fb9549e4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9501024f0 con 0x7fb950100bd0 2024-04-02T04:51:23.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.171+0000 7fb9549e4700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb950102630 con 0x7fb9500ff9d0 2024-04-02T04:51:23.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.171+0000 7fb94dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb950100bd0 0x7fb950101fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:23.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.171+0000 7fb94dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb950100bd0 0x7fb950101fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43824/0 (socket says 172.21.15.16:43824) 2024-04-02T04:51:23.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.171+0000 7fb94dd9b700 1 -- 172.21.15.16:0/3380603229 learned_addr learned my addr 172.21.15.16:0/3380603229 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:51:23.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.171+0000 7fb94e59c700 1 --2- 172.21.15.16:0/3380603229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9500ff9d0 0x7fb9501038c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:23.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.172+0000 7fb94dd9b700 1 -- 172.21.15.16:0/3380603229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9500ff9d0 msgr2=0x7fb9501038c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:23.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.172+0000 7fb94dd9b700 1 --2- 172.21.15.16:0/3380603229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9500ff9d0 0x7fb9501038c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:23.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.172+0000 7fb94dd9b700 1 -- 172.21.15.16:0/3380603229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9440095f0 con 0x7fb950100bd0 2024-04-02T04:51:23.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.172+0000 7fb94dd9b700 1 --2- 172.21.15.16:0/3380603229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb950100bd0 0x7fb950101fb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb944006010 tx=0x7fb94400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:23.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.172+0000 7fb93f7fe700 1 -- 172.21.15.16:0/3380603229 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb94401d070 con 0x7fb950100bd0 2024-04-02T04:51:23.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.172+0000 7fb93f7fe700 1 -- 172.21.15.16:0/3380603229 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb94400f460 con 0x7fb950100bd0 2024-04-02T04:51:23.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.172+0000 7fb9549e4700 1 -- 172.21.15.16:0/3380603229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9501028b0 con 0x7fb950100bd0 2024-04-02T04:51:23.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.172+0000 7fb93f7fe700 1 -- 172.21.15.16:0/3380603229 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb944021620 con 0x7fb950100bd0 2024-04-02T04:51:23.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.172+0000 7fb9549e4700 1 -- 172.21.15.16:0/3380603229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb950102da0 con 0x7fb950100bd0 2024-04-02T04:51:23.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.173+0000 7fb9549e4700 1 -- 172.21.15.16:0/3380603229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb95004ea80 con 0x7fb950100bd0 2024-04-02T04:51:23.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.174+0000 7fb93f7fe700 1 -- 172.21.15.16:0/3380603229 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fb94400f5d0 con 0x7fb950100bd0 2024-04-02T04:51:23.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.174+0000 7fb93f7fe700 1 --2- 172.21.15.16:0/3380603229 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fb93806dd60 0x7fb938070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:23.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.174+0000 7fb93f7fe700 1 -- 172.21.15.16:0/3380603229 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fb94408ee10 con 0x7fb950100bd0 2024-04-02T04:51:23.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.174+0000 7fb94e59c700 1 --2- 172.21.15.16:0/3380603229 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fb93806dd60 0x7fb938070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:23.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.175+0000 7fb94e59c700 1 --2- 172.21.15.16:0/3380603229 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fb93806dd60 0x7fb938070220 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb950100a30 tx=0x7fb940007400 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:23.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.179+0000 7fb93f7fe700 1 -- 172.21.15.16:0/3380603229 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb94405be30 con 0x7fb950100bd0 2024-04-02T04:51:23.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.397+0000 7fb9549e4700 1 -- 172.21.15.16:0/3380603229 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb9500fcf20 con 0x7fb93806dd60 2024-04-02T04:51:23.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.455+0000 7fb93f7fe700 1 -- 172.21.15.16:0/3380603229 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fb9500fcf20 con 0x7fb93806dd60 2024-04-02T04:51:23.459 INFO:teuthology.orchestra.run.smithi016.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:51:23.459 INFO:teuthology.orchestra.run.smithi016.stdout:alertmanager.smithi016 smithi016 *:9093,9094 running (15m) 5s ago 17m 24.8M - 0.25.0 c8568f914cd2 b95d8354b016 2024-04-02T04:51:23.459 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi016 smithi016 running (50s) 5s ago 17m 7898k - 19.0.0-1979-g6cec793d 78f830190839 1d3ba1f157b9 2024-04-02T04:51:23.459 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi027 smithi027 running (45s) 24s ago 16m 7558k - 19.0.0-1979-g6cec793d 78f830190839 3402997325e5 2024-04-02T04:51:23.459 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi016 smithi016 running (5m) 5s ago 17m 6660k - 19.0.0-1979-g6cec793d 78f830190839 dc63ce0eef58 2024-04-02T04:51:23.459 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi027 smithi027 running (5m) 24s ago 16m 6665k - 19.0.0-1979-g6cec793d 78f830190839 2beb077018e4 2024-04-02T04:51:23.459 INFO:teuthology.orchestra.run.smithi016.stdout:grafana.smithi016 smithi016 *:3000 running (15m) 5s ago 17m 87.1M - 9.4.7 954c08fa6188 7c43d133569c 2024-04-02T04:51:23.459 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.fyzrnm smithi016 running (116s) 5s ago 11m 16.3M - 19.0.0-1979-g6cec793d 78f830190839 98f0a83c6631 2024-04-02T04:51:23.459 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.uhgkgg smithi016 running (2m) 5s ago 11m 89.7M - 19.0.0-1979-g6cec793d 78f830190839 19be3989a243 2024-04-02T04:51:23.460 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.sdublg smithi027 running (64s) 24s ago 11m 12.8M - 19.0.0-1979-g6cec793d 78f830190839 7c307783baab 2024-04-02T04:51:23.460 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.zwrfoo smithi027 running (92s) 24s ago 11m 13.6M - 19.0.0-1979-g6cec793d 78f830190839 acc6430302ac 2024-04-02T04:51:23.460 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi016.eqnebx smithi016 *:8443,9283,8765 running (8m) 5s ago 18m 514M - 19.0.0-1979-g6cec793d 78f830190839 d61ed91b4577 2024-04-02T04:51:23.460 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi027.kjzcsc smithi027 *:8443,9283,8765 running (7m) 24s ago 16m 451M - 19.0.0-1979-g6cec793d 78f830190839 5072a5ffc761 2024-04-02T04:51:23.460 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi016 smithi016 running (7m) 5s ago 19m 54.7M 2048M 19.0.0-1979-g6cec793d 78f830190839 eabfc84f1cdf 2024-04-02T04:51:23.460 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi027 smithi027 running (6m) 24s ago 16m 44.0M 2048M 19.0.0-1979-g6cec793d 78f830190839 fcd68b20dcc5 2024-04-02T04:51:23.460 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi016 smithi016 *:9100 running (31s) 5s ago 17m 8434k - 1.5.0 0da6a335fe13 f3e285b06d71 2024-04-02T04:51:23.460 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi027 smithi027 *:9100 running (26s) 24s ago 16m 3649k - 1.5.0 0da6a335fe13 c8e26f58ca63 2024-04-02T04:51:23.460 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 smithi016 running (5m) 5s ago 15m 91.3M 1217M 19.0.0-1979-g6cec793d 78f830190839 421d9e1cc751 2024-04-02T04:51:23.460 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 smithi016 running (4m) 5s ago 14m 100M 1217M 19.0.0-1979-g6cec793d 78f830190839 3612a56236a3 2024-04-02T04:51:23.460 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 smithi016 running (4m) 5s ago 14m 91.7M 1217M 19.0.0-1979-g6cec793d 78f830190839 a8e9b8d0303c 2024-04-02T04:51:23.460 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 smithi027 running (3m) 24s ago 14m 112M 2240M 19.0.0-1979-g6cec793d 78f830190839 0dd13b5be8c0 2024-04-02T04:51:23.460 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 smithi027 running (3m) 24s ago 13m 89.3M 2240M 19.0.0-1979-g6cec793d 78f830190839 9025eacf8145 2024-04-02T04:51:23.460 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 smithi027 running (2m) 24s ago 13m 88.2M 2240M 19.0.0-1979-g6cec793d 78f830190839 c2787f157377 2024-04-02T04:51:23.460 INFO:teuthology.orchestra.run.smithi016.stdout:prometheus.smithi016 smithi016 *:9095 running (10s) 5s ago 16m 39.1M - 2.43.0 a07b618ecd1d 042152fa1836 2024-04-02T04:51:23.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.458+0000 7fb9549e4700 1 -- 172.21.15.16:0/3380603229 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fb93806dd60 msgr2=0x7fb938070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:23.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.458+0000 7fb9549e4700 1 --2- 172.21.15.16:0/3380603229 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fb93806dd60 0x7fb938070220 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb950100a30 tx=0x7fb940007400 comp rx=0 tx=0).stop 2024-04-02T04:51:23.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.458+0000 7fb9549e4700 1 -- 172.21.15.16:0/3380603229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb950100bd0 msgr2=0x7fb950101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:23.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.458+0000 7fb9549e4700 1 --2- 172.21.15.16:0/3380603229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb950100bd0 0x7fb950101fb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb944006010 tx=0x7fb94400bd70 comp rx=0 tx=0).stop 2024-04-02T04:51:23.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.458+0000 7fb9549e4700 1 -- 172.21.15.16:0/3380603229 shutdown_connections 2024-04-02T04:51:23.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.458+0000 7fb9549e4700 1 --2- 172.21.15.16:0/3380603229 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fb93806dd60 0x7fb938070220 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:23.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.458+0000 7fb9549e4700 1 --2- 172.21.15.16:0/3380603229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9500ff9d0 0x7fb9501038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:23.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.458+0000 7fb9549e4700 1 --2- 172.21.15.16:0/3380603229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb950100bd0 0x7fb950101fb0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:23.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.459+0000 7fb9549e4700 1 -- 172.21.15.16:0/3380603229 >> 172.21.15.16:0/3380603229 conn(0x7fb9500faf70 msgr2=0x7fb9500fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:23.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.459+0000 7fb9549e4700 1 -- 172.21.15.16:0/3380603229 shutdown_connections 2024-04-02T04:51:23.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.459+0000 7fb9549e4700 1 -- 172.21.15.16:0/3380603229 wait complete. 2024-04-02T04:51:23.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.577+0000 7f15ba59e700 1 -- 172.21.15.16:0/3065318935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15b4100bd0 msgr2=0x7f15b4101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:23.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.577+0000 7f15ba59e700 1 --2- 172.21.15.16:0/3065318935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15b4100bd0 0x7f15b4101030 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f15a8009910 tx=0x7f15a8009c20 comp rx=0 tx=0).stop 2024-04-02T04:51:23.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.577+0000 7f15ba59e700 1 -- 172.21.15.16:0/3065318935 shutdown_connections 2024-04-02T04:51:23.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.577+0000 7f15ba59e700 1 --2- 172.21.15.16:0/3065318935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15b4100bd0 0x7f15b4101030 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:23.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.577+0000 7f15ba59e700 1 --2- 172.21.15.16:0/3065318935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15b40ff9d0 0x7f15b40ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:23.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.577+0000 7f15ba59e700 1 -- 172.21.15.16:0/3065318935 >> 172.21.15.16:0/3065318935 conn(0x7f15b40faf70 msgr2=0x7f15b40fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:23.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.577+0000 7f15ba59e700 1 -- 172.21.15.16:0/3065318935 shutdown_connections 2024-04-02T04:51:23.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.577+0000 7f15ba59e700 1 -- 172.21.15.16:0/3065318935 wait complete. 2024-04-02T04:51:23.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.578+0000 7f15ba59e700 1 Processor -- start 2024-04-02T04:51:23.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.578+0000 7f15ba59e700 1 -- start start 2024-04-02T04:51:23.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.579+0000 7f15ba59e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15b40ff9d0 0x7f15b4196570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:23.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.579+0000 7f15ba59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15b4100bd0 0x7f15b4196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:23.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.579+0000 7f15ba59e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b41970d0 con 0x7f15b4100bd0 2024-04-02T04:51:23.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.579+0000 7f15ba59e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b4197210 con 0x7f15b40ff9d0 2024-04-02T04:51:23.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.579+0000 7f15b37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15b4100bd0 0x7f15b4196ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:23.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.579+0000 7f15b37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15b4100bd0 0x7f15b4196ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43840/0 (socket says 172.21.15.16:43840) 2024-04-02T04:51:23.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.579+0000 7f15b37fe700 1 -- 172.21.15.16:0/476685235 learned_addr learned my addr 172.21.15.16:0/476685235 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:51:23.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.579+0000 7f15b3fff700 1 --2- 172.21.15.16:0/476685235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15b40ff9d0 0x7f15b4196570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:23.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.579+0000 7f15b37fe700 1 -- 172.21.15.16:0/476685235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15b40ff9d0 msgr2=0x7f15b4196570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:23.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.579+0000 7f15b37fe700 1 --2- 172.21.15.16:0/476685235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15b40ff9d0 0x7f15b4196570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:23.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.579+0000 7f15b37fe700 1 -- 172.21.15.16:0/476685235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a80095f0 con 0x7f15b4100bd0 2024-04-02T04:51:23.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.579+0000 7f15b3fff700 1 --2- 172.21.15.16:0/476685235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15b40ff9d0 0x7f15b4196570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T04:51:23.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.579+0000 7f15b37fe700 1 --2- 172.21.15.16:0/476685235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15b4100bd0 0x7f15b4196ab0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f15a8006010 tx=0x7f15a800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:23.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.579+0000 7f15b17fa700 1 -- 172.21.15.16:0/476685235 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15a801d070 con 0x7f15b4100bd0 2024-04-02T04:51:23.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.580+0000 7f15b17fa700 1 -- 172.21.15.16:0/476685235 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f15a800f460 con 0x7f15b4100bd0 2024-04-02T04:51:23.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.580+0000 7f15b17fa700 1 -- 172.21.15.16:0/476685235 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15a8021640 con 0x7f15b4100bd0 2024-04-02T04:51:23.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.580+0000 7f15ba59e700 1 -- 172.21.15.16:0/476685235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15b419bbc0 con 0x7f15b4100bd0 2024-04-02T04:51:23.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.580+0000 7f15ba59e700 1 -- 172.21.15.16:0/476685235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15b419c0b0 con 0x7f15b4100bd0 2024-04-02T04:51:23.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.581+0000 7f15ba59e700 1 -- 172.21.15.16:0/476685235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15b404ea80 con 0x7f15b4100bd0 2024-04-02T04:51:23.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.581+0000 7f15b17fa700 1 -- 172.21.15.16:0/476685235 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f15a800fab0 con 0x7f15b4100bd0 2024-04-02T04:51:23.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.582+0000 7f15b17fa700 1 --2- 172.21.15.16:0/476685235 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f159c06dd60 0x7f159c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:23.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.582+0000 7f15b17fa700 1 -- 172.21.15.16:0/476685235 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f15a808ee30 con 0x7f15b4100bd0 2024-04-02T04:51:23.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.582+0000 7f15b3fff700 1 --2- 172.21.15.16:0/476685235 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f159c06dd60 0x7f159c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:23.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.582+0000 7f15b3fff700 1 --2- 172.21.15.16:0/476685235 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f159c06dd60 0x7f159c070220 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f15b4100a30 tx=0x7f15a400b670 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:23.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.587+0000 7f15b17fa700 1 -- 172.21.15.16:0/476685235 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f15a805bed0 con 0x7f15b4100bd0 2024-04-02T04:51:23.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.890+0000 7f15ba59e700 1 -- 172.21.15.16:0/476685235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f15b419c4b0 con 0x7f15b4100bd0 2024-04-02T04:51:23.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.891+0000 7f15b17fa700 1 -- 172.21.15.16:0/476685235 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f15a805ba60 con 0x7f15b4100bd0 2024-04-02T04:51:23.894 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:51:23.894 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": { 2024-04-02T04:51:23.894 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:51:23.894 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:51:23.894 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": { 2024-04-02T04:51:23.895 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:51:23.895 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:51:23.895 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": { 2024-04-02T04:51:23.895 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 6 2024-04-02T04:51:23.895 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:51:23.895 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-04-02T04:51:23.895 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 4 2024-04-02T04:51:23.895 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:51:23.895 INFO:teuthology.orchestra.run.smithi016.stdout: "overall": { 2024-04-02T04:51:23.895 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 14 2024-04-02T04:51:23.895 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-02T04:51:23.895 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:51:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.894+0000 7f15ba59e700 1 -- 172.21.15.16:0/476685235 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f159c06dd60 msgr2=0x7f159c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.894+0000 7f15ba59e700 1 --2- 172.21.15.16:0/476685235 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f159c06dd60 0x7f159c070220 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f15b4100a30 tx=0x7f15a400b670 comp rx=0 tx=0).stop 2024-04-02T04:51:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.894+0000 7f15ba59e700 1 -- 172.21.15.16:0/476685235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15b4100bd0 msgr2=0x7f15b4196ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.894+0000 7f15ba59e700 1 --2- 172.21.15.16:0/476685235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15b4100bd0 0x7f15b4196ab0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f15a8006010 tx=0x7f15a800be20 comp rx=0 tx=0).stop 2024-04-02T04:51:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.894+0000 7f15ba59e700 1 -- 172.21.15.16:0/476685235 shutdown_connections 2024-04-02T04:51:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.894+0000 7f15ba59e700 1 --2- 172.21.15.16:0/476685235 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f159c06dd60 0x7f159c070220 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.894+0000 7f15ba59e700 1 --2- 172.21.15.16:0/476685235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15b40ff9d0 0x7f15b4196570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.894+0000 7f15ba59e700 1 --2- 172.21.15.16:0/476685235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15b4100bd0 0x7f15b4196ab0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.894+0000 7f15ba59e700 1 -- 172.21.15.16:0/476685235 >> 172.21.15.16:0/476685235 conn(0x7f15b40faf70 msgr2=0x7f15b4068480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.894+0000 7f15ba59e700 1 -- 172.21.15.16:0/476685235 shutdown_connections 2024-04-02T04:51:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:23.894+0000 7f15ba59e700 1 -- 172.21.15.16:0/476685235 wait complete. 2024-04-02T04:51:24.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.021+0000 7fb1a7096700 1 -- 172.21.15.16:0/1582347968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1a0100bd0 msgr2=0x7fb1a0101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:24.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.021+0000 7fb1a7096700 1 --2- 172.21.15.16:0/1582347968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1a0100bd0 0x7fb1a0101030 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb18c009910 tx=0x7fb18c009c20 comp rx=0 tx=0).stop 2024-04-02T04:51:24.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.021+0000 7fb1a7096700 1 -- 172.21.15.16:0/1582347968 shutdown_connections 2024-04-02T04:51:24.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.021+0000 7fb1a7096700 1 --2- 172.21.15.16:0/1582347968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1a0100bd0 0x7fb1a0101030 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:24.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.021+0000 7fb1a7096700 1 --2- 172.21.15.16:0/1582347968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1a00ff9d0 0x7fb1a00ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:24.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.021+0000 7fb1a7096700 1 -- 172.21.15.16:0/1582347968 >> 172.21.15.16:0/1582347968 conn(0x7fb1a00faf70 msgr2=0x7fb1a00fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:24.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.021+0000 7fb1a7096700 1 -- 172.21.15.16:0/1582347968 shutdown_connections 2024-04-02T04:51:24.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.021+0000 7fb1a7096700 1 -- 172.21.15.16:0/1582347968 wait complete. 2024-04-02T04:51:24.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.022+0000 7fb1a7096700 1 Processor -- start 2024-04-02T04:51:24.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.022+0000 7fb1a7096700 1 -- start start 2024-04-02T04:51:24.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.022+0000 7fb1a7096700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1a00ff9d0 0x7fb1a0195410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:24.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.022+0000 7fb1a7096700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1a0100bd0 0x7fb1a0195950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:24.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.022+0000 7fb1a7096700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a0195f70 con 0x7fb1a00ff9d0 2024-04-02T04:51:24.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.022+0000 7fb1a7096700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a01960b0 con 0x7fb1a0100bd0 2024-04-02T04:51:24.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.022+0000 7fb1a4e32700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1a00ff9d0 0x7fb1a0195410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:24.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.022+0000 7fb1a4e32700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1a00ff9d0 0x7fb1a0195410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43854/0 (socket says 172.21.15.16:43854) 2024-04-02T04:51:24.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.022+0000 7fb1a4e32700 1 -- 172.21.15.16:0/3595071209 learned_addr learned my addr 172.21.15.16:0/3595071209 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:51:24.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.022+0000 7fb19ffff700 1 --2- 172.21.15.16:0/3595071209 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1a0100bd0 0x7fb1a0195950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:24.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.023+0000 7fb1a4e32700 1 -- 172.21.15.16:0/3595071209 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1a0100bd0 msgr2=0x7fb1a0195950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:24.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.023+0000 7fb1a4e32700 1 --2- 172.21.15.16:0/3595071209 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1a0100bd0 0x7fb1a0195950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:24.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.023+0000 7fb1a4e32700 1 -- 172.21.15.16:0/3595071209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb18c0095f0 con 0x7fb1a00ff9d0 2024-04-02T04:51:24.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.023+0000 7fb1a4e32700 1 --2- 172.21.15.16:0/3595071209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1a00ff9d0 0x7fb1a0195410 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb19400ea80 tx=0x7fb19400ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:24.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.023+0000 7fb19dffb700 1 -- 172.21.15.16:0/3595071209 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb194013070 con 0x7fb1a00ff9d0 2024-04-02T04:51:24.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.023+0000 7fb19dffb700 1 -- 172.21.15.16:0/3595071209 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb19400b940 con 0x7fb1a00ff9d0 2024-04-02T04:51:24.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.023+0000 7fb1a7096700 1 -- 172.21.15.16:0/3595071209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1a00684e0 con 0x7fb1a00ff9d0 2024-04-02T04:51:24.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.023+0000 7fb19dffb700 1 -- 172.21.15.16:0/3595071209 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb194003d20 con 0x7fb1a00ff9d0 2024-04-02T04:51:24.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.023+0000 7fb1a7096700 1 -- 172.21.15.16:0/3595071209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a0068ab0 con 0x7fb1a00ff9d0 2024-04-02T04:51:24.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.024+0000 7fb1a7096700 1 -- 172.21.15.16:0/3595071209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1a004ea80 con 0x7fb1a00ff9d0 2024-04-02T04:51:24.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.025+0000 7fb19dffb700 1 -- 172.21.15.16:0/3595071209 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fb194017410 con 0x7fb1a00ff9d0 2024-04-02T04:51:24.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.025+0000 7fb19dffb700 1 --2- 172.21.15.16:0/3595071209 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fb19006dd60 0x7fb190070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:24.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.025+0000 7fb19dffb700 1 -- 172.21.15.16:0/3595071209 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fb19408f620 con 0x7fb1a00ff9d0 2024-04-02T04:51:24.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.025+0000 7fb19ffff700 1 --2- 172.21.15.16:0/3595071209 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fb19006dd60 0x7fb190070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:24.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.026+0000 7fb19ffff700 1 --2- 172.21.15.16:0/3595071209 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fb19006dd60 0x7fb190070220 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00b800 tx=0x7fb18c009f70 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.034+0000 7fb19dffb700 1 -- 172.21.15.16:0/3595071209 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb19405c790 con 0x7fb1a00ff9d0 2024-04-02T04:51:24.279 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:24 smithi016 ceph-mon[93230]: from='client.34536 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:51:24.279 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:24 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:24.279 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:24 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:24.279 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:24 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:51:24.279 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:24 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:51:24.279 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:24 smithi016 ceph-mon[93230]: from='client.34540 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:51:24.279 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:24 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:24.279 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:24 smithi016 ceph-mon[93230]: pgmap v251: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:24.280 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:24 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-02T04:51:24.280 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:24 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:51:24.280 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:24 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:24.280 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:24 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:24.280 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:24 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:24.280 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:24 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:24.280 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:24 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:24.280 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:24 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:24.280 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:24 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:24.280 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:24 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:24.280 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:24 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:24.280 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:24 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:24.280 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:24 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:24.280 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:24 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:24.280 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:24 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:24.280 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:24 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:24.280 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:24 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/476685235' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:24.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.296+0000 7fb1a7096700 1 -- 172.21.15.16:0/3595071209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb1a0108530 con 0x7fb1a00ff9d0 2024-04-02T04:51:24.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.296+0000 7fb19dffb700 1 -- 172.21.15.16:0/3595071209 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1929 (secure 0 0 0) 0x7fb19405c320 con 0x7fb1a00ff9d0 2024-04-02T04:51:24.299 INFO:teuthology.orchestra.run.smithi016.stdout:e29 2024-04-02T04:51:24.300 INFO:teuthology.orchestra.run.smithi016.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:51:24.300 INFO:teuthology.orchestra.run.smithi016.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:51:24.300 INFO:teuthology.orchestra.run.smithi016.stdout:legacy client fscid: 1 2024-04-02T04:51:24.300 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:51:24.300 INFO:teuthology.orchestra.run.smithi016.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:51:24.300 INFO:teuthology.orchestra.run.smithi016.stdout:fs_name cephfs 2024-04-02T04:51:24.300 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 29 2024-04-02T04:51:24.300 INFO:teuthology.orchestra.run.smithi016.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:51:24.300 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-04-02T04:39:30.740168+0000 2024-04-02T04:51:24.301 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-04-02T04:50:20.104880+0000 2024-04-02T04:51:24.301 INFO:teuthology.orchestra.run.smithi016.stdout:tableserver 0 2024-04-02T04:51:24.301 INFO:teuthology.orchestra.run.smithi016.stdout:root 0 2024-04-02T04:51:24.301 INFO:teuthology.orchestra.run.smithi016.stdout:session_timeout 60 2024-04-02T04:51:24.301 INFO:teuthology.orchestra.run.smithi016.stdout:session_autoclose 300 2024-04-02T04:51:24.301 INFO:teuthology.orchestra.run.smithi016.stdout:max_file_size 1099511627776 2024-04-02T04:51:24.301 INFO:teuthology.orchestra.run.smithi016.stdout:max_xattr_size 65536 2024-04-02T04:51:24.301 INFO:teuthology.orchestra.run.smithi016.stdout:required_client_features {} 2024-04-02T04:51:24.301 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure 0 2024-04-02T04:51:24.301 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure_osd_epoch 84 2024-04-02T04:51:24.301 INFO:teuthology.orchestra.run.smithi016.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-02T04:51:24.301 INFO:teuthology.orchestra.run.smithi016.stdout:max_mds 1 2024-04-02T04:51:24.301 INFO:teuthology.orchestra.run.smithi016.stdout:in 0 2024-04-02T04:51:24.301 INFO:teuthology.orchestra.run.smithi016.stdout:up {0=34404} 2024-04-02T04:51:24.302 INFO:teuthology.orchestra.run.smithi016.stdout:failed 2024-04-02T04:51:24.302 INFO:teuthology.orchestra.run.smithi016.stdout:damaged 2024-04-02T04:51:24.302 INFO:teuthology.orchestra.run.smithi016.stdout:stopped 2024-04-02T04:51:24.302 INFO:teuthology.orchestra.run.smithi016.stdout:data_pools [3] 2024-04-02T04:51:24.302 INFO:teuthology.orchestra.run.smithi016.stdout:metadata_pool 2 2024-04-02T04:51:24.302 INFO:teuthology.orchestra.run.smithi016.stdout:inline_data enabled 2024-04-02T04:51:24.302 INFO:teuthology.orchestra.run.smithi016.stdout:balancer 2024-04-02T04:51:24.302 INFO:teuthology.orchestra.run.smithi016.stdout:bal_rank_mask -1 2024-04-02T04:51:24.303 INFO:teuthology.orchestra.run.smithi016.stdout:standby_count_wanted 1 2024-04-02T04:51:24.303 INFO:teuthology.orchestra.run.smithi016.stdout:qdb_cluster leader: 34404 members: 34404 2024-04-02T04:51:24.303 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.uhgkgg{0:34404} state up:active seq 21 join_fscid=1 addr [v2:172.21.15.16:6826/81744708,v1:172.21.15.16:6827/81744708] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T04:51:24.303 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:51:24.303 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:51:24.303 INFO:teuthology.orchestra.run.smithi016.stdout:Standby daemons: 2024-04-02T04:51:24.303 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:51:24.303 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.fyzrnm{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.16:6828/782430688,v1:172.21.15.16:6829/782430688] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T04:51:24.303 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.zwrfoo{-1:44359} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.27:6824/1016194753,v1:172.21.15.27:6825/1016194753] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T04:51:24.303 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.sdublg{-1:44383} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.27:6826/1507844210,v1:172.21.15.27:6827/1507844210] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T04:51:24.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.298+0000 7fb1a7096700 1 -- 172.21.15.16:0/3595071209 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fb19006dd60 msgr2=0x7fb190070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:24.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.298+0000 7fb1a7096700 1 --2- 172.21.15.16:0/3595071209 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fb19006dd60 0x7fb190070220 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00b800 tx=0x7fb18c009f70 comp rx=0 tx=0).stop 2024-04-02T04:51:24.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.298+0000 7fb1a7096700 1 -- 172.21.15.16:0/3595071209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1a00ff9d0 msgr2=0x7fb1a0195410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:24.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.298+0000 7fb1a7096700 1 --2- 172.21.15.16:0/3595071209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1a00ff9d0 0x7fb1a0195410 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb19400ea80 tx=0x7fb19400ed90 comp rx=0 tx=0).stop 2024-04-02T04:51:24.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.298+0000 7fb1a7096700 1 -- 172.21.15.16:0/3595071209 shutdown_connections 2024-04-02T04:51:24.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.298+0000 7fb1a7096700 1 --2- 172.21.15.16:0/3595071209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1a00ff9d0 0x7fb1a0195410 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:24.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.298+0000 7fb1a7096700 1 --2- 172.21.15.16:0/3595071209 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fb19006dd60 0x7fb190070220 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:24.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.298+0000 7fb1a7096700 1 --2- 172.21.15.16:0/3595071209 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1a0100bd0 0x7fb1a0195950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:24.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.298+0000 7fb1a7096700 1 -- 172.21.15.16:0/3595071209 >> 172.21.15.16:0/3595071209 conn(0x7fb1a00faf70 msgr2=0x7fb1a0104120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:24.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.299+0000 7fb1a7096700 1 -- 172.21.15.16:0/3595071209 shutdown_connections 2024-04-02T04:51:24.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.299+0000 7fb1a7096700 1 -- 172.21.15.16:0/3595071209 wait complete. 2024-04-02T04:51:24.305 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 29 2024-04-02T04:51:24.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:24 smithi027 ceph-mon[83036]: from='client.34536 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:51:24.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:24 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:24.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:24 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:24.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:24 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:51:24.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:24 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:51:24.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:24 smithi027 ceph-mon[83036]: from='client.34540 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:51:24.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:24 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:24.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:24 smithi027 ceph-mon[83036]: pgmap v251: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:24.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:24 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-02T04:51:24.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:24 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:51:24.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:24 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:24.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:24 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:24.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:24 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:24.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:24 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:24.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:24 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:24.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:24 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:24.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:24 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:24.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:24 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:24.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:24 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:24.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:24 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:24.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:24 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:24.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:24 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:24.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:24 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:24.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:24 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:24.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:24 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/476685235' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:24.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.411+0000 7fde6af95700 1 -- 172.21.15.16:0/3757532882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde640ffd10 msgr2=0x7fde64100190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:24.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.411+0000 7fde6af95700 1 --2- 172.21.15.16:0/3757532882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde640ffd10 0x7fde64100190 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fde50009910 tx=0x7fde50009c20 comp rx=0 tx=0).stop 2024-04-02T04:51:24.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.411+0000 7fde6af95700 1 -- 172.21.15.16:0/3757532882 shutdown_connections 2024-04-02T04:51:24.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.411+0000 7fde6af95700 1 --2- 172.21.15.16:0/3757532882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde640ffd10 0x7fde64100190 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:24.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.411+0000 7fde6af95700 1 --2- 172.21.15.16:0/3757532882 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde640ff3b0 0x7fde640ff7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:24.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.411+0000 7fde6af95700 1 -- 172.21.15.16:0/3757532882 >> 172.21.15.16:0/3757532882 conn(0x7fde640faf70 msgr2=0x7fde640fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:24.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.411+0000 7fde6af95700 1 -- 172.21.15.16:0/3757532882 shutdown_connections 2024-04-02T04:51:24.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.411+0000 7fde6af95700 1 -- 172.21.15.16:0/3757532882 wait complete. 2024-04-02T04:51:24.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.411+0000 7fde6af95700 1 Processor -- start 2024-04-02T04:51:24.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.411+0000 7fde6af95700 1 -- start start 2024-04-02T04:51:24.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.411+0000 7fde6af95700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde640ff3b0 0x7fde64198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:24.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.411+0000 7fde6af95700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde640ffd10 0x7fde64198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:24.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.411+0000 7fde6af95700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde64199270 con 0x7fde640ffd10 2024-04-02T04:51:24.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.411+0000 7fde6af95700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde641993b0 con 0x7fde640ff3b0 2024-04-02T04:51:24.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.412+0000 7fde63fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde640ffd10 0x7fde64198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:24.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.412+0000 7fde63fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde640ffd10 0x7fde64198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43876/0 (socket says 172.21.15.16:43876) 2024-04-02T04:51:24.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.412+0000 7fde63fff700 1 -- 172.21.15.16:0/3592320344 learned_addr learned my addr 172.21.15.16:0/3592320344 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:51:24.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.412+0000 7fde68d31700 1 --2- 172.21.15.16:0/3592320344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde640ff3b0 0x7fde64198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:24.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.412+0000 7fde63fff700 1 -- 172.21.15.16:0/3592320344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde640ff3b0 msgr2=0x7fde64198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:24.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.412+0000 7fde63fff700 1 --2- 172.21.15.16:0/3592320344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde640ff3b0 0x7fde64198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:24.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.412+0000 7fde63fff700 1 -- 172.21.15.16:0/3592320344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde500095f0 con 0x7fde640ffd10 2024-04-02T04:51:24.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.412+0000 7fde63fff700 1 --2- 172.21.15.16:0/3592320344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde640ffd10 0x7fde64198c50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fde50006010 tx=0x7fde5000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:24.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.412+0000 7fde61ffb700 1 -- 172.21.15.16:0/3592320344 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde5001d070 con 0x7fde640ffd10 2024-04-02T04:51:24.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.412+0000 7fde6af95700 1 -- 172.21.15.16:0/3592320344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde6419dd60 con 0x7fde640ffd10 2024-04-02T04:51:24.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.412+0000 7fde61ffb700 1 -- 172.21.15.16:0/3592320344 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde50005dc0 con 0x7fde640ffd10 2024-04-02T04:51:24.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.412+0000 7fde61ffb700 1 -- 172.21.15.16:0/3592320344 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde50021410 con 0x7fde640ffd10 2024-04-02T04:51:24.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.413+0000 7fde6af95700 1 -- 172.21.15.16:0/3592320344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde6419e2d0 con 0x7fde640ffd10 2024-04-02T04:51:24.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.414+0000 7fde6af95700 1 -- 172.21.15.16:0/3592320344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde64192850 con 0x7fde640ffd10 2024-04-02T04:51:24.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.414+0000 7fde61ffb700 1 -- 172.21.15.16:0/3592320344 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fde5000f460 con 0x7fde640ffd10 2024-04-02T04:51:24.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.415+0000 7fde61ffb700 1 --2- 172.21.15.16:0/3592320344 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fde5406dd70 0x7fde54070230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:24.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.415+0000 7fde61ffb700 1 -- 172.21.15.16:0/3592320344 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fde5008ee50 con 0x7fde640ffd10 2024-04-02T04:51:24.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.415+0000 7fde68d31700 1 --2- 172.21.15.16:0/3592320344 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fde5406dd70 0x7fde54070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:24.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.415+0000 7fde68d31700 1 --2- 172.21.15.16:0/3592320344 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fde5406dd70 0x7fde54070230 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fde58005950 tx=0x7fde5800b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:24.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.423+0000 7fde61ffb700 1 -- 172.21.15.16:0/3592320344 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fde5005be70 con 0x7fde640ffd10 2024-04-02T04:51:24.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.663+0000 7fde6af95700 1 -- 172.21.15.16:0/3592320344 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fde6402cd20 con 0x7fde5406dd70 2024-04-02T04:51:24.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.665+0000 7fde61ffb700 1 -- 172.21.15.16:0/3592320344 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+465 (secure 0 0 0) 0x7fde6402cd20 con 0x7fde5406dd70 2024-04-02T04:51:24.668 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:51:24.668 INFO:teuthology.orchestra.run.smithi016.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:51:24.668 INFO:teuthology.orchestra.run.smithi016.stdout: "in_progress": true, 2024-04-02T04:51:24.668 INFO:teuthology.orchestra.run.smithi016.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-02T04:51:24.668 INFO:teuthology.orchestra.run.smithi016.stdout: "services_complete": [ 2024-04-02T04:51:24.668 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr", 2024-04-02T04:51:24.669 INFO:teuthology.orchestra.run.smithi016.stdout: "osd", 2024-04-02T04:51:24.669 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-04-02T04:51:24.669 INFO:teuthology.orchestra.run.smithi016.stdout: "mon", 2024-04-02T04:51:24.669 INFO:teuthology.orchestra.run.smithi016.stdout: "mds", 2024-04-02T04:51:24.669 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph-exporter" 2024-04-02T04:51:24.669 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-04-02T04:51:24.669 INFO:teuthology.orchestra.run.smithi016.stdout: "progress": "18/23 daemons upgraded", 2024-04-02T04:51:24.669 INFO:teuthology.orchestra.run.smithi016.stdout: "message": "Currently upgrading alertmanager daemons", 2024-04-02T04:51:24.669 INFO:teuthology.orchestra.run.smithi016.stdout: "is_paused": false 2024-04-02T04:51:24.669 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:51:24.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.667+0000 7fde6af95700 1 -- 172.21.15.16:0/3592320344 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fde5406dd70 msgr2=0x7fde54070230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:24.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.667+0000 7fde6af95700 1 --2- 172.21.15.16:0/3592320344 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fde5406dd70 0x7fde54070230 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fde58005950 tx=0x7fde5800b410 comp rx=0 tx=0).stop 2024-04-02T04:51:24.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.667+0000 7fde6af95700 1 -- 172.21.15.16:0/3592320344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde640ffd10 msgr2=0x7fde64198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:24.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.667+0000 7fde6af95700 1 --2- 172.21.15.16:0/3592320344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde640ffd10 0x7fde64198c50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fde50006010 tx=0x7fde5000bd70 comp rx=0 tx=0).stop 2024-04-02T04:51:24.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.667+0000 7fde6af95700 1 -- 172.21.15.16:0/3592320344 shutdown_connections 2024-04-02T04:51:24.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.667+0000 7fde6af95700 1 --2- 172.21.15.16:0/3592320344 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fde5406dd70 0x7fde54070230 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:24.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.667+0000 7fde6af95700 1 --2- 172.21.15.16:0/3592320344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde640ff3b0 0x7fde64198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:24.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.667+0000 7fde6af95700 1 --2- 172.21.15.16:0/3592320344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde640ffd10 0x7fde64198c50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:24.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.667+0000 7fde6af95700 1 -- 172.21.15.16:0/3592320344 >> 172.21.15.16:0/3592320344 conn(0x7fde640faf70 msgr2=0x7fde64107db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:24.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.667+0000 7fde6af95700 1 -- 172.21.15.16:0/3592320344 shutdown_connections 2024-04-02T04:51:24.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.667+0000 7fde6af95700 1 -- 172.21.15.16:0/3592320344 wait complete. 2024-04-02T04:51:24.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.774+0000 7fd2ff542700 1 -- 172.21.15.16:0/1378435445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2f8103ed0 msgr2=0x7fd2f81062c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:24.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.774+0000 7fd2ff542700 1 --2- 172.21.15.16:0/1378435445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2f8103ed0 0x7fd2f81062c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8009910 tx=0x7fd2e8009c20 comp rx=0 tx=0).stop 2024-04-02T04:51:24.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.774+0000 7fd2ff542700 1 -- 172.21.15.16:0/1378435445 shutdown_connections 2024-04-02T04:51:24.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.774+0000 7fd2ff542700 1 --2- 172.21.15.16:0/1378435445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2f8103ed0 0x7fd2f81062c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:24.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.774+0000 7fd2ff542700 1 --2- 172.21.15.16:0/1378435445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2f81015a0 0x7fd2f8103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:24.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.774+0000 7fd2ff542700 1 -- 172.21.15.16:0/1378435445 >> 172.21.15.16:0/1378435445 conn(0x7fd2f80faf70 msgr2=0x7fd2f80fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:24.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.774+0000 7fd2ff542700 1 -- 172.21.15.16:0/1378435445 shutdown_connections 2024-04-02T04:51:24.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.774+0000 7fd2ff542700 1 -- 172.21.15.16:0/1378435445 wait complete. 2024-04-02T04:51:24.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.774+0000 7fd2ff542700 1 Processor -- start 2024-04-02T04:51:24.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.775+0000 7fd2ff542700 1 -- start start 2024-04-02T04:51:24.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.775+0000 7fd2ff542700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2f81015a0 0x7fd2f81964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:24.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.775+0000 7fd2ff542700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2f8103ed0 0x7fd2f81969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:24.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.775+0000 7fd2ff542700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f8197010 con 0x7fd2f81015a0 2024-04-02T04:51:24.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.775+0000 7fd2ff542700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f8197150 con 0x7fd2f8103ed0 2024-04-02T04:51:24.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.775+0000 7fd2fd2de700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2f81015a0 0x7fd2f81964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:24.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.775+0000 7fd2fd2de700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2f81015a0 0x7fd2f81964b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43882/0 (socket says 172.21.15.16:43882) 2024-04-02T04:51:24.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.775+0000 7fd2fd2de700 1 -- 172.21.15.16:0/71013766 learned_addr learned my addr 172.21.15.16:0/71013766 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:51:24.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.775+0000 7fd2fcadd700 1 --2- 172.21.15.16:0/71013766 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2f8103ed0 0x7fd2f81969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:24.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.775+0000 7fd2fd2de700 1 -- 172.21.15.16:0/71013766 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2f8103ed0 msgr2=0x7fd2f81969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:24.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.775+0000 7fd2fd2de700 1 --2- 172.21.15.16:0/71013766 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2f8103ed0 0x7fd2f81969f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:24.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.775+0000 7fd2fd2de700 1 -- 172.21.15.16:0/71013766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e80095f0 con 0x7fd2f81015a0 2024-04-02T04:51:24.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.775+0000 7fd2fd2de700 1 --2- 172.21.15.16:0/71013766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2f81015a0 0x7fd2f81964b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd2f400e920 tx=0x7fd2f400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:24.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.776+0000 7fd2ee7fc700 1 -- 172.21.15.16:0/71013766 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2f400c9d0 con 0x7fd2f81015a0 2024-04-02T04:51:24.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.776+0000 7fd2ee7fc700 1 -- 172.21.15.16:0/71013766 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd2f4010430 con 0x7fd2f81015a0 2024-04-02T04:51:24.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.776+0000 7fd2ee7fc700 1 -- 172.21.15.16:0/71013766 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2f4018630 con 0x7fd2f81015a0 2024-04-02T04:51:24.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.776+0000 7fd2ff542700 1 -- 172.21.15.16:0/71013766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2f819bbb0 con 0x7fd2f81015a0 2024-04-02T04:51:24.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.776+0000 7fd2ff542700 1 -- 172.21.15.16:0/71013766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2f819c100 con 0x7fd2f81015a0 2024-04-02T04:51:24.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.777+0000 7fd2ff542700 1 -- 172.21.15.16:0/71013766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2f804ea80 con 0x7fd2f81015a0 2024-04-02T04:51:24.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.777+0000 7fd2ee7fc700 1 -- 172.21.15.16:0/71013766 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fd2f40105a0 con 0x7fd2f81015a0 2024-04-02T04:51:24.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.778+0000 7fd2ee7fc700 1 --2- 172.21.15.16:0/71013766 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd2e406dd60 0x7fd2e4070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:24.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.778+0000 7fd2ee7fc700 1 -- 172.21.15.16:0/71013766 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fd2f4014070 con 0x7fd2f81015a0 2024-04-02T04:51:24.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.778+0000 7fd2fcadd700 1 --2- 172.21.15.16:0/71013766 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd2e406dd60 0x7fd2e4070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:24.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.778+0000 7fd2fcadd700 1 --2- 172.21.15.16:0/71013766 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd2e406dd60 0x7fd2e4070220 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8006010 tx=0x7fd2e8009f70 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:24.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:24.786+0000 7fd2ee7fc700 1 -- 172.21.15.16:0/71013766 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd2f4053280 con 0x7fd2f81015a0 2024-04-02T04:51:25.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:25.080+0000 7fd2ff542700 1 -- 172.21.15.16:0/71013766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd2f8109670 con 0x7fd2f81015a0 2024-04-02T04:51:25.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:25.081+0000 7fd2ee7fc700 1 -- 172.21.15.16:0/71013766 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fd2f4093020 con 0x7fd2f81015a0 2024-04-02T04:51:25.084 INFO:teuthology.orchestra.run.smithi016.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-02T04:51:25.084 INFO:teuthology.orchestra.run.smithi016.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-02T04:51:25.084 INFO:teuthology.orchestra.run.smithi016.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-02T04:51:25.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:25.082+0000 7fd2ff542700 1 -- 172.21.15.16:0/71013766 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd2e406dd60 msgr2=0x7fd2e4070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:25.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:25.082+0000 7fd2ff542700 1 --2- 172.21.15.16:0/71013766 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd2e406dd60 0x7fd2e4070220 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8006010 tx=0x7fd2e8009f70 comp rx=0 tx=0).stop 2024-04-02T04:51:25.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:25.083+0000 7fd2ff542700 1 -- 172.21.15.16:0/71013766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2f81015a0 msgr2=0x7fd2f81964b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:25.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:25.083+0000 7fd2ff542700 1 --2- 172.21.15.16:0/71013766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2f81015a0 0x7fd2f81964b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd2f400e920 tx=0x7fd2f400ec30 comp rx=0 tx=0).stop 2024-04-02T04:51:25.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:25.083+0000 7fd2ff542700 1 -- 172.21.15.16:0/71013766 shutdown_connections 2024-04-02T04:51:25.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:25.083+0000 7fd2ff542700 1 --2- 172.21.15.16:0/71013766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2f81015a0 0x7fd2f81964b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:25.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:25.083+0000 7fd2ff542700 1 --2- 172.21.15.16:0/71013766 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd2e406dd60 0x7fd2e4070220 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:25.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:25.083+0000 7fd2ff542700 1 --2- 172.21.15.16:0/71013766 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2f8103ed0 0x7fd2f81969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:25.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:25.083+0000 7fd2ff542700 1 -- 172.21.15.16:0/71013766 >> 172.21.15.16:0/71013766 conn(0x7fd2f80faf70 msgr2=0x7fd2f80fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:25.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:25.083+0000 7fd2ff542700 1 -- 172.21.15.16:0/71013766 shutdown_connections 2024-04-02T04:51:25.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:25.083+0000 7fd2ff542700 1 -- 172.21.15.16:0/71013766 wait complete. 2024-04-02T04:51:25.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:25 smithi027 ceph-mon[83036]: from='client.34544 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:51:25.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:25 smithi027 ceph-mon[83036]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-02T04:51:25.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:25 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/3595071209' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:51:25.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:25 smithi016 ceph-mon[93230]: from='client.34544 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:51:25.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:25 smithi016 ceph-mon[93230]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-02T04:51:25.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:25 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/3595071209' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:51:26.171 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:26 smithi016 ceph-mon[93230]: from='client.34556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:51:26.171 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:26 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/71013766' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:51:26.171 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:26 smithi016 ceph-mon[93230]: pgmap v252: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:26.171 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:26 smithi016 ceph-mon[93230]: Upgrade: Updating alertmanager.smithi016 2024-04-02T04:51:26.171 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:26 smithi016 ceph-mon[93230]: Deploying daemon alertmanager.smithi016 on smithi016 2024-04-02T04:51:26.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:26 smithi027 ceph-mon[83036]: from='client.34556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:51:26.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:26 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/71013766' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:51:26.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:26 smithi027 ceph-mon[83036]: pgmap v252: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:26.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:26 smithi027 ceph-mon[83036]: Upgrade: Updating alertmanager.smithi016 2024-04-02T04:51:26.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:26 smithi027 ceph-mon[83036]: Deploying daemon alertmanager.smithi016 on smithi016 2024-04-02T04:51:28.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:28 smithi016 ceph-mon[93230]: pgmap v253: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:28.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:28 smithi027 ceph-mon[83036]: pgmap v253: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:30.497 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:30 smithi016 ceph-mon[93230]: pgmap v254: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:30.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:30 smithi027 ceph-mon[83036]: pgmap v254: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:32.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:31 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:32.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:31 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:32.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:31 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:51:32.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:31 smithi016 ceph-mon[93230]: pgmap v255: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:32.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:31 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:32.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:31 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:32.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:31 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:51:32.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:31 smithi027 ceph-mon[83036]: pgmap v255: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:33.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:33 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:51:33.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:33 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:51:34.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:34 smithi016 ceph-mon[93230]: pgmap v256: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:34.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:34 smithi027 ceph-mon[83036]: pgmap v256: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:36.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:36 smithi016 ceph-mon[93230]: pgmap v257: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:36.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:36 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:36.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:36 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:36.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:36 smithi027 ceph-mon[83036]: pgmap v257: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:36.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:36 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:36.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:36 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:37.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:37 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:37.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:37 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:37.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:37 smithi016 ceph-mon[93230]: pgmap v258: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:37.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:37 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:37.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:37 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:37.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:37 smithi027 ceph-mon[83036]: pgmap v258: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:40.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:40 smithi016 ceph-mon[93230]: pgmap v259: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:40.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:40 smithi027 ceph-mon[83036]: pgmap v259: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:41.370 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:41 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:41.370 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:41 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:41.371 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:41 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:51:41.371 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:41 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:51:41.371 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:41 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:41.371 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:41 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-02T04:51:41.371 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:41 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:51:41.371 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:41 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:41.371 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:41 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:41.371 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:41 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:41.371 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:41 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:41.371 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:41 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:41.371 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:41 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:41.371 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:41 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:41.371 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:41 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:41.371 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:41 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:41.371 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:41 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:41.372 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:41 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:41.372 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:41 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:41.372 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:41 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:41.372 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:41 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:41.372 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:41 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:41.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:41 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:41.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:41 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:41.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:41 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:51:41.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:41 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:51:41.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:41 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:41.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:41 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-02T04:51:41.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:41 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:51:41.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:41 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:41.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:41 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:41.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:41 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:41.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:41 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:41.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:41 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:41.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:41 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:41.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:41 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:41.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:41 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:41.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:41 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:41.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:41 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:41.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:41 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:41.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:41 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:41.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:41 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:41.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:41 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:41.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:41 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:42.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:42 smithi027 ceph-mon[83036]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-02T04:51:42.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:42 smithi027 ceph-mon[83036]: pgmap v260: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:42.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:42 smithi016 ceph-mon[93230]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-02T04:51:42.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:42 smithi016 ceph-mon[93230]: pgmap v260: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:43.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:43 smithi027 ceph-mon[83036]: Upgrade: Updating grafana.smithi016 2024-04-02T04:51:43.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:43 smithi027 ceph-mon[83036]: Deploying daemon grafana.smithi016 on smithi016 2024-04-02T04:51:43.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:43 smithi016 ceph-mon[93230]: Upgrade: Updating grafana.smithi016 2024-04-02T04:51:43.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:43 smithi016 ceph-mon[93230]: Deploying daemon grafana.smithi016 on smithi016 2024-04-02T04:51:44.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:44 smithi027 ceph-mon[83036]: pgmap v261: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:44.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:44 smithi016 ceph-mon[93230]: pgmap v261: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:46.612 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:46 smithi016 ceph-mon[93230]: pgmap v262: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:46.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:46 smithi027 ceph-mon[83036]: pgmap v262: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:47.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:47 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:51:47.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:47 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:51:48.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:48 smithi027 ceph-mon[83036]: pgmap v263: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:48.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:48 smithi016 ceph-mon[93230]: pgmap v263: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:50.621 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:50 smithi016 ceph-mon[93230]: pgmap v264: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:50.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:50 smithi027 ceph-mon[83036]: pgmap v264: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:52.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:52 smithi027 ceph-mon[83036]: pgmap v265: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:52.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:52 smithi016 ceph-mon[93230]: pgmap v265: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:54.557 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:54 smithi016 ceph-mon[93230]: pgmap v266: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:54.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:54 smithi027 ceph-mon[83036]: pgmap v266: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:55.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.207+0000 7fb205bd0700 1 -- 172.21.15.16:0/1650688040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb200101d00 msgr2=0x7fb200102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:55.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.207+0000 7fb205bd0700 1 --2- 172.21.15.16:0/1650688040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb200101d00 0x7fb200102160 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4009910 tx=0x7fb1f4009c20 comp rx=0 tx=0).stop 2024-04-02T04:51:55.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.207+0000 7fb205bd0700 1 -- 172.21.15.16:0/1650688040 shutdown_connections 2024-04-02T04:51:55.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.207+0000 7fb205bd0700 1 --2- 172.21.15.16:0/1650688040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb200101d00 0x7fb200102160 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:55.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.207+0000 7fb205bd0700 1 --2- 172.21.15.16:0/1650688040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb200100b00 0x7fb200100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:55.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.207+0000 7fb205bd0700 1 -- 172.21.15.16:0/1650688040 >> 172.21.15.16:0/1650688040 conn(0x7fb2000fc0a0 msgr2=0x7fb2000fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:55.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.207+0000 7fb205bd0700 1 -- 172.21.15.16:0/1650688040 shutdown_connections 2024-04-02T04:51:55.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.207+0000 7fb205bd0700 1 -- 172.21.15.16:0/1650688040 wait complete. 2024-04-02T04:51:55.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.208+0000 7fb205bd0700 1 Processor -- start 2024-04-02T04:51:55.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.208+0000 7fb205bd0700 1 -- start start 2024-04-02T04:51:55.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.209+0000 7fb205bd0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb200100b00 0x7fb2001964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:55.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.209+0000 7fb205bd0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb200101d00 0x7fb200196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:55.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.209+0000 7fb205bd0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb200197040 con 0x7fb200100b00 2024-04-02T04:51:55.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.209+0000 7fb205bd0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb200197180 con 0x7fb200101d00 2024-04-02T04:51:55.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.209+0000 7fb1ff7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb200100b00 0x7fb2001964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:55.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.209+0000 7fb1ff7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb200100b00 0x7fb2001964e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51092/0 (socket says 172.21.15.16:51092) 2024-04-02T04:51:55.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.209+0000 7fb1ff7fe700 1 -- 172.21.15.16:0/39622458 learned_addr learned my addr 172.21.15.16:0/39622458 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:51:55.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.209+0000 7fb1feffd700 1 --2- 172.21.15.16:0/39622458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb200101d00 0x7fb200196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:55.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.209+0000 7fb1feffd700 1 -- 172.21.15.16:0/39622458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb200100b00 msgr2=0x7fb2001964e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:55.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.209+0000 7fb1feffd700 1 --2- 172.21.15.16:0/39622458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb200100b00 0x7fb2001964e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:55.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.209+0000 7fb1feffd700 1 -- 172.21.15.16:0/39622458 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f40095f0 con 0x7fb200101d00 2024-04-02T04:51:55.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.210+0000 7fb1feffd700 1 --2- 172.21.15.16:0/39622458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb200101d00 0x7fb200196a20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb1f40059d0 tx=0x7fb1f400be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:55.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.210+0000 7fb1fcff9700 1 -- 172.21.15.16:0/39622458 <== mon.1 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1f401d070 con 0x7fb200101d00 2024-04-02T04:51:55.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.210+0000 7fb1fcff9700 1 -- 172.21.15.16:0/39622458 <== mon.1 v2:172.21.15.27:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb1f4005dc0 con 0x7fb200101d00 2024-04-02T04:51:55.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.210+0000 7fb205bd0700 1 -- 172.21.15.16:0/39622458 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb20019bb80 con 0x7fb200101d00 2024-04-02T04:51:55.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.210+0000 7fb1fcff9700 1 -- 172.21.15.16:0/39622458 <== mon.1 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1f4021410 con 0x7fb200101d00 2024-04-02T04:51:55.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.210+0000 7fb205bd0700 1 -- 172.21.15.16:0/39622458 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb20019c0f0 con 0x7fb200101d00 2024-04-02T04:51:55.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.211+0000 7fb205bd0700 1 -- 172.21.15.16:0/39622458 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb200190680 con 0x7fb200101d00 2024-04-02T04:51:55.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.212+0000 7fb1fcff9700 1 -- 172.21.15.16:0/39622458 <== mon.1 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fb1f4003a60 con 0x7fb200101d00 2024-04-02T04:51:55.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.212+0000 7fb1fcff9700 1 --2- 172.21.15.16:0/39622458 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fb1e806dd60 0x7fb1e8070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:55.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.213+0000 7fb1fcff9700 1 -- 172.21.15.16:0/39622458 <== mon.1 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fb1f408ebe0 con 0x7fb200101d00 2024-04-02T04:51:55.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.213+0000 7fb1ff7fe700 1 --2- 172.21.15.16:0/39622458 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fb1e806dd60 0x7fb1e8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:55.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.213+0000 7fb1ff7fe700 1 --2- 172.21.15.16:0/39622458 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fb1e806dd60 0x7fb1e8070220 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb1f000bcd0 tx=0x7fb1f000b2c0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:55.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.218+0000 7fb1fcff9700 1 -- 172.21.15.16:0/39622458 <== mon.1 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb1f405bd30 con 0x7fb200101d00 2024-04-02T04:51:55.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.467+0000 7fb205bd0700 1 -- 172.21.15.16:0/39622458 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb20002ce10 con 0x7fb1e806dd60 2024-04-02T04:51:55.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.469+0000 7fb1fcff9700 1 -- 172.21.15.16:0/39622458 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+460 (secure 0 0 0) 0x7fb20002ce10 con 0x7fb1e806dd60 2024-04-02T04:51:55.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.471+0000 7fb205bd0700 1 -- 172.21.15.16:0/39622458 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fb1e806dd60 msgr2=0x7fb1e8070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:55.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.471+0000 7fb205bd0700 1 --2- 172.21.15.16:0/39622458 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fb1e806dd60 0x7fb1e8070220 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb1f000bcd0 tx=0x7fb1f000b2c0 comp rx=0 tx=0).stop 2024-04-02T04:51:55.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.471+0000 7fb205bd0700 1 -- 172.21.15.16:0/39622458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb200101d00 msgr2=0x7fb200196a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:55.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.471+0000 7fb205bd0700 1 --2- 172.21.15.16:0/39622458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb200101d00 0x7fb200196a20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb1f40059d0 tx=0x7fb1f400be20 comp rx=0 tx=0).stop 2024-04-02T04:51:55.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.471+0000 7fb205bd0700 1 -- 172.21.15.16:0/39622458 shutdown_connections 2024-04-02T04:51:55.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.471+0000 7fb205bd0700 1 --2- 172.21.15.16:0/39622458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb200100b00 0x7fb2001964e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:55.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.471+0000 7fb205bd0700 1 --2- 172.21.15.16:0/39622458 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fb1e806dd60 0x7fb1e8070220 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:55.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.471+0000 7fb205bd0700 1 --2- 172.21.15.16:0/39622458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb200101d00 0x7fb200196a20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:55.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.471+0000 7fb205bd0700 1 -- 172.21.15.16:0/39622458 >> 172.21.15.16:0/39622458 conn(0x7fb2000fc0a0 msgr2=0x7fb200105250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:55.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.471+0000 7fb205bd0700 1 -- 172.21.15.16:0/39622458 shutdown_connections 2024-04-02T04:51:55.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.471+0000 7fb205bd0700 1 -- 172.21.15.16:0/39622458 wait complete. 2024-04-02T04:51:55.489 INFO:teuthology.orchestra.run.smithi016.stdout:true 2024-04-02T04:51:55.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.648+0000 7f554bc5c700 1 -- 172.21.15.16:0/1943306084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5544100ad0 msgr2=0x7f5544100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.648+0000 7f554bc5c700 1 --2- 172.21.15.16:0/1943306084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5544100ad0 0x7f5544100ef0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5540009910 tx=0x7f5540009c20 comp rx=0 tx=0).stop 2024-04-02T04:51:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.648+0000 7f554bc5c700 1 -- 172.21.15.16:0/1943306084 shutdown_connections 2024-04-02T04:51:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.648+0000 7f554bc5c700 1 --2- 172.21.15.16:0/1943306084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5544101cd0 0x7f5544102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.648+0000 7f554bc5c700 1 --2- 172.21.15.16:0/1943306084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5544100ad0 0x7f5544100ef0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.648+0000 7f554bc5c700 1 -- 172.21.15.16:0/1943306084 >> 172.21.15.16:0/1943306084 conn(0x7f55440fc090 msgr2=0x7f55440fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.648+0000 7f554bc5c700 1 -- 172.21.15.16:0/1943306084 shutdown_connections 2024-04-02T04:51:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.648+0000 7f554bc5c700 1 -- 172.21.15.16:0/1943306084 wait complete. 2024-04-02T04:51:55.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.649+0000 7f554bc5c700 1 Processor -- start 2024-04-02T04:51:55.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.649+0000 7f554bc5c700 1 -- start start 2024-04-02T04:51:55.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.649+0000 7f554bc5c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5544100ad0 0x7f55441964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:55.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.649+0000 7f554bc5c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5544101cd0 0x7f5544196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:55.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.649+0000 7f554bc5c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5544197050 con 0x7f5544100ad0 2024-04-02T04:51:55.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.649+0000 7f554bc5c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5544197190 con 0x7f5544101cd0 2024-04-02T04:51:55.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.649+0000 7f55499f8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5544100ad0 0x7f55441964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:55.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.649+0000 7f55499f8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5544100ad0 0x7f55441964f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51110/0 (socket says 172.21.15.16:51110) 2024-04-02T04:51:55.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.650+0000 7f55499f8700 1 -- 172.21.15.16:0/934479769 learned_addr learned my addr 172.21.15.16:0/934479769 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:51:55.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.650+0000 7f55491f7700 1 --2- 172.21.15.16:0/934479769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5544101cd0 0x7f5544196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:55.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.650+0000 7f55499f8700 1 -- 172.21.15.16:0/934479769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5544101cd0 msgr2=0x7f5544196a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:55.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.650+0000 7f55499f8700 1 --2- 172.21.15.16:0/934479769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5544101cd0 0x7f5544196a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:55.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.650+0000 7f55499f8700 1 -- 172.21.15.16:0/934479769 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55400095f0 con 0x7f5544100ad0 2024-04-02T04:51:55.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.650+0000 7f55499f8700 1 --2- 172.21.15.16:0/934479769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5544100ad0 0x7f55441964f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f554000b620 tx=0x7f5540005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:55.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.650+0000 7f553affd700 1 -- 172.21.15.16:0/934479769 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f554001d070 con 0x7f5544100ad0 2024-04-02T04:51:55.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.650+0000 7f553affd700 1 -- 172.21.15.16:0/934479769 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f55400052a0 con 0x7f5544100ad0 2024-04-02T04:51:55.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.650+0000 7f553affd700 1 -- 172.21.15.16:0/934479769 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5540017400 con 0x7f5544100ad0 2024-04-02T04:51:55.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.650+0000 7f554bc5c700 1 -- 172.21.15.16:0/934479769 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f554419bb90 con 0x7f5544100ad0 2024-04-02T04:51:55.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.650+0000 7f554bc5c700 1 -- 172.21.15.16:0/934479769 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f554419c080 con 0x7f5544100ad0 2024-04-02T04:51:55.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.652+0000 7f554bc5c700 1 -- 172.21.15.16:0/934479769 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5544190650 con 0x7f5544100ad0 2024-04-02T04:51:55.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.652+0000 7f553affd700 1 -- 172.21.15.16:0/934479769 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f5540003680 con 0x7f5544100ad0 2024-04-02T04:51:55.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.653+0000 7f553affd700 1 --2- 172.21.15.16:0/934479769 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f553006dd70 0x7f5530070230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:55.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.653+0000 7f553affd700 1 -- 172.21.15.16:0/934479769 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f554008eea0 con 0x7f5544100ad0 2024-04-02T04:51:55.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.653+0000 7f55491f7700 1 --2- 172.21.15.16:0/934479769 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f553006dd70 0x7f5530070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:55.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.653+0000 7f55491f7700 1 --2- 172.21.15.16:0/934479769 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f553006dd70 0x7f5530070230 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5534009c60 tx=0x7f5534009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:55.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.658+0000 7f553affd700 1 -- 172.21.15.16:0/934479769 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f554005bf40 con 0x7f5544100ad0 2024-04-02T04:51:55.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.909+0000 7f554bc5c700 1 -- 172.21.15.16:0/934479769 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f554402ce10 con 0x7f553006dd70 2024-04-02T04:51:55.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.911+0000 7f553affd700 1 -- 172.21.15.16:0/934479769 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+460 (secure 0 0 0) 0x7f554402ce10 con 0x7f553006dd70 2024-04-02T04:51:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.913+0000 7f554bc5c700 1 -- 172.21.15.16:0/934479769 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f553006dd70 msgr2=0x7f5530070230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.913+0000 7f554bc5c700 1 --2- 172.21.15.16:0/934479769 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f553006dd70 0x7f5530070230 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5534009c60 tx=0x7f5534009260 comp rx=0 tx=0).stop 2024-04-02T04:51:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.913+0000 7f554bc5c700 1 -- 172.21.15.16:0/934479769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5544100ad0 msgr2=0x7f55441964f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.913+0000 7f554bc5c700 1 --2- 172.21.15.16:0/934479769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5544100ad0 0x7f55441964f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f554000b620 tx=0x7f5540005f00 comp rx=0 tx=0).stop 2024-04-02T04:51:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.913+0000 7f554bc5c700 1 -- 172.21.15.16:0/934479769 shutdown_connections 2024-04-02T04:51:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.913+0000 7f554bc5c700 1 --2- 172.21.15.16:0/934479769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5544100ad0 0x7f55441964f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.913+0000 7f554bc5c700 1 --2- 172.21.15.16:0/934479769 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f553006dd70 0x7f5530070230 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.913+0000 7f554bc5c700 1 --2- 172.21.15.16:0/934479769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5544101cd0 0x7f5544196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.913+0000 7f554bc5c700 1 -- 172.21.15.16:0/934479769 >> 172.21.15.16:0/934479769 conn(0x7f55440fc090 msgr2=0x7f55441052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.913+0000 7f554bc5c700 1 -- 172.21.15.16:0/934479769 shutdown_connections 2024-04-02T04:51:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:55.913+0000 7f554bc5c700 1 -- 172.21.15.16:0/934479769 wait complete. 2024-04-02T04:51:56.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.055+0000 7f0c4bf64700 1 -- 172.21.15.16:0/1644642810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c44101560 msgr2=0x7f0c44103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:56.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.055+0000 7f0c4bf64700 1 --2- 172.21.15.16:0/1644642810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c44101560 0x7f0c44103950 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0c40009910 tx=0x7f0c40009c20 comp rx=0 tx=0).stop 2024-04-02T04:51:56.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.055+0000 7f0c4bf64700 1 -- 172.21.15.16:0/1644642810 shutdown_connections 2024-04-02T04:51:56.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.055+0000 7f0c4bf64700 1 --2- 172.21.15.16:0/1644642810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c44103e90 0x7f0c44106280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:56.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.055+0000 7f0c4bf64700 1 --2- 172.21.15.16:0/1644642810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c44101560 0x7f0c44103950 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:56.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.055+0000 7f0c4bf64700 1 -- 172.21.15.16:0/1644642810 >> 172.21.15.16:0/1644642810 conn(0x7f0c440faf70 msgr2=0x7f0c440fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:56.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.055+0000 7f0c4bf64700 1 -- 172.21.15.16:0/1644642810 shutdown_connections 2024-04-02T04:51:56.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.055+0000 7f0c4bf64700 1 -- 172.21.15.16:0/1644642810 wait complete. 2024-04-02T04:51:56.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.056+0000 7f0c4bf64700 1 Processor -- start 2024-04-02T04:51:56.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.056+0000 7f0c4bf64700 1 -- start start 2024-04-02T04:51:56.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.056+0000 7f0c4bf64700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c44101560 0x7f0c44194290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:56.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.056+0000 7f0c4bf64700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c44103e90 0x7f0c441947d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:56.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.056+0000 7f0c4bf64700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c44194d60 con 0x7f0c44103e90 2024-04-02T04:51:56.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.056+0000 7f0c494ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c44103e90 0x7f0c441947d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:56.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.056+0000 7f0c4bf64700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c44194ea0 con 0x7f0c44101560 2024-04-02T04:51:56.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.056+0000 7f0c494ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c44103e90 0x7f0c441947d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51128/0 (socket says 172.21.15.16:51128) 2024-04-02T04:51:56.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.056+0000 7f0c494ff700 1 -- 172.21.15.16:0/169605798 learned_addr learned my addr 172.21.15.16:0/169605798 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:51:56.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.056+0000 7f0c49d00700 1 --2- 172.21.15.16:0/169605798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c44101560 0x7f0c44194290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:56.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.056+0000 7f0c494ff700 1 -- 172.21.15.16:0/169605798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c44101560 msgr2=0x7f0c44194290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:56.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.056+0000 7f0c494ff700 1 --2- 172.21.15.16:0/169605798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c44101560 0x7f0c44194290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:56.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.056+0000 7f0c494ff700 1 -- 172.21.15.16:0/169605798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c400095f0 con 0x7f0c44103e90 2024-04-02T04:51:56.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.056+0000 7f0c49d00700 1 --2- 172.21.15.16:0/169605798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c44101560 0x7f0c44194290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T04:51:56.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.057+0000 7f0c494ff700 1 --2- 172.21.15.16:0/169605798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c44103e90 0x7f0c441947d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0c3400c930 tx=0x7f0c3400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:56.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.057+0000 7f0c3affd700 1 -- 172.21.15.16:0/169605798 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c34007ad0 con 0x7f0c44103e90 2024-04-02T04:51:56.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.057+0000 7f0c3affd700 1 -- 172.21.15.16:0/169605798 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0c3400ce80 con 0x7f0c44103e90 2024-04-02T04:51:56.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.058+0000 7f0c3affd700 1 -- 172.21.15.16:0/169605798 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c34018630 con 0x7f0c44103e90 2024-04-02T04:51:56.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.058+0000 7f0c4bf64700 1 -- 172.21.15.16:0/169605798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c44199910 con 0x7f0c44103e90 2024-04-02T04:51:56.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.058+0000 7f0c4bf64700 1 -- 172.21.15.16:0/169605798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c44199e50 con 0x7f0c44103e90 2024-04-02T04:51:56.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.059+0000 7f0c4bf64700 1 -- 172.21.15.16:0/169605798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c4404ea80 con 0x7f0c44103e90 2024-04-02T04:51:56.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.059+0000 7f0c3affd700 1 -- 172.21.15.16:0/169605798 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f0c34007c30 con 0x7f0c44103e90 2024-04-02T04:51:56.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.060+0000 7f0c3affd700 1 --2- 172.21.15.16:0/169605798 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f0c3006dd80 0x7f0c30070240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:56.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.060+0000 7f0c3affd700 1 -- 172.21.15.16:0/169605798 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f0c3408db30 con 0x7f0c44103e90 2024-04-02T04:51:56.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.060+0000 7f0c49d00700 1 --2- 172.21.15.16:0/169605798 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f0c3006dd80 0x7f0c30070240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:56.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.060+0000 7f0c49d00700 1 --2- 172.21.15.16:0/169605798 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f0c3006dd80 0x7f0c30070240 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0c40006010 tx=0x7f0c4000b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:56.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.065+0000 7f0c3affd700 1 -- 172.21.15.16:0/169605798 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0c3405abd0 con 0x7f0c44103e90 2024-04-02T04:51:56.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.311+0000 7f0c4bf64700 1 -- 172.21.15.16:0/169605798 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c44105d80 con 0x7f0c3006dd80 2024-04-02T04:51:56.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.328+0000 7f0c3affd700 1 -- 172.21.15.16:0/169605798 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f0c44105d80 con 0x7f0c3006dd80 2024-04-02T04:51:56.333 INFO:teuthology.orchestra.run.smithi016.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:51:56.333 INFO:teuthology.orchestra.run.smithi016.stdout:alertmanager.smithi016 smithi016 *:9093,9094 running (25s) 20s ago 17m 16.2M - 0.25.0 c8568f914cd2 6c7b91689a07 2024-04-02T04:51:56.333 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi016 smithi016 running (83s) 20s ago 18m 7918k - 19.0.0-1979-g6cec793d 78f830190839 1d3ba1f157b9 2024-04-02T04:51:56.333 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi027 smithi027 running (78s) 57s ago 17m 7558k - 19.0.0-1979-g6cec793d 78f830190839 3402997325e5 2024-04-02T04:51:56.333 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi016 smithi016 running (6m) 20s ago 18m 6660k - 19.0.0-1979-g6cec793d 78f830190839 dc63ce0eef58 2024-04-02T04:51:56.333 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi027 smithi027 running (6m) 57s ago 16m 6665k - 19.0.0-1979-g6cec793d 78f830190839 2beb077018e4 2024-04-02T04:51:56.333 INFO:teuthology.orchestra.run.smithi016.stdout:grafana.smithi016 smithi016 *:3000 starting - - - - 2024-04-02T04:51:56.333 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.fyzrnm smithi016 running (2m) 20s ago 12m 16.4M - 19.0.0-1979-g6cec793d 78f830190839 98f0a83c6631 2024-04-02T04:51:56.333 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.uhgkgg smithi016 running (2m) 20s ago 12m 89.4M - 19.0.0-1979-g6cec793d 78f830190839 19be3989a243 2024-04-02T04:51:56.333 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.sdublg smithi027 running (97s) 57s ago 12m 12.8M - 19.0.0-1979-g6cec793d 78f830190839 7c307783baab 2024-04-02T04:51:56.333 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.zwrfoo smithi027 running (2m) 57s ago 12m 13.6M - 19.0.0-1979-g6cec793d 78f830190839 acc6430302ac 2024-04-02T04:51:56.333 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi016.eqnebx smithi016 *:8443,9283,8765 running (8m) 20s ago 19m 513M - 19.0.0-1979-g6cec793d 78f830190839 d61ed91b4577 2024-04-02T04:51:56.333 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi027.kjzcsc smithi027 *:8443,9283,8765 running (8m) 57s ago 16m 451M - 19.0.0-1979-g6cec793d 78f830190839 5072a5ffc761 2024-04-02T04:51:56.333 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi016 smithi016 running (7m) 20s ago 19m 55.3M 2048M 19.0.0-1979-g6cec793d 78f830190839 eabfc84f1cdf 2024-04-02T04:51:56.334 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi027 smithi027 running (7m) 57s ago 16m 44.0M 2048M 19.0.0-1979-g6cec793d 78f830190839 fcd68b20dcc5 2024-04-02T04:51:56.334 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi016 smithi016 *:9100 running (64s) 20s ago 18m 8468k - 1.5.0 0da6a335fe13 f3e285b06d71 2024-04-02T04:51:56.334 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi027 smithi027 *:9100 running (59s) 57s ago 16m 3649k - 1.5.0 0da6a335fe13 c8e26f58ca63 2024-04-02T04:51:56.334 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 smithi016 running (6m) 20s ago 15m 91.4M 1217M 19.0.0-1979-g6cec793d 78f830190839 421d9e1cc751 2024-04-02T04:51:56.334 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 smithi016 running (5m) 20s ago 15m 100M 1217M 19.0.0-1979-g6cec793d 78f830190839 3612a56236a3 2024-04-02T04:51:56.334 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 smithi016 running (4m) 20s ago 15m 91.8M 1217M 19.0.0-1979-g6cec793d 78f830190839 a8e9b8d0303c 2024-04-02T04:51:56.334 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 smithi027 running (4m) 57s ago 14m 112M 2240M 19.0.0-1979-g6cec793d 78f830190839 0dd13b5be8c0 2024-04-02T04:51:56.334 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 smithi027 running (3m) 57s ago 14m 89.3M 2240M 19.0.0-1979-g6cec793d 78f830190839 9025eacf8145 2024-04-02T04:51:56.334 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 smithi027 running (3m) 57s ago 13m 88.2M 2240M 19.0.0-1979-g6cec793d 78f830190839 c2787f157377 2024-04-02T04:51:56.334 INFO:teuthology.orchestra.run.smithi016.stdout:prometheus.smithi016 smithi016 *:9095 running (42s) 20s ago 17m 44.5M - 2.43.0 a07b618ecd1d 042152fa1836 2024-04-02T04:51:56.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.331+0000 7f0c4bf64700 1 -- 172.21.15.16:0/169605798 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f0c3006dd80 msgr2=0x7f0c30070240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:56.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.331+0000 7f0c4bf64700 1 --2- 172.21.15.16:0/169605798 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f0c3006dd80 0x7f0c30070240 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0c40006010 tx=0x7f0c4000b540 comp rx=0 tx=0).stop 2024-04-02T04:51:56.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.331+0000 7f0c4bf64700 1 -- 172.21.15.16:0/169605798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c44103e90 msgr2=0x7f0c441947d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:56.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.331+0000 7f0c4bf64700 1 --2- 172.21.15.16:0/169605798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c44103e90 0x7f0c441947d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0c3400c930 tx=0x7f0c3400cc40 comp rx=0 tx=0).stop 2024-04-02T04:51:56.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.331+0000 7f0c4bf64700 1 -- 172.21.15.16:0/169605798 shutdown_connections 2024-04-02T04:51:56.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.331+0000 7f0c4bf64700 1 --2- 172.21.15.16:0/169605798 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f0c3006dd80 0x7f0c30070240 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:56.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.331+0000 7f0c4bf64700 1 --2- 172.21.15.16:0/169605798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c44101560 0x7f0c44194290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:56.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.331+0000 7f0c4bf64700 1 --2- 172.21.15.16:0/169605798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c44103e90 0x7f0c441947d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:56.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.331+0000 7f0c4bf64700 1 -- 172.21.15.16:0/169605798 >> 172.21.15.16:0/169605798 conn(0x7f0c440faf70 msgr2=0x7f0c440fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:56.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.331+0000 7f0c4bf64700 1 -- 172.21.15.16:0/169605798 shutdown_connections 2024-04-02T04:51:56.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.331+0000 7f0c4bf64700 1 -- 172.21.15.16:0/169605798 wait complete. 2024-04-02T04:51:56.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.445+0000 7f8bebb87700 1 -- 172.21.15.16:0/1023333193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8be40ff9d0 msgr2=0x7f8be40ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:56.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.445+0000 7f8bebb87700 1 --2- 172.21.15.16:0/1023333193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8be40ff9d0 0x7f8be40ffdf0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8be0009910 tx=0x7f8be0009c20 comp rx=0 tx=0).stop 2024-04-02T04:51:56.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.445+0000 7f8bebb87700 1 -- 172.21.15.16:0/1023333193 shutdown_connections 2024-04-02T04:51:56.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.445+0000 7f8bebb87700 1 --2- 172.21.15.16:0/1023333193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8be4100bd0 0x7f8be4101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:56.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.445+0000 7f8bebb87700 1 --2- 172.21.15.16:0/1023333193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8be40ff9d0 0x7f8be40ffdf0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:56.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.446+0000 7f8bebb87700 1 -- 172.21.15.16:0/1023333193 >> 172.21.15.16:0/1023333193 conn(0x7f8be40faf70 msgr2=0x7f8be40fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:56.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.446+0000 7f8bebb87700 1 -- 172.21.15.16:0/1023333193 shutdown_connections 2024-04-02T04:51:56.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.446+0000 7f8bebb87700 1 -- 172.21.15.16:0/1023333193 wait complete. 2024-04-02T04:51:56.451 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:56 smithi016 ceph-mon[93230]: pgmap v267: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:56.451 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:56 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:56.451 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:56 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:56.451 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:56 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:51:56.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.446+0000 7f8bebb87700 1 Processor -- start 2024-04-02T04:51:56.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.446+0000 7f8bebb87700 1 -- start start 2024-04-02T04:51:56.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.447+0000 7f8bebb87700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8be40ff9d0 0x7f8be4068fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:56.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.447+0000 7f8bebb87700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8be4100bd0 0x7f8be419b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:56.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.447+0000 7f8bebb87700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be419b810 con 0x7f8be40ff9d0 2024-04-02T04:51:56.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.447+0000 7f8bebb87700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be419b950 con 0x7f8be4100bd0 2024-04-02T04:51:56.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.447+0000 7f8be9923700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8be40ff9d0 0x7f8be4068fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:56.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.447+0000 7f8be9923700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8be40ff9d0 0x7f8be4068fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51146/0 (socket says 172.21.15.16:51146) 2024-04-02T04:51:56.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.447+0000 7f8be9923700 1 -- 172.21.15.16:0/1498897590 learned_addr learned my addr 172.21.15.16:0/1498897590 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:51:56.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.447+0000 7f8be9122700 1 --2- 172.21.15.16:0/1498897590 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8be4100bd0 0x7f8be419b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:56.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.447+0000 7f8be9122700 1 -- 172.21.15.16:0/1498897590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8be40ff9d0 msgr2=0x7f8be4068fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:56.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.447+0000 7f8be9122700 1 --2- 172.21.15.16:0/1498897590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8be40ff9d0 0x7f8be4068fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:56.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.447+0000 7f8be9122700 1 -- 172.21.15.16:0/1498897590 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be00095f0 con 0x7f8be4100bd0 2024-04-02T04:51:56.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.447+0000 7f8be9122700 1 --2- 172.21.15.16:0/1498897590 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8be4100bd0 0x7f8be419b240 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800e970 tx=0x7f8bd800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:56.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.448+0000 7f8bd6ffd700 1 -- 172.21.15.16:0/1498897590 <== mon.1 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bd800c9d0 con 0x7f8be4100bd0 2024-04-02T04:51:56.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.448+0000 7f8bebb87700 1 -- 172.21.15.16:0/1498897590 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8be419bbe0 con 0x7f8be4100bd0 2024-04-02T04:51:56.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.448+0000 7f8bd6ffd700 1 -- 172.21.15.16:0/1498897590 <== mon.1 v2:172.21.15.27:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8bd8010430 con 0x7f8be4100bd0 2024-04-02T04:51:56.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.448+0000 7f8bd6ffd700 1 -- 172.21.15.16:0/1498897590 <== mon.1 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bd8018630 con 0x7f8be4100bd0 2024-04-02T04:51:56.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.448+0000 7f8bebb87700 1 -- 172.21.15.16:0/1498897590 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8be419c1b0 con 0x7f8be4100bd0 2024-04-02T04:51:56.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.449+0000 7f8bebb87700 1 -- 172.21.15.16:0/1498897590 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8be418f550 con 0x7f8be4100bd0 2024-04-02T04:51:56.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.449+0000 7f8bd6ffd700 1 -- 172.21.15.16:0/1498897590 <== mon.1 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f8bd80105a0 con 0x7f8be4100bd0 2024-04-02T04:51:56.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.450+0000 7f8bd6ffd700 1 --2- 172.21.15.16:0/1498897590 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f8bd006dd60 0x7f8bd0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:56.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.450+0000 7f8bd6ffd700 1 -- 172.21.15.16:0/1498897590 <== mon.1 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f8bd8014070 con 0x7f8be4100bd0 2024-04-02T04:51:56.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.450+0000 7f8be9923700 1 --2- 172.21.15.16:0/1498897590 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f8bd006dd60 0x7f8bd0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:56.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.450+0000 7f8be9923700 1 --2- 172.21.15.16:0/1498897590 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f8bd006dd60 0x7f8bd0070220 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8be000b800 tx=0x7f8be0009f70 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.455+0000 7f8bd6ffd700 1 -- 172.21.15.16:0/1498897590 <== mon.1 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8bd8053280 con 0x7f8be4100bd0 2024-04-02T04:51:56.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:56 smithi027 ceph-mon[83036]: pgmap v267: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:56.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:56 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:56.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:56 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:51:56.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:56 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:51:56.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.795+0000 7f8bebb87700 1 -- 172.21.15.16:0/1498897590 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8be404ea80 con 0x7f8be4100bd0 2024-04-02T04:51:56.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.796+0000 7f8bd6ffd700 1 -- 172.21.15.16:0/1498897590 <== mon.1 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f8bd805ae40 con 0x7f8be4100bd0 2024-04-02T04:51:56.801 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:51:56.801 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": { 2024-04-02T04:51:56.801 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:51:56.801 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:51:56.801 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": { 2024-04-02T04:51:56.801 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:51:56.801 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:51:56.801 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": { 2024-04-02T04:51:56.801 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 6 2024-04-02T04:51:56.801 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:51:56.801 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-04-02T04:51:56.801 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 4 2024-04-02T04:51:56.801 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:51:56.802 INFO:teuthology.orchestra.run.smithi016.stdout: "overall": { 2024-04-02T04:51:56.802 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 14 2024-04-02T04:51:56.802 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-02T04:51:56.802 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:51:56.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.799+0000 7f8bebb87700 1 -- 172.21.15.16:0/1498897590 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f8bd006dd60 msgr2=0x7f8bd0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:56.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.799+0000 7f8bebb87700 1 --2- 172.21.15.16:0/1498897590 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f8bd006dd60 0x7f8bd0070220 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8be000b800 tx=0x7f8be0009f70 comp rx=0 tx=0).stop 2024-04-02T04:51:56.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.799+0000 7f8bebb87700 1 -- 172.21.15.16:0/1498897590 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8be4100bd0 msgr2=0x7f8be419b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:56.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.799+0000 7f8bebb87700 1 --2- 172.21.15.16:0/1498897590 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8be4100bd0 0x7f8be419b240 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800e970 tx=0x7f8bd800ec80 comp rx=0 tx=0).stop 2024-04-02T04:51:56.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.799+0000 7f8bebb87700 1 -- 172.21.15.16:0/1498897590 shutdown_connections 2024-04-02T04:51:56.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.799+0000 7f8bebb87700 1 --2- 172.21.15.16:0/1498897590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8be40ff9d0 0x7f8be4068fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:56.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.799+0000 7f8bebb87700 1 --2- 172.21.15.16:0/1498897590 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f8bd006dd60 0x7f8bd0070220 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:56.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.799+0000 7f8bebb87700 1 --2- 172.21.15.16:0/1498897590 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8be4100bd0 0x7f8be419b240 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:56.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.799+0000 7f8bebb87700 1 -- 172.21.15.16:0/1498897590 >> 172.21.15.16:0/1498897590 conn(0x7f8be40faf70 msgr2=0x7f8be4104220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:56.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.799+0000 7f8bebb87700 1 -- 172.21.15.16:0/1498897590 shutdown_connections 2024-04-02T04:51:56.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.799+0000 7f8bebb87700 1 -- 172.21.15.16:0/1498897590 wait complete. 2024-04-02T04:51:56.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.917+0000 7f773ffea700 1 -- 172.21.15.16:0/3794343123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7738100360 msgr2=0x7f77381007c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:56.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.917+0000 7f773ffea700 1 --2- 172.21.15.16:0/3794343123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7738100360 0x7f77381007c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7728009910 tx=0x7f7728009c20 comp rx=0 tx=0).stop 2024-04-02T04:51:56.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.917+0000 7f773ffea700 1 -- 172.21.15.16:0/3794343123 shutdown_connections 2024-04-02T04:51:56.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.917+0000 7f773ffea700 1 --2- 172.21.15.16:0/3794343123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7738100360 0x7f77381007c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:56.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.917+0000 7f773ffea700 1 --2- 172.21.15.16:0/3794343123 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77380ff160 0x7f77380ff580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:56.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.917+0000 7f773ffea700 1 -- 172.21.15.16:0/3794343123 >> 172.21.15.16:0/3794343123 conn(0x7f77380fa700 msgr2=0x7f77380fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:56.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.917+0000 7f773ffea700 1 -- 172.21.15.16:0/3794343123 shutdown_connections 2024-04-02T04:51:56.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.917+0000 7f773ffea700 1 -- 172.21.15.16:0/3794343123 wait complete. 2024-04-02T04:51:56.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.918+0000 7f773ffea700 1 Processor -- start 2024-04-02T04:51:56.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.918+0000 7f773ffea700 1 -- start start 2024-04-02T04:51:56.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.918+0000 7f773ffea700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77380ff160 0x7f7738198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:56.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.918+0000 7f773ffea700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7738100360 0x7f7738198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:56.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.918+0000 7f773ffea700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7738199270 con 0x7f7738100360 2024-04-02T04:51:56.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.918+0000 7f773ffea700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77381993b0 con 0x7f77380ff160 2024-04-02T04:51:56.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.918+0000 7f773d585700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7738100360 0x7f7738198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:56.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.918+0000 7f773d585700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7738100360 0x7f7738198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51160/0 (socket says 172.21.15.16:51160) 2024-04-02T04:51:56.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.918+0000 7f773d585700 1 -- 172.21.15.16:0/1993038801 learned_addr learned my addr 172.21.15.16:0/1993038801 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:51:56.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.918+0000 7f773dd86700 1 --2- 172.21.15.16:0/1993038801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77380ff160 0x7f7738198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:56.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.919+0000 7f773d585700 1 -- 172.21.15.16:0/1993038801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77380ff160 msgr2=0x7f7738198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:56.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.919+0000 7f773d585700 1 --2- 172.21.15.16:0/1993038801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77380ff160 0x7f7738198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:56.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.919+0000 7f773d585700 1 -- 172.21.15.16:0/1993038801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77280095f0 con 0x7f7738100360 2024-04-02T04:51:56.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.919+0000 7f773d585700 1 --2- 172.21.15.16:0/1993038801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7738100360 0x7f7738198c50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f772800b6f0 tx=0x7f7728005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:56.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.919+0000 7f772effd700 1 -- 172.21.15.16:0/1993038801 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f772801d070 con 0x7f7738100360 2024-04-02T04:51:56.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.919+0000 7f772effd700 1 -- 172.21.15.16:0/1993038801 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f77280052a0 con 0x7f7738100360 2024-04-02T04:51:56.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.919+0000 7f773ffea700 1 -- 172.21.15.16:0/1993038801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f773819ddb0 con 0x7f7738100360 2024-04-02T04:51:56.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.919+0000 7f772effd700 1 -- 172.21.15.16:0/1993038801 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7728022470 con 0x7f7738100360 2024-04-02T04:51:56.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.920+0000 7f773ffea700 1 -- 172.21.15.16:0/1993038801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f773819e320 con 0x7f7738100360 2024-04-02T04:51:56.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.921+0000 7f773ffea700 1 -- 172.21.15.16:0/1993038801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7738104d30 con 0x7f7738100360 2024-04-02T04:51:56.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.921+0000 7f772effd700 1 -- 172.21.15.16:0/1993038801 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f772800f770 con 0x7f7738100360 2024-04-02T04:51:56.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.922+0000 7f772effd700 1 --2- 172.21.15.16:0/1993038801 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f772406dd60 0x7f7724070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:56.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.922+0000 7f772effd700 1 -- 172.21.15.16:0/1993038801 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f772808ef20 con 0x7f7738100360 2024-04-02T04:51:56.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.922+0000 7f773dd86700 1 --2- 172.21.15.16:0/1993038801 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f772406dd60 0x7f7724070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:56.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.923+0000 7f773dd86700 1 --2- 172.21.15.16:0/1993038801 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f772406dd60 0x7f7724070220 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f77381001c0 tx=0x7f7734009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:56.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:56.928+0000 7f772effd700 1 -- 172.21.15.16:0/1993038801 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f772805bfc0 con 0x7f7738100360 2024-04-02T04:51:57.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.187+0000 7f773ffea700 1 -- 172.21.15.16:0/1993038801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f773804ea80 con 0x7f7738100360 2024-04-02T04:51:57.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.188+0000 7f772effd700 1 -- 172.21.15.16:0/1993038801 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1929 (secure 0 0 0) 0x7f772805bb50 con 0x7f7738100360 2024-04-02T04:51:57.192 INFO:teuthology.orchestra.run.smithi016.stdout:e29 2024-04-02T04:51:57.192 INFO:teuthology.orchestra.run.smithi016.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:51:57.192 INFO:teuthology.orchestra.run.smithi016.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:51:57.192 INFO:teuthology.orchestra.run.smithi016.stdout:legacy client fscid: 1 2024-04-02T04:51:57.192 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:51:57.193 INFO:teuthology.orchestra.run.smithi016.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:51:57.193 INFO:teuthology.orchestra.run.smithi016.stdout:fs_name cephfs 2024-04-02T04:51:57.193 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 29 2024-04-02T04:51:57.193 INFO:teuthology.orchestra.run.smithi016.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:51:57.193 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-04-02T04:39:30.740168+0000 2024-04-02T04:51:57.193 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-04-02T04:50:20.104880+0000 2024-04-02T04:51:57.193 INFO:teuthology.orchestra.run.smithi016.stdout:tableserver 0 2024-04-02T04:51:57.193 INFO:teuthology.orchestra.run.smithi016.stdout:root 0 2024-04-02T04:51:57.193 INFO:teuthology.orchestra.run.smithi016.stdout:session_timeout 60 2024-04-02T04:51:57.193 INFO:teuthology.orchestra.run.smithi016.stdout:session_autoclose 300 2024-04-02T04:51:57.193 INFO:teuthology.orchestra.run.smithi016.stdout:max_file_size 1099511627776 2024-04-02T04:51:57.193 INFO:teuthology.orchestra.run.smithi016.stdout:max_xattr_size 65536 2024-04-02T04:51:57.193 INFO:teuthology.orchestra.run.smithi016.stdout:required_client_features {} 2024-04-02T04:51:57.193 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure 0 2024-04-02T04:51:57.193 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure_osd_epoch 84 2024-04-02T04:51:57.194 INFO:teuthology.orchestra.run.smithi016.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-02T04:51:57.194 INFO:teuthology.orchestra.run.smithi016.stdout:max_mds 1 2024-04-02T04:51:57.194 INFO:teuthology.orchestra.run.smithi016.stdout:in 0 2024-04-02T04:51:57.194 INFO:teuthology.orchestra.run.smithi016.stdout:up {0=34404} 2024-04-02T04:51:57.194 INFO:teuthology.orchestra.run.smithi016.stdout:failed 2024-04-02T04:51:57.194 INFO:teuthology.orchestra.run.smithi016.stdout:damaged 2024-04-02T04:51:57.194 INFO:teuthology.orchestra.run.smithi016.stdout:stopped 2024-04-02T04:51:57.194 INFO:teuthology.orchestra.run.smithi016.stdout:data_pools [3] 2024-04-02T04:51:57.194 INFO:teuthology.orchestra.run.smithi016.stdout:metadata_pool 2 2024-04-02T04:51:57.194 INFO:teuthology.orchestra.run.smithi016.stdout:inline_data enabled 2024-04-02T04:51:57.194 INFO:teuthology.orchestra.run.smithi016.stdout:balancer 2024-04-02T04:51:57.194 INFO:teuthology.orchestra.run.smithi016.stdout:bal_rank_mask -1 2024-04-02T04:51:57.194 INFO:teuthology.orchestra.run.smithi016.stdout:standby_count_wanted 1 2024-04-02T04:51:57.194 INFO:teuthology.orchestra.run.smithi016.stdout:qdb_cluster leader: 34404 members: 34404 2024-04-02T04:51:57.195 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.uhgkgg{0:34404} state up:active seq 21 join_fscid=1 addr [v2:172.21.15.16:6826/81744708,v1:172.21.15.16:6827/81744708] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T04:51:57.195 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:51:57.195 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:51:57.195 INFO:teuthology.orchestra.run.smithi016.stdout:Standby daemons: 2024-04-02T04:51:57.195 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:51:57.195 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.fyzrnm{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.16:6828/782430688,v1:172.21.15.16:6829/782430688] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T04:51:57.195 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.zwrfoo{-1:44359} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.27:6824/1016194753,v1:172.21.15.27:6825/1016194753] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T04:51:57.195 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.sdublg{-1:44383} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.27:6826/1507844210,v1:172.21.15.27:6827/1507844210] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T04:51:57.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.190+0000 7f773ffea700 1 -- 172.21.15.16:0/1993038801 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f772406dd60 msgr2=0x7f7724070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:57.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.190+0000 7f773ffea700 1 --2- 172.21.15.16:0/1993038801 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f772406dd60 0x7f7724070220 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f77381001c0 tx=0x7f7734009260 comp rx=0 tx=0).stop 2024-04-02T04:51:57.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.190+0000 7f773ffea700 1 -- 172.21.15.16:0/1993038801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7738100360 msgr2=0x7f7738198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:57.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.190+0000 7f773ffea700 1 --2- 172.21.15.16:0/1993038801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7738100360 0x7f7738198c50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f772800b6f0 tx=0x7f7728005e30 comp rx=0 tx=0).stop 2024-04-02T04:51:57.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.190+0000 7f773ffea700 1 -- 172.21.15.16:0/1993038801 shutdown_connections 2024-04-02T04:51:57.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.190+0000 7f773ffea700 1 --2- 172.21.15.16:0/1993038801 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f772406dd60 0x7f7724070220 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:57.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.190+0000 7f773ffea700 1 --2- 172.21.15.16:0/1993038801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77380ff160 0x7f7738198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:57.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.190+0000 7f773ffea700 1 --2- 172.21.15.16:0/1993038801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7738100360 0x7f7738198c50 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:57.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.190+0000 7f773ffea700 1 -- 172.21.15.16:0/1993038801 >> 172.21.15.16:0/1993038801 conn(0x7f77380fa700 msgr2=0x7f7738108e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:57.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.190+0000 7f773ffea700 1 -- 172.21.15.16:0/1993038801 shutdown_connections 2024-04-02T04:51:57.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.190+0000 7f773ffea700 1 -- 172.21.15.16:0/1993038801 wait complete. 2024-04-02T04:51:57.197 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 29 2024-04-02T04:51:57.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.304+0000 7f8eeb6f5700 1 -- 172.21.15.16:0/248955948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee40ff9d0 msgr2=0x7f8ee40ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:57.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.304+0000 7f8eeb6f5700 1 --2- 172.21.15.16:0/248955948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee40ff9d0 0x7f8ee40ffdf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0009910 tx=0x7f8ee0009c20 comp rx=0 tx=0).stop 2024-04-02T04:51:57.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.304+0000 7f8eeb6f5700 1 -- 172.21.15.16:0/248955948 shutdown_connections 2024-04-02T04:51:57.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.304+0000 7f8eeb6f5700 1 --2- 172.21.15.16:0/248955948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ee4100bd0 0x7f8ee4101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:57.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.304+0000 7f8eeb6f5700 1 --2- 172.21.15.16:0/248955948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee40ff9d0 0x7f8ee40ffdf0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:57.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.304+0000 7f8eeb6f5700 1 -- 172.21.15.16:0/248955948 >> 172.21.15.16:0/248955948 conn(0x7f8ee40faf70 msgr2=0x7f8ee40fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:57.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.305+0000 7f8eeb6f5700 1 -- 172.21.15.16:0/248955948 shutdown_connections 2024-04-02T04:51:57.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.305+0000 7f8eeb6f5700 1 -- 172.21.15.16:0/248955948 wait complete. 2024-04-02T04:51:57.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.305+0000 7f8eeb6f5700 1 Processor -- start 2024-04-02T04:51:57.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.305+0000 7f8eeb6f5700 1 -- start start 2024-04-02T04:51:57.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.305+0000 7f8eeb6f5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ee40ff9d0 0x7f8ee4196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:57.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.305+0000 7f8eeb6f5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee4100bd0 0x7f8ee4196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:57.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.305+0000 7f8eeb6f5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee41970e0 con 0x7f8ee4100bd0 2024-04-02T04:51:57.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.305+0000 7f8eeb6f5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee4197220 con 0x7f8ee40ff9d0 2024-04-02T04:51:57.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.305+0000 7f8ee8c90700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee4100bd0 0x7f8ee4196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:57.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.305+0000 7f8ee8c90700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee4100bd0 0x7f8ee4196ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51174/0 (socket says 172.21.15.16:51174) 2024-04-02T04:51:57.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.306+0000 7f8ee9491700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ee40ff9d0 0x7f8ee4196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:57.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.306+0000 7f8ee8c90700 1 -- 172.21.15.16:0/4271884849 learned_addr learned my addr 172.21.15.16:0/4271884849 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:51:57.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.306+0000 7f8ee8c90700 1 -- 172.21.15.16:0/4271884849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ee40ff9d0 msgr2=0x7f8ee4196580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:57.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.306+0000 7f8ee8c90700 1 --2- 172.21.15.16:0/4271884849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ee40ff9d0 0x7f8ee4196580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:57.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.306+0000 7f8ee8c90700 1 -- 172.21.15.16:0/4271884849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee00095f0 con 0x7f8ee4100bd0 2024-04-02T04:51:57.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.306+0000 7f8ee8c90700 1 --2- 172.21.15.16:0/4271884849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee4100bd0 0x7f8ee4196ac0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8ed400c8a0 tx=0x7f8ed400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:57.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.306+0000 7f8eda7fc700 1 -- 172.21.15.16:0/4271884849 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ed40078e0 con 0x7f8ee4100bd0 2024-04-02T04:51:57.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.306+0000 7f8eda7fc700 1 -- 172.21.15.16:0/4271884849 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ed400f450 con 0x7f8ee4100bd0 2024-04-02T04:51:57.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.306+0000 7f8eda7fc700 1 -- 172.21.15.16:0/4271884849 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ed400e5e0 con 0x7f8ee4100bd0 2024-04-02T04:51:57.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.306+0000 7f8eeb6f5700 1 -- 172.21.15.16:0/4271884849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ee4101e50 con 0x7f8ee4100bd0 2024-04-02T04:51:57.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.306+0000 7f8eeb6f5700 1 -- 172.21.15.16:0/4271884849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ee4102420 con 0x7f8ee4100bd0 2024-04-02T04:51:57.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.308+0000 7f8eeb6f5700 1 -- 172.21.15.16:0/4271884849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ee404ea80 con 0x7f8ee4100bd0 2024-04-02T04:51:57.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.308+0000 7f8eda7fc700 1 -- 172.21.15.16:0/4271884849 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f8ed400f5c0 con 0x7f8ee4100bd0 2024-04-02T04:51:57.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.308+0000 7f8eda7fc700 1 --2- 172.21.15.16:0/4271884849 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f8ed006dd70 0x7f8ed0070230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:57.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.308+0000 7f8eda7fc700 1 -- 172.21.15.16:0/4271884849 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f8ed408dac0 con 0x7f8ee4100bd0 2024-04-02T04:51:57.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.308+0000 7f8ee9491700 1 --2- 172.21.15.16:0/4271884849 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f8ed006dd70 0x7f8ed0070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:57.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.309+0000 7f8ee9491700 1 --2- 172.21.15.16:0/4271884849 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f8ed006dd70 0x7f8ed0070230 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0005090 tx=0x7f8ee000b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:57.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.314+0000 7f8eda7fc700 1 -- 172.21.15.16:0/4271884849 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8ed405ab60 con 0x7f8ee4100bd0 2024-04-02T04:51:57.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.543+0000 7f8eeb6f5700 1 -- 172.21.15.16:0/4271884849 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ee4068c00 con 0x7f8ed006dd70 2024-04-02T04:51:57.548 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:57 smithi016 ceph-mon[93230]: from='client.44429 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:51:57.548 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:57 smithi016 ceph-mon[93230]: from='client.34568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:51:57.548 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:57 smithi016 ceph-mon[93230]: from='client.34572 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:51:57.548 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:57 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/1498897590' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:57.548 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:57 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/1993038801' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:51:57.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.545+0000 7f8eda7fc700 1 -- 172.21.15.16:0/4271884849 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+460 (secure 0 0 0) 0x7f8ee4068c00 con 0x7f8ed006dd70 2024-04-02T04:51:57.549 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:51:57.549 INFO:teuthology.orchestra.run.smithi016.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae", 2024-04-02T04:51:57.549 INFO:teuthology.orchestra.run.smithi016.stdout: "in_progress": true, 2024-04-02T04:51:57.549 INFO:teuthology.orchestra.run.smithi016.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-02T04:51:57.549 INFO:teuthology.orchestra.run.smithi016.stdout: "services_complete": [ 2024-04-02T04:51:57.549 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr", 2024-04-02T04:51:57.549 INFO:teuthology.orchestra.run.smithi016.stdout: "osd", 2024-04-02T04:51:57.549 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-04-02T04:51:57.549 INFO:teuthology.orchestra.run.smithi016.stdout: "mon", 2024-04-02T04:51:57.550 INFO:teuthology.orchestra.run.smithi016.stdout: "mds", 2024-04-02T04:51:57.550 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph-exporter" 2024-04-02T04:51:57.550 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-04-02T04:51:57.550 INFO:teuthology.orchestra.run.smithi016.stdout: "progress": "18/23 daemons upgraded", 2024-04-02T04:51:57.550 INFO:teuthology.orchestra.run.smithi016.stdout: "message": "Currently upgrading grafana daemons", 2024-04-02T04:51:57.550 INFO:teuthology.orchestra.run.smithi016.stdout: "is_paused": false 2024-04-02T04:51:57.550 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:51:57.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.546+0000 7f8eeb6f5700 1 -- 172.21.15.16:0/4271884849 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f8ed006dd70 msgr2=0x7f8ed0070230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:57.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.546+0000 7f8eeb6f5700 1 --2- 172.21.15.16:0/4271884849 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f8ed006dd70 0x7f8ed0070230 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0005090 tx=0x7f8ee000b540 comp rx=0 tx=0).stop 2024-04-02T04:51:57.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.547+0000 7f8eeb6f5700 1 -- 172.21.15.16:0/4271884849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee4100bd0 msgr2=0x7f8ee4196ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:57.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.547+0000 7f8eeb6f5700 1 --2- 172.21.15.16:0/4271884849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee4100bd0 0x7f8ee4196ac0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8ed400c8a0 tx=0x7f8ed400cbb0 comp rx=0 tx=0).stop 2024-04-02T04:51:57.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.547+0000 7f8eeb6f5700 1 -- 172.21.15.16:0/4271884849 shutdown_connections 2024-04-02T04:51:57.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.547+0000 7f8eeb6f5700 1 --2- 172.21.15.16:0/4271884849 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f8ed006dd70 0x7f8ed0070230 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:57.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.547+0000 7f8eeb6f5700 1 --2- 172.21.15.16:0/4271884849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ee40ff9d0 0x7f8ee4196580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:57.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.547+0000 7f8eeb6f5700 1 --2- 172.21.15.16:0/4271884849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee4100bd0 0x7f8ee4196ac0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:57.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.547+0000 7f8eeb6f5700 1 -- 172.21.15.16:0/4271884849 >> 172.21.15.16:0/4271884849 conn(0x7f8ee40faf70 msgr2=0x7f8ee40684f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:57.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.547+0000 7f8eeb6f5700 1 -- 172.21.15.16:0/4271884849 shutdown_connections 2024-04-02T04:51:57.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.547+0000 7f8eeb6f5700 1 -- 172.21.15.16:0/4271884849 wait complete. 2024-04-02T04:51:57.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:57 smithi027 ceph-mon[83036]: from='client.44429 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:51:57.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:57 smithi027 ceph-mon[83036]: from='client.34568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:51:57.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:57 smithi027 ceph-mon[83036]: from='client.34572 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:51:57.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:57 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/1498897590' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:51:57.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:57 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/1993038801' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:51:57.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.654+0000 7ff14f1c0700 1 -- 172.21.15.16:0/484036710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff148100ae0 msgr2=0x7ff148100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:57.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.654+0000 7ff14f1c0700 1 --2- 172.21.15.16:0/484036710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff148100ae0 0x7ff148100f00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff13c009910 tx=0x7ff13c009c20 comp rx=0 tx=0).stop 2024-04-02T04:51:57.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.654+0000 7ff14f1c0700 1 -- 172.21.15.16:0/484036710 shutdown_connections 2024-04-02T04:51:57.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.654+0000 7ff14f1c0700 1 --2- 172.21.15.16:0/484036710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff148101ce0 0x7ff148102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:57.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.654+0000 7ff14f1c0700 1 --2- 172.21.15.16:0/484036710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff148100ae0 0x7ff148100f00 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:57.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.654+0000 7ff14f1c0700 1 -- 172.21.15.16:0/484036710 >> 172.21.15.16:0/484036710 conn(0x7ff1480fc0a0 msgr2=0x7ff1480fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:57.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.654+0000 7ff14f1c0700 1 -- 172.21.15.16:0/484036710 shutdown_connections 2024-04-02T04:51:57.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.654+0000 7ff14f1c0700 1 -- 172.21.15.16:0/484036710 wait complete. 2024-04-02T04:51:57.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.655+0000 7ff14f1c0700 1 Processor -- start 2024-04-02T04:51:57.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.655+0000 7ff14f1c0700 1 -- start start 2024-04-02T04:51:57.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.655+0000 7ff14f1c0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff148100ae0 0x7ff148198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:57.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.655+0000 7ff14f1c0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff148101ce0 0x7ff148198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:57.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.655+0000 7ff14f1c0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff148199260 con 0x7ff148100ae0 2024-04-02T04:51:57.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.655+0000 7ff14f1c0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1481993a0 con 0x7ff148101ce0 2024-04-02T04:51:57.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.655+0000 7ff14cf5c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff148100ae0 0x7ff148198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:57.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.655+0000 7ff14cf5c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff148100ae0 0x7ff148198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51184/0 (socket says 172.21.15.16:51184) 2024-04-02T04:51:57.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.655+0000 7ff14cf5c700 1 -- 172.21.15.16:0/4036612204 learned_addr learned my addr 172.21.15.16:0/4036612204 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:51:57.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.655+0000 7ff147fff700 1 --2- 172.21.15.16:0/4036612204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff148101ce0 0x7ff148198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:57.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.656+0000 7ff14cf5c700 1 -- 172.21.15.16:0/4036612204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff148101ce0 msgr2=0x7ff148198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:57.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.656+0000 7ff14cf5c700 1 --2- 172.21.15.16:0/4036612204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff148101ce0 0x7ff148198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:57.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.656+0000 7ff14cf5c700 1 -- 172.21.15.16:0/4036612204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff13c0095f0 con 0x7ff148100ae0 2024-04-02T04:51:57.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.656+0000 7ff147fff700 1 --2- 172.21.15.16:0/4036612204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff148101ce0 0x7ff148198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T04:51:57.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.656+0000 7ff14cf5c700 1 --2- 172.21.15.16:0/4036612204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff148100ae0 0x7ff148198700 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff13c00b620 tx=0x7ff13c005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:57.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.656+0000 7ff145ffb700 1 -- 172.21.15.16:0/4036612204 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff13c01d070 con 0x7ff148100ae0 2024-04-02T04:51:57.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.656+0000 7ff145ffb700 1 -- 172.21.15.16:0/4036612204 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff13c0052a0 con 0x7ff148100ae0 2024-04-02T04:51:57.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.656+0000 7ff14f1c0700 1 -- 172.21.15.16:0/4036612204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff14819dda0 con 0x7ff148100ae0 2024-04-02T04:51:57.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.656+0000 7ff145ffb700 1 -- 172.21.15.16:0/4036612204 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff13c017400 con 0x7ff148100ae0 2024-04-02T04:51:57.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.656+0000 7ff14f1c0700 1 -- 172.21.15.16:0/4036612204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff14819e2b0 con 0x7ff148100ae0 2024-04-02T04:51:57.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.658+0000 7ff14f1c0700 1 -- 172.21.15.16:0/4036612204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff14804eaf0 con 0x7ff148100ae0 2024-04-02T04:51:57.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.658+0000 7ff145ffb700 1 -- 172.21.15.16:0/4036612204 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7ff13c00f820 con 0x7ff148100ae0 2024-04-02T04:51:57.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.659+0000 7ff145ffb700 1 --2- 172.21.15.16:0/4036612204 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ff13806dd70 0x7ff138070230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:51:57.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.659+0000 7ff145ffb700 1 -- 172.21.15.16:0/4036612204 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7ff13c08eeb0 con 0x7ff148100ae0 2024-04-02T04:51:57.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.659+0000 7ff147fff700 1 --2- 172.21.15.16:0/4036612204 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ff13806dd70 0x7ff138070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:51:57.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.659+0000 7ff147fff700 1 --2- 172.21.15.16:0/4036612204 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ff13806dd70 0x7ff138070230 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff134011490 tx=0x7ff134009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:51:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.664+0000 7ff145ffb700 1 -- 172.21.15.16:0/4036612204 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff14804eaf0 con 0x7ff148100ae0 2024-04-02T04:51:57.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.990+0000 7ff14f1c0700 1 -- 172.21.15.16:0/4036612204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff148109640 con 0x7ff148100ae0 2024-04-02T04:51:57.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.991+0000 7ff145ffb700 1 -- 172.21.15.16:0/4036612204 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7ff13c05bf50 con 0x7ff148100ae0 2024-04-02T04:51:57.995 INFO:teuthology.orchestra.run.smithi016.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-02T04:51:57.995 INFO:teuthology.orchestra.run.smithi016.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-02T04:51:57.996 INFO:teuthology.orchestra.run.smithi016.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-02T04:51:57.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.993+0000 7ff14f1c0700 1 -- 172.21.15.16:0/4036612204 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ff13806dd70 msgr2=0x7ff138070230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:57.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.993+0000 7ff14f1c0700 1 --2- 172.21.15.16:0/4036612204 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ff13806dd70 0x7ff138070230 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff134011490 tx=0x7ff134009260 comp rx=0 tx=0).stop 2024-04-02T04:51:57.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.993+0000 7ff14f1c0700 1 -- 172.21.15.16:0/4036612204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff148100ae0 msgr2=0x7ff148198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:51:57.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.993+0000 7ff14f1c0700 1 --2- 172.21.15.16:0/4036612204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff148100ae0 0x7ff148198700 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff13c00b620 tx=0x7ff13c005f00 comp rx=0 tx=0).stop 2024-04-02T04:51:57.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.993+0000 7ff14f1c0700 1 -- 172.21.15.16:0/4036612204 shutdown_connections 2024-04-02T04:51:57.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.993+0000 7ff14f1c0700 1 --2- 172.21.15.16:0/4036612204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff148100ae0 0x7ff148198700 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:57.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.993+0000 7ff14f1c0700 1 --2- 172.21.15.16:0/4036612204 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ff13806dd70 0x7ff138070230 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:57.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.993+0000 7ff14f1c0700 1 --2- 172.21.15.16:0/4036612204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff148101ce0 0x7ff148198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:51:57.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.993+0000 7ff14f1c0700 1 -- 172.21.15.16:0/4036612204 >> 172.21.15.16:0/4036612204 conn(0x7ff1480fc0a0 msgr2=0x7ff148073300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:51:57.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.993+0000 7ff14f1c0700 1 -- 172.21.15.16:0/4036612204 shutdown_connections 2024-04-02T04:51:57.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:51:57.993+0000 7ff14f1c0700 1 -- 172.21.15.16:0/4036612204 wait complete. 2024-04-02T04:51:58.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:58 smithi016 ceph-mon[93230]: pgmap v268: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:58.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:58 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/4036612204' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:51:58.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:58 smithi027 ceph-mon[83036]: pgmap v268: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:51:58.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:58 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/4036612204' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:51:59.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:51:59 smithi027 ceph-mon[83036]: from='client.34584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:51:59.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:51:59 smithi016 ceph-mon[93230]: from='client.34584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:52:00.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:00 smithi016 ceph-mon[93230]: pgmap v269: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:00.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:00 smithi027 ceph-mon[83036]: pgmap v269: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:01.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:01 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:52:01.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:01 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:52:01.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:01 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:52:01.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:01 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:52:02.421 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:02 smithi016 ceph-mon[93230]: pgmap v270: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:02.421 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:02 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:52:02.421 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:02 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:52:02.421 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:02 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:52:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:02 smithi027 ceph-mon[83036]: pgmap v270: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:02 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:52:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:02 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:52:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:02 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:52:04.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:04 smithi027 ceph-mon[83036]: pgmap v271: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:04.432 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:04 smithi016 ceph-mon[93230]: pgmap v271: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:06.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: pgmap v272: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:06.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:52:06.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:52:06.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:06.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:52:06.397 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:52:06.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-02T04:52:06.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-02T04:52:06.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:52:06.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:06.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:06.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:06.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:06.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:06.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:06.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:06.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:06.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:06.398 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:06.399 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:06.399 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:06.399 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:06.399 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:06.399 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:06.399 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:06.399 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:06.399 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:06.399 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: Upgrade: Finalizing container_image settings 2024-04-02T04:52:06.399 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:52:06.399 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-02T04:52:06.399 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-02T04:52:06.399 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:52:06.399 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-02T04:52:06.399 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-02T04:52:06.400 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-04-02T04:52:06.400 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-02T04:52:06.400 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-04-02T04:52:06.400 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-02T04:52:06.400 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-04-02T04:52:06.400 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-02T04:52:06.400 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-02T04:52:06.400 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-02T04:52:06.400 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-02T04:52:06.400 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:52:06.400 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-02T04:52:06.400 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-02T04:52:06.400 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-02T04:52:06.400 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-02T04:52:06.400 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-02T04:52:06.401 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-02T04:52:06.401 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-02T04:52:06.401 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-02T04:52:06.401 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:52:06.401 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:52:06.401 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:52:06.401 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:52:06.401 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:52:06.401 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:52:06.401 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: Upgrade: Complete! 2024-04-02T04:52:06.401 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-02T04:52:06.401 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-02T04:52:06.401 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:52:06.401 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:06.401 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:52:06.402 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:52:06.402 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:52:06.402 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:06.402 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:52:06.402 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:52:06.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: pgmap v272: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:06.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:52:06.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:52:06.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:06.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:52:06.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:52:06.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-02T04:52:06.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-02T04:52:06.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:52:06.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:06.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:06.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:06.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:06.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:06.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:06.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:06.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:06.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:06.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:06.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:06.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:06.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:06.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:06.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:06.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:06.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:06.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:06.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: Upgrade: Finalizing container_image settings 2024-04-02T04:52:06.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:52:06.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-02T04:52:06.563 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-02T04:52:06.563 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:52:06.563 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-02T04:52:06.563 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-02T04:52:06.563 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-04-02T04:52:06.563 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-02T04:52:06.563 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-04-02T04:52:06.563 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-02T04:52:06.563 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-04-02T04:52:06.563 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-02T04:52:06.563 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-02T04:52:06.563 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-02T04:52:06.563 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-02T04:52:06.563 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:52:06.563 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-02T04:52:06.564 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-02T04:52:06.564 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-02T04:52:06.564 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-02T04:52:06.564 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-02T04:52:06.564 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-02T04:52:06.564 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-02T04:52:06.564 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-02T04:52:06.564 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:52:06.564 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:52:06.564 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:52:06.564 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:52:06.564 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:52:06.564 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-02T04:52:06.564 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: Upgrade: Complete! 2024-04-02T04:52:06.564 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-02T04:52:06.565 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-02T04:52:06.565 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:52:06.565 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:06.565 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:52:06.565 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:52:06.565 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:52:06.565 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:52:06.565 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:52:06.565 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:52:08.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:08 smithi016 ceph-mon[93230]: pgmap v273: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:08.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:08 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:52:08.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:08 smithi027 ceph-mon[83036]: pgmap v273: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:08.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:08 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:52:10.547 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:10 smithi027 ceph-mon[83036]: pgmap v274: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:10.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:10 smithi016 ceph-mon[93230]: pgmap v274: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:12.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:12 smithi016 ceph-mon[93230]: pgmap v275: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:12.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:12 smithi027 ceph-mon[83036]: pgmap v275: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:14.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:14 smithi016 ceph-mon[93230]: pgmap v276: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:14.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:14 smithi027 ceph-mon[83036]: pgmap v276: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:16.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:16 smithi016 ceph-mon[93230]: pgmap v277: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:16.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:16 smithi027 ceph-mon[83036]: pgmap v277: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:17.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:17 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:52:17.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:17 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:52:18.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:18 smithi016 ceph-mon[93230]: pgmap v278: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:18.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:18 smithi027 ceph-mon[83036]: pgmap v278: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:20.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:20 smithi016 ceph-mon[93230]: pgmap v279: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:20.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:20 smithi027 ceph-mon[83036]: pgmap v279: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:22.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:22 smithi016 ceph-mon[93230]: pgmap v280: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:22.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:22 smithi027 ceph-mon[83036]: pgmap v280: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:24.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:24 smithi016 ceph-mon[93230]: pgmap v281: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:24.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:24 smithi027 ceph-mon[83036]: pgmap v281: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:26.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:26 smithi016 ceph-mon[93230]: pgmap v282: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:26.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:26 smithi027 ceph-mon[83036]: pgmap v282: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:28.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.105+0000 7fe9d475f700 1 -- 172.21.15.16:0/2485186412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9cc0feb70 msgr2=0x7fe9cc0fefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:28.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.105+0000 7fe9d475f700 1 --2- 172.21.15.16:0/2485186412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9cc0feb70 0x7fe9cc0fefd0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc009910 tx=0x7fe9bc009c20 comp rx=0 tx=0).stop 2024-04-02T04:52:28.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.106+0000 7fe9d475f700 1 -- 172.21.15.16:0/2485186412 shutdown_connections 2024-04-02T04:52:28.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.106+0000 7fe9d475f700 1 --2- 172.21.15.16:0/2485186412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9cc0feb70 0x7fe9cc0fefd0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:28.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.106+0000 7fe9d475f700 1 --2- 172.21.15.16:0/2485186412 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9cc1020f0 0x7fe9cc102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:28.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.106+0000 7fe9d475f700 1 -- 172.21.15.16:0/2485186412 >> 172.21.15.16:0/2485186412 conn(0x7fe9cc0fa730 msgr2=0x7fe9cc0fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:52:28.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.106+0000 7fe9d475f700 1 -- 172.21.15.16:0/2485186412 shutdown_connections 2024-04-02T04:52:28.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.106+0000 7fe9d475f700 1 -- 172.21.15.16:0/2485186412 wait complete. 2024-04-02T04:52:28.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.106+0000 7fe9d475f700 1 Processor -- start 2024-04-02T04:52:28.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.106+0000 7fe9d475f700 1 -- start start 2024-04-02T04:52:28.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.106+0000 7fe9d475f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9cc0feb70 0x7fe9cc196570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:52:28.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.106+0000 7fe9d475f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9cc1020f0 0x7fe9cc196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:52:28.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.106+0000 7fe9d475f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc1970d0 con 0x7fe9cc1020f0 2024-04-02T04:52:28.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.106+0000 7fe9d475f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc197210 con 0x7fe9cc0feb70 2024-04-02T04:52:28.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.107+0000 7fe9d1cfa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9cc1020f0 0x7fe9cc196ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:52:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.107+0000 7fe9d1cfa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9cc1020f0 0x7fe9cc196ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37258/0 (socket says 172.21.15.16:37258) 2024-04-02T04:52:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.107+0000 7fe9d1cfa700 1 -- 172.21.15.16:0/2191177774 learned_addr learned my addr 172.21.15.16:0/2191177774 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:52:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.107+0000 7fe9d24fb700 1 --2- 172.21.15.16:0/2191177774 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9cc0feb70 0x7fe9cc196570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:52:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.107+0000 7fe9d1cfa700 1 -- 172.21.15.16:0/2191177774 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9cc0feb70 msgr2=0x7fe9cc196570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.107+0000 7fe9d1cfa700 1 --2- 172.21.15.16:0/2191177774 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9cc0feb70 0x7fe9cc196570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.107+0000 7fe9d1cfa700 1 -- 172.21.15.16:0/2191177774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9bc0095f0 con 0x7fe9cc1020f0 2024-04-02T04:52:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.107+0000 7fe9d1cfa700 1 --2- 172.21.15.16:0/2191177774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9cc1020f0 0x7fe9cc196ab0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc005b40 tx=0x7fe9bc0055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:52:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.107+0000 7fe9c37fe700 1 -- 172.21.15.16:0/2191177774 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9bc01d070 con 0x7fe9cc1020f0 2024-04-02T04:52:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.107+0000 7fe9c37fe700 1 -- 172.21.15.16:0/2191177774 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe9bc005dc0 con 0x7fe9cc1020f0 2024-04-02T04:52:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.107+0000 7fe9d475f700 1 -- 172.21.15.16:0/2191177774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9cc19bbc0 con 0x7fe9cc1020f0 2024-04-02T04:52:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.107+0000 7fe9c37fe700 1 -- 172.21.15.16:0/2191177774 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9bc021410 con 0x7fe9cc1020f0 2024-04-02T04:52:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.107+0000 7fe9d475f700 1 -- 172.21.15.16:0/2191177774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9cc19c130 con 0x7fe9cc1020f0 2024-04-02T04:52:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.108+0000 7fe9d475f700 1 -- 172.21.15.16:0/2191177774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9cc1906a0 con 0x7fe9cc1020f0 2024-04-02T04:52:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.109+0000 7fe9c37fe700 1 -- 172.21.15.16:0/2191177774 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fe9bc00f460 con 0x7fe9cc1020f0 2024-04-02T04:52:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.110+0000 7fe9c37fe700 1 --2- 172.21.15.16:0/2191177774 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fe9b806dd70 0x7fe9b8070230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:52:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.110+0000 7fe9c37fe700 1 -- 172.21.15.16:0/2191177774 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fe9bc08ec60 con 0x7fe9cc1020f0 2024-04-02T04:52:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.110+0000 7fe9d24fb700 1 --2- 172.21.15.16:0/2191177774 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fe9b806dd70 0x7fe9b8070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:52:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.110+0000 7fe9d24fb700 1 --2- 172.21.15.16:0/2191177774 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fe9b806dd70 0x7fe9b8070230 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc068d10 tx=0x7fe9c800b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:52:28.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.114+0000 7fe9c37fe700 1 -- 172.21.15.16:0/2191177774 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe9bc05bd00 con 0x7fe9cc1020f0 2024-04-02T04:52:28.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.337+0000 7fe9d475f700 1 -- 172.21.15.16:0/2191177774 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe9cc02cdf0 con 0x7fe9b806dd70 2024-04-02T04:52:28.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.339+0000 7fe9c37fe700 1 -- 172.21.15.16:0/2191177774 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fe9cc02cdf0 con 0x7fe9b806dd70 2024-04-02T04:52:28.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.340+0000 7fe9d475f700 1 -- 172.21.15.16:0/2191177774 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fe9b806dd70 msgr2=0x7fe9b8070230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:28.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.340+0000 7fe9d475f700 1 --2- 172.21.15.16:0/2191177774 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fe9b806dd70 0x7fe9b8070230 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc068d10 tx=0x7fe9c800b410 comp rx=0 tx=0).stop 2024-04-02T04:52:28.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.341+0000 7fe9d475f700 1 -- 172.21.15.16:0/2191177774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9cc1020f0 msgr2=0x7fe9cc196ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:28.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.341+0000 7fe9d475f700 1 --2- 172.21.15.16:0/2191177774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9cc1020f0 0x7fe9cc196ab0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc005b40 tx=0x7fe9bc0055f0 comp rx=0 tx=0).stop 2024-04-02T04:52:28.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.341+0000 7fe9d475f700 1 -- 172.21.15.16:0/2191177774 shutdown_connections 2024-04-02T04:52:28.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.341+0000 7fe9d475f700 1 --2- 172.21.15.16:0/2191177774 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fe9b806dd70 0x7fe9b8070230 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:28.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.341+0000 7fe9d475f700 1 --2- 172.21.15.16:0/2191177774 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9cc0feb70 0x7fe9cc196570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:28.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.341+0000 7fe9d475f700 1 --2- 172.21.15.16:0/2191177774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9cc1020f0 0x7fe9cc196ab0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:28.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.341+0000 7fe9d475f700 1 -- 172.21.15.16:0/2191177774 >> 172.21.15.16:0/2191177774 conn(0x7fe9cc0fa730 msgr2=0x7fe9cc105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:52:28.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.341+0000 7fe9d475f700 1 -- 172.21.15.16:0/2191177774 shutdown_connections 2024-04-02T04:52:28.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:28.341+0000 7fe9d475f700 1 -- 172.21.15.16:0/2191177774 wait complete. 2024-04-02T04:52:28.381 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:28 smithi016 ceph-mon[93230]: pgmap v283: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:28.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:28 smithi027 ceph-mon[83036]: pgmap v283: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:29.064 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'ceph orch ps' 2024-04-02T04:52:29.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:29 smithi016 ceph-mon[93230]: from='client.34592 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:52:29.319 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:52:29.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:29 smithi027 ceph-mon[83036]: from='client.34592 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:52:30.423 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:30 smithi016 ceph-mon[93230]: pgmap v284: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:30.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:30 smithi027 ceph-mon[83036]: pgmap v284: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:30.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:30.928+0000 7fde5f8d1700 1 -- 172.21.15.16:0/438507905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde58100a70 msgr2=0x7fde58100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:30.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:30.928+0000 7fde5f8d1700 1 --2- 172.21.15.16:0/438507905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde58100a70 0x7fde58100ef0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fde48009910 tx=0x7fde48009c20 comp rx=0 tx=0).stop 2024-04-02T04:52:30.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:30.928+0000 7fde5f8d1700 1 -- 172.21.15.16:0/438507905 shutdown_connections 2024-04-02T04:52:30.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:30.928+0000 7fde5f8d1700 1 --2- 172.21.15.16:0/438507905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde58100a70 0x7fde58100ef0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:30.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:30.928+0000 7fde5f8d1700 1 --2- 172.21.15.16:0/438507905 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde58106a70 0x7fde58106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:30.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:30.928+0000 7fde5f8d1700 1 -- 172.21.15.16:0/438507905 >> 172.21.15.16:0/438507905 conn(0x7fde580fc5a0 msgr2=0x7fde580fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:52:30.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:30.928+0000 7fde5f8d1700 1 -- 172.21.15.16:0/438507905 shutdown_connections 2024-04-02T04:52:30.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:30.928+0000 7fde5f8d1700 1 -- 172.21.15.16:0/438507905 wait complete. 2024-04-02T04:52:30.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:30.929+0000 7fde5f8d1700 1 Processor -- start 2024-04-02T04:52:30.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:30.929+0000 7fde5f8d1700 1 -- start start 2024-04-02T04:52:30.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:30.929+0000 7fde5f8d1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde58100a70 0x7fde5819c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:52:30.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:30.929+0000 7fde5f8d1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde58106a70 0x7fde5819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:52:30.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:30.929+0000 7fde5f8d1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5819d260 con 0x7fde58106a70 2024-04-02T04:52:30.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:30.929+0000 7fde5f8d1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde581967d0 con 0x7fde58100a70 2024-04-02T04:52:30.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:30.930+0000 7fde5ce6c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde58106a70 0x7fde5819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:52:30.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:30.930+0000 7fde5ce6c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde58106a70 0x7fde5819cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37278/0 (socket says 172.21.15.16:37278) 2024-04-02T04:52:30.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:30.930+0000 7fde5ce6c700 1 -- 172.21.15.16:0/2859124042 learned_addr learned my addr 172.21.15.16:0/2859124042 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:52:30.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:30.930+0000 7fde5d66d700 1 --2- 172.21.15.16:0/2859124042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde58100a70 0x7fde5819c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:52:30.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:30.930+0000 7fde5ce6c700 1 -- 172.21.15.16:0/2859124042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde58100a70 msgr2=0x7fde5819c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:30.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:30.930+0000 7fde5ce6c700 1 --2- 172.21.15.16:0/2859124042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde58100a70 0x7fde5819c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:30.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:30.930+0000 7fde5ce6c700 1 -- 172.21.15.16:0/2859124042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde480095f0 con 0x7fde58106a70 2024-04-02T04:52:30.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:30.930+0000 7fde5ce6c700 1 --2- 172.21.15.16:0/2859124042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde58106a70 0x7fde5819cbd0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fde48006010 tx=0x7fde4800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:52:30.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:30.930+0000 7fde4e7fc700 1 -- 172.21.15.16:0/2859124042 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde4801d070 con 0x7fde58106a70 2024-04-02T04:52:30.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:30.930+0000 7fde4e7fc700 1 -- 172.21.15.16:0/2859124042 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde480056e0 con 0x7fde58106a70 2024-04-02T04:52:30.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:30.930+0000 7fde5f8d1700 1 -- 172.21.15.16:0/2859124042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde58196a50 con 0x7fde58106a70 2024-04-02T04:52:30.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:30.930+0000 7fde4e7fc700 1 -- 172.21.15.16:0/2859124042 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde4800fdf0 con 0x7fde58106a70 2024-04-02T04:52:30.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:30.930+0000 7fde5f8d1700 1 -- 172.21.15.16:0/2859124042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde58196f40 con 0x7fde58106a70 2024-04-02T04:52:30.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:30.931+0000 7fde5f8d1700 1 -- 172.21.15.16:0/2859124042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde58109090 con 0x7fde58106a70 2024-04-02T04:52:30.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:30.932+0000 7fde4e7fc700 1 -- 172.21.15.16:0/2859124042 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fde4800f5f0 con 0x7fde58106a70 2024-04-02T04:52:30.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:30.933+0000 7fde4e7fc700 1 --2- 172.21.15.16:0/2859124042 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fde4406dd60 0x7fde44070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:52:30.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:30.933+0000 7fde4e7fc700 1 -- 172.21.15.16:0/2859124042 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fde4808ede0 con 0x7fde58106a70 2024-04-02T04:52:30.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:30.933+0000 7fde5d66d700 1 --2- 172.21.15.16:0/2859124042 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fde4406dd60 0x7fde44070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:52:30.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:30.933+0000 7fde5d66d700 1 --2- 172.21.15.16:0/2859124042 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fde4406dd60 0x7fde44070220 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fde54005950 tx=0x7fde5400b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:52:30.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:30.937+0000 7fde4e7fc700 1 -- 172.21.15.16:0/2859124042 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fde4805bf40 con 0x7fde58106a70 2024-04-02T04:52:31.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:31.152+0000 7fde5f8d1700 1 -- 172.21.15.16:0/2859124042 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fde58197d30 con 0x7fde4406dd60 2024-04-02T04:52:31.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:31.163+0000 7fde4e7fc700 1 -- 172.21.15.16:0/2859124042 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fde58197d30 con 0x7fde4406dd60 2024-04-02T04:52:31.166 INFO:teuthology.orchestra.run.smithi016.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:52:31.167 INFO:teuthology.orchestra.run.smithi016.stdout:alertmanager.smithi016 smithi016 *:9093,9094 running (60s) 30s ago 18m 16.2M - 0.25.0 c8568f914cd2 6c7b91689a07 2024-04-02T04:52:31.167 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi016 smithi016 running (118s) 30s ago 18m 7952k - 19.0.0-1979-g6cec793d 78f830190839 1d3ba1f157b9 2024-04-02T04:52:31.167 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi027 smithi027 running (113s) 91s ago 17m 7558k - 19.0.0-1979-g6cec793d 78f830190839 3402997325e5 2024-04-02T04:52:31.167 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi016 smithi016 running (7m) 30s ago 18m 6660k - 19.0.0-1979-g6cec793d 78f830190839 dc63ce0eef58 2024-04-02T04:52:31.167 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi027 smithi027 running (6m) 91s ago 17m 6665k - 19.0.0-1979-g6cec793d 78f830190839 2beb077018e4 2024-04-02T04:52:31.167 INFO:teuthology.orchestra.run.smithi016.stdout:grafana.smithi016 smithi016 *:3000 running (36s) 30s ago 18m 54.4M - 9.4.12 2bacad6d85d8 923c1f900a27 2024-04-02T04:52:31.167 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.fyzrnm smithi016 running (3m) 30s ago 12m 16.7M - 19.0.0-1979-g6cec793d 78f830190839 98f0a83c6631 2024-04-02T04:52:31.167 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.uhgkgg smithi016 running (3m) 30s ago 12m 89.8M - 19.0.0-1979-g6cec793d 78f830190839 19be3989a243 2024-04-02T04:52:31.167 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.sdublg smithi027 running (2m) 91s ago 12m 12.8M - 19.0.0-1979-g6cec793d 78f830190839 7c307783baab 2024-04-02T04:52:31.167 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.zwrfoo smithi027 running (2m) 91s ago 12m 13.6M - 19.0.0-1979-g6cec793d 78f830190839 acc6430302ac 2024-04-02T04:52:31.167 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi016.eqnebx smithi016 *:8443,9283,8765 running (9m) 30s ago 19m 520M - 19.0.0-1979-g6cec793d 78f830190839 d61ed91b4577 2024-04-02T04:52:31.167 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi027.kjzcsc smithi027 *:8443,9283,8765 running (8m) 91s ago 17m 451M - 19.0.0-1979-g6cec793d 78f830190839 5072a5ffc761 2024-04-02T04:52:31.167 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi016 smithi016 running (8m) 30s ago 20m 56.0M 2048M 19.0.0-1979-g6cec793d 78f830190839 eabfc84f1cdf 2024-04-02T04:52:31.167 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi027 smithi027 running (7m) 91s ago 17m 44.0M 2048M 19.0.0-1979-g6cec793d 78f830190839 fcd68b20dcc5 2024-04-02T04:52:31.167 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi016 smithi016 *:9100 running (99s) 30s ago 18m 8476k - 1.5.0 0da6a335fe13 f3e285b06d71 2024-04-02T04:52:31.168 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi027 smithi027 *:9100 running (94s) 91s ago 17m 3649k - 1.5.0 0da6a335fe13 c8e26f58ca63 2024-04-02T04:52:31.168 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 smithi016 running (6m) 30s ago 16m 91.5M 1217M 19.0.0-1979-g6cec793d 78f830190839 421d9e1cc751 2024-04-02T04:52:31.168 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 smithi016 running (5m) 30s ago 16m 100M 1217M 19.0.0-1979-g6cec793d 78f830190839 3612a56236a3 2024-04-02T04:52:31.168 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 smithi016 running (5m) 30s ago 15m 91.9M 1217M 19.0.0-1979-g6cec793d 78f830190839 a8e9b8d0303c 2024-04-02T04:52:31.168 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 smithi027 running (4m) 91s ago 15m 112M 2240M 19.0.0-1979-g6cec793d 78f830190839 0dd13b5be8c0 2024-04-02T04:52:31.168 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 smithi027 running (4m) 91s ago 14m 89.3M 2240M 19.0.0-1979-g6cec793d 78f830190839 9025eacf8145 2024-04-02T04:52:31.168 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 smithi027 running (3m) 91s ago 14m 88.2M 2240M 19.0.0-1979-g6cec793d 78f830190839 c2787f157377 2024-04-02T04:52:31.168 INFO:teuthology.orchestra.run.smithi016.stdout:prometheus.smithi016 smithi016 *:9095 running (77s) 30s ago 17m 46.2M - 2.43.0 a07b618ecd1d 042152fa1836 2024-04-02T04:52:31.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:31.165+0000 7fde5f8d1700 1 -- 172.21.15.16:0/2859124042 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fde4406dd60 msgr2=0x7fde44070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:31.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:31.165+0000 7fde5f8d1700 1 --2- 172.21.15.16:0/2859124042 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fde4406dd60 0x7fde44070220 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fde54005950 tx=0x7fde5400b410 comp rx=0 tx=0).stop 2024-04-02T04:52:31.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:31.165+0000 7fde5f8d1700 1 -- 172.21.15.16:0/2859124042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde58106a70 msgr2=0x7fde5819cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:31.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:31.165+0000 7fde5f8d1700 1 --2- 172.21.15.16:0/2859124042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde58106a70 0x7fde5819cbd0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fde48006010 tx=0x7fde4800bd70 comp rx=0 tx=0).stop 2024-04-02T04:52:31.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:31.165+0000 7fde5f8d1700 1 -- 172.21.15.16:0/2859124042 shutdown_connections 2024-04-02T04:52:31.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:31.165+0000 7fde5f8d1700 1 --2- 172.21.15.16:0/2859124042 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fde4406dd60 0x7fde44070220 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:31.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:31.166+0000 7fde5f8d1700 1 --2- 172.21.15.16:0/2859124042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde58100a70 0x7fde5819c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:31.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:31.166+0000 7fde5f8d1700 1 --2- 172.21.15.16:0/2859124042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde58106a70 0x7fde5819cbd0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:31.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:31.166+0000 7fde5f8d1700 1 -- 172.21.15.16:0/2859124042 >> 172.21.15.16:0/2859124042 conn(0x7fde580fc5a0 msgr2=0x7fde580fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:52:31.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:31.166+0000 7fde5f8d1700 1 -- 172.21.15.16:0/2859124042 shutdown_connections 2024-04-02T04:52:31.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:31.166+0000 7fde5f8d1700 1 -- 172.21.15.16:0/2859124042 wait complete. 2024-04-02T04:52:31.811 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'ceph orch upgrade status' 2024-04-02T04:52:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:52:32.493 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:32 smithi016 ceph-mon[93230]: pgmap v285: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:32.493 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:32 smithi016 ceph-mon[93230]: from='client.34596 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:52:32.493 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:32 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:52:32.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:32 smithi027 ceph-mon[83036]: pgmap v285: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:32.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:32 smithi027 ceph-mon[83036]: from='client.34596 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:52:32.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:32 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:52:33.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.609+0000 7f9c2b226700 1 -- 172.21.15.16:0/951421932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c24101ba0 msgr2=0x7f9c24101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:33.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.609+0000 7f9c2b226700 1 --2- 172.21.15.16:0/951421932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c24101ba0 0x7f9c24101f80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9c18009910 tx=0x7f9c18009c20 comp rx=0 tx=0).stop 2024-04-02T04:52:33.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.609+0000 7f9c2b226700 1 -- 172.21.15.16:0/951421932 shutdown_connections 2024-04-02T04:52:33.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.609+0000 7f9c2b226700 1 --2- 172.21.15.16:0/951421932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c24102550 0x7f9c2410aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:33.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.609+0000 7f9c2b226700 1 --2- 172.21.15.16:0/951421932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c24101ba0 0x7f9c24101f80 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:33.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.609+0000 7f9c2b226700 1 -- 172.21.15.16:0/951421932 >> 172.21.15.16:0/951421932 conn(0x7f9c240fb480 msgr2=0x7f9c240fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:52:33.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.609+0000 7f9c2b226700 1 -- 172.21.15.16:0/951421932 shutdown_connections 2024-04-02T04:52:33.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.609+0000 7f9c2b226700 1 -- 172.21.15.16:0/951421932 wait complete. 2024-04-02T04:52:33.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.609+0000 7f9c2b226700 1 Processor -- start 2024-04-02T04:52:33.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.610+0000 7f9c2b226700 1 -- start start 2024-04-02T04:52:33.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.610+0000 7f9c2b226700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c24101ba0 0x7f9c240ff890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:52:33.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.610+0000 7f9c2b226700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c24102550 0x7f9c240ffdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:52:33.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.610+0000 7f9c2b226700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c24100310 con 0x7f9c24101ba0 2024-04-02T04:52:33.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.610+0000 7f9c2b226700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c24100450 con 0x7f9c24102550 2024-04-02T04:52:33.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.610+0000 7f9c28fc2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c24101ba0 0x7f9c240ff890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:52:33.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.610+0000 7f9c28fc2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c24101ba0 0x7f9c240ff890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51880/0 (socket says 172.21.15.16:51880) 2024-04-02T04:52:33.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.610+0000 7f9c28fc2700 1 -- 172.21.15.16:0/2063882874 learned_addr learned my addr 172.21.15.16:0/2063882874 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:52:33.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.610+0000 7f9c23fff700 1 --2- 172.21.15.16:0/2063882874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c24102550 0x7f9c240ffdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:52:33.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.610+0000 7f9c28fc2700 1 -- 172.21.15.16:0/2063882874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c24102550 msgr2=0x7f9c240ffdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:33.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.610+0000 7f9c28fc2700 1 --2- 172.21.15.16:0/2063882874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c24102550 0x7f9c240ffdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:33.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.610+0000 7f9c28fc2700 1 -- 172.21.15.16:0/2063882874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c180095f0 con 0x7f9c24101ba0 2024-04-02T04:52:33.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.610+0000 7f9c23fff700 1 --2- 172.21.15.16:0/2063882874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c24102550 0x7f9c240ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T04:52:33.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.610+0000 7f9c28fc2700 1 --2- 172.21.15.16:0/2063882874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c24101ba0 0x7f9c240ff890 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9c18006010 tx=0x7f9c1800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:52:33.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.611+0000 7f9c21ffb700 1 -- 172.21.15.16:0/2063882874 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c1801d070 con 0x7f9c24101ba0 2024-04-02T04:52:33.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.611+0000 7f9c21ffb700 1 -- 172.21.15.16:0/2063882874 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9c180056e0 con 0x7f9c24101ba0 2024-04-02T04:52:33.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.611+0000 7f9c21ffb700 1 -- 172.21.15.16:0/2063882874 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c18017400 con 0x7f9c24101ba0 2024-04-02T04:52:33.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.611+0000 7f9c2b226700 1 -- 172.21.15.16:0/2063882874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c241006d0 con 0x7f9c24101ba0 2024-04-02T04:52:33.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.611+0000 7f9c2b226700 1 -- 172.21.15.16:0/2063882874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c241a2dd0 con 0x7f9c24101ba0 2024-04-02T04:52:33.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.613+0000 7f9c2b226700 1 -- 172.21.15.16:0/2063882874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c240695f0 con 0x7f9c24101ba0 2024-04-02T04:52:33.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.613+0000 7f9c21ffb700 1 -- 172.21.15.16:0/2063882874 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f9c1800f460 con 0x7f9c24101ba0 2024-04-02T04:52:33.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.613+0000 7f9c21ffb700 1 --2- 172.21.15.16:0/2063882874 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f9c1406db60 0x7f9c14070020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:52:33.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.614+0000 7f9c21ffb700 1 -- 172.21.15.16:0/2063882874 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f9c1808f020 con 0x7f9c24101ba0 2024-04-02T04:52:33.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.614+0000 7f9c23fff700 1 --2- 172.21.15.16:0/2063882874 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f9c1406db60 0x7f9c14070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:52:33.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.614+0000 7f9c23fff700 1 --2- 172.21.15.16:0/2063882874 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f9c1406db60 0x7f9c14070020 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9c10005950 tx=0x7f9c100058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:52:33.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.618+0000 7f9c21ffb700 1 -- 172.21.15.16:0/2063882874 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9c18058510 con 0x7f9c24101ba0 2024-04-02T04:52:33.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.841+0000 7f9c2b226700 1 -- 172.21.15.16:0/2063882874 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c241052e0 con 0x7f9c1406db60 2024-04-02T04:52:33.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.842+0000 7f9c21ffb700 1 -- 172.21.15.16:0/2063882874 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f9c241052e0 con 0x7f9c1406db60 2024-04-02T04:52:33.846 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:52:33.846 INFO:teuthology.orchestra.run.smithi016.stdout: "target_image": null, 2024-04-02T04:52:33.846 INFO:teuthology.orchestra.run.smithi016.stdout: "in_progress": false, 2024-04-02T04:52:33.846 INFO:teuthology.orchestra.run.smithi016.stdout: "which": "", 2024-04-02T04:52:33.846 INFO:teuthology.orchestra.run.smithi016.stdout: "services_complete": [], 2024-04-02T04:52:33.846 INFO:teuthology.orchestra.run.smithi016.stdout: "progress": null, 2024-04-02T04:52:33.846 INFO:teuthology.orchestra.run.smithi016.stdout: "message": "", 2024-04-02T04:52:33.846 INFO:teuthology.orchestra.run.smithi016.stdout: "is_paused": false 2024-04-02T04:52:33.846 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:52:33.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.844+0000 7f9c2b226700 1 -- 172.21.15.16:0/2063882874 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f9c1406db60 msgr2=0x7f9c14070020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:33.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.844+0000 7f9c2b226700 1 --2- 172.21.15.16:0/2063882874 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f9c1406db60 0x7f9c14070020 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9c10005950 tx=0x7f9c100058e0 comp rx=0 tx=0).stop 2024-04-02T04:52:33.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.844+0000 7f9c2b226700 1 -- 172.21.15.16:0/2063882874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c24101ba0 msgr2=0x7f9c240ff890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:33.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.844+0000 7f9c2b226700 1 --2- 172.21.15.16:0/2063882874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c24101ba0 0x7f9c240ff890 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9c18006010 tx=0x7f9c1800be20 comp rx=0 tx=0).stop 2024-04-02T04:52:33.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.844+0000 7f9c2b226700 1 -- 172.21.15.16:0/2063882874 shutdown_connections 2024-04-02T04:52:33.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.844+0000 7f9c2b226700 1 --2- 172.21.15.16:0/2063882874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c24101ba0 0x7f9c240ff890 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:33.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.844+0000 7f9c2b226700 1 --2- 172.21.15.16:0/2063882874 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f9c1406db60 0x7f9c14070020 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:33.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.844+0000 7f9c2b226700 1 --2- 172.21.15.16:0/2063882874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c24102550 0x7f9c240ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:33.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.844+0000 7f9c2b226700 1 -- 172.21.15.16:0/2063882874 >> 172.21.15.16:0/2063882874 conn(0x7f9c240fb480 msgr2=0x7f9c24102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:52:33.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.844+0000 7f9c2b226700 1 -- 172.21.15.16:0/2063882874 shutdown_connections 2024-04-02T04:52:33.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:33.845+0000 7f9c2b226700 1 -- 172.21.15.16:0/2063882874 wait complete. 2024-04-02T04:52:34.478 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:34 smithi016 ceph-mon[93230]: pgmap v286: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:34.541 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'ceph health detail' 2024-04-02T04:52:34.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:34 smithi027 ceph-mon[83036]: pgmap v286: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:34.762 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:52:35.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:35 smithi016 ceph-mon[93230]: from='client.34600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:52:35.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:35 smithi027 ceph-mon[83036]: from='client.34600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:52:36.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.340+0000 7f20de31e700 1 -- 172.21.15.16:0/1052657635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d80ff950 msgr2=0x7f20d80ffdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:36.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.340+0000 7f20de31e700 1 --2- 172.21.15.16:0/1052657635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d80ff950 0x7f20d80ffdd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f20c8009910 tx=0x7f20c8009c20 comp rx=0 tx=0).stop 2024-04-02T04:52:36.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.340+0000 7f20de31e700 1 -- 172.21.15.16:0/1052657635 shutdown_connections 2024-04-02T04:52:36.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.340+0000 7f20de31e700 1 --2- 172.21.15.16:0/1052657635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d80ff950 0x7f20d80ffdd0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:36.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.340+0000 7f20de31e700 1 --2- 172.21.15.16:0/1052657635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20d8105950 0x7f20d8105d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:36.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.340+0000 7f20de31e700 1 -- 172.21.15.16:0/1052657635 >> 172.21.15.16:0/1052657635 conn(0x7f20d8076640 msgr2=0x7f20d80fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:52:36.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.340+0000 7f20de31e700 1 -- 172.21.15.16:0/1052657635 shutdown_connections 2024-04-02T04:52:36.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.340+0000 7f20de31e700 1 -- 172.21.15.16:0/1052657635 wait complete. 2024-04-02T04:52:36.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.341+0000 7f20de31e700 1 Processor -- start 2024-04-02T04:52:36.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.341+0000 7f20de31e700 1 -- start start 2024-04-02T04:52:36.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.341+0000 7f20de31e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d80ff950 0x7f20d81977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:52:36.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.341+0000 7f20de31e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20d8105950 0x7f20d8197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:52:36.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.341+0000 7f20de31e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d81982e0 con 0x7f20d80ff950 2024-04-02T04:52:36.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.341+0000 7f20de31e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d8198420 con 0x7f20d8105950 2024-04-02T04:52:36.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.341+0000 7f20d7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d80ff950 0x7f20d81977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:52:36.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.341+0000 7f20d7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d80ff950 0x7f20d81977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51906/0 (socket says 172.21.15.16:51906) 2024-04-02T04:52:36.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.341+0000 7f20d7fff700 1 -- 172.21.15.16:0/1357998869 learned_addr learned my addr 172.21.15.16:0/1357998869 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:52:36.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.341+0000 7f20d77fe700 1 --2- 172.21.15.16:0/1357998869 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20d8105950 0x7f20d8197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:52:36.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.342+0000 7f20d7fff700 1 -- 172.21.15.16:0/1357998869 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20d8105950 msgr2=0x7f20d8197d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:36.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.342+0000 7f20d7fff700 1 --2- 172.21.15.16:0/1357998869 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20d8105950 0x7f20d8197d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:36.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.342+0000 7f20d7fff700 1 -- 172.21.15.16:0/1357998869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c80095f0 con 0x7f20d80ff950 2024-04-02T04:52:36.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.342+0000 7f20d7fff700 1 --2- 172.21.15.16:0/1357998869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d80ff950 0x7f20d81977d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f20cc00e920 tx=0x7f20cc00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:52:36.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.342+0000 7f20d57fa700 1 -- 172.21.15.16:0/1357998869 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20cc00c9d0 con 0x7f20d80ff950 2024-04-02T04:52:36.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.342+0000 7f20de31e700 1 -- 172.21.15.16:0/1357998869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d81a0840 con 0x7f20d80ff950 2024-04-02T04:52:36.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.342+0000 7f20d57fa700 1 -- 172.21.15.16:0/1357998869 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f20cc010430 con 0x7f20d80ff950 2024-04-02T04:52:36.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.342+0000 7f20d57fa700 1 -- 172.21.15.16:0/1357998869 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20cc018630 con 0x7f20d80ff950 2024-04-02T04:52:36.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.342+0000 7f20de31e700 1 -- 172.21.15.16:0/1357998869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20d81a0c50 con 0x7f20d80ff950 2024-04-02T04:52:36.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.343+0000 7f20de31e700 1 -- 172.21.15.16:0/1357998869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20d8107f70 con 0x7f20d80ff950 2024-04-02T04:52:36.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.344+0000 7f20d57fa700 1 -- 172.21.15.16:0/1357998869 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f20cc0105a0 con 0x7f20d80ff950 2024-04-02T04:52:36.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.344+0000 7f20d57fa700 1 --2- 172.21.15.16:0/1357998869 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f20c006dd60 0x7f20c0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:52:36.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.344+0000 7f20d57fa700 1 -- 172.21.15.16:0/1357998869 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f20cc014070 con 0x7f20d80ff950 2024-04-02T04:52:36.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.344+0000 7f20d77fe700 1 --2- 172.21.15.16:0/1357998869 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f20c006dd60 0x7f20c0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:52:36.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.345+0000 7f20d77fe700 1 --2- 172.21.15.16:0/1357998869 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f20c006dd60 0x7f20c0070220 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f20d8068c70 tx=0x7f20c8005940 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:52:36.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.349+0000 7f20d57fa700 1 -- 172.21.15.16:0/1357998869 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f20cc053280 con 0x7f20d80ff950 2024-04-02T04:52:36.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:36 smithi016 ceph-mon[93230]: pgmap v287: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.638+0000 7f20de31e700 1 -- 172.21.15.16:0/1357998869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f20d8068ac0 con 0x7f20d80ff950 2024-04-02T04:52:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.639+0000 7f20d57fa700 1 -- 172.21.15.16:0/1357998869 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f20cc057380 con 0x7f20d80ff950 2024-04-02T04:52:36.642 INFO:teuthology.orchestra.run.smithi016.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-02T04:52:36.642 INFO:teuthology.orchestra.run.smithi016.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-02T04:52:36.642 INFO:teuthology.orchestra.run.smithi016.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-02T04:52:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.641+0000 7f20de31e700 1 -- 172.21.15.16:0/1357998869 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f20c006dd60 msgr2=0x7f20c0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.641+0000 7f20de31e700 1 --2- 172.21.15.16:0/1357998869 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f20c006dd60 0x7f20c0070220 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f20d8068c70 tx=0x7f20c8005940 comp rx=0 tx=0).stop 2024-04-02T04:52:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.641+0000 7f20de31e700 1 -- 172.21.15.16:0/1357998869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d80ff950 msgr2=0x7f20d81977d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:36.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.641+0000 7f20de31e700 1 --2- 172.21.15.16:0/1357998869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d80ff950 0x7f20d81977d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f20cc00e920 tx=0x7f20cc00ec30 comp rx=0 tx=0).stop 2024-04-02T04:52:36.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.641+0000 7f20de31e700 1 -- 172.21.15.16:0/1357998869 shutdown_connections 2024-04-02T04:52:36.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.641+0000 7f20de31e700 1 --2- 172.21.15.16:0/1357998869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d80ff950 0x7f20d81977d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:36.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.641+0000 7f20de31e700 1 --2- 172.21.15.16:0/1357998869 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f20c006dd60 0x7f20c0070220 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:36.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.641+0000 7f20de31e700 1 --2- 172.21.15.16:0/1357998869 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20d8105950 0x7f20d8197d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:36.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.641+0000 7f20de31e700 1 -- 172.21.15.16:0/1357998869 >> 172.21.15.16:0/1357998869 conn(0x7f20d8076640 msgr2=0x7f20d80fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:52:36.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.641+0000 7f20de31e700 1 -- 172.21.15.16:0/1357998869 shutdown_connections 2024-04-02T04:52:36.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:36.641+0000 7f20de31e700 1 -- 172.21.15.16:0/1357998869 wait complete. 2024-04-02T04:52:36.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:36 smithi027 ceph-mon[83036]: pgmap v287: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:37.320 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'ceph versions' 2024-04-02T04:52:37.536 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:52:37.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:37 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/1357998869' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:52:37.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:37 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/1357998869' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-02T04:52:38.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:38 smithi016 ceph-mon[93230]: pgmap v288: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:38.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:38 smithi027 ceph-mon[83036]: pgmap v288: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:39.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.112+0000 7f3cc3eee700 1 -- 172.21.15.16:0/1111937802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cbc100a70 msgr2=0x7f3cbc100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:39.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.112+0000 7f3cc3eee700 1 --2- 172.21.15.16:0/1111937802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cbc100a70 0x7f3cbc100ef0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3cac009910 tx=0x7f3cac009c20 comp rx=0 tx=0).stop 2024-04-02T04:52:39.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.112+0000 7f3cc3eee700 1 -- 172.21.15.16:0/1111937802 shutdown_connections 2024-04-02T04:52:39.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.112+0000 7f3cc3eee700 1 --2- 172.21.15.16:0/1111937802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cbc100a70 0x7f3cbc100ef0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:39.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.112+0000 7f3cc3eee700 1 --2- 172.21.15.16:0/1111937802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cbc106a70 0x7f3cbc106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:39.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.112+0000 7f3cc3eee700 1 -- 172.21.15.16:0/1111937802 >> 172.21.15.16:0/1111937802 conn(0x7f3cbc0fc580 msgr2=0x7f3cbc0fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:52:39.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.112+0000 7f3cc3eee700 1 -- 172.21.15.16:0/1111937802 shutdown_connections 2024-04-02T04:52:39.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.112+0000 7f3cc3eee700 1 -- 172.21.15.16:0/1111937802 wait complete. 2024-04-02T04:52:39.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.113+0000 7f3cc3eee700 1 Processor -- start 2024-04-02T04:52:39.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.113+0000 7f3cc3eee700 1 -- start start 2024-04-02T04:52:39.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.113+0000 7f3cc3eee700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cbc100a70 0x7f3cbc19c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:52:39.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.113+0000 7f3cc3eee700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cbc106a70 0x7f3cbc19cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:52:39.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.113+0000 7f3cc3eee700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cbc19d250 con 0x7f3cbc106a70 2024-04-02T04:52:39.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.113+0000 7f3cc3eee700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cbc196810 con 0x7f3cbc100a70 2024-04-02T04:52:39.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.113+0000 7f3cc1489700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cbc106a70 0x7f3cbc19cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:52:39.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.113+0000 7f3cc1489700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cbc106a70 0x7f3cbc19cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51912/0 (socket says 172.21.15.16:51912) 2024-04-02T04:52:39.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.113+0000 7f3cc1489700 1 -- 172.21.15.16:0/3976731280 learned_addr learned my addr 172.21.15.16:0/3976731280 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:52:39.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.113+0000 7f3cc1c8a700 1 --2- 172.21.15.16:0/3976731280 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cbc100a70 0x7f3cbc19c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:52:39.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.113+0000 7f3cc1489700 1 -- 172.21.15.16:0/3976731280 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cbc100a70 msgr2=0x7f3cbc19c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:39.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.113+0000 7f3cc1489700 1 --2- 172.21.15.16:0/3976731280 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cbc100a70 0x7f3cbc19c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:39.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.113+0000 7f3cc1489700 1 -- 172.21.15.16:0/3976731280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cac0095f0 con 0x7f3cbc106a70 2024-04-02T04:52:39.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.114+0000 7f3cc1489700 1 --2- 172.21.15.16:0/3976731280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cbc106a70 0x7f3cbc19cbc0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3cac006010 tx=0x7f3cac00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:52:39.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.114+0000 7f3cb2ffd700 1 -- 172.21.15.16:0/3976731280 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cac01d070 con 0x7f3cbc106a70 2024-04-02T04:52:39.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.114+0000 7f3cb2ffd700 1 -- 172.21.15.16:0/3976731280 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3cac0056e0 con 0x7f3cbc106a70 2024-04-02T04:52:39.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.114+0000 7f3cc3eee700 1 -- 172.21.15.16:0/3976731280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cbc196a90 con 0x7f3cbc106a70 2024-04-02T04:52:39.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.114+0000 7f3cb2ffd700 1 -- 172.21.15.16:0/3976731280 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cac00fdf0 con 0x7f3cbc106a70 2024-04-02T04:52:39.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.114+0000 7f3cc3eee700 1 -- 172.21.15.16:0/3976731280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cbc196f80 con 0x7f3cbc106a70 2024-04-02T04:52:39.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.115+0000 7f3cc3eee700 1 -- 172.21.15.16:0/3976731280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cbc109090 con 0x7f3cbc106a70 2024-04-02T04:52:39.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.116+0000 7f3cb2ffd700 1 -- 172.21.15.16:0/3976731280 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f3cac00f5f0 con 0x7f3cbc106a70 2024-04-02T04:52:39.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.116+0000 7f3cb2ffd700 1 --2- 172.21.15.16:0/3976731280 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f3ca806dd60 0x7f3ca8070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:52:39.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.116+0000 7f3cb2ffd700 1 -- 172.21.15.16:0/3976731280 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f3cac08ef00 con 0x7f3cbc106a70 2024-04-02T04:52:39.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.116+0000 7f3cc1c8a700 1 --2- 172.21.15.16:0/3976731280 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f3ca806dd60 0x7f3ca8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:52:39.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.117+0000 7f3cc1c8a700 1 --2- 172.21.15.16:0/3976731280 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f3ca806dd60 0x7f3ca8070220 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8005950 tx=0x7f3cb800b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:52:39.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.121+0000 7f3cb2ffd700 1 -- 172.21.15.16:0/3976731280 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3cac05bf20 con 0x7f3cbc106a70 2024-04-02T04:52:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.412+0000 7f3cc3eee700 1 -- 172.21.15.16:0/3976731280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3cbc1021b0 con 0x7f3cbc106a70 2024-04-02T04:52:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.412+0000 7f3cb2ffd700 1 -- 172.21.15.16:0/3976731280 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f3cac05bab0 con 0x7f3cbc106a70 2024-04-02T04:52:39.416 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:52:39.416 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": { 2024-04-02T04:52:39.416 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:52:39.416 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:52:39.416 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": { 2024-04-02T04:52:39.416 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:52:39.416 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:52:39.416 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": { 2024-04-02T04:52:39.416 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 6 2024-04-02T04:52:39.416 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:52:39.417 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-04-02T04:52:39.417 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 4 2024-04-02T04:52:39.417 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:52:39.417 INFO:teuthology.orchestra.run.smithi016.stdout: "overall": { 2024-04-02T04:52:39.417 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 14 2024-04-02T04:52:39.417 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-02T04:52:39.417 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:52:39.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.415+0000 7f3cc3eee700 1 -- 172.21.15.16:0/3976731280 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f3ca806dd60 msgr2=0x7f3ca8070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:39.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.415+0000 7f3cc3eee700 1 --2- 172.21.15.16:0/3976731280 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f3ca806dd60 0x7f3ca8070220 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8005950 tx=0x7f3cb800b410 comp rx=0 tx=0).stop 2024-04-02T04:52:39.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.415+0000 7f3cc3eee700 1 -- 172.21.15.16:0/3976731280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cbc106a70 msgr2=0x7f3cbc19cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:52:39.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.415+0000 7f3cc3eee700 1 --2- 172.21.15.16:0/3976731280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cbc106a70 0x7f3cbc19cbc0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3cac006010 tx=0x7f3cac00bd70 comp rx=0 tx=0).stop 2024-04-02T04:52:39.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.415+0000 7f3cc3eee700 1 -- 172.21.15.16:0/3976731280 shutdown_connections 2024-04-02T04:52:39.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.415+0000 7f3cc3eee700 1 --2- 172.21.15.16:0/3976731280 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f3ca806dd60 0x7f3ca8070220 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:39.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.415+0000 7f3cc3eee700 1 --2- 172.21.15.16:0/3976731280 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cbc100a70 0x7f3cbc19c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:39.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.415+0000 7f3cc3eee700 1 --2- 172.21.15.16:0/3976731280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cbc106a70 0x7f3cbc19cbc0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:52:39.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.415+0000 7f3cc3eee700 1 -- 172.21.15.16:0/3976731280 >> 172.21.15.16:0/3976731280 conn(0x7f3cbc0fc580 msgr2=0x7f3cbc0fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:52:39.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.415+0000 7f3cc3eee700 1 -- 172.21.15.16:0/3976731280 shutdown_connections 2024-04-02T04:52:39.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:52:39.415+0000 7f3cc3eee700 1 -- 172.21.15.16:0/3976731280 wait complete. 2024-04-02T04:52:40.063 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-04-02T04:52:40.280 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:52:40.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:40 smithi016 ceph-mon[93230]: pgmap v289: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:40.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:40 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/3976731280' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:40.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:40 smithi027 ceph-mon[83036]: pgmap v289: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:40.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:40 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/3976731280' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:52:41.854 INFO:teuthology.orchestra.run.smithi016.stdout:wait for servicemap items w/ changing names to refresh 2024-04-02T04:52:42.492 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:42 smithi016 ceph-mon[93230]: pgmap v290: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:42.512 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'sleep 60' 2024-04-02T04:52:42.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:42 smithi027 ceph-mon[83036]: pgmap v290: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:42.738 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:52:44.539 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:44 smithi027 ceph-mon[83036]: pgmap v291: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:44.559 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:44 smithi016 ceph-mon[93230]: pgmap v291: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:46.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:46 smithi016 ceph-mon[93230]: pgmap v292: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:46.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:46 smithi027 ceph-mon[83036]: pgmap v292: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:47.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:47 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:52:47.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:47 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:52:48.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:48 smithi016 ceph-mon[93230]: pgmap v293: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:48.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:48 smithi027 ceph-mon[83036]: pgmap v293: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:50.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:50 smithi016 ceph-mon[93230]: pgmap v294: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:50.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:50 smithi027 ceph-mon[83036]: pgmap v294: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:52.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:52 smithi016 ceph-mon[93230]: pgmap v295: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:52.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:52 smithi027 ceph-mon[83036]: pgmap v295: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:54.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:54 smithi016 ceph-mon[93230]: pgmap v296: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:54.637 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:54 smithi027 ceph-mon[83036]: pgmap v296: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:56.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:56 smithi016 ceph-mon[93230]: pgmap v297: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:56.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:56 smithi027 ceph-mon[83036]: pgmap v297: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:58.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:52:58 smithi016 ceph-mon[93230]: pgmap v298: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:52:58.648 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:52:58 smithi027 ceph-mon[83036]: pgmap v298: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:00.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:00 smithi016 ceph-mon[93230]: pgmap v299: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:00.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:00 smithi027 ceph-mon[83036]: pgmap v299: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:02.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:02 smithi016 ceph-mon[93230]: pgmap v300: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:02.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:02 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:53:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:02 smithi027 ceph-mon[83036]: pgmap v300: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:02 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:53:04.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:04 smithi016 ceph-mon[93230]: pgmap v301: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:04.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:04 smithi027 ceph-mon[83036]: pgmap v301: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:06.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:06 smithi016 ceph-mon[93230]: pgmap v302: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:06.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:06 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:53:06.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:06 smithi027 ceph-mon[83036]: pgmap v302: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:06.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:06 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:53:07.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:07 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:53:07.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:07 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:53:07.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:07 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:53:07.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:07 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:53:07.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:07 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:53:07.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:07 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:53:08.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:08 smithi016 ceph-mon[93230]: pgmap v303: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:08.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:08 smithi027 ceph-mon[83036]: pgmap v303: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:10.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:10 smithi016 ceph-mon[93230]: pgmap v304: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:10.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:10 smithi027 ceph-mon[83036]: pgmap v304: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:12.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:12 smithi016 ceph-mon[93230]: pgmap v305: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:12.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:12 smithi027 ceph-mon[83036]: pgmap v305: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:14.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:14 smithi016 ceph-mon[93230]: pgmap v306: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:14.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:14 smithi027 ceph-mon[83036]: pgmap v306: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:16.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:16 smithi016 ceph-mon[93230]: pgmap v307: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:16.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:16 smithi027 ceph-mon[83036]: pgmap v307: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:17.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:17 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:53:17.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:17 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:53:18.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:18 smithi016 ceph-mon[93230]: pgmap v308: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:18.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:18 smithi027 ceph-mon[83036]: pgmap v308: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:20.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:20 smithi016 ceph-mon[93230]: pgmap v309: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:20.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:20 smithi027 ceph-mon[83036]: pgmap v309: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:22.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:22 smithi016 ceph-mon[93230]: pgmap v310: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:22.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:22 smithi027 ceph-mon[83036]: pgmap v310: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:24.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:24 smithi016 ceph-mon[93230]: pgmap v311: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:24.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:24 smithi027 ceph-mon[83036]: pgmap v311: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:26.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:26 smithi016 ceph-mon[93230]: pgmap v312: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:26.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:26 smithi027 ceph-mon[83036]: pgmap v312: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:28.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:28 smithi016 ceph-mon[93230]: pgmap v313: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:28.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:28 smithi027 ceph-mon[83036]: pgmap v313: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:30.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:30 smithi016 ceph-mon[93230]: pgmap v314: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:30.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:30 smithi027 ceph-mon[83036]: pgmap v314: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:32.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:32 smithi016 ceph-mon[93230]: pgmap v315: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:32.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:32 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:53:32.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:32 smithi027 ceph-mon[83036]: pgmap v315: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:32.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:32 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:53:34.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:34 smithi016 ceph-mon[93230]: pgmap v316: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:34.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:34 smithi027 ceph-mon[83036]: pgmap v316: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:36.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:36 smithi016 ceph-mon[93230]: pgmap v317: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:36.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:36 smithi027 ceph-mon[83036]: pgmap v317: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:38.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:38 smithi016 ceph-mon[93230]: pgmap v318: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:38.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:38 smithi027 ceph-mon[83036]: pgmap v318: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:40.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:40 smithi027 ceph-mon[83036]: pgmap v319: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:40.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:40 smithi016 ceph-mon[93230]: pgmap v319: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:42.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:42 smithi016 ceph-mon[93230]: pgmap v320: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:42.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:42 smithi027 ceph-mon[83036]: pgmap v320: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:44.393 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:44 smithi016 ceph-mon[93230]: pgmap v321: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:44.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:44 smithi027 ceph-mon[83036]: pgmap v321: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:44.677 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'ceph orch ps' 2024-04-02T04:53:44.896 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:53:46.360 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:46 smithi016 ceph-mon[93230]: pgmap v322: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.600+0000 7fee5ea8e700 1 -- 172.21.15.16:0/2065025371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee58105950 msgr2=0x7fee58105d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.600+0000 7fee5ea8e700 1 --2- 172.21.15.16:0/2065025371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee58105950 0x7fee58105d30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009910 tx=0x7fee4c009c20 comp rx=0 tx=0).stop 2024-04-02T04:53:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.600+0000 7fee5ea8e700 1 -- 172.21.15.16:0/2065025371 shutdown_connections 2024-04-02T04:53:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.600+0000 7fee5ea8e700 1 --2- 172.21.15.16:0/2065025371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee580ff950 0x7fee580ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.600+0000 7fee5ea8e700 1 --2- 172.21.15.16:0/2065025371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee58105950 0x7fee58105d30 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.600+0000 7fee5ea8e700 1 -- 172.21.15.16:0/2065025371 >> 172.21.15.16:0/2065025371 conn(0x7fee58076640 msgr2=0x7fee580fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.600+0000 7fee5ea8e700 1 -- 172.21.15.16:0/2065025371 shutdown_connections 2024-04-02T04:53:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.600+0000 7fee5ea8e700 1 -- 172.21.15.16:0/2065025371 wait complete. 2024-04-02T04:53:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.600+0000 7fee5ea8e700 1 Processor -- start 2024-04-02T04:53:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.600+0000 7fee5ea8e700 1 -- start start 2024-04-02T04:53:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.601+0000 7fee5ea8e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee580ff950 0x7fee5819c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.601+0000 7fee5ea8e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee58105950 0x7fee5819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.601+0000 7fee5ea8e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5819d1d0 con 0x7fee580ff950 2024-04-02T04:53:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.601+0000 7fee5ea8e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee58196820 con 0x7fee58105950 2024-04-02T04:53:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.601+0000 7fee5c82a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee580ff950 0x7fee5819c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.601+0000 7fee5c82a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee580ff950 0x7fee5819c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44092/0 (socket says 172.21.15.16:44092) 2024-04-02T04:53:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.601+0000 7fee5c82a700 1 -- 172.21.15.16:0/3707934993 learned_addr learned my addr 172.21.15.16:0/3707934993 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:53:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.601+0000 7fee57fff700 1 --2- 172.21.15.16:0/3707934993 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee58105950 0x7fee5819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.601+0000 7fee5c82a700 1 -- 172.21.15.16:0/3707934993 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee58105950 msgr2=0x7fee5819cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.601+0000 7fee5c82a700 1 --2- 172.21.15.16:0/3707934993 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee58105950 0x7fee5819cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.601+0000 7fee5c82a700 1 -- 172.21.15.16:0/3707934993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4c0095f0 con 0x7fee580ff950 2024-04-02T04:53:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.601+0000 7fee57fff700 1 --2- 172.21.15.16:0/3707934993 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee58105950 0x7fee5819cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T04:53:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.601+0000 7fee5c82a700 1 --2- 172.21.15.16:0/3707934993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee580ff950 0x7fee5819c690 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fee4c006010 tx=0x7fee4c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.601+0000 7fee55ffb700 1 -- 172.21.15.16:0/3707934993 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee4c01d070 con 0x7fee580ff950 2024-04-02T04:53:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.601+0000 7fee55ffb700 1 -- 172.21.15.16:0/3707934993 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fee4c00f460 con 0x7fee580ff950 2024-04-02T04:53:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.601+0000 7fee5ea8e700 1 -- 172.21.15.16:0/3707934993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee58196aa0 con 0x7fee580ff950 2024-04-02T04:53:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.602+0000 7fee55ffb700 1 -- 172.21.15.16:0/3707934993 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee4c017630 con 0x7fee580ff950 2024-04-02T04:53:46.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.602+0000 7fee5ea8e700 1 -- 172.21.15.16:0/3707934993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee58196f90 con 0x7fee580ff950 2024-04-02T04:53:46.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.603+0000 7fee5ea8e700 1 -- 172.21.15.16:0/3707934993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee58101090 con 0x7fee580ff950 2024-04-02T04:53:46.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.603+0000 7fee55ffb700 1 -- 172.21.15.16:0/3707934993 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fee4c00f5d0 con 0x7fee580ff950 2024-04-02T04:53:46.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.604+0000 7fee55ffb700 1 --2- 172.21.15.16:0/3707934993 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fee4806dd60 0x7fee48070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:46.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.604+0000 7fee55ffb700 1 -- 172.21.15.16:0/3707934993 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fee4c08f090 con 0x7fee580ff950 2024-04-02T04:53:46.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.604+0000 7fee57fff700 1 --2- 172.21.15.16:0/3707934993 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fee4806dd60 0x7fee48070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:46.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.604+0000 7fee57fff700 1 --2- 172.21.15.16:0/3707934993 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fee4806dd60 0x7fee48070220 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fee44005fd0 tx=0x7fee44005e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:46.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.609+0000 7fee55ffb700 1 -- 172.21.15.16:0/3707934993 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fee4c05c170 con 0x7fee580ff950 2024-04-02T04:53:46.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:46 smithi027 ceph-mon[83036]: pgmap v322: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:46.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.824+0000 7fee5ea8e700 1 -- 172.21.15.16:0/3707934993 --> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fee5810cb50 con 0x7fee4806dd60 2024-04-02T04:53:46.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.835+0000 7fee55ffb700 1 -- 172.21.15.16:0/3707934993 <== mgr.34102 v2:172.21.15.16:6800/835455925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fee5810cb50 con 0x7fee4806dd60 2024-04-02T04:53:46.839 INFO:teuthology.orchestra.run.smithi016.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T04:53:46.839 INFO:teuthology.orchestra.run.smithi016.stdout:alertmanager.smithi016 smithi016 *:9093,9094 running (2m) 106s ago 19m 16.2M - 0.25.0 c8568f914cd2 6c7b91689a07 2024-04-02T04:53:46.839 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi016 smithi016 running (3m) 106s ago 20m 7952k - 19.0.0-1979-g6cec793d 78f830190839 1d3ba1f157b9 2024-04-02T04:53:46.839 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-exporter.smithi027 smithi027 running (3m) 2m ago 18m 7558k - 19.0.0-1979-g6cec793d 78f830190839 3402997325e5 2024-04-02T04:53:46.839 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi016 smithi016 running (8m) 106s ago 19m 6660k - 19.0.0-1979-g6cec793d 78f830190839 dc63ce0eef58 2024-04-02T04:53:46.839 INFO:teuthology.orchestra.run.smithi016.stdout:crash.smithi027 smithi027 running (8m) 2m ago 18m 6665k - 19.0.0-1979-g6cec793d 78f830190839 2beb077018e4 2024-04-02T04:53:46.839 INFO:teuthology.orchestra.run.smithi016.stdout:grafana.smithi016 smithi016 *:3000 running (112s) 106s ago 19m 54.4M - 9.4.12 2bacad6d85d8 923c1f900a27 2024-04-02T04:53:46.839 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.fyzrnm smithi016 running (4m) 106s ago 14m 16.7M - 19.0.0-1979-g6cec793d 78f830190839 98f0a83c6631 2024-04-02T04:53:46.839 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi016.uhgkgg smithi016 running (4m) 106s ago 14m 89.8M - 19.0.0-1979-g6cec793d 78f830190839 19be3989a243 2024-04-02T04:53:46.840 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.sdublg smithi027 running (3m) 2m ago 14m 12.8M - 19.0.0-1979-g6cec793d 78f830190839 7c307783baab 2024-04-02T04:53:46.840 INFO:teuthology.orchestra.run.smithi016.stdout:mds.cephfs.smithi027.zwrfoo smithi027 running (3m) 2m ago 14m 13.6M - 19.0.0-1979-g6cec793d 78f830190839 acc6430302ac 2024-04-02T04:53:46.840 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi016.eqnebx smithi016 *:8443,9283,8765 running (10m) 106s ago 21m 520M - 19.0.0-1979-g6cec793d 78f830190839 d61ed91b4577 2024-04-02T04:53:46.840 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.smithi027.kjzcsc smithi027 *:8443,9283,8765 running (10m) 2m ago 18m 451M - 19.0.0-1979-g6cec793d 78f830190839 5072a5ffc761 2024-04-02T04:53:46.840 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi016 smithi016 running (9m) 106s ago 21m 56.0M 2048M 19.0.0-1979-g6cec793d 78f830190839 eabfc84f1cdf 2024-04-02T04:53:46.840 INFO:teuthology.orchestra.run.smithi016.stdout:mon.smithi027 smithi027 running (9m) 2m ago 18m 44.0M 2048M 19.0.0-1979-g6cec793d 78f830190839 fcd68b20dcc5 2024-04-02T04:53:46.840 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi016 smithi016 *:9100 running (2m) 106s ago 19m 8476k - 1.5.0 0da6a335fe13 f3e285b06d71 2024-04-02T04:53:46.840 INFO:teuthology.orchestra.run.smithi016.stdout:node-exporter.smithi027 smithi027 *:9100 running (2m) 2m ago 18m 3649k - 1.5.0 0da6a335fe13 c8e26f58ca63 2024-04-02T04:53:46.840 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 smithi016 running (7m) 106s ago 17m 91.5M 1217M 19.0.0-1979-g6cec793d 78f830190839 421d9e1cc751 2024-04-02T04:53:46.840 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 smithi016 running (7m) 106s ago 17m 100M 1217M 19.0.0-1979-g6cec793d 78f830190839 3612a56236a3 2024-04-02T04:53:46.840 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 smithi016 running (6m) 106s ago 16m 91.9M 1217M 19.0.0-1979-g6cec793d 78f830190839 a8e9b8d0303c 2024-04-02T04:53:46.840 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 smithi027 running (6m) 2m ago 16m 112M 2240M 19.0.0-1979-g6cec793d 78f830190839 0dd13b5be8c0 2024-04-02T04:53:46.840 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 smithi027 running (5m) 2m ago 16m 89.3M 2240M 19.0.0-1979-g6cec793d 78f830190839 9025eacf8145 2024-04-02T04:53:46.840 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 smithi027 running (4m) 2m ago 15m 88.2M 2240M 19.0.0-1979-g6cec793d 78f830190839 c2787f157377 2024-04-02T04:53:46.840 INFO:teuthology.orchestra.run.smithi016.stdout:prometheus.smithi016 smithi016 *:9095 running (2m) 106s ago 19m 46.2M - 2.43.0 a07b618ecd1d 042152fa1836 2024-04-02T04:53:46.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.838+0000 7fee5ea8e700 1 -- 172.21.15.16:0/3707934993 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fee4806dd60 msgr2=0x7fee48070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:46.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.838+0000 7fee5ea8e700 1 --2- 172.21.15.16:0/3707934993 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fee4806dd60 0x7fee48070220 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fee44005fd0 tx=0x7fee44005e30 comp rx=0 tx=0).stop 2024-04-02T04:53:46.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.838+0000 7fee5ea8e700 1 -- 172.21.15.16:0/3707934993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee580ff950 msgr2=0x7fee5819c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:46.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.838+0000 7fee5ea8e700 1 --2- 172.21.15.16:0/3707934993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee580ff950 0x7fee5819c690 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fee4c006010 tx=0x7fee4c00be20 comp rx=0 tx=0).stop 2024-04-02T04:53:46.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.838+0000 7fee5ea8e700 1 -- 172.21.15.16:0/3707934993 shutdown_connections 2024-04-02T04:53:46.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.838+0000 7fee5ea8e700 1 --2- 172.21.15.16:0/3707934993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee580ff950 0x7fee5819c690 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:46.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.838+0000 7fee5ea8e700 1 --2- 172.21.15.16:0/3707934993 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fee4806dd60 0x7fee48070220 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:46.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.838+0000 7fee5ea8e700 1 --2- 172.21.15.16:0/3707934993 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee58105950 0x7fee5819cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:46.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.838+0000 7fee5ea8e700 1 -- 172.21.15.16:0/3707934993 >> 172.21.15.16:0/3707934993 conn(0x7fee58076640 msgr2=0x7fee580fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:46.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.838+0000 7fee5ea8e700 1 -- 172.21.15.16:0/3707934993 shutdown_connections 2024-04-02T04:53:46.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:46.838+0000 7fee5ea8e700 1 -- 172.21.15.16:0/3707934993 wait complete. 2024-04-02T04:53:47.454 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:47 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:53:47.474 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'ceph versions' 2024-04-02T04:53:47.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:47 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:53:47.701 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:53:48.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:48 smithi027 ceph-mon[83036]: from='client.34612 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:53:48.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:48 smithi027 ceph-mon[83036]: pgmap v323: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:48.754 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:48 smithi016 ceph-mon[93230]: from='client.34612 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T04:53:48.754 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:48 smithi016 ceph-mon[93230]: pgmap v323: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:49.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.347+0000 7f5b5c20c700 1 -- 172.21.15.16:0/4125559791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b54100a90 msgr2=0x7f5b54100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:49.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.347+0000 7f5b5c20c700 1 --2- 172.21.15.16:0/4125559791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b54100a90 0x7f5b54100f10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5b44009910 tx=0x7f5b44009c20 comp rx=0 tx=0).stop 2024-04-02T04:53:49.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.347+0000 7f5b5c20c700 1 -- 172.21.15.16:0/4125559791 shutdown_connections 2024-04-02T04:53:49.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.347+0000 7f5b5c20c700 1 --2- 172.21.15.16:0/4125559791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b54100a90 0x7f5b54100f10 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:49.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.347+0000 7f5b5c20c700 1 --2- 172.21.15.16:0/4125559791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b54106a90 0x7f5b54106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:49.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.347+0000 7f5b5c20c700 1 -- 172.21.15.16:0/4125559791 >> 172.21.15.16:0/4125559791 conn(0x7f5b540fc5a0 msgr2=0x7f5b540fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:49.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.347+0000 7f5b5c20c700 1 -- 172.21.15.16:0/4125559791 shutdown_connections 2024-04-02T04:53:49.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.347+0000 7f5b5c20c700 1 -- 172.21.15.16:0/4125559791 wait complete. 2024-04-02T04:53:49.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.348+0000 7f5b5c20c700 1 Processor -- start 2024-04-02T04:53:49.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.348+0000 7f5b5c20c700 1 -- start start 2024-04-02T04:53:49.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.348+0000 7f5b5c20c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b54100a90 0x7f5b540730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:49.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.348+0000 7f5b5c20c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b54106a90 0x7f5b540735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:49.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.348+0000 7f5b5c20c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b54073b30 con 0x7f5b54106a90 2024-04-02T04:53:49.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.348+0000 7f5b5c20c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b54073c70 con 0x7f5b54100a90 2024-04-02T04:53:49.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.348+0000 7f5b597a7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b54106a90 0x7f5b540735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:49.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.348+0000 7f5b597a7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b54106a90 0x7f5b540735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44100/0 (socket says 172.21.15.16:44100) 2024-04-02T04:53:49.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.348+0000 7f5b597a7700 1 -- 172.21.15.16:0/226603416 learned_addr learned my addr 172.21.15.16:0/226603416 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:53:49.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.349+0000 7f5b59fa8700 1 --2- 172.21.15.16:0/226603416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b54100a90 0x7f5b540730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:49.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.349+0000 7f5b597a7700 1 -- 172.21.15.16:0/226603416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b54100a90 msgr2=0x7f5b540730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:49.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.349+0000 7f5b597a7700 1 --2- 172.21.15.16:0/226603416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b54100a90 0x7f5b540730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:49.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.349+0000 7f5b597a7700 1 -- 172.21.15.16:0/226603416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b440095f0 con 0x7f5b54106a90 2024-04-02T04:53:49.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.349+0000 7f5b59fa8700 1 --2- 172.21.15.16:0/226603416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b54100a90 0x7f5b540730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T04:53:49.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.349+0000 7f5b597a7700 1 --2- 172.21.15.16:0/226603416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b54106a90 0x7f5b540735f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5b4400bd70 tx=0x7f5b4400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:49.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.349+0000 7f5b4affd700 1 -- 172.21.15.16:0/226603416 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b4401d070 con 0x7f5b54106a90 2024-04-02T04:53:49.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.349+0000 7f5b4affd700 1 -- 172.21.15.16:0/226603416 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b4400f460 con 0x7f5b54106a90 2024-04-02T04:53:49.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.349+0000 7f5b5c20c700 1 -- 172.21.15.16:0/226603416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b54073ef0 con 0x7f5b54106a90 2024-04-02T04:53:49.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.349+0000 7f5b4affd700 1 -- 172.21.15.16:0/226603416 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b44017630 con 0x7f5b54106a90 2024-04-02T04:53:49.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.349+0000 7f5b5c20c700 1 -- 172.21.15.16:0/226603416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b541a2f10 con 0x7f5b54106a90 2024-04-02T04:53:49.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.350+0000 7f5b5c20c700 1 -- 172.21.15.16:0/226603416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b541090b0 con 0x7f5b54106a90 2024-04-02T04:53:49.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.351+0000 7f5b4affd700 1 -- 172.21.15.16:0/226603416 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f5b4400f5d0 con 0x7f5b54106a90 2024-04-02T04:53:49.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.351+0000 7f5b4affd700 1 --2- 172.21.15.16:0/226603416 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5b4006dd60 0x7f5b40070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:49.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.351+0000 7f5b4affd700 1 -- 172.21.15.16:0/226603416 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f5b4408f0e0 con 0x7f5b54106a90 2024-04-02T04:53:49.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.351+0000 7f5b59fa8700 1 --2- 172.21.15.16:0/226603416 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5b4006dd60 0x7f5b40070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:49.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.352+0000 7f5b59fa8700 1 --2- 172.21.15.16:0/226603416 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5b4006dd60 0x7f5b40070220 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5b500099d0 tx=0x7f5b50008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:49.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.356+0000 7f5b4affd700 1 -- 172.21.15.16:0/226603416 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5b440585d0 con 0x7f5b54106a90 2024-04-02T04:53:49.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.644+0000 7f5b5c20c700 1 -- 172.21.15.16:0/226603416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5b541021d0 con 0x7f5b54106a90 2024-04-02T04:53:49.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.645+0000 7f5b4affd700 1 -- 172.21.15.16:0/226603416 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f5b4405bc90 con 0x7f5b54106a90 2024-04-02T04:53:49.649 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-02T04:53:49.649 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": { 2024-04-02T04:53:49.649 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:53:49.649 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:53:49.649 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": { 2024-04-02T04:53:49.649 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 2 2024-04-02T04:53:49.649 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:53:49.649 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": { 2024-04-02T04:53:49.649 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 6 2024-04-02T04:53:49.649 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:53:49.649 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-04-02T04:53:49.650 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 4 2024-04-02T04:53:49.650 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-02T04:53:49.650 INFO:teuthology.orchestra.run.smithi016.stdout: "overall": { 2024-04-02T04:53:49.650 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)": 14 2024-04-02T04:53:49.650 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-02T04:53:49.650 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-02T04:53:49.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.647+0000 7f5b5c20c700 1 -- 172.21.15.16:0/226603416 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5b4006dd60 msgr2=0x7f5b40070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:49.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.647+0000 7f5b5c20c700 1 --2- 172.21.15.16:0/226603416 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5b4006dd60 0x7f5b40070220 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5b500099d0 tx=0x7f5b50008040 comp rx=0 tx=0).stop 2024-04-02T04:53:49.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.647+0000 7f5b5c20c700 1 -- 172.21.15.16:0/226603416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b54106a90 msgr2=0x7f5b540735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:49.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.647+0000 7f5b5c20c700 1 --2- 172.21.15.16:0/226603416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b54106a90 0x7f5b540735f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5b4400bd70 tx=0x7f5b4400be50 comp rx=0 tx=0).stop 2024-04-02T04:53:49.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.647+0000 7f5b5c20c700 1 -- 172.21.15.16:0/226603416 shutdown_connections 2024-04-02T04:53:49.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.647+0000 7f5b5c20c700 1 --2- 172.21.15.16:0/226603416 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f5b4006dd60 0x7f5b40070220 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:49.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.647+0000 7f5b5c20c700 1 --2- 172.21.15.16:0/226603416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b54100a90 0x7f5b540730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:49.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.647+0000 7f5b5c20c700 1 --2- 172.21.15.16:0/226603416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b54106a90 0x7f5b540735f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:49.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.647+0000 7f5b5c20c700 1 -- 172.21.15.16:0/226603416 >> 172.21.15.16:0/226603416 conn(0x7f5b540fc5a0 msgr2=0x7f5b540fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:49.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.647+0000 7f5b5c20c700 1 -- 172.21.15.16:0/226603416 shutdown_connections 2024-04-02T04:53:49.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:49.647+0000 7f5b5c20c700 1 -- 172.21.15.16:0/226603416 wait complete. 2024-04-02T04:53:50.266 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-04-02T04:53:50.492 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:53:50.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:50 smithi016 ceph-mon[93230]: pgmap v324: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:50.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:50 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/226603416' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:53:50.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:50 smithi027 ceph-mon[83036]: pgmap v324: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:50.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:50 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/226603416' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:53:52.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.188+0000 7f6a461f1700 1 -- 172.21.15.16:0/158722995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a400ff970 msgr2=0x7f6a400ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:52.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.188+0000 7f6a461f1700 1 --2- 172.21.15.16:0/158722995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a400ff970 0x7f6a400ffdf0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6a34009910 tx=0x7f6a34009c20 comp rx=0 tx=0).stop 2024-04-02T04:53:52.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.188+0000 7f6a461f1700 1 -- 172.21.15.16:0/158722995 shutdown_connections 2024-04-02T04:53:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.188+0000 7f6a461f1700 1 --2- 172.21.15.16:0/158722995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a400ff970 0x7f6a400ffdf0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.188+0000 7f6a461f1700 1 --2- 172.21.15.16:0/158722995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a40105970 0x7f6a40105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.188+0000 7f6a461f1700 1 -- 172.21.15.16:0/158722995 >> 172.21.15.16:0/158722995 conn(0x7f6a400fb480 msgr2=0x7f6a400fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.188+0000 7f6a461f1700 1 -- 172.21.15.16:0/158722995 shutdown_connections 2024-04-02T04:53:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.188+0000 7f6a461f1700 1 -- 172.21.15.16:0/158722995 wait complete. 2024-04-02T04:53:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.189+0000 7f6a461f1700 1 Processor -- start 2024-04-02T04:53:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.189+0000 7f6a461f1700 1 -- start start 2024-04-02T04:53:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.189+0000 7f6a461f1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a400ff970 0x7f6a40197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.189+0000 7f6a461f1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a40105970 0x7f6a40197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.189+0000 7f6a461f1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a40198360 con 0x7f6a40105970 2024-04-02T04:53:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.189+0000 7f6a461f1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a401984a0 con 0x7f6a400ff970 2024-04-02T04:53:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.190+0000 7f6a3effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a40105970 0x7f6a40197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.190+0000 7f6a3effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a40105970 0x7f6a40197d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40028/0 (socket says 172.21.15.16:40028) 2024-04-02T04:53:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.190+0000 7f6a3effd700 1 -- 172.21.15.16:0/780249800 learned_addr learned my addr 172.21.15.16:0/780249800 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:53:52.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.190+0000 7f6a3f7fe700 1 --2- 172.21.15.16:0/780249800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a400ff970 0x7f6a40197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:52.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.190+0000 7f6a3effd700 1 -- 172.21.15.16:0/780249800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a400ff970 msgr2=0x7f6a40197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:52.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.190+0000 7f6a3effd700 1 --2- 172.21.15.16:0/780249800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a400ff970 0x7f6a40197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:52.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.190+0000 7f6a3effd700 1 -- 172.21.15.16:0/780249800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a340095f0 con 0x7f6a40105970 2024-04-02T04:53:52.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.190+0000 7f6a3effd700 1 --2- 172.21.15.16:0/780249800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a40105970 0x7f6a40197d90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6a3400bd70 tx=0x7f6a3400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:52.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.190+0000 7f6a3cff9700 1 -- 172.21.15.16:0/780249800 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a3401d070 con 0x7f6a40105970 2024-04-02T04:53:52.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.190+0000 7f6a3cff9700 1 -- 172.21.15.16:0/780249800 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a3400f460 con 0x7f6a40105970 2024-04-02T04:53:52.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.190+0000 7f6a3cff9700 1 -- 172.21.15.16:0/780249800 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a34017630 con 0x7f6a40105970 2024-04-02T04:53:52.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.190+0000 7f6a461f1700 1 -- 172.21.15.16:0/780249800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a401a08c0 con 0x7f6a40105970 2024-04-02T04:53:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.190+0000 7f6a461f1700 1 -- 172.21.15.16:0/780249800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a401a0c20 con 0x7f6a40105970 2024-04-02T04:53:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.191+0000 7f6a461f1700 1 -- 172.21.15.16:0/780249800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a40107f90 con 0x7f6a40105970 2024-04-02T04:53:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.192+0000 7f6a3cff9700 1 -- 172.21.15.16:0/780249800 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f6a34003a60 con 0x7f6a40105970 2024-04-02T04:53:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.192+0000 7f6a3cff9700 1 --2- 172.21.15.16:0/780249800 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f6a2806dd60 0x7f6a28070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.193+0000 7f6a3f7fe700 1 --2- 172.21.15.16:0/780249800 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f6a2806dd60 0x7f6a28070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.193+0000 7f6a3cff9700 1 -- 172.21.15.16:0/780249800 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f6a3408f150 con 0x7f6a40105970 2024-04-02T04:53:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.193+0000 7f6a3f7fe700 1 --2- 172.21.15.16:0/780249800 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f6a2806dd60 0x7f6a28070220 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6a30005fd0 tx=0x7f6a3000a560 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:52.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.197+0000 7f6a3cff9700 1 -- 172.21.15.16:0/780249800 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6a34058600 con 0x7f6a40105970 2024-04-02T04:53:52.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.494+0000 7f6a461f1700 1 -- 172.21.15.16:0/780249800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6a401010b0 con 0x7f6a40105970 2024-04-02T04:53:52.498 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:52 smithi016 ceph-mon[93230]: pgmap v325: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:52.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.494+0000 7f6a3cff9700 1 -- 172.21.15.16:0/780249800 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f6a3405bcc0 con 0x7f6a40105970 2024-04-02T04:53:52.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.496+0000 7f6a461f1700 1 -- 172.21.15.16:0/780249800 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f6a2806dd60 msgr2=0x7f6a28070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:52.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.496+0000 7f6a461f1700 1 --2- 172.21.15.16:0/780249800 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f6a2806dd60 0x7f6a28070220 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6a30005fd0 tx=0x7f6a3000a560 comp rx=0 tx=0).stop 2024-04-02T04:53:52.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.496+0000 7f6a461f1700 1 -- 172.21.15.16:0/780249800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a40105970 msgr2=0x7f6a40197d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:52.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.496+0000 7f6a461f1700 1 --2- 172.21.15.16:0/780249800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a40105970 0x7f6a40197d90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6a3400bd70 tx=0x7f6a3400be50 comp rx=0 tx=0).stop 2024-04-02T04:53:52.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.496+0000 7f6a461f1700 1 -- 172.21.15.16:0/780249800 shutdown_connections 2024-04-02T04:53:52.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.496+0000 7f6a461f1700 1 --2- 172.21.15.16:0/780249800 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f6a2806dd60 0x7f6a28070220 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:52.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.496+0000 7f6a461f1700 1 --2- 172.21.15.16:0/780249800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a400ff970 0x7f6a40197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:52.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.496+0000 7f6a461f1700 1 --2- 172.21.15.16:0/780249800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a40105970 0x7f6a40197d90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:52.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.496+0000 7f6a461f1700 1 -- 172.21.15.16:0/780249800 >> 172.21.15.16:0/780249800 conn(0x7f6a400fb480 msgr2=0x7f6a400fc100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:52.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.496+0000 7f6a461f1700 1 -- 172.21.15.16:0/780249800 shutdown_connections 2024-04-02T04:53:52.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:52.496+0000 7f6a461f1700 1 -- 172.21.15.16:0/780249800 wait complete. 2024-04-02T04:53:52.510 INFO:teuthology.orchestra.run.smithi016.stdout:true 2024-04-02T04:53:52.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:52 smithi027 ceph-mon[83036]: pgmap v325: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:53.122 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-04-02T04:53:53.344 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:53:53.607 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:53 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/780249800' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:53:53.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:53 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/780249800' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:53:54.424 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:54 smithi016 ceph-mon[93230]: pgmap v326: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:54.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:54 smithi027 ceph-mon[83036]: pgmap v326: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:54.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:54.982+0000 7fc16a62b700 1 -- 172.21.15.16:0/400497053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc164100a70 msgr2=0x7fc164100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:54.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:54.982+0000 7fc16a62b700 1 --2- 172.21.15.16:0/400497053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc164100a70 0x7fc164100ef0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc158009910 tx=0x7fc158009c20 comp rx=0 tx=0).stop 2024-04-02T04:53:54.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:54.982+0000 7fc16a62b700 1 -- 172.21.15.16:0/400497053 shutdown_connections 2024-04-02T04:53:54.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:54.982+0000 7fc16a62b700 1 --2- 172.21.15.16:0/400497053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc164100a70 0x7fc164100ef0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:54.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:54.982+0000 7fc16a62b700 1 --2- 172.21.15.16:0/400497053 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc164106a70 0x7fc164106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:54.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:54.982+0000 7fc16a62b700 1 -- 172.21.15.16:0/400497053 >> 172.21.15.16:0/400497053 conn(0x7fc1640fc580 msgr2=0x7fc1640fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:54.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:54.982+0000 7fc16a62b700 1 -- 172.21.15.16:0/400497053 shutdown_connections 2024-04-02T04:53:54.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:54.982+0000 7fc16a62b700 1 -- 172.21.15.16:0/400497053 wait complete. 2024-04-02T04:53:54.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:54.983+0000 7fc16a62b700 1 Processor -- start 2024-04-02T04:53:54.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:54.983+0000 7fc16a62b700 1 -- start start 2024-04-02T04:53:54.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:54.983+0000 7fc16a62b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc164100a70 0x7fc1640730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:54.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:54.983+0000 7fc16a62b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc164106a70 0x7fc1640735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:54.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:54.983+0000 7fc16a62b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc164073b30 con 0x7fc164106a70 2024-04-02T04:53:54.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:54.983+0000 7fc16a62b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc164073c70 con 0x7fc164100a70 2024-04-02T04:53:54.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:54.984+0000 7fc1637fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc164106a70 0x7fc1640735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:54.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:54.984+0000 7fc1637fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc164106a70 0x7fc1640735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40050/0 (socket says 172.21.15.16:40050) 2024-04-02T04:53:54.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:54.984+0000 7fc1637fe700 1 -- 172.21.15.16:0/4181917553 learned_addr learned my addr 172.21.15.16:0/4181917553 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:53:54.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:54.984+0000 7fc163fff700 1 --2- 172.21.15.16:0/4181917553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc164100a70 0x7fc1640730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:54.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:54.984+0000 7fc1637fe700 1 -- 172.21.15.16:0/4181917553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc164100a70 msgr2=0x7fc1640730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:54.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:54.984+0000 7fc1637fe700 1 --2- 172.21.15.16:0/4181917553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc164100a70 0x7fc1640730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:54.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:54.984+0000 7fc1637fe700 1 -- 172.21.15.16:0/4181917553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1580095f0 con 0x7fc164106a70 2024-04-02T04:53:54.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:54.984+0000 7fc1637fe700 1 --2- 172.21.15.16:0/4181917553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc164106a70 0x7fc1640735f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc15800bd70 tx=0x7fc15800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:54.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:54.984+0000 7fc1617fa700 1 -- 172.21.15.16:0/4181917553 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc15801d070 con 0x7fc164106a70 2024-04-02T04:53:54.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:54.984+0000 7fc1617fa700 1 -- 172.21.15.16:0/4181917553 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc1580056e0 con 0x7fc164106a70 2024-04-02T04:53:54.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:54.984+0000 7fc16a62b700 1 -- 172.21.15.16:0/4181917553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc164073ef0 con 0x7fc164106a70 2024-04-02T04:53:54.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:54.984+0000 7fc16a62b700 1 -- 172.21.15.16:0/4181917553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1641a2dc0 con 0x7fc164106a70 2024-04-02T04:53:54.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:54.985+0000 7fc1617fa700 1 -- 172.21.15.16:0/4181917553 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc158017400 con 0x7fc164106a70 2024-04-02T04:53:54.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:54.985+0000 7fc16a62b700 1 -- 172.21.15.16:0/4181917553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc164109090 con 0x7fc164106a70 2024-04-02T04:53:54.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:54.986+0000 7fc1617fa700 1 -- 172.21.15.16:0/4181917553 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fc158003a60 con 0x7fc164106a70 2024-04-02T04:53:54.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:54.986+0000 7fc1617fa700 1 --2- 172.21.15.16:0/4181917553 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc14c06dd60 0x7fc14c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:54.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:54.987+0000 7fc1617fa700 1 -- 172.21.15.16:0/4181917553 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fc15808ef30 con 0x7fc164106a70 2024-04-02T04:53:54.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:54.987+0000 7fc163fff700 1 --2- 172.21.15.16:0/4181917553 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc14c06dd60 0x7fc14c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:54.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:54.987+0000 7fc163fff700 1 --2- 172.21.15.16:0/4181917553 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc14c06dd60 0x7fc14c070220 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc154005950 tx=0x7fc1540058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:54.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:54.991+0000 7fc1617fa700 1 -- 172.21.15.16:0/4181917553 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc1580584a0 con 0x7fc164106a70 2024-04-02T04:53:55.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:55.288+0000 7fc16a62b700 1 -- 172.21.15.16:0/4181917553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc1641021b0 con 0x7fc164106a70 2024-04-02T04:53:55.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:55.289+0000 7fc1617fa700 1 -- 172.21.15.16:0/4181917553 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fc15805bb60 con 0x7fc164106a70 2024-04-02T04:53:55.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:55.291+0000 7fc16a62b700 1 -- 172.21.15.16:0/4181917553 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc14c06dd60 msgr2=0x7fc14c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:55.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:55.291+0000 7fc16a62b700 1 --2- 172.21.15.16:0/4181917553 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc14c06dd60 0x7fc14c070220 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc154005950 tx=0x7fc1540058e0 comp rx=0 tx=0).stop 2024-04-02T04:53:55.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:55.291+0000 7fc16a62b700 1 -- 172.21.15.16:0/4181917553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc164106a70 msgr2=0x7fc1640735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:55.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:55.291+0000 7fc16a62b700 1 --2- 172.21.15.16:0/4181917553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc164106a70 0x7fc1640735f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc15800bd70 tx=0x7fc15800be50 comp rx=0 tx=0).stop 2024-04-02T04:53:55.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:55.291+0000 7fc16a62b700 1 -- 172.21.15.16:0/4181917553 shutdown_connections 2024-04-02T04:53:55.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:55.291+0000 7fc16a62b700 1 --2- 172.21.15.16:0/4181917553 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc14c06dd60 0x7fc14c070220 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:55.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:55.291+0000 7fc16a62b700 1 --2- 172.21.15.16:0/4181917553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc164100a70 0x7fc1640730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:55.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:55.291+0000 7fc16a62b700 1 --2- 172.21.15.16:0/4181917553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc164106a70 0x7fc1640735f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:55.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:55.291+0000 7fc16a62b700 1 -- 172.21.15.16:0/4181917553 >> 172.21.15.16:0/4181917553 conn(0x7fc1640fc580 msgr2=0x7fc1640fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:55.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:55.291+0000 7fc16a62b700 1 -- 172.21.15.16:0/4181917553 shutdown_connections 2024-04-02T04:53:55.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:55.291+0000 7fc16a62b700 1 -- 172.21.15.16:0/4181917553 wait complete. 2024-04-02T04:53:55.305 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 19.0.0-1979-g6cec793d (6cec793d9073f0594e2364b9b687ecaf22424cca) squid (dev)" 2024-04-02T04:53:55.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:55 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/4181917553' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:53:55.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:55 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/4181917553' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-02T04:53:55.895 DEBUG:teuthology.parallel:result is None 2024-04-02T04:53:55.895 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-02T04:53:55.908 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi016.front.sepia.ceph.com 2024-04-02T04:53:55.909 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 -- bash -c 'ceph fs dump' 2024-04-02T04:53:56.135 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/mon.smithi016/config 2024-04-02T04:53:56.553 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:56 smithi016 ceph-mon[93230]: pgmap v327: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:56.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:56 smithi027 ceph-mon[83036]: pgmap v327: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:57.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:57.691+0000 7f3a1e8ad700 1 -- 172.21.15.16:0/423345920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a18106a90 msgr2=0x7f3a18106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:57.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:57.691+0000 7f3a1e8ad700 1 --2- 172.21.15.16:0/423345920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a18106a90 0x7f3a18106e70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3a08009910 tx=0x7f3a08009c20 comp rx=0 tx=0).stop 2024-04-02T04:53:57.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:57.691+0000 7f3a1e8ad700 1 -- 172.21.15.16:0/423345920 shutdown_connections 2024-04-02T04:53:57.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:57.691+0000 7f3a1e8ad700 1 --2- 172.21.15.16:0/423345920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a18100a90 0x7f3a18100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:57.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:57.691+0000 7f3a1e8ad700 1 --2- 172.21.15.16:0/423345920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a18106a90 0x7f3a18106e70 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:57.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:57.692+0000 7f3a1e8ad700 1 -- 172.21.15.16:0/423345920 >> 172.21.15.16:0/423345920 conn(0x7f3a180fc580 msgr2=0x7f3a180fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:57.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:57.692+0000 7f3a1e8ad700 1 -- 172.21.15.16:0/423345920 shutdown_connections 2024-04-02T04:53:57.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:57.692+0000 7f3a1e8ad700 1 -- 172.21.15.16:0/423345920 wait complete. 2024-04-02T04:53:57.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:57.692+0000 7f3a1e8ad700 1 Processor -- start 2024-04-02T04:53:57.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:57.692+0000 7f3a1e8ad700 1 -- start start 2024-04-02T04:53:57.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:57.693+0000 7f3a1e8ad700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a18100a90 0x7f3a180730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:57.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:57.693+0000 7f3a1e8ad700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a18106a90 0x7f3a180735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:57.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:57.693+0000 7f3a1e8ad700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a18073b30 con 0x7f3a18106a90 2024-04-02T04:53:57.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:57.693+0000 7f3a1e8ad700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a18073c70 con 0x7f3a18100a90 2024-04-02T04:53:57.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:57.693+0000 7f3a177fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a18106a90 0x7f3a180735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:57.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:57.693+0000 7f3a177fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a18106a90 0x7f3a180735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40074/0 (socket says 172.21.15.16:40074) 2024-04-02T04:53:57.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:57.693+0000 7f3a177fe700 1 -- 172.21.15.16:0/4230315633 learned_addr learned my addr 172.21.15.16:0/4230315633 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:53:57.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:57.693+0000 7f3a17fff700 1 --2- 172.21.15.16:0/4230315633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a18100a90 0x7f3a180730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:57.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:57.693+0000 7f3a177fe700 1 -- 172.21.15.16:0/4230315633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a18100a90 msgr2=0x7f3a180730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:57.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:57.693+0000 7f3a177fe700 1 --2- 172.21.15.16:0/4230315633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a18100a90 0x7f3a180730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:57.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:57.693+0000 7f3a177fe700 1 -- 172.21.15.16:0/4230315633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a080095f0 con 0x7f3a18106a90 2024-04-02T04:53:57.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:57.693+0000 7f3a177fe700 1 --2- 172.21.15.16:0/4230315633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a18106a90 0x7f3a180735f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c00d8d0 tx=0x7f3a0c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:57.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:57.694+0000 7f3a157fa700 1 -- 172.21.15.16:0/4230315633 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a0c0098a0 con 0x7f3a18106a90 2024-04-02T04:53:57.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:57.694+0000 7f3a1e8ad700 1 -- 172.21.15.16:0/4230315633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a181a2b70 con 0x7f3a18106a90 2024-04-02T04:53:57.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:57.694+0000 7f3a157fa700 1 -- 172.21.15.16:0/4230315633 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a0c010460 con 0x7f3a18106a90 2024-04-02T04:53:57.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:57.694+0000 7f3a157fa700 1 -- 172.21.15.16:0/4230315633 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a0c00f5f0 con 0x7f3a18106a90 2024-04-02T04:53:57.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:57.694+0000 7f3a1e8ad700 1 -- 172.21.15.16:0/4230315633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a181a2ed0 con 0x7f3a18106a90 2024-04-02T04:53:57.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:57.695+0000 7f3a1e8ad700 1 -- 172.21.15.16:0/4230315633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a18109090 con 0x7f3a18106a90 2024-04-02T04:53:57.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:57.695+0000 7f3a157fa700 1 -- 172.21.15.16:0/4230315633 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f3a0c009a00 con 0x7f3a18106a90 2024-04-02T04:53:57.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:57.696+0000 7f3a157fa700 1 --2- 172.21.15.16:0/4230315633 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f3a0006dd60 0x7f3a00070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:57.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:57.696+0000 7f3a157fa700 1 -- 172.21.15.16:0/4230315633 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f3a0c08da90 con 0x7f3a18106a90 2024-04-02T04:53:57.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:57.696+0000 7f3a17fff700 1 --2- 172.21.15.16:0/4230315633 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f3a0006dd60 0x7f3a00070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:57.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:57.696+0000 7f3a17fff700 1 --2- 172.21.15.16:0/4230315633 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f3a0006dd60 0x7f3a00070220 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3a08000c00 tx=0x7f3a0801a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:57.700+0000 7f3a157fa700 1 -- 172.21.15.16:0/4230315633 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3a0c056f80 con 0x7f3a18106a90 2024-04-02T04:53:58.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.025+0000 7f3a1e8ad700 1 -- 172.21.15.16:0/4230315633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3a18073e30 con 0x7f3a18106a90 2024-04-02T04:53:58.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.025+0000 7f3a157fa700 1 -- 172.21.15.16:0/4230315633 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1929 (secure 0 0 0) 0x7f3a0c05a640 con 0x7f3a18106a90 2024-04-02T04:53:58.030 INFO:teuthology.orchestra.run.smithi016.stdout:e29 2024-04-02T04:53:58.031 INFO:teuthology.orchestra.run.smithi016.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T04:53:58.031 INFO:teuthology.orchestra.run.smithi016.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-02T04:53:58.031 INFO:teuthology.orchestra.run.smithi016.stdout:legacy client fscid: 1 2024-04-02T04:53:58.031 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:53:58.031 INFO:teuthology.orchestra.run.smithi016.stdout:Filesystem 'cephfs' (1) 2024-04-02T04:53:58.031 INFO:teuthology.orchestra.run.smithi016.stdout:fs_name cephfs 2024-04-02T04:53:58.031 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 29 2024-04-02T04:53:58.031 INFO:teuthology.orchestra.run.smithi016.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-02T04:53:58.031 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-04-02T04:39:30.740168+0000 2024-04-02T04:53:58.031 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-04-02T04:50:20.104880+0000 2024-04-02T04:53:58.031 INFO:teuthology.orchestra.run.smithi016.stdout:tableserver 0 2024-04-02T04:53:58.031 INFO:teuthology.orchestra.run.smithi016.stdout:root 0 2024-04-02T04:53:58.031 INFO:teuthology.orchestra.run.smithi016.stdout:session_timeout 60 2024-04-02T04:53:58.032 INFO:teuthology.orchestra.run.smithi016.stdout:session_autoclose 300 2024-04-02T04:53:58.032 INFO:teuthology.orchestra.run.smithi016.stdout:max_file_size 1099511627776 2024-04-02T04:53:58.032 INFO:teuthology.orchestra.run.smithi016.stdout:max_xattr_size 65536 2024-04-02T04:53:58.032 INFO:teuthology.orchestra.run.smithi016.stdout:required_client_features {} 2024-04-02T04:53:58.032 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure 0 2024-04-02T04:53:58.032 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure_osd_epoch 84 2024-04-02T04:53:58.032 INFO:teuthology.orchestra.run.smithi016.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-02T04:53:58.032 INFO:teuthology.orchestra.run.smithi016.stdout:max_mds 1 2024-04-02T04:53:58.032 INFO:teuthology.orchestra.run.smithi016.stdout:in 0 2024-04-02T04:53:58.032 INFO:teuthology.orchestra.run.smithi016.stdout:up {0=34404} 2024-04-02T04:53:58.032 INFO:teuthology.orchestra.run.smithi016.stdout:failed 2024-04-02T04:53:58.032 INFO:teuthology.orchestra.run.smithi016.stdout:damaged 2024-04-02T04:53:58.032 INFO:teuthology.orchestra.run.smithi016.stdout:stopped 2024-04-02T04:53:58.032 INFO:teuthology.orchestra.run.smithi016.stdout:data_pools [3] 2024-04-02T04:53:58.033 INFO:teuthology.orchestra.run.smithi016.stdout:metadata_pool 2 2024-04-02T04:53:58.033 INFO:teuthology.orchestra.run.smithi016.stdout:inline_data enabled 2024-04-02T04:53:58.033 INFO:teuthology.orchestra.run.smithi016.stdout:balancer 2024-04-02T04:53:58.033 INFO:teuthology.orchestra.run.smithi016.stdout:bal_rank_mask -1 2024-04-02T04:53:58.033 INFO:teuthology.orchestra.run.smithi016.stdout:standby_count_wanted 1 2024-04-02T04:53:58.033 INFO:teuthology.orchestra.run.smithi016.stdout:qdb_cluster leader: 34404 members: 34404 2024-04-02T04:53:58.033 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.uhgkgg{0:34404} state up:active seq 21 join_fscid=1 addr [v2:172.21.15.16:6826/81744708,v1:172.21.15.16:6827/81744708] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T04:53:58.033 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:53:58.033 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:53:58.033 INFO:teuthology.orchestra.run.smithi016.stdout:Standby daemons: 2024-04-02T04:53:58.033 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:53:58.033 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi016.fyzrnm{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.16:6828/782430688,v1:172.21.15.16:6829/782430688] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T04:53:58.033 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.zwrfoo{-1:44359} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.27:6824/1016194753,v1:172.21.15.27:6825/1016194753] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T04:53:58.033 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.cephfs.smithi027.sdublg{-1:44383} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.27:6826/1507844210,v1:172.21.15.27:6827/1507844210] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T04:53:58.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.027+0000 7f3a1e8ad700 1 -- 172.21.15.16:0/4230315633 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f3a0006dd60 msgr2=0x7f3a00070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:58.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.027+0000 7f3a1e8ad700 1 --2- 172.21.15.16:0/4230315633 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f3a0006dd60 0x7f3a00070220 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3a08000c00 tx=0x7f3a0801a040 comp rx=0 tx=0).stop 2024-04-02T04:53:58.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.027+0000 7f3a1e8ad700 1 -- 172.21.15.16:0/4230315633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a18106a90 msgr2=0x7f3a180735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:58.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.027+0000 7f3a1e8ad700 1 --2- 172.21.15.16:0/4230315633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a18106a90 0x7f3a180735f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c00d8d0 tx=0x7f3a0c00dbe0 comp rx=0 tx=0).stop 2024-04-02T04:53:58.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.028+0000 7f3a1e8ad700 1 -- 172.21.15.16:0/4230315633 shutdown_connections 2024-04-02T04:53:58.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.028+0000 7f3a1e8ad700 1 --2- 172.21.15.16:0/4230315633 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f3a0006dd60 0x7f3a00070220 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:58.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.028+0000 7f3a1e8ad700 1 --2- 172.21.15.16:0/4230315633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a18100a90 0x7f3a180730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:58.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.028+0000 7f3a1e8ad700 1 --2- 172.21.15.16:0/4230315633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a18106a90 0x7f3a180735f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:58.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.028+0000 7f3a1e8ad700 1 -- 172.21.15.16:0/4230315633 >> 172.21.15.16:0/4230315633 conn(0x7f3a180fc580 msgr2=0x7f3a180fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:58.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.028+0000 7f3a1e8ad700 1 -- 172.21.15.16:0/4230315633 shutdown_connections 2024-04-02T04:53:58.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.028+0000 7f3a1e8ad700 1 -- 172.21.15.16:0/4230315633 wait complete. 2024-04-02T04:53:58.035 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 29 2024-04-02T04:53:58.591 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:58 smithi016 ceph-mon[93230]: pgmap v328: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:58.591 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:58 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/4230315633' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:53:58.592 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-04-02T04:53:58.604 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T04:53:58.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:58 smithi027 ceph-mon[83036]: pgmap v328: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:53:58.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:58 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/4230315633' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T04:53:58.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.703+0000 7f42e08bd640 1 -- 172.21.15.16:0/2968632785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42dc0ff0a0 msgr2=0x7f42dc0ff480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:58.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.703+0000 7f42e08bd640 1 --2- 172.21.15.16:0/2968632785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42dc0ff0a0 0x7f42dc0ff480 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f42cc009880 tx=0x7f42cc02bce0 comp rx=0 tx=0).stop 2024-04-02T04:53:58.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.703+0000 7f42e08bd640 1 -- 172.21.15.16:0/2968632785 shutdown_connections 2024-04-02T04:53:58.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.703+0000 7f42e08bd640 1 --2- 172.21.15.16:0/2968632785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42dc0ffa50 0x7f42dc103a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:58.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.703+0000 7f42e08bd640 1 --2- 172.21.15.16:0/2968632785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42dc0ff0a0 0x7f42dc0ff480 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:58.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.703+0000 7f42e08bd640 1 -- 172.21.15.16:0/2968632785 >> 172.21.15.16:0/2968632785 conn(0x7f42dc075b10 msgr2=0x7f42dc0fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:58.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.703+0000 7f42e08bd640 1 -- 172.21.15.16:0/2968632785 shutdown_connections 2024-04-02T04:53:58.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.703+0000 7f42e08bd640 1 -- 172.21.15.16:0/2968632785 wait complete. 2024-04-02T04:53:58.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.704+0000 7f42e08bd640 1 Processor -- start 2024-04-02T04:53:58.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.704+0000 7f42e08bd640 1 -- start start 2024-04-02T04:53:58.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.704+0000 7f42e08bd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42dc0ff0a0 0x7f42dc196dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:58.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.704+0000 7f42e08bd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42dc197310 0x7f42dc197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:58.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.704+0000 7f42e08bd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc19fe40 con 0x7f42dc197310 2024-04-02T04:53:58.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.704+0000 7f42e08bd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc19ff80 con 0x7f42dc0ff0a0 2024-04-02T04:53:58.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.704+0000 7f42d1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42dc197310 0x7f42dc197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:58.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.704+0000 7f42d1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42dc197310 0x7f42dc197770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40088/0 (socket says 172.21.15.16:40088) 2024-04-02T04:53:58.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.704+0000 7f42d1d9b640 1 -- 172.21.15.16:0/3028358449 learned_addr learned my addr 172.21.15.16:0/3028358449 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:53:58.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.704+0000 7f42da59c640 1 --2- 172.21.15.16:0/3028358449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42dc0ff0a0 0x7f42dc196dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:58.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.705+0000 7f42d1d9b640 1 -- 172.21.15.16:0/3028358449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42dc0ff0a0 msgr2=0x7f42dc196dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:58.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.705+0000 7f42d1d9b640 1 --2- 172.21.15.16:0/3028358449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42dc0ff0a0 0x7f42dc196dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:58.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.705+0000 7f42d1d9b640 1 -- 172.21.15.16:0/3028358449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42cc009560 con 0x7f42dc197310 2024-04-02T04:53:58.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.705+0000 7f42d1d9b640 1 --2- 172.21.15.16:0/3028358449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42dc197310 0x7f42dc197770 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f42c400d6f0 tx=0x7f42c400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:58.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.705+0000 7f42d3fff640 1 -- 172.21.15.16:0/3028358449 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42c40051f0 con 0x7f42dc197310 2024-04-02T04:53:58.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.705+0000 7f42e08bd640 1 -- 172.21.15.16:0/3028358449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42dc1a0120 con 0x7f42dc197310 2024-04-02T04:53:58.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.705+0000 7f42d3fff640 1 -- 172.21.15.16:0/3028358449 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f42c4009ce0 con 0x7f42dc197310 2024-04-02T04:53:58.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.705+0000 7f42d3fff640 1 -- 172.21.15.16:0/3028358449 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42c4004e80 con 0x7f42dc197310 2024-04-02T04:53:58.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.705+0000 7f42e08bd640 1 -- 172.21.15.16:0/3028358449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42dc1a0680 con 0x7f42dc197310 2024-04-02T04:53:58.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.706+0000 7f42d3fff640 1 -- 172.21.15.16:0/3028358449 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f42c40046c0 con 0x7f42dc197310 2024-04-02T04:53:58.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.706+0000 7f42e08bd640 1 -- 172.21.15.16:0/3028358449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42dc100570 con 0x7f42dc197310 2024-04-02T04:53:58.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.707+0000 7f42d3fff640 1 --2- 172.21.15.16:0/3028358449 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f42c806dc70 0x7f42c8070130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:58.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.707+0000 7f42d3fff640 1 -- 172.21.15.16:0/3028358449 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f42c408d5e0 con 0x7f42dc197310 2024-04-02T04:53:58.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.707+0000 7f42da59c640 1 --2- 172.21.15.16:0/3028358449 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f42c806dc70 0x7f42c8070130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:58.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.707+0000 7f42da59c640 1 --2- 172.21.15.16:0/3028358449 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f42c806dc70 0x7f42c8070130 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f42cc006010 tx=0x7f42cc005d50 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:58.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.712+0000 7f42d3fff640 1 -- 172.21.15.16:0/3028358449 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f42c4056b80 con 0x7f42dc197310 2024-04-02T04:53:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.896+0000 7f42e08bd640 1 -- 172.21.15.16:0/3028358449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42dc04e030 con 0x7f42dc197310 2024-04-02T04:53:58.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.898+0000 7f42d3fff640 1 -- 172.21.15.16:0/3028358449 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16859 (secure 0 0 0) 0x7f42c405a240 con 0x7f42dc197310 2024-04-02T04:53:58.903 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:53:58.903 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":84,"fsid":"dc5403b4-f0a9-11ee-b647-cb9ed24678a4","created":"2024-04-02T04:32:23.014546+0000","modified":"2024-04-02T04:50:07.887023+0000","last_up_change":"2024-04-02T04:48:57.379242+0000","last_in_change":"2024-04-02T04:37:56.807214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T04:37:01.047338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-02T04:39:29.039519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-02T04:39:29.992892+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2b7be29b-0e2a-456b-b33e-c72abaad9136","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":78,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1136760904},{"type":"v1","addr":"172.21.15.16:6803","nonce":1136760904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1136760904},{"type":"v1","addr":"172.21.15.16:6805","nonce":1136760904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1136760904},{"type":"v1","addr":"172.21.15.16:6809","nonce":1136760904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1136760904},{"type":"v1","addr":"172.21.15.16:6807","nonce":1136760904}]},"public_addr":"172.21.15.16:6803/1136760904","cluster_addr":"172.21.15.16:6805/1136760904","heartbeat_back_addr":"172.21.15.16:6809/1136760904","heartbeat_front_addr":"172.21.15.16:6807/1136760904","state":["exists","up"]},{"osd":1,"uuid":"82578291-3c9c-4f3f-8de9-7a8710a4923e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":78,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2744685333},{"type":"v1","addr":"172.21.15.16:6811","nonce":2744685333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2744685333},{"type":"v1","addr":"172.21.15.16:6813","nonce":2744685333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2744685333},{"type":"v1","addr":"172.21.15.16:6817","nonce":2744685333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2744685333},{"type":"v1","addr":"172.21.15.16:6815","nonce":2744685333}]},"public_addr":"172.21.15.16:6811/2744685333","cluster_addr":"172.21.15.16:6813/2744685333","heartbeat_back_addr":"172.21.15.16:6817/2744685333","heartbeat_front_addr":"172.21.15.16:6815/2744685333","state":["exists","up"]},{"osd":2,"uuid":"046262b5-386e-4270-863c-e97a61a038b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":59,"up_from":63,"up_thru":78,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1004843818},{"type":"v1","addr":"172.21.15.16:6819","nonce":1004843818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1004843818},{"type":"v1","addr":"172.21.15.16:6821","nonce":1004843818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1004843818},{"type":"v1","addr":"172.21.15.16:6825","nonce":1004843818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1004843818},{"type":"v1","addr":"172.21.15.16:6823","nonce":1004843818}]},"public_addr":"172.21.15.16:6819/1004843818","cluster_addr":"172.21.15.16:6821/1004843818","heartbeat_back_addr":"172.21.15.16:6825/1004843818","heartbeat_front_addr":"172.21.15.16:6823/1004843818","state":["exists","up"]},{"osd":3,"uuid":"637f2a77-59a2-44fb-b47e-fcbd70f4423c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":64,"up_from":68,"up_thru":78,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":4092279016},{"type":"v1","addr":"172.21.15.27:6801","nonce":4092279016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4092279016},{"type":"v1","addr":"172.21.15.27:6803","nonce":4092279016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4092279016},{"type":"v1","addr":"172.21.15.27:6807","nonce":4092279016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4092279016},{"type":"v1","addr":"172.21.15.27:6805","nonce":4092279016}]},"public_addr":"172.21.15.27:6801/4092279016","cluster_addr":"172.21.15.27:6803/4092279016","heartbeat_back_addr":"172.21.15.27:6807/4092279016","heartbeat_front_addr":"172.21.15.27:6805/4092279016","state":["exists","up"]},{"osd":4,"uuid":"334b7b3d-505f-45b6-85b0-59ec8eeb8e91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":69,"up_from":73,"up_thru":78,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":2931886568},{"type":"v1","addr":"172.21.15.27:6809","nonce":2931886568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2931886568},{"type":"v1","addr":"172.21.15.27:6811","nonce":2931886568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2931886568},{"type":"v1","addr":"172.21.15.27:6815","nonce":2931886568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2931886568},{"type":"v1","addr":"172.21.15.27:6813","nonce":2931886568}]},"public_addr":"172.21.15.27:6809/2931886568","cluster_addr":"172.21.15.27:6811/2931886568","heartbeat_back_addr":"172.21.15.27:6815/2931886568","heartbeat_front_addr":"172.21.15.27:6813/2931886568","state":["exists","up"]},{"osd":5,"uuid":"f83802ee-4f47-47de-91dd-a667f6cd22e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":74,"up_from":78,"up_thru":78,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":401102788},{"type":"v1","addr":"172.21.15.27:6817","nonce":401102788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":401102788},{"type":"v1","addr":"172.21.15.27:6819","nonce":401102788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":401102788},{"type":"v1","addr":"172.21.15.27:6823","nonce":401102788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":401102788},{"type":"v1","addr":"172.21.15.27:6821","nonce":401102788}]},"public_addr":"172.21.15.27:6817/401102788","cluster_addr":"172.21.15.27:6819/401102788","heartbeat_back_addr":"172.21.15.27:6823/401102788","heartbeat_front_addr":"172.21.15.27:6821/401102788","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:36:02.027852+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:36:29.393593+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:36:56.847224+0000","dead_epoch":59},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:37:21.195876+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:37:47.992167+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:38:15.012477+0000","dead_epoch":74}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6827/4177192740":"2024-04-03T04:50:07.886976+0000","172.21.15.27:6824/1461455976":"2024-04-03T04:49:41.368223+0000","172.21.15.16:0/2790817266":"2024-04-03T04:44:17.011409+0000","172.21.15.16:0/1537212641":"2024-04-03T04:44:17.011409+0000","172.21.15.16:0/3646287833":"2024-04-03T04:44:17.011409+0000","172.21.15.16:6800/873531241":"2024-04-03T04:33:28.907145+0000","172.21.15.16:0/2255383579":"2024-04-03T04:32:56.644452+0000","172.21.15.16:6800/2776599951":"2024-04-03T04:44:17.011409+0000","172.21.15.16:0/3964929492":"2024-04-03T04:44:17.011409+0000","172.21.15.27:0/597316902":"2024-04-03T04:43:10.596160+0000","172.21.15.16:0/4067493740":"2024-04-03T04:33:28.907145+0000","172.21.15.16:6801/3912038174":"2024-04-03T04:34:42.713541+0000","172.21.15.16:6800/3912038174":"2024-04-03T04:34:42.713541+0000","172.21.15.16:0/1664387083":"2024-04-03T04:42:07.942513+0000","172.21.15.27:6825/1461455976":"2024-04-03T04:49:41.368223+0000","172.21.15.27:0/1314131858":"2024-04-03T04:43:10.596160+0000","172.21.15.27:6826/4177192740":"2024-04-03T04:50:07.886976+0000","172.21.15.16:6801/873531241":"2024-04-03T04:33:28.907145+0000","172.21.15.16:0/401549934":"2024-04-03T04:32:56.644452+0000","172.21.15.16:6801/3643600900":"2024-04-03T04:42:07.942513+0000","172.21.15.16:0/166410319":"2024-04-03T04:33:28.907145+0000","172.21.15.16:0/1884756010":"2024-04-03T04:32:56.644452+0000","172.21.15.16:6801/824107799":"2024-04-03T04:32:56.644452+0000","172.21.15.16:0/881481317":"2024-04-03T04:44:17.011409+0000","172.21.15.16:6800/3643600900":"2024-04-03T04:42:07.942513+0000","172.21.15.16:0/4209328561":"2024-04-03T04:34:42.713541+0000","172.21.15.16:0/1920858754":"2024-04-03T04:42:07.942513+0000","172.21.15.16:0/3733852550":"2024-04-03T04:34:42.713541+0000","172.21.15.16:6800/824107799":"2024-04-03T04:32:56.644452+0000","172.21.15.27:0/3186228192":"2024-04-03T04:43:10.596160+0000","172.21.15.27:0/1987939490":"2024-04-03T04:43:10.596160+0000","172.21.15.16:0/3405957790":"2024-04-03T04:34:42.713541+0000","172.21.15.16:0/1383394471":"2024-04-03T04:42:07.942513+0000","172.21.15.27:6828/1554339939":"2024-04-03T04:43:10.596160+0000","172.21.15.16:0/1228221358":"2024-04-03T04:33:28.907145+0000","172.21.15.16:0/2373084595":"2024-04-03T04:42:07.942513+0000","172.21.15.16:0/249069849":"2024-04-03T04:44:17.011409+0000","172.21.15.27:0/3624631480":"2024-04-03T04:43:10.596160+0000","172.21.15.27:0/2708474610":"2024-04-03T04:43:10.596160+0000","172.21.15.16:6826/71509234":"2024-04-03T04:39:46.861330+0000","172.21.15.16:6827/71509234":"2024-04-03T04:39:46.861330+0000","172.21.15.27:6829/1554339939":"2024-04-03T04:43:10.596160+0000","172.21.15.16:6801/2776599951":"2024-04-03T04:44:17.011409+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T04:53:58.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.900+0000 7f42e08bd640 1 -- 172.21.15.16:0/3028358449 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f42c806dc70 msgr2=0x7f42c8070130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:58.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.900+0000 7f42e08bd640 1 --2- 172.21.15.16:0/3028358449 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f42c806dc70 0x7f42c8070130 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f42cc006010 tx=0x7f42cc005d50 comp rx=0 tx=0).stop 2024-04-02T04:53:58.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.900+0000 7f42e08bd640 1 -- 172.21.15.16:0/3028358449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42dc197310 msgr2=0x7f42dc197770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:58.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.900+0000 7f42e08bd640 1 --2- 172.21.15.16:0/3028358449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42dc197310 0x7f42dc197770 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f42c400d6f0 tx=0x7f42c400dbb0 comp rx=0 tx=0).stop 2024-04-02T04:53:58.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.900+0000 7f42e08bd640 1 -- 172.21.15.16:0/3028358449 shutdown_connections 2024-04-02T04:53:58.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.900+0000 7f42e08bd640 1 --2- 172.21.15.16:0/3028358449 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f42c806dc70 0x7f42c8070130 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:58.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.900+0000 7f42e08bd640 1 --2- 172.21.15.16:0/3028358449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42dc197310 0x7f42dc197770 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:58.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.900+0000 7f42e08bd640 1 --2- 172.21.15.16:0/3028358449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42dc0ff0a0 0x7f42dc196dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:58.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.900+0000 7f42e08bd640 1 -- 172.21.15.16:0/3028358449 >> 172.21.15.16:0/3028358449 conn(0x7f42dc075b10 msgr2=0x7f42dc0fbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:58.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.900+0000 7f42e08bd640 1 -- 172.21.15.16:0/3028358449 shutdown_connections 2024-04-02T04:53:58.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:58.900+0000 7f42e08bd640 1 -- 172.21.15.16:0/3028358449 wait complete. 2024-04-02T04:53:58.915 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T04:37:01.047338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-02T04:39:29.039519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-02T04:39:29.992892+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T04:53:58.916 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T04:53:59.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.018+0000 7fcff5bf0640 1 -- 172.21.15.16:0/2563461782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcff01001c0 msgr2=0x7fcff01005a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:59.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.018+0000 7fcff5bf0640 1 --2- 172.21.15.16:0/2563461782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcff01001c0 0x7fcff01005a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4009880 tx=0x7fcfe402bce0 comp rx=0 tx=0).stop 2024-04-02T04:53:59.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.018+0000 7fcff5bf0640 1 -- 172.21.15.16:0/2563461782 shutdown_connections 2024-04-02T04:53:59.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.018+0000 7fcff5bf0640 1 --2- 172.21.15.16:0/2563461782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcff0100b70 0x7fcff0104bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:59.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.018+0000 7fcff5bf0640 1 --2- 172.21.15.16:0/2563461782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcff01001c0 0x7fcff01005a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:59.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.018+0000 7fcff5bf0640 1 -- 172.21.15.16:0/2563461782 >> 172.21.15.16:0/2563461782 conn(0x7fcff00fba20 msgr2=0x7fcff00fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:59.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.018+0000 7fcff5bf0640 1 -- 172.21.15.16:0/2563461782 shutdown_connections 2024-04-02T04:53:59.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.018+0000 7fcff5bf0640 1 -- 172.21.15.16:0/2563461782 wait complete. 2024-04-02T04:53:59.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.018+0000 7fcff5bf0640 1 Processor -- start 2024-04-02T04:53:59.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.018+0000 7fcff5bf0640 1 -- start start 2024-04-02T04:53:59.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.019+0000 7fcff5bf0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcff01001c0 0x7fcff0196010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:59.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.019+0000 7fcff5bf0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcff0196550 0x7fcff019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:59.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.019+0000 7fcff5bf0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff0196b50 con 0x7fcff01001c0 2024-04-02T04:53:59.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.019+0000 7fcff5bf0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff0196cc0 con 0x7fcff0196550 2024-04-02T04:53:59.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.019+0000 7fcfef7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcff01001c0 0x7fcff0196010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:59.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.019+0000 7fcfef7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcff01001c0 0x7fcff0196010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40100/0 (socket says 172.21.15.16:40100) 2024-04-02T04:53:59.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.019+0000 7fcfef7fe640 1 -- 172.21.15.16:0/2457343040 learned_addr learned my addr 172.21.15.16:0/2457343040 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:53:59.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.019+0000 7fcfeeffd640 1 --2- 172.21.15.16:0/2457343040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcff0196550 0x7fcff019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:59.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.019+0000 7fcfef7fe640 1 -- 172.21.15.16:0/2457343040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcff0196550 msgr2=0x7fcff019a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:59.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.019+0000 7fcfef7fe640 1 --2- 172.21.15.16:0/2457343040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcff0196550 0x7fcff019a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:59.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.019+0000 7fcfef7fe640 1 -- 172.21.15.16:0/2457343040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe4009560 con 0x7fcff01001c0 2024-04-02T04:53:59.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.019+0000 7fcfeeffd640 1 --2- 172.21.15.16:0/2457343040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcff0196550 0x7fcff019a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T04:53:59.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.019+0000 7fcfef7fe640 1 --2- 172.21.15.16:0/2457343040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcff01001c0 0x7fcff0196010 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4006010 tx=0x7fcfe4004a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:59.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.019+0000 7fcfecff9640 1 -- 172.21.15.16:0/2457343040 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfe4004420 con 0x7fcff01001c0 2024-04-02T04:53:59.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.019+0000 7fcff5bf0640 1 -- 172.21.15.16:0/2457343040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcff019ae90 con 0x7fcff01001c0 2024-04-02T04:53:59.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.019+0000 7fcfecff9640 1 -- 172.21.15.16:0/2457343040 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcfe4003ab0 con 0x7fcff01001c0 2024-04-02T04:53:59.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.019+0000 7fcfecff9640 1 -- 172.21.15.16:0/2457343040 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfe4005520 con 0x7fcff01001c0 2024-04-02T04:53:59.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.019+0000 7fcff5bf0640 1 -- 172.21.15.16:0/2457343040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcff019b3f0 con 0x7fcff01001c0 2024-04-02T04:53:59.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.021+0000 7fcff5bf0640 1 -- 172.21.15.16:0/2457343040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcff0101690 con 0x7fcff01001c0 2024-04-02T04:53:59.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.021+0000 7fcfecff9640 1 -- 172.21.15.16:0/2457343040 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fcfe4003c20 con 0x7fcff01001c0 2024-04-02T04:53:59.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.022+0000 7fcfecff9640 1 --2- 172.21.15.16:0/2457343040 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fcfd806dce0 0x7fcfd80701a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:59.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.022+0000 7fcfecff9640 1 -- 172.21.15.16:0/2457343040 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fcfe40afe80 con 0x7fcff01001c0 2024-04-02T04:53:59.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.022+0000 7fcfeeffd640 1 --2- 172.21.15.16:0/2457343040 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fcfd806dce0 0x7fcfd80701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:59.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.022+0000 7fcfeeffd640 1 --2- 172.21.15.16:0/2457343040 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fcfd806dce0 0x7fcfd80701a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcff0069400 tx=0x7fcfe0005f90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:59.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.027+0000 7fcfecff9640 1 -- 172.21.15.16:0/2457343040 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fcfe4079370 con 0x7fcff01001c0 2024-04-02T04:53:59.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.210+0000 7fcff5bf0640 1 -- 172.21.15.16:0/2457343040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcff01084c0 con 0x7fcff01001c0 2024-04-02T04:53:59.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.210+0000 7fcfecff9640 1 -- 172.21.15.16:0/2457343040 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcfe4041020 con 0x7fcff01001c0 2024-04-02T04:53:59.215 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-02T04:53:59.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.212+0000 7fcff5bf0640 1 -- 172.21.15.16:0/2457343040 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fcfd806dce0 msgr2=0x7fcfd80701a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:59.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.212+0000 7fcff5bf0640 1 --2- 172.21.15.16:0/2457343040 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fcfd806dce0 0x7fcfd80701a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcff0069400 tx=0x7fcfe0005f90 comp rx=0 tx=0).stop 2024-04-02T04:53:59.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.212+0000 7fcff5bf0640 1 -- 172.21.15.16:0/2457343040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcff01001c0 msgr2=0x7fcff0196010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:59.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.212+0000 7fcff5bf0640 1 --2- 172.21.15.16:0/2457343040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcff01001c0 0x7fcff0196010 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4006010 tx=0x7fcfe4004a10 comp rx=0 tx=0).stop 2024-04-02T04:53:59.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.212+0000 7fcff5bf0640 1 -- 172.21.15.16:0/2457343040 shutdown_connections 2024-04-02T04:53:59.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.212+0000 7fcff5bf0640 1 --2- 172.21.15.16:0/2457343040 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fcfd806dce0 0x7fcfd80701a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:59.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.212+0000 7fcff5bf0640 1 --2- 172.21.15.16:0/2457343040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcff0196550 0x7fcff019a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:59.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.212+0000 7fcff5bf0640 1 --2- 172.21.15.16:0/2457343040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcff01001c0 0x7fcff0196010 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:59.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.212+0000 7fcff5bf0640 1 -- 172.21.15.16:0/2457343040 >> 172.21.15.16:0/2457343040 conn(0x7fcff00fba20 msgr2=0x7fcff00fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:59.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.212+0000 7fcff5bf0640 1 -- 172.21.15.16:0/2457343040 shutdown_connections 2024-04-02T04:53:59.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.212+0000 7fcff5bf0640 1 -- 172.21.15.16:0/2457343040 wait complete. 2024-04-02T04:53:59.228 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-02T04:53:59.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.330+0000 7fc3ef2c4640 1 -- 172.21.15.16:0/2508829615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e81001c0 msgr2=0x7fc3e81005a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:59.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.330+0000 7fc3ef2c4640 1 --2- 172.21.15.16:0/2508829615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e81001c0 0x7fc3e81005a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4009880 tx=0x7fc3e402bce0 comp rx=0 tx=0).stop 2024-04-02T04:53:59.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.330+0000 7fc3ef2c4640 1 -- 172.21.15.16:0/2508829615 shutdown_connections 2024-04-02T04:53:59.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.330+0000 7fc3ef2c4640 1 --2- 172.21.15.16:0/2508829615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3e8100b70 0x7fc3e8104bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:59.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.330+0000 7fc3ef2c4640 1 --2- 172.21.15.16:0/2508829615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e81001c0 0x7fc3e81005a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:59.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.330+0000 7fc3ef2c4640 1 -- 172.21.15.16:0/2508829615 >> 172.21.15.16:0/2508829615 conn(0x7fc3e80fba20 msgr2=0x7fc3e80fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:59.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.330+0000 7fc3ef2c4640 1 -- 172.21.15.16:0/2508829615 shutdown_connections 2024-04-02T04:53:59.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.330+0000 7fc3ef2c4640 1 -- 172.21.15.16:0/2508829615 wait complete. 2024-04-02T04:53:59.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.331+0000 7fc3ef2c4640 1 Processor -- start 2024-04-02T04:53:59.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.331+0000 7fc3ef2c4640 1 -- start start 2024-04-02T04:53:59.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.331+0000 7fc3ef2c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e81001c0 0x7fc3e8196010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:59.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.331+0000 7fc3ef2c4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3e8196550 0x7fc3e819a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:59.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.331+0000 7fc3ef2c4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e8196b50 con 0x7fc3e81001c0 2024-04-02T04:53:59.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.331+0000 7fc3ef2c4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e8196cc0 con 0x7fc3e8196550 2024-04-02T04:53:59.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.331+0000 7fc3ed060640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e81001c0 0x7fc3e8196010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:59.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.331+0000 7fc3ed060640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e81001c0 0x7fc3e8196010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40116/0 (socket says 172.21.15.16:40116) 2024-04-02T04:53:59.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.331+0000 7fc3ed060640 1 -- 172.21.15.16:0/2283431230 learned_addr learned my addr 172.21.15.16:0/2283431230 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:53:59.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.331+0000 7fc3ec85f640 1 --2- 172.21.15.16:0/2283431230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3e8196550 0x7fc3e819a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:59.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.331+0000 7fc3ed060640 1 -- 172.21.15.16:0/2283431230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3e8196550 msgr2=0x7fc3e819a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:59.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.331+0000 7fc3ed060640 1 --2- 172.21.15.16:0/2283431230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3e8196550 0x7fc3e819a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:59.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.331+0000 7fc3ed060640 1 -- 172.21.15.16:0/2283431230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e4009560 con 0x7fc3e81001c0 2024-04-02T04:53:59.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.332+0000 7fc3ed060640 1 --2- 172.21.15.16:0/2283431230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e81001c0 0x7fc3e8196010 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4006010 tx=0x7fc3e4004a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:59.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.333+0000 7fc3de7fc640 1 -- 172.21.15.16:0/2283431230 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3e4004420 con 0x7fc3e81001c0 2024-04-02T04:53:59.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.333+0000 7fc3de7fc640 1 -- 172.21.15.16:0/2283431230 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc3e4003ab0 con 0x7fc3e81001c0 2024-04-02T04:53:59.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.333+0000 7fc3ef2c4640 1 -- 172.21.15.16:0/2283431230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3e819ae90 con 0x7fc3e81001c0 2024-04-02T04:53:59.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.333+0000 7fc3de7fc640 1 -- 172.21.15.16:0/2283431230 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3e4005520 con 0x7fc3e81001c0 2024-04-02T04:53:59.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.333+0000 7fc3ef2c4640 1 -- 172.21.15.16:0/2283431230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3e819b3f0 con 0x7fc3e81001c0 2024-04-02T04:53:59.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.334+0000 7fc3ef2c4640 1 -- 172.21.15.16:0/2283431230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3e8101690 con 0x7fc3e81001c0 2024-04-02T04:53:59.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.334+0000 7fc3de7fc640 1 -- 172.21.15.16:0/2283431230 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fc3e4003c20 con 0x7fc3e81001c0 2024-04-02T04:53:59.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.335+0000 7fc3de7fc640 1 --2- 172.21.15.16:0/2283431230 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc3d406dcd0 0x7fc3d4070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:59.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.335+0000 7fc3de7fc640 1 -- 172.21.15.16:0/2283431230 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fc3e40afea0 con 0x7fc3e81001c0 2024-04-02T04:53:59.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.335+0000 7fc3ec85f640 1 --2- 172.21.15.16:0/2283431230 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc3d406dcd0 0x7fc3d4070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:59.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.336+0000 7fc3ec85f640 1 --2- 172.21.15.16:0/2283431230 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc3d406dcd0 0x7fc3d4070190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8067ac0 tx=0x7fc3d8005f90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:59.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.340+0000 7fc3de7fc640 1 -- 172.21.15.16:0/2283431230 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc3e4079390 con 0x7fc3e81001c0 2024-04-02T04:53:59.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.523+0000 7fc3ef2c4640 1 -- 172.21.15.16:0/2283431230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fc3e819b6d0 con 0x7fc3e81001c0 2024-04-02T04:53:59.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.524+0000 7fc3de7fc640 1 -- 172.21.15.16:0/2283431230 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7fc3e4041020 con 0x7fc3e81001c0 2024-04-02T04:53:59.528 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-02T04:53:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.526+0000 7fc3ef2c4640 1 -- 172.21.15.16:0/2283431230 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc3d406dcd0 msgr2=0x7fc3d4070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.526+0000 7fc3ef2c4640 1 --2- 172.21.15.16:0/2283431230 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc3d406dcd0 0x7fc3d4070190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8067ac0 tx=0x7fc3d8005f90 comp rx=0 tx=0).stop 2024-04-02T04:53:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.526+0000 7fc3ef2c4640 1 -- 172.21.15.16:0/2283431230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e81001c0 msgr2=0x7fc3e8196010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.526+0000 7fc3ef2c4640 1 --2- 172.21.15.16:0/2283431230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e81001c0 0x7fc3e8196010 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4006010 tx=0x7fc3e4004a10 comp rx=0 tx=0).stop 2024-04-02T04:53:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.526+0000 7fc3ef2c4640 1 -- 172.21.15.16:0/2283431230 shutdown_connections 2024-04-02T04:53:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.526+0000 7fc3ef2c4640 1 --2- 172.21.15.16:0/2283431230 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc3d406dcd0 0x7fc3d4070190 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.526+0000 7fc3ef2c4640 1 --2- 172.21.15.16:0/2283431230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3e8196550 0x7fc3e819a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.526+0000 7fc3ef2c4640 1 --2- 172.21.15.16:0/2283431230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e81001c0 0x7fc3e8196010 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.526+0000 7fc3ef2c4640 1 -- 172.21.15.16:0/2283431230 >> 172.21.15.16:0/2283431230 conn(0x7fc3e80fba20 msgr2=0x7fc3e80fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.526+0000 7fc3ef2c4640 1 -- 172.21.15.16:0/2283431230 shutdown_connections 2024-04-02T04:53:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.526+0000 7fc3ef2c4640 1 -- 172.21.15.16:0/2283431230 wait complete. 2024-04-02T04:53:59.540 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:59 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/3028358449' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T04:53:59.540 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:53:59 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/2457343040' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T04:53:59.542 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-02T04:53:59.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:59 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/3028358449' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T04:53:59.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:53:59 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/2457343040' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T04:53:59.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.643+0000 7f9b729ce640 1 -- 172.21.15.16:0/1870113272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b6c067e60 msgr2=0x7f9b6c104ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:59.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.643+0000 7f9b729ce640 1 --2- 172.21.15.16:0/1870113272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b6c067e60 0x7f9b6c104ba0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9b60009880 tx=0x7f9b6002bce0 comp rx=0 tx=0).stop 2024-04-02T04:53:59.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.643+0000 7f9b729ce640 1 -- 172.21.15.16:0/1870113272 shutdown_connections 2024-04-02T04:53:59.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.643+0000 7f9b729ce640 1 --2- 172.21.15.16:0/1870113272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b6c067e60 0x7f9b6c104ba0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:59.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.643+0000 7f9b729ce640 1 --2- 172.21.15.16:0/1870113272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b6c0674b0 0x7f9b6c067890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:59.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.643+0000 7f9b729ce640 1 -- 172.21.15.16:0/1870113272 >> 172.21.15.16:0/1870113272 conn(0x7f9b6c074960 msgr2=0x7f9b6c074d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:59.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.643+0000 7f9b729ce640 1 -- 172.21.15.16:0/1870113272 shutdown_connections 2024-04-02T04:53:59.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.643+0000 7f9b729ce640 1 -- 172.21.15.16:0/1870113272 wait complete. 2024-04-02T04:53:59.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.643+0000 7f9b729ce640 1 Processor -- start 2024-04-02T04:53:59.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.644+0000 7f9b729ce640 1 -- start start 2024-04-02T04:53:59.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.644+0000 7f9b729ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b6c0674b0 0x7f9b6c196110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:59.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.644+0000 7f9b729ce640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b6c196650 0x7f9b6c19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:59.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.644+0000 7f9b729ce640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6c196c50 con 0x7f9b6c0674b0 2024-04-02T04:53:59.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.644+0000 7f9b729ce640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6c196dc0 con 0x7f9b6c196650 2024-04-02T04:53:59.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.644+0000 7f9b6bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b6c0674b0 0x7f9b6c196110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:59.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.644+0000 7f9b6bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b6c0674b0 0x7f9b6c196110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40142/0 (socket says 172.21.15.16:40142) 2024-04-02T04:53:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.644+0000 7f9b6bfff640 1 -- 172.21.15.16:0/1974733991 learned_addr learned my addr 172.21.15.16:0/1974733991 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:53:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.644+0000 7f9b6b7fe640 1 --2- 172.21.15.16:0/1974733991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b6c196650 0x7f9b6c19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.644+0000 7f9b6bfff640 1 -- 172.21.15.16:0/1974733991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b6c196650 msgr2=0x7f9b6c19a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.644+0000 7f9b6bfff640 1 --2- 172.21.15.16:0/1974733991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b6c196650 0x7f9b6c19a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.644+0000 7f9b6bfff640 1 -- 172.21.15.16:0/1974733991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b60009560 con 0x7f9b6c0674b0 2024-04-02T04:53:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.644+0000 7f9b6bfff640 1 --2- 172.21.15.16:0/1974733991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b6c0674b0 0x7f9b6c196110 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c00b450 tx=0x7f9b5c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.645+0000 7f9b697fa640 1 -- 172.21.15.16:0/1974733991 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b5c011860 con 0x7f9b6c0674b0 2024-04-02T04:53:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.645+0000 7f9b697fa640 1 -- 172.21.15.16:0/1974733991 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b5c011e80 con 0x7f9b6c0674b0 2024-04-02T04:53:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.645+0000 7f9b729ce640 1 -- 172.21.15.16:0/1974733991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b6c19aff0 con 0x7f9b6c0674b0 2024-04-02T04:53:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.645+0000 7f9b697fa640 1 -- 172.21.15.16:0/1974733991 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b5c00f570 con 0x7f9b6c0674b0 2024-04-02T04:53:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.645+0000 7f9b729ce640 1 -- 172.21.15.16:0/1974733991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b6c19b550 con 0x7f9b6c0674b0 2024-04-02T04:53:59.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.646+0000 7f9b729ce640 1 -- 172.21.15.16:0/1974733991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b6c101680 con 0x7f9b6c0674b0 2024-04-02T04:53:59.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.646+0000 7f9b697fa640 1 -- 172.21.15.16:0/1974733991 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f9b5c0103e0 con 0x7f9b6c0674b0 2024-04-02T04:53:59.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.647+0000 7f9b697fa640 1 --2- 172.21.15.16:0/1974733991 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f9b5406dcd0 0x7f9b54070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:59.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.647+0000 7f9b6b7fe640 1 --2- 172.21.15.16:0/1974733991 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f9b5406dcd0 0x7f9b54070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:59.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.647+0000 7f9b697fa640 1 -- 172.21.15.16:0/1974733991 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f9b5c08db20 con 0x7f9b6c0674b0 2024-04-02T04:53:59.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.647+0000 7f9b6b7fe640 1 --2- 172.21.15.16:0/1974733991 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f9b5406dcd0 0x7f9b54070190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9b60009230 tx=0x7f9b60005fd0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.651+0000 7f9b697fa640 1 -- 172.21.15.16:0/1974733991 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9b5c057010 con 0x7f9b6c0674b0 2024-04-02T04:53:59.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.835+0000 7f9b729ce640 1 -- 172.21.15.16:0/1974733991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f9b6c19b7d0 con 0x7f9b6c0674b0 2024-04-02T04:53:59.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.835+0000 7f9b697fa640 1 -- 172.21.15.16:0/1974733991 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7f9b5c05a6d0 con 0x7f9b6c0674b0 2024-04-02T04:53:59.840 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-02T04:53:59.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.837+0000 7f9b729ce640 1 -- 172.21.15.16:0/1974733991 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f9b5406dcd0 msgr2=0x7f9b54070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:59.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.837+0000 7f9b729ce640 1 --2- 172.21.15.16:0/1974733991 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f9b5406dcd0 0x7f9b54070190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9b60009230 tx=0x7f9b60005fd0 comp rx=0 tx=0).stop 2024-04-02T04:53:59.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.837+0000 7f9b729ce640 1 -- 172.21.15.16:0/1974733991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b6c0674b0 msgr2=0x7f9b6c196110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:59.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.837+0000 7f9b729ce640 1 --2- 172.21.15.16:0/1974733991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b6c0674b0 0x7f9b6c196110 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c00b450 tx=0x7f9b5c00b910 comp rx=0 tx=0).stop 2024-04-02T04:53:59.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.837+0000 7f9b729ce640 1 -- 172.21.15.16:0/1974733991 shutdown_connections 2024-04-02T04:53:59.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.837+0000 7f9b729ce640 1 --2- 172.21.15.16:0/1974733991 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f9b5406dcd0 0x7f9b54070190 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:59.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.837+0000 7f9b729ce640 1 --2- 172.21.15.16:0/1974733991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b6c196650 0x7f9b6c19a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:59.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.837+0000 7f9b729ce640 1 --2- 172.21.15.16:0/1974733991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b6c0674b0 0x7f9b6c196110 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:59.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.837+0000 7f9b729ce640 1 -- 172.21.15.16:0/1974733991 >> 172.21.15.16:0/1974733991 conn(0x7f9b6c074960 msgr2=0x7f9b6c103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:59.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.837+0000 7f9b729ce640 1 -- 172.21.15.16:0/1974733991 shutdown_connections 2024-04-02T04:53:59.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.837+0000 7f9b729ce640 1 -- 172.21.15.16:0/1974733991 wait complete. 2024-04-02T04:53:59.853 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T04:53:59.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.954+0000 7f922dcc4640 1 -- 172.21.15.16:0/1532048024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9228101a40 msgr2=0x7f9228109f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:59.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.954+0000 7f922dcc4640 1 --2- 172.21.15.16:0/1532048024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9228101a40 0x7f9228109f20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9210009880 tx=0x7f921002bd70 comp rx=0 tx=0).stop 2024-04-02T04:53:59.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.954+0000 7f922dcc4640 1 -- 172.21.15.16:0/1532048024 shutdown_connections 2024-04-02T04:53:59.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.954+0000 7f922dcc4640 1 --2- 172.21.15.16:0/1532048024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9228101a40 0x7f9228109f20 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:59.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.954+0000 7f922dcc4640 1 --2- 172.21.15.16:0/1532048024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9228101120 0x7f9228101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:59.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.954+0000 7f922dcc4640 1 -- 172.21.15.16:0/1532048024 >> 172.21.15.16:0/1532048024 conn(0x7f92280fa980 msgr2=0x7f92280fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:53:59.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.954+0000 7f922dcc4640 1 -- 172.21.15.16:0/1532048024 shutdown_connections 2024-04-02T04:53:59.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.954+0000 7f922dcc4640 1 -- 172.21.15.16:0/1532048024 wait complete. 2024-04-02T04:53:59.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.954+0000 7f922dcc4640 1 Processor -- start 2024-04-02T04:53:59.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.955+0000 7f922dcc4640 1 -- start start 2024-04-02T04:53:59.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.955+0000 7f922dcc4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9228101120 0x7f9228195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:59.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.955+0000 7f922dcc4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9228101a40 0x7f92281964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:59.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.955+0000 7f922dcc4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9228196b80 con 0x7f9228101120 2024-04-02T04:53:59.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.955+0000 7f922dcc4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922819a890 con 0x7f9228101a40 2024-04-02T04:53:59.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.955+0000 7f92277fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9228101120 0x7f9228195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:59.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.955+0000 7f92277fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9228101120 0x7f9228195fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40158/0 (socket says 172.21.15.16:40158) 2024-04-02T04:53:59.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.955+0000 7f92277fe640 1 -- 172.21.15.16:0/2432345867 learned_addr learned my addr 172.21.15.16:0/2432345867 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:53:59.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.955+0000 7f921effd640 1 --2- 172.21.15.16:0/2432345867 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9228101a40 0x7f92281964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:59.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.955+0000 7f92277fe640 1 -- 172.21.15.16:0/2432345867 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9228101a40 msgr2=0x7f92281964f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:53:59.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.955+0000 7f92277fe640 1 --2- 172.21.15.16:0/2432345867 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9228101a40 0x7f92281964f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:53:59.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.955+0000 7f92277fe640 1 -- 172.21.15.16:0/2432345867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9210009560 con 0x7f9228101120 2024-04-02T04:53:59.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.956+0000 7f92277fe640 1 --2- 172.21.15.16:0/2432345867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9228101120 0x7f9228195fb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f921800e970 tx=0x7f921800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:59.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.956+0000 7f92257fa640 1 -- 172.21.15.16:0/2432345867 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f921800cc90 con 0x7f9228101120 2024-04-02T04:53:59.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.956+0000 7f922dcc4640 1 -- 172.21.15.16:0/2432345867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f922819ab70 con 0x7f9228101120 2024-04-02T04:53:59.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.956+0000 7f92257fa640 1 -- 172.21.15.16:0/2432345867 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f921800cdf0 con 0x7f9228101120 2024-04-02T04:53:59.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.956+0000 7f92257fa640 1 -- 172.21.15.16:0/2432345867 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9218010430 con 0x7f9228101120 2024-04-02T04:53:59.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.956+0000 7f922dcc4640 1 -- 172.21.15.16:0/2432345867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f922819b0a0 con 0x7f9228101120 2024-04-02T04:53:59.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.957+0000 7f922dcc4640 1 -- 172.21.15.16:0/2432345867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9228068cc0 con 0x7f9228101120 2024-04-02T04:53:59.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.957+0000 7f92257fa640 1 -- 172.21.15.16:0/2432345867 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f92180046c0 con 0x7f9228101120 2024-04-02T04:53:59.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.958+0000 7f92257fa640 1 --2- 172.21.15.16:0/2432345867 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f921406dce0 0x7f92140701a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:53:59.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.958+0000 7f92257fa640 1 -- 172.21.15.16:0/2432345867 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f9218014070 con 0x7f9228101120 2024-04-02T04:53:59.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.958+0000 7f921effd640 1 --2- 172.21.15.16:0/2432345867 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f921406dce0 0x7f92140701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:53:59.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.958+0000 7f921effd640 1 --2- 172.21.15.16:0/2432345867 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f921406dce0 0x7f92140701a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9210006010 tx=0x7f9210005d50 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:53:59.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:53:59.962+0000 7f92257fa640 1 -- 172.21.15.16:0/2432345867 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9218057620 con 0x7f9228101120 2024-04-02T04:54:00.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.182+0000 7f922dcc4640 1 -- 172.21.15.16:0/2432345867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f922804e030 con 0x7f9228101120 2024-04-02T04:54:00.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.183+0000 7f92257fa640 1 -- 172.21.15.16:0/2432345867 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+5198 (secure 0 0 0) 0x7f921805ace0 con 0x7f9228101120 2024-04-02T04:54:00.188 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:54:00.188 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi016.fyzrnm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6829/782430688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":782430688},{"type":"v1","addr":"172.21.15.16:6829","nonce":782430688}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":44359,"name":"cephfs.smithi027.zwrfoo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6825/1016194753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1016194753},{"type":"v1","addr":"172.21.15.27:6825","nonce":1016194753}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":44383,"name":"cephfs.smithi027.sdublg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6827/1507844210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1507844210},{"type":"v1","addr":"172.21.15.27:6827","nonce":1507844210}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:39:30.740168+0000","modified":"2024-04-02T04:50:20.104880+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34404":{"gid":34404,"name":"cephfs.smithi016.uhgkgg","rank":0,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.16:6827/81744708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":81744708},{"type":"v1","addr":"172.21.15.16:6827","nonce":81744708}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34404,"qdb_cluster":[34404]},"id":1}]} 2024-04-02T04:54:00.188 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 29 2024-04-02T04:54:00.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.185+0000 7f922dcc4640 1 -- 172.21.15.16:0/2432345867 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f921406dce0 msgr2=0x7f92140701a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:00.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.185+0000 7f922dcc4640 1 --2- 172.21.15.16:0/2432345867 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f921406dce0 0x7f92140701a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9210006010 tx=0x7f9210005d50 comp rx=0 tx=0).stop 2024-04-02T04:54:00.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.185+0000 7f922dcc4640 1 -- 172.21.15.16:0/2432345867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9228101120 msgr2=0x7f9228195fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:00.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.185+0000 7f922dcc4640 1 --2- 172.21.15.16:0/2432345867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9228101120 0x7f9228195fb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f921800e970 tx=0x7f921800ed70 comp rx=0 tx=0).stop 2024-04-02T04:54:00.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.185+0000 7f922dcc4640 1 -- 172.21.15.16:0/2432345867 shutdown_connections 2024-04-02T04:54:00.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.185+0000 7f922dcc4640 1 --2- 172.21.15.16:0/2432345867 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f921406dce0 0x7f92140701a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:00.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.185+0000 7f922dcc4640 1 --2- 172.21.15.16:0/2432345867 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9228101a40 0x7f92281964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:00.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.185+0000 7f922dcc4640 1 --2- 172.21.15.16:0/2432345867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9228101120 0x7f9228195fb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:00.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.185+0000 7f922dcc4640 1 -- 172.21.15.16:0/2432345867 >> 172.21.15.16:0/2432345867 conn(0x7f92280fa980 msgr2=0x7f9228102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:00.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.185+0000 7f922dcc4640 1 -- 172.21.15.16:0/2432345867 shutdown_connections 2024-04-02T04:54:00.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.185+0000 7f922dcc4640 1 -- 172.21.15.16:0/2432345867 wait complete. 2024-04-02T04:54:00.202 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 15, 'max_mds': 1, 'flags': 18} 2024-04-02T04:54:00.202 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 16 2024-04-02T04:54:00.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.302+0000 7fe345d0a640 1 -- 172.21.15.16:0/3349286584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3400ffb70 msgr2=0x7fe340103aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:00.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.302+0000 7fe345d0a640 1 --2- 172.21.15.16:0/3349286584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3400ffb70 0x7fe340103aa0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe334009880 tx=0x7fe33402bce0 comp rx=0 tx=0).stop 2024-04-02T04:54:00.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.302+0000 7fe345d0a640 1 -- 172.21.15.16:0/3349286584 shutdown_connections 2024-04-02T04:54:00.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.302+0000 7fe345d0a640 1 --2- 172.21.15.16:0/3349286584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3400ffb70 0x7fe340103aa0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:00.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.302+0000 7fe345d0a640 1 --2- 172.21.15.16:0/3349286584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3400ff1c0 0x7fe3400ff5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:00.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.302+0000 7fe345d0a640 1 -- 172.21.15.16:0/3349286584 >> 172.21.15.16:0/3349286584 conn(0x7fe340074960 msgr2=0x7fe340074d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:00.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.302+0000 7fe345d0a640 1 -- 172.21.15.16:0/3349286584 shutdown_connections 2024-04-02T04:54:00.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.302+0000 7fe345d0a640 1 -- 172.21.15.16:0/3349286584 wait complete. 2024-04-02T04:54:00.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.303+0000 7fe345d0a640 1 Processor -- start 2024-04-02T04:54:00.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.303+0000 7fe345d0a640 1 -- start start 2024-04-02T04:54:00.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.303+0000 7fe345d0a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3400ff1c0 0x7fe340196e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:00.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.303+0000 7fe345d0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3401973b0 0x7fe340197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:00.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.303+0000 7fe345d0a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34019fee0 con 0x7fe3401973b0 2024-04-02T04:54:00.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.303+0000 7fe345d0a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3401a0020 con 0x7fe3400ff1c0 2024-04-02T04:54:00.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.303+0000 7fe33effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3401973b0 0x7fe340197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:00.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.303+0000 7fe33effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3401973b0 0x7fe340197810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40176/0 (socket says 172.21.15.16:40176) 2024-04-02T04:54:00.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.303+0000 7fe33effd640 1 -- 172.21.15.16:0/820295040 learned_addr learned my addr 172.21.15.16:0/820295040 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:54:00.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.303+0000 7fe33f7fe640 1 --2- 172.21.15.16:0/820295040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3400ff1c0 0x7fe340196e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:00.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.303+0000 7fe33effd640 1 -- 172.21.15.16:0/820295040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3400ff1c0 msgr2=0x7fe340196e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:00.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.303+0000 7fe33effd640 1 --2- 172.21.15.16:0/820295040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3400ff1c0 0x7fe340196e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:00.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.303+0000 7fe33effd640 1 -- 172.21.15.16:0/820295040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe334009560 con 0x7fe3401973b0 2024-04-02T04:54:00.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.304+0000 7fe33effd640 1 --2- 172.21.15.16:0/820295040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3401973b0 0x7fe340197810 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe334036040 tx=0x7fe33402c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:00.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.304+0000 7fe33cff9640 1 -- 172.21.15.16:0/820295040 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe334004420 con 0x7fe3401973b0 2024-04-02T04:54:00.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.304+0000 7fe33cff9640 1 -- 172.21.15.16:0/820295040 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe334003ab0 con 0x7fe3401973b0 2024-04-02T04:54:00.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.304+0000 7fe345d0a640 1 -- 172.21.15.16:0/820295040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3401a0160 con 0x7fe3401973b0 2024-04-02T04:54:00.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.304+0000 7fe33cff9640 1 -- 172.21.15.16:0/820295040 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe334005490 con 0x7fe3401973b0 2024-04-02T04:54:00.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.304+0000 7fe345d0a640 1 -- 172.21.15.16:0/820295040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3401a06c0 con 0x7fe3401973b0 2024-04-02T04:54:00.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.305+0000 7fe345d0a640 1 -- 172.21.15.16:0/820295040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe340100580 con 0x7fe3401973b0 2024-04-02T04:54:00.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.305+0000 7fe33cff9640 1 -- 172.21.15.16:0/820295040 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fe334003c20 con 0x7fe3401973b0 2024-04-02T04:54:00.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.306+0000 7fe33cff9640 1 --2- 172.21.15.16:0/820295040 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fe32806dcd0 0x7fe328070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:00.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.306+0000 7fe33cff9640 1 -- 172.21.15.16:0/820295040 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fe3340afba0 con 0x7fe3401973b0 2024-04-02T04:54:00.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.306+0000 7fe33f7fe640 1 --2- 172.21.15.16:0/820295040 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fe32806dcd0 0x7fe328070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:00.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.306+0000 7fe33f7fe640 1 --2- 172.21.15.16:0/820295040 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fe32806dcd0 0x7fe328070190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe330009bd0 tx=0x7fe330005f90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:00.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.310+0000 7fe33cff9640 1 -- 172.21.15.16:0/820295040 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe3340791c0 con 0x7fe3401973b0 2024-04-02T04:54:00.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.534+0000 7fe345d0a640 1 -- 172.21.15.16:0/820295040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 16, "format": "json"} v 0) v1 -- 0x7fe3400ffb70 con 0x7fe3401973b0 2024-04-02T04:54:00.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.534+0000 7fe33cff9640 1 -- 172.21.15.16:0/820295040 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 16, "format": "json"}]=0 dumped fsmap epoch 16 v29) v1 ==== 107+0+4078 (secure 0 0 0) 0x7fe334041090 con 0x7fe3401973b0 2024-04-02T04:54:00.539 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:54:00.539 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14488,"name":"cephfs.smithi016.fyzrnm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.16:6829/2578719859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2578719859},{"type":"v1","addr":"172.21.15.16:6829","nonce":2578719859}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24295,"name":"cephfs.smithi027.sdublg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.27:6827/4177192740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":4177192740},{"type":"v1","addr":"172.21.15.27:6827","nonce":4177192740}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:39:30.740168+0000","modified":"2024-04-02T04:39:49.875319+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24285":{"gid":24285,"name":"cephfs.smithi027.zwrfoo","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6825/1461455976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1461455976},{"type":"v1","addr":"172.21.15.27:6825","nonce":1461455976}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-02T04:54:00.539 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 16 2024-04-02T04:54:00.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.536+0000 7fe345d0a640 1 -- 172.21.15.16:0/820295040 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fe32806dcd0 msgr2=0x7fe328070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:00.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.536+0000 7fe345d0a640 1 --2- 172.21.15.16:0/820295040 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fe32806dcd0 0x7fe328070190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe330009bd0 tx=0x7fe330005f90 comp rx=0 tx=0).stop 2024-04-02T04:54:00.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.536+0000 7fe345d0a640 1 -- 172.21.15.16:0/820295040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3401973b0 msgr2=0x7fe340197810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:00.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.536+0000 7fe345d0a640 1 --2- 172.21.15.16:0/820295040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3401973b0 0x7fe340197810 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe334036040 tx=0x7fe33402c6f0 comp rx=0 tx=0).stop 2024-04-02T04:54:00.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.536+0000 7fe345d0a640 1 -- 172.21.15.16:0/820295040 shutdown_connections 2024-04-02T04:54:00.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.536+0000 7fe345d0a640 1 --2- 172.21.15.16:0/820295040 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fe32806dcd0 0x7fe328070190 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:00.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.536+0000 7fe345d0a640 1 --2- 172.21.15.16:0/820295040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3401973b0 0x7fe340197810 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:00.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.536+0000 7fe345d0a640 1 --2- 172.21.15.16:0/820295040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3400ff1c0 0x7fe340196e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:00.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.536+0000 7fe345d0a640 1 -- 172.21.15.16:0/820295040 >> 172.21.15.16:0/820295040 conn(0x7fe340074960 msgr2=0x7fe340102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:00.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.537+0000 7fe345d0a640 1 -- 172.21.15.16:0/820295040 shutdown_connections 2024-04-02T04:54:00.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.537+0000 7fe345d0a640 1 -- 172.21.15.16:0/820295040 wait complete. 2024-04-02T04:54:00.551 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:00 smithi016 ceph-mon[93230]: pgmap v329: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:00.551 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:00 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/2283431230' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-02T04:54:00.551 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:00 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/1974733991' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-02T04:54:00.551 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:00 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/2432345867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T04:54:00.553 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 17 2024-04-02T04:54:00.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:00 smithi027 ceph-mon[83036]: pgmap v329: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:00.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:00 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/2283431230' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-02T04:54:00.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:00 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/1974733991' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-02T04:54:00.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:00 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/2432345867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T04:54:00.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.651+0000 7f999ce51640 1 -- 172.21.15.16:0/5590146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99980ff1c0 msgr2=0x7f99980ff5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:00.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.651+0000 7f999ce51640 1 --2- 172.21.15.16:0/5590146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99980ff1c0 0x7f99980ff5a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9988009880 tx=0x7f998802bce0 comp rx=0 tx=0).stop 2024-04-02T04:54:00.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.651+0000 7f999ce51640 1 -- 172.21.15.16:0/5590146 shutdown_connections 2024-04-02T04:54:00.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.651+0000 7f999ce51640 1 --2- 172.21.15.16:0/5590146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99980ffb70 0x7f9998103aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:00.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.651+0000 7f999ce51640 1 --2- 172.21.15.16:0/5590146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99980ff1c0 0x7f99980ff5a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:00.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.651+0000 7f999ce51640 1 -- 172.21.15.16:0/5590146 >> 172.21.15.16:0/5590146 conn(0x7f9998074960 msgr2=0x7f9998074d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:00.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.651+0000 7f999ce51640 1 -- 172.21.15.16:0/5590146 shutdown_connections 2024-04-02T04:54:00.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.651+0000 7f999ce51640 1 -- 172.21.15.16:0/5590146 wait complete. 2024-04-02T04:54:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.652+0000 7f999ce51640 1 Processor -- start 2024-04-02T04:54:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.652+0000 7f999ce51640 1 -- start start 2024-04-02T04:54:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.652+0000 7f999ce51640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99980ff1c0 0x7f9998194f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.652+0000 7f999ce51640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99981954b0 0x7f9998199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.652+0000 7f999ce51640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9998195ab0 con 0x7f99980ff1c0 2024-04-02T04:54:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.652+0000 7f999ce51640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9998195c20 con 0x7f99981954b0 2024-04-02T04:54:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.652+0000 7f999659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99980ff1c0 0x7f9998194f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.652+0000 7f999659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99980ff1c0 0x7f9998194f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40196/0 (socket says 172.21.15.16:40196) 2024-04-02T04:54:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.652+0000 7f999659c640 1 -- 172.21.15.16:0/529522416 learned_addr learned my addr 172.21.15.16:0/529522416 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:54:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.652+0000 7f998dd9b640 1 --2- 172.21.15.16:0/529522416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99981954b0 0x7f9998199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.653+0000 7f999659c640 1 -- 172.21.15.16:0/529522416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99981954b0 msgr2=0x7f9998199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.653+0000 7f999659c640 1 --2- 172.21.15.16:0/529522416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99981954b0 0x7f9998199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.653+0000 7f999659c640 1 -- 172.21.15.16:0/529522416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9988009560 con 0x7f99980ff1c0 2024-04-02T04:54:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.653+0000 7f999659c640 1 --2- 172.21.15.16:0/529522416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99980ff1c0 0x7f9998194f70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9988006010 tx=0x7f998802cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.653+0000 7f998ffff640 1 -- 172.21.15.16:0/529522416 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99880044a0 con 0x7f99980ff1c0 2024-04-02T04:54:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.653+0000 7f998ffff640 1 -- 172.21.15.16:0/529522416 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9988003ab0 con 0x7f99980ff1c0 2024-04-02T04:54:00.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.653+0000 7f999ce51640 1 -- 172.21.15.16:0/529522416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9998199da0 con 0x7f99980ff1c0 2024-04-02T04:54:00.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.653+0000 7f998ffff640 1 -- 172.21.15.16:0/529522416 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9988005590 con 0x7f99980ff1c0 2024-04-02T04:54:00.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.653+0000 7f999ce51640 1 -- 172.21.15.16:0/529522416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f999819a300 con 0x7f99980ff1c0 2024-04-02T04:54:00.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.654+0000 7f999ce51640 1 -- 172.21.15.16:0/529522416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9998100580 con 0x7f99980ff1c0 2024-04-02T04:54:00.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.654+0000 7f998ffff640 1 -- 172.21.15.16:0/529522416 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f9988004dd0 con 0x7f99980ff1c0 2024-04-02T04:54:00.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.655+0000 7f998ffff640 1 --2- 172.21.15.16:0/529522416 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f998406dcd0 0x7f9984070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:00.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.655+0000 7f998ffff640 1 -- 172.21.15.16:0/529522416 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f99880afe70 con 0x7f99980ff1c0 2024-04-02T04:54:00.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.655+0000 7f998dd9b640 1 --2- 172.21.15.16:0/529522416 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f998406dcd0 0x7f9984070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:00.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.655+0000 7f998dd9b640 1 --2- 172.21.15.16:0/529522416 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f998406dcd0 0x7f9984070190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9980005fd0 tx=0x7f998000b390 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:00.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.659+0000 7f998ffff640 1 -- 172.21.15.16:0/529522416 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9988079410 con 0x7f99980ff1c0 2024-04-02T04:54:00.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.880+0000 7f999ce51640 1 -- 172.21.15.16:0/529522416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 17, "format": "json"} v 0) v1 -- 0x7f9998107380 con 0x7f99980ff1c0 2024-04-02T04:54:00.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.881+0000 7f998ffff640 1 -- 172.21.15.16:0/529522416 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 17, "format": "json"}]=0 dumped fsmap epoch 17 v29) v1 ==== 107+0+4919 (secure 0 0 0) 0x7f998807cad0 con 0x7f99980ff1c0 2024-04-02T04:54:00.886 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:54:00.886 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14488,"name":"cephfs.smithi016.fyzrnm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.16:6829/2578719859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2578719859},{"type":"v1","addr":"172.21.15.16:6829","nonce":2578719859}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24295,"name":"cephfs.smithi027.sdublg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.27:6827/4177192740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":4177192740},{"type":"v1","addr":"172.21.15.27:6827","nonce":4177192740}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34404,"name":"cephfs.smithi016.uhgkgg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6827/81744708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":81744708},{"type":"v1","addr":"172.21.15.16:6827","nonce":81744708}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:39:30.740168+0000","modified":"2024-04-02T04:39:49.875319+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24285":{"gid":24285,"name":"cephfs.smithi027.zwrfoo","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6825/1461455976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1461455976},{"type":"v1","addr":"172.21.15.27:6825","nonce":1461455976}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-02T04:54:00.886 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 17 2024-04-02T04:54:00.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.884+0000 7f999ce51640 1 -- 172.21.15.16:0/529522416 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f998406dcd0 msgr2=0x7f9984070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:00.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.884+0000 7f999ce51640 1 --2- 172.21.15.16:0/529522416 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f998406dcd0 0x7f9984070190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9980005fd0 tx=0x7f998000b390 comp rx=0 tx=0).stop 2024-04-02T04:54:00.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.884+0000 7f999ce51640 1 -- 172.21.15.16:0/529522416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99980ff1c0 msgr2=0x7f9998194f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:00.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.884+0000 7f999ce51640 1 --2- 172.21.15.16:0/529522416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99980ff1c0 0x7f9998194f70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9988006010 tx=0x7f998802cea0 comp rx=0 tx=0).stop 2024-04-02T04:54:00.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.884+0000 7f999ce51640 1 -- 172.21.15.16:0/529522416 shutdown_connections 2024-04-02T04:54:00.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.884+0000 7f999ce51640 1 --2- 172.21.15.16:0/529522416 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f998406dcd0 0x7f9984070190 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:00.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.884+0000 7f999ce51640 1 --2- 172.21.15.16:0/529522416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99981954b0 0x7f9998199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:00.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.884+0000 7f999ce51640 1 --2- 172.21.15.16:0/529522416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99980ff1c0 0x7f9998194f70 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:00.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.884+0000 7f999ce51640 1 -- 172.21.15.16:0/529522416 >> 172.21.15.16:0/529522416 conn(0x7f9998074960 msgr2=0x7f9998105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:00.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.884+0000 7f999ce51640 1 -- 172.21.15.16:0/529522416 shutdown_connections 2024-04-02T04:54:00.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:00.884+0000 7f999ce51640 1 -- 172.21.15.16:0/529522416 wait complete. 2024-04-02T04:54:00.900 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 18 2024-04-02T04:54:01.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.003+0000 7fd014759640 1 -- 172.21.15.16:0/3408710119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd00c101150 msgr2=0x7fd00c101530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:01.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.003+0000 7fd014759640 1 --2- 172.21.15.16:0/3408710119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd00c101150 0x7fd00c101530 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd0080098b0 tx=0x7fd00802bd60 comp rx=0 tx=0).stop 2024-04-02T04:54:01.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.003+0000 7fd014759640 1 -- 172.21.15.16:0/3408710119 shutdown_connections 2024-04-02T04:54:01.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.003+0000 7fd014759640 1 --2- 172.21.15.16:0/3408710119 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd00c101a70 0x7fd00c109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:01.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.003+0000 7fd014759640 1 --2- 172.21.15.16:0/3408710119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd00c101150 0x7fd00c101530 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:01.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.003+0000 7fd014759640 1 -- 172.21.15.16:0/3408710119 >> 172.21.15.16:0/3408710119 conn(0x7fd00c0fa930 msgr2=0x7fd00c0fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:01.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.003+0000 7fd014759640 1 -- 172.21.15.16:0/3408710119 shutdown_connections 2024-04-02T04:54:01.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.003+0000 7fd014759640 1 -- 172.21.15.16:0/3408710119 wait complete. 2024-04-02T04:54:01.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.004+0000 7fd014759640 1 Processor -- start 2024-04-02T04:54:01.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.004+0000 7fd014759640 1 -- start start 2024-04-02T04:54:01.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.004+0000 7fd014759640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd00c101150 0x7fd00c196010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:01.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.004+0000 7fd014759640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd00c101a70 0x7fd00c196550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:01.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.004+0000 7fd014759640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c196be0 con 0x7fd00c101a70 2024-04-02T04:54:01.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.004+0000 7fd014759640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c19a8f0 con 0x7fd00c101150 2024-04-02T04:54:01.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.004+0000 7fd011cf4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd00c101a70 0x7fd00c196550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:01.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.004+0000 7fd011cf4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd00c101a70 0x7fd00c196550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40210/0 (socket says 172.21.15.16:40210) 2024-04-02T04:54:01.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.004+0000 7fd011cf4640 1 -- 172.21.15.16:0/2563357781 learned_addr learned my addr 172.21.15.16:0/2563357781 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:54:01.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.004+0000 7fd0124f5640 1 --2- 172.21.15.16:0/2563357781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd00c101150 0x7fd00c196010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:01.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.004+0000 7fd011cf4640 1 -- 172.21.15.16:0/2563357781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd00c101150 msgr2=0x7fd00c196010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:01.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.004+0000 7fd011cf4640 1 --2- 172.21.15.16:0/2563357781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd00c101150 0x7fd00c196010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:01.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.004+0000 7fd011cf4640 1 -- 172.21.15.16:0/2563357781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd008009560 con 0x7fd00c101a70 2024-04-02T04:54:01.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.004+0000 7fd0124f5640 1 --2- 172.21.15.16:0/2563357781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd00c101150 0x7fd00c196010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T04:54:01.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.005+0000 7fd011cf4640 1 --2- 172.21.15.16:0/2563357781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd00c101a70 0x7fd00c196550 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fcffc00d6f0 tx=0x7fcffc00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:01.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.005+0000 7fd0037fe640 1 -- 172.21.15.16:0/2563357781 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcffc009a30 con 0x7fd00c101a70 2024-04-02T04:54:01.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.005+0000 7fd0037fe640 1 -- 172.21.15.16:0/2563357781 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcffc0054e0 con 0x7fd00c101a70 2024-04-02T04:54:01.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.005+0000 7fd014759640 1 -- 172.21.15.16:0/2563357781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd00c19abd0 con 0x7fd00c101a70 2024-04-02T04:54:01.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.005+0000 7fd0037fe640 1 -- 172.21.15.16:0/2563357781 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcffc010460 con 0x7fd00c101a70 2024-04-02T04:54:01.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.005+0000 7fd014759640 1 -- 172.21.15.16:0/2563357781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd00c19b130 con 0x7fd00c101a70 2024-04-02T04:54:01.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.006+0000 7fd014759640 1 -- 172.21.15.16:0/2563357781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd00c068ce0 con 0x7fd00c101a70 2024-04-02T04:54:01.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.007+0000 7fd0037fe640 1 -- 172.21.15.16:0/2563357781 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fcffc009b90 con 0x7fd00c101a70 2024-04-02T04:54:01.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.007+0000 7fd0037fe640 1 --2- 172.21.15.16:0/2563357781 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fcff806dcd0 0x7fcff8070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:01.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.007+0000 7fd0037fe640 1 -- 172.21.15.16:0/2563357781 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fcffc08d770 con 0x7fd00c101a70 2024-04-02T04:54:01.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.007+0000 7fd0124f5640 1 --2- 172.21.15.16:0/2563357781 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fcff806dcd0 0x7fcff8070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:01.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.008+0000 7fd0124f5640 1 --2- 172.21.15.16:0/2563357781 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fcff806dcd0 0x7fcff8070190 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd00802ca70 tx=0x7fd00802c8a0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:01.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.012+0000 7fd0037fe640 1 -- 172.21.15.16:0/2563357781 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fcffc056c60 con 0x7fd00c101a70 2024-04-02T04:54:01.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.232+0000 7fd014759640 1 -- 172.21.15.16:0/2563357781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7fd00c1047c0 con 0x7fd00c101a70 2024-04-02T04:54:01.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.232+0000 7fd0037fe640 1 -- 172.21.15.16:0/2563357781 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v29) v1 ==== 107+0+4140 (secure 0 0 0) 0x7fcffc05a320 con 0x7fd00c101a70 2024-04-02T04:54:01.237 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:54:01.237 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24295,"name":"cephfs.smithi027.sdublg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.27:6827/4177192740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":4177192740},{"type":"v1","addr":"172.21.15.27:6827","nonce":4177192740}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34404,"name":"cephfs.smithi016.uhgkgg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6827/81744708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":81744708},{"type":"v1","addr":"172.21.15.16:6827","nonce":81744708}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:39:30.740168+0000","modified":"2024-04-02T04:39:49.875319+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24285":{"gid":24285,"name":"cephfs.smithi027.zwrfoo","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6825/1461455976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1461455976},{"type":"v1","addr":"172.21.15.27:6825","nonce":1461455976}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-02T04:54:01.237 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 18 2024-04-02T04:54:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.234+0000 7fd014759640 1 -- 172.21.15.16:0/2563357781 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fcff806dcd0 msgr2=0x7fcff8070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.234+0000 7fd014759640 1 --2- 172.21.15.16:0/2563357781 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fcff806dcd0 0x7fcff8070190 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd00802ca70 tx=0x7fd00802c8a0 comp rx=0 tx=0).stop 2024-04-02T04:54:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.234+0000 7fd014759640 1 -- 172.21.15.16:0/2563357781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd00c101a70 msgr2=0x7fd00c196550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.234+0000 7fd014759640 1 --2- 172.21.15.16:0/2563357781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd00c101a70 0x7fd00c196550 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fcffc00d6f0 tx=0x7fcffc00dbb0 comp rx=0 tx=0).stop 2024-04-02T04:54:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.234+0000 7fd014759640 1 -- 172.21.15.16:0/2563357781 shutdown_connections 2024-04-02T04:54:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.234+0000 7fd014759640 1 --2- 172.21.15.16:0/2563357781 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fcff806dcd0 0x7fcff8070190 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:01.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.234+0000 7fd014759640 1 --2- 172.21.15.16:0/2563357781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd00c101a70 0x7fd00c196550 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:01.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.234+0000 7fd014759640 1 --2- 172.21.15.16:0/2563357781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd00c101150 0x7fd00c196010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:01.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.234+0000 7fd014759640 1 -- 172.21.15.16:0/2563357781 >> 172.21.15.16:0/2563357781 conn(0x7fd00c0fa930 msgr2=0x7fd00c102430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:01.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.234+0000 7fd014759640 1 -- 172.21.15.16:0/2563357781 shutdown_connections 2024-04-02T04:54:01.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.234+0000 7fd014759640 1 -- 172.21.15.16:0/2563357781 wait complete. 2024-04-02T04:54:01.250 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-04-02T04:54:01.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.353+0000 7f2f458b6640 1 -- 172.21.15.16:0/1155918440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f40067e60 msgr2=0x7f2f40104ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:01.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.353+0000 7f2f458b6640 1 --2- 172.21.15.16:0/1155918440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f40067e60 0x7f2f40104ba0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2f28009880 tx=0x7f2f2802bce0 comp rx=0 tx=0).stop 2024-04-02T04:54:01.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.353+0000 7f2f458b6640 1 -- 172.21.15.16:0/1155918440 shutdown_connections 2024-04-02T04:54:01.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.353+0000 7f2f458b6640 1 --2- 172.21.15.16:0/1155918440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f40067e60 0x7f2f40104ba0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:01.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.353+0000 7f2f458b6640 1 --2- 172.21.15.16:0/1155918440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f400674b0 0x7f2f40067890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:01.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.353+0000 7f2f458b6640 1 -- 172.21.15.16:0/1155918440 >> 172.21.15.16:0/1155918440 conn(0x7f2f40074960 msgr2=0x7f2f40074d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:01.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.353+0000 7f2f458b6640 1 -- 172.21.15.16:0/1155918440 shutdown_connections 2024-04-02T04:54:01.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.353+0000 7f2f458b6640 1 -- 172.21.15.16:0/1155918440 wait complete. 2024-04-02T04:54:01.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.353+0000 7f2f458b6640 1 Processor -- start 2024-04-02T04:54:01.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.353+0000 7f2f458b6640 1 -- start start 2024-04-02T04:54:01.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.354+0000 7f2f458b6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f400674b0 0x7f2f400ff020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:01.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.354+0000 7f2f458b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f400ff560 0x7f2f400ff9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:01.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.354+0000 7f2f458b6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f40100f80 con 0x7f2f400ff560 2024-04-02T04:54:01.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.354+0000 7f2f458b6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f401a2290 con 0x7f2f400674b0 2024-04-02T04:54:01.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.354+0000 7f2f3e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f400ff560 0x7f2f400ff9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:01.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.354+0000 7f2f3e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f400ff560 0x7f2f400ff9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40236/0 (socket says 172.21.15.16:40236) 2024-04-02T04:54:01.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.354+0000 7f2f3e7fc640 1 -- 172.21.15.16:0/997529768 learned_addr learned my addr 172.21.15.16:0/997529768 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:54:01.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.354+0000 7f2f3effd640 1 --2- 172.21.15.16:0/997529768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f400674b0 0x7f2f400ff020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:01.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.354+0000 7f2f3e7fc640 1 -- 172.21.15.16:0/997529768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f400674b0 msgr2=0x7f2f400ff020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:01.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.354+0000 7f2f3e7fc640 1 --2- 172.21.15.16:0/997529768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f400674b0 0x7f2f400ff020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:01.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.354+0000 7f2f3e7fc640 1 -- 172.21.15.16:0/997529768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f28009560 con 0x7f2f400ff560 2024-04-02T04:54:01.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.354+0000 7f2f3e7fc640 1 --2- 172.21.15.16:0/997529768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f400ff560 0x7f2f400ff9c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2f2802c220 tx=0x7f2f2802c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:01.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.354+0000 7f2f448b4640 1 -- 172.21.15.16:0/997529768 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f280035f0 con 0x7f2f400ff560 2024-04-02T04:54:01.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.354+0000 7f2f448b4640 1 -- 172.21.15.16:0/997529768 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2f28003c10 con 0x7f2f400ff560 2024-04-02T04:54:01.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.354+0000 7f2f458b6640 1 -- 172.21.15.16:0/997529768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f401a2430 con 0x7f2f400ff560 2024-04-02T04:54:01.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.354+0000 7f2f448b4640 1 -- 172.21.15.16:0/997529768 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f280055d0 con 0x7f2f400ff560 2024-04-02T04:54:01.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.354+0000 7f2f458b6640 1 -- 172.21.15.16:0/997529768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f401a2960 con 0x7f2f400ff560 2024-04-02T04:54:01.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.355+0000 7f2f458b6640 1 -- 172.21.15.16:0/997529768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f40101680 con 0x7f2f400ff560 2024-04-02T04:54:01.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.356+0000 7f2f448b4640 1 -- 172.21.15.16:0/997529768 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f2f28004dd0 con 0x7f2f400ff560 2024-04-02T04:54:01.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.356+0000 7f2f448b4640 1 --2- 172.21.15.16:0/997529768 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f2f2c06dcd0 0x7f2f2c070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:01.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.356+0000 7f2f3effd640 1 --2- 172.21.15.16:0/997529768 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f2f2c06dcd0 0x7f2f2c070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:01.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.356+0000 7f2f448b4640 1 -- 172.21.15.16:0/997529768 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f2f280afea0 con 0x7f2f400ff560 2024-04-02T04:54:01.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.357+0000 7f2f3effd640 1 --2- 172.21.15.16:0/997529768 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f2f2c06dcd0 0x7f2f2c070190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2f38005fd0 tx=0x7f2f38005e20 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:01.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.361+0000 7f2f448b4640 1 -- 172.21.15.16:0/997529768 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2f28079410 con 0x7f2f400ff560 2024-04-02T04:54:01.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:01 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/820295040' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-04-02T04:54:01.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:01 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/529522416' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-04-02T04:54:01.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:01 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/2563357781' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-04-02T04:54:01.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.584+0000 7f2f458b6640 1 -- 172.21.15.16:0/997529768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7f2f4004e030 con 0x7f2f400ff560 2024-04-02T04:54:01.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.585+0000 7f2f448b4640 1 -- 172.21.15.16:0/997529768 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v29) v1 ==== 107+0+4984 (secure 0 0 0) 0x7f2f28041090 con 0x7f2f400ff560 2024-04-02T04:54:01.590 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:54:01.590 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24295,"name":"cephfs.smithi027.sdublg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.27:6827/4177192740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":4177192740},{"type":"v1","addr":"172.21.15.27:6827","nonce":4177192740}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34404,"name":"cephfs.smithi016.uhgkgg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6827/81744708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":81744708},{"type":"v1","addr":"172.21.15.16:6827","nonce":81744708}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":34436,"name":"cephfs.smithi016.fyzrnm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6829/782430688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":782430688},{"type":"v1","addr":"172.21.15.16:6829","nonce":782430688}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:39:30.740168+0000","modified":"2024-04-02T04:39:49.875319+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24285":{"gid":24285,"name":"cephfs.smithi027.zwrfoo","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6825/1461455976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1461455976},{"type":"v1","addr":"172.21.15.27:6825","nonce":1461455976}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-02T04:54:01.590 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 19 2024-04-02T04:54:01.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.587+0000 7f2f458b6640 1 -- 172.21.15.16:0/997529768 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f2f2c06dcd0 msgr2=0x7f2f2c070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:01.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.587+0000 7f2f458b6640 1 --2- 172.21.15.16:0/997529768 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f2f2c06dcd0 0x7f2f2c070190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2f38005fd0 tx=0x7f2f38005e20 comp rx=0 tx=0).stop 2024-04-02T04:54:01.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.587+0000 7f2f458b6640 1 -- 172.21.15.16:0/997529768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f400ff560 msgr2=0x7f2f400ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:01.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.587+0000 7f2f458b6640 1 --2- 172.21.15.16:0/997529768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f400ff560 0x7f2f400ff9c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2f2802c220 tx=0x7f2f2802c7c0 comp rx=0 tx=0).stop 2024-04-02T04:54:01.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.587+0000 7f2f458b6640 1 -- 172.21.15.16:0/997529768 shutdown_connections 2024-04-02T04:54:01.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.587+0000 7f2f458b6640 1 --2- 172.21.15.16:0/997529768 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f2f2c06dcd0 0x7f2f2c070190 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:01.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.587+0000 7f2f458b6640 1 --2- 172.21.15.16:0/997529768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f400ff560 0x7f2f400ff9c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:01.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.587+0000 7f2f458b6640 1 --2- 172.21.15.16:0/997529768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f400674b0 0x7f2f400ff020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:01.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.587+0000 7f2f458b6640 1 -- 172.21.15.16:0/997529768 >> 172.21.15.16:0/997529768 conn(0x7f2f40074960 msgr2=0x7f2f40103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:01.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.588+0000 7f2f458b6640 1 -- 172.21.15.16:0/997529768 shutdown_connections 2024-04-02T04:54:01.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.588+0000 7f2f458b6640 1 -- 172.21.15.16:0/997529768 wait complete. 2024-04-02T04:54:01.603 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-04-02T04:54:01.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:01 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/820295040' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-04-02T04:54:01.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:01 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/529522416' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-04-02T04:54:01.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:01 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/2563357781' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-04-02T04:54:01.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.706+0000 7f622a11b640 1 -- 172.21.15.16:0/630964242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62240ff1c0 msgr2=0x7f62240ff5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:01.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.706+0000 7f622a11b640 1 --2- 172.21.15.16:0/630964242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62240ff1c0 0x7f62240ff5a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f620c009880 tx=0x7f620c02bce0 comp rx=0 tx=0).stop 2024-04-02T04:54:01.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.706+0000 7f622a11b640 1 -- 172.21.15.16:0/630964242 shutdown_connections 2024-04-02T04:54:01.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.706+0000 7f622a11b640 1 --2- 172.21.15.16:0/630964242 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62240ffb70 0x7f6224103aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:01.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.706+0000 7f622a11b640 1 --2- 172.21.15.16:0/630964242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62240ff1c0 0x7f62240ff5a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:01.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.706+0000 7f622a11b640 1 -- 172.21.15.16:0/630964242 >> 172.21.15.16:0/630964242 conn(0x7f6224074960 msgr2=0x7f6224074d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:01.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.706+0000 7f622a11b640 1 -- 172.21.15.16:0/630964242 shutdown_connections 2024-04-02T04:54:01.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.706+0000 7f622a11b640 1 -- 172.21.15.16:0/630964242 wait complete. 2024-04-02T04:54:01.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.707+0000 7f622a11b640 1 Processor -- start 2024-04-02T04:54:01.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.707+0000 7f622a11b640 1 -- start start 2024-04-02T04:54:01.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.707+0000 7f622a11b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62240ff1c0 0x7f6224196030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:01.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.707+0000 7f622a11b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6224196570 0x7f622419a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:01.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.707+0000 7f622a11b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6224196b70 con 0x7f6224196570 2024-04-02T04:54:01.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.707+0000 7f622a11b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6224196ce0 con 0x7f62240ff1c0 2024-04-02T04:54:01.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.707+0000 7f6222ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6224196570 0x7f622419a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:01.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.707+0000 7f6222ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6224196570 0x7f622419a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58188/0 (socket says 172.21.15.16:58188) 2024-04-02T04:54:01.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.707+0000 7f6222ffd640 1 -- 172.21.15.16:0/1747840589 learned_addr learned my addr 172.21.15.16:0/1747840589 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:54:01.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.708+0000 7f62237fe640 1 --2- 172.21.15.16:0/1747840589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62240ff1c0 0x7f6224196030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:01.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.708+0000 7f6222ffd640 1 -- 172.21.15.16:0/1747840589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62240ff1c0 msgr2=0x7f6224196030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:01.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.708+0000 7f6222ffd640 1 --2- 172.21.15.16:0/1747840589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62240ff1c0 0x7f6224196030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:01.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.708+0000 7f6222ffd640 1 -- 172.21.15.16:0/1747840589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f620c009560 con 0x7f6224196570 2024-04-02T04:54:01.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.708+0000 7f6222ffd640 1 --2- 172.21.15.16:0/1747840589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6224196570 0x7f622419a910 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f621400e970 tx=0x7f621400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:01.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.708+0000 7f6220ff9640 1 -- 172.21.15.16:0/1747840589 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f621400cc90 con 0x7f6224196570 2024-04-02T04:54:01.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.708+0000 7f622a11b640 1 -- 172.21.15.16:0/1747840589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f622419af10 con 0x7f6224196570 2024-04-02T04:54:01.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.708+0000 7f6220ff9640 1 -- 172.21.15.16:0/1747840589 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f621400cdf0 con 0x7f6224196570 2024-04-02T04:54:01.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.708+0000 7f6220ff9640 1 -- 172.21.15.16:0/1747840589 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6214010430 con 0x7f6224196570 2024-04-02T04:54:01.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.708+0000 7f622a11b640 1 -- 172.21.15.16:0/1747840589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f622419b470 con 0x7f6224196570 2024-04-02T04:54:01.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.709+0000 7f622a11b640 1 -- 172.21.15.16:0/1747840589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f622419b750 con 0x7f6224196570 2024-04-02T04:54:01.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.710+0000 7f6220ff9640 1 -- 172.21.15.16:0/1747840589 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f62140046c0 con 0x7f6224196570 2024-04-02T04:54:01.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.710+0000 7f6220ff9640 1 --2- 172.21.15.16:0/1747840589 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f621006dcd0 0x7f6210070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:01.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.710+0000 7f62237fe640 1 --2- 172.21.15.16:0/1747840589 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f621006dcd0 0x7f6210070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:01.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.710+0000 7f6220ff9640 1 -- 172.21.15.16:0/1747840589 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f6214014070 con 0x7f6224196570 2024-04-02T04:54:01.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.711+0000 7f62237fe640 1 --2- 172.21.15.16:0/1747840589 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f621006dcd0 0x7f6210070190 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f620c000c00 tx=0x7f620c02c820 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:01.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.715+0000 7f6220ff9640 1 -- 172.21.15.16:0/1747840589 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f62140576b0 con 0x7f6224196570 2024-04-02T04:54:01.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.937+0000 7f622a11b640 1 -- 172.21.15.16:0/1747840589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7f62240ffb70 con 0x7f6224196570 2024-04-02T04:54:01.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.938+0000 7f6220ff9640 1 -- 172.21.15.16:0/1747840589 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v29) v1 ==== 107+0+4194 (secure 0 0 0) 0x7f621405ad70 con 0x7f6224196570 2024-04-02T04:54:01.943 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:54:01.943 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24295,"name":"cephfs.smithi027.sdublg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.27:6827/4177192740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":4177192740},{"type":"v1","addr":"172.21.15.27:6827","nonce":4177192740}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34404,"name":"cephfs.smithi016.uhgkgg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6827/81744708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":81744708},{"type":"v1","addr":"172.21.15.16:6827","nonce":81744708}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":34436,"name":"cephfs.smithi016.fyzrnm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6829/782430688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":782430688},{"type":"v1","addr":"172.21.15.16:6829","nonce":782430688}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:39:30.740168+0000","modified":"2024-04-02T04:49:41.368233+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-02T04:54:01.943 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 20 2024-04-02T04:54:01.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.940+0000 7f622a11b640 1 -- 172.21.15.16:0/1747840589 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f621006dcd0 msgr2=0x7f6210070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:01.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.940+0000 7f622a11b640 1 --2- 172.21.15.16:0/1747840589 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f621006dcd0 0x7f6210070190 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f620c000c00 tx=0x7f620c02c820 comp rx=0 tx=0).stop 2024-04-02T04:54:01.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.940+0000 7f622a11b640 1 -- 172.21.15.16:0/1747840589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6224196570 msgr2=0x7f622419a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:01.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.940+0000 7f622a11b640 1 --2- 172.21.15.16:0/1747840589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6224196570 0x7f622419a910 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f621400e970 tx=0x7f621400ee30 comp rx=0 tx=0).stop 2024-04-02T04:54:01.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.940+0000 7f622a11b640 1 -- 172.21.15.16:0/1747840589 shutdown_connections 2024-04-02T04:54:01.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.940+0000 7f622a11b640 1 --2- 172.21.15.16:0/1747840589 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f621006dcd0 0x7f6210070190 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:01.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.940+0000 7f622a11b640 1 --2- 172.21.15.16:0/1747840589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6224196570 0x7f622419a910 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:01.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.940+0000 7f622a11b640 1 --2- 172.21.15.16:0/1747840589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62240ff1c0 0x7f6224196030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:01.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.940+0000 7f622a11b640 1 -- 172.21.15.16:0/1747840589 >> 172.21.15.16:0/1747840589 conn(0x7f6224074960 msgr2=0x7f62240675a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:01.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.940+0000 7f622a11b640 1 -- 172.21.15.16:0/1747840589 shutdown_connections 2024-04-02T04:54:01.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:01.940+0000 7f622a11b640 1 -- 172.21.15.16:0/1747840589 wait complete. 2024-04-02T04:54:01.956 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-04-02T04:54:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.054+0000 7f86e789d640 1 -- 172.21.15.16:0/3486296443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e00ffb70 msgr2=0x7f86e0103aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.054+0000 7f86e789d640 1 --2- 172.21.15.16:0/3486296443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e00ffb70 0x7f86e0103aa0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f86dc009880 tx=0x7f86dc02bce0 comp rx=0 tx=0).stop 2024-04-02T04:54:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.054+0000 7f86e789d640 1 -- 172.21.15.16:0/3486296443 shutdown_connections 2024-04-02T04:54:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.054+0000 7f86e789d640 1 --2- 172.21.15.16:0/3486296443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e00ffb70 0x7f86e0103aa0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.054+0000 7f86e789d640 1 --2- 172.21.15.16:0/3486296443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86e00ff1c0 0x7f86e00ff5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.054+0000 7f86e789d640 1 -- 172.21.15.16:0/3486296443 >> 172.21.15.16:0/3486296443 conn(0x7f86e0074960 msgr2=0x7f86e0074d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.055+0000 7f86e789d640 1 -- 172.21.15.16:0/3486296443 shutdown_connections 2024-04-02T04:54:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.055+0000 7f86e789d640 1 -- 172.21.15.16:0/3486296443 wait complete. 2024-04-02T04:54:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.055+0000 7f86e789d640 1 Processor -- start 2024-04-02T04:54:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.055+0000 7f86e789d640 1 -- start start 2024-04-02T04:54:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.055+0000 7f86e789d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86e00ff1c0 0x7f86e0196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.055+0000 7f86e789d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e0196560 0x7f86e019a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.055+0000 7f86e789d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e0196b60 con 0x7f86e0196560 2024-04-02T04:54:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.055+0000 7f86e789d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e0196cd0 con 0x7f86e00ff1c0 2024-04-02T04:54:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.055+0000 7f86e4e38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e0196560 0x7f86e019a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.055+0000 7f86e4e38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e0196560 0x7f86e019a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58212/0 (socket says 172.21.15.16:58212) 2024-04-02T04:54:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.055+0000 7f86e4e38640 1 -- 172.21.15.16:0/2127572475 learned_addr learned my addr 172.21.15.16:0/2127572475 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:54:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.056+0000 7f86e4e38640 1 -- 172.21.15.16:0/2127572475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86e00ff1c0 msgr2=0x7f86e0196020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.056+0000 7f86e4e38640 1 --2- 172.21.15.16:0/2127572475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86e00ff1c0 0x7f86e0196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.056+0000 7f86e4e38640 1 -- 172.21.15.16:0/2127572475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86dc009560 con 0x7f86e0196560 2024-04-02T04:54:02.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.056+0000 7f86e4e38640 1 --2- 172.21.15.16:0/2127572475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e0196560 0x7f86e019a900 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f86dc036040 tx=0x7f86dc02c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:02.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.056+0000 7f86d27fc640 1 -- 172.21.15.16:0/2127572475 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86dc004420 con 0x7f86e0196560 2024-04-02T04:54:02.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.056+0000 7f86d27fc640 1 -- 172.21.15.16:0/2127572475 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f86dc003ab0 con 0x7f86e0196560 2024-04-02T04:54:02.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.056+0000 7f86e789d640 1 -- 172.21.15.16:0/2127572475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86e019aea0 con 0x7f86e0196560 2024-04-02T04:54:02.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.056+0000 7f86d27fc640 1 -- 172.21.15.16:0/2127572475 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86dc005490 con 0x7f86e0196560 2024-04-02T04:54:02.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.056+0000 7f86e789d640 1 -- 172.21.15.16:0/2127572475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86e019b400 con 0x7f86e0196560 2024-04-02T04:54:02.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.057+0000 7f86e789d640 1 -- 172.21.15.16:0/2127572475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86e0100580 con 0x7f86e0196560 2024-04-02T04:54:02.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.058+0000 7f86d27fc640 1 -- 172.21.15.16:0/2127572475 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f86dc003c20 con 0x7f86e0196560 2024-04-02T04:54:02.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.058+0000 7f86d27fc640 1 --2- 172.21.15.16:0/2127572475 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f86cc06dcd0 0x7f86cc070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:02.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.058+0000 7f86d27fc640 1 -- 172.21.15.16:0/2127572475 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f86dc0afd90 con 0x7f86e0196560 2024-04-02T04:54:02.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.058+0000 7f86e5639640 1 --2- 172.21.15.16:0/2127572475 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f86cc06dcd0 0x7f86cc070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:02.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.058+0000 7f86e5639640 1 --2- 172.21.15.16:0/2127572475 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f86cc06dcd0 0x7f86cc070190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f86d40058c0 tx=0x7f86d4005850 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:02.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.063+0000 7f86d27fc640 1 -- 172.21.15.16:0/2127572475 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f86dc079280 con 0x7f86e0196560 2024-04-02T04:54:02.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.293+0000 7f86e789d640 1 -- 172.21.15.16:0/2127572475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7f86e019b650 con 0x7f86e0196560 2024-04-02T04:54:02.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.293+0000 7f86d27fc640 1 -- 172.21.15.16:0/2127572475 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v29) v1 ==== 107+0+4205 (secure 0 0 0) 0x7f86dc07c940 con 0x7f86e0196560 2024-04-02T04:54:02.298 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:54:02.298 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi016.uhgkgg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6827/81744708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":81744708},{"type":"v1","addr":"172.21.15.16:6827","nonce":81744708}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":34436,"name":"cephfs.smithi016.fyzrnm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6829/782430688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":782430688},{"type":"v1","addr":"172.21.15.16:6829","nonce":782430688}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:39:30.740168+0000","modified":"2024-04-02T04:49:41.373238+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24295":{"gid":24295,"name":"cephfs.smithi027.sdublg","rank":0,"incarnation":21,"state":"up:replay","state_seq":2,"addr":"172.21.15.27:6827/4177192740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":4177192740},{"type":"v1","addr":"172.21.15.27:6827","nonce":4177192740}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-02T04:54:02.298 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 21 2024-04-02T04:54:02.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.296+0000 7f86e789d640 1 -- 172.21.15.16:0/2127572475 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f86cc06dcd0 msgr2=0x7f86cc070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:02.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.296+0000 7f86e789d640 1 --2- 172.21.15.16:0/2127572475 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f86cc06dcd0 0x7f86cc070190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f86d40058c0 tx=0x7f86d4005850 comp rx=0 tx=0).stop 2024-04-02T04:54:02.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.296+0000 7f86e789d640 1 -- 172.21.15.16:0/2127572475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e0196560 msgr2=0x7f86e019a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:02.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.296+0000 7f86e789d640 1 --2- 172.21.15.16:0/2127572475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e0196560 0x7f86e019a900 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f86dc036040 tx=0x7f86dc02c6f0 comp rx=0 tx=0).stop 2024-04-02T04:54:02.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.296+0000 7f86e789d640 1 -- 172.21.15.16:0/2127572475 shutdown_connections 2024-04-02T04:54:02.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.296+0000 7f86e789d640 1 --2- 172.21.15.16:0/2127572475 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f86cc06dcd0 0x7f86cc070190 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:02.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.296+0000 7f86e789d640 1 --2- 172.21.15.16:0/2127572475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e0196560 0x7f86e019a900 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:02.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.296+0000 7f86e789d640 1 --2- 172.21.15.16:0/2127572475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86e00ff1c0 0x7f86e0196020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:02.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.296+0000 7f86e789d640 1 -- 172.21.15.16:0/2127572475 >> 172.21.15.16:0/2127572475 conn(0x7f86e0074960 msgr2=0x7f86e0067580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:02.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.296+0000 7f86e789d640 1 -- 172.21.15.16:0/2127572475 shutdown_connections 2024-04-02T04:54:02.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.296+0000 7f86e789d640 1 -- 172.21.15.16:0/2127572475 wait complete. 2024-04-02T04:54:02.311 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-04-02T04:54:02.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.412+0000 7f7e49d7e640 1 -- 172.21.15.16:0/1388985760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e44100b60 msgr2=0x7f7e44104ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:02.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.412+0000 7f7e49d7e640 1 --2- 172.21.15.16:0/1388985760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e44100b60 0x7f7e44104ba0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7e38009880 tx=0x7f7e3802bce0 comp rx=0 tx=0).stop 2024-04-02T04:54:02.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.412+0000 7f7e49d7e640 1 -- 172.21.15.16:0/1388985760 shutdown_connections 2024-04-02T04:54:02.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.412+0000 7f7e49d7e640 1 --2- 172.21.15.16:0/1388985760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e44100b60 0x7f7e44104ba0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:02.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.412+0000 7f7e49d7e640 1 --2- 172.21.15.16:0/1388985760 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e441001b0 0x7f7e44100590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:02.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.412+0000 7f7e49d7e640 1 -- 172.21.15.16:0/1388985760 >> 172.21.15.16:0/1388985760 conn(0x7f7e440fba10 msgr2=0x7f7e440fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:02.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.412+0000 7f7e49d7e640 1 -- 172.21.15.16:0/1388985760 shutdown_connections 2024-04-02T04:54:02.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.412+0000 7f7e49d7e640 1 -- 172.21.15.16:0/1388985760 wait complete. 2024-04-02T04:54:02.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.413+0000 7f7e49d7e640 1 Processor -- start 2024-04-02T04:54:02.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.413+0000 7f7e49d7e640 1 -- start start 2024-04-02T04:54:02.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.413+0000 7f7e49d7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e441001b0 0x7f7e440725c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:02.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.413+0000 7f7e49d7e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e44072b00 0x7f7e44072f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:02.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.413+0000 7f7e49d7e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e44074520 con 0x7f7e441001b0 2024-04-02T04:54:02.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.413+0000 7f7e49d7e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e441a21b0 con 0x7f7e44072b00 2024-04-02T04:54:02.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.413+0000 7f7e437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e441001b0 0x7f7e440725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:02.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.413+0000 7f7e437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e441001b0 0x7f7e440725c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58228/0 (socket says 172.21.15.16:58228) 2024-04-02T04:54:02.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.413+0000 7f7e437fe640 1 -- 172.21.15.16:0/2952861451 learned_addr learned my addr 172.21.15.16:0/2952861451 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:54:02.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.413+0000 7f7e42ffd640 1 --2- 172.21.15.16:0/2952861451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e44072b00 0x7f7e44072f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:02.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.413+0000 7f7e437fe640 1 -- 172.21.15.16:0/2952861451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e44072b00 msgr2=0x7f7e44072f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:02.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.413+0000 7f7e437fe640 1 --2- 172.21.15.16:0/2952861451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e44072b00 0x7f7e44072f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:02.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.413+0000 7f7e437fe640 1 -- 172.21.15.16:0/2952861451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e38009560 con 0x7f7e441001b0 2024-04-02T04:54:02.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.414+0000 7f7e437fe640 1 --2- 172.21.15.16:0/2952861451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e441001b0 0x7f7e440725c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7e3400e970 tx=0x7f7e3400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:02.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.414+0000 7f7e40ff9640 1 -- 172.21.15.16:0/2952861451 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e3400cc90 con 0x7f7e441001b0 2024-04-02T04:54:02.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.414+0000 7f7e40ff9640 1 -- 172.21.15.16:0/2952861451 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e3400cdf0 con 0x7f7e441001b0 2024-04-02T04:54:02.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.414+0000 7f7e49d7e640 1 -- 172.21.15.16:0/2952861451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e441a23b0 con 0x7f7e441001b0 2024-04-02T04:54:02.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.414+0000 7f7e40ff9640 1 -- 172.21.15.16:0/2952861451 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e34010430 con 0x7f7e441001b0 2024-04-02T04:54:02.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.414+0000 7f7e49d7e640 1 -- 172.21.15.16:0/2952861451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e441a28e0 con 0x7f7e441001b0 2024-04-02T04:54:02.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.415+0000 7f7e49d7e640 1 -- 172.21.15.16:0/2952861451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e44101680 con 0x7f7e441001b0 2024-04-02T04:54:02.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.415+0000 7f7e40ff9640 1 -- 172.21.15.16:0/2952861451 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f7e340046c0 con 0x7f7e441001b0 2024-04-02T04:54:02.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.416+0000 7f7e40ff9640 1 --2- 172.21.15.16:0/2952861451 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f7e2c06dcd0 0x7f7e2c070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:02.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.416+0000 7f7e40ff9640 1 -- 172.21.15.16:0/2952861451 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f7e34014070 con 0x7f7e441001b0 2024-04-02T04:54:02.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.416+0000 7f7e42ffd640 1 --2- 172.21.15.16:0/2952861451 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f7e2c06dcd0 0x7f7e2c070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:02.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.416+0000 7f7e42ffd640 1 --2- 172.21.15.16:0/2952861451 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f7e2c06dcd0 0x7f7e2c070190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7e3802efd0 tx=0x7f7e3802cf90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:02.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.423+0000 7f7e40ff9640 1 -- 172.21.15.16:0/2952861451 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7e34057640 con 0x7f7e441001b0 2024-04-02T04:54:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:02 smithi027 ceph-mon[83036]: pgmap v330: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:02 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/997529768' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-04-02T04:54:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:02 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/1747840589' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-04-02T04:54:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:02 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:54:02.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:02 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/2127572475' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-04-02T04:54:02.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.644+0000 7f7e49d7e640 1 -- 172.21.15.16:0/2952861451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7f7e4404e030 con 0x7f7e441001b0 2024-04-02T04:54:02.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.645+0000 7f7e40ff9640 1 -- 172.21.15.16:0/2952861451 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v29) v1 ==== 107+0+4210 (secure 0 0 0) 0x7f7e3405ad00 con 0x7f7e441001b0 2024-04-02T04:54:02.649 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:54:02.649 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi016.uhgkgg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6827/81744708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":81744708},{"type":"v1","addr":"172.21.15.16:6827","nonce":81744708}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":34436,"name":"cephfs.smithi016.fyzrnm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6829/782430688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":782430688},{"type":"v1","addr":"172.21.15.16:6829","nonce":782430688}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:39:30.740168+0000","modified":"2024-04-02T04:49:50.853317+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24295":{"gid":24295,"name":"cephfs.smithi027.sdublg","rank":0,"incarnation":21,"state":"up:reconnect","state_seq":153,"addr":"172.21.15.27:6827/4177192740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":4177192740},{"type":"v1","addr":"172.21.15.27:6827","nonce":4177192740}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-02T04:54:02.650 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 22 2024-04-02T04:54:02.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.647+0000 7f7e49d7e640 1 -- 172.21.15.16:0/2952861451 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f7e2c06dcd0 msgr2=0x7f7e2c070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:02.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.647+0000 7f7e49d7e640 1 --2- 172.21.15.16:0/2952861451 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f7e2c06dcd0 0x7f7e2c070190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7e3802efd0 tx=0x7f7e3802cf90 comp rx=0 tx=0).stop 2024-04-02T04:54:02.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.647+0000 7f7e49d7e640 1 -- 172.21.15.16:0/2952861451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e441001b0 msgr2=0x7f7e440725c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:02.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.647+0000 7f7e49d7e640 1 --2- 172.21.15.16:0/2952861451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e441001b0 0x7f7e440725c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7e3400e970 tx=0x7f7e3400ed70 comp rx=0 tx=0).stop 2024-04-02T04:54:02.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.647+0000 7f7e49d7e640 1 -- 172.21.15.16:0/2952861451 shutdown_connections 2024-04-02T04:54:02.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.647+0000 7f7e49d7e640 1 --2- 172.21.15.16:0/2952861451 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f7e2c06dcd0 0x7f7e2c070190 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:02.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.647+0000 7f7e49d7e640 1 --2- 172.21.15.16:0/2952861451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e44072b00 0x7f7e44072f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:02.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.647+0000 7f7e49d7e640 1 --2- 172.21.15.16:0/2952861451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e441001b0 0x7f7e440725c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:02.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.647+0000 7f7e49d7e640 1 -- 172.21.15.16:0/2952861451 >> 172.21.15.16:0/2952861451 conn(0x7f7e440fba10 msgr2=0x7f7e440fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:02.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.647+0000 7f7e49d7e640 1 -- 172.21.15.16:0/2952861451 shutdown_connections 2024-04-02T04:54:02.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.647+0000 7f7e49d7e640 1 -- 172.21.15.16:0/2952861451 wait complete. 2024-04-02T04:54:02.661 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:02 smithi016 ceph-mon[93230]: pgmap v330: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:02.662 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:02 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/997529768' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-04-02T04:54:02.662 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:02 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/1747840589' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-04-02T04:54:02.662 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:02 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:54:02.662 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:02 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/2127572475' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-04-02T04:54:02.663 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-04-02T04:54:02.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.763+0000 7f68399ab640 1 -- 172.21.15.16:0/3850095364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68340ffb70 msgr2=0x7f6834103aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:02.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.763+0000 7f68399ab640 1 --2- 172.21.15.16:0/3850095364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68340ffb70 0x7f6834103aa0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6828009880 tx=0x7f682802bce0 comp rx=0 tx=0).stop 2024-04-02T04:54:02.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.763+0000 7f68399ab640 1 -- 172.21.15.16:0/3850095364 shutdown_connections 2024-04-02T04:54:02.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.763+0000 7f68399ab640 1 --2- 172.21.15.16:0/3850095364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68340ffb70 0x7f6834103aa0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:02.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.763+0000 7f68399ab640 1 --2- 172.21.15.16:0/3850095364 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68340ff1c0 0x7f68340ff5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:02.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.763+0000 7f68399ab640 1 -- 172.21.15.16:0/3850095364 >> 172.21.15.16:0/3850095364 conn(0x7f6834074960 msgr2=0x7f6834074d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:02.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.763+0000 7f68399ab640 1 -- 172.21.15.16:0/3850095364 shutdown_connections 2024-04-02T04:54:02.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.763+0000 7f68399ab640 1 -- 172.21.15.16:0/3850095364 wait complete. 2024-04-02T04:54:02.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.763+0000 7f68399ab640 1 Processor -- start 2024-04-02T04:54:02.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.764+0000 7f68399ab640 1 -- start start 2024-04-02T04:54:02.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.764+0000 7f68399ab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68340ff1c0 0x7f6834194f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:02.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.764+0000 7f68399ab640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6834195450 0x7f68341997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:02.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.764+0000 7f68399ab640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6834195a50 con 0x7f68340ff1c0 2024-04-02T04:54:02.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.764+0000 7f68399ab640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6834195bc0 con 0x7f6834195450 2024-04-02T04:54:02.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.764+0000 7f6832ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68340ff1c0 0x7f6834194f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:02.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.764+0000 7f6832ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68340ff1c0 0x7f6834194f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58240/0 (socket says 172.21.15.16:58240) 2024-04-02T04:54:02.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.764+0000 7f6832ffd640 1 -- 172.21.15.16:0/4191847624 learned_addr learned my addr 172.21.15.16:0/4191847624 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:54:02.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.764+0000 7f68327fc640 1 --2- 172.21.15.16:0/4191847624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6834195450 0x7f68341997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:02.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.764+0000 7f6832ffd640 1 -- 172.21.15.16:0/4191847624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6834195450 msgr2=0x7f68341997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:02.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.764+0000 7f6832ffd640 1 --2- 172.21.15.16:0/4191847624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6834195450 0x7f68341997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:02.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.764+0000 7f6832ffd640 1 -- 172.21.15.16:0/4191847624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6828009560 con 0x7f68340ff1c0 2024-04-02T04:54:02.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.765+0000 7f6832ffd640 1 --2- 172.21.15.16:0/4191847624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68340ff1c0 0x7f6834194f10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f682400d900 tx=0x7f682400ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:02.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.765+0000 7f68389a9640 1 -- 172.21.15.16:0/4191847624 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6824009c00 con 0x7f68340ff1c0 2024-04-02T04:54:02.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.765+0000 7f68389a9640 1 -- 172.21.15.16:0/4191847624 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6824009d60 con 0x7f68340ff1c0 2024-04-02T04:54:02.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.765+0000 7f68399ab640 1 -- 172.21.15.16:0/4191847624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6834199df0 con 0x7f68340ff1c0 2024-04-02T04:54:02.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.765+0000 7f68389a9640 1 -- 172.21.15.16:0/4191847624 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f682400f710 con 0x7f68340ff1c0 2024-04-02T04:54:02.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.765+0000 7f68399ab640 1 -- 172.21.15.16:0/4191847624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f683419a350 con 0x7f68340ff1c0 2024-04-02T04:54:02.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.766+0000 7f68399ab640 1 -- 172.21.15.16:0/4191847624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6834100580 con 0x7f68340ff1c0 2024-04-02T04:54:02.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.766+0000 7f68389a9640 1 -- 172.21.15.16:0/4191847624 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f6824010460 con 0x7f68340ff1c0 2024-04-02T04:54:02.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.767+0000 7f68389a9640 1 --2- 172.21.15.16:0/4191847624 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f681c06dcd0 0x7f681c070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:02.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.767+0000 7f68389a9640 1 -- 172.21.15.16:0/4191847624 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f682408d8f0 con 0x7f68340ff1c0 2024-04-02T04:54:02.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.767+0000 7f68327fc640 1 --2- 172.21.15.16:0/4191847624 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f681c06dcd0 0x7f681c070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:02.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.767+0000 7f68327fc640 1 --2- 172.21.15.16:0/4191847624 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f681c06dcd0 0x7f681c070190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6828000c00 tx=0x7f6828005fd0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:02.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:02.771+0000 7f68389a9640 1 -- 172.21.15.16:0/4191847624 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6824056e90 con 0x7f68340ff1c0 2024-04-02T04:54:03.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.005+0000 7f68399ab640 1 -- 172.21.15.16:0/4191847624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7f683419a5d0 con 0x7f68340ff1c0 2024-04-02T04:54:03.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.006+0000 7f68389a9640 1 -- 172.21.15.16:0/4191847624 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v29) v1 ==== 107+0+5054 (secure 0 0 0) 0x7f682405a550 con 0x7f68340ff1c0 2024-04-02T04:54:03.011 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:54:03.011 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi016.uhgkgg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6827/81744708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":81744708},{"type":"v1","addr":"172.21.15.16:6827","nonce":81744708}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":34436,"name":"cephfs.smithi016.fyzrnm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6829/782430688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":782430688},{"type":"v1","addr":"172.21.15.16:6829","nonce":782430688}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":44359,"name":"cephfs.smithi027.zwrfoo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6825/1016194753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1016194753},{"type":"v1","addr":"172.21.15.27:6825","nonce":1016194753}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:39:30.740168+0000","modified":"2024-04-02T04:49:51.262480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24295":{"gid":24295,"name":"cephfs.smithi027.sdublg","rank":0,"incarnation":21,"state":"up:rejoin","state_seq":154,"addr":"172.21.15.27:6827/4177192740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":4177192740},{"type":"v1","addr":"172.21.15.27:6827","nonce":4177192740}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-02T04:54:03.011 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 23 2024-04-02T04:54:03.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.008+0000 7f68399ab640 1 -- 172.21.15.16:0/4191847624 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f681c06dcd0 msgr2=0x7f681c070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:03.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.008+0000 7f68399ab640 1 --2- 172.21.15.16:0/4191847624 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f681c06dcd0 0x7f681c070190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6828000c00 tx=0x7f6828005fd0 comp rx=0 tx=0).stop 2024-04-02T04:54:03.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.008+0000 7f68399ab640 1 -- 172.21.15.16:0/4191847624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68340ff1c0 msgr2=0x7f6834194f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:03.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.008+0000 7f68399ab640 1 --2- 172.21.15.16:0/4191847624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68340ff1c0 0x7f6834194f10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f682400d900 tx=0x7f682400ddc0 comp rx=0 tx=0).stop 2024-04-02T04:54:03.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.008+0000 7f68399ab640 1 -- 172.21.15.16:0/4191847624 shutdown_connections 2024-04-02T04:54:03.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.008+0000 7f68399ab640 1 --2- 172.21.15.16:0/4191847624 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f681c06dcd0 0x7f681c070190 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:03.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.008+0000 7f68399ab640 1 --2- 172.21.15.16:0/4191847624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6834195450 0x7f68341997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:03.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.008+0000 7f68399ab640 1 --2- 172.21.15.16:0/4191847624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68340ff1c0 0x7f6834194f10 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:03.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.008+0000 7f68399ab640 1 -- 172.21.15.16:0/4191847624 >> 172.21.15.16:0/4191847624 conn(0x7f6834074960 msgr2=0x7f6834102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:03.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.008+0000 7f68399ab640 1 -- 172.21.15.16:0/4191847624 shutdown_connections 2024-04-02T04:54:03.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.008+0000 7f68399ab640 1 -- 172.21.15.16:0/4191847624 wait complete. 2024-04-02T04:54:03.024 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-04-02T04:54:03.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.122+0000 7f96ac1d0640 1 -- 172.21.15.16:0/260724162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a40ff1c0 msgr2=0x7f96a40ff5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:03.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.122+0000 7f96ac1d0640 1 --2- 172.21.15.16:0/260724162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a40ff1c0 0x7f96a40ff5a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9698009880 tx=0x7f969802bce0 comp rx=0 tx=0).stop 2024-04-02T04:54:03.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.122+0000 7f96ac1d0640 1 -- 172.21.15.16:0/260724162 shutdown_connections 2024-04-02T04:54:03.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.122+0000 7f96ac1d0640 1 --2- 172.21.15.16:0/260724162 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96a40ffb70 0x7f96a4103aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:03.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.122+0000 7f96ac1d0640 1 --2- 172.21.15.16:0/260724162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a40ff1c0 0x7f96a40ff5a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:03.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.122+0000 7f96ac1d0640 1 -- 172.21.15.16:0/260724162 >> 172.21.15.16:0/260724162 conn(0x7f96a4074960 msgr2=0x7f96a4074d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.123+0000 7f96ac1d0640 1 -- 172.21.15.16:0/260724162 shutdown_connections 2024-04-02T04:54:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.123+0000 7f96ac1d0640 1 -- 172.21.15.16:0/260724162 wait complete. 2024-04-02T04:54:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.123+0000 7f96ac1d0640 1 Processor -- start 2024-04-02T04:54:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.123+0000 7f96ac1d0640 1 -- start start 2024-04-02T04:54:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.123+0000 7f96ac1d0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96a40ff1c0 0x7f96a41982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.123+0000 7f96ac1d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a4198800 0x7f96a419cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.123+0000 7f96ac1d0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a4198e00 con 0x7f96a4198800 2024-04-02T04:54:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.123+0000 7f96ac1d0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a4198f70 con 0x7f96a40ff1c0 2024-04-02T04:54:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.124+0000 7f96a976b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a4198800 0x7f96a419cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.124+0000 7f96a976b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a4198800 0x7f96a419cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58252/0 (socket says 172.21.15.16:58252) 2024-04-02T04:54:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.124+0000 7f96a976b640 1 -- 172.21.15.16:0/4028651019 learned_addr learned my addr 172.21.15.16:0/4028651019 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:54:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.124+0000 7f96a9f6c640 1 --2- 172.21.15.16:0/4028651019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96a40ff1c0 0x7f96a41982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.124+0000 7f96a976b640 1 -- 172.21.15.16:0/4028651019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96a40ff1c0 msgr2=0x7f96a41982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.124+0000 7f96a976b640 1 --2- 172.21.15.16:0/4028651019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96a40ff1c0 0x7f96a41982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.124+0000 7f96a976b640 1 -- 172.21.15.16:0/4028651019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9698009560 con 0x7f96a4198800 2024-04-02T04:54:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.124+0000 7f96a976b640 1 --2- 172.21.15.16:0/4028651019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a4198800 0x7f96a419cba0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f96a000d6f0 tx=0x7f96a000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.124+0000 7f9696ffd640 1 -- 172.21.15.16:0/4028651019 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96a00099f0 con 0x7f96a4198800 2024-04-02T04:54:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.124+0000 7f9696ffd640 1 -- 172.21.15.16:0/4028651019 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f96a0009b50 con 0x7f96a4198800 2024-04-02T04:54:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.124+0000 7f96ac1d0640 1 -- 172.21.15.16:0/4028651019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96a419d1a0 con 0x7f96a4198800 2024-04-02T04:54:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.124+0000 7f9696ffd640 1 -- 172.21.15.16:0/4028651019 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96a000f680 con 0x7f96a4198800 2024-04-02T04:54:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.124+0000 7f96ac1d0640 1 -- 172.21.15.16:0/4028651019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96a419d700 con 0x7f96a4198800 2024-04-02T04:54:03.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.125+0000 7f96ac1d0640 1 -- 172.21.15.16:0/4028651019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96a419d9d0 con 0x7f96a4198800 2024-04-02T04:54:03.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.126+0000 7f9696ffd640 1 -- 172.21.15.16:0/4028651019 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f96a0009cc0 con 0x7f96a4198800 2024-04-02T04:54:03.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.126+0000 7f9696ffd640 1 --2- 172.21.15.16:0/4028651019 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f969006dcd0 0x7f9690070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:03.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.126+0000 7f9696ffd640 1 -- 172.21.15.16:0/4028651019 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f96a008d940 con 0x7f96a4198800 2024-04-02T04:54:03.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.126+0000 7f96a9f6c640 1 --2- 172.21.15.16:0/4028651019 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f969006dcd0 0x7f9690070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:03.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.127+0000 7f96a9f6c640 1 --2- 172.21.15.16:0/4028651019 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f969006dcd0 0x7f9690070190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9698009230 tx=0x7f969802cf90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:03.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.134+0000 7f9696ffd640 1 -- 172.21.15.16:0/4028651019 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f96a0056f60 con 0x7f96a4198800 2024-04-02T04:54:03.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.423+0000 7f96ac1d0640 1 -- 172.21.15.16:0/4028651019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f96a4199610 con 0x7f96a4198800 2024-04-02T04:54:03.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.424+0000 7f9696ffd640 1 -- 172.21.15.16:0/4028651019 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v29) v1 ==== 107+0+5063 (secure 0 0 0) 0x7f96a005a620 con 0x7f96a4198800 2024-04-02T04:54:03.429 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:54:03.429 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi016.uhgkgg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6827/81744708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":81744708},{"type":"v1","addr":"172.21.15.16:6827","nonce":81744708}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":34436,"name":"cephfs.smithi016.fyzrnm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6829/782430688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":782430688},{"type":"v1","addr":"172.21.15.16:6829","nonce":782430688}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":44359,"name":"cephfs.smithi027.zwrfoo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6825/1016194753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1016194753},{"type":"v1","addr":"172.21.15.27:6825","nonce":1016194753}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:39:30.740168+0000","modified":"2024-04-02T04:49:53.265580+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24295":{"gid":24295,"name":"cephfs.smithi027.sdublg","rank":0,"incarnation":21,"state":"up:active","state_seq":155,"addr":"172.21.15.27:6827/4177192740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":4177192740},{"type":"v1","addr":"172.21.15.27:6827","nonce":4177192740}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24295,"qdb_cluster":[24295]},"id":1}]} 2024-04-02T04:54:03.429 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-04-02T04:54:03.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.426+0000 7f96ac1d0640 1 -- 172.21.15.16:0/4028651019 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f969006dcd0 msgr2=0x7f9690070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:03.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.427+0000 7f96ac1d0640 1 --2- 172.21.15.16:0/4028651019 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f969006dcd0 0x7f9690070190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9698009230 tx=0x7f969802cf90 comp rx=0 tx=0).stop 2024-04-02T04:54:03.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.427+0000 7f96ac1d0640 1 -- 172.21.15.16:0/4028651019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a4198800 msgr2=0x7f96a419cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:03.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.427+0000 7f96ac1d0640 1 --2- 172.21.15.16:0/4028651019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a4198800 0x7f96a419cba0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f96a000d6f0 tx=0x7f96a000dbb0 comp rx=0 tx=0).stop 2024-04-02T04:54:03.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.427+0000 7f96ac1d0640 1 -- 172.21.15.16:0/4028651019 shutdown_connections 2024-04-02T04:54:03.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.427+0000 7f96ac1d0640 1 --2- 172.21.15.16:0/4028651019 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f969006dcd0 0x7f9690070190 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:03.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.427+0000 7f96ac1d0640 1 --2- 172.21.15.16:0/4028651019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a4198800 0x7f96a419cba0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:03.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.427+0000 7f96ac1d0640 1 --2- 172.21.15.16:0/4028651019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96a40ff1c0 0x7f96a41982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:03.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.427+0000 7f96ac1d0640 1 -- 172.21.15.16:0/4028651019 >> 172.21.15.16:0/4028651019 conn(0x7f96a4074960 msgr2=0x7f96a4105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:03.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.427+0000 7f96ac1d0640 1 -- 172.21.15.16:0/4028651019 shutdown_connections 2024-04-02T04:54:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.427+0000 7f96ac1d0640 1 -- 172.21.15.16:0/4028651019 wait complete. 2024-04-02T04:54:03.442 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-04-02T04:54:03.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.541+0000 7f57b28a4640 1 -- 172.21.15.16:0/2490258980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57ac067e60 msgr2=0x7f57ac104ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:03.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.541+0000 7f57b28a4640 1 --2- 172.21.15.16:0/2490258980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57ac067e60 0x7f57ac104ba0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f57a0009880 tx=0x7f57a002bce0 comp rx=0 tx=0).stop 2024-04-02T04:54:03.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.541+0000 7f57b28a4640 1 -- 172.21.15.16:0/2490258980 shutdown_connections 2024-04-02T04:54:03.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.541+0000 7f57b28a4640 1 --2- 172.21.15.16:0/2490258980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57ac067e60 0x7f57ac104ba0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:03.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.541+0000 7f57b28a4640 1 --2- 172.21.15.16:0/2490258980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57ac0674b0 0x7f57ac067890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:03.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.541+0000 7f57b28a4640 1 -- 172.21.15.16:0/2490258980 >> 172.21.15.16:0/2490258980 conn(0x7f57ac074960 msgr2=0x7f57ac074d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:03.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.541+0000 7f57b28a4640 1 -- 172.21.15.16:0/2490258980 shutdown_connections 2024-04-02T04:54:03.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.541+0000 7f57b28a4640 1 -- 172.21.15.16:0/2490258980 wait complete. 2024-04-02T04:54:03.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.542+0000 7f57b28a4640 1 Processor -- start 2024-04-02T04:54:03.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.542+0000 7f57b28a4640 1 -- start start 2024-04-02T04:54:03.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.542+0000 7f57b28a4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57ac0674b0 0x7f57ac1960d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:03.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.542+0000 7f57b28a4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57ac196610 0x7f57ac19a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:03.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.542+0000 7f57b28a4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ac196c10 con 0x7f57ac196610 2024-04-02T04:54:03.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.542+0000 7f57b28a4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ac196d80 con 0x7f57ac0674b0 2024-04-02T04:54:03.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.542+0000 7f57ab7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57ac196610 0x7f57ac19a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:03.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.542+0000 7f57ab7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57ac196610 0x7f57ac19a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58266/0 (socket says 172.21.15.16:58266) 2024-04-02T04:54:03.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.542+0000 7f57ab7fe640 1 -- 172.21.15.16:0/855434814 learned_addr learned my addr 172.21.15.16:0/855434814 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:54:03.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.542+0000 7f57ab7fe640 1 -- 172.21.15.16:0/855434814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57ac0674b0 msgr2=0x7f57ac1960d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:03.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.542+0000 7f57abfff640 1 --2- 172.21.15.16:0/855434814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57ac0674b0 0x7f57ac1960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:03.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.542+0000 7f57ab7fe640 1 --2- 172.21.15.16:0/855434814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57ac0674b0 0x7f57ac1960d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:03.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.542+0000 7f57ab7fe640 1 -- 172.21.15.16:0/855434814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a0009560 con 0x7f57ac196610 2024-04-02T04:54:03.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.543+0000 7f57ab7fe640 1 --2- 172.21.15.16:0/855434814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57ac196610 0x7f57ac19a960 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f57a0036040 tx=0x7f57a002c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:03.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.543+0000 7f57a97fa640 1 -- 172.21.15.16:0/855434814 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57a00035f0 con 0x7f57ac196610 2024-04-02T04:54:03.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.543+0000 7f57a97fa640 1 -- 172.21.15.16:0/855434814 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f57a0003c10 con 0x7f57ac196610 2024-04-02T04:54:03.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.543+0000 7f57b28a4640 1 -- 172.21.15.16:0/855434814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57ac19af00 con 0x7f57ac196610 2024-04-02T04:54:03.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.543+0000 7f57a97fa640 1 -- 172.21.15.16:0/855434814 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57a00055c0 con 0x7f57ac196610 2024-04-02T04:54:03.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.543+0000 7f57b28a4640 1 -- 172.21.15.16:0/855434814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57ac19b460 con 0x7f57ac196610 2024-04-02T04:54:03.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.544+0000 7f57b28a4640 1 -- 172.21.15.16:0/855434814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57ac101680 con 0x7f57ac196610 2024-04-02T04:54:03.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.545+0000 7f57a97fa640 1 -- 172.21.15.16:0/855434814 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f57a0004e60 con 0x7f57ac196610 2024-04-02T04:54:03.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.545+0000 7f57a97fa640 1 --2- 172.21.15.16:0/855434814 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f579406dcd0 0x7f5794070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:03.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.545+0000 7f57a97fa640 1 -- 172.21.15.16:0/855434814 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f57a00b0010 con 0x7f57ac196610 2024-04-02T04:54:03.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.545+0000 7f57abfff640 1 --2- 172.21.15.16:0/855434814 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f579406dcd0 0x7f5794070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:03.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.546+0000 7f57abfff640 1 --2- 172.21.15.16:0/855434814 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f579406dcd0 0x7f5794070190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f579c0058c0 tx=0x7f579c005c60 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.550+0000 7f57a97fa640 1 -- 172.21.15.16:0/855434814 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f57a0079500 con 0x7f57ac196610 2024-04-02T04:54:03.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:03 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/2952861451' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-04-02T04:54:03.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:03 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/4191847624' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-04-02T04:54:03.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.771+0000 7f57b28a4640 1 -- 172.21.15.16:0/855434814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7f57ac19b680 con 0x7f57ac196610 2024-04-02T04:54:03.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.772+0000 7f57a97fa640 1 -- 172.21.15.16:0/855434814 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v29) v1 ==== 107+0+4262 (secure 0 0 0) 0x7f57a007cbc0 con 0x7f57ac196610 2024-04-02T04:54:03.776 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:54:03.776 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi016.uhgkgg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6827/81744708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":81744708},{"type":"v1","addr":"172.21.15.16:6827","nonce":81744708}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":34436,"name":"cephfs.smithi016.fyzrnm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6829/782430688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":782430688},{"type":"v1","addr":"172.21.15.16:6829","nonce":782430688}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":44359,"name":"cephfs.smithi027.zwrfoo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6825/1016194753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1016194753},{"type":"v1","addr":"172.21.15.27:6825","nonce":1016194753}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:39:30.740168+0000","modified":"2024-04-02T04:50:07.887292+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-02T04:54:03.777 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-04-02T04:54:03.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.774+0000 7f57b28a4640 1 -- 172.21.15.16:0/855434814 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f579406dcd0 msgr2=0x7f5794070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:03.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.774+0000 7f57b28a4640 1 --2- 172.21.15.16:0/855434814 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f579406dcd0 0x7f5794070190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f579c0058c0 tx=0x7f579c005c60 comp rx=0 tx=0).stop 2024-04-02T04:54:03.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.774+0000 7f57b28a4640 1 -- 172.21.15.16:0/855434814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57ac196610 msgr2=0x7f57ac19a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:03.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.774+0000 7f57b28a4640 1 --2- 172.21.15.16:0/855434814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57ac196610 0x7f57ac19a960 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f57a0036040 tx=0x7f57a002c7c0 comp rx=0 tx=0).stop 2024-04-02T04:54:03.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.774+0000 7f57b28a4640 1 -- 172.21.15.16:0/855434814 shutdown_connections 2024-04-02T04:54:03.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.774+0000 7f57b28a4640 1 --2- 172.21.15.16:0/855434814 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f579406dcd0 0x7f5794070190 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:03.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.774+0000 7f57b28a4640 1 --2- 172.21.15.16:0/855434814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57ac196610 0x7f57ac19a960 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:03.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.774+0000 7f57b28a4640 1 --2- 172.21.15.16:0/855434814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57ac0674b0 0x7f57ac1960d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:03.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.774+0000 7f57b28a4640 1 -- 172.21.15.16:0/855434814 >> 172.21.15.16:0/855434814 conn(0x7f57ac074960 msgr2=0x7f57ac103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:03.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.774+0000 7f57b28a4640 1 -- 172.21.15.16:0/855434814 shutdown_connections 2024-04-02T04:54:03.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.774+0000 7f57b28a4640 1 -- 172.21.15.16:0/855434814 wait complete. 2024-04-02T04:54:03.788 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:03 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/2952861451' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-04-02T04:54:03.788 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:03 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/4191847624' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-04-02T04:54:03.790 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-04-02T04:54:03.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.888+0000 7fe91afd7640 1 -- 172.21.15.16:0/2004924091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9140ff1c0 msgr2=0x7fe9140ff5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:03.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.888+0000 7fe91afd7640 1 --2- 172.21.15.16:0/2004924091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9140ff1c0 0x7fe9140ff5a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe908009880 tx=0x7fe90802bce0 comp rx=0 tx=0).stop 2024-04-02T04:54:03.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.888+0000 7fe91afd7640 1 -- 172.21.15.16:0/2004924091 shutdown_connections 2024-04-02T04:54:03.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.888+0000 7fe91afd7640 1 --2- 172.21.15.16:0/2004924091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9140ffb70 0x7fe914103aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.888+0000 7fe91afd7640 1 --2- 172.21.15.16:0/2004924091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9140ff1c0 0x7fe9140ff5a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.888+0000 7fe91afd7640 1 -- 172.21.15.16:0/2004924091 >> 172.21.15.16:0/2004924091 conn(0x7fe914074960 msgr2=0x7fe914074d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.888+0000 7fe91afd7640 1 -- 172.21.15.16:0/2004924091 shutdown_connections 2024-04-02T04:54:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.888+0000 7fe91afd7640 1 -- 172.21.15.16:0/2004924091 wait complete. 2024-04-02T04:54:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.888+0000 7fe91afd7640 1 Processor -- start 2024-04-02T04:54:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.888+0000 7fe91afd7640 1 -- start start 2024-04-02T04:54:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.889+0000 7fe91afd7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9140ff1c0 0x7fe914196070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.889+0000 7fe91afd7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9141965b0 0x7fe91419a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.889+0000 7fe91afd7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe914196b20 con 0x7fe9140ff1c0 2024-04-02T04:54:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.889+0000 7fe91afd7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe914196c90 con 0x7fe9141965b0 2024-04-02T04:54:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.889+0000 7fe918d73640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9140ff1c0 0x7fe914196070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.889+0000 7fe918d73640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9140ff1c0 0x7fe914196070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58278/0 (socket says 172.21.15.16:58278) 2024-04-02T04:54:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.889+0000 7fe918d73640 1 -- 172.21.15.16:0/1051887210 learned_addr learned my addr 172.21.15.16:0/1051887210 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:54:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.889+0000 7fe913fff640 1 --2- 172.21.15.16:0/1051887210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9141965b0 0x7fe91419a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.889+0000 7fe918d73640 1 -- 172.21.15.16:0/1051887210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9141965b0 msgr2=0x7fe91419a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:03.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.889+0000 7fe918d73640 1 --2- 172.21.15.16:0/1051887210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9141965b0 0x7fe91419a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:03.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.889+0000 7fe918d73640 1 -- 172.21.15.16:0/1051887210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe908009560 con 0x7fe9140ff1c0 2024-04-02T04:54:03.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.889+0000 7fe918d73640 1 --2- 172.21.15.16:0/1051887210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9140ff1c0 0x7fe914196070 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe90802c220 tx=0x7fe908004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:03.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.889+0000 7fe911ffb640 1 -- 172.21.15.16:0/1051887210 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9080035f0 con 0x7fe9140ff1c0 2024-04-02T04:54:03.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.890+0000 7fe91afd7640 1 -- 172.21.15.16:0/1051887210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe91419aea0 con 0x7fe9140ff1c0 2024-04-02T04:54:03.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.890+0000 7fe911ffb640 1 -- 172.21.15.16:0/1051887210 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe908003c10 con 0x7fe9140ff1c0 2024-04-02T04:54:03.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.890+0000 7fe911ffb640 1 -- 172.21.15.16:0/1051887210 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe908034440 con 0x7fe9140ff1c0 2024-04-02T04:54:03.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.890+0000 7fe91afd7640 1 -- 172.21.15.16:0/1051887210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe91419b3b0 con 0x7fe9140ff1c0 2024-04-02T04:54:03.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.891+0000 7fe91afd7640 1 -- 172.21.15.16:0/1051887210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe914100580 con 0x7fe9140ff1c0 2024-04-02T04:54:03.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.891+0000 7fe911ffb640 1 -- 172.21.15.16:0/1051887210 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fe908004f60 con 0x7fe9140ff1c0 2024-04-02T04:54:03.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.891+0000 7fe911ffb640 1 --2- 172.21.15.16:0/1051887210 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fe90406dcd0 0x7fe904070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:03.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.892+0000 7fe911ffb640 1 -- 172.21.15.16:0/1051887210 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fe9080afff0 con 0x7fe9140ff1c0 2024-04-02T04:54:03.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.892+0000 7fe913fff640 1 --2- 172.21.15.16:0/1051887210 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fe90406dcd0 0x7fe904070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:03.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.892+0000 7fe913fff640 1 --2- 172.21.15.16:0/1051887210 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fe90406dcd0 0x7fe904070190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe900005e10 tx=0x7fe900005da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:03.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:03.896+0000 7fe911ffb640 1 -- 172.21.15.16:0/1051887210 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe9080794e0 con 0x7fe9140ff1c0 2024-04-02T04:54:04.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.118+0000 7fe91afd7640 1 -- 172.21.15.16:0/1051887210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7fe914197320 con 0x7fe9140ff1c0 2024-04-02T04:54:04.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.119+0000 7fe911ffb640 1 -- 172.21.15.16:0/1051887210 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v29) v1 ==== 107+0+4341 (secure 0 0 0) 0x7fe90807cba0 con 0x7fe9140ff1c0 2024-04-02T04:54:04.124 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:54:04.124 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi016.fyzrnm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6829/782430688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":782430688},{"type":"v1","addr":"172.21.15.16:6829","nonce":782430688}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":44359,"name":"cephfs.smithi027.zwrfoo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6825/1016194753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1016194753},{"type":"v1","addr":"172.21.15.27:6825","nonce":1016194753}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:39:30.740168+0000","modified":"2024-04-02T04:50:07.897053+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34404":{"gid":34404,"name":"cephfs.smithi016.uhgkgg","rank":0,"incarnation":26,"state":"up:replay","state_seq":1,"addr":"172.21.15.16:6827/81744708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":81744708},{"type":"v1","addr":"172.21.15.16:6827","nonce":81744708}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-02T04:54:04.124 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-04-02T04:54:04.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.121+0000 7fe91afd7640 1 -- 172.21.15.16:0/1051887210 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fe90406dcd0 msgr2=0x7fe904070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:04.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.121+0000 7fe91afd7640 1 --2- 172.21.15.16:0/1051887210 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fe90406dcd0 0x7fe904070190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe900005e10 tx=0x7fe900005da0 comp rx=0 tx=0).stop 2024-04-02T04:54:04.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.121+0000 7fe91afd7640 1 -- 172.21.15.16:0/1051887210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9140ff1c0 msgr2=0x7fe914196070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:04.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.121+0000 7fe91afd7640 1 --2- 172.21.15.16:0/1051887210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9140ff1c0 0x7fe914196070 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe90802c220 tx=0x7fe908004040 comp rx=0 tx=0).stop 2024-04-02T04:54:04.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.121+0000 7fe91afd7640 1 -- 172.21.15.16:0/1051887210 shutdown_connections 2024-04-02T04:54:04.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.121+0000 7fe91afd7640 1 --2- 172.21.15.16:0/1051887210 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fe90406dcd0 0x7fe904070190 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:04.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.121+0000 7fe91afd7640 1 --2- 172.21.15.16:0/1051887210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9141965b0 0x7fe91419a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:04.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.121+0000 7fe91afd7640 1 --2- 172.21.15.16:0/1051887210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9140ff1c0 0x7fe914196070 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:04.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.121+0000 7fe91afd7640 1 -- 172.21.15.16:0/1051887210 >> 172.21.15.16:0/1051887210 conn(0x7fe914074960 msgr2=0x7fe914105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:04.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.121+0000 7fe91afd7640 1 -- 172.21.15.16:0/1051887210 shutdown_connections 2024-04-02T04:54:04.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.121+0000 7fe91afd7640 1 -- 172.21.15.16:0/1051887210 wait complete. 2024-04-02T04:54:04.137 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-04-02T04:54:04.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.241+0000 7feb14350640 1 -- 172.21.15.16:0/3266253060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb0c067e60 msgr2=0x7feb0c104ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:04.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.241+0000 7feb14350640 1 --2- 172.21.15.16:0/3266253060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb0c067e60 0x7feb0c104ba0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7feafc009880 tx=0x7feafc02bce0 comp rx=0 tx=0).stop 2024-04-02T04:54:04.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.242+0000 7feb14350640 1 -- 172.21.15.16:0/3266253060 shutdown_connections 2024-04-02T04:54:04.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.242+0000 7feb14350640 1 --2- 172.21.15.16:0/3266253060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb0c067e60 0x7feb0c104ba0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:04.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.242+0000 7feb14350640 1 --2- 172.21.15.16:0/3266253060 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb0c0674b0 0x7feb0c067890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:04.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.242+0000 7feb14350640 1 -- 172.21.15.16:0/3266253060 >> 172.21.15.16:0/3266253060 conn(0x7feb0c074960 msgr2=0x7feb0c074d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:04.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.242+0000 7feb14350640 1 -- 172.21.15.16:0/3266253060 shutdown_connections 2024-04-02T04:54:04.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.242+0000 7feb14350640 1 -- 172.21.15.16:0/3266253060 wait complete. 2024-04-02T04:54:04.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.242+0000 7feb14350640 1 Processor -- start 2024-04-02T04:54:04.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.242+0000 7feb14350640 1 -- start start 2024-04-02T04:54:04.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.242+0000 7feb14350640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb0c0674b0 0x7feb0c198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:04.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.242+0000 7feb14350640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb0c1988c0 0x7feb0c19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:04.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.242+0000 7feb14350640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c198ec0 con 0x7feb0c0674b0 2024-04-02T04:54:04.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.242+0000 7feb14350640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c199030 con 0x7feb0c1988c0 2024-04-02T04:54:04.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.242+0000 7feb120ec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb0c0674b0 0x7feb0c198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:04.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.243+0000 7feb120ec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb0c0674b0 0x7feb0c198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58292/0 (socket says 172.21.15.16:58292) 2024-04-02T04:54:04.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.243+0000 7feb120ec640 1 -- 172.21.15.16:0/2610824663 learned_addr learned my addr 172.21.15.16:0/2610824663 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:54:04.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.243+0000 7feb118eb640 1 --2- 172.21.15.16:0/2610824663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb0c1988c0 0x7feb0c19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:04.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.243+0000 7feb120ec640 1 -- 172.21.15.16:0/2610824663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb0c1988c0 msgr2=0x7feb0c19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:04.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.243+0000 7feb120ec640 1 --2- 172.21.15.16:0/2610824663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb0c1988c0 0x7feb0c19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:04.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.243+0000 7feb120ec640 1 -- 172.21.15.16:0/2610824663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feafc009560 con 0x7feb0c0674b0 2024-04-02T04:54:04.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.243+0000 7feb118eb640 1 --2- 172.21.15.16:0/2610824663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb0c1988c0 0x7feb0c19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T04:54:04.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.243+0000 7feb120ec640 1 --2- 172.21.15.16:0/2610824663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb0c0674b0 0x7feb0c198380 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7feb0800b450 tx=0x7feb0800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:04.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.243+0000 7feb037fe640 1 -- 172.21.15.16:0/2610824663 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb08011860 con 0x7feb0c0674b0 2024-04-02T04:54:04.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.243+0000 7feb037fe640 1 -- 172.21.15.16:0/2610824663 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb08011e80 con 0x7feb0c0674b0 2024-04-02T04:54:04.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.243+0000 7feb14350640 1 -- 172.21.15.16:0/2610824663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb0c19d260 con 0x7feb0c0674b0 2024-04-02T04:54:04.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.243+0000 7feb037fe640 1 -- 172.21.15.16:0/2610824663 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb0800f570 con 0x7feb0c0674b0 2024-04-02T04:54:04.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.243+0000 7feb14350640 1 -- 172.21.15.16:0/2610824663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb0c19d7c0 con 0x7feb0c0674b0 2024-04-02T04:54:04.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.244+0000 7feb14350640 1 -- 172.21.15.16:0/2610824663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb0c101680 con 0x7feb0c0674b0 2024-04-02T04:54:04.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.245+0000 7feb037fe640 1 -- 172.21.15.16:0/2610824663 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7feb080103e0 con 0x7feb0c0674b0 2024-04-02T04:54:04.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.245+0000 7feb037fe640 1 --2- 172.21.15.16:0/2610824663 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7feaf806dcd0 0x7feaf8070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:04.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.245+0000 7feb118eb640 1 --2- 172.21.15.16:0/2610824663 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7feaf806dcd0 0x7feaf8070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:04.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.245+0000 7feb037fe640 1 -- 172.21.15.16:0/2610824663 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7feb0808db20 con 0x7feb0c0674b0 2024-04-02T04:54:04.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.246+0000 7feb118eb640 1 --2- 172.21.15.16:0/2610824663 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7feaf806dcd0 0x7feaf8070190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7feafc009230 tx=0x7feafc02cb40 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:04.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.250+0000 7feb037fe640 1 -- 172.21.15.16:0/2610824663 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7feb08057010 con 0x7feb0c0674b0 2024-04-02T04:54:04.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.506+0000 7feb14350640 1 -- 172.21.15.16:0/2610824663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7feb0c04e030 con 0x7feb0c0674b0 2024-04-02T04:54:04.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.507+0000 7feb037fe640 1 -- 172.21.15.16:0/2610824663 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v29) v1 ==== 107+0+4345 (secure 0 0 0) 0x7feb0805a6d0 con 0x7feb0c0674b0 2024-04-02T04:54:04.512 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:54:04.512 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi016.fyzrnm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6829/782430688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":782430688},{"type":"v1","addr":"172.21.15.16:6829","nonce":782430688}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":44359,"name":"cephfs.smithi027.zwrfoo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6825/1016194753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1016194753},{"type":"v1","addr":"172.21.15.27:6825","nonce":1016194753}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:39:30.740168+0000","modified":"2024-04-02T04:50:17.366507+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34404":{"gid":34404,"name":"cephfs.smithi016.uhgkgg","rank":0,"incarnation":26,"state":"up:reconnect","state_seq":19,"addr":"172.21.15.16:6827/81744708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":81744708},{"type":"v1","addr":"172.21.15.16:6827","nonce":81744708}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-02T04:54:04.512 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 27 2024-04-02T04:54:04.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.509+0000 7feb14350640 1 -- 172.21.15.16:0/2610824663 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7feaf806dcd0 msgr2=0x7feaf8070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:04.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.509+0000 7feb14350640 1 --2- 172.21.15.16:0/2610824663 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7feaf806dcd0 0x7feaf8070190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7feafc009230 tx=0x7feafc02cb40 comp rx=0 tx=0).stop 2024-04-02T04:54:04.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.509+0000 7feb14350640 1 -- 172.21.15.16:0/2610824663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb0c0674b0 msgr2=0x7feb0c198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:04.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.509+0000 7feb14350640 1 --2- 172.21.15.16:0/2610824663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb0c0674b0 0x7feb0c198380 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7feb0800b450 tx=0x7feb0800b910 comp rx=0 tx=0).stop 2024-04-02T04:54:04.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.509+0000 7feb14350640 1 -- 172.21.15.16:0/2610824663 shutdown_connections 2024-04-02T04:54:04.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.509+0000 7feb14350640 1 --2- 172.21.15.16:0/2610824663 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7feaf806dcd0 0x7feaf8070190 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:04.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.509+0000 7feb14350640 1 --2- 172.21.15.16:0/2610824663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb0c1988c0 0x7feb0c19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:04.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.509+0000 7feb14350640 1 --2- 172.21.15.16:0/2610824663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb0c0674b0 0x7feb0c198380 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:04.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.509+0000 7feb14350640 1 -- 172.21.15.16:0/2610824663 >> 172.21.15.16:0/2610824663 conn(0x7feb0c074960 msgr2=0x7feb0c101d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:04.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.509+0000 7feb14350640 1 -- 172.21.15.16:0/2610824663 shutdown_connections 2024-04-02T04:54:04.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.510+0000 7feb14350640 1 -- 172.21.15.16:0/2610824663 wait complete. 2024-04-02T04:54:04.525 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:04 smithi016 ceph-mon[93230]: pgmap v331: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:04.525 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:04 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/4028651019' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-04-02T04:54:04.525 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:04 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/855434814' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-04-02T04:54:04.525 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:04 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/1051887210' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-04-02T04:54:04.526 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-04-02T04:54:04.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.626+0000 7f8a4d759640 1 -- 172.21.15.16:0/510219770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a480674b0 msgr2=0x7f8a48067890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:04.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.626+0000 7f8a4d759640 1 --2- 172.21.15.16:0/510219770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a480674b0 0x7f8a48067890 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8a40009880 tx=0x7f8a4002bce0 comp rx=0 tx=0).stop 2024-04-02T04:54:04.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.626+0000 7f8a4d759640 1 -- 172.21.15.16:0/510219770 shutdown_connections 2024-04-02T04:54:04.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.626+0000 7f8a4d759640 1 --2- 172.21.15.16:0/510219770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a48067e60 0x7f8a48104ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:04.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.626+0000 7f8a4d759640 1 --2- 172.21.15.16:0/510219770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a480674b0 0x7f8a48067890 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:04.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.626+0000 7f8a4d759640 1 -- 172.21.15.16:0/510219770 >> 172.21.15.16:0/510219770 conn(0x7f8a48074960 msgr2=0x7f8a48074d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:04.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.626+0000 7f8a4d759640 1 -- 172.21.15.16:0/510219770 shutdown_connections 2024-04-02T04:54:04.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.626+0000 7f8a4d759640 1 -- 172.21.15.16:0/510219770 wait complete. 2024-04-02T04:54:04.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.626+0000 7f8a4d759640 1 Processor -- start 2024-04-02T04:54:04.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.627+0000 7f8a4d759640 1 -- start start 2024-04-02T04:54:04.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.627+0000 7f8a4d759640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a480674b0 0x7f8a48196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:04.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.627+0000 7f8a4d759640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a48196540 0x7f8a4819a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:04.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.627+0000 7f8a4d759640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a48196b40 con 0x7f8a48196540 2024-04-02T04:54:04.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.627+0000 7f8a4d759640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a48196cb0 con 0x7f8a480674b0 2024-04-02T04:54:04.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.627+0000 7f8a467fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a48196540 0x7f8a4819a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:04.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.627+0000 7f8a467fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a48196540 0x7f8a4819a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58302/0 (socket says 172.21.15.16:58302) 2024-04-02T04:54:04.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.627+0000 7f8a467fc640 1 -- 172.21.15.16:0/1886041607 learned_addr learned my addr 172.21.15.16:0/1886041607 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:54:04.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.627+0000 7f8a46ffd640 1 --2- 172.21.15.16:0/1886041607 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a480674b0 0x7f8a48196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:04.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.627+0000 7f8a467fc640 1 -- 172.21.15.16:0/1886041607 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a480674b0 msgr2=0x7f8a48196000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:04.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.627+0000 7f8a467fc640 1 --2- 172.21.15.16:0/1886041607 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a480674b0 0x7f8a48196000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:04.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.627+0000 7f8a467fc640 1 -- 172.21.15.16:0/1886041607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a40009560 con 0x7f8a48196540 2024-04-02T04:54:04.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.627+0000 7f8a467fc640 1 --2- 172.21.15.16:0/1886041607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a48196540 0x7f8a4819a8e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00d900 tx=0x7f8a3c00ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:04.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.627+0000 7f8a2ffff640 1 -- 172.21.15.16:0/1886041607 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a3c009c00 con 0x7f8a48196540 2024-04-02T04:54:04.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.628+0000 7f8a4d759640 1 -- 172.21.15.16:0/1886041607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a4819aee0 con 0x7f8a48196540 2024-04-02T04:54:04.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.628+0000 7f8a2ffff640 1 -- 172.21.15.16:0/1886041607 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8a3c009d60 con 0x7f8a48196540 2024-04-02T04:54:04.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.628+0000 7f8a2ffff640 1 -- 172.21.15.16:0/1886041607 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a3c003e50 con 0x7f8a48196540 2024-04-02T04:54:04.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.628+0000 7f8a4d759640 1 -- 172.21.15.16:0/1886041607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a4819b440 con 0x7f8a48196540 2024-04-02T04:54:04.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.629+0000 7f8a4d759640 1 -- 172.21.15.16:0/1886041607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a48101680 con 0x7f8a48196540 2024-04-02T04:54:04.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.629+0000 7f8a2ffff640 1 -- 172.21.15.16:0/1886041607 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f8a3c004000 con 0x7f8a48196540 2024-04-02T04:54:04.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.629+0000 7f8a2ffff640 1 --2- 172.21.15.16:0/1886041607 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f8a3006dc80 0x7f8a30070140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:04.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.630+0000 7f8a46ffd640 1 --2- 172.21.15.16:0/1886041607 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f8a3006dc80 0x7f8a30070140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:04.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.630+0000 7f8a2ffff640 1 -- 172.21.15.16:0/1886041607 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f8a3c08d6f0 con 0x7f8a48196540 2024-04-02T04:54:04.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.630+0000 7f8a46ffd640 1 --2- 172.21.15.16:0/1886041607 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f8a3006dc80 0x7f8a30070140 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8a40006010 tx=0x7f8a40005d50 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:04.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.634+0000 7f8a2ffff640 1 -- 172.21.15.16:0/1886041607 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8a3c056c10 con 0x7f8a48196540 2024-04-02T04:54:04.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:04 smithi027 ceph-mon[83036]: pgmap v331: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:04.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:04 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/4028651019' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-04-02T04:54:04.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:04 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/855434814' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-04-02T04:54:04.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:04 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/1051887210' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-04-02T04:54:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.875+0000 7f8a4d759640 1 -- 172.21.15.16:0/1886041607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7f8a4819b6c0 con 0x7f8a48196540 2024-04-02T04:54:04.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.876+0000 7f8a2ffff640 1 -- 172.21.15.16:0/1886041607 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v29) v1 ==== 107+0+4342 (secure 0 0 0) 0x7f8a3c05a2d0 con 0x7f8a48196540 2024-04-02T04:54:04.880 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:54:04.880 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi016.fyzrnm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6829/782430688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":782430688},{"type":"v1","addr":"172.21.15.16:6829","nonce":782430688}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":44359,"name":"cephfs.smithi027.zwrfoo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6825/1016194753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1016194753},{"type":"v1","addr":"172.21.15.27:6825","nonce":1016194753}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T04:39:30.740168+0000","modified":"2024-04-02T04:50:18.101815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34404":{"gid":34404,"name":"cephfs.smithi016.uhgkgg","rank":0,"incarnation":26,"state":"up:rejoin","state_seq":20,"addr":"172.21.15.16:6827/81744708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":81744708},{"type":"v1","addr":"172.21.15.16:6827","nonce":81744708}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-02T04:54:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-04-02T04:54:04.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.878+0000 7f8a4d759640 1 -- 172.21.15.16:0/1886041607 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f8a3006dc80 msgr2=0x7f8a30070140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:04.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.878+0000 7f8a4d759640 1 --2- 172.21.15.16:0/1886041607 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f8a3006dc80 0x7f8a30070140 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8a40006010 tx=0x7f8a40005d50 comp rx=0 tx=0).stop 2024-04-02T04:54:04.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.878+0000 7f8a4d759640 1 -- 172.21.15.16:0/1886041607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a48196540 msgr2=0x7f8a4819a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:04.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.878+0000 7f8a4d759640 1 --2- 172.21.15.16:0/1886041607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a48196540 0x7f8a4819a8e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00d900 tx=0x7f8a3c00ddc0 comp rx=0 tx=0).stop 2024-04-02T04:54:04.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.878+0000 7f8a4d759640 1 -- 172.21.15.16:0/1886041607 shutdown_connections 2024-04-02T04:54:04.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.878+0000 7f8a4d759640 1 --2- 172.21.15.16:0/1886041607 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f8a3006dc80 0x7f8a30070140 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:04.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.878+0000 7f8a4d759640 1 --2- 172.21.15.16:0/1886041607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a48196540 0x7f8a4819a8e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:04.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.878+0000 7f8a4d759640 1 --2- 172.21.15.16:0/1886041607 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a480674b0 0x7f8a48196000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:04.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.878+0000 7f8a4d759640 1 -- 172.21.15.16:0/1886041607 >> 172.21.15.16:0/1886041607 conn(0x7f8a48074960 msgr2=0x7f8a48103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:04.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.878+0000 7f8a4d759640 1 -- 172.21.15.16:0/1886041607 shutdown_connections 2024-04-02T04:54:04.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:04.878+0000 7f8a4d759640 1 -- 172.21.15.16:0/1886041607 wait complete. 2024-04-02T04:54:04.894 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-02T04:54:04.906 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-02T04:54:04.906 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-02T04:54:04.906 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T04:54:04.923 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-02T04:54:04.923 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T04:54:04.979 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T04:54:05.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.120+0000 7f833c22d640 1 -- 172.21.15.16:0/4143581620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8334101a40 msgr2=0x7f8334109f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:05.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.120+0000 7f833c22d640 1 --2- 172.21.15.16:0/4143581620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8334101a40 0x7f8334109f20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8330009880 tx=0x7f833002bd70 comp rx=0 tx=0).stop 2024-04-02T04:54:05.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.120+0000 7f833c22d640 1 -- 172.21.15.16:0/4143581620 shutdown_connections 2024-04-02T04:54:05.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.120+0000 7f833c22d640 1 --2- 172.21.15.16:0/4143581620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8334101a40 0x7f8334109f20 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:05.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.120+0000 7f833c22d640 1 --2- 172.21.15.16:0/4143581620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8334101120 0x7f8334101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:05.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.120+0000 7f833c22d640 1 -- 172.21.15.16:0/4143581620 >> 172.21.15.16:0/4143581620 conn(0x7f83340fa940 msgr2=0x7f83340fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:05.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.120+0000 7f833c22d640 1 -- 172.21.15.16:0/4143581620 shutdown_connections 2024-04-02T04:54:05.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.120+0000 7f833c22d640 1 -- 172.21.15.16:0/4143581620 wait complete. 2024-04-02T04:54:05.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.121+0000 7f833c22d640 1 Processor -- start 2024-04-02T04:54:05.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.121+0000 7f833c22d640 1 -- start start 2024-04-02T04:54:05.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.121+0000 7f833c22d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8334101120 0x7f8334195fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:05.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.121+0000 7f833c22d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8334101a40 0x7f8334196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:05.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.121+0000 7f833c22d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8334196bb0 con 0x7f8334101120 2024-04-02T04:54:05.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.121+0000 7f833c22d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833419a8c0 con 0x7f8334101a40 2024-04-02T04:54:05.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.121+0000 7f8339fc9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8334101120 0x7f8334195fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:05.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.121+0000 7f8339fc9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8334101120 0x7f8334195fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58306/0 (socket says 172.21.15.16:58306) 2024-04-02T04:54:05.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.121+0000 7f8339fc9640 1 -- 172.21.15.16:0/3324673621 learned_addr learned my addr 172.21.15.16:0/3324673621 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:54:05.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.121+0000 7f83397c8640 1 --2- 172.21.15.16:0/3324673621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8334101a40 0x7f8334196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:05.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.121+0000 7f8339fc9640 1 -- 172.21.15.16:0/3324673621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8334101a40 msgr2=0x7f8334196520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:05.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.121+0000 7f8339fc9640 1 --2- 172.21.15.16:0/3324673621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8334101a40 0x7f8334196520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:05.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.121+0000 7f8339fc9640 1 -- 172.21.15.16:0/3324673621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8330009560 con 0x7f8334101120 2024-04-02T04:54:05.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.121+0000 7f83397c8640 1 --2- 172.21.15.16:0/3324673621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8334101a40 0x7f8334196520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T04:54:05.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.122+0000 7f8339fc9640 1 --2- 172.21.15.16:0/3324673621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8334101120 0x7f8334195fe0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f832400e970 tx=0x7f832400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:05.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.122+0000 7f832affd640 1 -- 172.21.15.16:0/3324673621 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f832400cc90 con 0x7f8334101120 2024-04-02T04:54:05.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.122+0000 7f832affd640 1 -- 172.21.15.16:0/3324673621 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f832400cdf0 con 0x7f8334101120 2024-04-02T04:54:05.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.122+0000 7f833c22d640 1 -- 172.21.15.16:0/3324673621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f833419aba0 con 0x7f8334101120 2024-04-02T04:54:05.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.122+0000 7f832affd640 1 -- 172.21.15.16:0/3324673621 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8324010430 con 0x7f8334101120 2024-04-02T04:54:05.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.122+0000 7f833c22d640 1 -- 172.21.15.16:0/3324673621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f833419b100 con 0x7f8334101120 2024-04-02T04:54:05.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.123+0000 7f833c22d640 1 -- 172.21.15.16:0/3324673621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83340686f0 con 0x7f8334101120 2024-04-02T04:54:05.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.123+0000 7f832affd640 1 -- 172.21.15.16:0/3324673621 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f83240046c0 con 0x7f8334101120 2024-04-02T04:54:05.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.124+0000 7f832affd640 1 --2- 172.21.15.16:0/3324673621 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f832006dcd0 0x7f8320070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:05.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.124+0000 7f83397c8640 1 --2- 172.21.15.16:0/3324673621 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f832006dcd0 0x7f8320070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:05.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.124+0000 7f832affd640 1 -- 172.21.15.16:0/3324673621 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f8324014070 con 0x7f8334101120 2024-04-02T04:54:05.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.125+0000 7f83397c8640 1 --2- 172.21.15.16:0/3324673621 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f832006dcd0 0x7f8320070190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8334197530 tx=0x7f8330005a20 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:05.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.128+0000 7f832affd640 1 -- 172.21.15.16:0/3324673621 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f83240576c0 con 0x7f8334101120 2024-04-02T04:54:05.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.379+0000 7f833c22d640 1 -- 172.21.15.16:0/3324673621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83341046f0 con 0x7f8334101120 2024-04-02T04:54:05.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.380+0000 7f832affd640 1 -- 172.21.15.16:0/3324673621 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16859 (secure 0 0 0) 0x7f832405ad80 con 0x7f8334101120 2024-04-02T04:54:05.385 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:54:05.385 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":84,"fsid":"dc5403b4-f0a9-11ee-b647-cb9ed24678a4","created":"2024-04-02T04:32:23.014546+0000","modified":"2024-04-02T04:50:07.887023+0000","last_up_change":"2024-04-02T04:48:57.379242+0000","last_in_change":"2024-04-02T04:37:56.807214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T04:37:01.047338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-02T04:39:29.039519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-02T04:39:29.992892+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2b7be29b-0e2a-456b-b33e-c72abaad9136","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":78,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1136760904},{"type":"v1","addr":"172.21.15.16:6803","nonce":1136760904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1136760904},{"type":"v1","addr":"172.21.15.16:6805","nonce":1136760904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1136760904},{"type":"v1","addr":"172.21.15.16:6809","nonce":1136760904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1136760904},{"type":"v1","addr":"172.21.15.16:6807","nonce":1136760904}]},"public_addr":"172.21.15.16:6803/1136760904","cluster_addr":"172.21.15.16:6805/1136760904","heartbeat_back_addr":"172.21.15.16:6809/1136760904","heartbeat_front_addr":"172.21.15.16:6807/1136760904","state":["exists","up"]},{"osd":1,"uuid":"82578291-3c9c-4f3f-8de9-7a8710a4923e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":78,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2744685333},{"type":"v1","addr":"172.21.15.16:6811","nonce":2744685333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2744685333},{"type":"v1","addr":"172.21.15.16:6813","nonce":2744685333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2744685333},{"type":"v1","addr":"172.21.15.16:6817","nonce":2744685333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2744685333},{"type":"v1","addr":"172.21.15.16:6815","nonce":2744685333}]},"public_addr":"172.21.15.16:6811/2744685333","cluster_addr":"172.21.15.16:6813/2744685333","heartbeat_back_addr":"172.21.15.16:6817/2744685333","heartbeat_front_addr":"172.21.15.16:6815/2744685333","state":["exists","up"]},{"osd":2,"uuid":"046262b5-386e-4270-863c-e97a61a038b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":59,"up_from":63,"up_thru":78,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1004843818},{"type":"v1","addr":"172.21.15.16:6819","nonce":1004843818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1004843818},{"type":"v1","addr":"172.21.15.16:6821","nonce":1004843818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1004843818},{"type":"v1","addr":"172.21.15.16:6825","nonce":1004843818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1004843818},{"type":"v1","addr":"172.21.15.16:6823","nonce":1004843818}]},"public_addr":"172.21.15.16:6819/1004843818","cluster_addr":"172.21.15.16:6821/1004843818","heartbeat_back_addr":"172.21.15.16:6825/1004843818","heartbeat_front_addr":"172.21.15.16:6823/1004843818","state":["exists","up"]},{"osd":3,"uuid":"637f2a77-59a2-44fb-b47e-fcbd70f4423c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":64,"up_from":68,"up_thru":78,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":4092279016},{"type":"v1","addr":"172.21.15.27:6801","nonce":4092279016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4092279016},{"type":"v1","addr":"172.21.15.27:6803","nonce":4092279016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4092279016},{"type":"v1","addr":"172.21.15.27:6807","nonce":4092279016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4092279016},{"type":"v1","addr":"172.21.15.27:6805","nonce":4092279016}]},"public_addr":"172.21.15.27:6801/4092279016","cluster_addr":"172.21.15.27:6803/4092279016","heartbeat_back_addr":"172.21.15.27:6807/4092279016","heartbeat_front_addr":"172.21.15.27:6805/4092279016","state":["exists","up"]},{"osd":4,"uuid":"334b7b3d-505f-45b6-85b0-59ec8eeb8e91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":69,"up_from":73,"up_thru":78,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":2931886568},{"type":"v1","addr":"172.21.15.27:6809","nonce":2931886568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2931886568},{"type":"v1","addr":"172.21.15.27:6811","nonce":2931886568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2931886568},{"type":"v1","addr":"172.21.15.27:6815","nonce":2931886568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2931886568},{"type":"v1","addr":"172.21.15.27:6813","nonce":2931886568}]},"public_addr":"172.21.15.27:6809/2931886568","cluster_addr":"172.21.15.27:6811/2931886568","heartbeat_back_addr":"172.21.15.27:6815/2931886568","heartbeat_front_addr":"172.21.15.27:6813/2931886568","state":["exists","up"]},{"osd":5,"uuid":"f83802ee-4f47-47de-91dd-a667f6cd22e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":74,"up_from":78,"up_thru":78,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":401102788},{"type":"v1","addr":"172.21.15.27:6817","nonce":401102788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":401102788},{"type":"v1","addr":"172.21.15.27:6819","nonce":401102788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":401102788},{"type":"v1","addr":"172.21.15.27:6823","nonce":401102788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":401102788},{"type":"v1","addr":"172.21.15.27:6821","nonce":401102788}]},"public_addr":"172.21.15.27:6817/401102788","cluster_addr":"172.21.15.27:6819/401102788","heartbeat_back_addr":"172.21.15.27:6823/401102788","heartbeat_front_addr":"172.21.15.27:6821/401102788","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:36:02.027852+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:36:29.393593+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:36:56.847224+0000","dead_epoch":59},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:37:21.195876+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:37:47.992167+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:38:15.012477+0000","dead_epoch":74}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6827/4177192740":"2024-04-03T04:50:07.886976+0000","172.21.15.27:6824/1461455976":"2024-04-03T04:49:41.368223+0000","172.21.15.16:0/2790817266":"2024-04-03T04:44:17.011409+0000","172.21.15.16:0/1537212641":"2024-04-03T04:44:17.011409+0000","172.21.15.16:0/3646287833":"2024-04-03T04:44:17.011409+0000","172.21.15.16:6800/873531241":"2024-04-03T04:33:28.907145+0000","172.21.15.16:0/2255383579":"2024-04-03T04:32:56.644452+0000","172.21.15.16:6800/2776599951":"2024-04-03T04:44:17.011409+0000","172.21.15.16:0/3964929492":"2024-04-03T04:44:17.011409+0000","172.21.15.27:0/597316902":"2024-04-03T04:43:10.596160+0000","172.21.15.16:0/4067493740":"2024-04-03T04:33:28.907145+0000","172.21.15.16:6801/3912038174":"2024-04-03T04:34:42.713541+0000","172.21.15.16:6800/3912038174":"2024-04-03T04:34:42.713541+0000","172.21.15.16:0/1664387083":"2024-04-03T04:42:07.942513+0000","172.21.15.27:6825/1461455976":"2024-04-03T04:49:41.368223+0000","172.21.15.27:0/1314131858":"2024-04-03T04:43:10.596160+0000","172.21.15.27:6826/4177192740":"2024-04-03T04:50:07.886976+0000","172.21.15.16:6801/873531241":"2024-04-03T04:33:28.907145+0000","172.21.15.16:0/401549934":"2024-04-03T04:32:56.644452+0000","172.21.15.16:6801/3643600900":"2024-04-03T04:42:07.942513+0000","172.21.15.16:0/166410319":"2024-04-03T04:33:28.907145+0000","172.21.15.16:0/1884756010":"2024-04-03T04:32:56.644452+0000","172.21.15.16:6801/824107799":"2024-04-03T04:32:56.644452+0000","172.21.15.16:0/881481317":"2024-04-03T04:44:17.011409+0000","172.21.15.16:6800/3643600900":"2024-04-03T04:42:07.942513+0000","172.21.15.16:0/4209328561":"2024-04-03T04:34:42.713541+0000","172.21.15.16:0/1920858754":"2024-04-03T04:42:07.942513+0000","172.21.15.16:0/3733852550":"2024-04-03T04:34:42.713541+0000","172.21.15.16:6800/824107799":"2024-04-03T04:32:56.644452+0000","172.21.15.27:0/3186228192":"2024-04-03T04:43:10.596160+0000","172.21.15.27:0/1987939490":"2024-04-03T04:43:10.596160+0000","172.21.15.16:0/3405957790":"2024-04-03T04:34:42.713541+0000","172.21.15.16:0/1383394471":"2024-04-03T04:42:07.942513+0000","172.21.15.27:6828/1554339939":"2024-04-03T04:43:10.596160+0000","172.21.15.16:0/1228221358":"2024-04-03T04:33:28.907145+0000","172.21.15.16:0/2373084595":"2024-04-03T04:42:07.942513+0000","172.21.15.16:0/249069849":"2024-04-03T04:44:17.011409+0000","172.21.15.27:0/3624631480":"2024-04-03T04:43:10.596160+0000","172.21.15.27:0/2708474610":"2024-04-03T04:43:10.596160+0000","172.21.15.16:6826/71509234":"2024-04-03T04:39:46.861330+0000","172.21.15.16:6827/71509234":"2024-04-03T04:39:46.861330+0000","172.21.15.27:6829/1554339939":"2024-04-03T04:43:10.596160+0000","172.21.15.16:6801/2776599951":"2024-04-03T04:44:17.011409+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T04:54:05.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.383+0000 7f833c22d640 1 -- 172.21.15.16:0/3324673621 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f832006dcd0 msgr2=0x7f8320070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:05.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.383+0000 7f833c22d640 1 --2- 172.21.15.16:0/3324673621 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f832006dcd0 0x7f8320070190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8334197530 tx=0x7f8330005a20 comp rx=0 tx=0).stop 2024-04-02T04:54:05.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.383+0000 7f833c22d640 1 -- 172.21.15.16:0/3324673621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8334101120 msgr2=0x7f8334195fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:05.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.383+0000 7f833c22d640 1 --2- 172.21.15.16:0/3324673621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8334101120 0x7f8334195fe0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f832400e970 tx=0x7f832400ed70 comp rx=0 tx=0).stop 2024-04-02T04:54:05.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.383+0000 7f833c22d640 1 -- 172.21.15.16:0/3324673621 shutdown_connections 2024-04-02T04:54:05.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.383+0000 7f833c22d640 1 --2- 172.21.15.16:0/3324673621 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f832006dcd0 0x7f8320070190 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:05.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.383+0000 7f833c22d640 1 --2- 172.21.15.16:0/3324673621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8334101a40 0x7f8334196520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:05.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.383+0000 7f833c22d640 1 --2- 172.21.15.16:0/3324673621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8334101120 0x7f8334195fe0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:05.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.383+0000 7f833c22d640 1 -- 172.21.15.16:0/3324673621 >> 172.21.15.16:0/3324673621 conn(0x7f83340fa940 msgr2=0x7f8334102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:05.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.383+0000 7f833c22d640 1 -- 172.21.15.16:0/3324673621 shutdown_connections 2024-04-02T04:54:05.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.383+0000 7f833c22d640 1 -- 172.21.15.16:0/3324673621 wait complete. 2024-04-02T04:54:05.399 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T04:37:01.047338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-02T04:39:29.039519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-02T04:39:29.992892+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T04:54:05.400 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T04:54:05.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.509+0000 7fc0be920640 1 -- 172.21.15.16:0/3225172330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0b80ffa40 msgr2=0x7fc0b8103a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:05.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.509+0000 7fc0be920640 1 --2- 172.21.15.16:0/3225172330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0b80ffa40 0x7fc0b8103a80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac009880 tx=0x7fc0ac02bce0 comp rx=0 tx=0).stop 2024-04-02T04:54:05.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.509+0000 7fc0be920640 1 -- 172.21.15.16:0/3225172330 shutdown_connections 2024-04-02T04:54:05.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.509+0000 7fc0be920640 1 --2- 172.21.15.16:0/3225172330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0b80ffa40 0x7fc0b8103a80 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:05.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.509+0000 7fc0be920640 1 --2- 172.21.15.16:0/3225172330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0b80ff090 0x7fc0b80ff470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:05.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.509+0000 7fc0be920640 1 -- 172.21.15.16:0/3225172330 >> 172.21.15.16:0/3225172330 conn(0x7fc0b80fa8f0 msgr2=0x7fc0b80fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:05.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.509+0000 7fc0be920640 1 -- 172.21.15.16:0/3225172330 shutdown_connections 2024-04-02T04:54:05.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.509+0000 7fc0be920640 1 -- 172.21.15.16:0/3225172330 wait complete. 2024-04-02T04:54:05.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.509+0000 7fc0be920640 1 Processor -- start 2024-04-02T04:54:05.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.509+0000 7fc0be920640 1 -- start start 2024-04-02T04:54:05.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.510+0000 7fc0be920640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0b80ff090 0x7fc0b8196dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:05.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.510+0000 7fc0be920640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0b8197310 0x7fc0b8197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:05.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.510+0000 7fc0be920640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b819fe40 con 0x7fc0b8197310 2024-04-02T04:54:05.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.510+0000 7fc0be920640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b819ff80 con 0x7fc0b80ff090 2024-04-02T04:54:05.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.510+0000 7fc0b77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0b8197310 0x7fc0b8197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:05.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.510+0000 7fc0b77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0b8197310 0x7fc0b8197770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58326/0 (socket says 172.21.15.16:58326) 2024-04-02T04:54:05.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.510+0000 7fc0b77fe640 1 -- 172.21.15.16:0/3267900482 learned_addr learned my addr 172.21.15.16:0/3267900482 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:54:05.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.510+0000 7fc0b7fff640 1 --2- 172.21.15.16:0/3267900482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0b80ff090 0x7fc0b8196dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:05.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.510+0000 7fc0b77fe640 1 -- 172.21.15.16:0/3267900482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0b80ff090 msgr2=0x7fc0b8196dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:05.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.510+0000 7fc0b77fe640 1 --2- 172.21.15.16:0/3267900482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0b80ff090 0x7fc0b8196dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:05.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.510+0000 7fc0b77fe640 1 -- 172.21.15.16:0/3267900482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0ac009560 con 0x7fc0b8197310 2024-04-02T04:54:05.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.511+0000 7fc0b77fe640 1 --2- 172.21.15.16:0/3267900482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0b8197310 0x7fc0b8197770 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac036040 tx=0x7fc0ac02c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:05.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.511+0000 7fc0b57fa640 1 -- 172.21.15.16:0/3267900482 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0ac0035f0 con 0x7fc0b8197310 2024-04-02T04:54:05.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.511+0000 7fc0b57fa640 1 -- 172.21.15.16:0/3267900482 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc0ac003c10 con 0x7fc0b8197310 2024-04-02T04:54:05.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.511+0000 7fc0b57fa640 1 -- 172.21.15.16:0/3267900482 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0ac0055f0 con 0x7fc0b8197310 2024-04-02T04:54:05.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.511+0000 7fc0be920640 1 -- 172.21.15.16:0/3267900482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0b81a00c0 con 0x7fc0b8197310 2024-04-02T04:54:05.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.511+0000 7fc0be920640 1 -- 172.21.15.16:0/3267900482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0b81a0590 con 0x7fc0b8197310 2024-04-02T04:54:05.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.512+0000 7fc0be920640 1 -- 172.21.15.16:0/3267900482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0b8100560 con 0x7fc0b8197310 2024-04-02T04:54:05.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.512+0000 7fc0b57fa640 1 -- 172.21.15.16:0/3267900482 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fc0ac004e60 con 0x7fc0b8197310 2024-04-02T04:54:05.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.513+0000 7fc0b57fa640 1 --2- 172.21.15.16:0/3267900482 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc0a006dcf0 0x7fc0a00701b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:05.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.513+0000 7fc0b57fa640 1 -- 172.21.15.16:0/3267900482 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fc0ac0afd70 con 0x7fc0b8197310 2024-04-02T04:54:05.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.513+0000 7fc0b7fff640 1 --2- 172.21.15.16:0/3267900482 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc0a006dcf0 0x7fc0a00701b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:05.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.513+0000 7fc0b7fff640 1 --2- 172.21.15.16:0/3267900482 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc0a006dcf0 0x7fc0a00701b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc0a800bbe0 tx=0x7fc0a800b390 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:05.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.520+0000 7fc0b57fa640 1 -- 172.21.15.16:0/3267900482 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc0ac079390 con 0x7fc0b8197310 2024-04-02T04:54:05.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:05 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/2610824663' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-04-02T04:54:05.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:05 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/1886041607' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-04-02T04:54:05.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.702+0000 7fc0be920640 1 -- 172.21.15.16:0/3267900482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc0b8107430 con 0x7fc0b8197310 2024-04-02T04:54:05.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.703+0000 7fc0b57fa640 1 -- 172.21.15.16:0/3267900482 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc0ac0410a0 con 0x7fc0b8197310 2024-04-02T04:54:05.707 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-02T04:54:05.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.705+0000 7fc0be920640 1 -- 172.21.15.16:0/3267900482 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc0a006dcf0 msgr2=0x7fc0a00701b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:05.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.705+0000 7fc0be920640 1 --2- 172.21.15.16:0/3267900482 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc0a006dcf0 0x7fc0a00701b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc0a800bbe0 tx=0x7fc0a800b390 comp rx=0 tx=0).stop 2024-04-02T04:54:05.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.705+0000 7fc0be920640 1 -- 172.21.15.16:0/3267900482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0b8197310 msgr2=0x7fc0b8197770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:05.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.705+0000 7fc0be920640 1 --2- 172.21.15.16:0/3267900482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0b8197310 0x7fc0b8197770 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac036040 tx=0x7fc0ac02c7c0 comp rx=0 tx=0).stop 2024-04-02T04:54:05.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.705+0000 7fc0be920640 1 -- 172.21.15.16:0/3267900482 shutdown_connections 2024-04-02T04:54:05.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.705+0000 7fc0be920640 1 --2- 172.21.15.16:0/3267900482 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fc0a006dcf0 0x7fc0a00701b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:05.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.705+0000 7fc0be920640 1 --2- 172.21.15.16:0/3267900482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0b8197310 0x7fc0b8197770 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:05.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.705+0000 7fc0be920640 1 --2- 172.21.15.16:0/3267900482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0b80ff090 0x7fc0b8196dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:05.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.705+0000 7fc0be920640 1 -- 172.21.15.16:0/3267900482 >> 172.21.15.16:0/3267900482 conn(0x7fc0b80fa8f0 msgr2=0x7fc0b80fbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:05.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.705+0000 7fc0be920640 1 -- 172.21.15.16:0/3267900482 shutdown_connections 2024-04-02T04:54:05.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.705+0000 7fc0be920640 1 -- 172.21.15.16:0/3267900482 wait complete. 2024-04-02T04:54:05.723 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:05 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/2610824663' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-04-02T04:54:05.723 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:05 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/1886041607' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-04-02T04:54:05.724 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-02T04:54:05.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.824+0000 7ff2affab640 1 -- 172.21.15.16:0/2998526456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a8101ac0 msgr2=0x7ff2a8109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:05.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.824+0000 7ff2affab640 1 --2- 172.21.15.16:0/2998526456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a8101ac0 0x7ff2a8109fa0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff29c009880 tx=0x7ff29c02bd70 comp rx=0 tx=0).stop 2024-04-02T04:54:05.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.824+0000 7ff2affab640 1 -- 172.21.15.16:0/2998526456 shutdown_connections 2024-04-02T04:54:05.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.824+0000 7ff2affab640 1 --2- 172.21.15.16:0/2998526456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a8101ac0 0x7ff2a8109fa0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:05.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.824+0000 7ff2affab640 1 --2- 172.21.15.16:0/2998526456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2a81011a0 0x7ff2a8101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:05.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.824+0000 7ff2affab640 1 -- 172.21.15.16:0/2998526456 >> 172.21.15.16:0/2998526456 conn(0x7ff2a80fa900 msgr2=0x7ff2a80fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:05.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.824+0000 7ff2affab640 1 -- 172.21.15.16:0/2998526456 shutdown_connections 2024-04-02T04:54:05.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.824+0000 7ff2affab640 1 -- 172.21.15.16:0/2998526456 wait complete. 2024-04-02T04:54:05.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.825+0000 7ff2affab640 1 Processor -- start 2024-04-02T04:54:05.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.825+0000 7ff2affab640 1 -- start start 2024-04-02T04:54:05.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.825+0000 7ff2affab640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2a81011a0 0x7ff2a8195f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:05.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.825+0000 7ff2affab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a8101ac0 0x7ff2a8196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:05.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.825+0000 7ff2affab640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a8196b20 con 0x7ff2a8101ac0 2024-04-02T04:54:05.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.825+0000 7ff2affab640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a819a830 con 0x7ff2a81011a0 2024-04-02T04:54:05.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.825+0000 7ff2ad546640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a8101ac0 0x7ff2a8196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:05.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.825+0000 7ff2ad546640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a8101ac0 0x7ff2a8196490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58336/0 (socket says 172.21.15.16:58336) 2024-04-02T04:54:05.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.825+0000 7ff2ad546640 1 -- 172.21.15.16:0/1828300041 learned_addr learned my addr 172.21.15.16:0/1828300041 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:54:05.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.825+0000 7ff2add47640 1 --2- 172.21.15.16:0/1828300041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2a81011a0 0x7ff2a8195f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:05.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.825+0000 7ff2ad546640 1 -- 172.21.15.16:0/1828300041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2a81011a0 msgr2=0x7ff2a8195f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:05.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.825+0000 7ff2ad546640 1 --2- 172.21.15.16:0/1828300041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2a81011a0 0x7ff2a8195f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:05.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.825+0000 7ff2ad546640 1 -- 172.21.15.16:0/1828300041 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29c009560 con 0x7ff2a8101ac0 2024-04-02T04:54:05.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.825+0000 7ff2add47640 1 --2- 172.21.15.16:0/1828300041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2a81011a0 0x7ff2a8195f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T04:54:05.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.825+0000 7ff2ad546640 1 --2- 172.21.15.16:0/1828300041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a8101ac0 0x7ff2a8196490 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff29c02c2b0 tx=0x7ff29c02c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:05.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.825+0000 7ff2a6ffd640 1 -- 172.21.15.16:0/1828300041 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff29c004520 con 0x7ff2a8101ac0 2024-04-02T04:54:05.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.826+0000 7ff2affab640 1 -- 172.21.15.16:0/1828300041 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a819aab0 con 0x7ff2a8101ac0 2024-04-02T04:54:05.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.826+0000 7ff2a6ffd640 1 -- 172.21.15.16:0/1828300041 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff29c003ab0 con 0x7ff2a8101ac0 2024-04-02T04:54:05.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.826+0000 7ff2a6ffd640 1 -- 172.21.15.16:0/1828300041 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff29c005500 con 0x7ff2a8101ac0 2024-04-02T04:54:05.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.826+0000 7ff2affab640 1 -- 172.21.15.16:0/1828300041 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a819b010 con 0x7ff2a8101ac0 2024-04-02T04:54:05.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.827+0000 7ff2affab640 1 -- 172.21.15.16:0/1828300041 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a8068be0 con 0x7ff2a8101ac0 2024-04-02T04:54:05.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.827+0000 7ff2a6ffd640 1 -- 172.21.15.16:0/1828300041 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7ff29c003c20 con 0x7ff2a8101ac0 2024-04-02T04:54:05.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.828+0000 7ff2a6ffd640 1 --2- 172.21.15.16:0/1828300041 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ff29006dcd0 0x7ff290070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:05.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.828+0000 7ff2a6ffd640 1 -- 172.21.15.16:0/1828300041 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7ff29c0b0680 con 0x7ff2a8101ac0 2024-04-02T04:54:05.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.828+0000 7ff2add47640 1 --2- 172.21.15.16:0/1828300041 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ff29006dcd0 0x7ff290070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:05.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.828+0000 7ff2add47640 1 --2- 172.21.15.16:0/1828300041 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ff29006dcd0 0x7ff290070190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0009b40 tx=0x7ff2a0005e50 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:05.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:05.832+0000 7ff2a6ffd640 1 -- 172.21.15.16:0/1828300041 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff29c079b70 con 0x7ff2a8101ac0 2024-04-02T04:54:06.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.013+0000 7ff2affab640 1 -- 172.21.15.16:0/1828300041 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7ff2a819ac40 con 0x7ff2a8101ac0 2024-04-02T04:54:06.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.014+0000 7ff2a6ffd640 1 -- 172.21.15.16:0/1828300041 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7ff2a819ac40 con 0x7ff2a8101ac0 2024-04-02T04:54:06.018 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-02T04:54:06.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.016+0000 7ff2affab640 1 -- 172.21.15.16:0/1828300041 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ff29006dcd0 msgr2=0x7ff290070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:06.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.016+0000 7ff2affab640 1 --2- 172.21.15.16:0/1828300041 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ff29006dcd0 0x7ff290070190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0009b40 tx=0x7ff2a0005e50 comp rx=0 tx=0).stop 2024-04-02T04:54:06.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.016+0000 7ff2affab640 1 -- 172.21.15.16:0/1828300041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a8101ac0 msgr2=0x7ff2a8196490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:06.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.016+0000 7ff2affab640 1 --2- 172.21.15.16:0/1828300041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a8101ac0 0x7ff2a8196490 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff29c02c2b0 tx=0x7ff29c02c850 comp rx=0 tx=0).stop 2024-04-02T04:54:06.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.016+0000 7ff2affab640 1 -- 172.21.15.16:0/1828300041 shutdown_connections 2024-04-02T04:54:06.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.016+0000 7ff2affab640 1 --2- 172.21.15.16:0/1828300041 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ff29006dcd0 0x7ff290070190 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:06.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.016+0000 7ff2affab640 1 --2- 172.21.15.16:0/1828300041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a8101ac0 0x7ff2a8196490 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:06.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.016+0000 7ff2affab640 1 --2- 172.21.15.16:0/1828300041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2a81011a0 0x7ff2a8195f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:06.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.016+0000 7ff2affab640 1 -- 172.21.15.16:0/1828300041 >> 172.21.15.16:0/1828300041 conn(0x7ff2a80fa900 msgr2=0x7ff2a8102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:06.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.016+0000 7ff2affab640 1 -- 172.21.15.16:0/1828300041 shutdown_connections 2024-04-02T04:54:06.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.016+0000 7ff2affab640 1 -- 172.21.15.16:0/1828300041 wait complete. 2024-04-02T04:54:06.031 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-02T04:54:06.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.132+0000 7f10f9415640 1 -- 172.21.15.16:0/1133678335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10f4067e60 msgr2=0x7f10f4104ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:06.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.132+0000 7f10f9415640 1 --2- 172.21.15.16:0/1133678335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10f4067e60 0x7f10f4104ba0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f10dc009880 tx=0x7f10dc02bce0 comp rx=0 tx=0).stop 2024-04-02T04:54:06.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.132+0000 7f10f9415640 1 -- 172.21.15.16:0/1133678335 shutdown_connections 2024-04-02T04:54:06.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.132+0000 7f10f9415640 1 --2- 172.21.15.16:0/1133678335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10f4067e60 0x7f10f4104ba0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:06.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.132+0000 7f10f9415640 1 --2- 172.21.15.16:0/1133678335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10f40674b0 0x7f10f4067890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:06.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.132+0000 7f10f9415640 1 -- 172.21.15.16:0/1133678335 >> 172.21.15.16:0/1133678335 conn(0x7f10f4074960 msgr2=0x7f10f4074d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:06.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.132+0000 7f10f9415640 1 -- 172.21.15.16:0/1133678335 shutdown_connections 2024-04-02T04:54:06.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.132+0000 7f10f9415640 1 -- 172.21.15.16:0/1133678335 wait complete. 2024-04-02T04:54:06.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.133+0000 7f10f9415640 1 Processor -- start 2024-04-02T04:54:06.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.133+0000 7f10f9415640 1 -- start start 2024-04-02T04:54:06.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.133+0000 7f10f9415640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10f40674b0 0x7f10f4196010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:06.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.133+0000 7f10f9415640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10f4196550 0x7f10f419a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:06.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.133+0000 7f10f9415640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f4196b50 con 0x7f10f4196550 2024-04-02T04:54:06.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.133+0000 7f10f9415640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f4196cc0 con 0x7f10f40674b0 2024-04-02T04:54:06.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.133+0000 7f10f27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10f4196550 0x7f10f419a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:06.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.133+0000 7f10f27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10f4196550 0x7f10f419a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58342/0 (socket says 172.21.15.16:58342) 2024-04-02T04:54:06.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.133+0000 7f10f27fc640 1 -- 172.21.15.16:0/534989691 learned_addr learned my addr 172.21.15.16:0/534989691 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:54:06.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.134+0000 7f10f2ffd640 1 --2- 172.21.15.16:0/534989691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10f40674b0 0x7f10f4196010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:06.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.134+0000 7f10f27fc640 1 -- 172.21.15.16:0/534989691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10f40674b0 msgr2=0x7f10f4196010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:06.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.134+0000 7f10f27fc640 1 --2- 172.21.15.16:0/534989691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10f40674b0 0x7f10f4196010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:06.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.134+0000 7f10f27fc640 1 -- 172.21.15.16:0/534989691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10dc009560 con 0x7f10f4196550 2024-04-02T04:54:06.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.134+0000 7f10f27fc640 1 --2- 172.21.15.16:0/534989691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10f4196550 0x7f10f419a8f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f10dc036040 tx=0x7f10dc02c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:06.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.134+0000 7f10ebfff640 1 -- 172.21.15.16:0/534989691 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10dc0044a0 con 0x7f10f4196550 2024-04-02T04:54:06.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.134+0000 7f10f9415640 1 -- 172.21.15.16:0/534989691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f419ae90 con 0x7f10f4196550 2024-04-02T04:54:06.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.134+0000 7f10ebfff640 1 -- 172.21.15.16:0/534989691 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f10dc003ab0 con 0x7f10f4196550 2024-04-02T04:54:06.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.134+0000 7f10ebfff640 1 -- 172.21.15.16:0/534989691 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10dc005560 con 0x7f10f4196550 2024-04-02T04:54:06.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.134+0000 7f10f9415640 1 -- 172.21.15.16:0/534989691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f419b3f0 con 0x7f10f4196550 2024-04-02T04:54:06.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.135+0000 7f10f9415640 1 -- 172.21.15.16:0/534989691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f4101680 con 0x7f10f4196550 2024-04-02T04:54:06.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.136+0000 7f10ebfff640 1 -- 172.21.15.16:0/534989691 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f10dc004dd0 con 0x7f10f4196550 2024-04-02T04:54:06.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.136+0000 7f10ebfff640 1 --2- 172.21.15.16:0/534989691 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f10e006dcd0 0x7f10e0070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:06.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.136+0000 7f10ebfff640 1 -- 172.21.15.16:0/534989691 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f10dc0afe40 con 0x7f10f4196550 2024-04-02T04:54:06.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.136+0000 7f10f2ffd640 1 --2- 172.21.15.16:0/534989691 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f10e006dcd0 0x7f10e0070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:06.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.137+0000 7f10f2ffd640 1 --2- 172.21.15.16:0/534989691 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f10e006dcd0 0x7f10e0070190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f10ec005fd0 tx=0x7f10ec009470 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:06.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.141+0000 7f10ebfff640 1 -- 172.21.15.16:0/534989691 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f10dc079330 con 0x7f10f4196550 2024-04-02T04:54:06.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.336+0000 7f10f9415640 1 -- 172.21.15.16:0/534989691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f10f4197310 con 0x7f10f4196550 2024-04-02T04:54:06.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.336+0000 7f10ebfff640 1 -- 172.21.15.16:0/534989691 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7f10dc041020 con 0x7f10f4196550 2024-04-02T04:54:06.341 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-02T04:54:06.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.338+0000 7f10f9415640 1 -- 172.21.15.16:0/534989691 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f10e006dcd0 msgr2=0x7f10e0070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:06.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.338+0000 7f10f9415640 1 --2- 172.21.15.16:0/534989691 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f10e006dcd0 0x7f10e0070190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f10ec005fd0 tx=0x7f10ec009470 comp rx=0 tx=0).stop 2024-04-02T04:54:06.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.338+0000 7f10f9415640 1 -- 172.21.15.16:0/534989691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10f4196550 msgr2=0x7f10f419a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:06.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.338+0000 7f10f9415640 1 --2- 172.21.15.16:0/534989691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10f4196550 0x7f10f419a8f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f10dc036040 tx=0x7f10dc02c7c0 comp rx=0 tx=0).stop 2024-04-02T04:54:06.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.338+0000 7f10f9415640 1 -- 172.21.15.16:0/534989691 shutdown_connections 2024-04-02T04:54:06.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.338+0000 7f10f9415640 1 --2- 172.21.15.16:0/534989691 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f10e006dcd0 0x7f10e0070190 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:06.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.338+0000 7f10f9415640 1 --2- 172.21.15.16:0/534989691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10f4196550 0x7f10f419a8f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:06.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.338+0000 7f10f9415640 1 --2- 172.21.15.16:0/534989691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10f40674b0 0x7f10f4196010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:06.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.338+0000 7f10f9415640 1 -- 172.21.15.16:0/534989691 >> 172.21.15.16:0/534989691 conn(0x7f10f4074960 msgr2=0x7f10f4103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:06.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.338+0000 7f10f9415640 1 -- 172.21.15.16:0/534989691 shutdown_connections 2024-04-02T04:54:06.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.338+0000 7f10f9415640 1 -- 172.21.15.16:0/534989691 wait complete. 2024-04-02T04:54:06.354 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-02T04:54:06.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.454+0000 7f7f55070640 1 -- 172.21.15.16:0/3040291718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f500ffa50 msgr2=0x7f7f50103a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:06.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.454+0000 7f7f55070640 1 --2- 172.21.15.16:0/3040291718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f500ffa50 0x7f7f50103a90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7f40009880 tx=0x7f7f4002bce0 comp rx=0 tx=0).stop 2024-04-02T04:54:06.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.454+0000 7f7f55070640 1 -- 172.21.15.16:0/3040291718 shutdown_connections 2024-04-02T04:54:06.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.454+0000 7f7f55070640 1 --2- 172.21.15.16:0/3040291718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f500ffa50 0x7f7f50103a90 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:06.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.454+0000 7f7f55070640 1 --2- 172.21.15.16:0/3040291718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f500ff0a0 0x7f7f500ff480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:06.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.454+0000 7f7f55070640 1 -- 172.21.15.16:0/3040291718 >> 172.21.15.16:0/3040291718 conn(0x7f7f50075b10 msgr2=0x7f7f500fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:06.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.454+0000 7f7f55070640 1 -- 172.21.15.16:0/3040291718 shutdown_connections 2024-04-02T04:54:06.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.454+0000 7f7f55070640 1 -- 172.21.15.16:0/3040291718 wait complete. 2024-04-02T04:54:06.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.455+0000 7f7f55070640 1 Processor -- start 2024-04-02T04:54:06.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.455+0000 7f7f55070640 1 -- start start 2024-04-02T04:54:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.456+0000 7f7f55070640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f500ff0a0 0x7f7f501960a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.456+0000 7f7f55070640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f501965e0 0x7f7f5019a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.456+0000 7f7f55070640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f50196be0 con 0x7f7f501965e0 2024-04-02T04:54:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.456+0000 7f7f55070640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f50196d50 con 0x7f7f500ff0a0 2024-04-02T04:54:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.456+0000 7f7f4e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f501965e0 0x7f7f5019a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.456+0000 7f7f4e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f501965e0 0x7f7f5019a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58354/0 (socket says 172.21.15.16:58354) 2024-04-02T04:54:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.456+0000 7f7f4e59c640 1 -- 172.21.15.16:0/3414988377 learned_addr learned my addr 172.21.15.16:0/3414988377 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:54:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.456+0000 7f7f4ed9d640 1 --2- 172.21.15.16:0/3414988377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f500ff0a0 0x7f7f501960a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.456+0000 7f7f4e59c640 1 -- 172.21.15.16:0/3414988377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f500ff0a0 msgr2=0x7f7f501960a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.456+0000 7f7f4e59c640 1 --2- 172.21.15.16:0/3414988377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f500ff0a0 0x7f7f501960a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.456+0000 7f7f4e59c640 1 -- 172.21.15.16:0/3414988377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f40009560 con 0x7f7f501965e0 2024-04-02T04:54:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.456+0000 7f7f4e59c640 1 --2- 172.21.15.16:0/3414988377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f501965e0 0x7f7f5019a980 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7f40036040 tx=0x7f7f40004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:06.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.456+0000 7f7f37fff640 1 -- 172.21.15.16:0/3414988377 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f400035f0 con 0x7f7f501965e0 2024-04-02T04:54:06.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.456+0000 7f7f55070640 1 -- 172.21.15.16:0/3414988377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f5019af20 con 0x7f7f501965e0 2024-04-02T04:54:06.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.456+0000 7f7f37fff640 1 -- 172.21.15.16:0/3414988377 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7f40003c10 con 0x7f7f501965e0 2024-04-02T04:54:06.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.456+0000 7f7f37fff640 1 -- 172.21.15.16:0/3414988377 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f40034440 con 0x7f7f501965e0 2024-04-02T04:54:06.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.456+0000 7f7f55070640 1 -- 172.21.15.16:0/3414988377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f5019b480 con 0x7f7f501965e0 2024-04-02T04:54:06.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.458+0000 7f7f55070640 1 -- 172.21.15.16:0/3414988377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f50100570 con 0x7f7f501965e0 2024-04-02T04:54:06.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.458+0000 7f7f37fff640 1 -- 172.21.15.16:0/3414988377 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f7f40004f60 con 0x7f7f501965e0 2024-04-02T04:54:06.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.458+0000 7f7f37fff640 1 --2- 172.21.15.16:0/3414988377 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f7f3806dcd0 0x7f7f38070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:06.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.459+0000 7f7f4ed9d640 1 --2- 172.21.15.16:0/3414988377 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f7f3806dcd0 0x7f7f38070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:06.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.459+0000 7f7f37fff640 1 -- 172.21.15.16:0/3414988377 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f7f400aff50 con 0x7f7f501965e0 2024-04-02T04:54:06.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.459+0000 7f7f4ed9d640 1 --2- 172.21.15.16:0/3414988377 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f7f3806dcd0 0x7f7f38070190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7f44009990 tx=0x7f7f44009110 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:06.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.463+0000 7f7f37fff640 1 -- 172.21.15.16:0/3414988377 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7f40079500 con 0x7f7f501965e0 2024-04-02T04:54:06.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:06 smithi027 ceph-mon[83036]: pgmap v332: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:06.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:06 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/3324673621' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T04:54:06.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:06 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/3267900482' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T04:54:06.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:06 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/1828300041' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-02T04:54:06.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:06 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/534989691' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-02T04:54:06.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.659+0000 7f7f55070640 1 -- 172.21.15.16:0/3414988377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7f5019b6a0 con 0x7f7f501965e0 2024-04-02T04:54:06.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.660+0000 7f7f37fff640 1 -- 172.21.15.16:0/3414988377 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 43 entries v84) v1 ==== 81+0+2535 (secure 0 0 0) 0x7f7f4007cbc0 con 0x7f7f501965e0 2024-04-02T04:54:06.664 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.27:6827/4177192740 2024-04-03T04:50:07.886976+0000 2024-04-02T04:54:06.664 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.27:6824/1461455976 2024-04-03T04:49:41.368223+0000 2024-04-02T04:54:06.664 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2790817266 2024-04-03T04:44:17.011409+0000 2024-04-02T04:54:06.664 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1537212641 2024-04-03T04:44:17.011409+0000 2024-04-02T04:54:06.665 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3646287833 2024-04-03T04:44:17.011409+0000 2024-04-02T04:54:06.665 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/873531241 2024-04-03T04:33:28.907145+0000 2024-04-02T04:54:06.665 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2255383579 2024-04-03T04:32:56.644452+0000 2024-04-02T04:54:06.665 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/2776599951 2024-04-03T04:44:17.011409+0000 2024-04-02T04:54:06.665 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3964929492 2024-04-03T04:44:17.011409+0000 2024-04-02T04:54:06.665 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.27:0/597316902 2024-04-03T04:43:10.596160+0000 2024-04-02T04:54:06.665 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/4067493740 2024-04-03T04:33:28.907145+0000 2024-04-02T04:54:06.665 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/3912038174 2024-04-03T04:34:42.713541+0000 2024-04-02T04:54:06.665 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/3912038174 2024-04-03T04:34:42.713541+0000 2024-04-02T04:54:06.665 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1664387083 2024-04-03T04:42:07.942513+0000 2024-04-02T04:54:06.665 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.27:6825/1461455976 2024-04-03T04:49:41.368223+0000 2024-04-02T04:54:06.665 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.27:0/1314131858 2024-04-03T04:43:10.596160+0000 2024-04-02T04:54:06.665 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.27:6826/4177192740 2024-04-03T04:50:07.886976+0000 2024-04-02T04:54:06.665 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/873531241 2024-04-03T04:33:28.907145+0000 2024-04-02T04:54:06.665 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/401549934 2024-04-03T04:32:56.644452+0000 2024-04-02T04:54:06.665 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/3643600900 2024-04-03T04:42:07.942513+0000 2024-04-02T04:54:06.666 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/166410319 2024-04-03T04:33:28.907145+0000 2024-04-02T04:54:06.666 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1884756010 2024-04-03T04:32:56.644452+0000 2024-04-02T04:54:06.666 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/824107799 2024-04-03T04:32:56.644452+0000 2024-04-02T04:54:06.666 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/881481317 2024-04-03T04:44:17.011409+0000 2024-04-02T04:54:06.666 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/3643600900 2024-04-03T04:42:07.942513+0000 2024-04-02T04:54:06.666 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/4209328561 2024-04-03T04:34:42.713541+0000 2024-04-02T04:54:06.666 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1920858754 2024-04-03T04:42:07.942513+0000 2024-04-02T04:54:06.666 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3733852550 2024-04-03T04:34:42.713541+0000 2024-04-02T04:54:06.666 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/824107799 2024-04-03T04:32:56.644452+0000 2024-04-02T04:54:06.666 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.27:0/3186228192 2024-04-03T04:43:10.596160+0000 2024-04-02T04:54:06.666 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.27:0/1987939490 2024-04-03T04:43:10.596160+0000 2024-04-02T04:54:06.666 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3405957790 2024-04-03T04:34:42.713541+0000 2024-04-02T04:54:06.666 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1383394471 2024-04-03T04:42:07.942513+0000 2024-04-02T04:54:06.666 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.27:6828/1554339939 2024-04-03T04:43:10.596160+0000 2024-04-02T04:54:06.667 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1228221358 2024-04-03T04:33:28.907145+0000 2024-04-02T04:54:06.667 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2373084595 2024-04-03T04:42:07.942513+0000 2024-04-02T04:54:06.667 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/249069849 2024-04-03T04:44:17.011409+0000 2024-04-02T04:54:06.667 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.27:0/3624631480 2024-04-03T04:43:10.596160+0000 2024-04-02T04:54:06.667 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.27:0/2708474610 2024-04-03T04:43:10.596160+0000 2024-04-02T04:54:06.667 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6826/71509234 2024-04-03T04:39:46.861330+0000 2024-04-02T04:54:06.667 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6827/71509234 2024-04-03T04:39:46.861330+0000 2024-04-02T04:54:06.667 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.27:6829/1554339939 2024-04-03T04:43:10.596160+0000 2024-04-02T04:54:06.667 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/2776599951 2024-04-03T04:44:17.011409+0000 2024-04-02T04:54:06.668 INFO:teuthology.orchestra.run.smithi016.stderr:listed 43 entries 2024-04-02T04:54:06.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.662+0000 7f7f55070640 1 -- 172.21.15.16:0/3414988377 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f7f3806dcd0 msgr2=0x7f7f38070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:06.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.662+0000 7f7f55070640 1 --2- 172.21.15.16:0/3414988377 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f7f3806dcd0 0x7f7f38070190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7f44009990 tx=0x7f7f44009110 comp rx=0 tx=0).stop 2024-04-02T04:54:06.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.662+0000 7f7f55070640 1 -- 172.21.15.16:0/3414988377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f501965e0 msgr2=0x7f7f5019a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:06.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.662+0000 7f7f55070640 1 --2- 172.21.15.16:0/3414988377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f501965e0 0x7f7f5019a980 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7f40036040 tx=0x7f7f40004040 comp rx=0 tx=0).stop 2024-04-02T04:54:06.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.662+0000 7f7f55070640 1 -- 172.21.15.16:0/3414988377 shutdown_connections 2024-04-02T04:54:06.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.662+0000 7f7f55070640 1 --2- 172.21.15.16:0/3414988377 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f7f3806dcd0 0x7f7f38070190 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:06.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.662+0000 7f7f55070640 1 --2- 172.21.15.16:0/3414988377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f501965e0 0x7f7f5019a980 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:06.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.662+0000 7f7f55070640 1 --2- 172.21.15.16:0/3414988377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f500ff0a0 0x7f7f501960a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:06.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.662+0000 7f7f55070640 1 -- 172.21.15.16:0/3414988377 >> 172.21.15.16:0/3414988377 conn(0x7f7f50075b10 msgr2=0x7f7f50067520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:06.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.662+0000 7f7f55070640 1 -- 172.21.15.16:0/3414988377 shutdown_connections 2024-04-02T04:54:06.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.662+0000 7f7f55070640 1 -- 172.21.15.16:0/3414988377 wait complete. 2024-04-02T04:54:06.677 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:06 smithi016 ceph-mon[93230]: pgmap v332: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:06.677 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:06 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/3324673621' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T04:54:06.677 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:06 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/3267900482' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T04:54:06.677 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:06 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/1828300041' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-02T04:54:06.677 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:06 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/534989691' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-02T04:54:06.678 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-02T04:54:06.678 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T04:54:06.694 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T04:54:06.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.835+0000 7fa959337640 1 -- 172.21.15.16:0/799563236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9540ff0a0 msgr2=0x7fa9540ff480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:06.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.835+0000 7fa959337640 1 --2- 172.21.15.16:0/799563236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9540ff0a0 0x7fa9540ff480 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa948009880 tx=0x7fa94802bce0 comp rx=0 tx=0).stop 2024-04-02T04:54:06.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.835+0000 7fa959337640 1 -- 172.21.15.16:0/799563236 shutdown_connections 2024-04-02T04:54:06.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.835+0000 7fa959337640 1 --2- 172.21.15.16:0/799563236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9540ffa50 0x7fa954103a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:06.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.835+0000 7fa959337640 1 --2- 172.21.15.16:0/799563236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9540ff0a0 0x7fa9540ff480 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:06.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.835+0000 7fa959337640 1 -- 172.21.15.16:0/799563236 >> 172.21.15.16:0/799563236 conn(0x7fa954075b10 msgr2=0x7fa9540fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:06.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.835+0000 7fa959337640 1 -- 172.21.15.16:0/799563236 shutdown_connections 2024-04-02T04:54:06.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.835+0000 7fa959337640 1 -- 172.21.15.16:0/799563236 wait complete. 2024-04-02T04:54:06.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.835+0000 7fa959337640 1 Processor -- start 2024-04-02T04:54:06.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.835+0000 7fa959337640 1 -- start start 2024-04-02T04:54:06.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.836+0000 7fa959337640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9540ff0a0 0x7fa954194e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:06.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.836+0000 7fa959337640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9541953d0 0x7fa954199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:06.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.836+0000 7fa959337640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9541959d0 con 0x7fa9541953d0 2024-04-02T04:54:06.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.836+0000 7fa959337640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa954195b40 con 0x7fa9540ff0a0 2024-04-02T04:54:06.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.836+0000 7fa9527fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9541953d0 0x7fa954199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:06.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.836+0000 7fa9527fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9541953d0 0x7fa954199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58368/0 (socket says 172.21.15.16:58368) 2024-04-02T04:54:06.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.836+0000 7fa9527fc640 1 -- 172.21.15.16:0/4240440293 learned_addr learned my addr 172.21.15.16:0/4240440293 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:54:06.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.836+0000 7fa952ffd640 1 --2- 172.21.15.16:0/4240440293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9540ff0a0 0x7fa954194e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:06.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.836+0000 7fa9527fc640 1 -- 172.21.15.16:0/4240440293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9540ff0a0 msgr2=0x7fa954194e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:06.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.836+0000 7fa9527fc640 1 --2- 172.21.15.16:0/4240440293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9540ff0a0 0x7fa954194e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:06.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.836+0000 7fa9527fc640 1 -- 172.21.15.16:0/4240440293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa948009560 con 0x7fa9541953d0 2024-04-02T04:54:06.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.836+0000 7fa952ffd640 1 --2- 172.21.15.16:0/4240440293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9540ff0a0 0x7fa954194e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T04:54:06.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.836+0000 7fa9527fc640 1 --2- 172.21.15.16:0/4240440293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9541953d0 0x7fa954199770 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa94400e9a0 tx=0x7fa94400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:06.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.836+0000 7fa93bfff640 1 -- 172.21.15.16:0/4240440293 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa94400cc90 con 0x7fa9541953d0 2024-04-02T04:54:06.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.836+0000 7fa959337640 1 -- 172.21.15.16:0/4240440293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa954199d70 con 0x7fa9541953d0 2024-04-02T04:54:06.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.836+0000 7fa93bfff640 1 -- 172.21.15.16:0/4240440293 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa94400cdf0 con 0x7fa9541953d0 2024-04-02T04:54:06.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.836+0000 7fa93bfff640 1 -- 172.21.15.16:0/4240440293 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa944010430 con 0x7fa9541953d0 2024-04-02T04:54:06.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.837+0000 7fa959337640 1 -- 172.21.15.16:0/4240440293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa95419a2d0 con 0x7fa9541953d0 2024-04-02T04:54:06.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.837+0000 7fa959337640 1 -- 172.21.15.16:0/4240440293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa95419a5b0 con 0x7fa9541953d0 2024-04-02T04:54:06.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.838+0000 7fa93bfff640 1 -- 172.21.15.16:0/4240440293 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fa944005020 con 0x7fa9541953d0 2024-04-02T04:54:06.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.839+0000 7fa93bfff640 1 --2- 172.21.15.16:0/4240440293 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fa93c06dcd0 0x7fa93c070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:06.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.839+0000 7fa93bfff640 1 -- 172.21.15.16:0/4240440293 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fa944014070 con 0x7fa9541953d0 2024-04-02T04:54:06.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.839+0000 7fa952ffd640 1 --2- 172.21.15.16:0/4240440293 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fa93c06dcd0 0x7fa93c070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:06.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.839+0000 7fa952ffd640 1 --2- 172.21.15.16:0/4240440293 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fa93c06dcd0 0x7fa93c070190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa948036040 tx=0x7fa948000bc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:06.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:06.843+0000 7fa93bfff640 1 -- 172.21.15.16:0/4240440293 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa944057670 con 0x7fa9541953d0 2024-04-02T04:54:07.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.076+0000 7fa959337640 1 -- 172.21.15.16:0/4240440293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa954196190 con 0x7fa9541953d0 2024-04-02T04:54:07.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.077+0000 7fa93bfff640 1 -- 172.21.15.16:0/4240440293 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16859 (secure 0 0 0) 0x7fa94405ad30 con 0x7fa9541953d0 2024-04-02T04:54:07.082 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-02T04:54:07.082 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":84,"fsid":"dc5403b4-f0a9-11ee-b647-cb9ed24678a4","created":"2024-04-02T04:32:23.014546+0000","modified":"2024-04-02T04:50:07.887023+0000","last_up_change":"2024-04-02T04:48:57.379242+0000","last_in_change":"2024-04-02T04:37:56.807214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T04:37:01.047338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-02T04:39:29.039519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-02T04:39:29.992892+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2b7be29b-0e2a-456b-b33e-c72abaad9136","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":78,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1136760904},{"type":"v1","addr":"172.21.15.16:6803","nonce":1136760904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1136760904},{"type":"v1","addr":"172.21.15.16:6805","nonce":1136760904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1136760904},{"type":"v1","addr":"172.21.15.16:6809","nonce":1136760904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1136760904},{"type":"v1","addr":"172.21.15.16:6807","nonce":1136760904}]},"public_addr":"172.21.15.16:6803/1136760904","cluster_addr":"172.21.15.16:6805/1136760904","heartbeat_back_addr":"172.21.15.16:6809/1136760904","heartbeat_front_addr":"172.21.15.16:6807/1136760904","state":["exists","up"]},{"osd":1,"uuid":"82578291-3c9c-4f3f-8de9-7a8710a4923e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":78,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2744685333},{"type":"v1","addr":"172.21.15.16:6811","nonce":2744685333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2744685333},{"type":"v1","addr":"172.21.15.16:6813","nonce":2744685333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2744685333},{"type":"v1","addr":"172.21.15.16:6817","nonce":2744685333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2744685333},{"type":"v1","addr":"172.21.15.16:6815","nonce":2744685333}]},"public_addr":"172.21.15.16:6811/2744685333","cluster_addr":"172.21.15.16:6813/2744685333","heartbeat_back_addr":"172.21.15.16:6817/2744685333","heartbeat_front_addr":"172.21.15.16:6815/2744685333","state":["exists","up"]},{"osd":2,"uuid":"046262b5-386e-4270-863c-e97a61a038b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":59,"up_from":63,"up_thru":78,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":1004843818},{"type":"v1","addr":"172.21.15.16:6819","nonce":1004843818}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":1004843818},{"type":"v1","addr":"172.21.15.16:6821","nonce":1004843818}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1004843818},{"type":"v1","addr":"172.21.15.16:6825","nonce":1004843818}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":1004843818},{"type":"v1","addr":"172.21.15.16:6823","nonce":1004843818}]},"public_addr":"172.21.15.16:6819/1004843818","cluster_addr":"172.21.15.16:6821/1004843818","heartbeat_back_addr":"172.21.15.16:6825/1004843818","heartbeat_front_addr":"172.21.15.16:6823/1004843818","state":["exists","up"]},{"osd":3,"uuid":"637f2a77-59a2-44fb-b47e-fcbd70f4423c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":64,"up_from":68,"up_thru":78,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":4092279016},{"type":"v1","addr":"172.21.15.27:6801","nonce":4092279016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4092279016},{"type":"v1","addr":"172.21.15.27:6803","nonce":4092279016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4092279016},{"type":"v1","addr":"172.21.15.27:6807","nonce":4092279016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4092279016},{"type":"v1","addr":"172.21.15.27:6805","nonce":4092279016}]},"public_addr":"172.21.15.27:6801/4092279016","cluster_addr":"172.21.15.27:6803/4092279016","heartbeat_back_addr":"172.21.15.27:6807/4092279016","heartbeat_front_addr":"172.21.15.27:6805/4092279016","state":["exists","up"]},{"osd":4,"uuid":"334b7b3d-505f-45b6-85b0-59ec8eeb8e91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":69,"up_from":73,"up_thru":78,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":2931886568},{"type":"v1","addr":"172.21.15.27:6809","nonce":2931886568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2931886568},{"type":"v1","addr":"172.21.15.27:6811","nonce":2931886568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2931886568},{"type":"v1","addr":"172.21.15.27:6815","nonce":2931886568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2931886568},{"type":"v1","addr":"172.21.15.27:6813","nonce":2931886568}]},"public_addr":"172.21.15.27:6809/2931886568","cluster_addr":"172.21.15.27:6811/2931886568","heartbeat_back_addr":"172.21.15.27:6815/2931886568","heartbeat_front_addr":"172.21.15.27:6813/2931886568","state":["exists","up"]},{"osd":5,"uuid":"f83802ee-4f47-47de-91dd-a667f6cd22e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":74,"up_from":78,"up_thru":78,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":401102788},{"type":"v1","addr":"172.21.15.27:6817","nonce":401102788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":401102788},{"type":"v1","addr":"172.21.15.27:6819","nonce":401102788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":401102788},{"type":"v1","addr":"172.21.15.27:6823","nonce":401102788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":401102788},{"type":"v1","addr":"172.21.15.27:6821","nonce":401102788}]},"public_addr":"172.21.15.27:6817/401102788","cluster_addr":"172.21.15.27:6819/401102788","heartbeat_back_addr":"172.21.15.27:6823/401102788","heartbeat_front_addr":"172.21.15.27:6821/401102788","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:36:02.027852+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:36:29.393593+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:36:56.847224+0000","dead_epoch":59},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:37:21.195876+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:37:47.992167+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T04:38:15.012477+0000","dead_epoch":74}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6827/4177192740":"2024-04-03T04:50:07.886976+0000","172.21.15.27:6824/1461455976":"2024-04-03T04:49:41.368223+0000","172.21.15.16:0/2790817266":"2024-04-03T04:44:17.011409+0000","172.21.15.16:0/1537212641":"2024-04-03T04:44:17.011409+0000","172.21.15.16:0/3646287833":"2024-04-03T04:44:17.011409+0000","172.21.15.16:6800/873531241":"2024-04-03T04:33:28.907145+0000","172.21.15.16:0/2255383579":"2024-04-03T04:32:56.644452+0000","172.21.15.16:6800/2776599951":"2024-04-03T04:44:17.011409+0000","172.21.15.16:0/3964929492":"2024-04-03T04:44:17.011409+0000","172.21.15.27:0/597316902":"2024-04-03T04:43:10.596160+0000","172.21.15.16:0/4067493740":"2024-04-03T04:33:28.907145+0000","172.21.15.16:6801/3912038174":"2024-04-03T04:34:42.713541+0000","172.21.15.16:6800/3912038174":"2024-04-03T04:34:42.713541+0000","172.21.15.16:0/1664387083":"2024-04-03T04:42:07.942513+0000","172.21.15.27:6825/1461455976":"2024-04-03T04:49:41.368223+0000","172.21.15.27:0/1314131858":"2024-04-03T04:43:10.596160+0000","172.21.15.27:6826/4177192740":"2024-04-03T04:50:07.886976+0000","172.21.15.16:6801/873531241":"2024-04-03T04:33:28.907145+0000","172.21.15.16:0/401549934":"2024-04-03T04:32:56.644452+0000","172.21.15.16:6801/3643600900":"2024-04-03T04:42:07.942513+0000","172.21.15.16:0/166410319":"2024-04-03T04:33:28.907145+0000","172.21.15.16:0/1884756010":"2024-04-03T04:32:56.644452+0000","172.21.15.16:6801/824107799":"2024-04-03T04:32:56.644452+0000","172.21.15.16:0/881481317":"2024-04-03T04:44:17.011409+0000","172.21.15.16:6800/3643600900":"2024-04-03T04:42:07.942513+0000","172.21.15.16:0/4209328561":"2024-04-03T04:34:42.713541+0000","172.21.15.16:0/1920858754":"2024-04-03T04:42:07.942513+0000","172.21.15.16:0/3733852550":"2024-04-03T04:34:42.713541+0000","172.21.15.16:6800/824107799":"2024-04-03T04:32:56.644452+0000","172.21.15.27:0/3186228192":"2024-04-03T04:43:10.596160+0000","172.21.15.27:0/1987939490":"2024-04-03T04:43:10.596160+0000","172.21.15.16:0/3405957790":"2024-04-03T04:34:42.713541+0000","172.21.15.16:0/1383394471":"2024-04-03T04:42:07.942513+0000","172.21.15.27:6828/1554339939":"2024-04-03T04:43:10.596160+0000","172.21.15.16:0/1228221358":"2024-04-03T04:33:28.907145+0000","172.21.15.16:0/2373084595":"2024-04-03T04:42:07.942513+0000","172.21.15.16:0/249069849":"2024-04-03T04:44:17.011409+0000","172.21.15.27:0/3624631480":"2024-04-03T04:43:10.596160+0000","172.21.15.27:0/2708474610":"2024-04-03T04:43:10.596160+0000","172.21.15.16:6826/71509234":"2024-04-03T04:39:46.861330+0000","172.21.15.16:6827/71509234":"2024-04-03T04:39:46.861330+0000","172.21.15.27:6829/1554339939":"2024-04-03T04:43:10.596160+0000","172.21.15.16:6801/2776599951":"2024-04-03T04:44:17.011409+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T04:54:07.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.079+0000 7fa959337640 1 -- 172.21.15.16:0/4240440293 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fa93c06dcd0 msgr2=0x7fa93c070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:07.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.079+0000 7fa959337640 1 --2- 172.21.15.16:0/4240440293 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fa93c06dcd0 0x7fa93c070190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa948036040 tx=0x7fa948000bc0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.079+0000 7fa959337640 1 -- 172.21.15.16:0/4240440293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9541953d0 msgr2=0x7fa954199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:07.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.079+0000 7fa959337640 1 --2- 172.21.15.16:0/4240440293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9541953d0 0x7fa954199770 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa94400e9a0 tx=0x7fa94400ee60 comp rx=0 tx=0).stop 2024-04-02T04:54:07.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.079+0000 7fa959337640 1 -- 172.21.15.16:0/4240440293 shutdown_connections 2024-04-02T04:54:07.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.079+0000 7fa959337640 1 --2- 172.21.15.16:0/4240440293 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fa93c06dcd0 0x7fa93c070190 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.079+0000 7fa959337640 1 --2- 172.21.15.16:0/4240440293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9541953d0 0x7fa954199770 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.079+0000 7fa959337640 1 --2- 172.21.15.16:0/4240440293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9540ff0a0 0x7fa954194e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.079+0000 7fa959337640 1 -- 172.21.15.16:0/4240440293 >> 172.21.15.16:0/4240440293 conn(0x7fa954075b10 msgr2=0x7fa9540fbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:07.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.079+0000 7fa959337640 1 -- 172.21.15.16:0/4240440293 shutdown_connections 2024-04-02T04:54:07.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.079+0000 7fa959337640 1 -- 172.21.15.16:0/4240440293 wait complete. 2024-04-02T04:54:07.096 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T04:37:01.047338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-02T04:39:29.039519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-02T04:39:29.992892+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T04:54:07.096 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T04:54:07.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.199+0000 7ff5a811b640 1 -- 172.21.15.16:0/1870986730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5a0101100 msgr2=0x7ff5a01014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:07.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.199+0000 7ff5a811b640 1 --2- 172.21.15.16:0/1870986730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5a0101100 0x7ff5a01014e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff590009880 tx=0x7ff59002bd70 comp rx=0 tx=0).stop 2024-04-02T04:54:07.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.199+0000 7ff5a811b640 1 -- 172.21.15.16:0/1870986730 shutdown_connections 2024-04-02T04:54:07.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.199+0000 7ff5a811b640 1 --2- 172.21.15.16:0/1870986730 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5a0101a20 0x7ff5a0109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.199+0000 7ff5a811b640 1 --2- 172.21.15.16:0/1870986730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5a0101100 0x7ff5a01014e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.199+0000 7ff5a811b640 1 -- 172.21.15.16:0/1870986730 >> 172.21.15.16:0/1870986730 conn(0x7ff5a0075b10 msgr2=0x7ff5a00fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:07.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.199+0000 7ff5a811b640 1 -- 172.21.15.16:0/1870986730 shutdown_connections 2024-04-02T04:54:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.199+0000 7ff5a811b640 1 -- 172.21.15.16:0/1870986730 wait complete. 2024-04-02T04:54:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.199+0000 7ff5a811b640 1 Processor -- start 2024-04-02T04:54:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.199+0000 7ff5a811b640 1 -- start start 2024-04-02T04:54:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.200+0000 7ff5a811b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5a0101100 0x7ff5a00fed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.200+0000 7ff5a811b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5a0101a20 0x7ff5a00ff290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.200+0000 7ff5a811b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a00ff7d0 con 0x7ff5a0101a20 2024-04-02T04:54:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.200+0000 7ff5a811b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a00ff940 con 0x7ff5a0101100 2024-04-02T04:54:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.200+0000 7ff5a56b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5a0101a20 0x7ff5a00ff290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.200+0000 7ff5a5eb7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5a0101100 0x7ff5a00fed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.200+0000 7ff5a56b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5a0101a20 0x7ff5a00ff290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58384/0 (socket says 172.21.15.16:58384) 2024-04-02T04:54:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.200+0000 7ff5a56b6640 1 -- 172.21.15.16:0/3029347187 learned_addr learned my addr 172.21.15.16:0/3029347187 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:54:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.200+0000 7ff5a56b6640 1 -- 172.21.15.16:0/3029347187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5a0101100 msgr2=0x7ff5a00fed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:07.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.200+0000 7ff5a56b6640 1 --2- 172.21.15.16:0/3029347187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5a0101100 0x7ff5a00fed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.200+0000 7ff5a56b6640 1 -- 172.21.15.16:0/3029347187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff590009560 con 0x7ff5a0101a20 2024-04-02T04:54:07.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.200+0000 7ff5a5eb7640 1 --2- 172.21.15.16:0/3029347187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5a0101100 0x7ff5a00fed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T04:54:07.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.200+0000 7ff5a56b6640 1 --2- 172.21.15.16:0/3029347187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5a0101a20 0x7ff5a00ff290 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00d8d0 tx=0x7ff59c00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:07.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.200+0000 7ff596ffd640 1 -- 172.21.15.16:0/3029347187 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff59c009a30 con 0x7ff5a0101a20 2024-04-02T04:54:07.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.201+0000 7ff596ffd640 1 -- 172.21.15.16:0/3029347187 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff59c009b90 con 0x7ff5a0101a20 2024-04-02T04:54:07.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.201+0000 7ff5a811b640 1 -- 172.21.15.16:0/3029347187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5a01a2140 con 0x7ff5a0101a20 2024-04-02T04:54:07.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.201+0000 7ff596ffd640 1 -- 172.21.15.16:0/3029347187 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff59c0049d0 con 0x7ff5a0101a20 2024-04-02T04:54:07.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.201+0000 7ff5a811b640 1 -- 172.21.15.16:0/3029347187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a01a24b0 con 0x7ff5a0101a20 2024-04-02T04:54:07.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.202+0000 7ff5a811b640 1 -- 172.21.15.16:0/3029347187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5a0067b20 con 0x7ff5a0101a20 2024-04-02T04:54:07.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.202+0000 7ff596ffd640 1 -- 172.21.15.16:0/3029347187 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7ff59c009d00 con 0x7ff5a0101a20 2024-04-02T04:54:07.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.203+0000 7ff596ffd640 1 --2- 172.21.15.16:0/3029347187 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ff58c06dce0 0x7ff58c0701a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:07.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.203+0000 7ff596ffd640 1 -- 172.21.15.16:0/3029347187 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7ff59c08d6f0 con 0x7ff5a0101a20 2024-04-02T04:54:07.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.203+0000 7ff5a5eb7640 1 --2- 172.21.15.16:0/3029347187 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ff58c06dce0 0x7ff58c0701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:07.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.203+0000 7ff5a5eb7640 1 --2- 172.21.15.16:0/3029347187 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ff58c06dce0 0x7ff58c0701a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff590006010 tx=0x7ff59002ded0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:07.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.207+0000 7ff596ffd640 1 -- 172.21.15.16:0/3029347187 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff59c056be0 con 0x7ff5a0101a20 2024-04-02T04:54:07.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.392+0000 7ff5a811b640 1 -- 172.21.15.16:0/3029347187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff5a004e030 con 0x7ff5a0101a20 2024-04-02T04:54:07.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.392+0000 7ff596ffd640 1 -- 172.21.15.16:0/3029347187 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff5a004e030 con 0x7ff5a0101a20 2024-04-02T04:54:07.397 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-04-02T04:54:07.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.394+0000 7ff5a811b640 1 -- 172.21.15.16:0/3029347187 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ff58c06dce0 msgr2=0x7ff58c0701a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:07.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.394+0000 7ff5a811b640 1 --2- 172.21.15.16:0/3029347187 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ff58c06dce0 0x7ff58c0701a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff590006010 tx=0x7ff59002ded0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.394+0000 7ff5a811b640 1 -- 172.21.15.16:0/3029347187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5a0101a20 msgr2=0x7ff5a00ff290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:07.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.394+0000 7ff5a811b640 1 --2- 172.21.15.16:0/3029347187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5a0101a20 0x7ff5a00ff290 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00d8d0 tx=0x7ff59c00dd90 comp rx=0 tx=0).stop 2024-04-02T04:54:07.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.394+0000 7ff5a811b640 1 -- 172.21.15.16:0/3029347187 shutdown_connections 2024-04-02T04:54:07.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.394+0000 7ff5a811b640 1 --2- 172.21.15.16:0/3029347187 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7ff58c06dce0 0x7ff58c0701a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.394+0000 7ff5a811b640 1 --2- 172.21.15.16:0/3029347187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5a0101a20 0x7ff5a00ff290 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.394+0000 7ff5a811b640 1 --2- 172.21.15.16:0/3029347187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5a0101100 0x7ff5a00fed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.394+0000 7ff5a811b640 1 -- 172.21.15.16:0/3029347187 >> 172.21.15.16:0/3029347187 conn(0x7ff5a0075b10 msgr2=0x7ff5a01023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:07.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.394+0000 7ff5a811b640 1 -- 172.21.15.16:0/3029347187 shutdown_connections 2024-04-02T04:54:07.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.394+0000 7ff5a811b640 1 -- 172.21.15.16:0/3029347187 wait complete. 2024-04-02T04:54:07.410 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-02T04:54:07.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.533+0000 7f01f4500640 1 -- 172.21.15.16:0/1385194258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01ec0ffb70 msgr2=0x7f01ec103aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:07.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.533+0000 7f01f4500640 1 --2- 172.21.15.16:0/1385194258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01ec0ffb70 0x7f01ec103aa0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f01dc009880 tx=0x7f01dc02bce0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.533+0000 7f01f4500640 1 -- 172.21.15.16:0/1385194258 shutdown_connections 2024-04-02T04:54:07.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.533+0000 7f01f4500640 1 --2- 172.21.15.16:0/1385194258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01ec0ffb70 0x7f01ec103aa0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.533+0000 7f01f4500640 1 --2- 172.21.15.16:0/1385194258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01ec0ff1c0 0x7f01ec0ff5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.533+0000 7f01f4500640 1 -- 172.21.15.16:0/1385194258 >> 172.21.15.16:0/1385194258 conn(0x7f01ec074960 msgr2=0x7f01ec074d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:07.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.533+0000 7f01f4500640 1 -- 172.21.15.16:0/1385194258 shutdown_connections 2024-04-02T04:54:07.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.533+0000 7f01f4500640 1 -- 172.21.15.16:0/1385194258 wait complete. 2024-04-02T04:54:07.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.534+0000 7f01f4500640 1 Processor -- start 2024-04-02T04:54:07.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.534+0000 7f01f4500640 1 -- start start 2024-04-02T04:54:07.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.534+0000 7f01f4500640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01ec0ff1c0 0x7f01ec196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:07.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.534+0000 7f01f4500640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01ec1973d0 0x7f01ec197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:07.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.534+0000 7f01f4500640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec19ff00 con 0x7f01ec0ff1c0 2024-04-02T04:54:07.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.534+0000 7f01f4500640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec1a0040 con 0x7f01ec1973d0 2024-04-02T04:54:07.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.534+0000 7f01f229c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01ec0ff1c0 0x7f01ec196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:07.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.534+0000 7f01f229c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01ec0ff1c0 0x7f01ec196e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58406/0 (socket says 172.21.15.16:58406) 2024-04-02T04:54:07.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.534+0000 7f01f229c640 1 -- 172.21.15.16:0/1757397726 learned_addr learned my addr 172.21.15.16:0/1757397726 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:54:07.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.534+0000 7f01f1a9b640 1 --2- 172.21.15.16:0/1757397726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01ec1973d0 0x7f01ec197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:07.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.535+0000 7f01f229c640 1 -- 172.21.15.16:0/1757397726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01ec1973d0 msgr2=0x7f01ec197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:07.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.535+0000 7f01f229c640 1 --2- 172.21.15.16:0/1757397726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01ec1973d0 0x7f01ec197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.535+0000 7f01f229c640 1 -- 172.21.15.16:0/1757397726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01dc009560 con 0x7f01ec0ff1c0 2024-04-02T04:54:07.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.535+0000 7f01f229c640 1 --2- 172.21.15.16:0/1757397726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01ec0ff1c0 0x7f01ec196e90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f01e800e9a0 tx=0x7f01e800eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:07.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.535+0000 7f01e37fe640 1 -- 172.21.15.16:0/1757397726 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01e800cd50 con 0x7f01ec0ff1c0 2024-04-02T04:54:07.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.535+0000 7f01e37fe640 1 -- 172.21.15.16:0/1757397726 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f01e800ceb0 con 0x7f01ec0ff1c0 2024-04-02T04:54:07.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.535+0000 7f01f4500640 1 -- 172.21.15.16:0/1757397726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01ec1a01e0 con 0x7f01ec0ff1c0 2024-04-02T04:54:07.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.535+0000 7f01e37fe640 1 -- 172.21.15.16:0/1757397726 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01e8010660 con 0x7f01ec0ff1c0 2024-04-02T04:54:07.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.535+0000 7f01f4500640 1 -- 172.21.15.16:0/1757397726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01ec1a0710 con 0x7f01ec0ff1c0 2024-04-02T04:54:07.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.537+0000 7f01f4500640 1 -- 172.21.15.16:0/1757397726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01ec100580 con 0x7f01ec0ff1c0 2024-04-02T04:54:07.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.537+0000 7f01e37fe640 1 -- 172.21.15.16:0/1757397726 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f01e80048a0 con 0x7f01ec0ff1c0 2024-04-02T04:54:07.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.537+0000 7f01e37fe640 1 --2- 172.21.15.16:0/1757397726 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f01d806dcd0 0x7f01d8070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:07.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.538+0000 7f01e37fe640 1 -- 172.21.15.16:0/1757397726 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f01e8014070 con 0x7f01ec0ff1c0 2024-04-02T04:54:07.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.538+0000 7f01f1a9b640 1 --2- 172.21.15.16:0/1757397726 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f01d806dcd0 0x7f01d8070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:07.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.538+0000 7f01f1a9b640 1 --2- 172.21.15.16:0/1757397726 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f01d806dcd0 0x7f01d8070190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f01dc000c00 tx=0x7f01dc005fd0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:07.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.542+0000 7f01e37fe640 1 -- 172.21.15.16:0/1757397726 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f01e805b0f0 con 0x7f01ec0ff1c0 2024-04-02T04:54:07.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:07 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/3414988377' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T04:54:07.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:07 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:54:07.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:07 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/4240440293' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T04:54:07.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.739+0000 7f01f4500640 1 -- 172.21.15.16:0/1757397726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f01ec04e030 con 0x7f01ec0ff1c0 2024-04-02T04:54:07.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.740+0000 7f01e37fe640 1 -- 172.21.15.16:0/1757397726 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7f01e805ac80 con 0x7f01ec0ff1c0 2024-04-02T04:54:07.744 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-02T04:54:07.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.742+0000 7f01f4500640 1 -- 172.21.15.16:0/1757397726 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f01d806dcd0 msgr2=0x7f01d8070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:07.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.742+0000 7f01f4500640 1 --2- 172.21.15.16:0/1757397726 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f01d806dcd0 0x7f01d8070190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f01dc000c00 tx=0x7f01dc005fd0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.742+0000 7f01f4500640 1 -- 172.21.15.16:0/1757397726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01ec0ff1c0 msgr2=0x7f01ec196e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:07.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.742+0000 7f01f4500640 1 --2- 172.21.15.16:0/1757397726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01ec0ff1c0 0x7f01ec196e90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f01e800e9a0 tx=0x7f01e800eda0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.742+0000 7f01f4500640 1 -- 172.21.15.16:0/1757397726 shutdown_connections 2024-04-02T04:54:07.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.742+0000 7f01f4500640 1 --2- 172.21.15.16:0/1757397726 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f01d806dcd0 0x7f01d8070190 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.742+0000 7f01f4500640 1 --2- 172.21.15.16:0/1757397726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01ec1973d0 0x7f01ec197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.742+0000 7f01f4500640 1 --2- 172.21.15.16:0/1757397726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01ec0ff1c0 0x7f01ec196e90 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.742+0000 7f01f4500640 1 -- 172.21.15.16:0/1757397726 >> 172.21.15.16:0/1757397726 conn(0x7f01ec074960 msgr2=0x7f01ec102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:07.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.742+0000 7f01f4500640 1 -- 172.21.15.16:0/1757397726 shutdown_connections 2024-04-02T04:54:07.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.742+0000 7f01f4500640 1 -- 172.21.15.16:0/1757397726 wait complete. 2024-04-02T04:54:07.756 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:07 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/3414988377' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T04:54:07.757 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:07 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:54:07.757 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:07 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/4240440293' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T04:54:07.758 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-02T04:54:07.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.863+0000 7f7238b0a640 1 -- 172.21.15.16:0/2088023532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72340ff1c0 msgr2=0x7f72340ff5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:07.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.863+0000 7f7238b0a640 1 --2- 172.21.15.16:0/2088023532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72340ff1c0 0x7f72340ff5a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7224009880 tx=0x7f722402bce0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.864+0000 7f7238b0a640 1 -- 172.21.15.16:0/2088023532 shutdown_connections 2024-04-02T04:54:07.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.864+0000 7f7238b0a640 1 --2- 172.21.15.16:0/2088023532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72340ffb70 0x7f7234103aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.864+0000 7f7238b0a640 1 --2- 172.21.15.16:0/2088023532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72340ff1c0 0x7f72340ff5a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.864+0000 7f7238b0a640 1 -- 172.21.15.16:0/2088023532 >> 172.21.15.16:0/2088023532 conn(0x7f7234074960 msgr2=0x7f7234074d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:07.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.864+0000 7f7238b0a640 1 -- 172.21.15.16:0/2088023532 shutdown_connections 2024-04-02T04:54:07.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.864+0000 7f7238b0a640 1 -- 172.21.15.16:0/2088023532 wait complete. 2024-04-02T04:54:07.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.864+0000 7f7238b0a640 1 Processor -- start 2024-04-02T04:54:07.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.864+0000 7f7238b0a640 1 -- start start 2024-04-02T04:54:07.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.865+0000 7f7238b0a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72340ff1c0 0x7f7234100b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:07.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.865+0000 7f7238b0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7234101080 0x7f72341014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:07.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.865+0000 7f7238b0a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7234102af0 con 0x7f7234101080 2024-04-02T04:54:07.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.865+0000 7f7238b0a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72341a21e0 con 0x7f72340ff1c0 2024-04-02T04:54:07.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.865+0000 7f7231d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7234101080 0x7f72341014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:07.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.865+0000 7f7231d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7234101080 0x7f72341014e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58418/0 (socket says 172.21.15.16:58418) 2024-04-02T04:54:07.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.865+0000 7f7231d9b640 1 -- 172.21.15.16:0/2296082574 learned_addr learned my addr 172.21.15.16:0/2296082574 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:54:07.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.865+0000 7f723259c640 1 --2- 172.21.15.16:0/2296082574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72340ff1c0 0x7f7234100b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:07.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.865+0000 7f7231d9b640 1 -- 172.21.15.16:0/2296082574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72340ff1c0 msgr2=0x7f7234100b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:07.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.865+0000 7f7231d9b640 1 --2- 172.21.15.16:0/2296082574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72340ff1c0 0x7f7234100b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:07.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.865+0000 7f7231d9b640 1 -- 172.21.15.16:0/2296082574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7224009560 con 0x7f7234101080 2024-04-02T04:54:07.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.865+0000 7f7231d9b640 1 --2- 172.21.15.16:0/2296082574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7234101080 0x7f72341014e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f722800e940 tx=0x7f722800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:07.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.866+0000 7f72237fe640 1 -- 172.21.15.16:0/2296082574 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f722800cc90 con 0x7f7234101080 2024-04-02T04:54:07.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.866+0000 7f72237fe640 1 -- 172.21.15.16:0/2296082574 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f722800cdf0 con 0x7f7234101080 2024-04-02T04:54:07.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.866+0000 7f7238b0a640 1 -- 172.21.15.16:0/2296082574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72341a2440 con 0x7f7234101080 2024-04-02T04:54:07.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.866+0000 7f72237fe640 1 -- 172.21.15.16:0/2296082574 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7228010430 con 0x7f7234101080 2024-04-02T04:54:07.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.866+0000 7f7238b0a640 1 -- 172.21.15.16:0/2296082574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72341a2970 con 0x7f7234101080 2024-04-02T04:54:07.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.867+0000 7f7238b0a640 1 -- 172.21.15.16:0/2296082574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7234100580 con 0x7f7234101080 2024-04-02T04:54:07.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.867+0000 7f72237fe640 1 -- 172.21.15.16:0/2296082574 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7f72280046c0 con 0x7f7234101080 2024-04-02T04:54:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.867+0000 7f72237fe640 1 --2- 172.21.15.16:0/2296082574 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f721c06dcd0 0x7f721c070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.868+0000 7f72237fe640 1 -- 172.21.15.16:0/2296082574 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f7228014070 con 0x7f7234101080 2024-04-02T04:54:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.868+0000 7f723259c640 1 --2- 172.21.15.16:0/2296082574 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f721c06dcd0 0x7f721c070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.868+0000 7f723259c640 1 --2- 172.21.15.16:0/2296082574 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f721c06dcd0 0x7f721c070190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7224009230 tx=0x7f722402cf90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:07.875+0000 7f72237fe640 1 -- 172.21.15.16:0/2296082574 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7228057560 con 0x7f7234101080 2024-04-02T04:54:08.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.198+0000 7f7238b0a640 1 -- 172.21.15.16:0/2296082574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f723404e030 con 0x7f7234101080 2024-04-02T04:54:08.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.198+0000 7f72237fe640 1 -- 172.21.15.16:0/2296082574 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7f722805ac20 con 0x7f7234101080 2024-04-02T04:54:08.203 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-04-02T04:54:08.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.200+0000 7f7238b0a640 1 -- 172.21.15.16:0/2296082574 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f721c06dcd0 msgr2=0x7f721c070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:08.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.200+0000 7f7238b0a640 1 --2- 172.21.15.16:0/2296082574 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f721c06dcd0 0x7f721c070190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7224009230 tx=0x7f722402cf90 comp rx=0 tx=0).stop 2024-04-02T04:54:08.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.200+0000 7f7238b0a640 1 -- 172.21.15.16:0/2296082574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7234101080 msgr2=0x7f72341014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:08.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.200+0000 7f7238b0a640 1 --2- 172.21.15.16:0/2296082574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7234101080 0x7f72341014e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f722800e940 tx=0x7f722800ee00 comp rx=0 tx=0).stop 2024-04-02T04:54:08.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.200+0000 7f7238b0a640 1 -- 172.21.15.16:0/2296082574 shutdown_connections 2024-04-02T04:54:08.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.200+0000 7f7238b0a640 1 --2- 172.21.15.16:0/2296082574 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7f721c06dcd0 0x7f721c070190 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:08.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.200+0000 7f7238b0a640 1 --2- 172.21.15.16:0/2296082574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7234101080 0x7f72341014e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:08.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.200+0000 7f7238b0a640 1 --2- 172.21.15.16:0/2296082574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72340ff1c0 0x7f7234100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:08.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.200+0000 7f7238b0a640 1 -- 172.21.15.16:0/2296082574 >> 172.21.15.16:0/2296082574 conn(0x7f7234074960 msgr2=0x7f7234067580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:08.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.200+0000 7f7238b0a640 1 -- 172.21.15.16:0/2296082574 shutdown_connections 2024-04-02T04:54:08.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.200+0000 7f7238b0a640 1 -- 172.21.15.16:0/2296082574 wait complete. 2024-04-02T04:54:08.216 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-02T04:54:08.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.329+0000 7fd577f92640 1 -- 172.21.15.16:0/2810828632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5700ffa70 msgr2=0x7fd570103ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:08.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.329+0000 7fd577f92640 1 --2- 172.21.15.16:0/2810828632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5700ffa70 0x7fd570103ab0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd560009880 tx=0x7fd56002bce0 comp rx=0 tx=0).stop 2024-04-02T04:54:08.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.329+0000 7fd577f92640 1 -- 172.21.15.16:0/2810828632 shutdown_connections 2024-04-02T04:54:08.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.329+0000 7fd577f92640 1 --2- 172.21.15.16:0/2810828632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5700ffa70 0x7fd570103ab0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:08.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.329+0000 7fd577f92640 1 --2- 172.21.15.16:0/2810828632 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5700ff0c0 0x7fd5700ff4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:08.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.329+0000 7fd577f92640 1 -- 172.21.15.16:0/2810828632 >> 172.21.15.16:0/2810828632 conn(0x7fd5700fa920 msgr2=0x7fd5700fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:08.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.329+0000 7fd577f92640 1 -- 172.21.15.16:0/2810828632 shutdown_connections 2024-04-02T04:54:08.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.329+0000 7fd577f92640 1 -- 172.21.15.16:0/2810828632 wait complete. 2024-04-02T04:54:08.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.330+0000 7fd577f92640 1 Processor -- start 2024-04-02T04:54:08.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.330+0000 7fd577f92640 1 -- start start 2024-04-02T04:54:08.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.330+0000 7fd577f92640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5700ff0c0 0x7fd570195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:08.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.330+0000 7fd577f92640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5701964f0 0x7fd57019a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:08.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.330+0000 7fd577f92640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd570196af0 con 0x7fd5701964f0 2024-04-02T04:54:08.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.330+0000 7fd577f92640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd570196c60 con 0x7fd5700ff0c0 2024-04-02T04:54:08.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.330+0000 7fd57552d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5701964f0 0x7fd57019a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:08.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.330+0000 7fd57552d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5701964f0 0x7fd57019a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58426/0 (socket says 172.21.15.16:58426) 2024-04-02T04:54:08.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.330+0000 7fd57552d640 1 -- 172.21.15.16:0/2964163452 learned_addr learned my addr 172.21.15.16:0/2964163452 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-02T04:54:08.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.330+0000 7fd575d2e640 1 --2- 172.21.15.16:0/2964163452 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5700ff0c0 0x7fd570195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:08.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.330+0000 7fd57552d640 1 -- 172.21.15.16:0/2964163452 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5700ff0c0 msgr2=0x7fd570195fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:08.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.330+0000 7fd57552d640 1 --2- 172.21.15.16:0/2964163452 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5700ff0c0 0x7fd570195fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:08.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.331+0000 7fd57552d640 1 -- 172.21.15.16:0/2964163452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd560009560 con 0x7fd5701964f0 2024-04-02T04:54:08.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.331+0000 7fd575d2e640 1 --2- 172.21.15.16:0/2964163452 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5700ff0c0 0x7fd570195fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T04:54:08.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.331+0000 7fd57552d640 1 --2- 172.21.15.16:0/2964163452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5701964f0 0x7fd57019a890 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd560036040 tx=0x7fd560004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:08.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.331+0000 7fd566ffd640 1 -- 172.21.15.16:0/2964163452 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5600035f0 con 0x7fd5701964f0 2024-04-02T04:54:08.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.331+0000 7fd566ffd640 1 -- 172.21.15.16:0/2964163452 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd560003c10 con 0x7fd5701964f0 2024-04-02T04:54:08.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.331+0000 7fd577f92640 1 -- 172.21.15.16:0/2964163452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd57019ae30 con 0x7fd5701964f0 2024-04-02T04:54:08.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.331+0000 7fd566ffd640 1 -- 172.21.15.16:0/2964163452 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd560034440 con 0x7fd5701964f0 2024-04-02T04:54:08.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.331+0000 7fd577f92640 1 -- 172.21.15.16:0/2964163452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd57019b390 con 0x7fd5701964f0 2024-04-02T04:54:08.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.332+0000 7fd577f92640 1 -- 172.21.15.16:0/2964163452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd570100590 con 0x7fd5701964f0 2024-04-02T04:54:08.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.332+0000 7fd566ffd640 1 -- 172.21.15.16:0/2964163452 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 38) v1 ==== 91861+0+0 (secure 0 0 0) 0x7fd560004f60 con 0x7fd5701964f0 2024-04-02T04:54:08.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.333+0000 7fd566ffd640 1 --2- 172.21.15.16:0/2964163452 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd55c06dcd0 0x7fd55c070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T04:54:08.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.333+0000 7fd566ffd640 1 -- 172.21.15.16:0/2964163452 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fd5600b0010 con 0x7fd5701964f0 2024-04-02T04:54:08.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.333+0000 7fd575d2e640 1 --2- 172.21.15.16:0/2964163452 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd55c06dcd0 0x7fd55c070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T04:54:08.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.333+0000 7fd575d2e640 1 --2- 172.21.15.16:0/2964163452 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd55c06dcd0 0x7fd55c070190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd56c005fd0 tx=0x7fd56c005ea0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T04:54:08.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.338+0000 7fd566ffd640 1 -- 172.21.15.16:0/2964163452 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd560079500 con 0x7fd5701964f0 2024-04-02T04:54:08.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.533+0000 7fd577f92640 1 -- 172.21.15.16:0/2964163452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd57004e030 con 0x7fd5701964f0 2024-04-02T04:54:08.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.533+0000 7fd566ffd640 1 -- 172.21.15.16:0/2964163452 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 43 entries v84) v1 ==== 81+0+2535 (secure 0 0 0) 0x7fd56007cbc0 con 0x7fd5701964f0 2024-04-02T04:54:08.538 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.27:6827/4177192740 2024-04-03T04:50:07.886976+0000 2024-04-02T04:54:08.538 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.27:6824/1461455976 2024-04-03T04:49:41.368223+0000 2024-04-02T04:54:08.538 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2790817266 2024-04-03T04:44:17.011409+0000 2024-04-02T04:54:08.538 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1537212641 2024-04-03T04:44:17.011409+0000 2024-04-02T04:54:08.538 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3646287833 2024-04-03T04:44:17.011409+0000 2024-04-02T04:54:08.538 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/873531241 2024-04-03T04:33:28.907145+0000 2024-04-02T04:54:08.538 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2255383579 2024-04-03T04:32:56.644452+0000 2024-04-02T04:54:08.538 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/2776599951 2024-04-03T04:44:17.011409+0000 2024-04-02T04:54:08.538 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3964929492 2024-04-03T04:44:17.011409+0000 2024-04-02T04:54:08.538 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.27:0/597316902 2024-04-03T04:43:10.596160+0000 2024-04-02T04:54:08.538 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/4067493740 2024-04-03T04:33:28.907145+0000 2024-04-02T04:54:08.539 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/3912038174 2024-04-03T04:34:42.713541+0000 2024-04-02T04:54:08.539 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/3912038174 2024-04-03T04:34:42.713541+0000 2024-04-02T04:54:08.539 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1664387083 2024-04-03T04:42:07.942513+0000 2024-04-02T04:54:08.539 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.27:6825/1461455976 2024-04-03T04:49:41.368223+0000 2024-04-02T04:54:08.539 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.27:0/1314131858 2024-04-03T04:43:10.596160+0000 2024-04-02T04:54:08.539 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.27:6826/4177192740 2024-04-03T04:50:07.886976+0000 2024-04-02T04:54:08.539 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/873531241 2024-04-03T04:33:28.907145+0000 2024-04-02T04:54:08.539 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/401549934 2024-04-03T04:32:56.644452+0000 2024-04-02T04:54:08.539 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/3643600900 2024-04-03T04:42:07.942513+0000 2024-04-02T04:54:08.539 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/166410319 2024-04-03T04:33:28.907145+0000 2024-04-02T04:54:08.539 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1884756010 2024-04-03T04:32:56.644452+0000 2024-04-02T04:54:08.540 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/824107799 2024-04-03T04:32:56.644452+0000 2024-04-02T04:54:08.540 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/881481317 2024-04-03T04:44:17.011409+0000 2024-04-02T04:54:08.540 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/3643600900 2024-04-03T04:42:07.942513+0000 2024-04-02T04:54:08.540 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/4209328561 2024-04-03T04:34:42.713541+0000 2024-04-02T04:54:08.540 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1920858754 2024-04-03T04:42:07.942513+0000 2024-04-02T04:54:08.540 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3733852550 2024-04-03T04:34:42.713541+0000 2024-04-02T04:54:08.540 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/824107799 2024-04-03T04:32:56.644452+0000 2024-04-02T04:54:08.540 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.27:0/3186228192 2024-04-03T04:43:10.596160+0000 2024-04-02T04:54:08.540 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.27:0/1987939490 2024-04-03T04:43:10.596160+0000 2024-04-02T04:54:08.540 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3405957790 2024-04-03T04:34:42.713541+0000 2024-04-02T04:54:08.540 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1383394471 2024-04-03T04:42:07.942513+0000 2024-04-02T04:54:08.540 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.27:6828/1554339939 2024-04-03T04:43:10.596160+0000 2024-04-02T04:54:08.540 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1228221358 2024-04-03T04:33:28.907145+0000 2024-04-02T04:54:08.540 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2373084595 2024-04-03T04:42:07.942513+0000 2024-04-02T04:54:08.540 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/249069849 2024-04-03T04:44:17.011409+0000 2024-04-02T04:54:08.541 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.27:0/3624631480 2024-04-03T04:43:10.596160+0000 2024-04-02T04:54:08.541 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.27:0/2708474610 2024-04-03T04:43:10.596160+0000 2024-04-02T04:54:08.541 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6826/71509234 2024-04-03T04:39:46.861330+0000 2024-04-02T04:54:08.541 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6827/71509234 2024-04-03T04:39:46.861330+0000 2024-04-02T04:54:08.541 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.27:6829/1554339939 2024-04-03T04:43:10.596160+0000 2024-04-02T04:54:08.541 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/2776599951 2024-04-03T04:44:17.011409+0000 2024-04-02T04:54:08.542 INFO:teuthology.orchestra.run.smithi016.stderr:listed 43 entries 2024-04-02T04:54:08.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.535+0000 7fd577f92640 1 -- 172.21.15.16:0/2964163452 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd55c06dcd0 msgr2=0x7fd55c070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:08.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.535+0000 7fd577f92640 1 --2- 172.21.15.16:0/2964163452 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd55c06dcd0 0x7fd55c070190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd56c005fd0 tx=0x7fd56c005ea0 comp rx=0 tx=0).stop 2024-04-02T04:54:08.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.535+0000 7fd577f92640 1 -- 172.21.15.16:0/2964163452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5701964f0 msgr2=0x7fd57019a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T04:54:08.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.535+0000 7fd577f92640 1 --2- 172.21.15.16:0/2964163452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5701964f0 0x7fd57019a890 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd560036040 tx=0x7fd560004040 comp rx=0 tx=0).stop 2024-04-02T04:54:08.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.535+0000 7fd577f92640 1 -- 172.21.15.16:0/2964163452 shutdown_connections 2024-04-02T04:54:08.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.535+0000 7fd577f92640 1 --2- 172.21.15.16:0/2964163452 >> [v2:172.21.15.16:6800/835455925,v1:172.21.15.16:6801/835455925] conn(0x7fd55c06dcd0 0x7fd55c070190 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:08.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.535+0000 7fd577f92640 1 --2- 172.21.15.16:0/2964163452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5701964f0 0x7fd57019a890 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:08.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.535+0000 7fd577f92640 1 --2- 172.21.15.16:0/2964163452 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5700ff0c0 0x7fd570195fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T04:54:08.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.535+0000 7fd577f92640 1 -- 172.21.15.16:0/2964163452 >> 172.21.15.16:0/2964163452 conn(0x7fd5700fa920 msgr2=0x7fd570068820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T04:54:08.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.535+0000 7fd577f92640 1 -- 172.21.15.16:0/2964163452 shutdown_connections 2024-04-02T04:54:08.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-02T04:54:08.535+0000 7fd577f92640 1 -- 172.21.15.16:0/2964163452 wait complete. 2024-04-02T04:54:08.550 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:08 smithi016 ceph-mon[93230]: pgmap v333: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:08.550 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:08 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/3029347187' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T04:54:08.550 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:08 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:54:08.550 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:08 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:54:08.550 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:08 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/1757397726' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-02T04:54:08.550 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:08 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:54:08.550 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:08 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:54:08.550 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:08 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:54:08.551 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:08 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/2296082574' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-02T04:54:08.551 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2024-04-02T04:54:08.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T04:54:08.552 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-02T04:54:08.578 INFO:teuthology.orchestra.run:waiting for 300 2024-04-02T04:54:08.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:08 smithi027 ceph-mon[83036]: pgmap v333: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:08.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:08 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/3029347187' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T04:54:08.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:08 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:54:08.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:08 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:54:08.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:08 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/1757397726' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-02T04:54:08.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:08 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:54:08.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:08 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:54:08.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:08 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:54:08.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:08 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/2296082574' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-02T04:54:09.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:09 smithi027 ceph-mon[83036]: from='client.? 172.21.15.16:0/2964163452' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T04:54:09.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:09 smithi016 ceph-mon[93230]: from='client.? 172.21.15.16:0/2964163452' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T04:54:10.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:10 smithi027 ceph-mon[83036]: pgmap v334: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:10.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:10 smithi016 ceph-mon[93230]: pgmap v334: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:12.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:12 smithi027 ceph-mon[83036]: pgmap v335: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:12.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:12 smithi016 ceph-mon[93230]: pgmap v335: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:14.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:14 smithi027 ceph-mon[83036]: pgmap v336: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:14.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:14 smithi016 ceph-mon[93230]: pgmap v336: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:16.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:16 smithi027 ceph-mon[83036]: pgmap v337: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:16.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:16 smithi016 ceph-mon[93230]: pgmap v337: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:17.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:17 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:54:17.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:17 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:54:18.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:18 smithi016 ceph-mon[93230]: pgmap v338: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:18.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:18 smithi027 ceph-mon[83036]: pgmap v338: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:20.636 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:20 smithi016 ceph-mon[93230]: pgmap v339: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:20.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:20 smithi027 ceph-mon[83036]: pgmap v339: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:22.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:22 smithi027 ceph-mon[83036]: pgmap v340: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:22.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:22 smithi016 ceph-mon[93230]: pgmap v340: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:24.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:24 smithi027 ceph-mon[83036]: pgmap v341: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:24.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:24 smithi016 ceph-mon[93230]: pgmap v341: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:26.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:26 smithi027 ceph-mon[83036]: pgmap v342: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:26.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:26 smithi016 ceph-mon[93230]: pgmap v342: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:28.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:28 smithi027 ceph-mon[83036]: pgmap v343: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:28.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:28 smithi016 ceph-mon[93230]: pgmap v343: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:30.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:30 smithi027 ceph-mon[83036]: pgmap v344: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:30.746 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:30 smithi016 ceph-mon[93230]: pgmap v344: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:32.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:32 smithi027 ceph-mon[83036]: pgmap v345: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:32.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:32 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:54:32.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:32 smithi016 ceph-mon[93230]: pgmap v345: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:32.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:32 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:54:34.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:34 smithi016 ceph-mon[93230]: pgmap v346: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:34.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:34 smithi027 ceph-mon[83036]: pgmap v346: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:36.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:36 smithi016 ceph-mon[93230]: pgmap v347: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:36.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:36 smithi027 ceph-mon[83036]: pgmap v347: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:38.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:38 smithi016 ceph-mon[93230]: pgmap v348: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:38.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:38 smithi027 ceph-mon[83036]: pgmap v348: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:40.680 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:40 smithi027 ceph-mon[83036]: pgmap v349: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:40.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:40 smithi016 ceph-mon[93230]: pgmap v349: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:42.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:42 smithi016 ceph-mon[93230]: pgmap v350: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:42.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:42 smithi027 ceph-mon[83036]: pgmap v350: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:44.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:44 smithi016 ceph-mon[93230]: pgmap v351: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:44.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:44 smithi027 ceph-mon[83036]: pgmap v351: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:45.729 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:45 smithi027 ceph-mon[83036]: pgmap v352: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:45.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:45 smithi016 ceph-mon[93230]: pgmap v352: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:47.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:47 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:54:47.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:47 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:54:48.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:48 smithi016 ceph-mon[93230]: pgmap v353: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:48.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:48 smithi027 ceph-mon[83036]: pgmap v353: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:50.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:50 smithi016 ceph-mon[93230]: pgmap v354: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:50.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:50 smithi027 ceph-mon[83036]: pgmap v354: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:52.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:52 smithi016 ceph-mon[93230]: pgmap v355: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:52.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:52 smithi027 ceph-mon[83036]: pgmap v355: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:54.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:54 smithi016 ceph-mon[93230]: pgmap v356: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:54.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:54 smithi027 ceph-mon[83036]: pgmap v356: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:56.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:56 smithi016 ceph-mon[93230]: pgmap v357: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:56.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:56 smithi027 ceph-mon[83036]: pgmap v357: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:58.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:54:58 smithi016 ceph-mon[93230]: pgmap v358: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:54:58.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:54:58 smithi027 ceph-mon[83036]: pgmap v358: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:00.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:55:00 smithi016 ceph-mon[93230]: pgmap v359: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:00.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:55:00 smithi027 ceph-mon[83036]: pgmap v359: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:02.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:55:02 smithi016 ceph-mon[93230]: pgmap v360: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:02.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:55:02 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:55:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:55:02 smithi027 ceph-mon[83036]: pgmap v360: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:55:02 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:55:04.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:55:04 smithi016 ceph-mon[93230]: pgmap v361: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:04.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:55:04 smithi027 ceph-mon[83036]: pgmap v361: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:06.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:55:06 smithi016 ceph-mon[93230]: pgmap v362: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:06.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:55:06 smithi027 ceph-mon[83036]: pgmap v362: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:08.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:55:08 smithi016 ceph-mon[93230]: pgmap v363: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:08.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:55:08 smithi027 ceph-mon[83036]: pgmap v363: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:09.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:55:09 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:55:09.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:55:09 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:55:09.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:55:09 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:55:09.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:55:09 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:55:09.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:55:09 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:55:09.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:55:09 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:55:09.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:55:09 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:55:09.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:55:09 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:55:10.547 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:55:10 smithi027 ceph-mon[83036]: pgmap v364: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:10.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:55:10 smithi016 ceph-mon[93230]: pgmap v364: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:12.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:55:12 smithi016 ceph-mon[93230]: pgmap v365: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:12.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:55:12 smithi027 ceph-mon[83036]: pgmap v365: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:14.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:55:14 smithi016 ceph-mon[93230]: pgmap v366: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:14.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:55:14 smithi027 ceph-mon[83036]: pgmap v366: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:16.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:55:16 smithi027 ceph-mon[83036]: pgmap v367: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:16.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:55:16 smithi016 ceph-mon[93230]: pgmap v367: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:17.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:55:17 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:55:17.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:55:17 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:55:18.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:55:18 smithi016 ceph-mon[93230]: pgmap v368: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:18.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:55:18 smithi027 ceph-mon[83036]: pgmap v368: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:20.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:55:20 smithi016 ceph-mon[93230]: pgmap v369: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:20.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:55:20 smithi027 ceph-mon[83036]: pgmap v369: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:22.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:55:22 smithi016 ceph-mon[93230]: pgmap v370: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:22.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:55:22 smithi027 ceph-mon[83036]: pgmap v370: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:24.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:55:24 smithi016 ceph-mon[93230]: pgmap v371: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:24.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:55:24 smithi027 ceph-mon[83036]: pgmap v371: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:26.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:55:26 smithi027 ceph-mon[83036]: pgmap v372: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:26.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:55:26 smithi016 ceph-mon[93230]: pgmap v372: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:28.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:55:28 smithi016 ceph-mon[93230]: pgmap v373: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:28.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:55:28 smithi027 ceph-mon[83036]: pgmap v373: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:30.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:55:30 smithi016 ceph-mon[93230]: pgmap v374: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:30.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:55:30 smithi027 ceph-mon[83036]: pgmap v374: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:32.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:55:32 smithi016 ceph-mon[93230]: pgmap v375: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:32.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:55:32 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:55:32.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:55:32 smithi027 ceph-mon[83036]: pgmap v375: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:32.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:55:32 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:55:34.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:55:34 smithi016 ceph-mon[93230]: pgmap v376: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:34.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:55:34 smithi027 ceph-mon[83036]: pgmap v376: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:36.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:55:36 smithi016 ceph-mon[93230]: pgmap v377: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:36.605 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:55:36 smithi027 ceph-mon[83036]: pgmap v377: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:38.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:55:38 smithi016 ceph-mon[93230]: pgmap v378: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:38.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:55:38 smithi027 ceph-mon[83036]: pgmap v378: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:40.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:55:40 smithi016 ceph-mon[93230]: pgmap v379: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:40.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:55:40 smithi027 ceph-mon[83036]: pgmap v379: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:42.562 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:55:42 smithi016 ceph-mon[93230]: pgmap v380: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:42.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:55:42 smithi027 ceph-mon[83036]: pgmap v380: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:44.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:55:44 smithi016 ceph-mon[93230]: pgmap v381: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:44.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:55:44 smithi027 ceph-mon[83036]: pgmap v381: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:46.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:55:46 smithi016 ceph-mon[93230]: pgmap v382: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:46.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:55:46 smithi027 ceph-mon[83036]: pgmap v382: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:47.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:55:47 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:55:47.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:55:47 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:55:48.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:55:48 smithi016 ceph-mon[93230]: pgmap v383: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:48.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:55:48 smithi027 ceph-mon[83036]: pgmap v383: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:50.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:55:50 smithi016 ceph-mon[93230]: pgmap v384: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:50.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:55:50 smithi027 ceph-mon[83036]: pgmap v384: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:52.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:55:52 smithi016 ceph-mon[93230]: pgmap v385: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:52.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:55:52 smithi027 ceph-mon[83036]: pgmap v385: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:54.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:55:54 smithi016 ceph-mon[93230]: pgmap v386: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:54.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:55:54 smithi027 ceph-mon[83036]: pgmap v386: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:56.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:55:56 smithi016 ceph-mon[93230]: pgmap v387: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:56.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:55:56 smithi027 ceph-mon[83036]: pgmap v387: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:58.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:55:58 smithi016 ceph-mon[93230]: pgmap v388: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:55:58.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:55:58 smithi027 ceph-mon[83036]: pgmap v388: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:00.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:00 smithi016 ceph-mon[93230]: pgmap v389: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:00.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:00 smithi027 ceph-mon[83036]: pgmap v389: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:02.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:02 smithi016 ceph-mon[93230]: pgmap v390: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:02.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:02 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:56:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:02 smithi027 ceph-mon[83036]: pgmap v390: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:02.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:02 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:56:04.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:04 smithi016 ceph-mon[93230]: pgmap v391: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:04.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:04 smithi027 ceph-mon[83036]: pgmap v391: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:06.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:06 smithi027 ceph-mon[83036]: pgmap v392: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:06.801 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:06 smithi016 ceph-mon[93230]: pgmap v392: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:08.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:08 smithi027 ceph-mon[83036]: pgmap v393: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:08.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:08 smithi016 ceph-mon[93230]: pgmap v393: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:09.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:09 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:56:09.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:09 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:56:10.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:10 smithi027 ceph-mon[83036]: pgmap v394: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:10.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:10 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:56:10.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:10 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:56:10.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:10 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:56:10.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:10 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:56:10.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:10 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:56:10.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:10 smithi016 ceph-mon[93230]: pgmap v394: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:10.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:10 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:56:10.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:10 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:56:10.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:10 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:56:10.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:10 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:56:10.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:10 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:56:12.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:12 smithi027 ceph-mon[83036]: pgmap v395: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:12.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:12 smithi016 ceph-mon[93230]: pgmap v395: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:14.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:14 smithi027 ceph-mon[83036]: pgmap v396: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:14.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:14 smithi016 ceph-mon[93230]: pgmap v396: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:16.611 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:16 smithi016 ceph-mon[93230]: pgmap v397: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:16.622 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:16 smithi027 ceph-mon[83036]: pgmap v397: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:17.625 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:17 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:56:17.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:17 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:56:18.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:18 smithi027 ceph-mon[83036]: pgmap v398: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:18.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:18 smithi016 ceph-mon[93230]: pgmap v398: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:20.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:20 smithi027 ceph-mon[83036]: pgmap v399: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:20.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:20 smithi016 ceph-mon[93230]: pgmap v399: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:22.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:22 smithi027 ceph-mon[83036]: pgmap v400: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:22.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:22 smithi016 ceph-mon[93230]: pgmap v400: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:24.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:24 smithi027 ceph-mon[83036]: pgmap v401: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:24.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:24 smithi016 ceph-mon[93230]: pgmap v401: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:26.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:26 smithi027 ceph-mon[83036]: pgmap v402: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:26.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:26 smithi016 ceph-mon[93230]: pgmap v402: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:28.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:28 smithi027 ceph-mon[83036]: pgmap v403: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:28.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:28 smithi016 ceph-mon[93230]: pgmap v403: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:30.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:30 smithi027 ceph-mon[83036]: pgmap v404: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:30.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:30 smithi016 ceph-mon[93230]: pgmap v404: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:32.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:32 smithi027 ceph-mon[83036]: pgmap v405: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:32.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:32 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:56:32.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:32 smithi016 ceph-mon[93230]: pgmap v405: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:32.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:32 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:56:34.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:34 smithi027 ceph-mon[83036]: pgmap v406: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:34.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:34 smithi016 ceph-mon[93230]: pgmap v406: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:36.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:36 smithi027 ceph-mon[83036]: pgmap v407: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:36.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:36 smithi016 ceph-mon[93230]: pgmap v407: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:38.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:38 smithi027 ceph-mon[83036]: pgmap v408: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:38.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:38 smithi016 ceph-mon[93230]: pgmap v408: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:40.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:40 smithi027 ceph-mon[83036]: pgmap v409: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:40.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:40 smithi016 ceph-mon[93230]: pgmap v409: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:42.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:42 smithi016 ceph-mon[93230]: pgmap v410: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:42.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:42 smithi027 ceph-mon[83036]: pgmap v410: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:44.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:44 smithi027 ceph-mon[83036]: pgmap v411: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:44.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:44 smithi016 ceph-mon[93230]: pgmap v411: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:46.611 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:46 smithi016 ceph-mon[93230]: pgmap v412: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:46.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:46 smithi027 ceph-mon[83036]: pgmap v412: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:47.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:47 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:56:47.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:47 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:56:48.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:48 smithi027 ceph-mon[83036]: pgmap v413: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:48.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:48 smithi016 ceph-mon[93230]: pgmap v413: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:50.649 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:50 smithi027 ceph-mon[83036]: pgmap v414: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:50.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:50 smithi016 ceph-mon[93230]: pgmap v414: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:52.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:52 smithi016 ceph-mon[93230]: pgmap v415: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:52.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:52 smithi027 ceph-mon[83036]: pgmap v415: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:54.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:54 smithi027 ceph-mon[83036]: pgmap v416: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:54.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:54 smithi016 ceph-mon[93230]: pgmap v416: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:56.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:56 smithi027 ceph-mon[83036]: pgmap v417: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:56.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:56 smithi016 ceph-mon[93230]: pgmap v417: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:58.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:56:58 smithi027 ceph-mon[83036]: pgmap v418: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:56:58.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:56:58 smithi016 ceph-mon[93230]: pgmap v418: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:00.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:57:00 smithi027 ceph-mon[83036]: pgmap v419: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:00.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:57:00 smithi016 ceph-mon[93230]: pgmap v419: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:57:02 smithi027 ceph-mon[83036]: pgmap v420: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:02.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:57:02 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:57:02.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:57:02 smithi016 ceph-mon[93230]: pgmap v420: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:02.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:57:02 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:57:04.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:57:04 smithi027 ceph-mon[83036]: pgmap v421: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:04.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:57:04 smithi016 ceph-mon[93230]: pgmap v421: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:06.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:57:06 smithi027 ceph-mon[83036]: pgmap v422: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:06.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:57:06 smithi016 ceph-mon[93230]: pgmap v422: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:08.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:57:08 smithi027 ceph-mon[83036]: pgmap v423: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:08.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:57:08 smithi016 ceph-mon[93230]: pgmap v423: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:10.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:57:10 smithi027 ceph-mon[83036]: pgmap v424: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:10.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:57:10 smithi016 ceph-mon[93230]: pgmap v424: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:11.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:57:11 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:57:11.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:57:11 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:57:11.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:57:11 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:57:11.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:57:11 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:57:11.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:57:11 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:57:11.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:57:11 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:57:11.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:57:11 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:57:11.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:57:11 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:57:12.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:57:12 smithi027 ceph-mon[83036]: pgmap v425: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:12.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:57:12 smithi016 ceph-mon[93230]: pgmap v425: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:14.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:57:14 smithi027 ceph-mon[83036]: pgmap v426: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:14.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:57:14 smithi016 ceph-mon[93230]: pgmap v426: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:16.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:57:16 smithi016 ceph-mon[93230]: pgmap v427: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:16.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:57:16 smithi027 ceph-mon[83036]: pgmap v427: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:17.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:57:17 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:57:17.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:57:17 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:57:18.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:57:18 smithi016 ceph-mon[93230]: pgmap v428: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:18.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:57:18 smithi027 ceph-mon[83036]: pgmap v428: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:20.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:57:20 smithi016 ceph-mon[93230]: pgmap v429: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:20.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:57:20 smithi027 ceph-mon[83036]: pgmap v429: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:22.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:57:22 smithi027 ceph-mon[83036]: pgmap v430: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:22.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:57:22 smithi016 ceph-mon[93230]: pgmap v430: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:24.310 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:57:23 smithi016 ceph-mon[93230]: pgmap v431: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:24.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:57:23 smithi027 ceph-mon[83036]: pgmap v431: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:26.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:57:26 smithi016 ceph-mon[93230]: pgmap v432: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:26.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:57:26 smithi027 ceph-mon[83036]: pgmap v432: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:28.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:57:28 smithi016 ceph-mon[93230]: pgmap v433: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:28.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:57:28 smithi027 ceph-mon[83036]: pgmap v433: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:30.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:57:30 smithi016 ceph-mon[93230]: pgmap v434: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:30.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:57:30 smithi027 ceph-mon[83036]: pgmap v434: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:32.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:57:32 smithi016 ceph-mon[93230]: pgmap v435: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:32.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:57:32 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:57:32.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:57:32 smithi027 ceph-mon[83036]: pgmap v435: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:32.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:57:32 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:57:34.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:57:34 smithi016 ceph-mon[93230]: pgmap v436: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:34.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:57:34 smithi027 ceph-mon[83036]: pgmap v436: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:36.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:57:36 smithi016 ceph-mon[93230]: pgmap v437: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:36.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:57:36 smithi027 ceph-mon[83036]: pgmap v437: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:38.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:57:38 smithi016 ceph-mon[93230]: pgmap v438: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:38.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:57:38 smithi027 ceph-mon[83036]: pgmap v438: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:40.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:57:40 smithi016 ceph-mon[93230]: pgmap v439: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:40.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:57:40 smithi027 ceph-mon[83036]: pgmap v439: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:42.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:57:42 smithi016 ceph-mon[93230]: pgmap v440: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:42.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:57:42 smithi027 ceph-mon[83036]: pgmap v440: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:44.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:57:44 smithi016 ceph-mon[93230]: pgmap v441: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:44.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:57:44 smithi027 ceph-mon[83036]: pgmap v441: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:46.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:57:46 smithi016 ceph-mon[93230]: pgmap v442: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:46.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:57:46 smithi027 ceph-mon[83036]: pgmap v442: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:47.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:57:47 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:57:47.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:57:47 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:57:48.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:57:48 smithi016 ceph-mon[93230]: pgmap v443: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:48.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:57:48 smithi027 ceph-mon[83036]: pgmap v443: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:50.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:57:50 smithi016 ceph-mon[93230]: pgmap v444: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:50.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:57:50 smithi027 ceph-mon[83036]: pgmap v444: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:52.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:57:52 smithi016 ceph-mon[93230]: pgmap v445: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:52.562 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:57:52 smithi027 ceph-mon[83036]: pgmap v445: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:54.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:57:54 smithi016 ceph-mon[93230]: pgmap v446: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:54.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:57:54 smithi027 ceph-mon[83036]: pgmap v446: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:56.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:57:56 smithi016 ceph-mon[93230]: pgmap v447: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:56.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:57:56 smithi027 ceph-mon[83036]: pgmap v447: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:58.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:57:58 smithi016 ceph-mon[93230]: pgmap v448: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:57:58.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:57:58 smithi027 ceph-mon[83036]: pgmap v448: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:00.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:58:00 smithi016 ceph-mon[93230]: pgmap v449: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:00.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:58:00 smithi027 ceph-mon[83036]: pgmap v449: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:02.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:58:02 smithi016 ceph-mon[93230]: pgmap v450: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:02.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:58:02 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:58:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:58:02 smithi027 ceph-mon[83036]: pgmap v450: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:58:02 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:58:04.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:58:04 smithi016 ceph-mon[93230]: pgmap v451: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:04.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:58:04 smithi027 ceph-mon[83036]: pgmap v451: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:06.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:58:06 smithi016 ceph-mon[93230]: pgmap v452: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:06.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:58:06 smithi027 ceph-mon[83036]: pgmap v452: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:08.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:58:08 smithi016 ceph-mon[93230]: pgmap v453: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:08.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:58:08 smithi027 ceph-mon[83036]: pgmap v453: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:10.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:58:10 smithi016 ceph-mon[93230]: pgmap v454: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:10.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:58:10 smithi027 ceph-mon[83036]: pgmap v454: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:11.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:58:11 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:58:11.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:58:11 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T04:58:12.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:58:12 smithi016 ceph-mon[93230]: pgmap v455: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:12.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:58:12 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:58:12.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:58:12 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:58:12.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:58:12 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:58:12.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:58:12 smithi027 ceph-mon[83036]: pgmap v455: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:12.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:58:12 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T04:58:12.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:58:12 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T04:58:12.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:58:12 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' 2024-04-02T04:58:14.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:58:14 smithi016 ceph-mon[93230]: pgmap v456: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:14.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:58:14 smithi027 ceph-mon[83036]: pgmap v456: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:16.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:58:16 smithi016 ceph-mon[93230]: pgmap v457: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:16.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:58:16 smithi027 ceph-mon[83036]: pgmap v457: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:17.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:58:17 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:58:17.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:58:17 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:58:18.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:58:18 smithi016 ceph-mon[93230]: pgmap v458: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:18.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:58:18 smithi027 ceph-mon[83036]: pgmap v458: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:20.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:58:20 smithi016 ceph-mon[93230]: pgmap v459: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:20.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:58:20 smithi027 ceph-mon[83036]: pgmap v459: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:22.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:58:22 smithi016 ceph-mon[93230]: pgmap v460: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:22.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:58:22 smithi027 ceph-mon[83036]: pgmap v460: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:24.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:58:24 smithi027 ceph-mon[83036]: pgmap v461: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:24.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:58:24 smithi016 ceph-mon[93230]: pgmap v461: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:26.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:58:26 smithi027 ceph-mon[83036]: pgmap v462: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:26.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:58:26 smithi016 ceph-mon[93230]: pgmap v462: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:28.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:58:28 smithi027 ceph-mon[83036]: pgmap v463: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:28.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:58:28 smithi016 ceph-mon[93230]: pgmap v463: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:30.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:58:30 smithi027 ceph-mon[83036]: pgmap v464: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:30.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:58:30 smithi016 ceph-mon[93230]: pgmap v464: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:32.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:58:32 smithi027 ceph-mon[83036]: pgmap v465: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:32.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:58:32 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:58:32.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:58:32 smithi016 ceph-mon[93230]: pgmap v465: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:32.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:58:32 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:58:34.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:58:34 smithi027 ceph-mon[83036]: pgmap v466: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:34.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:58:34 smithi016 ceph-mon[93230]: pgmap v466: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:36.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:58:36 smithi027 ceph-mon[83036]: pgmap v467: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:36.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:58:36 smithi016 ceph-mon[93230]: pgmap v467: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:38.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:58:38 smithi027 ceph-mon[83036]: pgmap v468: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:38.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:58:38 smithi016 ceph-mon[93230]: pgmap v468: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:40.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:58:40 smithi027 ceph-mon[83036]: pgmap v469: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:40.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:58:40 smithi016 ceph-mon[93230]: pgmap v469: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:42.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:58:42 smithi027 ceph-mon[83036]: pgmap v470: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:42.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:58:42 smithi016 ceph-mon[93230]: pgmap v470: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:44.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:58:44 smithi027 ceph-mon[83036]: pgmap v471: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:44.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:58:44 smithi016 ceph-mon[93230]: pgmap v471: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:46.612 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:58:46 smithi016 ceph-mon[93230]: pgmap v472: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:46.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:58:46 smithi027 ceph-mon[83036]: pgmap v472: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:47.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:58:47 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:58:47.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:58:47 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:58:48.396 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:58:48 smithi027 ceph-mon[83036]: pgmap v473: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:48.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:58:48 smithi016 ceph-mon[93230]: pgmap v473: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:50.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:58:50 smithi027 ceph-mon[83036]: pgmap v474: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:50.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:58:50 smithi016 ceph-mon[93230]: pgmap v474: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:52.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:58:52 smithi027 ceph-mon[83036]: pgmap v475: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:52.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:58:52 smithi016 ceph-mon[93230]: pgmap v475: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:54.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:58:54 smithi027 ceph-mon[83036]: pgmap v476: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:54.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:58:54 smithi016 ceph-mon[93230]: pgmap v476: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:56.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:58:56 smithi027 ceph-mon[83036]: pgmap v477: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:56.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:58:56 smithi016 ceph-mon[93230]: pgmap v477: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:58.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:58:58 smithi027 ceph-mon[83036]: pgmap v478: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:58:58.655 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:58:58 smithi016 ceph-mon[93230]: pgmap v478: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:59:00.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:59:00 smithi027 ceph-mon[83036]: pgmap v479: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:59:00.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:59:00 smithi016 ceph-mon[93230]: pgmap v479: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:59:02.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:59:02 smithi027 ceph-mon[83036]: pgmap v480: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:59:02.647 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:59:02 smithi027 ceph-mon[83036]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:59:02.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:59:02 smithi016 ceph-mon[93230]: pgmap v480: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:59:02.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:59:02 smithi016 ceph-mon[93230]: from='mgr.34102 172.21.15.16:0/447853927' entity='mgr.smithi016.eqnebx' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-02T04:59:04.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:59:04 smithi027 ceph-mon[83036]: pgmap v481: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:59:04.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:59:04 smithi016 ceph-mon[93230]: pgmap v481: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:59:06.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:59:06 smithi027 ceph-mon[83036]: pgmap v482: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:59:06.810 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:59:06 smithi016 ceph-mon[93230]: pgmap v482: 65 pgs: 65 active+clean; 260 MiB data, 1006 MiB used, 535 GiB / 536 GiB avail 2024-04-02T04:59:07.651 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-04-02T04:59:07.651 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_6cec793d9073f0594e2364b9b687ecaf22424cca/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_6cec793d9073f0594e2364b9b687ecaf22424cca/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-02T04:59:07.653 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-02T04:59:07.665 INFO:tasks.cephadm:Teardown begin 2024-04-02T04:59:07.665 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph_6cec793d9073f0594e2364b9b687ecaf22424cca/qa/tasks/cephadm.py", line 1807, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_6cec793d9073f0594e2364b9b687ecaf22424cca/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_6cec793d9073f0594e2364b9b687ecaf22424cca/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-02T04:59:07.666 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-02T04:59:07.696 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-02T04:59:07.735 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-02T04:59:07.735 DEBUG:teuthology.orchestra.run.smithi016:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-02T04:59:07.750 DEBUG:teuthology.orchestra.run.smithi027:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-02T04:59:07.793 INFO:tasks.cephadm:Stopping all daemons... 2024-04-02T04:59:07.794 INFO:tasks.cephadm.mon.smithi016:Stopping mon.smithi016... 2024-04-02T04:59:07.794 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@mon.smithi016 2024-04-02T04:59:08.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:59:07 smithi016 systemd[1]: Stopping Ceph mon.smithi016 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4... 2024-04-02T04:59:08.560 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:59:08 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi016[93226]: 2024-04-02T04:59:08.097+0000 7f13245b5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi016 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T04:59:08.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:59:08 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi016[93226]: 2024-04-02T04:59:08.097+0000 7f13245b5700 -1 mon.smithi016@0(leader) e3 *** Got Signal Terminated *** 2024-04-02T04:59:08.561 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:59:08 smithi016 podman[153832]: 2024-04-02 04:59:08.341432218 +0000 UTC m=+0.360930649 container died eabfc84f1cdff5cd60b50713584c6364a79145f2998e417d49bd432ff2679bf6 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi016, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-6cec793, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS) 2024-04-02T04:59:09.060 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:59:08 smithi016 podman[153832]: 2024-04-02 04:59:08.823794449 +0000 UTC m=+0.843292874 container cleanup eabfc84f1cdff5cd60b50713584c6364a79145f2998e417d49bd432ff2679bf6 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi016, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=squid-6cec793) 2024-04-02T04:59:09.061 INFO:journalctl@ceph.mon.smithi016.smithi016.stdout:Apr 02 04:59:08 smithi016 bash[153832]: ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi016 2024-04-02T04:59:09.196 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@mon.smithi016.service' 2024-04-02T04:59:09.240 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T04:59:09.240 INFO:tasks.cephadm.mon.smithi016:Stopped mon.smithi016 2024-04-02T04:59:09.240 INFO:tasks.cephadm.mon.smithi027:Stopping mon.smithi027... 2024-04-02T04:59:09.241 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@mon.smithi027 2024-04-02T04:59:09.543 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:59:09 smithi027 systemd[1]: Stopping Ceph mon.smithi027 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4... 2024-04-02T04:59:09.826 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:59:09 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi027[83031]: 2024-04-02T04:59:09.538+0000 7f92b31b5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi027 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T04:59:09.827 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:59:09 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi027[83031]: 2024-04-02T04:59:09.538+0000 7f92b31b5700 -1 mon.smithi027@1(peon) e3 *** Got Signal Terminated *** 2024-04-02T04:59:09.827 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:59:09 smithi027 podman[125022]: 2024-04-02 04:59:09.559869663 +0000 UTC m=+0.117976563 container died fcd68b20dcc5e6d1d630af96bc8cb0ac28513a73371305294bea2f8d6f363a08 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi027, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=squid-6cec793, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326) 2024-04-02T04:59:10.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:59:10 smithi027 podman[125022]: 2024-04-02 04:59:10.022983271 +0000 UTC m=+0.581090170 container cleanup fcd68b20dcc5e6d1d630af96bc8cb0ac28513a73371305294bea2f8d6f363a08 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi027, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-6cec793, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-02T04:59:10.146 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:59:10 smithi027 bash[125022]: ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi027 2024-04-02T04:59:10.439 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Apr 02 04:59:10 smithi027 podman[125036]: 2024-04-02 04:59:10.299799933 +0000 UTC m=+0.742190911 container remove fcd68b20dcc5e6d1d630af96bc8cb0ac28513a73371305294bea2f8d6f363a08 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-mon-smithi027, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-6cec793, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True) 2024-04-02T04:59:10.457 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@mon.smithi027.service' 2024-04-02T04:59:10.508 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T04:59:10.508 INFO:tasks.cephadm.mon.smithi027:Stopped mon.smithi027 2024-04-02T04:59:10.508 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-02T04:59:10.508 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@osd.0 2024-04-02T04:59:10.810 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:59:10 smithi016 systemd[1]: Stopping Ceph osd.0 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4... 2024-04-02T04:59:11.310 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:59:10 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0[102237]: 2024-04-02T04:59:10.812+0000 7f8394b86700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T04:59:11.310 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:59:10 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0[102237]: 2024-04-02T04:59:10.812+0000 7f8394b86700 -1 osd.0 84 *** Got signal Terminated *** 2024-04-02T04:59:11.311 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:59:10 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0[102237]: 2024-04-02T04:59:10.812+0000 7f8394b86700 -1 osd.0 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T04:59:16.458 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:59:16 smithi016 podman[153965]: 2024-04-02 04:59:16.187952504 +0000 UTC m=+5.490330414 container died 421d9e1cc751685a0849fcb4c1d0592e6671836c7a718a8bc6731ce954bdaaef (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-6cec793, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-02T04:59:16.783 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:59:16 smithi016 podman[153965]: 2024-04-02 04:59:16.537545759 +0000 UTC m=+5.839923664 container cleanup 421d9e1cc751685a0849fcb4c1d0592e6671836c7a718a8bc6731ce954bdaaef (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, RELEASE=squid-6cec793, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD) 2024-04-02T04:59:16.784 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:59:16 smithi016 bash[153965]: ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0 2024-04-02T04:59:17.060 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:59:16 smithi016 podman[153983]: 2024-04-02 04:59:16.780711139 +0000 UTC m=+0.589697882 container remove 421d9e1cc751685a0849fcb4c1d0592e6671836c7a718a8bc6731ce954bdaaef (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-6cec793, ceph=True) 2024-04-02T04:59:17.540 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:59:17 smithi016 podman[154073]: 2024-04-02 04:59:17.139013897 +0000 UTC m=+0.019913877 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:59:17.540 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:59:17 smithi016 podman[154073]: 2024-04-02 04:59:17.265359518 +0000 UTC m=+0.146259473 container create 825827a20d283f10f0afb0daadd0656389cacf2efc6853d8668e626a65038286 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0-deactivate, ceph=True, RELEASE=squid-6cec793, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS) 2024-04-02T04:59:17.810 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:59:17 smithi016 podman[154073]: 2024-04-02 04:59:17.536859638 +0000 UTC m=+0.417759594 container init 825827a20d283f10f0afb0daadd0656389cacf2efc6853d8668e626a65038286 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0-deactivate, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-6cec793, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T04:59:17.810 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:59:17 smithi016 podman[154073]: 2024-04-02 04:59:17.540152991 +0000 UTC m=+0.421052954 container start 825827a20d283f10f0afb0daadd0656389cacf2efc6853d8668e626a65038286 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, RELEASE=squid-6cec793, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-02T04:59:17.811 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:59:17 smithi016 podman[154073]: 2024-04-02 04:59:17.628726092 +0000 UTC m=+0.509626052 container attach 825827a20d283f10f0afb0daadd0656389cacf2efc6853d8668e626a65038286 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-6cec793, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T04:59:18.310 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:59:17 smithi016 podman[154073]: 2024-04-02 04:59:17.841937093 +0000 UTC m=+0.722837065 container died 825827a20d283f10f0afb0daadd0656389cacf2efc6853d8668e626a65038286 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-6cec793, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-02T04:59:18.605 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@osd.0.service' 2024-04-02T04:59:18.630 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:59:18 smithi016 podman[154099]: 2024-04-02 04:59:18.582778071 +0000 UTC m=+0.730882866 container remove 825827a20d283f10f0afb0daadd0656389cacf2efc6853d8668e626a65038286 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=squid-6cec793, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T04:59:18.630 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:59:18 smithi016 systemd[1]: ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@osd.0.service: Deactivated successfully. 2024-04-02T04:59:18.630 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:59:18 smithi016 systemd[1]: Stopped Ceph osd.0 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4. 2024-04-02T04:59:18.630 INFO:journalctl@ceph.osd.0.smithi016.stdout:Apr 02 04:59:18 smithi016 systemd[1]: ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@osd.0.service: Consumed 8.351s CPU time. 2024-04-02T04:59:18.646 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T04:59:18.647 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-02T04:59:18.647 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-02T04:59:18.647 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@osd.1 2024-04-02T04:59:18.906 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:59:18 smithi016 systemd[1]: Stopping Ceph osd.1 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4... 2024-04-02T04:59:19.310 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:59:19 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1[108244]: 2024-04-02T04:59:19.019+0000 7ff4989ad700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T04:59:19.310 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:59:19 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1[108244]: 2024-04-02T04:59:19.019+0000 7ff4989ad700 -1 osd.1 84 *** Got signal Terminated *** 2024-04-02T04:59:19.310 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:59:19 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1[108244]: 2024-04-02T04:59:19.019+0000 7ff4989ad700 -1 osd.1 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T04:59:24.797 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:59:24 smithi016 podman[154178]: 2024-04-02 04:59:24.490266364 +0000 UTC m=+5.597021313 container died 3612a56236a3d6e0438927443f396a9af27ea390e7218efc062ed7d9105b9828 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-6cec793, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-02T04:59:24.798 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:59:24 smithi016 podman[154178]: 2024-04-02 04:59:24.794467253 +0000 UTC m=+5.901222211 container cleanup 3612a56236a3d6e0438927443f396a9af27ea390e7218efc062ed7d9105b9828 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-6cec793, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326) 2024-04-02T04:59:25.060 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:59:24 smithi016 bash[154178]: ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1 2024-04-02T04:59:25.060 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:59:25 smithi016 podman[154206]: 2024-04-02 04:59:25.029400103 +0000 UTC m=+0.538963663 container remove 3612a56236a3d6e0438927443f396a9af27ea390e7218efc062ed7d9105b9828 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-6cec793, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T04:59:25.753 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:59:25 smithi016 podman[154297]: 2024-04-02 04:59:25.273786773 +0000 UTC m=+0.018150351 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:59:25.753 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:59:25 smithi016 podman[154297]: 2024-04-02 04:59:25.372199297 +0000 UTC m=+0.116562853 container create 6b8dbf7695b4bdc3ead289a0ffcfd948c419d6fe383b6f2cca25746e5634dd19 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, RELEASE=squid-6cec793, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-02T04:59:25.753 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:59:25 smithi016 podman[154297]: 2024-04-02 04:59:25.749763556 +0000 UTC m=+0.494127112 container init 6b8dbf7695b4bdc3ead289a0ffcfd948c419d6fe383b6f2cca25746e5634dd19 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-6cec793, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-02T04:59:26.056 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:59:25 smithi016 podman[154297]: 2024-04-02 04:59:25.753202249 +0000 UTC m=+0.497565809 container start 6b8dbf7695b4bdc3ead289a0ffcfd948c419d6fe383b6f2cca25746e5634dd19 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-6cec793) 2024-04-02T04:59:26.056 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:59:25 smithi016 podman[154297]: 2024-04-02 04:59:25.833129052 +0000 UTC m=+0.577492631 container attach 6b8dbf7695b4bdc3ead289a0ffcfd948c419d6fe383b6f2cca25746e5634dd19 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-6cec793, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-02T04:59:26.310 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 02 04:59:26 smithi016 podman[154297]: 2024-04-02 04:59:26.052880602 +0000 UTC m=+0.797244164 container died 6b8dbf7695b4bdc3ead289a0ffcfd948c419d6fe383b6f2cca25746e5634dd19 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-1-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-6cec793, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-02T04:59:26.624 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@osd.1.service' 2024-04-02T04:59:26.666 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T04:59:26.666 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-02T04:59:26.666 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-02T04:59:26.666 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@osd.2 2024-04-02T04:59:27.044 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:59:26 smithi016 systemd[1]: Stopping Ceph osd.2 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4... 2024-04-02T04:59:27.310 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:59:27 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2[113667]: 2024-04-02T04:59:27.040+0000 7f02bd3de700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T04:59:27.310 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:59:27 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2[113667]: 2024-04-02T04:59:27.040+0000 7f02bd3de700 -1 osd.2 84 *** Got signal Terminated *** 2024-04-02T04:59:27.310 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:59:27 smithi016 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2[113667]: 2024-04-02T04:59:27.040+0000 7f02bd3de700 -1 osd.2 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T04:59:32.769 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:59:32 smithi016 podman[154402]: 2024-04-02 04:59:32.375517723 +0000 UTC m=+5.469742475 container died a8e9b8d0303c9ac8dd63db1cc767a668e0668b8847e1fafecf767a56b17c42cd (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2, org.label-schema.build-date=20240326, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-6cec793, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-02T04:59:33.060 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:59:32 smithi016 podman[154402]: 2024-04-02 04:59:32.85694766 +0000 UTC m=+5.951172422 container cleanup a8e9b8d0303c9ac8dd63db1cc767a668e0668b8847e1fafecf767a56b17c42cd (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-6cec793, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-04-02T04:59:33.060 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:59:32 smithi016 bash[154402]: ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2 2024-04-02T04:59:33.509 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:59:33 smithi016 podman[154420]: 2024-04-02 04:59:33.125142748 +0000 UTC m=+0.746122714 container remove a8e9b8d0303c9ac8dd63db1cc767a668e0668b8847e1fafecf767a56b17c42cd (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-6cec793, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-02T04:59:33.764 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:59:33 smithi016 podman[154511]: 2024-04-02 04:59:33.409443516 +0000 UTC m=+0.019379514 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:59:33.764 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:59:33 smithi016 podman[154511]: 2024-04-02 04:59:33.660131074 +0000 UTC m=+0.270067043 container create b2bde80265bc398c42a5a6c89b636c98fa0359ec006e2ad287bb77facb89ad94 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-6cec793, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T04:59:34.060 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:59:33 smithi016 podman[154511]: 2024-04-02 04:59:33.896347962 +0000 UTC m=+0.506283943 container init b2bde80265bc398c42a5a6c89b636c98fa0359ec006e2ad287bb77facb89ad94 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2-deactivate, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-6cec793, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-02T04:59:34.060 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:59:33 smithi016 podman[154511]: 2024-04-02 04:59:33.900056269 +0000 UTC m=+0.509992245 container start b2bde80265bc398c42a5a6c89b636c98fa0359ec006e2ad287bb77facb89ad94 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-6cec793, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-02T04:59:34.061 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:59:34 smithi016 podman[154511]: 2024-04-02 04:59:34.00443379 +0000 UTC m=+0.614369768 container attach b2bde80265bc398c42a5a6c89b636c98fa0359ec006e2ad287bb77facb89ad94 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-6cec793, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T04:59:34.560 INFO:journalctl@ceph.osd.2.smithi016.stdout:Apr 02 04:59:34 smithi016 podman[154511]: 2024-04-02 04:59:34.235305291 +0000 UTC m=+0.845241271 container died b2bde80265bc398c42a5a6c89b636c98fa0359ec006e2ad287bb77facb89ad94 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-6cec793, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.build-date=20240326) 2024-04-02T04:59:34.814 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@osd.2.service' 2024-04-02T04:59:34.855 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T04:59:34.855 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-02T04:59:34.855 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-02T04:59:34.856 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@osd.3 2024-04-02T04:59:35.146 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:59:34 smithi027 systemd[1]: Stopping Ceph osd.3 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4... 2024-04-02T04:59:35.646 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:59:35 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3[93204]: 2024-04-02T04:59:35.285+0000 7f8d543cb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T04:59:35.647 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:59:35 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3[93204]: 2024-04-02T04:59:35.285+0000 7f8d543cb700 -1 osd.3 84 *** Got signal Terminated *** 2024-04-02T04:59:35.647 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:59:35 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3[93204]: 2024-04-02T04:59:35.285+0000 7f8d543cb700 -1 osd.3 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T04:59:37.646 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:59:37 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4[98165]: 2024-04-02T04:59:37.146+0000 7f5e6ff47700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-04-02T04:59:13.669168+0000 front 2024-04-02T04:59:13.669015+0000 (oldest deadline 2024-04-02T04:59:37.069716+0000) 2024-04-02T04:59:38.396 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:59:38 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4[98165]: 2024-04-02T04:59:38.143+0000 7f5e6ff47700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-04-02T04:59:13.669168+0000 front 2024-04-02T04:59:13.669015+0000 (oldest deadline 2024-04-02T04:59:37.069716+0000) 2024-04-02T04:59:39.396 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:59:39 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4[98165]: 2024-04-02T04:59:39.143+0000 7f5e6ff47700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-04-02T04:59:13.669168+0000 front 2024-04-02T04:59:13.669015+0000 (oldest deadline 2024-04-02T04:59:37.069716+0000) 2024-04-02T04:59:39.397 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:39 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[103101]: 2024-04-02T04:59:39.096+0000 7fd117bd2700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-04-02T04:59:15.206745+0000 front 2024-04-02T04:59:15.206672+0000 (oldest deadline 2024-04-02T04:59:38.706848+0000) 2024-04-02T04:59:40.396 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:59:40 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4[98165]: 2024-04-02T04:59:40.185+0000 7f5e6ff47700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-04-02T04:59:13.669168+0000 front 2024-04-02T04:59:13.669015+0000 (oldest deadline 2024-04-02T04:59:37.069716+0000) 2024-04-02T04:59:40.397 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:40 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[103101]: 2024-04-02T04:59:40.143+0000 7fd117bd2700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-04-02T04:59:15.206745+0000 front 2024-04-02T04:59:15.206672+0000 (oldest deadline 2024-04-02T04:59:38.706848+0000) 2024-04-02T04:59:40.896 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:59:40 smithi027 podman[125157]: 2024-04-02 04:59:40.691519997 +0000 UTC m=+5.636159631 container died 0dd13b5be8c048af76377f18225b037644f2f03f37e11ee29d6ceef0a43a66d3 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-6cec793, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240326) 2024-04-02T04:59:41.208 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:41 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[103101]: 2024-04-02T04:59:41.179+0000 7fd117bd2700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-04-02T04:59:15.206745+0000 front 2024-04-02T04:59:15.206672+0000 (oldest deadline 2024-04-02T04:59:38.706848+0000) 2024-04-02T04:59:41.208 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:59:41 smithi027 podman[125157]: 2024-04-02 04:59:41.018717172 +0000 UTC m=+5.963356801 container cleanup 0dd13b5be8c048af76377f18225b037644f2f03f37e11ee29d6ceef0a43a66d3 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-6cec793) 2024-04-02T04:59:41.208 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:59:41 smithi027 bash[125157]: ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3 2024-04-02T04:59:41.595 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:59:41 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4[98165]: 2024-04-02T04:59:41.204+0000 7f5e6ff47700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-04-02T04:59:13.669168+0000 front 2024-04-02T04:59:13.669015+0000 (oldest deadline 2024-04-02T04:59:37.069716+0000) 2024-04-02T04:59:41.596 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:59:41 smithi027 podman[125178]: 2024-04-02 04:59:41.253671282 +0000 UTC m=+0.560383576 container remove 0dd13b5be8c048af76377f18225b037644f2f03f37e11ee29d6ceef0a43a66d3 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3, GIT_CLEAN=True, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-6cec793, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T04:59:41.864 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:59:41 smithi027 podman[125276]: 2024-04-02 04:59:41.621918773 +0000 UTC m=+0.018260817 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:59:41.864 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:59:41 smithi027 podman[125276]: 2024-04-02 04:59:41.74653733 +0000 UTC m=+0.142879395 container create b3e3dc8f13a0981ce24d55232f3f6933dab331a10b42c0bcdfd62e9925e18952 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-6cec793, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-02T04:59:42.146 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:59:41 smithi027 podman[125276]: 2024-04-02 04:59:41.974214645 +0000 UTC m=+0.370556674 container init b3e3dc8f13a0981ce24d55232f3f6933dab331a10b42c0bcdfd62e9925e18952 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, RELEASE=squid-6cec793, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T04:59:42.147 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:59:41 smithi027 podman[125276]: 2024-04-02 04:59:41.977423428 +0000 UTC m=+0.373765450 container start b3e3dc8f13a0981ce24d55232f3f6933dab331a10b42c0bcdfd62e9925e18952 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=squid-6cec793, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-02T04:59:42.147 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:59:42 smithi027 podman[125276]: 2024-04-02 04:59:42.040566043 +0000 UTC m=+0.436908077 container attach b3e3dc8f13a0981ce24d55232f3f6933dab331a10b42c0bcdfd62e9925e18952 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3-deactivate, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-6cec793, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, ceph=True) 2024-04-02T04:59:42.630 INFO:journalctl@ceph.osd.3.smithi027.stdout:Apr 02 04:59:42 smithi027 podman[125276]: 2024-04-02 04:59:42.284454328 +0000 UTC m=+0.680796341 container died b3e3dc8f13a0981ce24d55232f3f6933dab331a10b42c0bcdfd62e9925e18952 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-6cec793) 2024-04-02T04:59:42.630 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:42 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[103101]: 2024-04-02T04:59:42.207+0000 7fd117bd2700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-04-02T04:59:15.206745+0000 front 2024-04-02T04:59:15.206672+0000 (oldest deadline 2024-04-02T04:59:38.706848+0000) 2024-04-02T04:59:42.630 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:59:42 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4[98165]: 2024-04-02T04:59:42.229+0000 7f5e6ff47700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-04-02T04:59:13.669168+0000 front 2024-04-02T04:59:13.669015+0000 (oldest deadline 2024-04-02T04:59:37.069716+0000) 2024-04-02T04:59:42.943 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@osd.3.service' 2024-04-02T04:59:42.989 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T04:59:42.989 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-02T04:59:42.989 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-02T04:59:42.989 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@osd.4 2024-04-02T04:59:43.253 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:59:43 smithi027 systemd[1]: Stopping Ceph osd.4 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4... 2024-04-02T04:59:43.615 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:43 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[103101]: 2024-04-02T04:59:43.249+0000 7fd117bd2700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-04-02T04:59:15.206745+0000 front 2024-04-02T04:59:15.206672+0000 (oldest deadline 2024-04-02T04:59:38.706848+0000) 2024-04-02T04:59:43.616 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:59:43 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4[98165]: 2024-04-02T04:59:43.262+0000 7f5e6ff47700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-04-02T04:59:13.669168+0000 front 2024-04-02T04:59:13.669015+0000 (oldest deadline 2024-04-02T04:59:37.069716+0000) 2024-04-02T04:59:43.616 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:59:43 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4[98165]: 2024-04-02T04:59:43.296+0000 7f5e7715a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T04:59:43.616 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:59:43 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4[98165]: 2024-04-02T04:59:43.296+0000 7f5e7715a700 -1 osd.4 84 *** Got signal Terminated *** 2024-04-02T04:59:43.616 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:59:43 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4[98165]: 2024-04-02T04:59:43.296+0000 7f5e7715a700 -1 osd.4 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T04:59:44.646 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:59:44 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4[98165]: 2024-04-02T04:59:44.240+0000 7f5e6ff47700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-04-02T04:59:13.669168+0000 front 2024-04-02T04:59:13.669015+0000 (oldest deadline 2024-04-02T04:59:37.069716+0000) 2024-04-02T04:59:44.647 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:44 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[103101]: 2024-04-02T04:59:44.205+0000 7fd117bd2700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-04-02T04:59:15.206745+0000 front 2024-04-02T04:59:15.206672+0000 (oldest deadline 2024-04-02T04:59:38.706848+0000) 2024-04-02T04:59:45.646 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:59:45 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4[98165]: 2024-04-02T04:59:45.243+0000 7f5e6ff47700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-04-02T04:59:13.669168+0000 front 2024-04-02T04:59:13.669015+0000 (oldest deadline 2024-04-02T04:59:37.069716+0000) 2024-04-02T04:59:45.646 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:59:45 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4[98165]: 2024-04-02T04:59:45.243+0000 7f5e6ff47700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.1 since back 2024-04-02T04:59:22.970669+0000 front 2024-04-02T04:59:22.970773+0000 (oldest deadline 2024-04-02T04:59:44.670767+0000) 2024-04-02T04:59:45.647 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:45 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[103101]: 2024-04-02T04:59:45.255+0000 7fd117bd2700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-04-02T04:59:15.206745+0000 front 2024-04-02T04:59:15.206672+0000 (oldest deadline 2024-04-02T04:59:38.706848+0000) 2024-04-02T04:59:45.647 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:45 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[103101]: 2024-04-02T04:59:45.255+0000 7fd117bd2700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.1 since back 2024-04-02T04:59:24.007841+0000 front 2024-04-02T04:59:24.007783+0000 (oldest deadline 2024-04-02T04:59:45.107728+0000) 2024-04-02T04:59:46.646 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:59:46 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4[98165]: 2024-04-02T04:59:46.228+0000 7f5e6ff47700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-04-02T04:59:13.669168+0000 front 2024-04-02T04:59:13.669015+0000 (oldest deadline 2024-04-02T04:59:37.069716+0000) 2024-04-02T04:59:46.647 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:59:46 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4[98165]: 2024-04-02T04:59:46.228+0000 7f5e6ff47700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.1 since back 2024-04-02T04:59:22.970669+0000 front 2024-04-02T04:59:22.970773+0000 (oldest deadline 2024-04-02T04:59:44.670767+0000) 2024-04-02T04:59:46.647 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:46 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[103101]: 2024-04-02T04:59:46.221+0000 7fd117bd2700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-04-02T04:59:15.206745+0000 front 2024-04-02T04:59:15.206672+0000 (oldest deadline 2024-04-02T04:59:38.706848+0000) 2024-04-02T04:59:46.647 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:46 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[103101]: 2024-04-02T04:59:46.221+0000 7fd117bd2700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.1 since back 2024-04-02T04:59:24.007841+0000 front 2024-04-02T04:59:24.007783+0000 (oldest deadline 2024-04-02T04:59:45.107728+0000) 2024-04-02T04:59:47.646 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:47 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[103101]: 2024-04-02T04:59:47.230+0000 7fd117bd2700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-04-02T04:59:15.206745+0000 front 2024-04-02T04:59:15.206672+0000 (oldest deadline 2024-04-02T04:59:38.706848+0000) 2024-04-02T04:59:47.647 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:47 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[103101]: 2024-04-02T04:59:47.230+0000 7fd117bd2700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.1 since back 2024-04-02T04:59:24.007841+0000 front 2024-04-02T04:59:24.007783+0000 (oldest deadline 2024-04-02T04:59:45.107728+0000) 2024-04-02T04:59:47.647 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:59:47 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4[98165]: 2024-04-02T04:59:47.245+0000 7f5e6ff47700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-04-02T04:59:13.669168+0000 front 2024-04-02T04:59:13.669015+0000 (oldest deadline 2024-04-02T04:59:37.069716+0000) 2024-04-02T04:59:47.647 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:59:47 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4[98165]: 2024-04-02T04:59:47.245+0000 7f5e6ff47700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.1 since back 2024-04-02T04:59:22.970669+0000 front 2024-04-02T04:59:22.970773+0000 (oldest deadline 2024-04-02T04:59:44.670767+0000) 2024-04-02T04:59:48.646 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:59:48 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4[98165]: 2024-04-02T04:59:48.294+0000 7f5e6ff47700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-04-02T04:59:13.669168+0000 front 2024-04-02T04:59:13.669015+0000 (oldest deadline 2024-04-02T04:59:37.069716+0000) 2024-04-02T04:59:48.646 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:59:48 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4[98165]: 2024-04-02T04:59:48.294+0000 7f5e6ff47700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.1 since back 2024-04-02T04:59:22.970669+0000 front 2024-04-02T04:59:22.970773+0000 (oldest deadline 2024-04-02T04:59:44.670767+0000) 2024-04-02T04:59:48.646 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:48 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[103101]: 2024-04-02T04:59:48.233+0000 7fd117bd2700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-04-02T04:59:15.206745+0000 front 2024-04-02T04:59:15.206672+0000 (oldest deadline 2024-04-02T04:59:38.706848+0000) 2024-04-02T04:59:48.646 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:48 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[103101]: 2024-04-02T04:59:48.233+0000 7fd117bd2700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.1 since back 2024-04-02T04:59:24.007841+0000 front 2024-04-02T04:59:24.007783+0000 (oldest deadline 2024-04-02T04:59:45.107728+0000) 2024-04-02T04:59:48.896 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:59:48 smithi027 podman[125383]: 2024-04-02 04:59:48.686251538 +0000 UTC m=+5.513380394 container died 9025eacf8145aaf2fee8b614698330e23898a09bd8fadebcb47768b3936a1fe7 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=squid-6cec793, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-02T04:59:49.203 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:59:49 smithi027 podman[125383]: 2024-04-02 04:59:49.0058339 +0000 UTC m=+5.832962747 container cleanup 9025eacf8145aaf2fee8b614698330e23898a09bd8fadebcb47768b3936a1fe7 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4, RELEASE=squid-6cec793, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-04-02T04:59:49.203 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:59:49 smithi027 bash[125383]: ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4 2024-04-02T04:59:49.558 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:59:49 smithi027 podman[125401]: 2024-04-02 04:59:49.235278118 +0000 UTC m=+0.538808941 container remove 9025eacf8145aaf2fee8b614698330e23898a09bd8fadebcb47768b3936a1fe7 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-6cec793, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-02T04:59:49.558 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:49 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[103101]: 2024-04-02T04:59:49.199+0000 7fd117bd2700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-04-02T04:59:15.206745+0000 front 2024-04-02T04:59:15.206672+0000 (oldest deadline 2024-04-02T04:59:38.706848+0000) 2024-04-02T04:59:49.558 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:49 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[103101]: 2024-04-02T04:59:49.199+0000 7fd117bd2700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.1 since back 2024-04-02T04:59:24.007841+0000 front 2024-04-02T04:59:24.007783+0000 (oldest deadline 2024-04-02T04:59:45.107728+0000) 2024-04-02T04:59:49.814 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:59:49 smithi027 podman[125494]: 2024-04-02 04:59:49.457959292 +0000 UTC m=+0.018397989 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:59:49.814 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:59:49 smithi027 podman[125494]: 2024-04-02 04:59:49.566478696 +0000 UTC m=+0.126917419 container create 11c8fd7e053bb78ce3a92d7f4ee2a8cfc8884f801d6323f9ce358415135f8447 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-6cec793, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0) 2024-04-02T04:59:49.814 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:59:49 smithi027 podman[125494]: 2024-04-02 04:59:49.811429058 +0000 UTC m=+0.371867746 container init 11c8fd7e053bb78ce3a92d7f4ee2a8cfc8884f801d6323f9ce358415135f8447 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-6cec793, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0) 2024-04-02T04:59:50.130 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:59:49 smithi027 podman[125494]: 2024-04-02 04:59:49.8149317 +0000 UTC m=+0.375370378 container start 11c8fd7e053bb78ce3a92d7f4ee2a8cfc8884f801d6323f9ce358415135f8447 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, RELEASE=squid-6cec793, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True) 2024-04-02T04:59:50.130 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:59:49 smithi027 podman[125494]: 2024-04-02 04:59:49.90236741 +0000 UTC m=+0.462806119 container attach 11c8fd7e053bb78ce3a92d7f4ee2a8cfc8884f801d6323f9ce358415135f8447 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4-deactivate, RELEASE=squid-6cec793, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-04-02T04:59:50.396 INFO:journalctl@ceph.osd.4.smithi027.stdout:Apr 02 04:59:50 smithi027 podman[125494]: 2024-04-02 04:59:50.127414807 +0000 UTC m=+0.687853501 container died 11c8fd7e053bb78ce3a92d7f4ee2a8cfc8884f801d6323f9ce358415135f8447 (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, RELEASE=squid-6cec793, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-02T04:59:50.396 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:50 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[103101]: 2024-04-02T04:59:50.175+0000 7fd117bd2700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-04-02T04:59:15.206745+0000 front 2024-04-02T04:59:15.206672+0000 (oldest deadline 2024-04-02T04:59:38.706848+0000) 2024-04-02T04:59:50.396 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:50 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[103101]: 2024-04-02T04:59:50.175+0000 7fd117bd2700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.1 since back 2024-04-02T04:59:24.007841+0000 front 2024-04-02T04:59:24.007783+0000 (oldest deadline 2024-04-02T04:59:45.107728+0000) 2024-04-02T04:59:50.762 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@osd.4.service' 2024-04-02T04:59:50.810 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T04:59:50.810 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-02T04:59:50.811 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-02T04:59:50.811 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@osd.5 2024-04-02T04:59:51.031 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:50 smithi027 systemd[1]: Stopping Ceph osd.5 for dc5403b4-f0a9-11ee-b647-cb9ed24678a4... 2024-04-02T04:59:51.396 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:51 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[103101]: 2024-04-02T04:59:51.124+0000 7fd11addd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T04:59:51.397 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:51 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[103101]: 2024-04-02T04:59:51.124+0000 7fd11addd700 -1 osd.5 84 *** Got signal Terminated *** 2024-04-02T04:59:51.397 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:51 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[103101]: 2024-04-02T04:59:51.124+0000 7fd11addd700 -1 osd.5 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T04:59:51.397 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:51 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[103101]: 2024-04-02T04:59:51.209+0000 7fd117bd2700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-04-02T04:59:15.206745+0000 front 2024-04-02T04:59:15.206672+0000 (oldest deadline 2024-04-02T04:59:38.706848+0000) 2024-04-02T04:59:51.397 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:51 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[103101]: 2024-04-02T04:59:51.209+0000 7fd117bd2700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.1 since back 2024-04-02T04:59:24.007841+0000 front 2024-04-02T04:59:24.007783+0000 (oldest deadline 2024-04-02T04:59:45.107728+0000) 2024-04-02T04:59:52.646 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:52 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[103101]: 2024-04-02T04:59:52.169+0000 7fd117bd2700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-04-02T04:59:15.206745+0000 front 2024-04-02T04:59:15.206672+0000 (oldest deadline 2024-04-02T04:59:38.706848+0000) 2024-04-02T04:59:52.647 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:52 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[103101]: 2024-04-02T04:59:52.169+0000 7fd117bd2700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.1 since back 2024-04-02T04:59:24.007841+0000 front 2024-04-02T04:59:24.007783+0000 (oldest deadline 2024-04-02T04:59:45.107728+0000) 2024-04-02T04:59:53.396 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:53 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[103101]: 2024-04-02T04:59:53.122+0000 7fd117bd2700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-04-02T04:59:15.206745+0000 front 2024-04-02T04:59:15.206672+0000 (oldest deadline 2024-04-02T04:59:38.706848+0000) 2024-04-02T04:59:53.396 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:53 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[103101]: 2024-04-02T04:59:53.122+0000 7fd117bd2700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.1 since back 2024-04-02T04:59:24.007841+0000 front 2024-04-02T04:59:24.007783+0000 (oldest deadline 2024-04-02T04:59:45.107728+0000) 2024-04-02T04:59:54.396 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:54 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[103101]: 2024-04-02T04:59:54.090+0000 7fd117bd2700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-04-02T04:59:15.206745+0000 front 2024-04-02T04:59:15.206672+0000 (oldest deadline 2024-04-02T04:59:38.706848+0000) 2024-04-02T04:59:54.396 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:54 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[103101]: 2024-04-02T04:59:54.090+0000 7fd117bd2700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.1 since back 2024-04-02T04:59:24.007841+0000 front 2024-04-02T04:59:24.007783+0000 (oldest deadline 2024-04-02T04:59:45.107728+0000) 2024-04-02T04:59:55.396 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:55 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[103101]: 2024-04-02T04:59:55.077+0000 7fd117bd2700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-04-02T04:59:15.206745+0000 front 2024-04-02T04:59:15.206672+0000 (oldest deadline 2024-04-02T04:59:38.706848+0000) 2024-04-02T04:59:55.397 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:55 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[103101]: 2024-04-02T04:59:55.077+0000 7fd117bd2700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.1 since back 2024-04-02T04:59:24.007841+0000 front 2024-04-02T04:59:24.007783+0000 (oldest deadline 2024-04-02T04:59:45.107728+0000) 2024-04-02T04:59:56.396 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:56 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[103101]: 2024-04-02T04:59:56.053+0000 7fd117bd2700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-04-02T04:59:15.206745+0000 front 2024-04-02T04:59:15.206672+0000 (oldest deadline 2024-04-02T04:59:38.706848+0000) 2024-04-02T04:59:56.396 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:56 smithi027 ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5[103101]: 2024-04-02T04:59:56.053+0000 7fd117bd2700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.16:6814 osd.1 since back 2024-04-02T04:59:24.007841+0000 front 2024-04-02T04:59:24.007783+0000 (oldest deadline 2024-04-02T04:59:45.107728+0000) 2024-04-02T04:59:56.855 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:56 smithi027 podman[125600]: 2024-04-02 04:59:56.578578743 +0000 UTC m=+5.579526254 container died c2787f15737767bbcdb2ea71c8b69b5549819535c8237f3ad2c2b677738519ba (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, RELEASE=squid-6cec793, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-02T04:59:57.146 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:56 smithi027 podman[125600]: 2024-04-02 04:59:56.950382048 +0000 UTC m=+5.951329558 container cleanup c2787f15737767bbcdb2ea71c8b69b5549819535c8237f3ad2c2b677738519ba (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-6cec793, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-02T04:59:57.147 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:56 smithi027 bash[125600]: ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5 2024-04-02T04:59:57.578 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:57 smithi027 podman[125617]: 2024-04-02 04:59:57.226828914 +0000 UTC m=+0.645444430 container remove c2787f15737767bbcdb2ea71c8b69b5549819535c8237f3ad2c2b677738519ba (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-6cec793, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T04:59:57.836 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:57 smithi027 podman[125709]: 2024-04-02 04:59:57.478462957 +0000 UTC m=+0.019250480 image pull 78f830190839e38cac46d0f24741058bd5f6b4363a488caf8a9e060d9857e808 quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae 2024-04-02T04:59:57.837 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:57 smithi027 podman[125709]: 2024-04-02 04:59:57.594695372 +0000 UTC m=+0.135482938 container create 04993b70c47132cb5747ee39e114e797e7fcad2f07664b34aa027c29b883b11d (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, RELEASE=squid-6cec793, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5) 2024-04-02T04:59:58.146 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:57 smithi027 podman[125709]: 2024-04-02 04:59:57.947135329 +0000 UTC m=+0.487922832 container init 04993b70c47132cb5747ee39e114e797e7fcad2f07664b34aa027c29b883b11d (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-6cec793, ceph=True) 2024-04-02T04:59:58.146 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:57 smithi027 podman[125709]: 2024-04-02 04:59:57.950359103 +0000 UTC m=+0.491146609 container start 04993b70c47132cb5747ee39e114e797e7fcad2f07664b34aa027c29b883b11d (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-6cec793, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-02T04:59:58.147 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:58 smithi027 podman[125709]: 2024-04-02 04:59:58.01363759 +0000 UTC m=+0.554425094 container attach 04993b70c47132cb5747ee39e114e797e7fcad2f07664b34aa027c29b883b11d (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5-deactivate, RELEASE=squid-6cec793, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326) 2024-04-02T04:59:58.544 INFO:journalctl@ceph.osd.5.smithi027.stdout:Apr 02 04:59:58 smithi027 podman[125709]: 2024-04-02 04:59:58.251793339 +0000 UTC m=+0.792580842 container died 04993b70c47132cb5747ee39e114e797e7fcad2f07664b34aa027c29b883b11d (image=quay.ceph.io/ceph-ci/ceph@sha256:7bd5b84926d4a0d6c51e5562e965bff211cec2740c9f4dec9b5e00a2bbb75eae, name=ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4-osd-5-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-6cec793) 2024-04-02T04:59:58.869 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc5403b4-f0a9-11ee-b647-cb9ed24678a4@osd.5.service' 2024-04-02T04:59:58.914 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T04:59:58.914 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-02T04:59:58.915 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 --force --keep-logs 2024-04-02T04:59:59.069 INFO:teuthology.orchestra.run.smithi016.stdout:Deleting cluster with fsid: dc5403b4-f0a9-11ee-b647-cb9ed24678a4 2024-04-02T05:00:03.212 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[76979]: fuse finished with error 0 and tester_r 0 2024-04-02T05:00:21.089 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 --force --keep-logs 2024-04-02T05:00:21.258 INFO:teuthology.orchestra.run.smithi027.stdout:Deleting cluster with fsid: dc5403b4-f0a9-11ee-b647-cb9ed24678a4 2024-04-02T05:00:37.028 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-02T05:00:37.062 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-02T05:00:37.095 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-02T05:00:37.097 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/crash to /home/teuthworker/archive/teuthology-2024-04-01_21:08:08-orch-squid-distro-default-smithi/7634838/remote/smithi016/crash 2024-04-02T05:00:37.098 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/crash -- . 2024-04-02T05:00:37.130 INFO:teuthology.orchestra.run.smithi016.stderr:tar: /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/crash: Cannot open: No such file or directory 2024-04-02T05:00:37.130 INFO:teuthology.orchestra.run.smithi016.stderr:tar: Error is not recoverable: exiting now 2024-04-02T05:00:37.132 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/crash to /home/teuthworker/archive/teuthology-2024-04-01_21:08:08-orch-squid-distro-default-smithi/7634838/remote/smithi027/crash 2024-04-02T05:00:37.133 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/crash -- . 2024-04-02T05:00:37.165 INFO:teuthology.orchestra.run.smithi027.stderr:tar: /var/lib/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/crash: Cannot open: No such file or directory 2024-04-02T05:00:37.165 INFO:teuthology.orchestra.run.smithi027.stderr:tar: Error is not recoverable: exiting now 2024-04-02T05:00:37.166 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-02T05:00:37.167 DEBUG:teuthology.orchestra.run.smithi016:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-04-02T05:00:37.205 INFO:tasks.cephadm:Compressing logs... 2024-04-02T05:00:37.206 DEBUG:teuthology.orchestra.run.smithi016:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-02T05:00:37.249 DEBUG:teuthology.orchestra.run.smithi027:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-02T05:00:37.274 INFO:teuthology.orchestra.run.smithi016.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-02T05:00:37.275 INFO:teuthology.orchestra.run.smithi016.stderr:: No such file or directory 2024-04-02T05:00:37.275 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-04-02T05:00:37.276 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-mgr.smithi016.eqnebx.log 2024-04-02T05:00:37.276 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-mds.cephfs.smithi016.uhgkgg.log 2024-04-02T05:00:37.276 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-mon.smithi016.log 2024-04-02T05:00:37.276 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-mgr.smithi016.eqnebx.log: gzip -5 --verbose -- /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-client.ceph-exporter.smithi016.log 2024-04-02T05:00:37.278 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-mds.cephfs.smithi016.uhgkgg.log: /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-mon.smithi016.log: gzip -5 --verbose -- /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph.audit.log 2024-04-02T05:00:37.278 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-client.ceph-exporter.smithi016.log: gzip -5 --verbose -- /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-mds.cephfs.smithi016.fyzrnm.log 2024-04-02T05:00:37.278 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-osd.1.log 2024-04-02T05:00:37.279 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-mds.cephfs.smithi016.fyzrnm.log: 94.2% -- replaced with /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-client.ceph-exporter.smithi016.log.gz 2024-04-02T05:00:37.280 INFO:teuthology.orchestra.run.smithi027.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-02T05:00:37.280 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-02T05:00:37.280 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph.audit.log 2024-04-02T05:00:37.281 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-mds.cephfs.smithi027.zwrfoo.log 2024-04-02T05:00:37.281 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-osd.4.log 2024-04-02T05:00:37.281 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-mds.cephfs.smithi027.zwrfoo.log: gzip -5 --verbose -- /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-mgr.smithi027.kjzcsc.log 2024-04-02T05:00:37.286 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-client.ceph-exporter.smithi027.log 2024-04-02T05:00:37.286 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-mgr.smithi027.kjzcsc.log: gzip -5 --verbose -- /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-osd.5.log 2024-04-02T05:00:37.287 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-client.ceph-exporter.smithi027.log: gzip -5 --verbose 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-02T05:00:37.287 INFO:teuthology.orchestra.run.smithi027.stderr: -- /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-osd.3.log 2024-04-02T05:00:37.287 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-mon.smithi027.log 2024-04-02T05:00:37.287 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph.cephadm.log 2024-04-02T05:00:37.287 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-mon.smithi027.log: 94.3% -- replaced with /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-client.ceph-exporter.smithi027.log.gz 2024-04-02T05:00:37.287 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- 91.6% -- replaced with /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph.audit.log.gz 2024-04-02T05:00:37.287 INFO:teuthology.orchestra.run.smithi027.stderr: /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph.log 2024-04-02T05:00:37.287 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-volume.log 2024-04-02T05:00:37.287 INFO:teuthology.orchestra.run.smithi027.stderr: 85.8% -- replaced with /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph.cephadm.log.gz 2024-04-02T05:00:37.287 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph.log: gzip -5 --verbose -- /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-mds.cephfs.smithi027.sdublg.log 2024-04-02T05:00:37.291 INFO:teuthology.orchestra.run.smithi016.stderr: 91.4% -- replaced with /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph.audit.log.gz 2024-04-02T05:00:37.291 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-osd.0.log 2024-04-02T05:00:37.291 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph.cephadm.log 2024-04-02T05:00:37.291 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-osd.0.log: 94.4% -- replaced with /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-mds.cephfs.smithi016.fyzrnm.log.gz 2024-04-02T05:00:37.291 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph.log 2024-04-02T05:00:37.292 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph.cephadm.log: 85.8% -- replaced with /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph.cephadm.log.gz 2024-04-02T05:00:37.292 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-osd.2.log 2024-04-02T05:00:37.292 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph.log: 88.5% -- replaced with /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph.log.gz 2024-04-02T05:00:37.292 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-volume.log 2024-04-02T05:00:37.293 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-volume.log: 88.5% -- replaced with /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph.log.gz 2024-04-02T05:00:37.293 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.log 2024-04-02T05:00:37.302 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-osd.2.log: 90.7% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-02T05:00:37.310 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-mds.cephfs.smithi027.sdublg.log: /var/log/ceph/ceph-client.1.log: 91.0% -- replaced with /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-volume.log.gz 2024-04-02T05:00:37.328 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-volume.log: 91.2% -- replaced with /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-volume.log.gz 2024-04-02T05:00:37.331 INFO:teuthology.orchestra.run.smithi027.stderr: 89.2% -- replaced with /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-mgr.smithi027.kjzcsc.log.gz 2024-04-02T05:00:37.710 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-mgr.smithi016.eqnebx.log.gz 2024-04-02T05:00:37.725 INFO:teuthology.orchestra.run.smithi027.stderr: 92.6% -- replaced with /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-mon.smithi027.log.gz 2024-04-02T05:00:38.536 INFO:teuthology.orchestra.run.smithi016.stderr: 90.5% -- replaced with /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-mon.smithi016.log.gz 2024-04-02T05:00:41.676 INFO:teuthology.orchestra.run.smithi016.stderr: 93.7% -- replaced with /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-osd.0.log.gz 2024-04-02T05:00:41.885 INFO:teuthology.orchestra.run.smithi027.stderr: 93.8% -- replaced with /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-osd.5.log.gz 2024-04-02T05:00:42.016 INFO:teuthology.orchestra.run.smithi016.stderr: 93.9% -- replaced with /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-osd.1.log.gz 2024-04-02T05:00:42.232 INFO:teuthology.orchestra.run.smithi027.stderr: 93.7% -- replaced with /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-osd.4.log.gz 2024-04-02T05:00:42.410 INFO:teuthology.orchestra.run.smithi016.stderr: 93.8% -- replaced with /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-osd.2.log.gz 2024-04-02T05:00:43.496 INFO:teuthology.orchestra.run.smithi016.stderr: 94.7% -- replaced with /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-mds.cephfs.smithi016.uhgkgg.log.gz 2024-04-02T05:00:43.989 INFO:teuthology.orchestra.run.smithi027.stderr: 93.8% -- replaced with /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-osd.3.log.gz 2024-04-02T05:00:45.211 INFO:teuthology.orchestra.run.smithi027.stderr: 94.8% -- replaced with /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-mds.cephfs.smithi027.sdublg.log.gz 2024-04-02T05:00:52.079 INFO:teuthology.orchestra.run.smithi027.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-04-02T05:00:52.080 INFO:teuthology.orchestra.run.smithi027.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.1.log.gz 2024-04-02T05:00:53.493 INFO:teuthology.orchestra.run.smithi016.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-04-02T05:00:53.495 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-04-02T05:00:53.495 INFO:teuthology.orchestra.run.smithi016.stderr:real 0m16.234s 2024-04-02T05:00:53.495 INFO:teuthology.orchestra.run.smithi016.stderr:user 0m37.191s 2024-04-02T05:00:53.496 INFO:teuthology.orchestra.run.smithi016.stderr:sys 0m1.173s 2024-04-02T05:02:09.533 INFO:teuthology.orchestra.run.smithi027.stderr: 92.6% -- replaced with /var/log/ceph/dc5403b4-f0a9-11ee-b647-cb9ed24678a4/ceph-mds.cephfs.smithi027.zwrfoo.log.gz 2024-04-02T05:02:09.535 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-04-02T05:02:09.535 INFO:teuthology.orchestra.run.smithi027.stderr:real 1m32.271s 2024-04-02T05:02:09.535 INFO:teuthology.orchestra.run.smithi027.stderr:user 2m8.041s 2024-04-02T05:02:09.535 INFO:teuthology.orchestra.run.smithi027.stderr:sys 0m3.490s 2024-04-02T05:02:09.536 INFO:tasks.cephadm:Archiving logs... 2024-04-02T05:02:09.537 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-01_21:08:08-orch-squid-distro-default-smithi/7634838/remote/smithi016/log 2024-04-02T05:02:09.538 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-02T05:02:14.397 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-01_21:08:08-orch-squid-distro-default-smithi/7634838/remote/smithi027/log 2024-04-02T05:02:14.398 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-02T05:02:33.148 INFO:tasks.cephadm:Removing cluster... 2024-04-02T05:02:33.148 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 --force 2024-04-02T05:02:33.308 INFO:teuthology.orchestra.run.smithi016.stdout:Deleting cluster with fsid: dc5403b4-f0a9-11ee-b647-cb9ed24678a4 2024-04-02T05:02:33.600 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid dc5403b4-f0a9-11ee-b647-cb9ed24678a4 --force 2024-04-02T05:02:33.767 INFO:teuthology.orchestra.run.smithi027.stdout:Deleting cluster with fsid: dc5403b4-f0a9-11ee-b647-cb9ed24678a4 2024-04-02T05:02:34.159 INFO:tasks.cephadm:Removing cephadm ... 2024-04-02T05:02:34.159 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-02T05:02:34.175 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-02T05:02:34.191 INFO:tasks.cephadm:Teardown complete 2024-04-02T05:02:34.191 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-02T05:02:34.203 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_6cec793d9073f0594e2364b9b687ecaf22424cca/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_6cec793d9073f0594e2364b9b687ecaf22424cca/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-02T05:02:34.203 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-02T05:02:34.204 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-02T05:02:34.217 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-02T05:02:34.317 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-02T05:02:34.328 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-02T05:02:34.328 DEBUG:teuthology.orchestra.run.smithi016:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T05:02:34.331 DEBUG:teuthology.orchestra.run.smithi027:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T05:02:34.342 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2024-04-02T05:02:34.345 INFO:teuthology.orchestra.run.smithi027.stderr:bash: line 1: ntpq: command not found 2024-04-02T05:02:34.347 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T05:02:34.347 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-04-02T05:02:34.347 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 55 -1202us[-1204us] +/- 98ms 2024-04-02T05:02:34.347 INFO:teuthology.orchestra.run.smithi016.stdout:^* hv02.front.sepia.ceph.com 3 6 377 50 -2348us[-2350us] +/- 66ms 2024-04-02T05:02:34.348 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 53 +5061us[+5059us] +/- 90ms 2024-04-02T05:02:34.348 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T05:02:34.350 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T05:02:34.350 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2024-04-02T05:02:34.350 INFO:teuthology.orchestra.run.smithi027.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 53 -1423us[-1423us] +/- 98ms 2024-04-02T05:02:34.350 INFO:teuthology.orchestra.run.smithi027.stdout:^* hv02.front.sepia.ceph.com 3 6 377 56 -2567us[-2566us] +/- 66ms 2024-04-02T05:02:34.350 INFO:teuthology.orchestra.run.smithi027.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 +4838us[+4838us] +/- 90ms 2024-04-02T05:02:34.351 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T05:02:34.351 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-02T05:02:34.363 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-02T05:02:34.364 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-02T05:02:34.390 DEBUG:teuthology.orchestra.run.smithi016:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-02T05:02:34.476 DEBUG:teuthology.orchestra.run.smithi027:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-02T05:02:34.564 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T05:02:34.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T05:02:34.595 DEBUG:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T05:02:34.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T05:02:34.630 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-02T05:02:34.642 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-02T05:02:34.707 INFO:teuthology.task.internal:Duration was 2266.017109 seconds 2024-04-02T05:02:34.707 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-02T05:02:34.739 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-02T05:02:34.739 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-02T05:02:34.742 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-02T05:02:34.783 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T05:02:34.793 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T05:02:35.190 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-02T05:02:35.190 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi016.front.sepia.ceph.com 2024-04-02T05:02:35.191 DEBUG:teuthology.orchestra.run.smithi016:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-02T05:02:35.210 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi027.front.sepia.ceph.com 2024-04-02T05:02:35.210 DEBUG:teuthology.orchestra.run.smithi027:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-02T05:02:35.231 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-02T05:02:35.231 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-02T05:02:35.253 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-02T05:02:35.304 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-02T05:02:35.305 DEBUG:teuthology.orchestra.run.smithi016:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-02T05:02:35.321 DEBUG:teuthology.orchestra.run.smithi027:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-02T05:02:35.971 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-02T05:02:35.984 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-02T05:02:35.984 DEBUG:teuthology.orchestra.run.smithi016:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-02T05:02:36.010 DEBUG:teuthology.orchestra.run.smithi027:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-02T05:02:36.043 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-02T05:02:36.055 DEBUG:teuthology.orchestra.run.smithi016:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-02T05:02:36.058 DEBUG:teuthology.orchestra.run.smithi027:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-02T05:02:36.080 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = core 2024-04-02T05:02:36.114 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = core 2024-04-02T05:02:36.141 DEBUG:teuthology.orchestra.run.smithi016:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-02T05:02:36.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T05:02:36.157 DEBUG:teuthology.orchestra.run.smithi027:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-02T05:02:36.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T05:02:36.197 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-02T05:02:36.208 INFO:teuthology.task.internal:Transferring archived files... 2024-04-02T05:02:36.209 DEBUG:teuthology.misc:Transferring archived files from smithi016:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-01_21:08:08-orch-squid-distro-default-smithi/7634838/remote/smithi016 2024-04-02T05:02:36.209 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-02T05:02:36.260 DEBUG:teuthology.misc:Transferring archived files from smithi027:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-01_21:08:08-orch-squid-distro-default-smithi/7634838/remote/smithi027 2024-04-02T05:02:36.260 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-02T05:02:36.309 INFO:teuthology.task.internal:Removing archive directory... 2024-04-02T05:02:36.309 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-02T05:02:36.311 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-02T05:02:36.358 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-02T05:02:36.373 INFO:teuthology.task.internal:Not uploading archives. 2024-04-02T05:02:36.373 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-02T05:02:36.398 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-02T05:02:36.398 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-02T05:02:36.401 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-02T05:02:36.412 INFO:teuthology.orchestra.run.smithi016.stdout: 262166 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Apr 2 05:02 /home/ubuntu/cephtest 2024-04-02T05:02:36.412 INFO:teuthology.orchestra.run.smithi016.stdout: 554382 4 d--------- 2 ubuntu ubuntu 4096 Apr 2 04:40 /home/ubuntu/cephtest/mnt.0 2024-04-02T05:02:36.413 INFO:teuthology.orchestra.run.smithi016.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-04-02T05:02:36.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T05:02:36.414 INFO:teuthology.orchestra.run.smithi016.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-04-02T05:02:36.414 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_6cec793d9073f0594e2364b9b687ecaf22424cca/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_6cec793d9073f0594e2364b9b687ecaf22424cca/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi016 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-04-02T05:02:36.415 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-02T05:02:36.433 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-02T05:02:36.467 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-02T05:02:36.467 INFO:teuthology.nuke:Checking targets against current locks 2024-04-02T05:02:36.499 DEBUG:teuthology.nuke:shortname: smithi016 2024-04-02T05:02:36.499 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-02T05:02:36.526 DEBUG:teuthology.nuke:shortname: smithi027 2024-04-02T05:02:36.526 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-02T05:02:36.530 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-01_21:08:08-orch-squid-distro-default-smithi/7634838', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 04:14:49.476152', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHRZqVhwCciLOONC/HznwhHfKmFctk5739tiOraMTl2h1nkmzK2GPAaBjw6l3WlquIg1JsPPi0NANUoJYARBiF8='} 2024-04-02T05:02:36.552 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi027.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-01_21:08:08-orch-squid-distro-default-smithi/7634838', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 04:14:49.477957', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH5+d8o2yMuNlHGs4c4hyCuLs2fdq07eanTjpGjK1YZciNmYtpq/Y0+kMTtZ1cweEeM4h1GlamBOqh+N7bcpRlM='} 2024-04-02T05:02:36.581 INFO:teuthology.orchestra.console.smithi016:Power off 2024-04-02T05:02:36.581 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-02T05:02:36.605 DEBUG:teuthology.orchestra.console.smithi016:power off output: Chassis Power Control: Down/Off 2024-04-02T05:02:36.605 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T05:02:36.620 INFO:teuthology.orchestra.console.smithi027:Power off 2024-04-02T05:02:36.621 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-02T05:02:36.630 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is on 2024-04-02T05:02:36.642 DEBUG:teuthology.orchestra.console.smithi027:power off output: Chassis Power Control: Down/Off 2024-04-02T05:02:36.642 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T05:02:36.666 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is on 2024-04-02T05:02:40.631 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T05:02:40.667 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T05:02:40.742 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is on 2024-04-02T05:02:40.777 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is on 2024-04-02T05:02:44.746 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T05:02:44.778 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T05:02:44.856 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is off 2024-04-02T05:02:44.887 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is off 2024-04-02T05:02:44.958 INFO:teuthology.orchestra.console.smithi016:Power off completed 2024-04-02T05:02:44.988 INFO:teuthology.orchestra.console.smithi027:Power off completed 2024-04-02T05:02:45.093 INFO:teuthology.run:Summary data: description: orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 2266.0171086788177 failure_reason: reached maximum tries (51) after waiting for 300 seconds flavor: default owner: scheduled_teuthology@teuthology status: fail success: false 2024-04-02T05:02:45.093 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-02T05:02:45.179 INFO:teuthology.run:FAIL